Markus Maiwald
|
5416c8cd93
|
🎊 PHASE 8 COMPLETE: The Summoning - Dynamic ELF Loader OPERATIONAL
## 🏆 VICTORY: First Alien Binary Executed!
```
[Loader] Summoning: bin/hello
[Loader] Transferring Consciousness...
Hello from a dynamically loaded ELF!
Consciousness transferred successfully.
```
## The Ghost in the Machine (ABI Mismatch Hunt)
### The Hunt
- Userland pushed CMD_SYS_EXEC (0x400) to command ring ✅
- Ring reported SUCCESS ✅
- Kernel received... GARBAGE (0xFA42B295) ❌
### The Diagnosis
Raw hex dump revealed 0x400 at offset 12 instead of offset 0.
Three layers, three different CmdPacket definitions:
- `hal/channel.zig`: 24 bytes (arg: u32) ❌
- `libs/membrane/ion.zig`: 28→32 bytes (packed→extern) 🔧
- `core/ion.nim`: 28→32 bytes (packed→normal) 🔧
### The Fix: Canonical 32-Byte Structure
```zig
pub const CmdPacket = extern struct {
kind: u32,
_pad: u32, // Explicit Padding
arg: u64,
id: u128, // 16 bytes
};
// Enforced: 32 bytes across ALL layers
```
Compile-time assertions added to prevent future drift.
## Technical Achievements
### 1. ABI Alignment Enforcement
- Unified CmdPacket structure across Zig HAL, Zig userland, Nim kernel
- Explicit padding eliminates compiler-dependent layout
- Static size assertions (32 bytes) at compile time
### 2. Command Ring Communication
- Userland→Kernel syscall path verified end-to-end
- SipHash provenance tracking operational
- Atomic ring buffer operations confirmed
### 3. ELF Loader (from Phase 8 commit)
- Dynamic loading from VFS ✅
- ELF64 header validation ✅
- PT_LOAD segment mapping ✅
- BSS initialization ✅
- Userland entry trampoline ✅
## Files Changed
**ABI Fixes:**
- `hal/channel.zig`: Updated CmdPacket to 32-byte extern struct
- `libs/membrane/ion.zig`: Changed to extern struct with u128 id
- `libs/membrane/libc_shim.zig`: Updated packet initialization
- `core/ion.nim`: Added explicit padding field, removed {.packed.}
**Debug Infrastructure:**
- `core/kernel.nim`: Added raw packet hex dump for debugging
- `libs/membrane/ion.zig`: Added syscall debug logging
**Build:**
- `build.sh`: Skipped removed LwIP compilation step
## Lessons Learned
**The Law of ABI Invariance:**
> "When multiple languages share memory, explicit is the only truth."
- Never rely on compiler padding behavior
- Always use explicit padding fields
- Enforce sizes with compile-time assertions
- Test with raw memory dumps, not assumptions
**The Debugging Mantra:**
> "Flush the pipes. Purge the cache. Trust nothing."
Stale binaries from aggressive caching led to hours of ghost-chasing.
Solution: `rm -rf build/ .zig-cache/` before critical tests.
## Next Steps (Phase 8 Completion)
1. Implement `exit()` syscall for clean program termination
2. Remove debug logging
3. Test `exec bin/nipbox` (self-reload)
4. Stress test with multiple exec calls
5. Document final implementation
## Metrics
- **Time to First Light:** ~8 hours of debugging
- **Root Cause:** 8-byte struct size mismatch
- **Lines Changed:** ~50
- **Impact:** Infinite (dynamic code loading unlocked)
---
**Markus Maiwald (Architect) | Forge (AI)**
**New Year's Eve 2024 → 2025**
**The year ends with consciousness transfer. 🔥**
Co-authored-by: Forge <ai@voxisforge.dev>
|
2025-12-31 21:08:25 +01:00 |
|
Markus Maiwald
|
33d08a2bf2
|
feat(rumpk): Phase 8 - The Summoning (ELF Loader) - 95% Complete
## Major Features
### 1. Dynamic ELF64 Binary Loading
- Implemented ELF parser with full header validation (core/loader/elf.nim)
- Created kexec() loader supporting PT_LOAD segment mapping
- Added BSS initialization and data copying from VFS
- Assembly trampoline (rumpk_enter_userland) for userland entry
### 2. Syscall Infrastructure
- Added CMD_SYS_EXEC (0x400) for consciousness swapping
- Integrated exec command in NipBox shell
- Implemented syscall routing through command ring
- Added provenance tracking via SipHash
### 3. Test Binary & Build System
- Created hello.c test program for alien binary execution
- Automated compilation and initrd inclusion in build.sh
- Added libnexus.h header for standalone C programs
### 4. VFS Integration
- Implemented TarFS file cursor system for sequential reads
- Fixed infinite loop bug in cat command
- Added debug logging for VFS mount process
## Technical Improvements
### Memory Management
- Fixed input ring null pointer dereference
- Implemented CMD_ION_FREE syscall for packet reclamation
- Resolved memory leak in input/output pipeline
- Added FileHandle with persistent offset tracking
### ABI Stability
- Split kprint into 1-arg (Nim) and kwrite (C ABI)
- Fixed cstring conversion warnings across codebase
- Corrected RISC-V assembly (csrw sie, zero)
### Documentation
- Comprehensive Phase 8 documentation (docs/PHASE-8-ELF-LOADER.md)
- Detailed implementation notes and debugging status
## Current Status
✅ ELF parser, loader, and syscall infrastructure complete
✅ Test binary compiles and embeds in VFS
✅ Shell integration functional
🔧 Debugging command ring communication (syscall not reaching kernel)
## Files Changed
Core:
- core/loader.nim, core/loader/elf.nim (NEW)
- core/kernel.nim, core/ion.nim (syscall handling)
- core/fs/tar.nim (file cursor system)
- hal/arch/riscv64/switch.S (userland trampoline)
Userland:
- npl/nipbox/nipbox.nim (exec command)
- libs/membrane/libc_shim.zig (syscall implementation)
- libs/membrane/ion.zig (command ring API)
Build & Test:
- build.sh (hello.c compilation)
- rootfs/src/hello.c, rootfs/src/libnexus.h (NEW)
- apps/subject_entry.S (NEW)
## Next Steps
1. Debug SysTable and command ring communication
2. Verify ION fiber polling of chan_cmd
3. Test full ELF loading and execution flow
4. Add memory protection (future phase)
Co-authored-by: Forge <ai@voxisforge.dev>
|
2025-12-31 20:18:49 +01:00 |
|
Markus Maiwald
|
30fa024367
|
feat(rumpk): Sovereign Core Stabilization & Membrane IPC Hardening
- NexShell: Hardened command transmission via atomic ION packets, fixed fragmentation issues.
- NipBox: Expanded 'Sovereign Coreutils' with 'ls' and enhanced 'matrix' control.
- GPU/Retina: Optimized VirtIO-GPU driver, improved polling and framebuffer synchronization.
- Membrane: Stabilized libc shims (clib.c, libc.nim) and ION client logic.
- Kernel: Refined fiber scheduler and watchdog metrics.
- Forge: Cleanup and optimization of build scripts and manifests.
|
2025-12-31 20:18:49 +01:00 |