diff --git a/README.md b/README.md index b3276ce..6654f79 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,94 @@ --- +## Build Profiles & Architecture + +**NexFS** implements a **modular Baukasten architecture** (SPEC-004) with three build profiles. Not one binary - three configurations compiled from modules: + +### Three Build Profiles + +| Profile | Modules | Footprint | Use Case | +|---------|---------|-----------|----------| +| **nexfs-core** | CAS + Block Valve + Hash | ~40KB | IoT, Satellite, Sensor | +| **nexfs-sovereign** | core + CDC + DAG | ~120KB | Phone, Laptop, Desktop | +| **nexfs-mesh** | sovereign + UTCP + Gossip | ~200KB | Home Box, Chapter Node | + +### Module Compilation + +NexFS uses Nim's compile-time conditionals for module selection: + +```nim +# nexfs/config.nim +const + nexfs_cas* = defined(nexfs_cas) # CAS Layer (always) + nexfs_cdc* = defined(nexfs_cdc) # Content-Defined Chunking + nexfs_mesh* = defined(nexfs_mesh) # UTCP Cluster + Gossip + nexfs_dag* = defined(nexfs_dag) # Merkle DAG Directories + nexfs_dedup* = defined(nexfs_dedup) # Cross-Node Dedup +``` + +**Build Commands:** +```bash +# Core profile (IoT sensor) +nim c -d:nexfs_cas nexfs + +# Sovereign profile (laptop) +nim c -d:nexfs_cas -d:nexfs_cdc -d:nexfs_dag nexfs + +# Mesh profile (home box) +nim c -d:nexfs_cas -d:nexfs_cdc -d:nexfs_dag -d:nexfs_mesh nexfs +``` + +### Profile Comparison + +**nexfs-core** is your LittleFS successor: +- ✅ Robust, power-safe +- ✅ Content-addressed storage +- ✅ Block-level wear leveling +- ❌ No network code +- ❌ No chunker +- ❌ No gossip +- **Target:** Root partition for everything that blinks and beeps + +**nexfs-sovereign** adds local intelligence: +- ✅ CDC-Chunking for efficient snapshots +- ✅ Local deduplication +- ✅ Merkle DAGs for Time-Travel +- ❌ Still single-node +- **Target:** Phone, laptop, desktop + +**nexfs-mesh** enables network layer: +- ✅ UTCP cluster communication +- ✅ Gossip-based peer discovery +- ✅ Cross-node deduplication +- ✅ Chapter-Mesh integration +- ✅ Kinetic Credits economy +- **Target:** Home box, Chapter node + +--- + +## Specifications + +NexFS is defined by three SPECs that build on each other: + +| SPEC | Name | Scope | Size | +|------|------|-------|------| +| **SPEC-084** | NexFS Core Format | Superblock, Block Pointers, Hash-ID, Profile-Byte | Foundation | +| **SPEC-085** | NexFS Sovereign Extensions | CDC Chunking, Merkle DAG, Local Dedup, Time Travel | +80KB | +| **SPEC-704** | UTCP Storage Extensions | UTCP Storage Ops, Gossip, Chapter-Mesh, Kinetic Credits | +80KB | + +**Dependency Chain:** +``` +SPEC-084 (Core) → SPEC-085 (Sovereign) → SPEC-704 (Mesh) + ↓ ↓ ↓ + 40KB +80KB +80KB + IoT Laptop Home Box +``` + +Each SPEC is **independently implementable**. `core` compiles without the others. `sovereign` needs `core`. `mesh` needs both. + +--- + ## Use Cases ### 1. Libertaria Mesh Nodes @@ -155,6 +243,147 @@ pub const FlashInterface = struct { --- +## Chapter-Mesh Architecture (nexfs-mesh) + +### The Strategic Moat + +When a user installs a Sovereign device, NexFS creates a two-partition layout: + +``` +┌─ /dev/nvme0 ─────────────────────────────────┐ +│ │ +│ Partition 1: nexfs-sovereign (Root, Private)│ +│ ├── /Cas (System, Immutable) │ +│ ├── /Nexus (Config, Encrypted) │ +│ └── /Data (User, Encrypted) │ +│ │ +│ Partition 2: nexfs-mesh (Chapter Pool) │ +│ ├── Encrypted at rest (Monolith) │ +│ ├── Auto-joins Chapter via UTCP Discovery │ +│ ├── Contributes: Storage + Bandwidth │ +│ └── Receives: Redundancy + Content Access │ +│ │ +└────────────────────────────────────────────────┘ +``` + +### Installation Flow + +```bash +# During Sovereign installation +nexus forge --chapter "frankfurt-01" + +# User prompt: +# "How much storage do you want to contribute to the Chapter Network?" +# [Slider: 10GB ... 500GB ... 2TB] +# Default: 10% of disk or 50GB, whichever is smaller + +nexfs format /dev/nvme0p2 --mode sovereign-mesh \ + --chapter-id "frankfurt-01" \ + --quota 50G \ + --replication-factor 3 +``` + +### What Happens in the Background + +1. **UTCP Discovery** (SPEC-703 Pattern): + - Box broadcasts `PEER_ANNOUNCE` on local network + - Connects to known Chapter gateways + +2. **Gossip Sync**: + - Learns which other Sovereigns are online + - Exchanges peer lists and chunk availability + +3. **Lazy Replication**: + - Popular chunks (Nip packages, community content) automatically cached + - Replication factor enforced (default: 3 copies) + +4. **Sealed Storage**: + - User cannot see Chapter partition as normal directory + - It's an opaque block pool + - No user snooping on transit data possible + +### The Economy (Kinetic Credits) + +Integration with **Kinetic Economy Model** (SPEC-053): + +``` +You give: 100GB Mesh-Storage + Uptime +You get: Kinetic Credits +You use: Credits for Package-Downloads, Mesh-Access, Priority-Routing +``` + +**Engineered Reciprocity** - No altruism, no charity: +- More contribution = more credits +- No contribution = still can read (Commonwealth packages are free) +- Premium content and priority bandwidth cost credits + +### The Torrent Killer + +**Why NexFS Chapter-Mesh beats BitTorrent:** + +| Feature | BitTorrent | NexFS Chapter-Mesh | +|---------|------------|-------------------| +| **FS Integration** | Separate client | Native FS (`nip install firefox`) | +| **Deduplication** | Full file transfer | CAS-Dedup (5% delta transfer) | +| **Encryption** | Optional | Encrypted at rest by default | +| **NAT Traversal** | Problematic | UTCP CellID routing (no NAT issues) | +| **Discovery** | Tracker/DHT | Gossip-based peer discovery | +| **Economy** | None | Kinetic Credits | + +**Example:** Firefox 120.0.1 and 120.0.2 share 95% of chunks: +- BitTorrent: Transfer entire `.tar.gz` again +- NexFS: Transfer only 5% difference + +### Superblock Extension + +One byte in the superblock determines the profile: + +```nim +type NexfsSuperblock* = object + magic*: uint32 # "NXFS" + version*: uint16 + hash_algo*: uint8 # 0x00=BLAKE3-256, 0x01=BLAKE3-128, 0x02=XXH3-128 + profile*: uint8 # 0x00=core, 0x01=sovereign, 0x02=mesh ← NEW + chapter_id*: CellID # 128-bit (only for mesh, otherwise 0) + mesh_quota*: uint64 # Bytes dedicated to mesh (only for mesh) + replication_target*: uint8 # Desired copies (default 3) + # ... rest as usual +``` + +### Filesystem Hierarchy Extension + +``` +/Data/ +├── Volume/ +│ ├── Local/ # Standard +│ ├── Remote/ # Cloud-Mounts +│ ├── External/ # USB +│ └── Mesh/ # NEW: Chapter Pool +│ ├── .quota # 50GB +│ ├── .chapter-id # frankfurt-01 +│ └── .peers # Known CellIDs +``` + +### UTCP Storage Protocol (SPEC-704) + +**Wire Protocol Extensions:** + +| Category | Operations | Size | +|----------|-----------|------| +| **Block Ops** | GET, PUT, DELETE, REPLICATE, VERIFY, REPAIR, STATS | 7 ops | +| **DAG Ops** | RESOLVE, ANCESTORS, DIFF | 3 ops | +| **Peer Ops** | ANNOUNCE, LEAVE, HEARTBEAT | 3 ops | + +All operations use **16-byte extension header** on existing UTCP frames. + +**Key Features:** +- Gossip-based peer discovery +- Credit-based flow control (SPEC-702 pattern reuse) +- Replication-factor enforcement +- Chapter-Mesh integration with Kinetic Credits + +--- + ## Quick Start ### Installation @@ -344,24 +573,43 @@ const Config = struct { ## Roadmap -### Version 0.2.0 (Q2 2026) +### Version 0.2.0 (Q2 2026) - Sovereign Profile +**Focus:** SPEC-085 Implementation +- [ ] FastCDC chunker implementation +- [ ] CAS store with inline dedup +- [ ] Merkle DAG directory structure +- [ ] TimeWarp snapshots (`nexfs snap create/rollback/diff`) +- [ ] Build profiles: core vs sovereign - [ ] Active wear leveling algorithm - [ ] Bad block management -- [ ] Power-loss recovery improvements -- [ ] Extended file attributes (xattr) -### Version 0.3.0 (Q3 2026) +### Version 0.3.0 (Q3 2026) - Mesh Profile +**Focus:** SPEC-704 Implementation +- [ ] UTCP storage protocol (7 Block Ops, 3 DAG Ops, 3 Peer Ops) +- [ ] Gossip-based peer discovery +- [ ] Cross-node deduplication +- [ ] Chapter-Mesh integration +- [ ] Kinetic Credits integration +- [ ] Credit-based flow control - [ ] Compression support (LZ4) - [ ] Defragmentation tool - [ ] Filesystem check utility (fsck) -- [ ] Performance benchmarks -### Version 1.0.0 (Q4 2026) +### Version 1.0.0 (Q4 2026) - Production Hardening +**Focus:** Production Readiness - [ ] Encryption support (XChaCha20-Poly1305) -- [ ] Snapshot support +- [ ] CRDT for concurrent DAG edits +- [ ] Multi-Chapter federation +- [ ] Performance benchmarks - [ ] Production-hardened - [ ] Full Libertaria stack integration +### Future Considerations +- **CRDT-Specification** for concurrent DAG edits (Phase 50+) +- **Multi-Chapter-Federation** for cross-chapter chunk exchange (Phase 50+) + +**Note:** Features grow into existing SPECs as Sections when needed, not as standalone documents. + --- ## Testing