docs: Major update - Build profiles, SPECs, Chapter-Mesh architecture
🚀 Architecture Evolution:
**Build Profiles (Baukasten):**
- nexfs-core (~40KB): CAS + Block Valve + Hash (IoT, Sensor)
- nexfs-sovereign (~120KB): core + CDC + DAG (Phone, Laptop)
- nexfs-mesh (~200KB): sovereign + UTCP + Gossip (Home Box, Chapter Node)
**Specifications:**
- SPEC-084: NexFS Core Format (Superblock, Block Pointers, Hash-ID)
- SPEC-085: NexFS Sovereign Extensions (CDC, Merkle DAG, Time Travel)
- SPEC-704: UTCP Storage Extensions (Gossip, Chapter-Mesh, Kinetic Credits)
**Chapter-Mesh Architecture:**
- Two-partition layout (sovereign root + mesh pool)
- Automatic Chapter discovery via UTCP
- Sealed storage with Monolith encryption
- Kinetic Credits economy integration
- Torrent-killer features (CAS dedup, native FS integration)
**UTCP Storage Protocol:**
- 7 Block Operations (GET, PUT, DELETE, REPLICATE, etc.)
- 3 DAG Operations (RESOLVE, ANCESTORS, DIFF)
- 3 Peer Operations (ANNOUNCE, LEAVE, HEARTBEAT)
**Updated Roadmap:**
- v0.2.0 (Q2 2026): Sovereign profile (SPEC-085)
- v0.3.0 (Q3 2026): Mesh profile (SPEC-704)
- v1.0.0 (Q4 2026): Production hardening
724 lines (+248 lines), 19KB total
This commit is contained in:
Janus Agent
parent
7b98c21243
commit
69b7c74eee
GPG Key ID:
07DDBEA3CBDC090A
262
README.md
262
README.md
|
|
@ -22,6 +22,94 @@
|
|||
|
||||
---
|
||||
|
||||
## Build Profiles & Architecture
|
||||
|
||||
**NexFS** implements a **modular Baukasten architecture** (SPEC-004) with three build profiles. Not one binary - three configurations compiled from modules:
|
||||
|
||||
### Three Build Profiles
|
||||
|
||||
| Profile | Modules | Footprint | Use Case |
|
||||
|---------|---------|-----------|----------|
|
||||
| **nexfs-core** | CAS + Block Valve + Hash | ~40KB | IoT, Satellite, Sensor |
|
||||
| **nexfs-sovereign** | core + CDC + DAG | ~120KB | Phone, Laptop, Desktop |
|
||||
| **nexfs-mesh** | sovereign + UTCP + Gossip | ~200KB | Home Box, Chapter Node |
|
||||
|
||||
### Module Compilation
|
||||
|
||||
NexFS uses Nim's compile-time conditionals for module selection:
|
||||
|
||||
```nim
|
||||
# nexfs/config.nim
|
||||
const
|
||||
nexfs_cas* = defined(nexfs_cas) # CAS Layer (always)
|
||||
nexfs_cdc* = defined(nexfs_cdc) # Content-Defined Chunking
|
||||
nexfs_mesh* = defined(nexfs_mesh) # UTCP Cluster + Gossip
|
||||
nexfs_dag* = defined(nexfs_dag) # Merkle DAG Directories
|
||||
nexfs_dedup* = defined(nexfs_dedup) # Cross-Node Dedup
|
||||
```
|
||||
|
||||
**Build Commands:**
|
||||
```bash
|
||||
# Core profile (IoT sensor)
|
||||
nim c -d:nexfs_cas nexfs
|
||||
|
||||
# Sovereign profile (laptop)
|
||||
nim c -d:nexfs_cas -d:nexfs_cdc -d:nexfs_dag nexfs
|
||||
|
||||
# Mesh profile (home box)
|
||||
nim c -d:nexfs_cas -d:nexfs_cdc -d:nexfs_dag -d:nexfs_mesh nexfs
|
||||
```
|
||||
|
||||
### Profile Comparison
|
||||
|
||||
**nexfs-core** is your LittleFS successor:
|
||||
- ✅ Robust, power-safe
|
||||
- ✅ Content-addressed storage
|
||||
- ✅ Block-level wear leveling
|
||||
- ❌ No network code
|
||||
- ❌ No chunker
|
||||
- ❌ No gossip
|
||||
- **Target:** Root partition for everything that blinks and beeps
|
||||
|
||||
**nexfs-sovereign** adds local intelligence:
|
||||
- ✅ CDC-Chunking for efficient snapshots
|
||||
- ✅ Local deduplication
|
||||
- ✅ Merkle DAGs for Time-Travel
|
||||
- ❌ Still single-node
|
||||
- **Target:** Phone, laptop, desktop
|
||||
|
||||
**nexfs-mesh** enables network layer:
|
||||
- ✅ UTCP cluster communication
|
||||
- ✅ Gossip-based peer discovery
|
||||
- ✅ Cross-node deduplication
|
||||
- ✅ Chapter-Mesh integration
|
||||
- ✅ Kinetic Credits economy
|
||||
- **Target:** Home box, Chapter node
|
||||
|
||||
---
|
||||
|
||||
## Specifications
|
||||
|
||||
NexFS is defined by three SPECs that build on each other:
|
||||
|
||||
| SPEC | Name | Scope | Size |
|
||||
|------|------|-------|------|
|
||||
| **SPEC-084** | NexFS Core Format | Superblock, Block Pointers, Hash-ID, Profile-Byte | Foundation |
|
||||
| **SPEC-085** | NexFS Sovereign Extensions | CDC Chunking, Merkle DAG, Local Dedup, Time Travel | +80KB |
|
||||
| **SPEC-704** | UTCP Storage Extensions | UTCP Storage Ops, Gossip, Chapter-Mesh, Kinetic Credits | +80KB |
|
||||
|
||||
**Dependency Chain:**
|
||||
```
|
||||
SPEC-084 (Core) → SPEC-085 (Sovereign) → SPEC-704 (Mesh)
|
||||
↓ ↓ ↓
|
||||
40KB +80KB +80KB
|
||||
IoT Laptop Home Box
|
||||
```
|
||||
|
||||
Each SPEC is **independently implementable**. `core` compiles without the others. `sovereign` needs `core`. `mesh` needs both.
|
||||
|
||||
---
|
||||
|
||||
## Use Cases
|
||||
|
||||
### 1. Libertaria Mesh Nodes
|
||||
|
|
@ -155,6 +243,147 @@ pub const FlashInterface = struct {
|
|||
|
||||
---
|
||||
|
||||
## Chapter-Mesh Architecture (nexfs-mesh)
|
||||
|
||||
### The Strategic Moat
|
||||
|
||||
When a user installs a Sovereign device, NexFS creates a two-partition layout:
|
||||
|
||||
```
|
||||
┌─ /dev/nvme0 ─────────────────────────────────┐
|
||||
│ │
|
||||
│ Partition 1: nexfs-sovereign (Root, Private)│
|
||||
│ ├── /Cas (System, Immutable) │
|
||||
│ ├── /Nexus (Config, Encrypted) │
|
||||
│ └── /Data (User, Encrypted) │
|
||||
│ │
|
||||
│ Partition 2: nexfs-mesh (Chapter Pool) │
|
||||
│ ├── Encrypted at rest (Monolith) │
|
||||
│ ├── Auto-joins Chapter via UTCP Discovery │
|
||||
│ ├── Contributes: Storage + Bandwidth │
|
||||
│ └── Receives: Redundancy + Content Access │
|
||||
│ │
|
||||
└────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Installation Flow
|
||||
|
||||
```bash
|
||||
# During Sovereign installation
|
||||
nexus forge --chapter "frankfurt-01"
|
||||
|
||||
# User prompt:
|
||||
# "How much storage do you want to contribute to the Chapter Network?"
|
||||
# [Slider: 10GB ... 500GB ... 2TB]
|
||||
# Default: 10% of disk or 50GB, whichever is smaller
|
||||
|
||||
nexfs format /dev/nvme0p2 --mode sovereign-mesh \
|
||||
--chapter-id "frankfurt-01" \
|
||||
--quota 50G \
|
||||
--replication-factor 3
|
||||
```
|
||||
|
||||
### What Happens in the Background
|
||||
|
||||
1. **UTCP Discovery** (SPEC-703 Pattern):
|
||||
- Box broadcasts `PEER_ANNOUNCE` on local network
|
||||
- Connects to known Chapter gateways
|
||||
|
||||
2. **Gossip Sync**:
|
||||
- Learns which other Sovereigns are online
|
||||
- Exchanges peer lists and chunk availability
|
||||
|
||||
3. **Lazy Replication**:
|
||||
- Popular chunks (Nip packages, community content) automatically cached
|
||||
- Replication factor enforced (default: 3 copies)
|
||||
|
||||
4. **Sealed Storage**:
|
||||
- User cannot see Chapter partition as normal directory
|
||||
- It's an opaque block pool
|
||||
- No user snooping on transit data possible
|
||||
|
||||
### The Economy (Kinetic Credits)
|
||||
|
||||
Integration with **Kinetic Economy Model** (SPEC-053):
|
||||
|
||||
```
|
||||
You give: 100GB Mesh-Storage + Uptime
|
||||
You get: Kinetic Credits
|
||||
You use: Credits for Package-Downloads, Mesh-Access, Priority-Routing
|
||||
```
|
||||
|
||||
**Engineered Reciprocity** - No altruism, no charity:
|
||||
- More contribution = more credits
|
||||
- No contribution = still can read (Commonwealth packages are free)
|
||||
- Premium content and priority bandwidth cost credits
|
||||
|
||||
### The Torrent Killer
|
||||
|
||||
**Why NexFS Chapter-Mesh beats BitTorrent:**
|
||||
|
||||
| Feature | BitTorrent | NexFS Chapter-Mesh |
|
||||
|---------|------------|-------------------|
|
||||
| **FS Integration** | Separate client | Native FS (`nip install firefox`) |
|
||||
| **Deduplication** | Full file transfer | CAS-Dedup (5% delta transfer) |
|
||||
| **Encryption** | Optional | Encrypted at rest by default |
|
||||
| **NAT Traversal** | Problematic | UTCP CellID routing (no NAT issues) |
|
||||
| **Discovery** | Tracker/DHT | Gossip-based peer discovery |
|
||||
| **Economy** | None | Kinetic Credits |
|
||||
|
||||
**Example:** Firefox 120.0.1 and 120.0.2 share 95% of chunks:
|
||||
- BitTorrent: Transfer entire `.tar.gz` again
|
||||
- NexFS: Transfer only 5% difference
|
||||
|
||||
### Superblock Extension
|
||||
|
||||
One byte in the superblock determines the profile:
|
||||
|
||||
```nim
|
||||
type NexfsSuperblock* = object
|
||||
magic*: uint32 # "NXFS"
|
||||
version*: uint16
|
||||
hash_algo*: uint8 # 0x00=BLAKE3-256, 0x01=BLAKE3-128, 0x02=XXH3-128
|
||||
profile*: uint8 # 0x00=core, 0x01=sovereign, 0x02=mesh ← NEW
|
||||
chapter_id*: CellID # 128-bit (only for mesh, otherwise 0)
|
||||
mesh_quota*: uint64 # Bytes dedicated to mesh (only for mesh)
|
||||
replication_target*: uint8 # Desired copies (default 3)
|
||||
# ... rest as usual
|
||||
```
|
||||
|
||||
### Filesystem Hierarchy Extension
|
||||
|
||||
```
|
||||
/Data/
|
||||
├── Volume/
|
||||
│ ├── Local/ # Standard
|
||||
│ ├── Remote/ # Cloud-Mounts
|
||||
│ ├── External/ # USB
|
||||
│ └── Mesh/ # NEW: Chapter Pool
|
||||
│ ├── .quota # 50GB
|
||||
│ ├── .chapter-id # frankfurt-01
|
||||
│ └── .peers # Known CellIDs
|
||||
```
|
||||
|
||||
### UTCP Storage Protocol (SPEC-704)
|
||||
|
||||
**Wire Protocol Extensions:**
|
||||
|
||||
| Category | Operations | Size |
|
||||
|----------|-----------|------|
|
||||
| **Block Ops** | GET, PUT, DELETE, REPLICATE, VERIFY, REPAIR, STATS | 7 ops |
|
||||
| **DAG Ops** | RESOLVE, ANCESTORS, DIFF | 3 ops |
|
||||
| **Peer Ops** | ANNOUNCE, LEAVE, HEARTBEAT | 3 ops |
|
||||
|
||||
All operations use **16-byte extension header** on existing UTCP frames.
|
||||
|
||||
**Key Features:**
|
||||
- Gossip-based peer discovery
|
||||
- Credit-based flow control (SPEC-702 pattern reuse)
|
||||
- Replication-factor enforcement
|
||||
- Chapter-Mesh integration with Kinetic Credits
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Installation
|
||||
|
|
@ -344,24 +573,43 @@ const Config = struct {
|
|||
|
||||
## Roadmap
|
||||
|
||||
### Version 0.2.0 (Q2 2026)
|
||||
### Version 0.2.0 (Q2 2026) - Sovereign Profile
|
||||
**Focus:** SPEC-085 Implementation
|
||||
- [ ] FastCDC chunker implementation
|
||||
- [ ] CAS store with inline dedup
|
||||
- [ ] Merkle DAG directory structure
|
||||
- [ ] TimeWarp snapshots (`nexfs snap create/rollback/diff`)
|
||||
- [ ] Build profiles: core vs sovereign
|
||||
- [ ] Active wear leveling algorithm
|
||||
- [ ] Bad block management
|
||||
- [ ] Power-loss recovery improvements
|
||||
- [ ] Extended file attributes (xattr)
|
||||
|
||||
### Version 0.3.0 (Q3 2026)
|
||||
### Version 0.3.0 (Q3 2026) - Mesh Profile
|
||||
**Focus:** SPEC-704 Implementation
|
||||
- [ ] UTCP storage protocol (7 Block Ops, 3 DAG Ops, 3 Peer Ops)
|
||||
- [ ] Gossip-based peer discovery
|
||||
- [ ] Cross-node deduplication
|
||||
- [ ] Chapter-Mesh integration
|
||||
- [ ] Kinetic Credits integration
|
||||
- [ ] Credit-based flow control
|
||||
- [ ] Compression support (LZ4)
|
||||
- [ ] Defragmentation tool
|
||||
- [ ] Filesystem check utility (fsck)
|
||||
- [ ] Performance benchmarks
|
||||
|
||||
### Version 1.0.0 (Q4 2026)
|
||||
### Version 1.0.0 (Q4 2026) - Production Hardening
|
||||
**Focus:** Production Readiness
|
||||
- [ ] Encryption support (XChaCha20-Poly1305)
|
||||
- [ ] Snapshot support
|
||||
- [ ] CRDT for concurrent DAG edits
|
||||
- [ ] Multi-Chapter federation
|
||||
- [ ] Performance benchmarks
|
||||
- [ ] Production-hardened
|
||||
- [ ] Full Libertaria stack integration
|
||||
|
||||
### Future Considerations
|
||||
- **CRDT-Specification** for concurrent DAG edits (Phase 50+)
|
||||
- **Multi-Chapter-Federation** for cross-chapter chunk exchange (Phase 50+)
|
||||
|
||||
**Note:** Features grow into existing SPECs as Sections when needed, not as standalone documents.
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
|
|
|||
Loading…
Reference in New Issue