docs: Major update - Build profiles, SPECs, Chapter-Mesh architecture
🚀 Architecture Evolution:
**Build Profiles (Baukasten):**
- nexfs-core (~40KB): CAS + Block Valve + Hash (IoT, Sensor)
- nexfs-sovereign (~120KB): core + CDC + DAG (Phone, Laptop)
- nexfs-mesh (~200KB): sovereign + UTCP + Gossip (Home Box, Chapter Node)
**Specifications:**
- SPEC-084: NexFS Core Format (Superblock, Block Pointers, Hash-ID)
- SPEC-085: NexFS Sovereign Extensions (CDC, Merkle DAG, Time Travel)
- SPEC-704: UTCP Storage Extensions (Gossip, Chapter-Mesh, Kinetic Credits)
**Chapter-Mesh Architecture:**
- Two-partition layout (sovereign root + mesh pool)
- Automatic Chapter discovery via UTCP
- Sealed storage with Monolith encryption
- Kinetic Credits economy integration
- Torrent-killer features (CAS dedup, native FS integration)
**UTCP Storage Protocol:**
- 7 Block Operations (GET, PUT, DELETE, REPLICATE, etc.)
- 3 DAG Operations (RESOLVE, ANCESTORS, DIFF)
- 3 Peer Operations (ANNOUNCE, LEAVE, HEARTBEAT)
**Updated Roadmap:**
- v0.2.0 (Q2 2026): Sovereign profile (SPEC-085)
- v0.3.0 (Q3 2026): Mesh profile (SPEC-704)
- v1.0.0 (Q4 2026): Production hardening
724 lines (+248 lines), 19KB total
This commit is contained in:
Janus Agent
parent
7b98c21243
commit
69b7c74eee
GPG Key ID:
07DDBEA3CBDC090A
262
README.md
262
README.md
|
|
@ -22,6 +22,94 @@
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Build Profiles & Architecture
|
||||||
|
|
||||||
|
**NexFS** implements a **modular Baukasten architecture** (SPEC-004) with three build profiles. Not one binary - three configurations compiled from modules:
|
||||||
|
|
||||||
|
### Three Build Profiles
|
||||||
|
|
||||||
|
| Profile | Modules | Footprint | Use Case |
|
||||||
|
|---------|---------|-----------|----------|
|
||||||
|
| **nexfs-core** | CAS + Block Valve + Hash | ~40KB | IoT, Satellite, Sensor |
|
||||||
|
| **nexfs-sovereign** | core + CDC + DAG | ~120KB | Phone, Laptop, Desktop |
|
||||||
|
| **nexfs-mesh** | sovereign + UTCP + Gossip | ~200KB | Home Box, Chapter Node |
|
||||||
|
|
||||||
|
### Module Compilation
|
||||||
|
|
||||||
|
NexFS uses Nim's compile-time conditionals for module selection:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
# nexfs/config.nim
|
||||||
|
const
|
||||||
|
nexfs_cas* = defined(nexfs_cas) # CAS Layer (always)
|
||||||
|
nexfs_cdc* = defined(nexfs_cdc) # Content-Defined Chunking
|
||||||
|
nexfs_mesh* = defined(nexfs_mesh) # UTCP Cluster + Gossip
|
||||||
|
nexfs_dag* = defined(nexfs_dag) # Merkle DAG Directories
|
||||||
|
nexfs_dedup* = defined(nexfs_dedup) # Cross-Node Dedup
|
||||||
|
```
|
||||||
|
|
||||||
|
**Build Commands:**
|
||||||
|
```bash
|
||||||
|
# Core profile (IoT sensor)
|
||||||
|
nim c -d:nexfs_cas nexfs
|
||||||
|
|
||||||
|
# Sovereign profile (laptop)
|
||||||
|
nim c -d:nexfs_cas -d:nexfs_cdc -d:nexfs_dag nexfs
|
||||||
|
|
||||||
|
# Mesh profile (home box)
|
||||||
|
nim c -d:nexfs_cas -d:nexfs_cdc -d:nexfs_dag -d:nexfs_mesh nexfs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Profile Comparison
|
||||||
|
|
||||||
|
**nexfs-core** is your LittleFS successor:
|
||||||
|
- ✅ Robust, power-safe
|
||||||
|
- ✅ Content-addressed storage
|
||||||
|
- ✅ Block-level wear leveling
|
||||||
|
- ❌ No network code
|
||||||
|
- ❌ No chunker
|
||||||
|
- ❌ No gossip
|
||||||
|
- **Target:** Root partition for everything that blinks and beeps
|
||||||
|
|
||||||
|
**nexfs-sovereign** adds local intelligence:
|
||||||
|
- ✅ CDC-Chunking for efficient snapshots
|
||||||
|
- ✅ Local deduplication
|
||||||
|
- ✅ Merkle DAGs for Time-Travel
|
||||||
|
- ❌ Still single-node
|
||||||
|
- **Target:** Phone, laptop, desktop
|
||||||
|
|
||||||
|
**nexfs-mesh** enables network layer:
|
||||||
|
- ✅ UTCP cluster communication
|
||||||
|
- ✅ Gossip-based peer discovery
|
||||||
|
- ✅ Cross-node deduplication
|
||||||
|
- ✅ Chapter-Mesh integration
|
||||||
|
- ✅ Kinetic Credits economy
|
||||||
|
- **Target:** Home box, Chapter node
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Specifications
|
||||||
|
|
||||||
|
NexFS is defined by three SPECs that build on each other:
|
||||||
|
|
||||||
|
| SPEC | Name | Scope | Size |
|
||||||
|
|------|------|-------|------|
|
||||||
|
| **SPEC-084** | NexFS Core Format | Superblock, Block Pointers, Hash-ID, Profile-Byte | Foundation |
|
||||||
|
| **SPEC-085** | NexFS Sovereign Extensions | CDC Chunking, Merkle DAG, Local Dedup, Time Travel | +80KB |
|
||||||
|
| **SPEC-704** | UTCP Storage Extensions | UTCP Storage Ops, Gossip, Chapter-Mesh, Kinetic Credits | +80KB |
|
||||||
|
|
||||||
|
**Dependency Chain:**
|
||||||
|
```
|
||||||
|
SPEC-084 (Core) → SPEC-085 (Sovereign) → SPEC-704 (Mesh)
|
||||||
|
↓ ↓ ↓
|
||||||
|
40KB +80KB +80KB
|
||||||
|
IoT Laptop Home Box
|
||||||
|
```
|
||||||
|
|
||||||
|
Each SPEC is **independently implementable**. `core` compiles without the others. `sovereign` needs `core`. `mesh` needs both.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Use Cases
|
## Use Cases
|
||||||
|
|
||||||
### 1. Libertaria Mesh Nodes
|
### 1. Libertaria Mesh Nodes
|
||||||
|
|
@ -155,6 +243,147 @@ pub const FlashInterface = struct {
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Chapter-Mesh Architecture (nexfs-mesh)
|
||||||
|
|
||||||
|
### The Strategic Moat
|
||||||
|
|
||||||
|
When a user installs a Sovereign device, NexFS creates a two-partition layout:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─ /dev/nvme0 ─────────────────────────────────┐
|
||||||
|
│ │
|
||||||
|
│ Partition 1: nexfs-sovereign (Root, Private)│
|
||||||
|
│ ├── /Cas (System, Immutable) │
|
||||||
|
│ ├── /Nexus (Config, Encrypted) │
|
||||||
|
│ └── /Data (User, Encrypted) │
|
||||||
|
│ │
|
||||||
|
│ Partition 2: nexfs-mesh (Chapter Pool) │
|
||||||
|
│ ├── Encrypted at rest (Monolith) │
|
||||||
|
│ ├── Auto-joins Chapter via UTCP Discovery │
|
||||||
|
│ ├── Contributes: Storage + Bandwidth │
|
||||||
|
│ └── Receives: Redundancy + Content Access │
|
||||||
|
│ │
|
||||||
|
└────────────────────────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Installation Flow
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# During Sovereign installation
|
||||||
|
nexus forge --chapter "frankfurt-01"
|
||||||
|
|
||||||
|
# User prompt:
|
||||||
|
# "How much storage do you want to contribute to the Chapter Network?"
|
||||||
|
# [Slider: 10GB ... 500GB ... 2TB]
|
||||||
|
# Default: 10% of disk or 50GB, whichever is smaller
|
||||||
|
|
||||||
|
nexfs format /dev/nvme0p2 --mode sovereign-mesh \
|
||||||
|
--chapter-id "frankfurt-01" \
|
||||||
|
--quota 50G \
|
||||||
|
--replication-factor 3
|
||||||
|
```
|
||||||
|
|
||||||
|
### What Happens in the Background
|
||||||
|
|
||||||
|
1. **UTCP Discovery** (SPEC-703 Pattern):
|
||||||
|
- Box broadcasts `PEER_ANNOUNCE` on local network
|
||||||
|
- Connects to known Chapter gateways
|
||||||
|
|
||||||
|
2. **Gossip Sync**:
|
||||||
|
- Learns which other Sovereigns are online
|
||||||
|
- Exchanges peer lists and chunk availability
|
||||||
|
|
||||||
|
3. **Lazy Replication**:
|
||||||
|
- Popular chunks (Nip packages, community content) automatically cached
|
||||||
|
- Replication factor enforced (default: 3 copies)
|
||||||
|
|
||||||
|
4. **Sealed Storage**:
|
||||||
|
- User cannot see Chapter partition as normal directory
|
||||||
|
- It's an opaque block pool
|
||||||
|
- No user snooping on transit data possible
|
||||||
|
|
||||||
|
### The Economy (Kinetic Credits)
|
||||||
|
|
||||||
|
Integration with **Kinetic Economy Model** (SPEC-053):
|
||||||
|
|
||||||
|
```
|
||||||
|
You give: 100GB Mesh-Storage + Uptime
|
||||||
|
You get: Kinetic Credits
|
||||||
|
You use: Credits for Package-Downloads, Mesh-Access, Priority-Routing
|
||||||
|
```
|
||||||
|
|
||||||
|
**Engineered Reciprocity** - No altruism, no charity:
|
||||||
|
- More contribution = more credits
|
||||||
|
- No contribution = still can read (Commonwealth packages are free)
|
||||||
|
- Premium content and priority bandwidth cost credits
|
||||||
|
|
||||||
|
### The Torrent Killer
|
||||||
|
|
||||||
|
**Why NexFS Chapter-Mesh beats BitTorrent:**
|
||||||
|
|
||||||
|
| Feature | BitTorrent | NexFS Chapter-Mesh |
|
||||||
|
|---------|------------|-------------------|
|
||||||
|
| **FS Integration** | Separate client | Native FS (`nip install firefox`) |
|
||||||
|
| **Deduplication** | Full file transfer | CAS-Dedup (5% delta transfer) |
|
||||||
|
| **Encryption** | Optional | Encrypted at rest by default |
|
||||||
|
| **NAT Traversal** | Problematic | UTCP CellID routing (no NAT issues) |
|
||||||
|
| **Discovery** | Tracker/DHT | Gossip-based peer discovery |
|
||||||
|
| **Economy** | None | Kinetic Credits |
|
||||||
|
|
||||||
|
**Example:** Firefox 120.0.1 and 120.0.2 share 95% of chunks:
|
||||||
|
- BitTorrent: Transfer entire `.tar.gz` again
|
||||||
|
- NexFS: Transfer only 5% difference
|
||||||
|
|
||||||
|
### Superblock Extension
|
||||||
|
|
||||||
|
One byte in the superblock determines the profile:
|
||||||
|
|
||||||
|
```nim
|
||||||
|
type NexfsSuperblock* = object
|
||||||
|
magic*: uint32 # "NXFS"
|
||||||
|
version*: uint16
|
||||||
|
hash_algo*: uint8 # 0x00=BLAKE3-256, 0x01=BLAKE3-128, 0x02=XXH3-128
|
||||||
|
profile*: uint8 # 0x00=core, 0x01=sovereign, 0x02=mesh ← NEW
|
||||||
|
chapter_id*: CellID # 128-bit (only for mesh, otherwise 0)
|
||||||
|
mesh_quota*: uint64 # Bytes dedicated to mesh (only for mesh)
|
||||||
|
replication_target*: uint8 # Desired copies (default 3)
|
||||||
|
# ... rest as usual
|
||||||
|
```
|
||||||
|
|
||||||
|
### Filesystem Hierarchy Extension
|
||||||
|
|
||||||
|
```
|
||||||
|
/Data/
|
||||||
|
├── Volume/
|
||||||
|
│ ├── Local/ # Standard
|
||||||
|
│ ├── Remote/ # Cloud-Mounts
|
||||||
|
│ ├── External/ # USB
|
||||||
|
│ └── Mesh/ # NEW: Chapter Pool
|
||||||
|
│ ├── .quota # 50GB
|
||||||
|
│ ├── .chapter-id # frankfurt-01
|
||||||
|
│ └── .peers # Known CellIDs
|
||||||
|
```
|
||||||
|
|
||||||
|
### UTCP Storage Protocol (SPEC-704)
|
||||||
|
|
||||||
|
**Wire Protocol Extensions:**
|
||||||
|
|
||||||
|
| Category | Operations | Size |
|
||||||
|
|----------|-----------|------|
|
||||||
|
| **Block Ops** | GET, PUT, DELETE, REPLICATE, VERIFY, REPAIR, STATS | 7 ops |
|
||||||
|
| **DAG Ops** | RESOLVE, ANCESTORS, DIFF | 3 ops |
|
||||||
|
| **Peer Ops** | ANNOUNCE, LEAVE, HEARTBEAT | 3 ops |
|
||||||
|
|
||||||
|
All operations use **16-byte extension header** on existing UTCP frames.
|
||||||
|
|
||||||
|
**Key Features:**
|
||||||
|
- Gossip-based peer discovery
|
||||||
|
- Credit-based flow control (SPEC-702 pattern reuse)
|
||||||
|
- Replication-factor enforcement
|
||||||
|
- Chapter-Mesh integration with Kinetic Credits
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
### Installation
|
### Installation
|
||||||
|
|
@ -344,24 +573,43 @@ const Config = struct {
|
||||||
|
|
||||||
## Roadmap
|
## Roadmap
|
||||||
|
|
||||||
### Version 0.2.0 (Q2 2026)
|
### Version 0.2.0 (Q2 2026) - Sovereign Profile
|
||||||
|
**Focus:** SPEC-085 Implementation
|
||||||
|
- [ ] FastCDC chunker implementation
|
||||||
|
- [ ] CAS store with inline dedup
|
||||||
|
- [ ] Merkle DAG directory structure
|
||||||
|
- [ ] TimeWarp snapshots (`nexfs snap create/rollback/diff`)
|
||||||
|
- [ ] Build profiles: core vs sovereign
|
||||||
- [ ] Active wear leveling algorithm
|
- [ ] Active wear leveling algorithm
|
||||||
- [ ] Bad block management
|
- [ ] Bad block management
|
||||||
- [ ] Power-loss recovery improvements
|
|
||||||
- [ ] Extended file attributes (xattr)
|
|
||||||
|
|
||||||
### Version 0.3.0 (Q3 2026)
|
### Version 0.3.0 (Q3 2026) - Mesh Profile
|
||||||
|
**Focus:** SPEC-704 Implementation
|
||||||
|
- [ ] UTCP storage protocol (7 Block Ops, 3 DAG Ops, 3 Peer Ops)
|
||||||
|
- [ ] Gossip-based peer discovery
|
||||||
|
- [ ] Cross-node deduplication
|
||||||
|
- [ ] Chapter-Mesh integration
|
||||||
|
- [ ] Kinetic Credits integration
|
||||||
|
- [ ] Credit-based flow control
|
||||||
- [ ] Compression support (LZ4)
|
- [ ] Compression support (LZ4)
|
||||||
- [ ] Defragmentation tool
|
- [ ] Defragmentation tool
|
||||||
- [ ] Filesystem check utility (fsck)
|
- [ ] Filesystem check utility (fsck)
|
||||||
- [ ] Performance benchmarks
|
|
||||||
|
|
||||||
### Version 1.0.0 (Q4 2026)
|
### Version 1.0.0 (Q4 2026) - Production Hardening
|
||||||
|
**Focus:** Production Readiness
|
||||||
- [ ] Encryption support (XChaCha20-Poly1305)
|
- [ ] Encryption support (XChaCha20-Poly1305)
|
||||||
- [ ] Snapshot support
|
- [ ] CRDT for concurrent DAG edits
|
||||||
|
- [ ] Multi-Chapter federation
|
||||||
|
- [ ] Performance benchmarks
|
||||||
- [ ] Production-hardened
|
- [ ] Production-hardened
|
||||||
- [ ] Full Libertaria stack integration
|
- [ ] Full Libertaria stack integration
|
||||||
|
|
||||||
|
### Future Considerations
|
||||||
|
- **CRDT-Specification** for concurrent DAG edits (Phase 50+)
|
||||||
|
- **Multi-Chapter-Federation** for cross-chapter chunk exchange (Phase 50+)
|
||||||
|
|
||||||
|
**Note:** Features grow into existing SPECs as Sections when needed, not as standalone documents.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Testing
|
## Testing
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue