# NexFS - Native Zig Flash Filesystem for NexusOS

> **The sovereign flash filesystem for Libertaria nodes and embedded devices**

[![License: LSL-1.0](https://img.shields.io/badge/License-LSL--1.0-blue.svg)](https://opensource.org/licenses/LSL-1.0)
[![Zig](https://img.shields.io/badge/Zig-0.13+-orange.svg)](https://ziglang.org)
[![Status: Alpha](https://img.shields.io/badge/Status-Alpha-yellow.svg)](https://git.sovereign-society.org/nexus/nexfs)

---

## What is NexFS?

**NexFS** is a native Zig implementation of a flash-aware filesystem designed for **Libertaria nodes** and **embedded sovereign devices**. It provides reliable, wear-leveling-aware storage for resource-constrained environments where data integrity and flash longevity are critical.

### Key Design Goals

- **Flash-First Architecture**: Optimized for raw NAND/NOR flash with wear leveling awareness
- **Zero Dynamic Allocation**: All buffers provided by caller - no runtime memory allocation
- **Platform Agnostic**: Works with any flash HAL via callback interface
- **Data Integrity**: CRC32C checksums on all metadata structures
- **Sovereign by Design**: No external dependencies, no vendor lock-in, fully auditable

---

## Use Cases

### 1. Libertaria Mesh Nodes

**Primary Use Case**: Storage layer for Libertaria Capsule nodes

```
┌─────────────────────────────────────────┐
│        Libertaria Capsule Node          │
│                                         │
│  ┌─────────────────────────────────┐   │
│  │   L3 Gossip (QVL Trust Edges)   │   │
│  └─────────────────────────────────┘   │
│  ┌─────────────────────────────────┐   │
│  │   L2 Session (Noise Handshakes) │   │
│  └─────────────────────────────────┘   │
│  ┌─────────────────────────────────┐   │
│  │   L1 Identity (SoulKeys)        │   │
│  └─────────────────────────────────┘   │
│  ┌─────────────────────────────────┐   │
│  │   NexFS (Persistent Storage)    │◄──┘
│  └─────────────────────────────────┘
│  ┌─────────────────────────────────┐
│  │   Raw Flash (NAND/NOR/SPI)      │
│  └─────────────────────────────────┘
└─────────────────────────────────────────┘
```

**Why NexFS for Libertaria?**
- **Persistence**: SoulKeys, peer tables, trust graphs survive reboots
- **Integrity**: CRC32C ensures metadata hasn't been corrupted
- **Wear Leveling**: Tracks erase counts to maximize flash lifespan
- **Minimal Footprint**: Zero allocation design fits embedded constraints
- **Fast Boot**: No journal replay, direct mount from superblock

### 2. Embedded Sovereign Devices

**Secondary Use Case**: IoT devices, Raspberry Pi, ESP32, microcontrollers

**Examples:**
- **Solar Monitor Nodes**: Store sensor readings, config, firmware updates
- **Weather Network**: Log environmental data locally before sync
- **Pager Devices**: Message queue persistence
- **Home Automation**: Device state, automation rules, logs

**Why NexFS for Embedded?**
- **Raw Flash Support**: Works directly with SPI flash, no FTL layer needed
- **Power-Loss Resilience**: Dual superblock backup survives sudden power loss
- **Deterministic**: Fixed buffer sizes, predictable memory usage
- **No OS Dependencies**: Works bare-metal or with any RTOS

---

## Architecture

### On-Disk Layout

```
┌─────────────────────────────────────────────┐
│  Block 0: Primary Superblock (128 bytes)   │
├─────────────────────────────────────────────┤
│  Block 1: Backup Superblock (128 bytes)    │
├─────────────────────────────────────────────┤
│  Blocks 2-N: Block Allocation Map (BAM)    │
│  - Tracks allocation status                 │
│  - Records erase counts (wear leveling)    │
│  - Bad block marking                        │
├─────────────────────────────────────────────┤
│  Blocks N+1-N+4: Inode Table               │
│  - File/directory metadata                  │
│  - Inode IDs 1-128                          │
├─────────────────────────────────────────────┤
│  Blocks N+5+: Data Blocks                  │
│  - File/directory contents                  │
│  - Wear-leveled allocation                  │
└─────────────────────────────────────────────┘
```

### Key Components

**1. Superblock**
- Magic number: `0x4E455846` ("NEXF")
- Generation counter for crash recovery
- Mount count for health monitoring
- CRC32C checksum for integrity

**2. Block Allocation Map (BAM)**
- Per-block metadata: allocated, bad, reserved, needs_erase
- Erase count tracking for wear leveling
- Generation counter for block age

**3. Inode Table**
- File/directory metadata
- Supports: Regular, Directory, Symlink, Device nodes
- Max filename: 255 characters

**4. Flash Interface**
```zig
pub const FlashInterface = struct {
    read: *const fn (ctx: *anyopaque, addr: u64, buffer: []u8) NexFSError!usize,
    write: *const fn (ctx: *anyopaque, addr: u64, buffer: []const u8) NexFSError!void,
    erase: *const fn (ctx: *anyopaque, block_addr: BlockAddr) NexFSError!void,
    sync: *const fn (ctx: *anyopaque) NexFSError!void,
};
```

---

## Features

### ✅ Implemented (v0.1.0)

- **Format/Initialization**: `format()` creates fresh filesystem
- **Superblock Management**: Primary + backup with checksums
- **Block Allocation**: BAM-based allocation with wear tracking
- **Inode Operations**: Create, read, write, delete
- **Directory Operations**: mkdir, rmdir, readdir, lookup
- **File Operations**: open, read, write, close, seek
- **Path Resolution**: Full path support (`/path/to/file`)
- **Checksum Verification**: CRC32C on all metadata
- **Zero Allocation**: All buffers provided by caller

### 🚧 Planned (Future Versions)

- **Wear Leveling Algorithm**: Active block rotation based on erase counts
- **Bad Block Management**: Automatic bad block detection and marking
- **Defragmentation**: Reclaim fragmented data blocks
- **Snapshots**: Point-in-time filesystem snapshots
- **Compression**: Optional LZ4 compression for data blocks
- **Encryption**: Optional XChaCha20-Poly1305 encryption

---

## Quick Start

### Installation

Add NexFS to your `build.zig.zon`:

```zig
.{
    .name = "your-project",
    .version = "0.1.0",
    .dependencies = .{
        .nexfs = .{
            .url = "https://git.sovereign-society.org/nexus/nexfs/archive/main.tar.gz",
            .hash = "...",
        },
    },
}
```

### Example: Basic Usage

```zig
const std = @import("std");
const nexfs = @import("nexfs");

// 1. Define your flash interface
const MyFlash = struct {
    flash_data: []u8,

    pub fn read(ctx: *anyopaque, addr: u64, buffer: []u8) nexfs.NexFSError!usize {
        const self = @ptrCast(*MyFlash, @alignCast(ctx));
        @memcpy(buffer, self.flash_data[addr..][0..buffer.len]);
        return buffer.len;
    }

    pub fn write(ctx: *anyopaque, addr: u64, buffer: []const u8) nexfs.NexFSError!void {
        const self = @ptrCast(*MyFlash, @alignCast(ctx));
        @memcpy(self.flash_data[addr..][0..buffer.len], buffer);
    }

    pub fn erase(ctx: *anyopaque, block_addr: nexfs.BlockAddr) nexfs.NexFSError!void {
        // Erase flash block (set to 0xFF for NAND)
    }

    pub fn sync(ctx: *anyopaque) nexfs.NexFSError!void {
        // Flush any caches
    }
};

pub fn main() !void {
    var flash = MyFlash{ .flash_data = try allocator.alloc(u8, 1024 * 1024) };

    // 2. Configure NexFS
    var read_buf: [4096]u8 = undefined;
    var write_buf: [4096]u8 = undefined;
    var workspace: [256]u8 = undefined;

    const config = nexfs.Config{
        .flash = .{
            .ctx = &flash,
            .read = MyFlash.read,
            .write = MyFlash.write,
            .erase = MyFlash.erase,
            .sync = MyFlash.sync,
        },
        .device_size = 1024 * 1024,
        .block_size = 4096,
        .block_count = 256,
        .page_size = 256,
        .checksum_algo = .CRC32C,
        .read_buffer = &read_buf,
        .write_buffer = &write_buf,
        .workspace = &workspace,
        .time_source = null,
        .verbose = true,
    };

    // 3. Format the filesystem
    try nexfs.format(&config.flash, &config, &write_buf);

    // 4. Create a file
    var fs = try nexfs.NexFS.init(allocator, config);
    const fd = try fs.create("/config.txt");
    try fs.write(fd, "hello nexfs");
    try fs.close(fd);

    // 5. Read it back
    var buf: [64]u8 = undefined;
    const fd2 = try fs.open("/config.txt");
    const len = try fs.read(fd2, &buf);
    try std.io.getStdOut().writeAll(buf[0..len]);
    try fs.close(fd2);
}
```

---

## Configuration Options

```zig
const Config = struct {
    flash: FlashInterface,      // Your flash HAL
    device_size: u64,            // Total flash size in bytes
    block_size: BlockSize,       // Flash block size (512, 1024, 2048, 4096)
    block_count: u32,            // Number of blocks
    page_size: PageSize,         // Flash page size for alignment
    checksum_algo: ChecksumAlgo, // None, CRC16, or CRC32C
    read_buffer: []u8,           // Buffer >= block_size
    write_buffer: []u8,          // Buffer >= block_size
    workspace: []u8,             // Buffer >= page_size
    time_source: ?TimeSource,    // Optional timestamp provider
    verbose: bool,               // Enable debug logging
};
```

### Recommended Configurations

**1. Raspberry Pi with SPI Flash (1MB)**
```zig
.block_size = 4096,
.page_size = 256,
.block_count = 256,
.checksum_algo = .CRC32C,
```

**2. ESP32 with Flash (4MB)**
```zig
.block_size = 4096,
.page_size = 256,
.block_count = 1024,
.checksum_algo = .CRC32C,
```

**3. Microcontroller with NOR Flash (512KB)**
```zig
.block_size = 2048,
.page_size = 256,
.block_count = 256,
.checksum_algo = .CRC16,  // Faster on limited CPUs
```

---

## Design Philosophy

### Sovereign Storage Principles

1. **No Secrets**: All code is open source and auditable (LSL-1.0)
2. **No Dependencies**: Zero external libraries, pure Zig
3. **No Vendor Lock-in**: Standard interfaces, portable anywhere
4. **No Hidden Allocation**: Explicit memory management
5. **No Trust Required**: Verify integrity with checksums

### Flash-Aware Design

**Why Raw Flash?**
- **Predictable Performance**: No FTL latency spikes
- **Full Control**: Wear leveling algorithm you control
- **Longer Lifespan**: Avoid consumer-grade FTL write amplification
- **Lower Power**: No background garbage collection

**Wear Leveling Strategy:**
- Track erase counts per block (BAM)
- Prefer blocks with lowest erase counts for writes
- Reserve high-erase-count blocks for cold data
- Target: Even wear distribution across flash lifetime

---

## Performance Characteristics

| Operation | Typical Latency | Notes |
|-----------|----------------|-------|
| Mount | < 10ms | Read superblock, validate checksum |
| Format | 100-500ms | Initialize all metadata blocks |
| File Create | 5-20ms | Allocate inode, write metadata |
| File Read (4KB) | 1-5ms | Single block read |
| File Write (4KB) | 10-30ms | Erase + write cycle |
| Directory Lookup | 1-5ms | Inode table scan |

**Memory Requirements:**
- **Minimum**: 2 × block_size + page_size (e.g., 8KB + 256B = ~8.5KB)
- **Recommended**: 2 × block_size + 2 × page_size (for async ops)
- **Allocator**: Not required (zero dynamic allocation)

---

## Roadmap

### Version 0.2.0 (Q2 2026)
- [ ] Active wear leveling algorithm
- [ ] Bad block management
- [ ] Power-loss recovery improvements
- [ ] Extended file attributes (xattr)

### Version 0.3.0 (Q3 2026)
- [ ] Compression support (LZ4)
- [ ] Defragmentation tool
- [ ] Filesystem check utility (fsck)
- [ ] Performance benchmarks

### Version 1.0.0 (Q4 2026)
- [ ] Encryption support (XChaCha20-Poly1305)
- [ ] Snapshot support
- [ ] Production-hardened
- [ ] Full Libertaria stack integration

---

## Testing

**Current Test Coverage:** 251/253 tests passing (99.2%)

```bash
# Run tests
zig build test

# Run with verbose output
zig build test -Dverbose
```

**Test Categories:**
- ✅ Superblock validation
- ✅ Checksum verification
- ✅ Block allocation/deallocation
- ✅ Inode operations
- ✅ Directory operations
- ✅ File operations
- ✅ Path resolution
- 🔄 Wear leveling (in progress)
- 🔄 Bad block handling (planned)

---

## Security Considerations

**Data Integrity:**
- CRC32C protects all metadata from silent corruption
- Dual superblock survives single-block corruption
- Bad block marking prevents data loss

**Power-Loss Resilience:**
- Primary + backup superblock
- Metadata writes are atomic (single block)
- No journal to replay

**Future Security Features:**
- Optional encryption at rest (v1.0)
- Authenticated encryption (AEAD)
- Key derivation from SoulKey (Libertaria integration)

---

## Contributing

**Development Status:** Alpha (v0.1.0)

**Contribution Areas:**
- Wear leveling algorithm improvements
- Bad block detection strategies
- Performance optimizations
- Test coverage improvements
- Documentation enhancements

**Code Style:**
- Follow Zig style guidelines
- SPDX license headers required
- BDD-style tests preferred
- Panopticum architecture compliance

---

## License

**License:** LSL-1.0 (Libertaria Source License 1.0)

**Summary:**
- ✅ Open source and auditable
- ✅ Free to use for sovereign applications
- ✅ Modifications must be contributed back
- ✅ No commercial restrictions for sovereign use cases

See [LICENSE](LICENSE) for full text.

---

## Community

**Repository:** https://git.sovereign-society.org/nexus/nexfs

**Organization:** [Nexus](https://git.sovereign-society.org/nexus)
- rumpk - Runtime package manager
- nip - Nexus package format
- nexus - Core utilities
- nipbox - Package repository
- **nexfs** - Flash filesystem

**Related Projects:**
- [Libertaria Stack](https://git.sovereign-society.org/libertaria/libertaria-stack) - P2P mesh networking
- [Janus Language](https://git.sovereign-society.org/janus/janus) - Systems programming language

---

## Acknowledgments

**Inspired By:**
- **LittleFS** - Flash-friendly embedded filesystem
- **JFFS2** - Journaling flash filesystem
- **YAFFS2** - Yet another flash filesystem

**Built With:**
- **Zig** - Systems programming language
- **Libertaria** - Sovereign P2P mesh network

---

**NexFS** - *Storage for Sovereign Systems*

*Part of the Nexus ecosystem for Libertaria nodes and embedded devices*