nexus
/
nip
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
nip/docs/NIPPELS_DEVELOPER_GUIDE.md

770 lines
18 KiB
Markdown
Raw Blame History

# Nippels Developer Guide
**Complete guide for developers working with Nippels (NimPak Cells)**
---
## Table of Contents
1. [Architecture Overview](#architecture-overview)
2. [Core Components](#core-components)
3. [API Reference](#api-reference)
4. [Extension Points](#extension-points)
5. [Development Setup](#development-setup)
6. [Testing](#testing)
7. [Contributing](#contributing)
---
## Architecture Overview
### High-Level Architecture
```
┌─────────────────────────────────────────────────────────┐
│ Nippel Manager │
│ (nippels.nim - Orchestration & Public API) │
├─────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Profile │ │ Namespace │ │ XDG │ │
│ │ Manager │ │ Subsystem │ │ Enforcer │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ CAS │ │ Merkle │ │ UTCP │ │
│ │ Backend │ │ Tree │ │ Protocol │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Nexter │ │ Decentralized│ │
│ │ Comm │ │ Architecture│ │
│ └──────────────┘ └──────────────┘ │
└─────────────────────────────────────────────────────────┘
```
### Module Dependencies
```
nippels.nim (main orchestrator)
├── nippel_types.nim (shared types)
├── profile_manager.nim (security profiles)
├── namespace_subsystem.nim (Linux namespaces)
├── xdg_enforcer.nim (XDG directory enforcement)
├── cas_backend.nim (content-addressable storage)
├── merkle_tree.nim (integrity verification)
├── utcp_protocol.nim (AI-addressability)
├── nexter_comm.nim (Nexter communication)
└── decentralized.nim (P2P features)
```
## Core Components
### 1. Nippel Manager (nippels.nim)
**Purpose:** Main orchestration and public API
**Key Types:**
```nim
type
NippelManager* = ref object
cells*: Table[string, Nippel]
profileManager*: ProfileManager
activeCells*: HashSet[string]
merkleTrees*: Table[string, MerkleTree]
Nippel* = object
name*: string
profile*: SecurityProfile
settings*: ProfileSettings
rootPath*: string
state*: NippelState
namespaceHandle*: Option[NamespaceHandle]
merkleRoot*: string
utcpAddress*: UTCPAddress
```
**Key Functions:**
```nim
proc createNippel*(manager: NippelManager, name: string,
profile: SecurityProfile): Result[Nippel, NippelError]
proc activateNippel*(manager: NippelManager, name: string): Result[void, NippelError]
proc deactivateNippel*(manager: NippelManager, name: string): Result[void, NippelError]
proc removeNippel*(manager: NippelManager, name: string,
purge: bool = false): Result[void, NippelError]
```
### 2. Profile Manager (profile_manager.nim)
**Purpose:** Security profile management
**Key Types:**
```nim
type
SecurityProfile* = enum
Workstation, Homestation, Satellite, NetworkIOT, Server
ProfileSettings* = object
isolationLevel*: IsolationLevel
desktopIntegration*: bool
networkAccess*: NetworkAccess
resourceLimits*: ResourceLimits
xdgStrategy*: XDGStrategy
ProfileOverrides* = object
isolationLevel*: Option[IsolationLevel]
desktopIntegration*: Option[bool]
networkAccess*: Option[NetworkAccess]
```
**Key Functions:**
```nim
proc loadProfile*(manager: ProfileManager,
profile: SecurityProfile): ProfileSettings
proc applyOverrides*(settings: ProfileSettings,
overrides: ProfileOverrides): ProfileSettings
```
### 3. Namespace Subsystem (namespace_subsystem.nim)
**Purpose:** Linux namespace management
**Key Types:**
```nim
type
IsolationLevel* = enum
None, Standard, Strict, Quantum
NamespaceConfig* = object
mountNS*: bool
pidNS*: bool
ipcNS*: bool
networkNS*: bool
utsNS*: bool
userNS*: bool
NamespaceHandle* = object
pid*: int
namespaces*: NamespaceConfig
```
**Key Functions:**
```nim
proc createNamespaces*(config: NamespaceConfig): Result[NamespaceHandle, string]
proc enterNamespace*(handle: NamespaceHandle): Result[void, string]
proc exitNamespace*(handle: NamespaceHandle): Result[void, string]
proc destroyNamespaces*(handle: NamespaceHandle): Result[void, string]
```
### 4. XDG Enforcer (xdg_enforcer.nim)
**Purpose:** XDG directory enforcement
**Key Types:**
```nim
type
XDGStrategy* = enum
Portable, SystemIntegrated
XDGDirectories* = object
dataHome*: string
configHome*: string
cacheHome*: string
stateHome*: string
```
**Key Functions:**
```nim
proc createXDGStructure*(rootPath: string,
strategy: XDGStrategy): Result[XDGDirectories, string]
proc setXDGEnvironment*(dirs: XDGDirectories): Result[void, string]
proc redirectLegacyPaths*(dirs: XDGDirectories): Result[void, string]
```
### 5. CAS Backend (cas_backend.nim)
**Purpose:** Content-addressable storage with deduplication
**Key Types:**
```nim
type
CASBackend* = ref object
storePath*: string
hashAlgorithm*: HashAlgorithm
refCounts*: Table[string, int]
HashAlgorithm* = enum
Xxh3_128, Blake2b512
```
**Key Functions:**
```nim
proc storeFile*(cas: CASBackend, filePath: string): Result[string, string]
proc retrieveFile*(cas: CASBackend, hash: string,
destPath: string): Result[void, string]
proc garbageCollect*(cas: CASBackend): Result[int, string]
```
### 6. Merkle Tree (merkle_tree.nim)
**Purpose:** Integrity verification
**Key Types:**
```nim
type
MerkleTree* = ref object
root*: MerkleNode
algorithm*: HashAlgorithm
MerkleNode* = ref object
hash*: string
path*: string
children*: seq[MerkleNode]
```
**Key Functions:**
```nim
proc buildTreeFromFiles*(files: seq[string],
algorithm: HashAlgorithm): Result[MerkleTree, string]
proc verifyTree*(tree: MerkleTree): Result[bool, string]
proc updateTree*(tree: MerkleTree,
changes: seq[FileChange]): Result[MerkleTree, string]
proc diffTrees*(tree1, tree2: MerkleTree): seq[FileDiff]
```
### 7. UTCP Protocol (utcp_protocol.nim)
**Purpose:** AI-addressability
**Key Types:**
```nim
type
UTCPAddress* = object
scheme*: string
host*: string
resource*: string
port*: int
UTCPRequest* = object
address*: UTCPAddress
method*: string
headers*: Table[string, string]
body*: string
UTCPResponse* = object
status*: int
headers*: Table[string, string]
body*: string
```
**Key Functions:**
```nim
proc assignUTCPAddress*(nippelName: string): UTCPAddress
proc handleUTCPRequest*(request: UTCPRequest): Result[UTCPResponse, string]
proc formatUTCPAddress*(address: UTCPAddress): string
```
## API Reference
### Creating a Nippel
```nim
import nippels, nippel_types, profile_manager
# Create manager
let manager = newNippelManager()
# Create Nippel with default profile
let result = manager.createNippel("my-app", Workstation)
if result.isOk:
echo "Nippel created successfully"
else:
echo "Error: ", result.error
# Create with custom overrides
let overrides = ProfileOverrides(
isolationLevel: some(Strict),
networkAccess: some(Limited)
)
let result2 = manager.createNippelWithOverrides("secure-app", Satellite, overrides)
```
### Activating a Nippel
```nim
# Activate Nippel
let activateResult = manager.activateNippel("my-app")
if activateResult.isOk:
echo "Nippel activated"
# Now in isolated environment
else:
echo "Activation failed: ", activateResult.error
# Check if active
if manager.isNippelActive("my-app"):
echo "Nippel is active"
# Deactivate
let deactivateResult = manager.deactivateNippel("my-app")
```
### Working with Profiles
```nim
import profile_manager
let profileMgr = newProfileManager()
# Load profile
let settings = profileMgr.loadProfile(Workstation)
echo "Isolation level: ", settings.isolationLevel
echo "Desktop integration: ", settings.desktopIntegration
# Apply overrides
let overrides = ProfileOverrides(
desktopIntegration: some(false)
)
let customSettings = settings.applyOverrides(overrides)
```
### Using CAS Backend
```nim
import cas_backend
let cas = newCASBackend("/var/lib/nip/cas", Xxh3_128)
# Store file
let storeResult = cas.storeFile("/path/to/file.txt")
if storeResult.isOk:
let hash = storeResult.value
echo "File stored with hash: ", hash
# Retrieve file
let retrieveResult = cas.retrieveFile(hash, "/path/to/dest.txt")
# Garbage collect
let gcResult = cas.garbageCollect()
if gcResult.isOk:
echo "Removed ", gcResult.value, " unreferenced entries"
```
### Building Merkle Trees
```nim
import merkle_tree
# Build tree from files
let files = @["/path/to/file1.txt", "/path/to/file2.txt"]
let treeResult = buildTreeFromFiles(files, Xxh3_128)
if treeResult.isOk:
let tree = treeResult.value
echo "Merkle root: ", tree.root.hash
# Verify tree
let verifyResult = tree.verifyTree()
if verifyResult.isOk and verifyResult.value:
echo "Tree verified successfully"
```
### UTCP Protocol
```nim
import utcp_protocol
# Assign UTCP address
let address = assignUTCPAddress("my-app")
echo "UTCP address: ", formatUTCPAddress(address)
# Output: utcp://localhost/nippel/my-app
# Create request
let request = UTCPRequest(
address: address,
method: "GET",
headers: {"Accept": "application/json"}.toTable,
body: ""
)
# Handle request
let response = handleUTCPRequest(request)
if response.isOk:
echo "Response: ", response.value.body
```
## Extension Points
### Custom Security Profiles
You can define custom security profiles:
```nim
# In profile_manager.nim
proc loadCustomProfile*(name: string): ProfileSettings =
case name
of "my-custom-profile":
result = ProfileSettings(
isolationLevel: Strict,
desktopIntegration: true,
networkAccess: Limited,
resourceLimits: ResourceLimits(
maxMemory: 2_000_000_000, # 2GB
maxCPU: 50
),
xdgStrategy: Portable
)
else:
raise newException(ValueError, "Unknown profile: " & name)
```
### Custom Hash Algorithms
Add support for new hash algorithms:
```nim
# In cas_backend.nim
type
HashAlgorithm* = enum
Xxh3_128, Blake2b512, MyCustomHash
proc computeHash*(data: string, algorithm: HashAlgorithm): string =
case algorithm
of Xxh3_128:
result = xxh3_128(data)
of Blake2b512:
result = blake2b_512(data)
of MyCustomHash:
result = myCustomHashFunction(data)
```
### Custom UTCP Methods
Extend UTCP protocol with custom methods:
```nim
# In utcp_protocol.nim
proc handleCustomMethod*(request: UTCPRequest): Result[UTCPResponse, string] =
case request.method
of "CUSTOM_METHOD":
# Handle custom method
let response = UTCPResponse(
status: 200,
headers: {"Content-Type": "application/json"}.toTable,
body: """{"result": "success"}"""
)
return ok(response)
else:
return err("Unknown method: " & request.method)
```
### Custom Namespace Configurations
Define custom namespace configurations:
```nim
# In namespace_subsystem.nim
proc getCustomNamespaceConfig*(level: string): NamespaceConfig =
case level
of "my-custom-level":
result = NamespaceConfig(
mountNS: true,
pidNS: true,
ipcNS: true,
networkNS: false, # Custom: no network isolation
utsNS: true,
userNS: false
)
else:
raise newException(ValueError, "Unknown isolation level: " & level)
```
## Development Setup
### Prerequisites
- Nim 2.0.0 or later
- Linux kernel with namespace support
- xxHash library (`nimble install xxhash`)
- Standard Nim libraries
### Building from Source
```bash
# Clone repository
git clone https://github.com/nexusos/nip.git
cd nip
# Install dependencies
nimble install -d
# Build
nimble build
# Run tests
nimble test
# Install
nimble install
```
### Development Environment
```bash
# Set up development environment
export NIP_DEV_MODE=1
export NIP_LOG_LEVEL=debug
# Build with debug symbols
nim c -d:debug --debugger:native nip/src/nimpak/nippels.nim
# Run with debugger
gdb --args ./nippels
```
### Code Style
Follow Nim standard style guide:
```nim
# Good
proc createNippel*(manager: NippelManager, name: string): Result[Nippel, NippelError] =
## Creates a new Nippel with the given name
if name.len == 0:
return err(NippelError(code: InvalidName, message: "Name cannot be empty"))
# Implementation
ok(nippel)
# Bad
proc CreateNippel(manager:NippelManager,name:string):Result[Nippel,NippelError]=
if name.len==0:return err(NippelError(code:InvalidName,message:"Name cannot be empty"))
ok(nippel)
```
### Documentation Standards
All public APIs must be documented:
```nim
proc createNippel*(manager: NippelManager, name: string,
profile: SecurityProfile): Result[Nippel, NippelError] =
## Creates a new Nippel with the specified security profile.
##
## Parameters:
## - manager: The NippelManager instance
## - name: Unique name for the Nippel
## - profile: Security profile to use
##
## Returns:
## - Ok(Nippel) on success
## - Err(NippelError) on failure
##
## Example:
## ```nim
## let manager = newNippelManager()
## let result = manager.createNippel("my-app", Workstation)
## if result.isOk:
## echo "Created: ", result.value.name
## ```
# Implementation
```
## Testing
### Unit Tests
```nim
# test_nippels.nim
import unittest, nippels, nippel_types
suite "Nippel Manager Tests":
setup:
let manager = newNippelManager()
test "Create Nippel":
let result = manager.createNippel("test-app", Workstation)
check result.isOk
check result.value.name == "test-app"
test "Activate Nippel":
discard manager.createNippel("test-app", Workstation)
let result = manager.activateNippel("test-app")
check result.isOk
check manager.isNippelActive("test-app")
test "Deactivate Nippel":
discard manager.createNippel("test-app", Workstation)
discard manager.activateNippel("test-app")
let result = manager.deactivateNippel("test-app")
check result.isOk
check not manager.isNippelActive("test-app")
```
### Integration Tests
```nim
# test_integration.nim
import unittest, nippels, profile_manager, namespace_subsystem
suite "Integration Tests":
test "Full Nippel Lifecycle":
let manager = newNippelManager()
# Create
let createResult = manager.createNippel("integration-test", Workstation)
check createResult.isOk
# Activate
let activateResult = manager.activateNippel("integration-test")
check activateResult.isOk
# Verify active
check manager.isNippelActive("integration-test")
# Deactivate
let deactivateResult = manager.deactivateNippel("integration-test")
check deactivateResult.isOk
# Remove
let removeResult = manager.removeNippel("integration-test")
check removeResult.isOk
```
### Performance Tests
```nim
# test_performance.nim
import unittest, nippels, times
suite "Performance Tests":
test "Nippel Creation Performance":
let manager = newNippelManager()
let start = cpuTime()
for i in 1..100:
discard manager.createNippel("perf-test-" & $i, Workstation)
let elapsed = cpuTime() - start
let avgTime = elapsed / 100.0
echo "Average creation time: ", avgTime * 1000, " ms"
check avgTime < 0.1 # Should be < 100ms
test "Nippel Activation Performance":
let manager = newNippelManager()
discard manager.createNippel("perf-test", Workstation)
let start = cpuTime()
discard manager.activateNippel("perf-test")
let elapsed = cpuTime() - start
echo "Activation time: ", elapsed * 1000, " ms"
check elapsed < 0.05 # Should be < 50ms
```
### Running Tests
```bash
# Run all tests
nimble test
# Run specific test suite
nim c -r tests/test_nippels.nim
# Run with coverage
nim c -d:coverage -r tests/test_nippels.nim
# Run performance tests
nim c -d:release -r tests/test_performance.nim
```
## Contributing
### Contribution Guidelines
1. **Fork the repository**
2. **Create a feature branch**
```bash
git checkout -b feature/my-new-feature
```
3. **Make your changes**
- Follow code style guidelines
- Add tests for new features
- Update documentation
4. **Run tests**
```bash
nimble test
```
5. **Commit your changes**
```bash
git commit -am "Add new feature: description"
```
6. **Push to your fork**
```bash
git push origin feature/my-new-feature
```
7. **Create a Pull Request**
### Code Review Process
- All PRs require at least one review
- Tests must pass
- Documentation must be updated
- Code style must be consistent
### Areas for Contribution
- **New security profiles**
- **Additional hash algorithms**
- **UTCP protocol extensions**
- **Performance optimizations**
- **Documentation improvements**
- **Bug fixes**
- **Test coverage improvements**
---
## See Also
- [Nippels User Guide](./NIPPELS_USER_GUIDE.md) - User documentation
- [Nippels Troubleshooting](./NIPPELS_TROUBLESHOOTING.md) - Troubleshooting guide
- [Nippels Requirements](../../.kiro/specs/nip-nippels/requirements.md) - Requirements
- [Nippels Design](../../.kiro/specs/nip-nippels/design.md) - Design document
---
**Version:** 1.0
**Last Updated:** November 19, 2025
**Status:** Developer Documentation
**Target Audience:** Developers contributing to Nippels
Reference in New Issue View Git Blame Copy Permalink