Enigma's engine, checkpoint codec, and state validation are Matrix. Bash supplies toolchain glue; Python 3 supplies the POSIX filesystem host and fault/restart verification. No other database implements Enigma's persistence.
Use the public NIL Framework setup instructions. The upstream Linux prerequisites include a C++17 compiler, CMake 3.21 or newer, Python 3.12 or newer, LLVM development packages, and Pandoc. Follow upstream for the current dependency details and other platforms.
For example, build a clean public checkout beside this project:
git clone https://github.com/exec-software/exec-nil-framework.git ../exec-nil-framework
cd ../exec-nil-framework
CMAKE_BUILD_PARALLEL_LEVEL=4 CTEST_PARALLEL_LEVEL=4 ./scripts/build-sdk.shThe framework's build runs its tests and writes sdk/. It expects a clean
checkout. Enigma does not download dependencies or build the framework implicitly.
SDK discovery in scripts/nil.sh uses this order:
- Explicit
NIL_SDK, which must name a complete SDK directory. - The ignored project-local
.sdkdirectory or symlink. - The sibling
../exec-nil-framework/sdk. - The directory containing
nilbuildfound onPATH.
Tools reside directly in the SDK directory. If a PATH entry is a launcher or a
symlink outside that directory, set NIL_SDK explicitly.
The seed is validated with a local development SDK reporting 1.0.0-rc.1.
That SDK reports a dirty source tree, so its version string is not a reproducible
upstream dependency pin. Clean upstream SDK qualification, a pinned toolchain,
and hosted CI are follow-up work; no prebuilt release is assumed.
From the Enigma project directory:
./scripts/nil.sh describe
./scripts/nil.sh check
./scripts/nil.sh build --target all
./scripts/nil.sh run --target native -- --version
./scripts/nil.sh run -- --demo
./scripts/nil.sh run -- --surface
./scripts/build-storage.sh --target all
./scripts/test.shThe wrapper resolves the project independently of the invocation directory and
sets NIL_STRICT_BOUNDS=1, including for tests. This is required because unchecked
array access would undermine storage correctness as the engine develops. The
environment setting is a runtime check, not a database security boundary.
scripts/test.sh checks the engine, companion, and storage-worker projects,
runs Matrix tests, builds the requested execution targets, checks the CLI and
package examples, and exercises Python filesystem faults and process recovery.
Its default is both VM and native; --target vm or --target native selects one.
The installed SDK's atomic file publication lacks a parent-directory sync
barrier and file locking; its ordinary stream flush is not an fsync. The
POSIX-only scripts/store.py supplies those operations without changing the NIL
framework. Python 3 is required for this path. It invokes the separately built
Matrix worker and does not build it implicitly.
./scripts/build-storage.sh --target vm
mkdir -p data
python3 scripts/store.py init data/example --scope work
python3 scripts/store.py serve data/exampleAdd --target native to use the native worker after building that target. Keep
the parent directory available and trusted; init exclusively creates the
private database directory and will not replace an existing one. serve opens
only an existing store and holds its nonblocking writer lock for the session.
It rotates the incarnation and durably retires every previous session grant
before ready; re-admission and new grants are explicit host operations.
See the persistence guide for line-delimited JSON requests,
receipt semantics, and recovery behavior. Do not infer a durable acknowledgement
from a direct Matrix worker response or an enigma.client call. Only the host's
receipt follows file and directory synchronization.
Focused host fault tests can run without a built Matrix worker:
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p test_durable_host.pyThose tests inject a private worker/syscall fixture. They establish host ordering and failure handling, not Matrix policy semantics or hardware power-cut qualification. Full verification also uses real VM/native worker processes.
Build products stay in the ignored build/ directory. Keep local databases,
backups, keys, and credentials out of Git; use synthetic fixtures in tests.
General library gaps found while building Enigma should feed back into NIL.
The first contributions add POSIX durable byte publication, directory sync, and
context-owned advisory file locks (NIL-0028), then owner-only directory
creation, private-entry metadata verdicts, and sha256_hex (NIL-0029), each
with paired Ny/Matrix VM, AOT, and JIT checks. The
consumer contract exercises those primitives
against a real Enigma revoked-grant payload using a matching development SDK:
NIL_SDK=/absolute/path/to/matching/development-sdk ./scripts/test-nil-io.shThe existing host remains the qualified persistence path. A Matrix host now
exists under host/ and satisfies the same black-box
session contract on the VM and native routes, with either host able to serve a
database the other created:
NIL_SDK=/absolute/path/to/matching/development-sdk ./scripts/test-matrix-host.shThat runner also injects storage faults from the filesystem and the process environment, against both hosts: a commit that cannot be published is never acknowledged, an unsafe or multiply-linked checkpoint is refused, a rewritten payload fails its digest, and an oversized request ends a session rather than being truncated into a different one.
scripts/store.py stays the qualified path because the Matrix host needs
NIL-0028 through NIL-0030 and a native random.hex, so making it the default
would leave ./scripts/test.sh unrunnable without a development build. Short
writes, a crash at a chosen commit phase, forged worker replies, and the
host's response to an uncertain publication still need a seam inside a host
and are not claimed. See ENI-0003. Preserve
database policy in Enigma; only general mechanisms belong in the framework
standard library.
Use book enigma.<area> with files under src/enigma/ and .matrix extensions.
Prefer explicit imports and named call arguments. Put begin test declarations
beside the code they exercise; niltest discovers tests through the entrypoint's
import graph. Run the test script with the SDK used to build the executable.