# Provium

> A KVM-backed test harness — tests written in Lua, run inside real virtual machines against a real kernel.

---

# What is Provium

_Provium / Getting started_

> Provium is a KVM-backed test harness for kernel and system code. You write tests in Lua, and they run inside real virtual machines against a real Linux kernel.

Provium is a test harness for software that needs a real operating system underneath it. It runs each test file in its own Lua state, boots real VMs on demand, and exposes a Lua API for driving the guest (commands, files, syscalls, networking, faults), and reports results through the same `passed`/`failed`/`skipped` shape any other test runner would.

It is the runner Peios uses for kernel-side and system-service tests, but there is nothing Peios-specific in the harness — any guest that ships the Provium agent binary works.

## Why Provium exists

Most kernel-adjacent code is hard to test. Unit tests can verify pure logic, but they cannot exercise:

- Driver behaviour against real hardware abstractions (disks, NICs, clocks).
- Networking behaviour under partition, latency, packet loss, or bandwidth caps.
- Multi-host topologies — two guests on the same bridge, three guests in a cluster.
- The boot path itself.
- Failure injection (EIO, slow I/O, link-down, kernel panic recovery).
- Snapshots, restores, and time travel.

The alternatives — containers, mock kernels, in-process simulators — each give up something important. Provium chooses the path of "use a real VM, make it cheap to spawn, and put a high-leverage API on top."

A single test file looks like this:

```lua
test("guest can ping its bridge peer", function(t)
    local lan = provium:bridge("lan")
    local a = provium:vm("a", "peios"):boot()
    local b = provium:vm("b", "peios"):boot()
    lan:attach({a, b})
    a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
    b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")
    a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
end)
```

When the file runs, Provium boots two VMs, sets up a real Linux bridge with TAP interfaces, attaches both, runs the commands inside the guests, and tears everything down at the end. No mocks, no shims. The two `ip addr` lines are ordinary guest commands: Provium wires layer 2 and leaves IP addressing to the guest.

## What Provium gives you

| Capability | What it does |
|---|---|
| **VM lifecycle** | Boot, snapshot, restore, pause, resume, reset, power-button. Snapshots survive across files via the fixture cache. |
| **Layer-1 ops** | `vm:run("cmd")`, `vm:read_file`, `vm:write_file`, `vm:stat`, `vm:mkdir` — the things you'd normally do over SSH, but driven through a vsock agent so they stay fast and self-contained. |
| **Layer-0 ops** | `vm:syscall`, `vm:ioctl` — direct invocations against the guest kernel, with byte-buffer support for in/out parameters. |
| **File handles** | Open, read, write, seek, tell, close, tail. Mirrors POSIX semantics. |
| **Async processes** | `vm:run_async` returns a `Process` userdata you can `:wait`, `:kill`, `:signal`, write `stdin` into, and stream `stdout`/`stderr` from. |
| **Workers** | `vm:spawn_worker()` lets a test concurrently exercise the guest from multiple agent connections without spinning up another VM. |
| **Networking** | Real Linux bridges with TAP interfaces. `bridge:partition`, `bridge:add_latency`, `bridge:drop_rate`, `bridge:bandwidth_limit`, `bridge:isolate`, `bridge:capture` (pcap), and uplink/NAT for outbound traffic. |
| **Disks** | Attach images, read sectors, write sectors, inject `eio_read` / `eio_write` / `slow` faults. |
| **Console** | Read the boot log, stream the chardev, write input. Useful for tests that exercise early-boot behaviour or interactive prompts. |
| **Clock control** | `vm:clock():set`, `:advance`, `:sleep`. Tests that depend on time can move time deterministically. |
| **Streams** | `Tail`/`Capture`/`Console` streams all share `next` / `read_until` / `expect` / `drain` / `close` / `eof` so log-watching, pcap-watching, and console-watching all feel the same. |
| **Fixtures** | `provium:vm_fixture("base")` builds a snapshot once, caches it on disk, and restores it for every test that asks for it. Lab fixtures (`provium:lab_fixture(...)`) cache whole multi-VM topologies the same way. |
| **Resource pool** | A scheduler with memory and CPU budgets. Test files declare what they need with `provium:claim({memory="2G", cpus=4})`; the scheduler runs as many in parallel as the budget allows. |
| **Observability** | Every host-side action emits a structured msgpack event. Pipe it to `provium-coverage`, save to a file, multiplex over a Unix socket, or just watch the human-readable summary. |
| **Determinism aids** | `boot_opts.rng_seed`, `boot_opts.initial_time`, fixture-cache keying that folds in kernel + initrd identity. |

## How it compares

| | Provium | LXC / Docker | KUnit | Mocked I/O |
|---|---|---|---|---|
| **Real kernel** | Yes (per VM) | Shared with host | Yes (single test kernel) | No |
| **Driver-level testing** | Yes | Limited | Limited | No |
| **Network impairments** | Built-in | External tools | No | No |
| **Multi-host topologies** | Built-in | Compose / k8s | No | No |
| **Snapshot + restore** | Built-in (fixtures) | Manual | No | N/A |
| **Fault injection** | Built-in (`fault_inject`, `clock:advance`) | Limited | Limited | Yes |
| **Per-test isolation** | Fresh VM per file | Container per test | Test-binary boundary | Process |
| **Wire protocol exposed** | Yes (`vm:syscall`, `vm:ioctl`) | No | Direct in-kernel | N/A |
| **Test language** | Lua 5.4 | Shell / Go / Python | C | Any |
| **Dependencies** | QEMU, KVM, iproute2, nftables | Docker daemon, etc. | Kernel build | Test framework |

## How it works

```mermaid
flowchart LR
    cli[provium CLI] --> disp[Dispatcher]
    disp --> runner1[File runner #1]
    disp --> runner2[File runner #2]
    runner1 --> lua1[Lua state]
    runner2 --> lua2[Lua state]
    lua1 -->|host bindings| host1[VM, Bridge, Lab]
    host1 -->|QEMU launch| vm1[Guest VM]
    vm1 -->|vsock| agent1[provium-agent]
    agent1 -->|exec, syscall, file ops| guest_kernel[Guest kernel]
```

When you run `provium tests/`, the CLI:

1. Walks `tests/` for `*.test.lua` files.
2. Hands each file to the dispatcher, which acquires resources from the pool.
3. Spins up a Lua state per file, installs the `provium` global, and executes the file.
4. Inside the file, calls like `provium:vm("a", "peios"):boot()` launch real QEMU processes and hand back userdata wrappers around them.
5. Operations against those VMs (`vm:run`, `vm:read_file`, `vm:syscall`) are dispatched over vsock to the `provium-agent` running in the guest.
6. At file end (or per-test if `provium.reset_between_tests = true`), the resource graph is walked in reverse-dependency order and every stream, process, file, worker, VM, and bridge is closed.

The whole loop is observable: every VM spawn, every file dispatched, every fixture cache hit, every test pass/fail, every pool-state snapshot becomes an event on the wire.

## What's not here

- **No browser / GUI testing.** Provium drives guests through agent ops, not graphical interfaces.
- **No Windows guests.** The agent is Linux-only in v1.
- **No live cluster orchestration.** Provium is a test harness, not Terraform.
- **No mock VMM.** Tests run against real KVM. The `--vmm local` mode exists for ad-hoc dev runs without KVM but does not boot a kernel — it just gives the host bindings something to dispatch against.

---

# Quick start

_Provium / Getting started_

> Install Provium, configure a profile, write a first test, and run it. Five minutes from a clean checkout to a passing test.

This guide walks through the minimum setup needed to run one Provium test against a real VM. It assumes you have KVM, QEMU, iproute2, and nftables installed; see [project structure](/provium/getting-started/project-structure.md) for prerequisites detail.

## Install

Provium is a Cargo workspace. Build the host binary:

```
cd provium
cargo build --release --bin provium
```

The binary lands at `target/release/provium`. Optionally, install it onto your `PATH`:

```
cargo install --path provium-host --bin provium
```

Verify:

```
provium --help
```

## Pre-flight check

Provium runs a startup pre-flight on every invocation that can reach a VM (only `lsp-setup` and `prepare` skip it). There is no standalone pre-flight command, so any cheap run — `provium list`, say — exercises it before you start writing tests. To skip the checks (useful in containers without KVM):

```
provium tests/ --no-preflight
```

The pre-flight checks for:

| Check | Recovery |
|---|---|
| `/dev/kvm` exists and is openable | `sudo modprobe kvm-intel` (or `kvm-amd`); add yourself to the `kvm` group |
| `/dev/vhost-vsock` exists | `sudo modprobe vhost_vsock` |
| `ip` and `tc` on `PATH` | `apt install iproute2` / `pacman -S iproute2` |
| `nft` on `PATH` | `apt install nftables` / `pacman -S nftables` |
| `qemu-system-x86_64` on `PATH` | `apt install qemu-system-x86_64` / `pacman -S qemu-base` |
| Effective `CAP_NET_ADMIN` | `sudo setcap cap_net_admin,cap_net_raw=eip $(which provium)` |

## Provide a kernel and initrd

Provium boots VMs by direct kernel boot (`-kernel` / `-initrd`). You need:

- A bzImage-format kernel.
- An initramfs with a working `/init`. Almost any initrd works — a vanilla distro initramfs, a buildroot image, a from-scratch cpio with just `/init` and busybox.

Provium injects its own agent into your initrd at launch (concatenated cpio, content-hash cached) and chains to your `/init`. You don't need to bake `provium-agent` into your image. To opt out (e.g. when your image already bundles an agent), set `inject_agent = false` on the profile — see [provium.toml reference](/provium/configuration/provium-toml.md).

For Peios, the kernel + initrd are built by the Peios image-build pipeline. For ad-hoc use, see the [project structure](/provium/getting-started/project-structure.md) section.

## Create `provium.toml`

Provium loads `./provium.toml` (override with `--config`). The minimum useful config declares one profile:

```toml
[provium]
roots = ["tests"]

[profiles.peios]
kernel  = "/path/to/bzImage"
initrd  = "/path/to/provium-initrd.cpio.gz"
cmdline = "console=ttyS0 quiet"
```

`[profiles.<name>]` is the dictionary `provium:vm("name", "<profile>")` looks up. The `roots` setting is the list of directories scanned for `*.test.lua` and `*.fixture.lua` files; it also controls where `require("helper")` resolves.

See [provium.toml reference](/provium/configuration/provium-toml.md) for every field.

## Write your first test

Create `tests/smoke.test.lua`:

```lua
test("guest boots and runs commands", function(t)
    local vm = provium:vm("smoke", "peios"):boot()
    local r = vm:run("uname -a")
    r:assert_ok()
    t:assert(r.stdout:find("Linux"), "uname should report Linux: " .. r.stdout)
end)

test("filesystem ops round-trip", function(t)
    local vm = provium:vm("smoke", "peios"):boot()
    vm:write_file("/tmp/hello", "world")
    t:assert_eq(vm:read_file("/tmp/hello"), "world")
end)
```

Two things to know:

- Each `test(...)` body runs in its own scope. The two `provium:vm("smoke", "peios")` calls above each create a fresh VM in their respective test scope; the `"smoke"` name in test 1 and test 2 are independent and don't collide. Each VM is silently shut down at end of test.
- To share a VM across tests, declare it at file top-level (`local vm = provium:vm("v", "peios"):boot()`) and either capture the userdata as a Lua local or look it up from inside tests via the 1-arg form `provium:vm("v")`.

## Run it

```
provium tests/
```

You should see:

```
PASS tests/smoke.test.lua (2 passed, 0 failed, 0 skipped)

1 file(s); 2 passed, 0 failed, 0 skipped; 3.42s
```

Useful flags for the development loop:

- `provium tests/ --watch` — re-runs on file change (poll-based, 500 ms).
- `provium tests/smoke.test.lua` — run a single file.
- `provium tests/ --filter smoke` — substring match against the relative path.
- `provium tests/ --rerun-failed` — re-run only files that failed last time.
- `provium tests/ -v` — show passing tests too.
- `provium tests/ --json` — line-delimited JSON output, one object per file.

See [the CLI reference](/provium/reference/cli.md) for every flag.

## Add an assertion that should fail

Sanity-check the failure path:

```lua
test("fails on purpose", function(t)
    t:assert_eq(1, 2, "1 must equal 2")
end)
```

```
FAIL tests/smoke.test.lua (2 passed, 1 failed, 0 skipped)
    FAIL fails on purpose
        1 must equal 2: 1 ~= 2
```

Provium's exit code is `0` when every file passed (or skipped), `1` when one or more files had a failure or timed out, and `2` for internal errors (config load failure, pre-flight failure, etc.).

## What's next

- Read [project structure](/provium/getting-started/project-structure.md) to understand how `provium.toml`, `tests/`, fixtures, and the cache directory all fit together.
- Read the [test-framework reference](/provium/reference/test-framework.md) for everything the `test()` and `t` API expose.
- Read [VMs and profiles](/provium/writing-tests/vms-and-profiles.md) for the full set of `provium:vm(...)` options.
- Skim the [CLI reference](/provium/reference/cli.md) to see what's available for the running side.

---

# Project structure

_Provium / Getting started_

> A Provium project is a directory with provium.toml, test roots holding *.test.lua and *.fixture.lua files, and a per-user fixture cache. Nothing else.

Provium has no scaffolding command. A project is whatever directory `provium.toml` lives in. This page documents what each piece does.

## Directory layout

```
my-project/
  provium.toml                  # Required. Profiles + scan roots.
  tests/                        # One of the directories listed in `roots`.
    smoke.test.lua              # Run by `provium`.
    networking/
      partition.test.lua
      uplink.test.lua
    helpers/
      assert_pingable.lua       # `require("helpers.assert_pingable")`
    fixtures/
      booted-pair.fixture.lua   # Built once, restored per test.
  ~/.cache/provium/
    fixtures/                   # Per-user fixture cache.
      <hash>.snap               # Single-VM fixture snapshots.
      <hash>.lab/               # Multi-VM lab-fixture directories.
    rerun.json                  # `--rerun-failed` state.
```

The only required file is `provium.toml`. Tests, helpers, and fixtures all live under whatever directory you list in `[provium].roots`.

## `provium.toml`

The configuration file at the project root. Two sections:

```toml
[provium]
roots = ["tests", "vendor/upstream-tests"]
cache_dir       = "/var/cache/provium/fixtures"   # optional
cache_max_size  = "20G"                            # optional

[profiles.peios]
kernel   = "/build/peios/bzImage"
initrd   = "/build/peios/initrd.cpio.gz"
cmdline  = "console=ttyS0 quiet"
guest_os = "peios"

[profiles.peios-debug]
kernel   = "/build/peios-debug/bzImage"
initrd   = "/build/peios-debug/initrd.cpio.gz"
cmdline  = "console=ttyS0 debug loglevel=7"
```

`[profiles.<name>]` blocks declare named (kernel, initrd, cmdline) tuples. Test code looks them up by name: `provium:vm("a", "peios")` boots the VM using `[profiles.peios]`. You can have any number of profiles; tests pick whichever they need.

`[provium].roots` is the list of directories scanned for `*.test.lua` files. It also controls where `require("helper.module")` resolves; `helpers/foo.lua` under any root is reachable as `require("helpers.foo")`.

`cache_dir` and `cache_max_size` configure where fixture snapshots are stored and how big the cache is allowed to grow. Defaults: `~/.cache/provium/fixtures/`, `20 GiB`. See [provium.toml reference](/provium/configuration/provium-toml.md) for full detail.

## Test files

Files matching `*.test.lua` under any `roots` directory are picked up by `provium`. Each file is run in a fresh Lua state with a fresh root `Lab`.

A test file consists of `test(name, [meta,] fn)` calls. Tests run in declaration order, and each test runs sequentially (one at a time within a file). The harness builds a fresh `t` context for each test and passes it as the function's first argument.

```lua
test("simple", function(t)
    t:assert_eq(1 + 1, 2)
end)

test("with metadata", {tags = {"smoke"}, timeout = "30s"}, function(t)
    t:assert(true)
end)

test("declaratively skipped", {skip = "still figuring out the spec"}, function(t)
    -- never runs
end)
```

See [the test-framework reference](/provium/reference/test-framework.md) for `test()`, the `t` context, `todo()`, and `wait_until()`.

## Fixture files

Files matching `*.fixture.lua` are not picked up by the test runner directly. Instead, test files reference them by name:

```lua
test("uses a pre-booted VM", function(t)
    local vm = provium:vm_fixture("booted-base")
    -- vm is restored from a cached snapshot
end)
```

When the test runs, Provium:

1. Locates `<root>/booted-base.fixture.lua` (under any `roots` directory).
2. Hashes the file's source bytes plus every transitive dependency (other fixtures it references via `vm_fixture`/`lab_fixture`, every helper it `require`s, and any host file declared with `depends_on_file`) plus every profile's kernel and initrd identity and the wire-protocol version into a cache key.
3. Looks up `<cache_dir>/<key>.snap`. If present, restores it and hands the test a fresh VM.
4. If absent, builds the fixture by running its chunk under a build-time Lua state, takes the resulting snapshot, sparse-zstd-compresses it, and installs it into the cache.

A fixture file's chunk must end in a `return` statement that hands back either a `vm:snapshot()` (single VM) or a `provium:snapshot()` (whole lab). Example:

```lua
-- booted-base.fixture.lua
local vm = provium:vm("base", "peios"):boot()
vm:run("mkdir -p /srv/state"):assert_ok()
return vm:snapshot()
```

See [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) for the cache lifecycle, eviction policy, and what triggers a rebuild.

## Helper files

Plain Lua files anywhere under a `roots` directory are reachable through `require`. The `roots` directories are prepended to `package.path` at file-runner setup, so `tests/helpers/assert_pingable.lua` can be loaded as:

```lua
local pingable = require("helpers.assert_pingable")
```

Helpers are not detected as tests (they don't end in `.test.lua`) and not detected as fixtures. Editing a helper invalidates the cache key of every fixture that transitively requires it.

## The fixture cache

```
~/.cache/provium/fixtures/
  c5e6f8….snap           # Single-VM fixture snapshot (sparse, zstd).
  c5e6f8….lock           # Per-key build lock.
  abc123….lab/           # Lab-fixture directory.
    lab.json             # Per-VM snapshot index.
    base.snap            # Per-VM snapshot.
    extra.snap
  abc123….lab.lock
```

The cache is per-user by default (`~/.cache/provium/fixtures/`). Override it system-wide with `[provium].cache_dir` in `provium.toml`. The cache is shared across runs and across files within a run — concurrent file runners building the same fixture coordinate via the per-key lock file.

LRU eviction runs at `provium` startup before tests dispatch: the harness sums file sizes under `cache_dir`, sorts entries by access time, and deletes the oldest until the total is under `cache_max_size`.

Inspect or manage the cache from the CLI:

```
provium fixture list      # Show every cached entry, with sizes.
provium fixture build P   # Force-build the named fixture.
provium fixture rebuild P # Evict and rebuild.
provium fixture clean     # Wipe the cache.
provium fixture stale     # List fixtures whose source hashes don't match any cache entry.
```

## The rerun state file

After every run that produced at least one failure, Provium writes the canonical paths of the failing files to `~/.cache/provium/rerun.json` (override with `$PROVIUM_RERUN_STATE`). The `--rerun-failed` flag intersects discovered files against this list, so:

```
provium tests/ --rerun-failed
```

re-runs only files that failed last time. A clean run leaves the file alone, so the failed set persists until you fix the failures.

## Required external binaries

| Binary | Used for | Provided by |
|---|---|---|
| `qemu-system-x86_64` | VMM backend | `qemu-system-x86` package |
| `ip` | Bridge / TAP / link operations | `iproute2` |
| `tc` | Latency / drop-rate / bandwidth qdiscs | `iproute2` |
| `nft` | Per-bridge partition rules, NAT for uplink | `nftables` |
| `tcpdump` | `bridge:capture()` and `nic:capture()` | `tcpdump` |

Provium's startup pre-flight checks for `/dev/kvm`, `/dev/vhost-vsock`, `ip`, `tc`, `nft`, `qemu-system-x86_64`, and effective `CAP_NET_ADMIN`. Missing pieces fail with an actionable message before any test runs. The exception is `tcpdump`: the pre-flight does not check for it, because it is only needed once a test calls `capture()`.

## What lives outside the project tree

- The fixture cache (default: `~/.cache/provium/fixtures/`).
- The rerun-state file (default: `~/.cache/provium/rerun.json`).
- Optional `--save-events` / `--events-socket` paths (your choice).
- The kernel and initrd files referenced by `[profiles.<name>]`.

Everything else — tests, fixtures, helpers, configuration — lives inside the project directory and is the project author's responsibility to keep version-controlled.

---

# Writing tests with test() and t

_Provium / Writing tests_

> How to register tests, organise a file, use assertions, skip cleanly, log diagnostic data, and pick per-test metadata.

A Provium test file is a sequence of `test(...)` calls. The harness runs them in declaration order, gives each one a fresh `t` context, and records pass / fail / skip. This page covers the patterns for getting the most out of that loop.

The exhaustive reference is on [test framework](/provium/reference/test-framework.md) and [meta tags](/provium/reference/meta-tags.md).

## Anatomy of a test file

```lua
-- tests/file_handles.test.lua

provium.timeout = "10s"        -- file-default per-test timeout

test("baseline I/O round-trips", function(t)
    local vm = provium:vm("v", "peios"):boot()
    vm:write_file("/tmp/data", "hello")
    t:assert_eq(vm:read_file("/tmp/data"), "hello")
end)

test("permission bits stick on create", {tags = {"perms"}}, function(t)
    local vm = provium:vm("v", "peios"):boot()
    local h = vm:open_file("/tmp/secret", {write=true, create=true, perm=0x180})  -- 0o600
    h:close()
    t:assert_eq(vm:stat("/tmp/secret").perm, 0x180)
end)
```

Two things to know up front:

1. **Each `test()` body runs in its own scope.** The `provium:vm("v", …)` calls in the two tests above each create their own VM in their test scope; they're independent and do not share state. The VMs are silently shut down at test end. To share a VM across tests, declare it at file scope (top-level `local vm = provium:vm("v", "peios"):boot()`) and look it up by name (`provium:vm("v")`) or capture the userdata as a Lua local.
2. **Each test gets a fresh `t` context.** `t.name` and `t.meta` are per-test; assertions and skips are scoped to the running test.

For the full scoping rules (lookup fallthrough, shadow detection, fixtures), see [labs and scope](/provium/writing-tests/labs-and-scope.md).

## Naming tests

Test names must be unique within a file. The harness raises at registration time on a duplicate:

```
test: duplicate name `boots` in this file
```

Pick names that read like the assertion, not the implementation. `"boots and runs uname"` is better than `"test_boot_uname"` — the renderer prefixes them with status (`PASS`, `FAIL`) and indents under the file path, so the name reads as a sentence.

## Assertions

The `t` context exposes:

| Assertion | Use when |
|---|---|
| `t:assert(cond, msg?)` | You're checking any boolean condition. |
| `t:assert_eq(a, b, msg?)` | Two values must be equal. The error message includes both values. |
| `t:assert_neq(a, b, msg?)` | Two values must be unequal. |
| `t:assert_contains(haystack, needle, msg?)` | A string must appear inside another string. Both args must be strings. |
| `t:assert_raises(fn, msg?)` | Calling `fn` must raise; returns the error value. |
| `t:fail(msg?)` | You've decided the test failed for reasons that don't fit an assertion. |

Every assertion that fires raises (with `error(..., 2)` so the call site, not the assertion implementation, is in the message). The harness catches the raise and records the test as Failed.

Even when you wrap a body in `pcall` and swallow the error, the harness still detects the failure — an assertion marks the test failed before it raises, and that mark is sticky:

```lua
test("expected to detect a failure", function(t)
    local ok = pcall(function() t:assert(false, "should fail") end)
    -- ok is false (we caught the assert), but the test still
    -- reports Failed: the failure was recorded before the raise.
end)
```

Use `t:assert_raises(fn)` if you want the inverse — "this should raise":

```lua
test("write to a closed handle raises", function(t)
    local h = vm:open_file("/tmp/x", {write=true, create=true})
    h:close()
    local err = t:assert_raises(function() h:write("oops") end)
    t:assert_contains(tostring(err), "closed")
end)
```

## Skipping

Three ways to skip:

### Inline (`t:skip(reason)`)

```lua
test("only on btrfs", function(t)
    if not is_btrfs(vm) then t:skip("not a btrfs root") end
    -- … rest of test body …
end)
```

`t:skip` raises an internal sentinel that the harness treats as Skipped. Useful when the skip condition can only be evaluated at runtime.

### Declarative (`{skip = …}`)

```lua
test("not implemented yet", {skip = true}, function(t) … end)
test("waiting on RFC", {skip = "see issue #42"}, function(t) … end)
```

The body never runs. Cleaner than inline when the test is permanently disabled or pending an unrelated change.

### File-scope (`todo("reason")`)

```lua
todo("waiting on the spec")
test("a", function(t) … end)
test("b", function(t) … end)
```

Every registered test is reported Skipped with the given reason. Use this when an entire test file is non-applicable temporarily — do not delete the tests, just mark the file pending.

## Logging diagnostic data

`t:log(msg)` appends to a per-test log array. The harness includes the log in the file outcome and the `TestPassed` / `TestFailed` events — useful for diagnostic context when something goes wrong:

```lua
test("retries until success", function(t)
    for i = 1, 5 do
        local r = vm:run("flaky-command")
        t:log("attempt " .. i .. " -> exit " .. r.exit_code)
        if r:ok() then return end
    end
    t:fail("flaky-command failed 5 times")
end)
```

Avoid `print` in tests — `print` writes to stdout and gets interleaved with the harness's own output, while `t:log` is structured and tied to the specific test.

## Per-test metadata

The optional second arg to `test(...)` is a metadata table. Provium inspects a handful of well-known keys:

| Key | Purpose |
|---|---|
| `slow = true` | Skip unless `--include-slow` is passed. |
| `skip = …` | Declarative skip. |
| `tags = {...}` | Tag-based filtering (`--tag`, `--no-tag`). |
| `timeout = "30s"` | Per-test wall-clock timeout. |
| `spec = "PSD-…"` | Spec linkage for `provium-coverage`. |

Anything else passes through to event consumers. See [meta tags](/provium/reference/meta-tags.md) for the full reference.

```lua
test("federation handshake", {
    tags    = {"federation", "slow"},
    timeout = "5m",
    spec    = "PSD-FEDERATION §3.1",
}, function(t) … end)
```

## Polling with `wait_until`

`wait_until(predicate, opts?)` calls `predicate` repeatedly until it returns truthy. Use it for guest-side conditions that don't have a stream interface:

```lua
test("nginx comes up", function(t)
    vm:run("systemctl start nginx"):assert_ok()
    local pid = wait_until(function()
        local r = vm:run("pidof nginx")
        if r:ok() then return r.stdout:match("%d+") end
    end, {timeout = "30s", interval = "200ms", desc = "nginx running"})
    t:assert(pid)
end)
```

For things that produce a stream (logs, console output, captured stdout), prefer `:expect` on the stream over `wait_until` — it gets event-driven semantics and much tighter feedback.

## File-default timeouts

Set `provium.timeout` at file scope to put a default on every test:

```lua
provium.timeout = "30s"

test("…", function(t) … end)        -- 30s
test("…", {timeout = "5m"}, function(t) … end)  -- per-test wins; 5m
```

Per-test `meta.timeout` always wins over the file default. When a per-test timeout fires, the watchdog tears down the **entire file's lab** — there's no finer-grained cancellation in v1. See [time and timeouts](/provium/reference/test-framework.md#time-and-timeouts) for the scope-limitation note.

## Reset-between-tests

Set `provium.reset_between_tests = true` at file scope to take a baseline snapshot after the file's top-level chunk and restore it between every test:

```lua
provium.reset_between_tests = true

local vm = provium:vm("v", "peios"):boot()
vm:write_file("/etc/marker", "baseline")  -- snapshot baseline includes this

test("a", function(t)
    vm:run("rm /etc/marker"):assert_ok()
    -- After this test, the lab is restored: the marker is back.
end)

test("b", function(t)
    vm:run("test -f /etc/marker"):assert_ok()  -- still present, baseline restored
end)
```

**Mutually exclusive with file-scope open streams.** Opening a `tail_file`, `console:read`, `bridge:capture`, etc. at top-level errors at chunk load with the offending stream's creation site named. Move stream opens into `test()` bodies.

## Common patterns

### One-test-per-VM

When tests are independent and a fresh VM per test is acceptable, opt into reset-between-tests:

```lua
provium.reset_between_tests = true

test("a", function(t)
    local vm = provium:vm("v", "peios"):boot()
    -- … tests against fresh state …
end)
```

### One-VM-many-tests, ordered

When tests build on each other, leave `reset_between_tests` off and let state accumulate:

```lua
local vm = provium:vm("v", "peios"):boot()

test("create user", function(t)
    vm:run("useradd alice"):assert_ok()
end)

test("user appears in /etc/passwd", function(t)
    t:assert(vm:read_file("/etc/passwd"):find("^alice:"))
end)
```

Order matters here. If you delete the first test, the second will fail; that's intentional.

### Fixture-backed setup

When the setup is expensive (install packages, fetch data, build a config), put it in a `*.fixture.lua` file and call `provium:vm_fixture("name")`:

```lua
-- tests/fixtures/corpus.fixture.lua
local vm = provium:vm("base", "peios"):boot()
vm:run("seq 1000000 > /srv/corpus"):assert_ok()
return vm:snapshot()

-- tests/uses-corpus.test.lua
test("corpus is pre-built", function(t)
    local vm = provium:vm_fixture("fixtures/corpus")
    vm:run("test -s /srv/corpus"):assert_ok()
end)
```

The fixture is built once, cached on disk, and restored per test that asks for it. See [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) for the cache lifecycle.

## See also

- [Test framework reference](/provium/reference/test-framework.md) — every method on `t`, plus `wait_until` and `todo`.
- [Meta tags reference](/provium/reference/meta-tags.md) — every well-known meta key.
- [Labs and scope](/provium/writing-tests/labs-and-scope.md) — `lab:claim`, `lab:barrier`, `reset_between_tests`.

---

# VMs and profiles

_Provium / Writing tests_

> Create, boot, snapshot, restore, pause, and reset VMs; how profiles map names to kernel and initrd choices, and how boot opts split between create and boot.

Every Provium test ultimately drives one or more guest VMs. This page covers the lifecycle: creating a VM, picking its profile, controlling boot, taking snapshots, and tearing down.

The exhaustive reference is on [VM](/provium/reference/vm.md).

## Creating a VM

```lua
local vm = provium:vm("name", "profile")
```

The first arg is the VM's name (per-scope unique). The second arg is a profile name from `provium.toml`. The VM is in `Created` state until you call `:boot()`.

**Two forms.** `provium:vm(name, profile)` *creates* a VM in the current scope. `provium:vm(name)` *looks one up* by name, walking from the current scope to the file root. Inside a `test()` body, the current scope is the per-test scope; at file top-level, it's the file root. See [labs and scope](/provium/writing-tests/labs-and-scope.md) for the full rules.

Optional third arg is a sizing table — the two keys the scheduler and QEMU need before boot:

```lua
local vm = provium:vm("v", "peios", {memory = "2G", cpus = 4})
```

Everything else about a boot — kernel command line, determinism seeds, injected files — is passed to `vm:boot(opts)` instead:

```lua
vm:boot({
    kernel_cmdline = "console=ttyS0 quiet earlyprintk=ttyS0",
    rng_seed = 0xdeadbeef,
    initial_time = 1700000000,
    files = {
        {path = "/etc/hostname", content = "v"},
        {path = "/root/.ssh/authorized_keys", content = pubkey},
    },
})
```

The split is deliberate: `memory` and `cpus` size the VM, and the scheduler needs them when the VM is declared; the boot opts shape one particular boot and merge per field into the VM's pending boot options.

## Booting

```lua
local vm = provium:vm("v", "peios"):boot()
```

`:boot()` returns `self` so you can chain. The VM is in `Booted` state on return. A `vm_spawned` event fires as soon as the agent has handshaken; consumers see it before `:boot()` returns.

For multi-VM tests, you can boot each individually:

```lua
local a = provium:vm("a", "peios"):boot()
local b = provium:vm("b", "peios"):boot()
```

Or batch-boot via the lab:

```lua
provium:vm("a", "peios")
provium:vm("b", "peios")
provium:boot()                -- boots both
```

The batch form is mainly useful when you've declared a topology in a fixture builder and want to bring it up atomically.

## Profiles

A profile is a `[profiles.<name>]` block in `provium.toml`:

```toml
[profiles.peios]
kernel   = "/build/peios/bzImage"
initrd   = "/build/peios/initrd.cpio.gz"
cmdline  = "console=ttyS0 quiet"
guest_os = "peios"
```

Each profile names a (kernel, initrd, cmdline) tuple. A test picks which profile to use by name:

```lua
provium:vm("a", "peios")           -- uses [profiles.peios]
provium:vm("a", "peios-debug")     -- uses [profiles.peios-debug]
```

You can have any number of profiles. Common patterns:

| Pattern | Profiles |
|---|---|
| Test against multiple kernel versions | `peios-stable`, `peios-mainline` |
| Compare optimised and debug builds | `peios`, `peios-debug` |
| Test pre/post a feature flag | `peios-prefeatx`, `peios-postfeatx` |

Multi-profile fixtures invalidate every cached fixture when any profile's kernel or initrd identifier changes — see [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md).

## Lifecycle methods

A VM's lifecycle: `:boot()` takes it from `Created` to `Booted`; `:pause()` / `:resume()` toggle between `Booted` and `Paused`; `:reset()` warm-reboots (still `Booted`); and `:shutdown()` or `:power_button()` end in `Shutdown`. The full state machine and per-method semantics are in the [VM reference](/provium/reference/vm.md#state-machine).

Operations against a VM in the wrong state error cleanly with a hint: `VM not booted` on a `Created` VM, `VM is paused; use resume()` on a `Paused` one, `VM is shutdown; create a new one` after shutdown.

You typically don't call `:shutdown()` explicitly. The harness's resource-graph walker tears every VM down at the appropriate scope boundary: test-scope VMs at the end of their `test()` body, file-scope VMs at file end. (`reset_between_tests = true` snapshots and restores instead — see [labs and scope](/provium/writing-tests/labs-and-scope.md).)

## Boot opts in detail

The opts you'll use most often are `memory` and `cpus` (sizing, at creation), `kernel_cmdline` (per-VM `loglevel`, `nokaslr`, etc. — replaces the profile's `cmdline`; at boot), and `files` (inject config before init runs; at boot):

```lua
local vm = provium:vm("v", "peios", {memory = "1G", cpus = 2})
vm:boot({
    files = {
        {path = "/etc/test.conf", content = "key=value\n"},
    },
})
```

This is a subset — the full boot-opts table (types, defaults, `rng_seed`, `initial_time`) is in the [VM reference](/provium/reference/vm.md#boot-opts).

## Querying VM state

The accessors you'll use most are `vm:state()` (returns `"created"`, `"booted"`, `"paused"`, `"shutdown"`, or `"dead"`) and `vm:is_quiescent()` (true when there are no in-flight ops, open files, or open streams). The full accessor list — `name`, `profile`, `cid`, `open_file_count`, `open_stream_count` — is in the [VM reference](/provium/reference/vm.md#accessors).

`is_quiescent` and the open-count accessors are useful for snapshot precondition asserts:

```lua
test("snapshot is taken at quiescence", function(t)
    local vm = provium:vm("v", "peios"):boot()
    vm:run("…")
    t:assert(vm:is_quiescent(), "VM must be quiescent before snapshot")
    local s = vm:snapshot()
    t:assert(s:size() > 0)
end)
```

## Snapshots

```lua
local s = vm:snapshot()              -- writes to a tempfile
local s = vm:snapshot("/tmp/x.snap") -- writes to that path
```

Returns a [Snapshot](/provium/reference/snapshot.md) userdata wrapping the path. Use it to:

- Restore later in the same test: `vm:shutdown(); vm:restore(s)`.
- Inspect size: `s:size()`.
- Delete: `s:delete()` (idempotent).

The snapshot file is what fixture builders return. If the snapshot fails because of an open stream, the error names the stream's creation site — close streams before snapshotting:

```lua
test("snapshot needs no open streams", function(t)
    local vm = provium:vm("v", "peios"):boot()
    local stream = vm:tail_file("/var/log/messages")
    -- vm:snapshot() would error here; close first.
    stream:close()
    local s = vm:snapshot()
end)
```

## Restoring

Restore from a Snapshot userdata or a bare path string:

```lua
vm:shutdown()
vm:restore(s)            -- from snapshot userdata
vm:restore("/tmp/x.snap") -- from path
```

The VM moves through `Shutdown → Created → Booted` (the restored state is already `Booted`). Restoring requires the VM to be in `Created` or `Shutdown` first.

## Determinism patterns

For tests that depend on randomness or wall-clock time, fix both at boot:

```lua
local vm = provium:vm("v", "peios")
vm:boot({
    rng_seed = 0xdeadbeef,
    initial_time = 1700000000,
})
```

After boot, you can move time forward (or backward) with `vm:clock():advance(N)` — see [Clock reference](/provium/reference/clock.md).

## Pausing for inspection

`vm:pause()` freezes the guest's vCPUs. Useful for:

- Time-sensitive tests where you need to read multiple bits of state without races.
- Snapshotting (the snapshot path will pause anyway, but explicit pause makes the test's intent clear).

```lua
vm:pause()
local before = vm:read_file("/proc/loadavg")
vm:resume()
```

## Reset and power-button

`vm:reset()` warm-reboots the guest — same VM, same RAM image initially, then init re-runs. Stays in `Booted`.

`vm:power_button()` sends ACPI power-button. The guest's init handles it as a graceful shutdown signal (typically: stop services, sync filesystems, kernel halts). Ends in `Shutdown`.

```lua
test("guest survives reset", function(t)
    local vm = provium:vm("v", "peios"):boot()
    vm:write_file("/tmp/before", "1")
    vm:reset()
    -- /tmp is tmpfs in the test profile; vm:reset() is a warm
    -- reboot, so /tmp is fresh. Check that the test's expectation
    -- matches what the profile actually does.
    local r = vm:run("test ! -f /tmp/before")
    r:assert_ok()
end)
```

## Multi-VM topologies

The two-VM pattern is the workhorse of networking tests:

```lua
local lan = provium:bridge("lan")
local a   = provium:vm("a", "peios"):boot()
local b   = provium:vm("b", "peios"):boot()
lan:attach({a, b})
a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")

test("a can reach b", function(t)
    a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
end)
```

For larger topologies, use sub-labs to keep names organised — see [labs and scope](/provium/writing-tests/labs-and-scope.md).

## See also

- [VM reference](/provium/reference/vm.md) — every method, every option.
- [Snapshot reference](/provium/reference/snapshot.md) — snapshot/lab-snapshot userdata.
- [provium.toml reference](/provium/configuration/provium-toml.md) — profile configuration.
- [Bridges and impairments](/provium/writing-tests/bridges-and-impairments.md) — wiring VMs together.

---

# Running commands inside the guest

_Provium / Writing tests_

> Patterns for vm:run — shell form vs direct exec, env, cwd, stdin, timeouts — plus vm:run_async process handles, signals, and workers.

Provium gives you several ways to run commands inside a guest. This page is the practical guide; the canonical method reference is on [VM](/provium/reference/vm.md).

## Sync exec: `vm:run`

The default. Returns a [RunResult](/provium/reference/vm.md#runresult) with `exit_code`, `stdout`, `stderr`, `status`, `signal`, `timed_out`.

```lua
local r = vm:run("echo hello")
r:assert_ok()
t:assert_eq(r.stdout, "hello\n")
```

Two call shapes:

### Shell form: `vm:run(string)`

```lua
vm:run("echo hello | tr a-z A-Z")
vm:run("ls /etc/*.conf | head -3")
```

The string is run through `/bin/sh -c "<string>"` so shell metacharacters work. Convenient for one-liners; quote-handling is the shell's problem.

### Direct exec: `vm:run(cmd, {args, ...})` or `vm:run(cmd, {arr})`

```lua
vm:run("echo", {args = {"hello"}})            -- canonical
vm:run("echo", {"hello"})                     -- legacy bare-array form
```

No shell. The first arg is the executable; positional args go in `args` (or as a bare array when no opts keys are present). Use this when:

- The args contain shell metacharacters you don't want interpreted.
- You don't want a shell process in your tree (PID, signal handling, etc.).
- You're passing user-controlled data that you'd otherwise have to quote.

The detection between the two table forms is by recognised keys: the presence of any of `env`, `env_clear`, `cwd`, `stdin`, `timeout`, `timeout_ms`, or `args` selects the opts form; a table without any of them is treated as the legacy direct-args array.

## Environment, cwd, stdin

```lua
vm:run("printenv FOO", {env = {FOO = "bar"}})
vm:run("ls", {cwd = "/etc"})
vm:run("cat", {stdin = "hello\n"})
```

Combine freely:

```lua
local r = vm:run("python3 -c 'import os, sys; print(os.environ[\"X\"], file=sys.stderr); print(sys.stdin.read())'", {
    env  = {X = "from-env"},
    stdin = "from-stdin\n",
})
t:assert_eq(r.stderr, "from-env\n")
t:assert_eq(r.stdout, "from-stdin\n")
```

`env_clear = true` makes the guest see ONLY the keys you supplied:

```lua
vm:run("env", {env = {PATH = "/usr/bin"}, env_clear = true})
```

Without `env_clear`, your `env` merges with the agent's environment.

## Timeouts

Two equivalent keys: `timeout_ms` (int, milliseconds) and `timeout` (number seconds, or string with suffix).

```lua
vm:run("sleep 100", {timeout_ms = 500})        -- TimedOut
vm:run("sleep 100", {timeout = 0.5})           -- Same
vm:run("sleep 100", {timeout = "500ms"})       -- Same
vm:run("sleep 100", {timeout = "5s"})          -- 5s
vm:run("sleep 100", {timeout = "1m"})          -- 1 minute
```

When the timeout fires, the agent kills the process and returns a RunResult with `status = "timed_out"`, `timed_out = true`, and `exit_code = -2`. Use `r.timed_out` to disambiguate from a clean exit with code -2.

```lua
local r = vm:run("flaky", {timeout = "2s"})
if r.timed_out then
    t:fail("flaky timed out")
elseif not r:ok() then
    t:fail("flaky failed: " .. r.stderr)
end
```

## RunResult fields and helpers

Most tests only need three things from a RunResult:

```lua
local r = vm:run("…")

r:assert_ok() -- raise if not ok; message includes status, stdout, stderr
r.stdout      -- captured stdout (bytes)
r:ok()        -- shorthand for exit_code == 0
```

When a command can fail in more than one way, check `r.status` (`"exited"` / `"signalled"` / `"timed_out"`) rather than pattern-matching `exit_code` — the sentinel exit codes and the `signal` field are documented in the [RunResult reference](/provium/reference/vm.md#runresult).

## Async exec: `vm:run_async`

Returns a [Process](/provium/reference/process.md) userdata immediately. The agent does NOT auto-kill; you control the lifetime.

```lua
local proc = vm:run_async("python3", {args = {"server.py"}})
-- … do other things …
proc:kill("term")
local r = proc:wait("2s")
```

The opts shape mirrors `vm:run`'s, **except** that passing `timeout` / `timeout_ms` is rejected — use `proc:wait(timeout)` instead. (Silently honouring `timeout` here would be a footgun: the agent doesn't auto-kill, so the timeout would do nothing.)

### Stdin pipe

```lua
local proc = vm:run_async("cat")
proc:stdin_write("hello\n")
proc:stdin_write("world\n")
proc:close_stdin()
local r = proc:wait("5s")
t:assert_eq(r.stdout, "hello\nworld\n")
```

### Streaming stdout / stderr

```lua
local proc = vm:run_async("server")
local out  = proc:stdout_stream()
out:expect("listening on port 8080", "10s")
-- now exercise the server
```

See [streams and tails](/provium/writing-tests/streams-and-tails.md) for the full stream API.

### Signals

```lua
proc:kill()                  -- defaults to SIGTERM
proc:kill(9)                 -- SIGKILL by number
proc:kill("kill")            -- by name
proc:kill("usr1")            -- SIGUSR1
proc:signal("usr2")          -- alias for kill(); reads better for non-fatal signals
```

Signals are accepted by friendly name (`term`, `kill`, `usr1`, …), with a `sig` prefix (`sigterm`), or as a bare integer. The full recognised-name list is in the [Process reference](/provium/reference/process.md#prockillsig).

### Inspecting the process

```lua
proc:pid()       -- live kernel PID inside the guest
proc:handle()    -- opaque agent-side handle id
proc:status()    -- non-blocking poll
```

`pid()` calls into the agent every time. `handle()` is a stable in-memory id that never changes.

### Waiting

```lua
proc:wait()              -- wait forever
proc:wait("5s")          -- wait at most 5 seconds
proc:wait(0.5)           -- 500 ms
```

Don't pass `0` — the harness rejects it and points you at `proc:status()` for non-blocking polling. The [Process reference](/provium/reference/process.md#procwaittimeout) explains why a literal-zero timeout is a footgun.

## Workers: parallel commands in the same VM

`vm:spawn_worker()` returns a [Worker](/provium/reference/worker.md) — a sub-agent connection. Useful for driving the same VM from multiple threads of control:

```lua
local w1 = vm:spawn_worker()
local w2 = vm:spawn_worker()

-- Two writers in parallel.
local p1 = w1:run_async("dd if=/dev/urandom of=/tmp/a bs=1M count=8")
local p2 = w2:run_async("dd if=/dev/urandom of=/tmp/b bs=1M count=8")

p1:wait("10s"):assert_ok()
p2:wait("10s"):assert_ok()
```

Workers expose the same surface as the VM (`run`, `run_async`, `open_file`, `syscall`, `kill`, `join`, `close`). Files and processes allocated under a worker live in the worker's namespace; cleanup is per-worker.

For coordination between workers' guest processes, use a guest-side primitive (file on a shared mount, fifo, network message). [`lab:barrier(name, count, timeout?)`](/provium/reference/lab.md#labbarriername-count-timeout) is a host-side rendezvous and can't be reached from inside a guest — see [Labs and scope — Barriers](/provium/writing-tests/labs-and-scope.md#barriers).

## Common patterns

### "Did the fixture build correctly?"

```lua
test("fixture data is present", function(t)
    local vm = provium:vm_fixture("base")
    local r = vm:run("wc -c < /srv/corpus")
    r:assert_ok()
    t:assert(tonumber(r.stdout) > 0)
end)
```

### "Run a server, hit it, tear down"

```lua
test("nginx serves index", function(t)
    local vm = provium:vm("v", "peios"):boot()
    local proc = vm:run_async("nginx", {args = {"-g", "daemon off;"}})
    local out  = proc:stdout_stream()
    -- nginx writes nothing to stdout in this mode; use wait_until on
    -- the listening port.
    wait_until(function()
        return vm:run("ss -ltn | grep :80"):ok()
    end, {timeout = "10s", desc = "nginx listening on :80"})
    local r = vm:run("curl -s http://localhost/")
    r:assert_ok()
    t:assert(r.stdout:find("Welcome to nginx"))
end)
```

### "Compose multiple ops in one round trip"

```lua
local results = vm:batch(function(b)
    b:write_file("/tmp/a", "1")
    b:write_file("/tmp/b", "2")
    b:read_file("/tmp/a")
    b:stat("/tmp/missing")
end)
-- results[3].ok == "1"
-- results[4].err contains "No such file"
```

`vm:batch` is one wire round-trip for N ops. Useful when latency dominates (many small ops back-to-back) or when you want to inspect the ordered outcome.

## See also

- [VM reference](/provium/reference/vm.md) — every method.
- [Process reference](/provium/reference/process.md) — async-process surface.
- [Worker reference](/provium/reference/worker.md) — concurrent agent connections.
- [Streams and tails](/provium/writing-tests/streams-and-tails.md) — `proc:stdout_stream` / `proc:stderr_stream`.

---

# Files and handles

_Provium / Writing tests_

> Guest-side file I/O — read_file and write_file vs open_file, mode tables, seek and tell, tail streams, stat, listdir, mkdir, unlink, and rename.

Provium gives you two layers for guest-side file I/O. Most tests use the high-level `vm:read_file` / `vm:write_file` / `vm:stat` calls; tests that need cursor control, partial reads, or POSIX semantics drop to `vm:open_file` and the [File](/provium/reference/file-handle.md) userdata.

## High-level: `read_file` / `write_file` / `stat`

These are one-shot operations. Each is a single round-trip to the agent.

```lua
vm:write_file("/etc/test.conf", "key=value\n")
local body = vm:read_file("/etc/test.conf")
local meta = vm:stat("/etc/test.conf")
```

### `vm:read_file(path)`

Returns the entire file as a Lua string. Errors on agent-side `read` failure (ENOENT, EACCES, etc.).

```lua
local content = vm:read_file("/etc/hostname")
```

### `vm:write_file(path, data)`

Replaces the file's contents. Creates if absent. The mode bits follow agent defaults — use `vm:open_file` if you need explicit `perm`.

```lua
vm:write_file("/tmp/data", "binary\0bytes")
```

### `vm:push_file(host_path, guest_path, opts?)`

Read a file on the host and write its bytes to `guest_path` in the guest. Equivalent to reading the host file in Lua and passing the bytes to `vm:write_file`, but with one important extra: inside a fixture, the host file is folded into the fixture's cache key automatically, so rebuilding the host artifact (e.g. a binary you're testing) invalidates the snapshot.

```lua
vm:push_file("../peios-uapi/target/release/peios-uapi", "/usr/bin/peios-uapi")
```

Relative `host_path` is resolved against the fixture (or helper) file's directory, not the cwd at invocation time. Pass `{auto_dep = false}` to skip the auto-fold for a single call (e.g. a large test corpus you don't want included in the key):

```lua
vm:push_file("../big-corpus.tar", "/data.tar", {auto_dep = false})
```

The auto-fold relies on static scanning of the call site, so non-literal host paths (variables, concatenation) are NOT tracked. Use [`lab:depends_on_file`](/provium/reference/lab.md#labdepends-on-filehost-path) with a literal string to declare them explicitly. See [Fixtures and dependencies — External host-file deps](/provium/running-tests/fixtures-and-dependencies.md#external-host-file-deps) for the full model.

Mode bits follow `write_file`'s agent defaults — `vm:run("chmod +x …")` after the push if you need executable bits.

### `vm:stat(path)`

Returns a table:

```lua
local m = vm:stat("/etc/test.conf")
print(m.size)        -- 10
print(m.perm)        -- POSIX mode bits
print(m.entry_type)  -- "file", "directory", "symlink", …
```

The full field table (including the complete `entry_type` value list) is in the [VM reference](/provium/reference/vm.md#vmstatpath). Two fields carry the modification time: use `mtime_ns` for exact comparisons; use `mtime` (float seconds) when "around what o'clock" is enough.

`perm` is in the POSIX range. Lua 5.4 doesn't accept `0o…` literals — use decimal or hex (`0x180` for 0o600). `4095` is `0o7777`, the upper bound.

### `vm:listdir(path)`

Returns an array of `{name, entry_type}` tables:

```lua
for _, e in ipairs(vm:listdir("/etc")) do
    print(e.name, e.entry_type)
end
```

### `vm:mkdir(path, opts?)`

Create a directory:

```lua
vm:mkdir("/tmp/d")
vm:mkdir("/tmp/d/e/f", {parents = true})
vm:mkdir("/tmp/private", {perm = 0x1c0})  -- 0o700
```

### `vm:unlink(path)`

Remove a file or empty directory. Errors on non-empty directory (use `vm:run("rm -rf …")` for that).

### `vm:rename(from, to)`

Atomic rename within the guest filesystem.

## Low-level: `vm:open_file`

Returns a [File](/provium/reference/file-handle.md) userdata that you can read, write, seek, and close.

```lua
local h = vm:open_file("/tmp/data", {read=true, write=true, create=true, truncate=true})
h:write("hello")
h:seek(0)
local s = h:read(5)
h:close()
```

### Mode table

At least one of `read`, `write`, `append` must be true; an empty mode table errors at open time. The flags you'll combine most often are `read` / `write`, `create` (create if absent), `truncate`, and `perm` (POSIX mode for newly-created files):

```lua
-- Write-only, create, truncate, mode 0o600.
local h = vm:open_file("/etc/secret", {write=true, create=true, truncate=true, perm=0x180})
```

The full mode-table reference (including `append` and `exclusive` / `O_EXCL`) is on [vm:open_file](/provium/reference/vm.md#vmopen-filepath-mode-table).

### Reading

```lua
local h = vm:open_file("/etc/hostname", {read=true})

local first = h:read(64)         -- up to 64 bytes
local rest  = h:read_all()       -- drain to EOF
h:close()
```

`h:read(n)` returns up to `n` bytes; at EOF it returns the empty string `""`, never an error — so a read loop terminates on `chunk == ""`. `h:read_all()` drains from the cursor to EOF in one call. See [EOF semantics](/provium/reference/file-handle.md#eof-semantics) in the File reference.

### Writing

```lua
local h = vm:open_file("/tmp/data", {write=true, create=true, truncate=true})
local n = h:write("hello world")
-- n is 11 (bytes actually written, may be less than #data on partial write)
h:close()
```

### Seek and tell

```lua
local h = vm:open_file("/tmp/x", {read=true})
h:seek(10, "set")             -- absolute offset 10
h:seek(5, "cur")              -- relative +5 from current
h:seek(-1, "end")             -- 1 byte before EOF
h:tell()                      -- current offset
```

`tell()` reports the authoritative agent-side position, not a host-side cache — the [File reference](/provium/reference/file-handle.md#filetell) explains how.

### Closing

```lua
h:close()
h:close()  -- second close is fine; idempotent
h:read(1)  -- raises "file is closed"
```

The harness's resource walker auto-closes files at scope end via the `_provium_close_test_scope` hook, so you can usually omit explicit closes. Closing manually is good practice when the file's lifetime is bounded by a clear point in the test.

### Raw fd

```lua
local fd = h:fd()              -- u64 handle id
vm:ioctl(fd, MY_IOCTL_NUM)
```

`fd()` returns `0` on a closed file, otherwise the handle's u64 value. Useful for `vm:ioctl(fd, …)` and `vm:syscall(…)` invocations that take a file descriptor.

## Tailing files

Two ways to tail:

### `vm:tail_file(path, opts?)`

Subscribe to bytes appended to a file. Returns a [Tail](/provium/reference/streams.md).

```lua
local stream = vm:tail_file("/var/log/messages")
vm:run("logger 'hello'"):assert_ok()
local line = stream:read_until("\n", "5s")
t:assert(line:find("hello"))
```

`opts.start` controls the starting position — `"end"` (the default: only bytes appended after the call), `"beginning"` (replay from byte 0, then continue tailing), or a byte offset. Negative-offset and float handling are in the [VM reference](/provium/reference/vm.md#vmtail-filepath-opts).

### `file:tail_stream()`

Open a tail rooted at the file handle's current cursor:

```lua
local h = vm:open_file("/var/log/messages", {read=true})
h:seek(0, "end")
local stream = h:tail_stream()
-- stream now subscribes from current EOF onwards
```

Useful when you've already seek-d to a known position.

### `vm:fd_stream(fd_or_file)`

Open a tail against an existing file handle (by fd integer or by the File userdata directly):

```lua
local h = vm:open_file("/dev/console", {read=true})
local stream = vm:fd_stream(h)
```

## Common patterns

### Writing then reading back

```lua
test("config round-trips", function(t)
    local body = "key1=value1\nkey2=value2\n"
    vm:write_file("/etc/test.conf", body)
    t:assert_eq(vm:read_file("/etc/test.conf"), body)
    t:assert_eq(vm:stat("/etc/test.conf").size, #body)
end)
```

### Asserting permission bits

Lua 5.4 has no `0o…` literal. Use decimal or hex:

```lua
test("private file is 0600", function(t)
    local h = vm:open_file("/etc/secret", {write=true, create=true, perm=0x180})  -- 0o600
    h:close()
    t:assert_eq(vm:stat("/etc/secret").perm, 0x180)
end)
```

### Tailing a log while the test acts

```lua
test("logger writes are persisted", function(t)
    local stream = vm:tail_file("/var/log/messages")
    vm:run("logger 'event 1'"):assert_ok()
    vm:run("logger 'event 2'"):assert_ok()
    stream:expect("event 1", "5s")
    stream:expect("event 2", "5s")
end)
```

### Listing then filtering

```lua
test("/tmp has the expected files", function(t)
    vm:write_file("/tmp/a", "")
    vm:write_file("/tmp/b", "")
    local names = {}
    for _, e in ipairs(vm:listdir("/tmp")) do
        if e.entry_type == "file" then table.insert(names, e.name) end
    end
    table.sort(names)
    t:assert(names[1] == "a" and names[2] == "b")
end)
```

## Files inside a worker

`worker:open_file(path, mode)` allocates the file under the worker's namespace. The returned File auto-registers with the test scope:

```lua
local w = vm:spawn_worker()
local h = w:open_file("/tmp/from-worker", {write=true, create=true})
h:write("hi")
h:close()
```

Otherwise the API is identical.

## Batch I/O for low latency

For many small ops back-to-back, batch them in one round trip:

```lua
local results = vm:batch(function(b)
    b:write_file("/tmp/a", "1")
    b:write_file("/tmp/b", "2")
    b:write_file("/tmp/c", "3")
    b:read_file("/tmp/a")
    b:stat("/tmp/b")
end)
```

Each entry in `results` is `{ok=value}` or `{err=msg}`. A failure on one op doesn't short-circuit the rest. See [VM batch](/provium/reference/vm.md#vmbatchfn) for the per-op return shape.

## See also

- [VM reference](/provium/reference/vm.md) — `read_file`, `write_file`, `stat`, `mkdir`, `listdir`, `unlink`, `rename`, `open_file`, `tail_file`, `fd_stream`.
- [File handle reference](/provium/reference/file-handle.md) — every method on the File userdata.
- [Streams and tails](/provium/writing-tests/streams-and-tails.md) — patterns for tail streams.

---

# Disks and fault injection

_Provium / Writing tests_

> How to attach disks, read and write sectors directly, and inject EIO and slow I/O faults to exercise guest-side error recovery.

Provium's disk support has two goals: give the test direct sector-level access to the backing image, and inject faults that exercise the guest's error-handling paths.

The exhaustive method reference is on [Disk](/provium/reference/disk.md).

## Attaching a disk

```lua
local img = "/tmp/test.img"
-- Pre-create a backing file; Provium does not auto-create.
io.open(img, "w"):write(string.rep("\0", 1024 * 1024)):close()

local vm = provium:vm("v", "peios"):boot()
local disk = vm:attach_disk({id = "vda", size = 1024 * 1024, image = img})
```

The two opts that matter for fault-injection work are `id` (names the disk so you can re-look it up via `vm:disk(id)`) and `image` (the backing file — required for `read_sectors` / `write_sectors`). The full opts table, types, and defaults are in the [Disk reference](/provium/reference/disk.md#constructing).

## Reading and writing sectors

Sectors are 512 bytes throughout. Offsets and counts are in sectors, not bytes.

```lua
-- Read sector 0 (the first 512 bytes).
local sec0 = disk:read_sectors(0, 1)

-- Read 4 sectors starting at sector 100 (bytes 51200..53247).
local block = disk:read_sectors(100, 4)
assert(#block == 4 * 512)

-- Write at sector 50.
disk:write_sectors(50, "hello, sector 50")
```

Without a backing image, both ops error with a `no backing image — disk:with_image required` message.

## Fault injection

Three modes:

| Mode | Effect |
|---|---|
| `eio_read` | Every `read_sectors` short-circuits to EIO. |
| `eio_write` | Every `write_sectors` short-circuits to EIO. |
| `slow` | Every `read_sectors` / `write_sectors` sleeps 50 ms before doing the I/O. |

Modes are activated with `disk:fault_inject(mode)` and cleared with `disk:clear_faults()`. Multiple modes can be active simultaneously — with `slow` + `eio_read` both set, the EIO check wins: the read errors immediately, without the 50 ms delay.

### Inject EIO

```lua
test("guest sees EIO on read", function(t)
    -- … attach disk with backing image …
    disk:fault_inject("eio_read")
    local ok, err = pcall(function() disk:read_sectors(0, 1) end)
    t:assert(not ok)
    t:assert(tostring(err):find("EIO"))
end)
```

### Inject slow I/O

`slow` delays each sector op but doesn't change its outcome. Assert both halves: the op took the hit, and it still worked. The guest [Clock](/provium/reference/clock.md) gives you a sub-millisecond time source (`os.time()` only has 1-second resolution):

```lua
test("write completes despite slowness", function(t)
    disk:fault_inject("slow")

    local clock = vm:clock()
    local before = clock:get_ns()
    disk:write_sectors(0, "data")            -- sleeps ~50 ms, then writes
    local elapsed_ms = (clock:get_ns() - before) / 1e6

    t:assert(elapsed_ms >= 50, "slow fault should add at least 50 ms")
    t:assert_eq(disk:read_sectors(0, 1):sub(1, 4), "data")  -- the write still landed
end)
```

The 50 ms delay is fixed per call; it is not currently configurable from test code.

### Combine modes

```lua
test("slow EIO is still EIO", function(t)
    disk:fault_inject("slow")
    disk:fault_inject("eio_read")
    local ok, err = pcall(function() disk:read_sectors(0, 1) end)
    t:assert(not ok)  -- slow doesn't change the outcome
    t:assert(tostring(err):find("EIO"))
end)
```

### Concurrent injection during I/O

The harness re-checks the fault set after the slow-fault sleep AND after the actual I/O completes, so an EIO fault that lands mid-call still takes effect. There is currently no way to exercise this from a test, though: test code runs on a single thread, and workers run guest processes — they can't call `disk:fault_inject`. Inject faults up-front, act, then clear; the mid-call recheck is defensive insurance in the harness, not a pattern you can drive.

### Inspecting state

```lua
local active = disk:active_faults()  -- {"eio_read", "slow"}
disk:is_detached()                   -- false
```

### Clearing

```lua
disk:clear_faults()
local r = disk:read_sectors(0, 1)    -- succeeds
```

## Detaching a disk

```lua
disk:detach()
local ok = pcall(function() disk:read_sectors(0, 1) end)
assert(not ok)  -- "disk is detached"
```

`disk:detach()` issues a best-effort QMP `device_del` against the parent VM and marks the local handle detached; after that, sector ops error. The exact behaviour for disks that were never QMP-added is in the [Disk reference](/provium/reference/disk.md#diskdetach).

## Common patterns

### "Does the guest retry after a transient EIO?"

```lua
test("guest retries on transient EIO", function(t)
    -- Inject EIO.
    disk:fault_inject("eio_read")

    -- Have the guest start a read in the background.
    local proc = vm:run_async("dd if=/dev/vda of=/tmp/out bs=512 count=1")

    -- Wait briefly, then clear so retry succeeds.
    vm:clock():sleep("100ms")
    disk:clear_faults()

    local r = proc:wait("5s")
    -- If the guest's driver retries, this succeeds. Otherwise dd
    -- returned an I/O error.
    r:assert_ok()
end)
```

### "Does the filesystem remount read-only after EIO?"

```lua
test("filesystem goes read-only after persistent EIO", function(t)
    disk:fault_inject("eio_write")
    vm:run("echo data > /mnt/test/file"):assert_ok()  -- might succeed or fail
    -- Force a sync to surface the write.
    vm:run("sync")
    -- Now check the kernel's view: errors=remount-ro should kick in.
    local r = vm:run("findmnt /mnt/test -o OPTIONS")
    t:assert(r.stdout:find("ro"))
end)
```

### "Does the guest panic on EIO at boot?"

```lua
test("EIO at boot does not panic", function(t)
    disk:fault_inject("eio_read")
    vm:reset()
    -- vm:reset() returns to Booted; check the console for panic strings.
    local log = vm:console():read_log()
    t:assert(not log:find("kernel panic"))
end)
```

## Multiple disks per VM

```lua
local data = vm:attach_disk({id = "vdb", size = 1024 * 1024, image = "/tmp/data.img"})
local logs = vm:attach_disk({id = "vdc", size = 1024 * 1024, image = "/tmp/logs.img"})

-- Inject EIO on data only; logs is unaffected.
data:fault_inject("eio_read")
```

Use `vm:disk(id)` to look up an already-attached disk:

```lua
local data = vm:disk("vdb")
data:fault_inject("slow")
```

## Caveats

- **Sector size is fixed at 512 bytes.** Tests that need 4 KiB sectors should expect their guest to layer that on top.
- **`read_sectors` and `write_sectors` go directly to the host file**, not through QEMU's block backend. This means a test that exercises QEMU's block translation (sparse holes, compression, etc.) will not see those layers — the disk userdata is a direct view of the underlying image bytes.
- **`disk:size()` reports the live image file size when an image is attached.** Test code that resizes the underlying file (`truncate`, `fallocate`) sees the new size, not the modelled `size` from `attach_disk`.

## See also

- [Disk reference](/provium/reference/disk.md) — every method, every error message.
- [VM reference](/provium/reference/vm.md) — `vm:attach_disk`, `vm:disk`.

---

# Bridges and impairments

_Provium / Writing tests_

> Wire VMs into bridges, partition them, inject latency, drop rate, and bandwidth limits, isolate VMs, capture pcap, and reach the outside world via uplink.

Provium's networking is real Linux bridges with TAP attachments. You declare a topology, the harness realises it as the VMs boot, and you can mutate it (partition, impair, capture) at runtime.

The exhaustive method reference is on [Bridge](/provium/reference/bridge.md) and [Nic](/provium/reference/nic.md).

> [!NOTE]
> Provium wires layer 2 only. A bridge gives each attached VM a NIC on a shared segment; IP addressing, routing, and name resolution inside the guests are the guest image's business. The examples on this page assign addresses with ordinary `ip addr` commands run in the guests.

## Declared vs realised

Provium tracks resources in two layers: the resource graph records what a test has declared (bridges, attachments, partitions, impairments), and the harness separately realises that graph on the host — bridge interfaces, TAPs, `tc` qdiscs, nft rules, QMP calls — once the backing pieces exist, typically when an attached VM boots. A call marked **graph-state only** updates the declared graph without touching the host: it's recorded and visible to inspection methods, but on its own it doesn't change anything real.

## Wiring up

```lua
local lan = provium:bridge("lan")
local a   = provium:vm("a", "peios"):boot()
local b   = provium:vm("b", "peios"):boot()
lan:attach({a, b})  -- atomic; either all attach or none do

a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")

-- a can reach b.
a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
```

`bridge:attach` takes a single VM or an array (validated atomically — a bad element fails before anything is recorded). The bare-string form and its graph-state-only caveat are in the [Bridge reference](/provium/reference/bridge.md#membership).

The host-side bridge interface and per-VM TAPs come up the first time any attached VM boots.

## Multi-bridge topologies

```lua
local mgmt = provium:bridge("mgmt")
local data = provium:bridge("data")
local a    = provium:vm("a", "peios"):boot()
local b    = provium:vm("b", "peios"):boot()

mgmt:attach({a, b})    -- both VMs on mgmt
data:attach({a, b})    -- both VMs on data too

-- a:nic("mgmt") and a:nic("data") return separate Nic handles.
local mgmt_nic = a:nic("mgmt")
local data_nic = a:nic("data")
```

Inside the guest, each NIC shows up as a separate interface. The mapping from bridge name to guest-side interface name (eth0, eth1, …) is determined by attachment order sorted by bridge name. For portable tests, prefer `vm:nic("mgmt")` (by bridge name) over `vm:nic("eth0")` (by guest-name index).

## Partitions

A partition is a network-layer drop between two specific VMs. Two flavours:

### Symmetric

```lua
lan:partition(a, b)        -- A↔B traffic dropped both ways
lan:unpartition(a, b)      -- restore
```

Symmetric partitions are graph-state — they install drop rules at boot via nft and lift cleanly.

### Directional

```lua
lan:partition({from = a, to = b})    -- only A→B dropped; B→A still flows
lan:unpartition({from = a, to = b})
```

Directional partitions install per-TAP nft rules and require **both endpoints to be already attached and booted** — otherwise they error with a "call bridge:attach first" pointer. The (more lenient) `unpartition` semantics are in the [Bridge reference](/provium/reference/bridge.md#partitions).

### Whole-bridge

```lua
lan:partition_all()         -- every pair partitioned
lan:restore_all()           -- every partition lifted
```

### Inspect

```lua
if lan:is_partitioned(a, b) then
    -- A↔B is currently partitioned (symmetric or A→B directional)
end
```

## Impairments

Three knobs: latency, drop rate, bandwidth limit. Each accepts either a scalar (whole-bridge) or a directional table.

### Latency

```lua
lan:add_latency(50)                                -- 50 ms one-way to every flow
lan:add_latency({from = a, to = b, ms = 50})       -- 50 ms recorded for A→B
```

Implemented as netem qdiscs. A directional impairment is recorded per `(from, to)` pair but realised per **source**: each VM's outbound pairs collapse into one qdisc chain on its own TAP, taking the worst case across its pairs — so in practice the `{from = a, to = b}` latency above delays everything leaving `a`, not just its traffic to `b`. The endpoint attachment check applies (same as directional partitions).

### Drop rate

```lua
lan:drop_rate(10)                                  -- ~10 % loss, both ways
lan:drop_rate({from = a, to = b, p = 10})          -- recorded for A→B
```

### Bandwidth limit

```lua
lan:bandwidth_limit(1024 * 1024)                       -- 1 Mbit/s, both ways
lan:bandwidth_limit({from = a, to = b, bps = 500000})  -- 500 kbit/s leaving A
```

The number is **bits per second**, not bytes — matches `tc rate Nbit`. Two realisation caveats matter when you design a test: the directional form shapes every packet leaving the source TAP (not just traffic to the named `to`), and whole-bridge bandwidth is not enforced while whole-bridge latency/drop is also set — for combined shaping use the directional form on each source. The full realisation details (TBF vs HTB, `max(bps)` collapse across pairs) are in the [Bridge reference](/provium/reference/bridge.md#bridgebandwidth-limitbps-or-table).

Combine directional bandwidth with directional latency / drop on the same source for a complete profile:

```lua
lan:bandwidth_limit({from = a, to = b, bps = 1_000_000})
lan:add_latency({from = a, to = b, ms = 25})
lan:drop_rate({from = a, to = b, p = 1})
```

### Reset

```lua
lan:reset()    -- tear down every netem/tbf qdisc, clear every partition
```

`reset` is a clean way to go back to "default" without enumerating every impairment you applied. Isolation, uplink, and L3 routes survive a `reset` — they're topology, not impairments.

### Inspect

```lua
lan:latency_ms()        -- current whole-bridge latency
lan:drop_rate_pct()     -- current whole-bridge drop rate
lan:bandwidth_bps()     -- current whole-bridge bandwidth cap
```

These return the most recently applied whole-bridge value. They don't enumerate per-direction impairments.

## Isolation

Isolation puts one VM behind a hairpin filter — it can't reach any other VM on the bridge, but the bridge stays up:

```lua
lan:isolate(a)
local r = a:run("ping -c 1 -W 1 10.0.0.2")
-- r:ok() is false; a is isolated

lan:unisolate(a)
a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
```

`lan:is_isolated(vm)` returns `true` if the VM is currently isolated.

## NICs

```lua
local nic = a:nic("lan")              -- by bridge name
local nic = a:nic("eth0")             -- by guest-name index
local nic = lan:nic(a)                -- equivalent
```

The Nic gives you per-NIC capabilities the bridge can't. The ones you'll use most:

```lua
nic:counters()         -- per-NIC traffic counters, guest's perspective
nic:disconnect()       -- link-down via QMP set_link(false)
nic:reconnect()        -- link-up
```

`disconnect` / `reconnect` drive QMP `set_link` so the guest sees a real link-down event. The full method list, the counter field table (and its guest-perspective mapping), and the graph-state-only bare-string case are in the [Nic reference](/provium/reference/nic.md).

## Packet capture

Two scopes:

### Bridge-wide capture

```lua
local cap = lan:capture()
a:run("ping -c 5 10.0.0.2")
local frames = cap:drain("2s")
local pcap = table.concat(frames)
-- pcap is now standard pcap-format bytes, parseable by tshark, etc.
```

`bridge:capture()` returns a [Capture](/provium/reference/streams.md) stream of pcap-format bytes. It requires `tcpdump` on `PATH`, and a live capture blocks `vm:snapshot()` (no half-captured pcap) — capability requirements and the mechanism are in the [Bridge reference](/provium/reference/bridge.md#bridgecapture).

### Per-NIC capture

```lua
local nic = a:nic("lan")
local cap = nic:capture()    -- captures only A's TAP, not the whole bridge
```

Useful when multiple VMs are on the bridge and you only want one VM's perspective.

`nic:capture()` errors if the VM hasn't been booted yet (the per-VM TAP doesn't exist):

```
nic:capture: vm `a` has no TAP on bridge `lan` (not booted?). Call lab:boot() / vm:boot() first.
```

## Uplink (NAT to the outside world)

```lua
lan:enable_uplink()
vm:run("curl -s https://example.com/"):assert_ok()
lan:disable_uplink()
```

`enable_uplink` installs an nft NAT masquerade rule between the bridge and the host's default-route interface. Failures (e.g. no default-route interface) error with the underlying detail.

## L3 routing (preview)

```lua
lan:route(other_bridge)
```

`bridge:route` records the routing intent in the graph but installs no nft forward rules in v1 — cross-bridge IP traffic does not actually flow yet, and the first call per bridge prints a one-shot warning saying so. `bridge:routes()` returns the recorded routes. Plan tests around the limitation. See the [Bridge reference](/provium/reference/bridge.md#l3-routing-preview).

## Common patterns

### Test split-brain recovery

```lua
test("application recovers after partition heals", function(t)
    local lan = provium:bridge("lan")
    local a, b = provium:vm("a", "peios"):boot(), provium:vm("b", "peios"):boot()
    lan:attach({a, b})
    a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
    b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")

    -- Baseline: a can talk to b.
    a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()

    -- Partition.
    lan:partition(a, b)
    local r = a:run("ping -c 1 -W 1 10.0.0.2")
    t:assert(not r:ok())

    -- Heal.
    lan:unpartition(a, b)
    -- Wait for ARP / route to re-converge.
    wait_until(function()
        return a:run("ping -c 1 -W 1 10.0.0.2"):ok()
    end, {timeout = "10s", desc = "post-heal connectivity"})
end)
```

### Test latency-sensitive code

```lua
test("client retries on slow link", function(t)
    local lan = provium:bridge("lan")
    -- … attach VMs …
    lan:add_latency(500)  -- 500 ms each way ≈ 1s RTT
    local r = client:run("curl --max-time 0.5 http://10.0.0.2/")
    t:assert(not r:ok())  -- timeout fires

    lan:reset()
    r = client:run("curl --max-time 0.5 http://10.0.0.2/")
    r:assert_ok()
end)
```

### Test packet loss tolerance

```lua
test("client succeeds with 30% drop", function(t)
    lan:drop_rate(30)
    -- Apps should retry; some will succeed.
    local successes = 0
    for _ = 1, 20 do
        if client:run("curl --max-time 5 http://10.0.0.2/"):ok() then
            successes = successes + 1
        end
    end
    t:assert(successes >= 5)  -- pessimistic floor
end)
```

### Inspect packet flow with capture

```lua
test("DNS query produces UDP traffic", function(t)
    lan:enable_uplink()
    local cap = lan:capture()
    a:run("dig @8.8.8.8 example.com")
    local pcap = table.concat(cap:drain("3s"))
    cap:close()
    -- Pipe pcap through tshark or pyshark to assert UDP/53.
end)
```

## See also

- [Bridge reference](/provium/reference/bridge.md) — every method on the Bridge userdata.
- [Nic reference](/provium/reference/nic.md) — per-NIC handle.
- [Streams reference](/provium/reference/streams.md) — what `bridge:capture()` and `nic:capture()` return.

---

# Streams and tails

_Provium / Writing tests_

> The three stream types — when to use next, read_until, expect, or drain, how EOF works, and how snapshots interact with live streams.

Provium has three stream userdata types — Tail, Capture, and ConsoleStream — that share a common surface: `next` / `read_until` / `expect` / `drain` / `close` / `eof` / `creation_site`. This page is the practical guide.

The exhaustive method reference is on [Streams](/provium/reference/streams.md).

## What returns what

| Source | Type | Use for |
|---|---|---|
| `vm:tail_file(path, opts?)` | Tail | Following a file as it grows. |
| `vm:fd_stream(fd_or_file)` | Tail | Following an open file handle. |
| `file:tail_stream()` | Tail | Following a file from its current cursor. |
| `proc:stdout_stream()` / `proc:stderr_stream()` | Tail | Following an async process's output. |
| `bridge:capture()` | Capture | Sniffing every packet on a bridge. |
| `nic:capture()` | Capture | Sniffing one VM's TAP. |
| `console:read()` | ConsoleStream | Reading the guest's serial console. |

All three types support the same operations. The differences are in the underlying transport and the per-frame shape.

## Operations

Four reading operations cover almost every test. Their exact semantics — frame shapes per type, default timeouts, error strings, and the pending-bytes buffer — are in the [Streams reference](/provium/reference/streams.md#common-methods); this section shows how each is used.

One shared property matters for correctness: bytes past an `expect`/`read_until` match are kept and replayed on the next call, so a sequence of reads never silently loses data.

### `:next(timeout?)` — pull the next chunk

Returns the next chunk of bytes as a Lua string, or `nil` at EOF / timeout.

```lua
local stream = vm:tail_file("/var/log/messages")
vm:run("logger 'event'"):assert_ok()
local frame = stream:next("5s")
print(frame)  -- "Jan  1 00:00:00 v: event\n"
```

### `:read_until(pattern, timeout?)` — read until a substring

```lua
local line = stream:read_until("\n", "5s")
```

Pulls frames until `pattern` (a Lua string of bytes) appears. Returns the prefix up to AND including the matched bytes. Errors with the pattern in the message on timeout.

### `:expect(pattern, timeout?)` — assert and discard

```lua
stream:expect("ready", "30s")
-- next() / read_until() will see anything past "ready"
```

Like `read_until`, but discards the matched prefix. Returns nothing. Use this when you want the assertion semantics — "this stream produced X" — without caring about the bytes themselves.

### `:drain(timeout?)` — collect everything available

```lua
local chunks = stream:drain("2s")
-- chunks is a Lua array of strings
local body = table.concat(chunks)
```

Read frames until the stream goes quiet, hits EOF, or the timeout lapses. Useful for "what did the stream produce in this window?" Note its default timeout is deliberately short (0.5 s vs 10 s for the other ops).

### Housekeeping: `:close()`, `:eof()`, `:creation_site()`

`stream:close()` drops the underlying transport (idempotent; a closed stream reads as EOF). `stream:eof()` tells you the stream is finished and fully consumed. `stream:creation_site()` reports where the stream was opened — you'll mostly meet it in snapshot-refusal errors. Exact semantics for all three are in the [Streams reference](/provium/reference/streams.md#common-methods).

## Choosing between next, read_until, expect, drain

| Scenario | Use |
|---|---|
| "Tell me when X happens." | `:expect("X", timeout)` |
| "What's the next line?" | `:read_until("\n", timeout)` |
| "Pull bytes until I have enough." | Loop on `:next(timeout)` |
| "Snapshot everything in this window." | `:drain(timeout)` |
| "Has the stream finished?" | `:eof()` after `:next()` returns nil. |

`:expect` is the workhorse for log-watching tests — it has tight, unambiguous error messages on timeout, and it doesn't dump bytes you don't want into your handler.

`:read_until` is the workhorse when you need the matched bytes (parsing structured log lines, checking that the prefix matches an expected shape).

`:next` is most useful in loops where you want to inspect each frame before deciding what to do.

`:drain` is most useful at the end of a test to confirm "nothing weird snuck through" or to capture a quiet window for offline analysis.

## EOF semantics

A closed stream is semantically EOF — `:next` returns `nil`, `:read_until` and `:expect` error. Idiomatic loops:

```lua
while true do
    local frame = stream:next("100ms")
    if not frame then break end       -- EOF or timeout
    process(frame)
end
```

For Tail specifically, the closed-stream-returns-nil behaviour mirrors Capture and ConsoleStream so the `while s:next() do … end` idiom works across all three types.

## Tail-specific: starting position

```lua
vm:tail_file("/var/log/messages")                    -- start at end (default)
vm:tail_file("/var/log/messages", {start = "beginning"})  -- replay from byte 0
vm:tail_file("/var/log/messages", {start = -512})    -- last 512 bytes then follow
```

`"end"` is the most common — only bytes appended after the call are streamed. `"beginning"` is useful for tests that need to assert on the whole file. Negative integers mean "N bytes before EOF". The full `start` value table (exact offsets, clamping, float handling) is in the [Streams reference](/provium/reference/streams.md#tail-only-behaviour).

## Capture: pcap bytes

`bridge:capture()` and `nic:capture()` produce raw pcap bytes (the standard pcap-savefile format, not pcap-ng). Concatenate the chunks and pipe into `tshark`, `tcpdump -r -`, or a parser library:

```lua
local cap = lan:capture()
a:run("ping -c 5 10.0.0.2")
local pcap = table.concat(cap:drain("3s"))
cap:close()

-- Analyse inside a guest without touching the host disk.
local r = a:run("tcpdump -r /dev/stdin -n", {stdin = pcap})
print(r.stdout)
```

A live capture blocks `vm:snapshot()` on purpose — no half-captured pcaps. Close the capture before snapshotting. (The mechanism is described in the [Streams reference](/provium/reference/streams.md#capture-only-notes).)

## ConsoleStream: bytes from the chardev

```lua
local console = vm:console()
local stream  = console:read()

stream:expect("login:", "30s")
console:write("root\n")
stream:expect("Password:", "5s")
console:write("toor\n")
stream:expect("# ", "5s")
```

ConsoleStream reads raw bytes from QEMU's console chardev — typically the boot log, login prompt, and anything the guest has written to `/dev/ttyS0` since the last read. A VM reset or shutdown reads as console EOF, not an error. Transport details (socket, timeouts, error mapping) are in the [Streams reference](/provium/reference/streams.md#consolestream-only-notes).

## Process streams

`proc:stdout_stream()` and `proc:stderr_stream()` open Tail streams subscribed to captured output:

```lua
local proc = vm:run_async("server")
local out  = proc:stdout_stream()
out:expect("listening on :8080", "10s")
-- now hit the server
```

The stream's `creation_site` and `kind`/`detail` carry the process handle id, so snapshot diagnostics can name "proc_stdout_stream(handle=42)" when something refuses a snapshot.

## File streams

`file:tail_stream()` opens a Tail rooted at the file's current cursor:

```lua
local h = vm:open_file("/var/log/messages", {read=true})
h:seek(0, "end")  -- start at current EOF
local stream = h:tail_stream()
-- stream subscribes from EOF onwards, just like vm:tail_file with start="end"
```

`vm:fd_stream(fd_or_file)` is the lower-level form — accepts either an integer fd (from `file:fd()`) or the File userdata directly:

```lua
local h = vm:open_file("/dev/console", {read=true})
local stream = vm:fd_stream(h)
```

## Snapshots and live streams

Snapshots refuse to run while a stream is live. The error names the stream:

```
provium: vm:snapshot() refused — file `tests/x.test.lua` has live streams:
  - tail_file("/var/log/messages") at tests/x.test.lua:42
  - proc_stdout_stream(handle=7) at tests/x.test.lua:55
Close the streams before snapshotting (or remove the snapshot).
```

Two ways out:

1. Close the streams explicitly before `vm:snapshot()`.
2. Move the snapshot earlier in the test, before the streams are opened.

The pre-condition is enforced for both single-VM (`vm:snapshot()`) and lab (`provium:snapshot()`) snapshots. It also blocks `provium.reset_between_tests = true` files at chunk-load time when file-scope streams are open.

## Common patterns

### "Wait for a log line"

```lua
local stream = vm:tail_file("/var/log/messages")
vm:run("trigger-something"):assert_ok()
stream:expect("trigger landed", "5s")
```

### "Race a process boot against its readiness signal"

```lua
local proc = vm:run_async("server")
local out  = proc:stdout_stream()
out:expect("ready", "10s")
-- safe to hit the server now
```

### "Capture pcap during a specific operation"

```lua
local cap = lan:capture()
a:run("ping -c 3 10.0.0.2"):assert_ok()
local pcap = table.concat(cap:drain("2s"))
cap:close()
```

### "Drive an interactive prompt over the console"

```lua
local console = vm:console()
local stream  = console:read()
stream:expect("login:", "30s")
console:write("root\n")
stream:expect("# ", "10s")
```

### "Confirm nothing weird snuck through"

```lua
local cap = lan:capture()
vm:run("…benign workload…"):assert_ok()
local frames = cap:drain("1s")
cap:close()
local pcap = table.concat(frames)
local r = vm:run("tcpdump -r /dev/stdin -n", {stdin = pcap})
t:assert(not r.stdout:find("malformed"))
```

## See also

- [Streams reference](/provium/reference/streams.md) — every method, EOF semantics, type-specific notes.
- [Console reference](/provium/reference/console.md) — `console:read` and `console:expect`.
- [VM reference](/provium/reference/vm.md) — `vm:tail_file`, `vm:fd_stream`.
- [File handle reference](/provium/reference/file-handle.md) — `file:tail_stream`.

---

# Labs and scope

_Provium / Writing tests_

> Sub-labs for multi-VM topologies, resource claims against the pool, barriers, whole-lab snapshot and restore, and cached lab fixtures.

A Lab is the unit of resource ownership in Provium. The `provium` global is the root Lab; sub-labs let you carve out scoped subsets. This page covers the patterns for using labs effectively.

The exhaustive method reference is on [Lab](/provium/reference/lab.md).

## Per-test scope

Each `test()` body runs in its own ephemeral sub-Lab. Resources you create inside a test are local to that test; resources declared at file scope are visible (via lookup fallthrough) and persist across tests.

```lua
-- File scope: persists across all tests
local shared = provium:vm("shared", "peios"):boot()

test("first", function(t)
    local local_vm = provium:vm("local", "peios"):boot()  -- test scope
    -- both `shared` and `local_vm` work here
end)
-- `local_vm` is shutdown silently here

test("second", function(t)
    local local_vm = provium:vm("local", "peios"):boot()  -- DIFFERENT VM
    -- The "local" name in test 1 and test 2 are independent — no collision.
    -- `shared` is still here, with whatever state test 1 left.
end)
```

The rules:

| Operation | What happens in a test() body |
|---|---|
| `provium:vm(name, profile)` | Creates in the test scope. Auto-shutdown at test end. |
| `provium:vm(name)` | Looks up name: test scope first, then file root. Errors if not found. |
| `provium.foo` (dot access) | Same lookup as above; returns `nil` on miss. |
| `provium:bridge(name, opts?)` | Same as VMs: create local, lookup walks up. |
| `provium:lab(name)` | Creates a sub-lab in the test scope (auto-cleaned). |
| `provium:vm_fixture(name)` / `provium:lab_fixture(name)` | Always materialises at file root, regardless of where called. The fixture cache stays warm across tests. |

**Shadow detection.** Declaring a name at test scope when it already exists at file scope is an error, not a silent shadow:

```lua
local shared = provium:vm("shared", "peios"):boot()

test("oops", function(t)
    -- This errors: "name already declared at parent scope (lab `provium`).
    -- Pick a different name, or use `provium:vm("shared")` to reuse it."
    local v = provium:vm("shared", "peios")
end)
```

The intent: if you wanted the file-scope VM, use the lookup form (`provium:vm("shared")`). If you wanted a fresh independent VM, pick a different name.

**Federation sub-labs are isolated.** A sub-lab created via `provium:lab("dc1")` has no parent chain — `dc1.web` doesn't fall through to find a sibling DC's "web". This is intentional; federation models distinct sites.

**Things that don't walk parents** (each by design):

- `vm_names` / `bridge_names` / `members` — return only the local scope's contents.
- `boot` / `shutdown` / `pause` / `resume` — batch ops on the local scope's VMs.
- `snapshot` / `restore` — operate on the local scope plus its sub-labs (downward, not upward).
- `claim`, `barrier` — file-scope coordination primitives; test-scope claims/barriers are isolated.

## Per-test scope vs `reset_between_tests`

These solve different problems and compose well together — they're not alternatives.

| | Per-test scope (default, automatic) | `reset_between_tests = true` (opt-in) |
|---|---|---|
| **What it isolates** | New declarations made inside a `test()` body. | Mutable state of file-scope resources. |
| **Mechanism** | Test-scope sub-Lab; auto-shutdown at test end. | Snapshot after file setup; restore between tests. |
| **Per-test cost** | Booting the test-scope VMs you declared. | Snapshot restore (cheaper than full boot, slower than nothing). |
| **What it doesn't help with** | File-scope VMs accumulating cruft across tests. | Name collisions across tests for ad-hoc VMs. |

The typical heavy test file uses both:

```lua
provium.reset_between_tests = true

-- File-scope setup: expensive cluster build, snapshot baseline.
local lan = provium:bridge("lan")
local web = provium:vm("web", "peios"):boot()
local db  = provium:vm("db",  "peios"):boot()
lan:attach({web, db})
web:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
db:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")
db:run("psql -c 'CREATE TABLE t (...)'")  -- baseline schema

test("api writes propagate to db", function(t)
    -- web and db are restored to baseline at start of every test.
    web:run("curl -X POST http://localhost/items -d '...'")
    local r = db:run("psql -tA -c 'SELECT count(*) FROM t'")
    t:assert_eq(r.stdout:match("%d+"), "1")
end)

test("scratch VM joins the network", function(t)
    -- Throwaway VM, test-scope, auto-shutdown at test end.
    local probe = provium:vm("probe", "peios"):boot()
    lan:attach(probe)  -- bridge is file-scope, found via parent walk
    probe:run("ip addr add 10.0.0.3/24 dev eth0 && ip link set eth0 up")
    probe:run("curl -s http://10.0.0.1/health"):assert_ok()
end)
```

Without per-test scope, the second test couldn't introduce `probe` without colliding (or having to pick a unique name per test). Without `reset_between_tests`, the row the first test inserted would still be in the database for every later test.

When to use which:

- **Per-test scope alone** is enough when each test is self-contained: declare what you need, do work, done. The smoke-test shape — `local vm = provium:vm("v", "peios"):boot(); vm:run(...)` repeated per test.
- **Add `reset_between_tests`** when you have a non-trivial setup at file scope (configured cluster, populated DB, attached topology) and tests mutate it. Pays for itself once a per-test setup-cost crosses the snapshot/restore time.
- **Skip both** by declaring `reset_between_tests = false` AND being careful with naming when you actually want state to accumulate across tests (e.g. progressive integration scenarios).

## Why sub-labs

Most simple tests use only the root lab — `provium:vm("a", "peios")`, `provium:bridge("lan")`. Sub-labs are useful when:

- You're modelling multi-DC topologies (`provium:lab("dc1")`, `provium:lab("dc2")`).
- You want to snapshot or restore a coherent subset of resources.
- You want to use `lab_fixture` to cache an entire topology.

```lua
local dc1 = provium:lab("dc1")
dc1:vm("a", "peios")
dc1:vm("b", "peios")
dc1:bridge("lan"):attach({dc1.a, dc1.b})

local dc2 = provium:lab("dc2")
dc2:vm("c", "peios")
dc2:bridge("lan"):attach(dc2.c)
```

`dc1.a` is shorthand for `dc1:vm("a")`. The dot lookup tries vm → bridge → sub-lab in order; missing names return `nil` (no error).

## Anonymous sub-labs

```lua
local sub = provium:lab()  -- name auto-generated: __lab_0, __lab_1, ...
```

Useful for one-shot subsets that don't need a stable name.

## Membership: include and remove

You can move resources between labs:

```lua
local v = provium:vm("v", "peios")
local sub = provium:lab("sub")
sub:include(v)            -- v is now in sub
provium:remove(v)         -- and gone from root
```

`lab:include` accepts a single VM, Bridge, or sub-Lab userdata, or an array of those. It errors on duplicate names, reserved names, and shadow conflicts — the exact error cases are in the [Lab reference](/provium/reference/lab.md#labincluderesource-or-list).

`lab:remove` is [graph-state only](/provium/writing-tests/bridges-and-impairments.md#declared-vs-realised) — the underlying VM, bridge, or sub-lab is NOT shut down. It's removed from the lab's child list. Useful when you want to take ownership of a resource somewhere else.

## Listing members

```lua
provium:vm_names()          -- {"v", "v2"}
provium:bridge_names()      -- {"lan"}
provium:sub_lab_names()     -- {"dc1", "dc2"}
provium:members()           -- [{kind="vm", name="v"}, {kind="bridge", name="lan"}, …]
```

`members()` is the unified accessor; the others are sliced by kind.

## Batch lifecycle

The lifecycle methods on a lab apply to every direct VM child (not recursive):

```lua
provium:boot()       -- boot every VM in the root lab
provium:shutdown()   -- shutdown every VM
provium:pause()
provium:resume()
```

Useful when a test sets up the topology declaratively and wants to bring it up atomically:

```lua
local lan = provium:bridge("lan")
local a   = provium:vm("a", "peios")
local b   = provium:vm("b", "peios")
lan:attach({a, b})
provium:boot()                -- boots a and b together
```

For per-sub-lab control:

```lua
local dc1 = provium:lab("dc1")
dc1:vm("a", "peios"):boot()
dc1:vm("b", "peios"):boot()
-- dc1 boots independently of the root lab
```

## Resource claims

Each test file may make at most one claim against the dispatcher's resource pool:

```lua
provium:claim({memory = "4G", cpus = 4})

test("…", function(t) end)
test("…", function(t) end)
```

The claim sits across the file's lifetime; it's released at file end. A second `:claim` errors — `lab claim already held; one-shot per lab`. The accepted field shapes and the no-pool behaviour are in the [Lab reference](/provium/reference/lab.md#resource-claims).

Why claim? The dispatcher won't oversubscribe — it tracks total RAM and CPU budget across files and only schedules a file when its claim plus the per-file overhead fits. A file that needs 4 VMs at 2 GiB each should claim ~10 GiB so it doesn't get scheduled alongside other heavy files and OOM the host.

```lua
-- For a 3-VM, 2-CPU-each, 1-GiB-each test:
provium:claim({memory = "5G", cpus = 7})  -- 3 GiB + per-file overhead, 6 + 1 vCPU
```

## Barriers

`lab:barrier(name, count, timeout?)` is an N-arrival rendezvous: it blocks the caller until `count` callers have hit the same `name`-keyed barrier, then returns `true` for all of them. On timeout it returns `false` — it does not raise. The default timeout, count lock-in, and round-reuse semantics are in the [Lab reference](/provium/reference/lab.md#labbarriername-count-timeout).

```lua
-- count = 1 is satisfied immediately: a labelled checkpoint.
provium:barrier("setup-done", 1)

-- An unmet count times out and returns false rather than raising.
local ok = provium:barrier("both-ready", 2, 0.5)
if not ok then
    error("second arriver never showed up")
end
```

A count above 1 needs a second concurrent host-side caller. Test files execute on a single thread, and workers run guest processes (they can't call back into the test's Lua), so today a `count > 1` barrier in an ordinary test file simply times out. The multi-caller form is aimed at thread-mode workers (`vm:spawn_worker({thread = true})`), which aren't supported yet.

For inter-VM coordination (e.g. between two guests), use a guest-side primitive (file in a shared mount, fifo, network message). Barriers are host-side only.

## Snapshot and restore

```lua
local snap = provium:snapshot()           -- to a tempdir, returns LabSnapshot
local snap = provium:snapshot("/tmp/x")   -- to that path, returns the path string
```

The `LabSnapshot` userdata's accessors are in the [Snapshot reference](/provium/reference/snapshot.md). Restore with:

```lua
provium:restore(snap)             -- from LabSnapshot userdata
provium:restore("/tmp/x")         -- from path
```

The same precondition that blocks `vm:snapshot()` blocks `lab:snapshot()` — open streams cause the snapshot to refuse with the offending stream's creation site.

## Lab fixtures

A lab fixture builds a multi-VM topology once, caches the snapshot, and restores it per test:

```lua
-- tests/fixtures/cluster.fixture.lua
local lan = provium:bridge("lan")
local a = provium:vm("a", "peios"):boot()
local b = provium:vm("b", "peios"):boot()
lan:attach({a, b})
a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")
return provium:snapshot()
```

```lua
-- tests/uses-cluster.test.lua
test("cluster has both VMs reachable", function(t)
    local cluster = provium:lab_fixture("fixtures/cluster")
    cluster.a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
end)
```

`provium:lab_fixture(path)` returns a Lab userdata wrapping a fresh sub-lab with the fixture restored. For single-VM fixtures, use `provium:vm_fixture(path)` — same cache mechanism, returns a VM directly. What goes into the cache key (and therefore what triggers a rebuild) is covered in [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md).

## Common patterns

### Multi-DC topology

```lua
local function dc(name, vms)
    local d = provium:lab(name)
    for _, n in ipairs(vms) do d:vm(n, "peios") end
    d:bridge("lan"):attach(d:vm_names())
    return d
end

local dc1 = dc("dc1", {"a", "b"})
local dc2 = dc("dc2", {"c", "d"})
provium:boot()  -- boot every VM
```

### File-level resource claim

```lua
provium:claim({memory = "8G", cpus = 6})

test("3-VM cluster", function(t)
    -- The dispatcher won't run another file alongside this one
    -- if the pool can't afford 8G + 6 CPUs.
end)
```

### Cached cluster fixture

```lua
-- fixture builder
return provium:snapshot()  -- after building the topology

-- test usage
local cluster = provium:lab_fixture("fixtures/big-cluster")
local a = cluster.a  -- look up by name from the restored sub-lab
```

### Reset-between-tests with a sub-lab snapshot

`provium.reset_between_tests = true` snapshots the root lab. To reset only a subset:

```lua
local sub = provium:lab("workspace")
local v = sub:vm("v", "peios"):boot()
v:write_file("/tmp/data", "")
local snap = sub:snapshot()

test("a", function(t)
    v:write_file("/tmp/data", "1")
end)

test("b", function(t)
    sub:restore(snap)  -- explicit restore
    t:assert_eq(v:read_file("/tmp/data"), "")  -- back to baseline
end)
```

This pattern is more verbose than `provium.reset_between_tests = true` but works when only part of the lab needs resetting.

## Auto-close ordering

When a test scope ends (per-test or per-file), the harness's resource graph walker closes resources in reverse-dependency order:

1. Streams (Tails, Captures, ConsoleStreams).
2. Processes (`vm:run_async`, `worker:run_async`).
3. Files (`vm:open_file`, `worker:open_file`).
4. Workers.
5. Bridges.
6. VMs.

This is implemented as a Lua-side registry that every `register_resource` call appends to. Resources are closed in priority order, then within a priority by registration order.

You don't usually need to call `:close` yourself — the walker fires it automatically. Calling explicitly is fine (the methods are idempotent) and useful when the resource's lifetime is bounded by a clear point in the test.

## See also

- [Lab reference](/provium/reference/lab.md) — every method.
- [provium global](/provium/reference/provium-module.md) — file-scope configuration globals.
- [VM reference](/provium/reference/vm.md), [Bridge reference](/provium/reference/bridge.md) — what lives in a lab.
- [Pool and parallelism](/provium/running-tests/pools-and-parallelism.md) — how the dispatcher uses claims.

---

# Running tests with the CLI

_Provium / Running tests_

> Patterns for the provium binary — discovery, filtering, the watch loop, --rerun-failed, --since, --tag, --include-slow, JSON output, and exit codes for CI.

The `provium` binary is what you run. This page covers the day-to-day patterns. The full flag inventory is on [the CLI reference](/provium/reference/cli.md).

## The basic invocation

```
provium tests/
```

Walks `tests/` for `*.test.lua` files and runs each. Default output:

```
PASS tests/smoke.test.lua (3 passed, 0 failed, 0 skipped)
FAIL tests/networking/partition.test.lua (2 passed, 1 failed, 0 skipped)
    FAIL split-brain
        client A could not reach client B after partition

2 file(s); 5 passed, 1 failed, 0 skipped; 12.34s
```

A single file works too:

```
provium tests/networking/partition.test.lua
```

If you omit paths entirely, the current directory is scanned recursively.

## Selecting what runs

### `--filter <substring>`

Match against the test-root-relative path:

```
provium tests/ --filter networking
provium tests/ --filter partition.test
```

### `--rerun-failed`

After every run with at least one failure, Provium writes the canonical paths to `~/.cache/provium/rerun.json`. `--rerun-failed` intersects discovered files against this list:

```
provium tests/ --rerun-failed
```

If no prior failure state exists, `--rerun-failed` exits cleanly with a notice (running the FULL suite is almost never what the user wanted).

A clean run leaves the failed-state file alone, so the failed set persists until you fix the failures.

### `--since <reference-file>`

Run only files whose mtime is newer than the reference file:

```
provium tests/ --since main.lua          # files edited after main.lua
provium tests/ --since .last-checkpoint  # custom marker
```

Useful for "what's changed since I last ran".

### Per-test filters

Tag and slow filtering happen inside each file (the test body's `meta` is what's matched):

```
provium tests/ --tag smoke
provium tests/ --tag smoke --tag fast    # OR: any tag matches
provium tests/ --no-tag flaky            # exclude tests tagged flaky
provium tests/ --include-slow            # include meta.slow tests
```

`--no-tag` wins over `--tag` (intersect semantics).

For filtering on arbitrary meta fields (not just `tags`), use `--tag-meta KEY=VALUE`:

```
provium tests/ --tag-meta subsystems=peinit
provium tests/ --tag-meta subsystems=peinit --tag-meta subsystems=loregd   # OR within KEY
provium tests/ --tag-meta subsystems=peinit --tag-meta area=boot            # AND across KEYs
provium tests/ --no-tag-meta flaky=true
```

This is the recommended pattern when test files are organised by user-visible scenario rather than by code subsystem — annotate tests with `subsystems = {"peinit", "loregd"}` (or any other axis you care about) and let CI compute "what to run" from the changeset:

```bash
# CI: PR touched the peinit subsystem
provium tests/ --tag-meta subsystems=peinit
```

See [meta tags](/provium/reference/meta-tags.md) for the convention.

Skipped tests still appear in outcomes — you'll see `SKIP <name>` in `-v` mode, with the filter expression as the reason.

## Output modes

### Default (compact)

```
FAIL tests/x.test.lua (2 passed, 1 failed, 0 skipped)
    FAIL the failing case
        message lines
```

### Verbose (`-v`)

Every test status, not just failures:

```
PASS tests/x.test.lua (3 passed, 0 failed, 0 skipped)
    PASS first
    PASS second
    PASS third
```

### Quiet (`-q`)

Only failures show. Files with all passing / skipped are silent. The summary line still prints.

### JSON (`--json`)

Line-delimited JSON, one object per file:

```json
{"path":"tests/x.test.lua","timeout":"in_time","passed":true,"chunk_error":null,"tests":[{"name":"…","status":"passed","message":null,"log":[]}]}
```

Useful for piping into another tool. Mutually exclusive with `--events-stdout`.

### Msgpack events (`--events-stdout`)

Length-prefixed msgpack frames over stdout. The human-readable / JSON renderer redirects to stderr automatically. Useful with `provium-coverage`:

```
provium tests/ --events-stdout | provium-coverage
```

See [events and coverage](/provium/running-tests/events-and-coverage.md) for the full event-stream story.

## Watch mode

```
provium tests/ --watch
```

Polls every 500 ms for stat-changed files; re-runs when one changes. Use Ctrl-C to exit. The empty-set case is fine — the watcher rescans every tick, so dropping a `*.test.lua` into the watched root after launch triggers discovery.

`--rerun-failed` is automatically dropped under `--watch` so file-edit detection works as expected (otherwise each tick would re-run only the original failed set forever).

## Per-file timeouts

```
provium tests/ --timeout 300            # 5 min (default)
provium tests/ --timeout 30s
provium tests/ --timeout 10m
provium tests/ --timeout 500ms
provium tests/ --timeout 0              # disable
```

Bare integers are seconds. Suffixes accepted: `ms`, `s`, `m`, `h`. `0` disables the timeout entirely.

When a file's timeout fires, the harness records it as `timed_out`, marks the file as failed for exit-code purposes, and emits a `file_completed` event with `status = "timed_out"`.

## Pool and CPU controls

By default Provium uses 80 % of host RAM and the full host CPU count for the resource pool. Override:

```
provium tests/ --mem 16G                 # explicit pool memory
provium tests/ --cpus 8                  # explicit pool vCPUs
provium tests/ --cpu-overcommit 1.5      # multiplier on --cpus
```

See [pools and parallelism](/provium/running-tests/pools-and-parallelism.md) for how the pool, claims, and dispatcher interact — including when overcommit is safe. Flag defaults and the clamp range are in the [CLI reference](/provium/reference/cli.md#vmm-and-resources).

## Dev-mode flags

### `--no-preflight`

Skip the `/dev/kvm` / iproute2 / nft / qemu / `CAP_NET_ADMIN` checks. Use in containers or CI environments where you know the environment is fine but the checks would fail.

### `--no-ksm`

Skip Kernel Same-page Merging tuning at startup. Default tunes `/sys/kernel/mm/ksm/*` per the design's pool-density goals; pass this on shared dev hosts where you don't want global tuning.

### `--vmm local`

Use the in-process LocalAgent backend instead of QEMU. Useful when KVM isn't available (CI, dev-on-laptop). The local backend does not actually boot a kernel — it gives the host bindings something to dispatch against, but tests that rely on guest-side behaviour (running commands, file I/O) won't work.

```
provium tests/ --vmm local
```

## Subcommands

When a subcommand is given, the test-runner mode is suppressed.

### `provium repl <profile>`

Boot a VM and drop into an interactive Lua REPL:

```
provium repl peios                  # cold-boot the peios profile
provium repl peios --name dev       # custom VM name
provium repl --fixture base         # resume from a fixture
```

Useful for poking at a guest interactively. The full Provium API is available — `vm:run`, `vm:read_file`, `vm:tail_file`, etc.

### `provium fixture <op>`

Manage the fixture cache:

```
provium fixture list                       # show every cached entry
provium fixture build path/to/fixture      # force-build
provium fixture rebuild path/to/fixture    # evict and rebuild
provium fixture clean                      # wipe the cache
provium fixture stale                      # list fixtures whose source doesn't match any cache entry
```

See [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) for when each is useful.

### `provium prepare [profile]`

Run dynamic profiles' `build` commands without booting anything — one profile by name, or every profile that declares a `build`. Pre-warm images with `provium prepare && provium tests/ --no-build`; see [dynamic profiles](/provium/configuration/dynamic-profiles.md).

### `provium list`

List discovered tests / fixtures without running anything:

```
provium list                 # tests
provium list --fixtures      # fixtures
```

### `provium lsp-setup [dir]`

Write Lua Language Server stubs and a `.luarc.json` into a test directory so the harness globals resolve in your editor — see [the CLI reference](/provium/reference/cli.md#provium-lsp-setup-dir---force).

## Exit codes

For CI, three values: `0` means every file passed (or skipped), `1` means at least one file did not finish cleanly, `2` means an internal error (config, pre-flight, dispatcher). The exit code is deliberately clamped to those three — the precise triggers and the clamping rationale are in the [CLI reference](/provium/reference/cli.md#exit-codes).

## CI patterns

### "Run everything, fail on any failure"

```
#!/bin/sh
provium tests/
```

Exit code is your test result. Capture stdout for the summary, stderr for the human-readable lines (when `--json` is set).

### "Run with structured output"

```
provium tests/ --json | tee results.jsonl
```

Each line is a complete `{"path":…,"tests":[…]}` object. Easy to pipe into a custom dashboard or per-test reporter.

### "Save events for post-hoc analysis"

```
provium tests/ --save-events events.msgpack
provium-coverage --from events.msgpack > coverage.html
```

The msgpack file is portable; you can analyse it on a different host.

### "Fail fast"

```
provium tests/ --fail-fast
```

Stops after the first failed file. Useful for tight inner loops where you want to see the first failure quickly.

### "Slow CI vs fast CI"

```
# Fast: skip slow tests by default.
provium tests/

# Slow / nightly: run everything.
provium tests/ --include-slow
```

`meta.slow = true` tests are skipped by default. Tag them in the test bodies, and your dev loop stays fast while nightly catches the slow stuff.

### "Parallel CI sharding"

There's no built-in shard splitter, but `provium` accepts any number of path arguments, so split the file list in your CI script and pass each shard its own files:

```
# $SHARDS = total shard count, $INDEX = this shard (0-based).
git ls-files 'tests/**/*.test.lua' | awk "NR % $SHARDS == $INDEX" | xargs provium
```

Each shard runs a disjoint subset of files. Don't try to build a shard with `--filter` — it's a single substring match against the relative path, not a pattern list.

## See also

- [CLI reference](/provium/reference/cli.md) — every flag, every env var.
- [Events and coverage](/provium/running-tests/events-and-coverage.md) — `--save-events`, `--events-stdout`, `--coverage`.
- [Pools and parallelism](/provium/running-tests/pools-and-parallelism.md) — how the resource pool works.

---

# Events and coverage

_Provium / Running tests_

> How to capture Provium's structured event stream — to a file, to stdout, or over a Unix socket — and feed it to provium-coverage or your own consumer.

Provium emits a structured msgpack event stream covering every file, test, VM, fixture, and pool transition. This page covers the practical side of consuming it; the wire format is on [events](/provium/reference/events.md).

## Why events

The human-readable `PASS / FAIL` summary is fine for the dev loop, but not enough for:

- Coverage reports (which test exercised which spec section?).
- Dashboards (live view of pool state, in-flight files, fixture builds).
- Post-hoc analysis ("why did this CI run take 4× longer than yesterday?").
- Debugging fixture rebuilds (which file is waiting on which build lock?).

For all of these, the event stream is what you want.

## Output channels

Provium can route events to three places independently. Combine flags freely.

### `--save-events <path>` — to a file

```
provium tests/ --save-events events.msgpack
```

Writes length-prefixed msgpack frames to the file. Use for post-hoc analysis or replay.

Under `--watch`, the file is truncated at each iteration so it contains exactly the most recent run.

### `--events-stdout` — to stdout

```
provium tests/ --events-stdout | provium-coverage
```

Streams events on stdout. The human-readable / JSON renderer redirects to stderr so consumers piping `provium --events-stdout | …` don't see human text interleaved into their msgpack parser. (`provium-coverage` reads stdin when `--from` is omitted, so the bare pipe above is the whole pipeline.)

Mutually exclusive with `--json`.

### `--events-socket <path>` — over a Unix socket

```
provium tests/ --watch --events-socket /tmp/provium.sock
# in another terminal — any client that can read a Unix socket:
socat - UNIX-CONNECT:/tmp/provium.sock | provium-coverage
```

The binary listens, accepts connections, and fans out frames. Useful for live dashboards. The socket persists across `--watch` iterations so connected clients aren't dropped on every re-run.

## Coverage post-run

`--coverage` is sugar over `--save-events` + a post-run pipe to `provium-coverage`:

```
provium tests/ --coverage
```

Internally:

1. If `--save-events <path>` is also set, Provium uses that file.
2. Otherwise, Provium tees events to a scratch tempfile (`$TMPDIR/provium-coverage-<pid>.msgpack`) with a sibling marker file.
3. After the run, runs `provium-coverage --from <path>`.
4. Cleans up the tempfile if Provium owns it (the marker is the proof).

Failures from the post-run pipe propagate as the exit code — a coverage failure shows up as exit `1`.

The cleanup is robust: SIGINT / SIGTERM handlers and an `atexit` hook clear the tempfile even on a killed run, so CI won't accumulate orphaned msgpack files.

## What's in the stream

The stream covers five areas: file lifecycle (`file_discovered` → `file_dispatched` → `file_completed`, with `file_blocked` when a file waits on the pool), test outcomes (`test_started` then one of `test_passed` / `test_failed` / `test_skipped`), VM lifecycle (`vm_spawned` / `vm_shutdown`), pool and claim activity, and fixture builds (`fixture_build_started` / `_done` / `_waiting` and `fixture_cache_hit`).

Every frame is `{ts, kind, payload}`. The full variant list and every payload field are in the [events reference](/provium/reference/events.md).

## Replay and analysis

```
provium tests/ --save-events events.msgpack
provium-coverage --from events.msgpack
```

The msgpack file is portable. You can analyse it on a different host than the one that produced it, or run multiple consumers against the same file:

```
provium-coverage --from events.msgpack > coverage.html
my-custom-tool --from events.msgpack > metrics.json
```

## Building your own consumer

The protocol is `provium-protocol`'s `EventFrame` over length-prefixed msgpack frames (the same framing as the agent wire protocol). The framing helpers in `provium-protocol::frame` are exposed for reuse.

A minimal Rust consumer:

```rust
use provium_protocol::events::{Event, EventFrame};
use provium_protocol::frame::{read_frame, DEFAULT_MAX_FRAME_BYTES};
use std::fs::File;
use std::io::BufReader;

fn main() -> std::io::Result<()> {
    let f = File::open("events.msgpack")?;
    let mut r = BufReader::new(f);
    loop {
        let frame: EventFrame = match read_frame(&mut r, DEFAULT_MAX_FRAME_BYTES) {
            Ok(f) => f,
            Err(_) => break,  // EOF or malformed; production code should distinguish
        };
        match frame.event {
            Event::TestPassed(t) => println!("PASS {}::{}", t.path, t.name),
            Event::TestFailed(t) => println!("FAIL {}::{} — {}", t.path, t.name, t.reason),
            _ => {}
        }
    }
    Ok(())
}
```

For other languages, use any msgpack library that handles the length-prefix framing (e.g. read 4 bytes BE length, then read N bytes, then deserialise as msgpack).

## Live multiplexing for dashboards

`--events-socket` is the right shape when you have a long-running dashboard:

```
# Provium runs in watch mode; events flow over the socket.
provium tests/ --watch --events-socket /tmp/provium.sock

# Dashboard connects once and stays connected across reruns.
my-dashboard --from-socket /tmp/provium.sock
```

The socket persists across `--watch` iterations precisely so dashboards don't have to reconnect on every re-run.

For non-dashboard pipelines (CI feeding a coverage tool), prefer `--events-stdout` — simpler, no socket-file management.

## Observability event guarantees

The harness guarantees ordering and pairing invariants (exactly one `file_completed` per dispatched file, exactly one terminal event per started test, paired claim and fixture-build events) — the full list is in [events reference — Event guarantees](/provium/reference/events.md#event-guarantees). Consumers can rely on these to track in-flight state, e.g. counting "in-flight files" as `file_dispatched - file_completed`.

## Practical patterns

### "Why did this run take so long?"

```
provium tests/ --save-events events.msgpack
# parse events.msgpack to find the longest file_completed.duration_ns
```

### "Which fixtures rebuilt?"

```
provium tests/ --save-events events.msgpack
# count fixture_build_started events
```

If many fixtures are rebuilding, something invalidated the cache — usually a kernel swap or a helper edit.

### "Spec coverage report"

```
provium tests/ --include-slow --coverage
```

`provium-coverage` reads `meta.spec` from each `test_passed` / `test_failed` event and produces a spec-section report.

### "What was the guest doing when this test failed?"

The `test_failed` event includes `console_excerpt` — the last 4 KiB of every booted VM's console log captured at the failure moment. Useful when failures are kernel-side and reproducing locally is expensive.

## See also

- [Events reference](/provium/reference/events.md) — every event, every field.
- [Protocol version](/provium/reference/protocol-version.md) — wire-shape pinning.
- [The CLI](/provium/running-tests/the-cli.md) — `--save-events`, `--events-stdout`, `--events-socket`, `--coverage`.

---

# Fixtures and dependencies

_Provium / Running tests_

> How fixtures are discovered, cached, keyed, locked, and evicted. When a fixture rebuilds. How to manage the cache from the CLI.

Fixtures are pre-built VM (or whole-lab) snapshots that test files restore from instead of re-doing setup work. This page covers the cache model end-to-end.

The exhaustive method reference is on [Lab](/provium/reference/lab.md) (`vm_fixture`, `lab_fixture`); the userdata is on [Snapshot](/provium/reference/snapshot.md).

## Discovery

A fixture is any `*.fixture.lua` file under one of the `roots` directories listed in `provium.toml`. Test files reference fixtures by their test-root-relative path with the `.fixture.lua` suffix omitted:

```
tests/
  fixtures/
    base.fixture.lua            -- referenced as "fixtures/base"
    cluster.fixture.lua         -- referenced as "fixtures/cluster"
  uses-base.test.lua
```

```lua
-- in uses-base.test.lua:
local vm = provium:vm_fixture("fixtures/base")
```

If the same name resolves under multiple roots, the first-listed root wins. Missing fixtures error at restore time with `fixture \`X\` not found in any test root`.

## Two fixture shapes

### Single-VM fixture (`vm_fixture`)

The chunk ends with `return vm:snapshot()`:

```lua
-- tests/fixtures/base.fixture.lua
local vm = provium:vm("base", "peios"):boot()
vm:run("seq 1000000 > /srv/corpus"):assert_ok()
return vm:snapshot()
```

Cached as `<key>.snap` — a single sparse zstd-compressed file.

### Lab fixture (`lab_fixture`)

The chunk ends with `return provium:snapshot()`:

```lua
-- tests/fixtures/cluster.fixture.lua
local lan = provium:bridge("lan")
local a = provium:vm("a", "peios"):boot()
local b = provium:vm("b", "peios"):boot()
lan:attach({a, b})
return provium:snapshot()
```

Cached as `<key>.lab/` — a directory containing per-VM `.snap` files plus a `lab.json` index.

## What ends up in the cache key

The cache key is a SHA-256 of, in order:

1. The fixture file's source bytes.
2. Every transitively-referenced fixture's key (so `vm_fixture("derived")` calling `vm_fixture("base")` invalidates when `base` changes).
3. Every `require()`d helper's source bytes (recursively — helpers that require other helpers fold in too).
4. The kernel and initrd identifier of EVERY profile in `provium.toml` (sorted by name for determinism).
5. Every external host-file declared with `vm:push_file("…", …)` or `lab:depends_on_file("…")` (path + mtime + size). See [External host-file deps](#external-host-file-deps).

Edit any of those, and the next run rebuilds the fixture. This is intentional:

- A fixture builder editing the file → rebuild.
- A helper module the fixture requires gets edited → rebuild.
- A different fixture the fixture references gets rebuilt → rebuild this one too.
- A new kernel image is dropped in → every fixture rebuilds.

Multi-profile cache-key folding means a kernel change on profile B invalidates fixtures even if they only ever boot under profile A. This is a deliberate over-invalidation: it's safer than serving a fixture built against a now-stale kernel.

## External host-file deps

Fixtures routinely push host-side files — a freshly-built binary, a config template, a test corpus — into the guest. The cache key folds those host files in automatically so editing the file on the host invalidates the snapshot.

### `vm:push_file(host_path, guest_path, opts?)`

Reads `host_path` on the host and writes its bytes to `guest_path` in the guest. The host file's identifier (path + mtime + size) is folded into the fixture's cache key by default — rebuild the binary, get a fresh fixture next run.

```lua
-- tests/fixtures/uapi-ready.fixture.lua
local vm = provium:vm("uapi", "peios"):boot()
vm:push_file("../peios-uapi/target/release/peios-uapi", "/usr/bin/peios-uapi")
vm:run("/usr/bin/peios-uapi --setup"):assert_ok()
return vm:snapshot()
```

Relative `host_path` is resolved against the directory containing the fixture file (or the directory of the require'd helper that calls `push_file`), not the cwd at invocation time. That keeps things stable regardless of where `provium` was launched from.

To opt out for a specific call — e.g. a large test corpus you don't want included in the key — pass `auto_dep = false`:

```lua
vm:push_file("../big-corpus.tar", "/data.tar", {auto_dep = false})
```

The opt-out is detected by static source scanning, so the `auto_dep = false` must be a literal in the call site. A variable like `{auto_dep = x}` does NOT opt out (the default is to track).

### `lab:depends_on_file(host_path)`

Declares an external host-file as a fixture-cache dependency without doing any I/O. Useful when the fixture reads the file on the host side — e.g. a config template parsed in Lua, or a binary the build script invokes locally before pushing post-processed output.

```lua
provium:depends_on_file("../templates/sshd_config.in")
local rendered = render_template("../templates/sshd_config.in", {port = 2222})
vm:write_file("/etc/ssh/sshd_config", rendered)
return vm:snapshot()
```

`provium:` is the root lab; sub-labs (`provium:lab("dc1"):depends_on_file(...)`) work too. All declarations on any lab in the fixture fold into the same fixture-level cache key.

### Limits of static scanning

The fold is driven by scanning the fixture and its helpers for literal-string arguments to `push_file` and `depends_on_file`. Two cases fall outside that:

- **Variable host paths**: `vm:push_file(my_path, "/foo")` doesn't fold (the scanner can't resolve `my_path`). Either inline the literal or follow up with an explicit `lab:depends_on_file("…")`.
- **Generated paths**: `vm:push_file("build/" .. arch, "/bin/foo")` doesn't fold either.

The fix is the same in both cases: add a separate literal `lab:depends_on_file("…")` declaration alongside the dynamic call. If a fixture's deps genuinely can't be expressed as literals, fall back to `provium fixture rebuild <name>` after host-side changes.

## Cache layout

```
~/.cache/provium/fixtures/
  c5e6f8….snap            -- single-VM fixture snapshot (sparse, zstd)
  c5e6f8….lock            -- per-key build lock
  abc123….lab/            -- lab-fixture directory
    lab.json
    a.snap
    b.snap
  abc123….lab.lock
```

Override the cache directory in `provium.toml`:

```toml
[provium]
cache_dir = "/var/cache/provium/fixtures"
```

Default: `~/.cache/provium/fixtures/`.

## Build flow

When a test calls `provium:vm_fixture("name")`:

```mermaid
flowchart TD
  Start[vm_fixture call] --> KeyComp[Compute key — source + deps + kernels]
  KeyComp --> Lookup{Key in cache?}
  Lookup -->|yes| Restore[Restore from cache]
  Lookup -->|no| Lock[Acquire build lock]
  Lock --> Recheck{Peer built it while we waited?}
  Recheck -->|yes| Restore
  Recheck -->|no| Build[Build under lock]
  Build --> Compress[Sparse + zstd]
  Compress --> Install[Atomic install]
  Install --> Restore
  Restore --> Return[Return VM]
```

Build under lock ensures only one process builds a given fixture even when 16 test files reference the same fixture. Other files queue on the lock and either restore from the freshly-installed cache (if the holder finished cleanly) or rebuild themselves (if the holder crashed).

`fixture_build_started` / `fixture_build_done` events frame each build. `fixture_build_waiting` fires when a file queues behind a peer; the payload includes `held_by_file` so dashboards can show "file X is waiting on file Y to build fixture Z."

## Atomic install

Lab fixtures use `renameat2(RENAME_EXCHANGE)` to swap the freshly-built `<key>.lab/` with any existing one. If the kernel doesn't support the syscall, falls back to a sibling-rename + cleanup.

Single-VM fixtures use plain `rename` after sparse + zstd compression. Both paths leave the cache in a coherent state — readers either see the old version or the new, never half-installed.

## Eviction

LRU eviction runs at every `provium` startup, before tests dispatch:

1. Sum file sizes under `cache_dir`.
2. Sort entries by access time (atime).
3. Delete oldest until total is ≤ `cache_max_size`.

Default `cache_max_size`: `20 GiB`. Override in `provium.toml`:

```toml
[provium]
cache_max_size = "100G"
```

Each successful restore bumps the entry's atime so frequently-used fixtures stay hot. Without the bump, relatime would let every entry's atime collapse together and eviction would degrade to filesystem order.

## Corrupt entries

If a restore fails (decompression error, version mismatch not caught by the key), the harness:

1. Prints a warning to stderr naming the fixture.
2. Evicts the entry.
3. Falls through to the rebuild path.

This is logged but not fatal — the next restore attempt builds fresh.

## CLI management

### `provium fixture list`

```
   23.45MiB  vm   c5e6f8…
   78.12MiB  lab  abc123…

2 entries, 101.57MiB
```

Each entry shows size, kind (`vm` or `lab`), and the cache key. Useful for "what's hot in my cache?"

### `provium fixture build <path>`

Force-build the named fixture. If the entry is already cached, prints `already built: <path>` and exits.

```
provium fixture build fixtures/base
```

Useful in CI to warm the cache before the test run.

### `provium fixture rebuild <path>`

Evict and rebuild. Useful when you know the fixture should change but the harness's cache key didn't catch it (rare):

```
provium fixture rebuild fixtures/base
```

### `provium fixture clean`

Wipe the entire cache directory:

```
provium fixture clean
```

The next run rebuilds everything from scratch. Slow but safe.

### `provium fixture stale`

List fixtures whose source / dep / kernel hash doesn't match any cached entry:

```
provium fixture stale
  tests/fixtures/base.fixture.lua

1 stale fixture(s)
```

Useful to check "what would the next `provium` run rebuild?" without running anything.

## Performance notes

Fixture restores are fast — typical single-VM snapshot restore is < 1 second on warm caches because:

1. The snapshot file is sparse-zstd compressed at build time, so reading and decompressing is dominated by kernel buffer cache hits.
2. The harness uses `renameat2(RENAME_EXCHANGE)` for atomic install (no observable in-between state for readers).
3. Each successful restore bumps atime, so popular fixtures stay LRU-hot.

Fixture builds are slow — they involve actually booting a VM, doing the setup, and snapshotting. Build them once per cache-key change and amortise across every test that references them.

## Common patterns

### "Make a clean baseline available everywhere"

```lua
-- tests/fixtures/clean.fixture.lua
local vm = provium:vm("clean", "peios"):boot()
return vm:snapshot()
```

Then every test starts from a fresh boot without paying the boot cost:

```lua
test("…", function(t)
    local vm = provium:vm_fixture("fixtures/clean")
    -- … fresh-boot semantics …
end)
```

### "Stack fixtures to amortise expensive setup"

```lua
-- tests/fixtures/with-corpus.fixture.lua
local vm = provium:vm_fixture("fixtures/clean")
vm:run("seq 1000000 > /srv/corpus"):assert_ok()
return vm:snapshot()
```

`with-corpus` is keyed off `clean`'s key — when `clean` rebuilds, `with-corpus` rebuilds too. When `clean` is a cache hit, `with-corpus` only pays the corpus-build cost.

### "Cache a multi-VM topology"

```lua
-- tests/fixtures/two-node-cluster.fixture.lua
local lan = provium:bridge("lan")
local a, b = provium:vm("a", "peios"):boot(), provium:vm("b", "peios"):boot()
lan:attach({a, b})
a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")
return provium:snapshot()
```

```lua
test("…", function(t)
    local cluster = provium:lab_fixture("fixtures/two-node-cluster")
    cluster.a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
end)
```

The whole topology — bridge, both VMs, partition state, attached disks, everything — is one cached unit.

### "Pre-warm the cache in CI"

Add to your CI script:

```sh
provium fixture build fixtures/clean
provium fixture build fixtures/with-corpus
provium fixture build fixtures/two-node-cluster
provium tests/  # now every test sees cache hits
```

Without pre-warming, the first test to reference each fixture pays the build cost serially.

## What can invalidate the cache

| Change | Invalidates |
|---|---|
| Edit `<fixture>.fixture.lua` | That fixture only. |
| Edit a helper that the fixture `require`s | The fixture and every fixture that references it. |
| Edit a fixture that another fixture references via `vm_fixture`/`lab_fixture` | Both. |
| New kernel or initrd image (any profile) | Every fixture. |
| Change `[profiles.<name>].kernel` or `.initrd` path | Every fixture. |
| Edit a file declared with `vm:push_file` or `lab:depends_on_file` | The fixture (and any fixture that references it). |
| `provium fixture rebuild` / `clean` | Per command. |

Adding a new profile invalidates the cache (the new profile's kernel/initrd are folded in even if no test uses the new profile). This is by design.

## See also

- [Lab reference](/provium/reference/lab.md) — `vm_fixture`, `lab_fixture`.
- [Snapshot reference](/provium/reference/snapshot.md) — what fixture builders return.
- [provium.toml reference](/provium/configuration/provium-toml.md) — `cache_dir`, `cache_max_size`.
- [The CLI](/provium/running-tests/the-cli.md) — `provium fixture …` subcommands.

---

# Pools and parallelism

_Provium / Running tests_

> How the resource pool, per-file claims, PSI throttling, and CPU overcommit shape parallel test execution. How to tune for your hardware.

Provium's dispatcher runs multiple test files in parallel against a shared resource pool. Each file declares what it needs (or accepts the per-file overhead default), and the dispatcher schedules as many as fit. This page covers how to think about that for your test corpus.

## The pool

When `provium` starts, it builds one pool with two budgets:

| Budget | Default | Override |
|---|---|---|
| Memory | 80 % of host RAM | `--mem 16G` |
| vCPUs | host online CPUs | `--cpus 8` (then `--cpu-overcommit <multiplier>`) |

The pool tracks total and available; every dispatch takes from available, every release returns. A `pool_state` event fires every second with `{used, available}` so dashboards see live utilisation.

If you want to be conservative on a busy host:

```
provium tests/ --mem 8G --cpus 4
```

If you have a beefy CI box and want maximum throughput:

```
provium tests/ --mem 64G --cpus 32 --cpu-overcommit 1.5
```

`--cpu-overcommit` is clamped to `[0.5, 8.0]`. Default `1.0` (strict — no oversubscription). `1.5` or `2.0` allows oversubscription if the workload tolerates scheduling jitter.

## Per-file claims

Each test file may declare what it needs:

```lua
provium:claim({memory = "4G", cpus = 4})

test("…", function(t) end)
test("…", function(t) end)
```

The claim is taken at file dispatch and released at file completion. A second `:claim` errors — `lab claim already held; one-shot per lab`.

Without a claim, the dispatcher uses the per-file overhead default (50 MiB memory, 0 CPU). A file that boots 3 VMs at 1 GiB each but doesn't claim anything gets through the pool gate immediately and then might OOM the host because the overhead default is way under the actual demand.

**Rule of thumb:** claim memory equal to the sum of expected VM memory budgets plus a small buffer; claim CPUs equal to the sum of expected VM vCPU budgets.

```lua
-- 3 VMs × (2G memory, 2 CPUs each):
provium:claim({memory = "7G", cpus = 7})  -- 6G VMs + 1G buffer; 6 vCPUs + 1
```

## File dispatch flow

```mermaid
flowchart LR
  Disc[file_discovered] --> Wait{pool has room?}
  Wait -->|no| Block[file_blocked]
  Block --> Wait
  Wait -->|yes| Disp[file_dispatched]
  Disp --> Acq[claim_acquired]
  Acq --> Run[Run tests in file]
  Run --> Comp[file_completed]
  Comp --> Rel[claim_released]
```

Files queue at the pool until their reservation fits. The order is stable — the dispatcher takes files in discovery order, so a file that doesn't fit blocks subsequent files (FIFO). For now there's no priority or backfill; if you need a specific file to run first, list its path explicitly.

## PSI throttling

On Linux hosts with pressure stall information (PSI) available, Provium spawns a pressure monitor at startup. It polls CPU and memory pressure once a second; when either crosses the threshold (10 % some-pressure averaged over 10 seconds), the dispatcher pauses new file dispatches until pressure drops.

The threshold and poll interval are fixed — they are not configurable from the CLI.

Files that are already in flight continue. The throttling only delays new dispatch.

When PSI throttling is the cause of a `file_blocked`, the event's `reason` field is `psi_pressure` (versus `pool_full` for "the pool can't afford this file's claim").

## KSM tuning

Kernel Same-page Merging deduplicates identical pages across VMs. When you boot many VMs from the same kernel + initrd, KSM can reclaim significant memory. Provium tunes `/sys/kernel/mm/ksm/*` at startup unless `--no-ksm` is passed.

The KSM tuning runs once per `provium` invocation, setting `run = 1`, `pages_to_scan = 1000`, and `sleep_millisecs = 20`. Best-effort: a one-line summary goes to stderr.

```
provium: ksm: tuned (3 knobs)
```

If your host is shared with non-Provium workloads, pass `--no-ksm` so Provium doesn't change global tuning.

## What blocks parallelism

Several things prevent unlimited parallelism even when the pool has room:

| Thing | Why |
|---|---|
| Fixture build lock | One process at a time per fixture key. Other files queue on `fixture_build_waiting`. |
| Pool reservation | A file with a 16 G claim won't run alongside other big files until pool has 16 G free. |
| PSI pressure | High CPU pressure pauses new dispatches. |
| `--fail-fast` | Stops new dispatch after the first file failure. |

Of these, fixture build lock is the most common surprise. If 16 test files all reference `fixtures/base` and the cache is cold, all 16 queue on the build lock; only one builds, the rest wait. Pre-warm with `provium fixture build fixtures/base` to amortise.

## Inspecting parallelism in a run

The event stream gives you the full picture:

```
provium tests/ --save-events events.msgpack
```

Then walk the stream:

- `file_discovered` events at the start tell you the universe.
- `file_dispatched` events tell you what actually ran in parallel (count of in-flight = `file_dispatched - file_completed`).
- `file_blocked` events with `reason` tell you why something queued.
- `pool_state` events at 1 Hz give you a usage timeline.

`provium-coverage` summarises this in its run report; for ad-hoc debugging, parse the msgpack with the snippet in [events and coverage](/provium/running-tests/events-and-coverage.md).

## Tuning patterns

### "I want to maximise throughput"

```
provium tests/ --cpus $(nproc) --mem $(awk '/MemTotal/ {printf "%dG", $2/1024/1024 - 4}' /proc/meminfo)
```

Use everything except 4 GiB of RAM and overcommit CPUs gently:

```
provium tests/ --cpu-overcommit 1.5
```

### "I want to be conservative on a shared host"

```
provium tests/ --cpus 4 --mem 8G --no-ksm
```

### "I want to detect over-subscription"

Watch the `pool_state` and `file_blocked` event streams. Frequent `file_blocked` with `reason = pool_full` means files are claiming more than the pool can serve in parallel — either the pool is too small or claims are too generous. Frequent `file_blocked` with `reason = psi_pressure` means the host is genuinely overloaded — reduce parallelism (`--cpus`, `--mem`) or move other workloads off the host.

### "I want to debug a slow run"

```
provium tests/ --save-events events.msgpack
```

Parse the msgpack for the longest `file_completed.duration_ns`. Then look at that file's `test_started` / `test_passed` events to see which test(s) are slow. Cross-reference with `vm_spawned` events to see how many VMs were involved.

### "I want to test for resource leaks"

The pool's available budget should return to its full value after each `claim_released`. Watch `pool_state` over time — if available is drifting down monotonically, something is leaking. The most common culprits:

- A test that exits via panic without releasing a claim — but the dispatcher releases on `file_completed` regardless of how it ended, so this shouldn't happen.
- A bug in the harness — file an issue with the event stream.

## VMs vs files

A common confusion: the pool tracks per-file resources, not per-VM. The dispatcher reserves the file's full claim at dispatch and holds it until file completion, regardless of how many VMs the file actually boots concurrently.

```lua
-- This file claims 4G, but only ever has one VM live at a time:
provium:claim({memory = "4G"})

test("a", function(t)
    local vm = provium:vm("v", "peios"):boot()
    vm:shutdown()  -- VM gone, but claim still held
end)

test("b", function(t)
    local vm = provium:vm("v", "peios"):boot()
end)
```

The claim doesn't release between tests. If you want fine-grained reservation, you'd need a smaller claim and rely on the dispatcher's per-file overhead — but the trade-off is potential OOM if the claim is too small for the actual peak.

For the typical case, claim for the file's worst-case peak.

## See also

- [The CLI](/provium/running-tests/the-cli.md) — `--mem`, `--cpus`, `--cpu-overcommit`, `--no-ksm`.
- [Lab reference](/provium/reference/lab.md) — `lab:claim`.
- [Events](/provium/reference/events.md) — `pool_state`, `file_blocked`, `claim_acquired`, `claim_released`.
- [Fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) — fixture build lock as a parallelism limiter.

---

# provium.toml

_Provium / Configuration_

> Every field in provium.toml — the [provium] section (roots, cache_dir, cache_max_size) and the [profiles.<name>] blocks (kernel, initrd, cmdline, guest_os).

`provium.toml` is the only config file Provium reads. The default location is `./provium.toml`; override with `--config <path>`.

The schema has two top-level keys: `[provium]` for runner settings, and `[profiles.<name>]` for per-VM-type tuples. Both are optional individually — an empty `provium.toml` parses cleanly but won't be useful for anything (no profiles means `provium:vm(name, profile)` won't find any profile to look up).

## Minimum useful config

```toml
[provium]
roots = ["tests"]

[profiles.peios]
kernel  = "/path/to/bzImage"
initrd  = "/path/to/initrd.cpio.gz"
cmdline = "console=ttyS0 quiet"
```

This declares one test root and one profile. `provium tests/` will scan `tests/` for `*.test.lua`, and tests can call `provium:vm("name", "peios")`.

## `[provium]` section

### `roots`

```toml
[provium]
roots = ["tests", "vendor/upstream-tests"]
```

| Type | Default | Description |
|---|---|---|
| array of strings | `[]` (empty) | Directories scanned for `*.test.lua` and `*.fixture.lua` files. Also prepended to `package.path` so `require("helper.module")` resolves under any root. |

When `roots` is empty (or unset), `provium <paths>` scans whichever paths you pass on the CLI; if you also omit those, it scans the current directory.

For most projects, set `roots = ["tests"]` (or whichever directory contains your tests) so that:

- `provium` (no arg) walks the test tree.
- `provium fixture build foo` finds `tests/foo.fixture.lua`.
- Tests can `require("helpers.assert_pingable")` and Provium resolves it as `tests/helpers/assert_pingable.lua`.

### `cache_dir`

```toml
[provium]
cache_dir = "/var/cache/provium/fixtures"
```

| Type | Default | Description |
|---|---|---|
| path string | `~/.cache/provium/fixtures` | Where fixture snapshots are stored. |

Useful for:

- Per-machine caches on shared hosts (default is per-user, set to a system-wide path for shared CI runners).
- Fast scratch storage (point at an SSD or tmpfs for build performance).
- Large dedicated cache (point at a partition with more headroom than `~/.cache`).

The directory is created on first build. If the path is unreadable / unwritable at startup, eviction silently skips and the next build attempt errors with the underlying I/O error.

### `cache_max_size`

```toml
[provium]
cache_max_size = "100G"
```

| Type | Default | Description |
|---|---|---|
| string with `K`/`M`/`G`/`T` suffix, or bare bytes | `"20G"` | LRU eviction target. The cache is allowed to grow beyond this between runs; eviction trims at the next `provium` startup. |

When the cache exceeds the cap, Provium sorts entries by access time and deletes oldest until total size is under the cap. Each successful restore bumps the entry's atime so popular fixtures stay hot.

## `[profiles.<name>]` blocks

Each `[profiles.<name>]` block declares one (kernel, initrd, cmdline, guest_os) tuple. Test code looks them up by name: `provium:vm("v", "<name>")`.

You can have any number of profiles; tests pick whichever they need.

### `kernel`

```toml
[profiles.peios]
kernel = "/build/peios/bzImage"
```

| Type | Required | Description |
|---|---|---|
| path string | yes | Path to a bzImage-format kernel image. Booted via QEMU's `-kernel` option. |

Path validation (does the file actually exist?) happens at VM-boot time, not config-load time. This lets a single `provium.toml` be portable across machines that have different kernel layouts.

### `initrd`

```toml
[profiles.peios]
initrd = "/build/peios/initrd.cpio.gz"
```

| Type | Required | Description |
|---|---|---|
| path string | yes | Path to an initramfs. Booted via QEMU's `-initrd`. |

By default, Provium injects the `provium-agent` binary at `/sbin/provium-agent` by concatenating a small overlay cpio onto your initrd at launch (the kernel unpacks concatenated gzip cpios into a single rootfs). The agent boots as PID 1 and forks immediately: the child becomes the vsock listener, the parent execs your initrd's `/init`, which therefore takes over PID 1. In this chained layout the agent deliberately mounts nothing first — your init owns the mount and pivot sequence exactly as it would alone. (Only on an agent-only initrd, with no user `/init`, does the agent perform the pseudo-FS mounts itself.) Userspace runs as it would have on its own; the agent runs alongside as PID 2.

Three consequences:

- **Your initrd doesn't need to know about Provium.** A vanilla distro initramfs, a buildroot image, or a from-scratch cpio with just `/init` will all work.
- **Your `/init` runs as PID 1.** If your tests need PID-1 init duties (signals, child reaping), they happen in your init as before.
- **The merged initrd is content-hash cached** under `<scratch_root>/agent-overlay-cache/<sha256>.cpio.gz`. Subsequent boots of the same `(initrd, overlay)` pair pay nothing.

To opt out (e.g. when your initrd already bundles an agent at the path Provium would set), set `inject_agent = false` on the profile.

### `inject_agent`

```toml
[profiles.peios]
inject_agent = false
```

| Type | Default | Description |
|---|---|---|
| bool | `true` | When `true`, Provium concatenates the agent overlay onto `initrd` at launch and appends `rdinit=/sbin/provium-agent` to the kernel cmdline. Set `false` to use the initrd as-is. |

If your cmdline already pins a different `rdinit=PATH`, the launch errors with a clear conflict message rather than silently overriding. Either remove the conflicting `rdinit=`, or set `inject_agent = false`.

### `agent_overlay_path`

```toml
[profiles.peios]
agent_overlay_path = "/usr/local/share/provium/agent-overlay.cpio.gz"
```

| Type | Default | Description |
|---|---|---|
| path string | unset | Override the path to the agent overlay cpio. Useful for distribution-installed Provium where the overlay isn't co-located with the binary. |

When unset, Provium tries (in order): the `PROVIUM_OVERLAY` env var, then `<provium-binary-dir>/../share/provium/agent-overlay.cpio.gz`, then walks up the binary's directory tree looking for `dist/agent-overlay.cpio.gz` (covers in-development runs from `target/`). Set this field — or the env var — when none of those apply.

### `cmdline`

```toml
[profiles.peios]
cmdline = "console=ttyS0 quiet"
```

| Type | Required | Description |
|---|---|---|
| string | one of `cmdline` / `cmdline_file` | Inline kernel command line. May be empty or omitted **when `cmdline_file` is set** — the two compose. A profile with neither a non-empty `cmdline` nor a `cmdline_file` is rejected with `profile \`<name>\` has empty cmdline and no \`cmdline_file\`; set at least one`. |

`vm:boot({kernel_cmdline = "..."})` overrides the whole command line for one boot.

Provium **always appends `console=ttyS0`** if absent. The kernel happily uses multiple `console=` directives, so any user-set values are preserved alongside. The reason: the QEMU command wires the serial port into Provium's console capture, so the kernel's printk needs to land there for failure diagnostics to be visible. Without `console=ttyS0`, PID 1's `/dev/console` may resolve to a device QEMU doesn't capture, and writes to stderr can fail and crash userspace.

Common additions:

- `loglevel=7` — verbose kernel logging for debugging.
- `nokaslr` — for reproducible kernel addresses in panic dumps.
- `panic=1` — panic immediately rather than hanging on unrecoverable errors.

### `cmdline_file`

```toml
[profiles.peios]
cmdline_file = "../peiso/out/root/boot/cmdline"
cmdline      = "loglevel=7"   # optional; appended after the file
```

| Type | Required | Description |
|---|---|---|
| path string | one of `cmdline` / `cmdline_file` | Path to a file whose contents are the **base** command line — typically an image builder's generated `cmdline`. Read at VM-boot time; resolved relative to the current directory, like `kernel`/`initrd`. |

All whitespace in the file — including newlines — is collapsed to single spaces, then the inline `cmdline` (if any) is appended **after**. Because the kernel applies last-wins semantics to most repeated parameters (`init=`, `loglevel=`, …), an inline token overrides the file's value for those.

The point is to stop the command line from drifting. If a builder bakes `init=/usr/bin/protoinit` into the image's cmdline and you hand-copy that into `cmdline`, the two silently diverge the next time the builder changes. Pointing `cmdline_file` at the builder's output means Provium reads the authoritative value every boot. See [Dynamic profiles](/provium/configuration/dynamic-profiles.md#cmdline-from-the-builder).

### `guest_os`

```toml
[profiles.peios]
guest_os = "peios"
```

| Type | Default | Description |
|---|---|---|
| string | `"peios"` | Guest OS identifier. v1 only supports `"peios"`. |

Validated at config load time. A profile with `guest_os = "linux"` (or any other value) errors with `profile \`<name>\` has guest_os = \`linux\`, only \`peios\` is supported in v1`.

The field exists to future-proof the schema for other ports — when there's a Windows agent, this would be how a profile selects it.

### `build`

```toml
[profiles.peios]
build        = "peiso build manifests/peios.toml --out {out}"
kernel       = "{out}/root/usr/lib/modules/<release>/vmlinuz-<release>"
initrd       = "{out}/initrd.img"
cmdline_file = "{out}/root/boot/cmdline"
```

| Type | Default | Description |
|---|---|---|
| string | unset | Shell command (run with `sh -c`) that produces this profile's boot artifacts. Runs **once before any VM boots**. A non-zero exit aborts the run. |

The literal token `{out}` — in `build` and in the path fields — expands to this profile's build-output directory (see `build_out`), so the command's `--out` and the `kernel`/`initrd`/`cmdline_file` Provium later reads are the same path and cannot drift.

Provium tracks no staleness: the command runs every invocation. Making rebuilds cheap when nothing changed is the builder's job, not Provium's. Skip the hook with `--no-build`, or run it without booting via `provium prepare`. Full treatment on [Dynamic profiles](/provium/configuration/dynamic-profiles.md).

### `build_out`

```toml
[profiles.peios]
build_out = "/tmp/provium-builds/peios"
```

| Type | Default | Description |
|---|---|---|
| path string | `$XDG_CACHE_HOME/provium/builds/<profile>/` | The directory `{out}` expands to. |

When unset, the default base is resolved like the fixture cache — `$PROVIUM_BUILD_DIR`, then `$XDG_CACHE_HOME/provium/builds`, then `~/.cache/provium/builds`, then `/tmp/provium-builds` — with the profile name appended. Provium creates the directory before running `build` but **never wipes it**: the `build` command owns its contents (so it can keep its own incremental-build caches there).

## Multiple profiles

You can declare any number of `[profiles.<name>]` blocks; tests pick one by name per VM. Patterns for multi-profile setups (debug builds, cross-version testing, feature-flag gating) are on [Profiles](/provium/configuration/profiles.md).

**Important:** the fixture cache key folds in EVERY profile's kernel + initrd identifier. Adding a new profile invalidates the entire fixture cache. This is intentional: a new kernel could behave differently, so existing fixtures are no longer trustworthy. See [Profiles — What happens to the cache when profiles change](/provium/configuration/profiles.md#what-happens-to-the-cache-when-profiles-change).

## Configuration loading

| Step | What happens |
|---|---|
| 1. Read | `provium.toml` is read from `--config <path>` (default `./provium.toml`). |
| 2. Parse | TOML is parsed. Parse errors include the file path. |
| 3. Validate | Each profile is validated (a non-empty `cmdline` **or** a `cmdline_file`; `guest_os = "peios"`). |
| 4. Expand | The `{out}` token in each profile's `build` command and path fields is replaced with that profile's resolved build-output directory. |
| 5. Use | The `Config` struct is wrapped in an `Arc` and passed to every file runner. |

Errors:

| Error | Cause |
|---|---|
| `read \`<path>\`: <io error>` | File missing or unreadable. |
| `parse \`<path>\`: <toml error>` | Malformed TOML. |
| `invalid config in \`<path>\`: <message>` | Validation failed (empty `cmdline`, unsupported `guest_os`, etc.). |

All three abort the run with exit code `2`.

## Worked example

```toml
[provium]
roots          = ["tests", "internal-tests"]
cache_dir      = "/srv/provium-cache"
cache_max_size = "200G"

[profiles.peios]
kernel   = "/srv/peios-builds/latest/bzImage"
initrd   = "/srv/peios-builds/latest/initrd.cpio.gz"
cmdline  = "console=ttyS0 quiet panic=1"
guest_os = "peios"

[profiles.peios-mainline]
kernel   = "/srv/peios-builds/mainline/bzImage"
initrd   = "/srv/peios-builds/mainline/initrd.cpio.gz"
cmdline  = "console=ttyS0 quiet panic=1"
guest_os = "peios"

[profiles.peios-debug]
kernel   = "/srv/peios-builds/debug/bzImage"
initrd   = "/srv/peios-builds/debug/initrd.cpio.gz"
cmdline  = "console=ttyS0 debug loglevel=7 nokaslr panic=1"
guest_os = "peios"
```

This declares three profiles (production, mainline, debug), centralises the cache on a 200 GiB dedicated mount, and scans both `tests/` and `internal-tests/`.

## See also

- [Project structure](/provium/getting-started/project-structure.md) — the broader project layout.
- [VM reference](/provium/reference/vm.md) — `boot_opts.kernel_cmdline` overrides.
- [Profiles](/provium/configuration/profiles.md) — patterns for using multiple profiles.

---

# Profiles

_Provium / Configuration_

> Patterns for multiple profiles — kernel-version testing, debug builds, feature-flag gating, and what happens to the fixture cache when profiles change.

A profile is one named `(kernel, initrd, cmdline, guest_os)` tuple in `provium.toml`. Tests pick a profile by name when they create a VM. This page covers the practical patterns.

The configuration field reference is on [provium.toml](/provium/configuration/provium-toml.md). A profile can also *build* its own artifacts before booting rather than pointing at files already on disk — see [Dynamic profiles](/provium/configuration/dynamic-profiles.md).

## When you need more than one profile

For most projects, one profile is enough — `peios`, pointing at the latest build. You'd add more when:

- You're testing across kernel versions (`peios-stable`, `peios-mainline`).
- You want a debug build available for diagnosing failures (`peios-debug`).
- You're gating tests on a feature flag in the kernel (`peios-prefeatx`, `peios-postfeatx`).
- You're testing a port to a new architecture (`peios-arm64`).

## Declaring multiple profiles

```toml
[profiles.peios]
kernel   = "/build/peios/bzImage"
initrd   = "/build/peios/initrd.cpio.gz"
cmdline  = "console=ttyS0 quiet"

[profiles.peios-debug]
kernel   = "/build/peios-debug/bzImage"
initrd   = "/build/peios-debug/initrd.cpio.gz"
cmdline  = "console=ttyS0 debug loglevel=7 nokaslr"

[profiles.peios-stable]
kernel   = "/build/peios-stable/bzImage"
initrd   = "/build/peios-stable/initrd.cpio.gz"
cmdline  = "console=ttyS0 quiet"
```

Tests pick:

```lua
provium:vm("v",        "peios")
provium:vm("debug-v",  "peios-debug")
provium:vm("stable-v", "peios-stable")
```

A test can mix profiles in the same file — useful for compatibility testing:

```lua
test("stable client can talk to mainline server", function(t)
    local lan    = provium:bridge("lan")
    local server = provium:vm("server", "peios"):boot()
    local client = provium:vm("client", "peios-stable"):boot()
    lan:attach({server, client})
    server:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
    client:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")

    -- Run a real protocol against the mismatched pair.
    server:run_async("python3", {args = {"-m", "http.server", "8080"}})
    wait_until(function()
        return server:run("ss -ltn | grep :8080"):ok()
    end, {timeout = "5s"})
    client:run("curl -s http://10.0.0.1:8080/"):assert_ok()
end)
```

## Profile choice patterns

### Default vs debug

The most common split: a fast `peios` build for the default loop, and a `peios-debug` build with `loglevel=7`, `nokaslr`, and (probably) `KASAN`/`UBSAN` enabled. Use the debug profile when reproducing a kernel bug:

```lua
test("debug build reveals the leak", {tags = {"slow", "debug"}}, function(t)
    local vm = provium:vm("v", "peios-debug"):boot()
    vm:run("…workload that leaks…")
    local r = vm:run("dmesg | grep -i 'BUG:'")
    -- KASAN / UBSAN reports show up here in the debug build.
end)
```

### Cross-version compatibility

Two profiles, one per release line. Ship a small handful of cross-version tests:

```lua
test("rolling upgrade: stable client + mainline server", function(t)
    -- … as above …
end)

test("rolling upgrade: mainline client + stable server", function(t)
    -- … inverse …
end)
```

Tag these `cross-version` and gate via `--no-tag cross-version` in the dev loop. CI runs them on a nightly cadence with `--include-slow`.

### Feature-flag gating

Two profiles built from the same source with one feature toggled:

```toml
[profiles.peios-prefeatx]
kernel  = "/build/peios-prefeatx/bzImage"   # FEATX disabled
# …

[profiles.peios-postfeatx]
kernel  = "/build/peios-postfeatx/bzImage"  # FEATX enabled
# …
```

Tests that exercise FEATX-specific behaviour pick `peios-postfeatx`; regression tests run against both.

```lua
test("featx changes API behaviour", function(t)
    local before = provium:vm("v1", "peios-prefeatx"):boot()
    local after  = provium:vm("v2", "peios-postfeatx"):boot()

    local r1 = before:run("…")
    local r2 = after:run("…")
    t:assert_neq(r1.stdout, r2.stdout)
end)
```

## Per-VM cmdline overrides

The profile's `cmdline` is the default; `kernel_cmdline` in `vm:boot(opts)` overrides it for one VM:

```lua
local vm = provium:vm("v", "peios")
vm:boot({kernel_cmdline = "console=ttyS0 quiet maxcpus=1 isolcpus=0"})
```

This is useful for one-off testing of cmdline-sensitive features. For cmdline configurations you use repeatedly, declare a dedicated profile.

## Default profile selection

A few CLI subcommands take an optional profile arg — when omitted, they fall back to "the first profile in `provium.toml` sorted by name":

| Subcommand | Behaviour |
|---|---|
| `provium repl --fixture <path>` (no profile) | First profile by sorted name. |
| `provium fixture build <path>` | Uses the first profile's kernel/initrd as the cache-key kernel inputs. |

> [!WARNING]
> With more than one profile configured, `provium fixture build` folds only that first profile into the key, while the test runner folds every profile — so a pre-warmed entry may not be the one the runner looks up. Pre-warming is reliable on single-profile configs.

To make the default predictable, name your most common profile so it sorts first alphabetically. `aaa-default` is ugly but it works; `peios` is fine when it's your only profile.

## What happens to the cache when profiles change

The fixture cache key folds in the kernel and initrd identifiers of **every profile** in `provium.toml`, sorted by name for determinism (the full key model is on [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md#what-ends-up-in-the-cache-key)). Practical consequences for profile changes specifically:

| Action | Cache effect |
|---|---|
| Edit a kernel image (any profile) | Every fixture invalidates. |
| Edit an initrd image (any profile) | Every fixture invalidates. |
| Add a new profile | Every fixture invalidates (the new profile's kernel/initrd are folded in). |
| Remove a profile | Every fixture invalidates. |
| Rename a profile | Usually invalidates everything (the fold order follows profile names). |
| Edit `cmdline` on a profile | Cache is unaffected (cmdline isn't in the key). |
| Change `cache_dir` | Cache is unaffected — the new dir is just empty. |

This is deliberately conservative. A new profile means a new kernel could behave differently, so the existing fixtures might be subtly stale. Rather than guess, the harness rebuilds.

If you need to add a profile without invalidating the cache, you can't. Either accept the rebuild or use a sibling `provium.toml` with `--config alt-config.toml`.

## Multi-arch profiles (preview)

In v1, the QemuVmm backend invokes `qemu-system-x86_64` exclusively — there's no per-profile arch field yet. To run on aarch64, you'd have to swap the binary at the harness level (not currently exposed).

When multi-arch lands, the profile is the natural place for an `arch = "x86_64"` / `arch = "aarch64"` field.

## See also

- [provium.toml reference](/provium/configuration/provium-toml.md) — every field.
- [VMs and profiles](/provium/writing-tests/vms-and-profiles.md) — `provium:vm(name, profile, opts?)`.
- [Fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) — what triggers a fixture rebuild.

---

# Dynamic profiles

_Provium / Configuration_

> Make a profile build its own image before the VM boots — the build command, the {out} token, cmdline from the builder, and prepare / --no-build.

A static profile points at artifacts that already exist on disk:

```toml
[profiles.peios]
kernel  = "/build/peios/bzImage"
initrd  = "/build/peios/initrd.cpio.gz"
cmdline = "console=ttyS0 quiet"
```

That leaves a gap. You build the image with one tool, then run Provium with another, and nothing connects the two — so it's easy to test last week's image because you forgot to rebuild. A *dynamic* profile closes the gap: the profile owns a `build` command, Provium runs it, and the VM boots the fresh result.

This is the same idea as [`cmdline_file`](/provium/configuration/provium-toml.md#cmdline-file) — let the test consume the authoritative build instead of a hand-copied snapshot — applied to the whole image. This page covers both.

## Cmdline from the builder

Start with the smaller case. Image builders usually emit a kernel command line of their own — `peiso`, for instance, writes one to `out/root/boot/cmdline`. If you copy that string into your profile's `cmdline`, the two drift the moment the builder changes it (a new `init=`, a new console). Point `cmdline_file` at the builder's file instead:

```toml
[profiles.peios]
kernel       = "../peiso/out/root/usr/lib/modules/<release>/vmlinuz-<release>"
initrd       = "../peiso/out/initrd.img"
cmdline_file = "../peiso/out/root/boot/cmdline"
```

Provium reads the file at boot, collapses its whitespace to single spaces, and uses it as the command line. You can still add tokens inline — they're appended *after* the file, so they win for last-wins kernel parameters:

```toml
cmdline_file = "../peiso/out/root/boot/cmdline"
cmdline      = "loglevel=7"     # verbose, on top of whatever the builder set
```

No manual copy means nothing to forget to update.

## Building the whole image

The `build` field takes a shell command. Provium runs it — once, before any VM boots — and only then reads the profile's `kernel` / `initrd` / `cmdline_file`:

```toml
[profiles.peios-full]
build        = "peiso build manifests/peios-full.toml --out {out}"
kernel       = "{out}/root/usr/lib/modules/<release>/vmlinuz-<release>"
initrd       = "{out}/initrd.img"
cmdline_file = "{out}/root/boot/cmdline"
```

Now `provium` builds `peios-full` before the suite runs, and tests boot exactly what was just built:

```lua
local vm = provium:vm("v", "peios-full"):boot()
```

The command runs with `sh -c`, from Provium's working directory, inheriting your environment and streaming its output straight to the terminal. Reference inputs (the manifest above) relative to the suite so the suite stays portable across checkouts — there is deliberately no "build working directory" knob to tie it to one machine's layout.

## `{out}`: one directory, no drift

Look again at the example: `{out}` appears in the `build` command's `--out` *and* in every path field. That's the point. `{out}` expands — once, when the config loads — to this profile's build-output directory, so the place the build *writes* and the places Provium *reads* are guaranteed to be the same directory. You can't update one and forget the other, because there's only one.

Where does `{out}` point? By default, a per-profile directory under the Provium cache:

```
$XDG_CACHE_HOME/provium/builds/<profile>/     # e.g. ~/.cache/provium/builds/peios-full/
```

Set [`build_out`](/provium/configuration/provium-toml.md#build-out) to pin it somewhere specific — but you still only write the path once:

```toml
[profiles.peios-full]
build_out    = "/tmp/provium-builds/peios-full"
build        = "peiso build manifests/peios-full.toml --out {out}"
kernel       = "{out}/root/usr/lib/modules/<release>/vmlinuz-<release>"
# …
```

Provium creates the directory before running the build, but never wipes it — the build command owns its contents, so it's free to keep its own incremental-build caches there.

## When the build runs

| You run | What gets built |
|---|---|
| `provium` (the test suite) | Every profile that declares a `build`, up front, before the scheduler starts. |
| `provium console <profile>` | Just that profile. |
| `provium repl <profile>` | Just that profile. |
| `provium prepare [profile]` | That profile, or — with no argument — every profile with a `build`. No VM boots. |

The test runner builds *all* dynamic profiles rather than only the ones a run will touch, because tests choose their profiles at runtime from Lua (`provium:vm(name, profile)`) — Provium can't know in advance which a given run needs. If you keep several dynamic profiles and want to build only one, use `provium prepare <profile>` followed by `provium --no-build`.

Two controls shape this:

- **`provium prepare`** builds without booting. It even skips the pre-flight checks (`/dev/kvm`, networking, `CAP_NET_ADMIN`), so you can build an image on a machine that isn't set up to run VMs.
- **`provium --no-build`** skips the hook entirely for one run — for when you know the artifacts are already current and just want to boot.

## No staleness — that's the builder's job

Provium does not try to decide whether your image is up to date. It runs the `build` command **every** invocation and trusts the builder to be cheap when nothing has changed. Tracking inputs, hashing sources, and skipping unnecessary work is what build tools are for; reimplementing that inside a test harness would only get it subtly wrong.

The practical consequence: if your builder does a full rebuild every time, `provium` pays that cost every time. That's usually fine — Provium tends to be the *last* gate you run, where a clean build is what you want anyway — but if it bites, make the builder incremental (or use `--no-build` between source changes). Don't expect Provium to shortcut it.

## Fixtures rebuild when the image changes

Provium's [fixture cache](/provium/running-tests/fixtures-and-dependencies.md) keys each snapshot partly on the kernel and initrd's path, size, and modification time. A dynamic profile rebuilds its image every run, which usually changes those timestamps — so fixtures captured against the old image are treated as stale and rebuilt. That's correct (a freshly built kernel can behave differently), but it does mean a dynamic profile plus a large fixture tree pays a fixture rebuild each run on top of the image build. If that's too slow for the inner loop, `--no-build` keeps the image — and therefore its fixtures — warm between source changes.

## Failures abort — never fall through

A `build` command that exits non-zero stops the run:

```
provium: building profile `peios-full` → /home/you/.cache/provium/builds/peios-full
… builder output …
provium: profile `peios-full`: build command failed (exit status: 1)
```

This matters because a half-finished build often leaves the *previous* run's artifacts in place — they'd pass an existence check and boot cleanly, and you'd be testing a stale image while believing the build succeeded. Provium refuses to do that: a failed build is a failed run (exit code `2`), full stop.

## Worked example

A self-contained suite that builds its own image. The manifest lives in the suite, so a fresh clone can `provium` with nothing else set up:

```toml
# test-suite/provium.toml
[provium]
roots = ["tests"]

[profiles.peios-full]
build        = "peiso build manifests/peios-full.toml --out {out}"
kernel       = "{out}/root/usr/lib/modules/<release>/vmlinuz-<release>"
initrd       = "{out}/initrd.img"
cmdline_file = "{out}/root/boot/cmdline"
```

```
test-suite/
├── provium.toml
├── manifests/
│   └── peios-full.toml        # the image recipe, versioned with the tests
└── tests/
    └── smoke.test.lua
```

Day-to-day:

```bash
provium                       # build the image, then run the whole suite
provium --no-build            # skip the build; boot the last-built image
provium prepare               # build the image, don't run anything
provium console peios-full    # build, then drop to the guest's console
```

## See also

- [provium.toml reference](/provium/configuration/provium-toml.md) — the `build`, `build_out`, and `cmdline_file` fields.
- [Profiles](/provium/configuration/profiles.md) — patterns for static and multi-profile setups.
- [CLI reference](/provium/reference/cli.md) — `provium prepare` and `--no-build`.

---

# provium global

_Provium / Reference_

> The `provium` global is the entry point to every Provium API. It is itself a Lab, so every method documented on Lab is reachable directly off provium.

The `provium` global is installed onto every test file's Lua state. It is the root [Lab](/provium/reference/lab.md) and the only API surface a test author needs to import — there is no `require`.

```lua
provium:vm("a", "peios")              -- create a VM in the root lab
provium:bridge("lan")                 -- create a bridge in the root lab
provium:lab("subset")                 -- create a sub-lab
provium:vm_fixture("base")            -- restore a fixture into a VM
provium:lab_fixture("ha-pair")        -- restore a fixture into a sub-lab
provium:claim({memory="4G", cpus=2})  -- file-level resource reservation
provium:boot()                        -- batch-boot every VM in the lab
provium:shutdown()                    -- batch-shutdown every VM in the lab
provium:snapshot()                    -- whole-lab snapshot
provium:restore(snap)                 -- whole-lab restore
provium:pack(...) / provium:unpack(...)  -- Lua 5.4 string.pack/unpack
```

See [Lab reference](/provium/reference/lab.md) for the canonical enumeration of these methods. This page documents only the things that are unique to the top-level `provium` global.

## File-scope configuration globals

Two top-level Lua globals on the file's `provium` table change harness behaviour for every test in the file:

### `provium.reset_between_tests`

```lua
provium.reset_between_tests = true

test("a", function(t) provium:vm("v", "peios"):boot() end)
test("b", function(t) provium:vm("v", "peios"):boot() end)  -- v was reset
```

When set to `true` at file scope, Provium takes a baseline snapshot of the root lab right after the file's top-level chunk finishes (and before the first test runs). After every test's body returns, the lab is restored to that baseline. Each `test()` runs against an identical starting state.

Mutually exclusive with file-scope open streams. If the top-level chunk opens a stream (`vm:tail_file`, `vm:console():read()`, `bridge:capture()`, etc.) and `reset_between_tests = true`, the file errors at chunk-load time with an explicit "would auto-snapshot a live stream" message naming the stream type and creation site.

### `provium.timeout`

```lua
provium.timeout = "30s"

test("a", function(t) … end)         -- per-test deadline = 30s
test("b", {timeout = "5m"}, function(t) … end)  -- per-test wins; 5m
```

File-default per-test timeout. Per-test `meta.timeout` wins. Accepts numeric seconds or a duration string with `ms`/`s`/`m`/`h` suffix. When a per-test timeout fires, the watchdog tears the entire root lab down (see [time and timeouts](/provium/reference/test-framework.md#time-and-timeouts) for the scope-limitation note).

## Dot-sugar lookup

`provium.<name>` is shorthand for whichever named resource matches `<name>` in the root lab. The lookup order is:

| Reserved keys | Resolves to |
|---|---|
| `provium.pack`, `provium.unpack` | Lua's standard `string.pack` / `string.unpack`. Reserved against shadowing. |
| `provium.vm_fixture`, `provium.lab_fixture` | Functions equivalent to the `:vm_fixture(...)` / `:lab_fixture(...)` methods. |

After reserved keys, the lookup tries:

1. A VM declared by `provium:vm("<name>", ...)` — returns the VM userdata.
2. A bridge declared by `provium:bridge("<name>")` — returns the Bridge userdata.
3. A sub-lab declared by `provium:lab("<name>")` — returns the LabUd userdata.
4. `nil` if nothing matches.

Examples:

```lua
local lan = provium:bridge("lan")
local a   = provium:vm("a", "peios")
local b   = provium:vm("b", "peios")

-- Later in the test:
provium.lan:attach({provium.a, provium.b})
provium.a:boot()
if provium.dc1 then provium.dc1:boot() end
```

The dot form is a strict lookup. It does not create new resources; calling `provium.unknown` returns `nil`. Use the method form (`provium:vm("name", "profile")`) to create.

## Fixture sugar

`provium.vm_fixture` and `provium.lab_fixture` are reserved keys that resolve to the same callable surface as the `:vm_fixture(...)` / `:lab_fixture(...)` methods. Both call shapes work:

```lua
local vm  = provium:vm_fixture("base")     -- method form
local vm2 = provium.vm_fixture("base")     -- function form (same effect)
```

Use whichever reads better in context. The function form is convenient inside a `wait_until` predicate or a higher-order helper that captures the function rather than the lab.

## Binary helpers (`pack` / `unpack`)

`provium:pack(fmt, ...)` and `provium:unpack(fmt, s)` are pass-throughs to Lua 5.4's `string.pack` / `string.unpack`. They exist on the `provium` global so test code that uses binary protocols doesn't have to reach into `string.*`:

```lua
local frame = provium:pack(">I4 I4 s2", id, seq, payload)
local id, seq, payload = provium:unpack(">I4 I4 s2", frame)
```

The full Lua 5.4 format-string grammar applies. See the [Lua reference manual on string.pack](https://www.lua.org/manual/5.4/manual.html#6.4.2) for the format specifiers.

## See also

- [Lab reference](/provium/reference/lab.md) — every method `provium` carries through inheritance.
- [VM reference](/provium/reference/vm.md) — what `provium:vm(...)` returns.
- [test framework reference](/provium/reference/test-framework.md) — `test()`, `t`, `wait_until`, `todo`.

---

# json

_Provium / Reference_

> The `json` global — encode and decode JSON between Lua values and strings, backed by serde_json.

`json` is a top-level Lua global. Two methods, both pure host-side functions (no VM round-trip).

## `json.encode(value)`

Serialise a Lua value to a JSON string. Tables, numbers, booleans, strings, and `nil` round-trip naturally.

```lua
json.encode({a = 1, b = "x"})        -- '{"a":1,"b":"x"}'
json.encode({10, 20, 30})            -- '[10,20,30]'
json.encode(true)                    -- 'true'
json.encode(nil)                     -- 'null'
json.encode({a = 1, b = nil})        -- '{"a":1}'  (b isn't in the table — Lua semantics)
json.encode({[1]=1, [3]=3})          -- '[1,null,3]'  (array padded to max int key)
```

Array vs object detection: a table whose keys are positive integers (`1..=N`, possibly with gaps) encodes as a JSON array of length `max(key)`; gaps emit `null`. Anything else encodes as an object. Mixed tables (`{1, 2, name = "x"}`) fall into the object case, with integer keys stringified.

Encoding errors (cycles, function/userdata values, non-string-coercible keys) raise a Lua error.

## `json.decode(string)`

Parse a JSON string into a Lua value.

```lua
local t = json.decode('{"x": 42}')
print(t.x)                           -- 42

local t = json.decode('{"a": {"b": ["c", "d"]}}')
print(t.a.b[2])                      -- "d"
```

### Number precision

JSON integers in `[i64::MIN, i64::MAX]` decode to Lua integers exactly — full 64-bit precision, no rounding through `f64`. Integers above `i64::MAX` (i.e. u64-only values, up to `0xFFFFFFFFFFFFFFFF`) preserve their bit pattern via a wrap-cast: `0xFFFFFFFFFFFFFFFF` decodes to `-1`, exactly matching Lua 5.4's own `tonumber("0xFFFFFFFFFFFFFFFF")`. Bitwise ops still work correctly on the result (the bits are the bits).

Non-integer JSON (`3.14`, `1e10`) decodes to Lua's float type as expected.

You can drop the "emit as hex string and `tonumber()` it" workaround that older code used — large unsigned values now round-trip through `json.decode` directly.

### Null handling

Lua tables can't hold `nil` as a value — assigning `nil` to a key removes it. We follow the standard Lua JSON-library convention (dkjson, lua-cjson):

| Input | Lua result | Notes |
|---|---|---|
| `null` | `nil` | Top-level scalar. |
| `{"k": null}` | A table with no `k` key. | `t.k == nil`, `next(t)` doesn't yield `k`. |
| `[1, null, 3]` | `{1, nil, 3}` — array-with-hole. | `t[2] == nil`; `#t` is implementation-defined per Lua. |

This is lossy versus the original `"key exists but is null"`: round-tripping `decode` → `encode` drops null fields. Tests that need to distinguish `null` from absent should keep the source string and re-check.

Decoding errors raise a Lua error with the parse position.

## Typical use

Reading config the guest emitted:

```lua
local body = vm:read_file("/var/state/peios/state.json")
local state = json.decode(body)
t:assert_eq(state.ready, true)
```

Building a request body for a test client:

```lua
local body = json.encode({op = "write", key = "k", value = "v"})
vm:run("curl", {"-X", "POST", "-d", body, "http://localhost:8080/api"}):assert_ok()
```

## Performance

`encode` and `decode` are pure-host serde_json calls — microseconds for typical config-sized inputs. No VM round-trip, no agent involvement. Safe to call from inside tight test loops.

---

# VM

_Provium / Reference_

> The VM userdata — lifecycle, run and run_async, file I/O, syscalls and ioctls, disks, NICs, console, clock, workers, batch, snapshots, and RunResult.

A VM userdata wraps one guest. You get one from `lab:vm("name", "profile")` (or the dot-sugar `lab.<name>` after creation), and every operation against the guest dispatches through it.

VM handles are cheap to clone — passing them around or storing them in tables doesn't duplicate state.

## Constructing

| Source | Returns |
|---|---|
| `lab:vm("name", "profile")` | New VM in `lab`, in the `Created` state. |
| `lab:vm("name", "profile", opts)` | Same, with sizing opts (`memory`, `cpus`). See [boot opts](#boot-opts). |
| `lab:vm("name")` | Lookup the VM previously declared with that name. Errors if absent. |
| `lab.name` | Same lookup as `lab:vm("name")`, returns `nil` on miss. |
| `provium:vm_fixture("path")` | Restored from a cached fixture; already booted. |

## State machine

```
Created ──:boot()──> Booted ──:pause()──> Paused
                       │                    │
                       ├─:reset()→Booted    ├─:resume()→Booted
                       ├─:power_button()──> Shutdown
                       └─:shutdown()─────> Shutdown
```

`vm:state()` returns `"created"`, `"booted"`, `"paused"`, `"shutdown"`, or `"dead"`. Operations against a VM in the wrong state error cleanly with the offending state in the message.

## Boot opts

Boot opts split between the two call sites. The sizing keys go in the third arg to `lab:vm(name, profile, opts)` — the scheduler and QEMU need them when the VM is declared. The boot-shaping keys go to `vm:boot(opts)` and merge per field into the VM's pending boot options.

| Key | Where | Type | Description |
|---|---|---|---|
| `memory` | `lab:vm` opts | int (bytes) or string `"512M"`/`"2G"` | VM memory cap. Hands to QEMU as `-m <size>`. |
| `cpus` | `lab:vm` opts | int | vCPU count. Hands to QEMU as `-smp <n>`. |
| `kernel_cmdline` | `vm:boot` opts | string | Replaces the profile's `cmdline`. |
| `rng_seed` | `vm:boot` opts | int (u64) | Seeds the guest's `virtio-rng`. Use for determinism. |
| `initial_time` | `vm:boot` opts | int or float (seconds since epoch) | Sets the guest's wall clock at boot. |
| `files` | `vm:boot` opts | array of `{path=string, content=string}` | Files to inject into the guest's filesystem before init runs. |

Example:

```lua
local vm = provium:vm("v", "peios", {memory = "2G", cpus = 4})
vm:boot({
    rng_seed = 0xdeadbeef,
    initial_time = 1700000000,
    files = {
        {path = "/etc/hostname", content = "v"},
    },
})
```

## Lifecycle methods

### `vm:boot(opts?)`

Boots the guest. Returns `self` so calls chain (`provium:vm("a", "peios"):boot()`). The optional `opts` table (`kernel_cmdline`, `rng_seed`, `initial_time`, `files`) merges per field into the VM's pending boot options. Errors if already booted.

Emits a `vm_spawned` event after the guest is up and the agent has handshaken. The event payload carries `{file, vm_name, profile, memory_bytes, cid}`.

### `vm:pause()`

Pauses the guest's vCPUs. Returns nothing. Errors if not in `Booted`.

### `vm:resume()`

Resumes a paused guest. Returns nothing. Errors if not in `Paused`.

### `vm:shutdown()`

Tears down the guest. Idempotent in the sense that `Created`, `Booted`, `Paused`, and `Dead` all transition to `Shutdown`. Emits a `vm_shutdown` event with `{file, vm_name, duration_ns}`.

### `vm:reset()`

Warm reboot — guest re-runs its boot path. Errors if not in `Booted`. The post-reset state is still `Booted`.

### `vm:power_button()`

Sends ACPI power-button to the guest. Triggers a graceful shutdown sequence. Errors if not in `Booted`. Post-call state is `Shutdown`.

### `vm:close()`

Auto-close hook used by the resource walker. Idempotent. Calls `:shutdown()` if not already in `Shutdown`. Test code rarely calls this directly — the harness fires it for every VM at file end (or per-test if `provium.reset_between_tests = true`).

## Snapshot and restore

### `vm:snapshot(path?)`

Take a snapshot. With an explicit `path`, writes there; without, writes to a tempfile. Returns a [Snapshot](/provium/reference/snapshot.md) userdata wrapping the path.

```lua
local s = vm:snapshot()
local s2 = vm:snapshot("/tmp/before-mutation.snap")
```

### `vm:restore(snap_or_path)`

Restore from a `Snapshot` userdata (preferred) or a bare path string. Errors if not in `Created` or `Shutdown`.

```lua
vm:shutdown()
vm:restore(s)
```

## Layer-1 ops

Layer-1 ops are the high-level "do it like SSH" primitives. They dispatch through the agent's exec / file / stat handlers.

### `vm:run(cmd_or_args, opts?)`

Run a command. Two forms:

- `vm:run("echo hi")` — runs through `/bin/sh -c "echo hi"`. Shell metacharacters work.
- `vm:run("echo", {"hi"})` — direct exec. No shell. The second arg is a positional-arg array.
- `vm:run("echo", {args={"hi"}, env={K="v"}, cwd="/tmp", stdin="…", env_clear=true, timeout="5s"})` — opts form. Auto-detected by the presence of a recognised opts key (`env`, `env_clear`, `cwd`, `stdin`, `timeout`, `timeout_ms`, `args`); a table with none of them is treated as the legacy array. Mixed array entries with `args=` are not allowed; use one form or the other.

Returns a [RunResult](#runresult) userdata.

| Opts key | Type | Description |
|---|---|---|
| `args` | array of strings | Direct positional args (only used with the table-as-opts form). |
| `env` | string→string map | Environment variables. |
| `env_clear` | bool | When true, the guest sees only `env`; otherwise env merges with the agent's environment. |
| `cwd` | string | Working directory inside the guest. |
| `stdin` | string | Bytes piped to the process's stdin. |
| `timeout_ms` | int | Hard wall-clock timeout in milliseconds. |
| `timeout` | int / float / string | Same, expressed as seconds or `"5s"` / `"500ms"` / `"2m"` / `"1h"`. Negative or NaN errors. |

### `vm:run_async(cmd, opts?)`

Like `:run`, but returns a [Process](/provium/reference/process.md) userdata immediately. The agent does not auto-kill; you control the lifetime via `proc:kill()` / `proc:wait()`. Passing `timeout` here is rejected — use `proc:wait(timeout)` instead.

### `vm:read_file(path)`

Returns the file contents as a Lua string. Errors on agent-side `read` failure (ENOENT, EACCES, etc.).

### `vm:write_file(path, data)`

Replaces the file's contents with `data`. Creates the file if absent. Returns nothing.

### `vm:push_file(host_path, guest_path, opts?)`

Read a file from the host filesystem and write its bytes to `guest_path` in the guest. Inside a fixture, the host file is folded into the fixture's cache key automatically — rebuilding the host file (typical case: a binary under test) invalidates the snapshot. Relative `host_path` is resolved against the directory of the file containing the call. Pass `{auto_dep = false}` to skip the auto-fold for a single call. See [files and handles](/provium/writing-tests/files-and-handles.md#vmpush-filehost-path-guest-path-opts) and [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md#external-host-file-deps).

### `vm:stat(path)`

Returns a table:

| Field | Type | Description |
|---|---|---|
| `size` | int (bytes) | File size. |
| `mtime` | float (seconds since epoch) | Modification time. |
| `mtime_ns` | int (ns since epoch) | Same, full precision. |
| `perm` | int | POSIX mode bits (≤ 4095, i.e. `0o7777`). |
| `entry_type` | string | One of `"file"`, `"directory"`, `"symlink"`, `"fifo"`, `"socket"`, `"block_device"`, `"char_device"`, `"other"`. |

### `vm:listdir(path)`

Returns an array of `{name=string, entry_type=string}` tables.

### `vm:mkdir(path, opts?)`

Create a directory. Opts: `{parents=bool, perm=int}`. With `parents=true`, intermediate directories are created (`mkdir -p`). `perm` is the POSIX mode for the new directory.

### `vm:unlink(path)`

Remove a file or empty directory.

### `vm:rename(from, to)`

Atomic rename within the guest filesystem.

### `vm:open_file(path, mode_table)`

Returns a [File](/provium/reference/file-handle.md) userdata.

The mode table accepts:

| Key | Type | Effect |
|---|---|---|
| `read` | bool | Open for reading. |
| `write` | bool | Open for writing. |
| `create` | bool | Create if absent. |
| `truncate` | bool | Truncate to zero bytes on open. |
| `append` | bool | Append-only writes. |
| `exclusive` | bool | Combine with `create=true` to require the file not already exist (`O_EXCL`). |
| `perm` | int | POSIX mode for newly-created files. |

At least one of `read`, `write`, or `append` must be true; an empty mode table errors at open time with a pointer.

### `vm:tail_file(path, opts?)`

Open a streaming subscription to a file. Returns a [Tail](/provium/reference/streams.md) userdata. Opts: `{start = "beginning" | "end" | <offset>}` (`"start"` is accepted as an alias for `"beginning"`). Default is `"end"` — only bytes appended after the call are streamed. `<offset>` may be a non-negative integer (absolute byte offset), a negative integer (N bytes before EOF — provium stats and resolves), or a finite float (truncated toward zero, same sign semantics).

### `vm:fd_stream(fd_or_file)`

Open a streaming subscription to an existing file handle. Accepts either an integer handle id (from `file:fd()`) or a `File` userdata directly. Returns a [Tail](/provium/reference/streams.md).

## Layer-0 ops

Layer-0 ops are direct kernel-level primitives — `syscall(2)` and `ioctl(2)`. Use these to exercise driver paths or to test syscall semantics directly.

### `vm:syscall(nr, ...)`

Two call shapes:

- **Integer-only:** `vm:syscall(nr, a1, a2, a3, a4, a5, a6)` — up to 6 integer arguments.
- **Table form:** `vm:syscall(nr, {args={a1, a2, …}, bufs={"…", …}, ptrs={1, 3}, nested={…}})`. Buffers are byte arrays; `ptrs[i]` says which arg slot (0-indexed) `bufs[i]` should be spliced into. The buffer's address is filled in for the kernel.

**Nested pointers.** `nested` is a list of `{parent=N, child=M, offset=K}` entries (`parent`/`child` are **1-based** indices into `bufs`). Before the arg-slot `ptrs` are applied, the agent writes `bufs[M]`'s address into `bufs[N]` at byte `offset` (an 8-byte native-endian pointer). This is how you pass a **struct argument that itself contains a pointer** — a buffer pointing at another buffer. Use it for ioctls invoked via raw `syscall(SYS_ioctl, fd, cmd, &struct)` whose struct has an inner output-buffer pointer (a query/info ioctl), or for syscalls whose argument struct embeds pointers (`kacs_access_check`-style). Each pointed-at buffer is returned post-call in `out_bufs` at its own index.

```lua
-- ioctl(fd, QUERY, &args) where struct kacs_query_args { u32 class; u32 buf_len; u64 buf_ptr; }
-- buf_ptr (offset 8) must point at the output buffer.
local args = string.pack("<I4I4I8", PRIVS_CLASS, 4096, 0)  -- buf_ptr filled in by nested
local out  = string.rep("\0", 4096)
local r = vm:syscall(SYS_ioctl, {
    args   = { fd, QUERY_CMD, 0 },     -- arg slot 2 receives the struct pointer
    bufs   = { args, out },            -- buf 1 = struct, buf 2 = output
    ptrs   = { 2 },                    -- buf 1 → arg slot 2
    nested = { { parent = 1, child = 2, offset = 8 } },  -- buf 2's addr → buf 1 @ offset 8
})
local data = r.out_bufs[2]             -- the QUERY output buffer
```

Returns a table:

| Field | Type | Description |
|---|---|---|
| `ret` | int | Syscall return value. |
| `result` | int | Same as `ret` (alias for clarity). |
| `errno` | int | Errno if the syscall returned negative; 0 otherwise. |
| `out_bufs` | array of strings | Post-syscall contents of the supplied `bufs` (including nested-pointer target buffers). Useful for read-style syscalls. |

The same `bufs` / `ptrs` / `nested` options are available on [`worker:syscall`](/provium/reference/worker.md#workersyscallnr-).

### `vm:ioctl(fd, cmd, data?, opts?)`

Direct `ioctl(2)`. `fd` and `cmd` are integers. `data` is an optional byte string passed as the third arg's pointer. `opts.bufs` and `opts.ptr_offsets` work like `:syscall`'s `bufs` / `ptrs`.

Returns `{ret, result, out_data, out_bufs}`. `out_data` is the post-call value of `data` (for ioctls that write back through the same buffer).

## Hypervisor resources

### `vm:nic(name)`

Returns a [Nic](/provium/reference/nic.md) userdata for the named bridge attachment, or for a guest-style interface name (`eth0`, `enp0s3`, `ens3`). Guest-name lookup maps to the Nth attached bridge sorted by bridge name.

### `vm:disk(id)`

Returns a [Disk](/provium/reference/disk.md) userdata for an already-attached disk. Errors if no disk with that id.

### `vm:attach_disk(opts?)`

Attaches a new disk and returns a [Disk](/provium/reference/disk.md) userdata. Opts: `{id=string, size=int, image=string}`. Defaults: `id` = `attached-<vm_name>`, `size` = 4 GiB, `image` = no backing file.

## Console and clock

### `vm:console()`

Returns a [Console](/provium/reference/console.md) userdata.

### `vm:clock()`

Returns a [Clock](/provium/reference/clock.md) userdata.

## Workers

### `vm:spawn_worker(opts?)`

Spawns a sub-agent connection so test code can dispatch concurrent ops against the same guest. Returns a [Worker](/provium/reference/worker.md) userdata. Opts are reserved for future use; passing `{thread=true}` errors with a future-work pointer.

## Batch operations

### `vm:batch(fn)`

Collect multiple ops inside `fn(b)` and dispatch them as one wire round-trip. `b` exposes:

- `b:run(...)` (with the same shape as `vm:run`)
- `b:read_file(path)`
- `b:write_file(path, data)`
- `b:stat(path)`
- `b:listdir(path)`
- `b:mkdir(path, opts)`
- `b:unlink(path)`
- `b:rename(from, to)`
- `b:syscall(nr, a, b, …)` — integer-args form only; the table form is rejected here.

Returns a Lua array of `{ok=value}` or `{err=msg}` entries, one per op. A failure on op N does not short-circuit the rest of the batch.

```lua
local results = vm:batch(function(b)
    b:write_file("/tmp/a", "1")
    b:write_file("/tmp/b", "2")
    b:read_file("/tmp/a")
    b:stat("/tmp/missing")
end)
-- results = {{ok=nil}, {ok=nil}, {ok="1"}, {err="No such file or directory"}}
```

## Accessors

### `vm:name()` — Returns the VM's name as given to `lab:vm`.
### `vm:profile()` — Returns the profile name.
### `vm:state()` — Returns `"created"`, `"booted"`, `"paused"`, `"shutdown"`, or `"dead"`.
### `vm:cid()` — Returns the assigned vsock CID, or `nil` before boot.
### `vm:is_quiescent()` — `true` if no in-flight ops, no open files or streams.
### `vm:open_file_count()` — Number of files currently open through the agent.
### `vm:open_stream_count()` — Number of streams currently active (tails, fd-streams, console-reads, captures).

## RunResult

The return value of `vm:run(...)` (and `worker:run(...)`, `proc:wait()`).

Fields:

| Field | Type | Description |
|---|---|---|
| `exit_code` | int | Exit code. `-1` for signalled, `-2` for timed_out. Use `signal` and `timed_out` to disambiguate. |
| `stdout` | string | Captured stdout. |
| `stderr` | string | Captured stderr. |
| `status` | string | `"exited"`, `"signalled"`, `"timed_out"`. |
| `timed_out` | bool | `true` if the timeout fired. |
| `signal` | int or nil | Signal number when status is `signalled`, else nil. |

Methods:

- `result:ok()` — convenience for `result.exit_code == 0`.
- `result:assert_ok()` — raises if not OK; the error message includes status, stdout, and stderr (with signal name when signalled).

## See also

- [Lab](/provium/reference/lab.md) — what creates and contains VMs.
- [Process](/provium/reference/process.md) — what `vm:run_async` returns.
- [Worker](/provium/reference/worker.md) — what `vm:spawn_worker` returns.
- [Disk](/provium/reference/disk.md), [Nic](/provium/reference/nic.md) — hypervisor-side handles.
- [Console](/provium/reference/console.md), [Clock](/provium/reference/clock.md) — guest-side accessors.
- [Streams](/provium/reference/streams.md) — what `vm:tail_file` and `vm:fd_stream` return.

---

# Bridge

_Provium / Reference_

> The Bridge userdata — wire VMs together, partition, inject latency and drops, limit bandwidth, capture packets, isolate VMs, and NAT outbound traffic.

A Bridge represents one Linux bridge with TAP-interface attachments. It is created via `lab:bridge("name")` and lazily realised — the host-side bridge and per-VM TAPs come up the first time any attached VM boots.

## Constructing

| Source | Returns |
|---|---|
| `lab:bridge("name")` | New bridge. |
| `lab:bridge("name", opts)` | Same. Opts are reserved for future use. |
| `lab:bridge("name")` (already created) | The same bridge. |
| `lab.<name>` | Same lookup as `lab:bridge("name")` for declared bridges. Returns nil otherwise. |

A bridge is a graph object until a VM attached to it boots; only at that point does the host-side bridge interface exist.

## Membership

### `bridge:attach(vm)` / `bridge:attach({vms…})`

Records that `vm` is on this bridge. When `vm` is booted the host installs a TAP, slaves it to the bridge, and adds a `-netdev tap` flag to the QEMU launch.

Accepts:

- A single VM userdata — `lan:attach(a)`.
- A bare string — `lan:attach("a")`. Graph-state only; no VM handle is wired, so future link-down via `bridge:detach(vm)` will not drive QMP.
- An array — `lan:attach({a, b, c})`. Validated atomically: a bad type at index N fails before any element is recorded.

### `bridge:detach(vm)`

Remove `vm` from the bridge. When `vm` is a VM userdata, also issues a QMP `set_link(false)` against the matching netdev so the guest sees a link-down event.

### `bridge:members()`

Returns an array of attached VM names.

## Partitions

Partitions are network-layer drops between specific pairs of VMs. Two forms:

### `bridge:partition(a, b)` (symmetric)

Drop both directions of A↔B traffic. Accepts VM userdata or bare strings. Graph-state only — drops install when each VM is booted.

### `bridge:partition({from=A, to=B})` (directional)

Drop only A→B traffic. B→A continues to flow. Both endpoints must already be attached and booted; otherwise it errors with a "call bridge:attach first" pointer (the rule would otherwise install onto a non-existent TAP).

### `bridge:unpartition(a, b)` / `bridge:unpartition({from=A, to=B})`

Undo the matching partition. The directional unpartition does NOT require the endpoints to still be attached — if you `:detach()` a VM and then `:unpartition` it, the call is a clean no-op rather than an error (detach already cleared the directional rule).

### `bridge:partition_all()`

Drop every pair of attached VMs. Equivalent to calling `bridge:partition` for every pair.

### `bridge:restore_all()`

Undo every partition on the bridge.

### `bridge:is_partitioned(a, b)`

Returns `true` if A↔B is currently partitioned (either symmetric or directional from A to B).

## Impairments

Impairments install `tc` qdiscs on the bridge or per-TAP. All three accept either a scalar (whole-bridge) or a directional table.

### `bridge:add_latency(ms_or_table)`

Whole-bridge form: `bridge:add_latency(50)` adds 50 ms of one-way delay to every packet through this bridge.

Directional form: `bridge:add_latency({from=A, to=B, ms=50})` records 50 ms for the A→B pair. Realisation is per **source**: each VM's outbound `(from, *)` pairs collapse — worst case per axis across latency, drop, and bandwidth — into one qdisc chain on that VM's TAP, so the delay in practice applies to everything leaving `A`. Endpoint-attachment check applies (see Partitions).

### `bridge:drop_rate(pct_or_table)`

Whole-bridge: `bridge:drop_rate(10)` drops ~10 % of packets via netem.

Directional: `bridge:drop_rate({from=A, to=B, p=10})`. Same per-source realisation as directional latency.

### `bridge:bandwidth_limit(bps_or_table)`

Whole-bridge: `bridge:bandwidth_limit(1_000_000)` caps the bridge to 1 Mbit/s via `tc tbf`.

Directional: `bridge:bandwidth_limit({from=A, to=B, bps=500_000})` installs an HTB-root + one rate-limited class on `A`'s source TAP, with a `netem` child when latency/drop is also set for the same source. Bits per second, not bytes. The `to` is graph-recorded but the realisation shapes every packet leaving the source TAP — HTB at the bridge layer can't select by destination MAC. Multiple `(from=A, to=*)` pairs collapse to `max(bps)` on `A`'s TAP so no pair is over-shaped.

Whole-bridge bandwidth does **not** combine with whole-bridge latency/drop on the same bridge: when netem is also configured, the latency/drop shaping is what's realised and the bandwidth cap is recorded in the graph but not enforced (combined netem+tbf needs class-based HTB — out of scope for v1). For combined shaping use the directional form on each source.

### `bridge:add_directional_latency({from=A, to=B, ms=N})`

Explicit directional form of `:add_latency`. Same effect as the table form above.

### `bridge:directional_drop_rate({from=A, to=B, p=N})`

Explicit directional form of `:drop_rate`.

### `bridge:reset()`

Tear down every installed netem / tbf qdisc and clear every partition. Isolation, uplink, and L3 routes are preserved — they're topology, not impairments.

### Introspection

- `bridge:latency_ms()` — Current whole-bridge latency.
- `bridge:drop_rate_pct()` — Current whole-bridge drop rate.
- `bridge:bandwidth_bps()` — Current whole-bridge bandwidth cap.

## Isolation

Isolation puts a single VM behind a hairpin filter so it cannot reach any other VM on the bridge (but the bridge itself remains alive).

### `bridge:isolate(vm)` / `bridge:unisolate(vm)`

Toggle isolation for one VM. Accepts a VM userdata or a bare string.

### `bridge:is_isolated(vm)`

Returns `true` if `vm` is currently isolated on this bridge.

## L3 routing (preview)

### `bridge:route(other_bridge_or_list)`

Records that traffic should be routed to `other_bridge`. **Graph-state only in v1.** No nft forward rules are installed — the call records the intent but cross-bridge IP traffic does not actually flow yet. The first call to `:route()` per bridge prints a one-shot warning to stderr so test authors notice at the call site rather than chasing dropped packets.

### `bridge:routes()`

Returns the array of bridge names this bridge is currently routed to.

## Uplink

Uplink installs an nft NAT masquerade rule between the bridge and the host's default-route interface, so VMs on the bridge can reach the outside world.

### `bridge:enable_uplink()` / `bridge:disable_uplink()`

Enable / disable NAT for this bridge. Both can fail if the host doesn't have a default-route interface or if nft commands fail; the error includes the underlying detail.

## NICs and capture

### `bridge:nic(vm)`

Returns a [Nic](/provium/reference/nic.md) bound to the (bridge, vm) pair. When `vm` is a VM userdata, the Nic carries the VM handle through so `nic:disconnect()` / `nic:reconnect()` can drive QMP `set_link`. With a bare string, the Nic is [graph-state only](/provium/writing-tests/bridges-and-impairments.md#declared-vs-realised).

### `bridge:capture()`

Spawn `tcpdump -i <bridge> -U -w -` and return a [Capture](/provium/reference/streams.md) stream. Reads pcap bytes off the tcpdump child's stdout. Requires `tcpdump` on `PATH` plus `CAP_NET_RAW` (in addition to `CAP_NET_ADMIN`).

The capture holds a guard that pins the bridge's `active_captures` counter, so `vm:snapshot()` can detect "stream live, snapshot would race" and refuse.

## Auto-close

### `bridge:close()`

Auto-close hook used by the resource walker. Tears down host-side networking — TAPs, qdiscs, nft tables, the bridge itself. Idempotent.

## Accessors

- `bridge:name()` — Returns the bridge's name.

## See also

- [Nic](/provium/reference/nic.md) — per-(bridge, vm) handle for link-state and per-NIC capture.
- [Streams](/provium/reference/streams.md) — what `bridge:capture()` returns.
- [Lab](/provium/reference/lab.md) — bridges live in a lab.

---

# Lab

_Provium / Reference_

> The Lab userdata — VMs, bridges, and sub-labs, plus claims, barriers, whole-lab snapshots, and fixtures. The provium global is the root Lab.

A Lab is the unit of resource ownership in Provium. The `provium` global is the root Lab; you can carve out sub-labs with `provium:lab("name")`. Both shapes expose the same surface, so this page is the canonical reference.

## Membership

### `lab:vm(name, profile?, opts?)`

Three call shapes:

- `lab:vm("name")` — lookup. Errors if no VM with that name has been declared.
- `lab:vm("name", "profile")` — create.
- `lab:vm("name", "profile", opts)` — create with sizing opts (`memory`, `cpus`). See [VM boot opts](/provium/reference/vm.md#boot-opts) for the split between creation-time and boot-time keys.

Returns a [VM](/provium/reference/vm.md) userdata.

### `lab:bridge(name, opts?)`

Two call shapes:

- `lab:bridge("name")` — lookup or create. If the bridge already exists in the lab, returns it; otherwise creates it.
- `lab:bridge("name", opts)` — same. Opts are reserved.

Returns a [Bridge](/provium/reference/bridge.md) userdata.

### `lab:lab(name?)`

Create a sub-lab. Without a name, an anonymous name is generated (`__lab_0`, `__lab_1`, …). Returns a Lab userdata. Sub-labs share the parent's config, VMM, event sink, and pool, but have their own membership graph.

```lua
local dc1 = provium:lab("dc1")
dc1:vm("a", "peios")
dc1:vm("b", "peios")
dc1:bridge("lan"):attach({dc1.a, dc1.b})
```

### `lab:depends_on_file(host_path)`

Declare an external host-file as a fixture-cache dependency. The call itself is a no-op at runtime — its purpose is to be picked up by the cache-key scanner so that editing `host_path` invalidates the fixture next run.

```lua
provium:depends_on_file("../templates/sshd_config.in")
```

Relative paths are resolved against the directory of the file containing the call. The path must be a string literal at the call site; non-literal paths (variables, concatenation) are not detected. Folds path + mtime + size into the cache key. See [Fixtures and dependencies — External host-file deps](/provium/running-tests/fixtures-and-dependencies.md#external-host-file-deps).

Files pushed into the guest via [`vm:push_file`](/provium/reference/vm.md#vmpush-filehost-path-guest-path-opts) already track themselves — `depends_on_file` is for files the fixture reads on the host side.

### `lab:include(resource_or_list)`

Add an existing resource (VM, Bridge, or sub-Lab) to this lab. Accepts:

- A single userdata — `lab:include(other_lab.shared_vm)`.
- An array — `lab:include({a, b, c})`.

Names must not already be taken in `lab` (errors with `DuplicateVmName` / `DuplicateBridgeName`). Reserved names (`pack`, `unpack`, `vm_fixture`, `lab_fixture`) are also rejected. When called from a test scope, names that already exist in a parent scope (the file root) also error to avoid silent shadowing — see [labs and scope](/provium/writing-tests/labs-and-scope.md#per-test-scope).

### `lab:remove(name_or_userdata)`

Remove a resource. Accepts a bare name (tries vm → bridge → sub-lab in order) or a userdata. Removal is [graph-state only](/provium/writing-tests/bridges-and-impairments.md#declared-vs-realised) — the underlying VM, bridge, or sub-lab is NOT shut down or unrealised.

### `lab:members()`

Returns an array of `{kind=string, name=string}` entries listing every direct child of the lab. `kind` is `"vm"`, `"bridge"`, or `"sub_lab"`.

### `lab:vm_names()` / `lab:bridge_names()` / `lab:sub_lab_names()`

Arrays of names of each kind directly in this lab (does not recurse into sub-labs).

### `lab.<name>` (dot sugar)

Looks up a named resource. Lookup order:

1. Reserved keys: `pack`, `unpack`, `vm_fixture`, `lab_fixture`.
2. VM with that name (walks parent scope chain on miss).
3. Bridge with that name (walks parent scope chain on miss).
4. Sub-lab with that name (walks parent scope chain on miss).
5. Returns `nil`.

The dot form does not create. Use `lab:vm("name", "profile")` for that. The parent-scope walk applies when this lab is a per-test scope (the runner sets `provium` to a test-scope LabUd whose parent chain includes the file root). Federation sub-labs created via `lab:lab(name)` have an empty parent chain and don't fall through to siblings.

## Batch lifecycle

Each method here applies to every direct VM child of the lab. They do not recurse into sub-labs.

### `lab:boot()`

Boot every VM in the lab. Returns `self` so you can chain.

### `lab:shutdown()`

Shutdown every VM.

### `lab:pause()` / `lab:resume()`

Pause / resume every VM. Errors if any VM is in the wrong state for the transition.

## Resource claims

Each test file may make at most one claim against the dispatcher's resource pool. The claim sits across the entire file's lifetime; it is released at file end (or scope-walker end).

### `lab:claim({memory=…, cpus=…})`

Reserve resources from the pool. Either field may be omitted (treated as zero). Both are accepted as integers; `memory` may also be a string with `K`/`M`/`G` suffix.

| Field | Type | Effect |
|---|---|---|
| `memory` | int (bytes) or string `"512M"` / `"2G"` | Memory budget. |
| `cpus` | int | vCPU budget. |

```lua
provium:claim({memory = "4G", cpus = 4})

test("…", function(t) end)
test("…", function(t) end)
```

A second `:claim` call on the same file errors — `lab claim already held; one-shot per lab`. When no pool is wired (REPL / single-file ad-hoc runs) the call still records the one-shot constraint but is otherwise a no-op.

The claim emits a `claim_acquired` event when it goes through; a matching `claim_released` fires at file end.

## Barriers

### `lab:barrier(name, count, timeout?)`

Block until `count` callers have hit the same `name`-keyed barrier. Returns `true` when the barrier releases and `false` on timeout — it does not raise. Default timeout is 60 seconds. The timeout argument accepts a number (seconds) or nil. The count is locked in by the first arrival; a later call with a mismatching count for the same name returns `false` with a host-side warning. Barriers are reusable across rounds under the same name.

```lua
-- count = 1: satisfied immediately.
assert(provium:barrier("checkpoint", 1))

-- Unmet count: times out and returns false.
assert(not provium:barrier("pair", 2, 0.1))
```

Meeting a `count > 1` requires a second concurrent host-side caller. Test files run single-threaded, and workers run guest processes (which can't call `barrier`), so a `count > 1` barrier currently times out in an ordinary test file; the multi-caller form is aimed at thread-mode workers, which aren't supported yet. See [Labs and scope — Barriers](/provium/writing-tests/labs-and-scope.md#barriers).

## Snapshot and restore

### `lab:snapshot(dir?)`

Take a whole-lab snapshot. With an explicit `dir` arg, writes there and returns the dir path as a string. Without args, writes to a fresh tempdir and returns a [LabSnapshot](/provium/reference/snapshot.md) userdata.

### `lab:restore(dir_or_snapshot)`

Restore from a directory path or a `LabSnapshot` userdata. Reads `lab.json` for the per-VM index plus each VM's snapshot file under the same dir.

## Fixtures

### `lab:vm_fixture(path)`

Build (or restore from cache) a single-VM fixture. `path` is the test-root-relative path of a `*.fixture.lua` file with the `.fixture.lua` suffix omitted.

```lua
local vm = provium:vm_fixture("base")
local vm = provium:vm_fixture("setups/networked")
```

The cached snapshot is hashed by source bytes + every transitive `vm_fixture`/`lab_fixture` reference + every `require()`d helper + every profile's kernel and initrd identifier. Editing any of those invalidates the cache for that fixture (and every fixture that transitively depends on it).

Returns a [VM](/provium/reference/vm.md) userdata in the `Booted` state.

### `lab:lab_fixture(path)`

Build (or restore from cache) a multi-VM lab fixture. The fixture's chunk must end with `return provium:snapshot()`. Cached as a directory `<key>.lab/` containing the per-VM snapshots and `lab.json`.

Returns a Lab userdata wrapping a fresh sub-lab into which the fixture has been restored.

## Binary helpers

### `lab:pack(fmt, ...)` / `lab:unpack(fmt, s)`

Pass-throughs to Lua 5.4's `string.pack` / `string.unpack`. Available so test code that processes binary protocols doesn't need to reach into `string.*`. Same call shape and semantics as the standard library.

## Accessors

- `lab:name()` — Returns the lab's name. The root lab is named `"provium"`.

## See also

- [provium global](/provium/reference/provium-module.md) — file-scope configuration globals on top of the root Lab.
- [VM](/provium/reference/vm.md), [Bridge](/provium/reference/bridge.md) — what lives in a lab.
- [Snapshot](/provium/reference/snapshot.md) — what `lab:snapshot()` returns.
- [Fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) — for the cache-key model and rebuild triggers.

---

# Disk

_Provium / Reference_

> The Disk userdata — read and write sectors directly, inject EIO and slow-I/O faults, and detach the disk from the live guest.

A Disk wraps one virtio-blk disk attached to a VM. It exposes block-level operations and a fault-injection surface useful for exercising I/O error paths in the guest.

## Constructing

| Source | Returns |
|---|---|
| `vm:attach_disk({id="vda", size=…, image="…"})` | New disk attached to `vm`. |
| `vm:disk("id")` | Lookup of an already-attached disk. Errors if absent. |

`vm:attach_disk` opts:

| Field | Type | Default | Description |
|---|---|---|---|
| `id` | string | `"attached-<vm_name>"` | Disk identifier within the VM. |
| `size` | int (bytes) | `4 GiB` | Modelled disk size. Used for `:size()` when no image is attached. |
| `image` | string (path) | none | Backing file. Sector ops require an image. |

Disks are 512-byte sectors throughout. The `read_sectors` and `write_sectors` ops express offsets and counts in sectors.

## Methods

### `disk:size()`

Returns the disk's size in bytes. When a backing image is attached, returns the live `stat()` size of the image file (so a test that resized the underlying file with `truncate` / `fallocate` reads honest output). Without an image, returns the modelled size from `attach_disk`.

### `disk:read_sectors(offset, n)`

Read `n` sectors starting at sector `offset`. Returns the bytes as a Lua string. Errors if:

- The disk is detached (`disk:read_sectors: disk is detached`).
- No backing image (`disk:read_sectors: no backing image — disk:with_image required`).
- An `eio_read` fault is active (`disk:read_sectors: EIO (fault_inject)`).
- The host-side `read` fails (passes the underlying error through).

Honours active faults:

- `eio_read` — short-circuits to EIO before any I/O.
- `slow` — sleeps 50 ms per call. After the sleep, re-checks for `eio_read` so a concurrent fault injection during the sleep still takes effect. After the actual I/O, checks one more time for the same reason.

### `disk:write_sectors(offset, data)`

Write `data` (a Lua string) starting at sector `offset`. The data does not have to be a multiple of 512 bytes; it is written verbatim from the offset. Returns nothing.

Errors and fault handling mirror `read_sectors`:

- Detached → error.
- No backing image → error.
- `eio_write` active before the I/O → EIO.
- `slow` → 50 ms sleep, with `eio_write` re-checks after the sleep and after the I/O.

### `disk:fault_inject(mode)`

Activate a fault. Valid modes: `"eio_read"`, `"eio_write"`, `"slow"`. Unknown modes error with the valid list (`disk:fault_inject: unknown mode \`X\` (valid: eio_read, eio_write, slow)`).

Multiple modes can be active simultaneously. When `slow` and the matching `eio_*` are both active at call time, the EIO check runs first — the call errors immediately, without the 50 ms delay. (The post-sleep re-checks only matter for an EIO fault set after the call started.)

### `disk:clear_faults()`

Clear every active fault. Subsequent reads / writes succeed normally.

### `disk:active_faults()`

Returns a Lua array of the currently-active fault mode names. Useful for tests that need to assert the harness state.

### `disk:detach()`

Mark the disk as detached and issue a best-effort QMP `device_del` against the parent VM. Subsequent `read_sectors` / `write_sectors` error. The QMP call is best-effort: a disk that was never QMP-added (host-bookkeeping-only attach) silently surfaces "Device not found" but the local detached flag is still set.

### `disk:is_detached()` / `disk:id()`

Accessors. `:id()` returns the disk identifier as given to `attach_disk`.

## Example: full fault-injection pattern

```lua
test("guest sees EIO and recovers after clear", function(t)
    local img = "/tmp/test-image"
    -- Pre-create the backing file.
    local f = io.open(img, "w"); f:write(string.rep("\0", 512 * 1024)); f:close()

    local vm = provium:vm("v", "peios"):boot()
    local disk = vm:attach_disk({id = "vda", size = 512 * 1024, image = img})

    -- Confirm baseline read works.
    local body = disk:read_sectors(0, 1)
    t:assert_eq(#body, 512)

    -- Inject EIO.
    disk:fault_inject("eio_read")
    local ok, err = pcall(function() disk:read_sectors(0, 1) end)
    t:assert(not ok)
    t:assert(tostring(err):find("EIO"))

    -- Clear and confirm reads recover.
    disk:clear_faults()
    body = disk:read_sectors(0, 1)
    t:assert_eq(#body, 512)
end)
```

## See also

- [VM](/provium/reference/vm.md) — disks come from `vm:attach_disk` / `vm:disk`.
- [Disks and fault injection](/provium/writing-tests/disks-and-fault-injection.md) — patterns for exercising I/O error paths.

---

# Nic

_Provium / Reference_

> A Nic userdata is a per-(bridge, vm) handle. Use it to read NIC counters, capture per-NIC pcap, or simulate link-down / link-up on a single attachment.

A Nic represents one virtual NIC: the binding between a VM and a Bridge. You get one from either side of that pair (`vm:nic(bridge_name)` or `bridge:nic(vm)`), and it gives you per-NIC observability and link control.

## Constructing

| Source | Returns |
|---|---|
| `vm:nic("bridge_name")` | NIC for the named bridge attachment. |
| `vm:nic("eth0")` | NIC for the Nth attached bridge sorted by bridge name (`eth<N>`, `enp0s<N>`, `ens<N>`). |
| `bridge:nic(vm)` | NIC for the (bridge, vm) pair. When `vm` is a userdata, the link-state ops drive QMP. |

The guest-name lookup (`eth0`) is deterministic per build but may not match the kernel's actual device-probe order on every distro. For portable tests, prefer `vm:nic("bridge_name")` and `bridge:nic(vm)`.

## Methods

### `nic:counters()`

Returns a table of per-NIC counters read from `/sys/class/net/<tap>/statistics/`. The counters are presented from the **guest's perspective**: the host's `tx_bytes` on the TAP is what the guest received, so it appears as `rx_bytes` in the table.

| Field | Description |
|---|---|
| `rx_bytes` | Bytes the guest received (host TAP's `tx_bytes`). |
| `tx_bytes` | Bytes the guest sent (host TAP's `rx_bytes`). |
| `rx_packets` | Packets received. |
| `tx_packets` | Packets sent. |
| `errors` | Sum of host TAP's `rx_errors` and `tx_errors`. |

Returns zeros when the host TAP isn't yet realised (VM not booted). If you need to assert a NIC has produced traffic, gate the read on `vm:state() == "booted"` or `wait_until` on a non-zero counter.

### `nic:capture()`

Spawn `tcpdump -i <tap>` on this NIC's host TAP interface. Returns a [Capture](/provium/reference/streams.md) stream. Errors if the VM hasn't been booted yet (the per-VM TAP doesn't exist):

```
nic:capture: vm `a` has no TAP on bridge `lan` (not booted?). Call lab:boot() / vm:boot() first.
```

The capture pins the bridge's `active_captures` counter, so `vm:snapshot()` can detect a live capture and refuse the snapshot (no half-captured pcap).

Compare with `bridge:capture()` which captures the whole bridge, not a single NIC.

### `nic:disconnect()`

Detach the NIC from the bridge **and** issue a QMP `set_link(false)` against the matching netdev so the guest sees a link-down event. Without a VM userdata at construction time (bare-string `bridge:nic("name")` form), the QMP step is skipped — the call is [graph-state only](/provium/writing-tests/bridges-and-impairments.md#declared-vs-realised).

### `nic:reconnect()`

Reverse of `:disconnect()`. Re-attaches the NIC and issues a `set_link(true)`.

### `nic:vm_name()` / `nic:bridge()`

Accessors. `:vm_name()` returns the VM's name; `:bridge()` returns the bridge's name.

## Example: link-down recovery

```lua
test("guest reconnects after link flap", function(t)
    local lan = provium:bridge("lan")
    local a   = provium:vm("a", "peios"):boot()
    local b   = provium:vm("b", "peios"):boot()
    lan:attach({a, b})
    a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
    b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")

    -- Baseline.
    a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()

    local nic = a:nic("lan")
    nic:disconnect()
    -- Pings should fail now.
    local r = a:run("ping -c 1 -W 1 10.0.0.2")
    t:assert(not r:ok())

    nic:reconnect()
    -- And succeed after reconnect.
    a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
end)
```

## See also

- [Bridge](/provium/reference/bridge.md) — bridge-wide controls (partition, impairments, capture).
- [Streams](/provium/reference/streams.md) — what `nic:capture()` returns.
- [Bridges and impairments](/provium/writing-tests/bridges-and-impairments.md) — patterns for using NIC and bridge handles together.

---

# File

_Provium / Reference_

> A File userdata wraps an open file descriptor inside the guest. Use it to read, write, seek, tell, fd-to-int, tail, and close. Mirrors POSIX semantics.

A File is an open guest-side file handle, returned by `vm:open_file(path, mode)` or `worker:open_file(path, mode)`. Once closed, all ops error with `file is closed`. Idempotent: closing twice is safe.

## Constructing

| Source | Returns |
|---|---|
| `vm:open_file(path, mode)` | New file handle in the guest. |
| `worker:open_file(path, mode)` | Same but allocated under a worker's namespace. |

Mode table fields: `read`, `write`, `create`, `truncate`, `append`, `exclusive`, `perm`. See [VM open_file](/provium/reference/vm.md#vmopen-filepath-mode-table) for the full mode-table reference.

At least one of `read`, `write`, or `append` must be true.

## Methods

### `file:read(n)`

Read up to `n` bytes from the current cursor. Returns the bytes as a Lua string. Returns the empty string at EOF (POSIX `read` semantics — never errors on EOF).

```lua
local f = vm:open_file("/etc/hostname", {read=true})
local s = f:read(64)
```

### `file:read_all()`

Drain the file from the current cursor to EOF. Returns the bytes as a Lua string. Internally chunked at 64 KiB. At EOF, returns the empty string.

### `file:write(data)`

Write `data` (a Lua string of bytes) at the current cursor. Returns the number of bytes actually written (which may be less than `#data` on a partial write — POSIX semantics).

### `file:seek(offset, whence?)`

Reposition the cursor. `whence` defaults to `"set"`; valid values: `"set"`, `"cur"`, `"end"`. Anything else errors with `seek: whence must be set/cur/end, got \`X\``.

Returns the new absolute offset.

### `file:tell()`

Returns the current cursor position. This authoritatively re-reads from the agent (a no-op `Seek(Cur, 0)`) rather than trusting the host-side cursor cache, so `tell()` stays honest when a previous read or write returned mid-flight.

### `file:close()`

Close the file. Idempotent: a second `:close()` does not error. After close, every other op errors with `file is closed`.

### `file:fd()`

Returns the raw u64 handle id. Useful for `vm:ioctl(fd, …)`, `vm:syscall(…)` (where the fd is one of the integer args), or `vm:fd_stream(fd)` to open a streaming subscription against the same file.

After close, returns `0`.

### `file:tail_stream()`

Open a [Tail](/provium/reference/streams.md) stream rooted at the file's current cursor. Streams new bytes appended after the current position. Useful for "give me a stream that starts here" without re-opening the file.

Errors with `file:tail_stream needs the source path; opened via wrap() not wrap_with_path()` if the File was constructed without a recorded path (which never happens through `vm:open_file` — it always records the path).

## Example

```lua
test("file handle round-trip", function(t)
    local vm = provium:vm("v", "peios"):boot()
    local h = vm:open_file("/tmp/data", {write=true, create=true, truncate=true, perm=0x180})  -- 0o600
    local n = h:write("hello world")
    t:assert_eq(n, 11)
    h:close()

    local r = vm:open_file("/tmp/data", {read=true})
    r:seek(6, "set")
    t:assert_eq(r:read(5), "world")
    t:assert_eq(r:tell(), 11)
    r:close()
end)
```

## EOF semantics

Reading past EOF returns the empty string `""`, not nil and not an error. Tests that loop "read until empty" should use:

```lua
while true do
    local chunk = h:read(4096)
    if chunk == "" then break end
    -- process chunk
end
```

## See also

- [VM](/provium/reference/vm.md) — `vm:open_file`, `vm:read_file`, `vm:write_file`, `vm:fd_stream`.
- [Worker](/provium/reference/worker.md) — `worker:open_file` for files allocated under a worker's namespace.
- [Streams](/provium/reference/streams.md) — what `file:tail_stream()` returns.
- [Files and handles](/provium/writing-tests/files-and-handles.md) — patterns for guest-side file I/O.

---

# Process

_Provium / Reference_

> A Process userdata is a handle to an async-spawned guest process. Use it to wait, kill, signal, write stdin, stream stdout/stderr, and query exit status.

A Process is what `vm:run_async(...)` (or `worker:run_async(...)`) returns: a handle to a guest process that the agent has launched but is not waiting on. You control the lifetime — the agent does not auto-kill it.

## Constructing

| Source | Returns |
|---|---|
| `vm:run_async(cmd, opts?)` | New process under the VM's main agent. |
| `worker:run_async(cmd, opts?)` | New process under a worker namespace. |

Opts mirror `vm:run`'s shape (`args`, `env`, `env_clear`, `cwd`), but the `timeout` / `timeout_ms` keys are rejected with a pointer to use `proc:wait(timeout)` instead. `stdin` works (initial bytes go through `RunAsyncArgs`); subsequent input goes through `proc:stdin_write`.

## Methods

### `proc:wait(timeout?)`

Wait for the process to exit. Returns a [RunResult](/provium/reference/vm.md#runresult) userdata.

`timeout` accepts:
- `nil` — wait forever.
- A number — seconds.
- A string — `"5s"` / `"500ms"` / `"5m"` / `"2h"`.

Passing `0` is rejected with a pointer to use `proc:status()` for non-blocking polling — a literal-zero timeout would otherwise SIGKILL the process immediately because the agent's `wait_with_timeout(0)` sees the deadline already past.

After `:wait()` returns, the agent-side slot is gone. The Process is "consumed"; subsequent ops still work, but `:close()` short-circuits without re-killing.

### `proc:kill(sig?)`

Send a signal. `sig` accepts:

- `nil` — defaults to `SIGTERM`.
- An integer — that signal number.
- A string — friendly name. Recognised: `term`/`sigterm`/`15`, `kill`/`sigkill`/`9`, `int`/`sigint`/`2`, `hup`/`sighup`/`1`, `quit`/`sigquit`/`3`, `stop`/`sigstop`, `cont`/`sigcont`, `usr1`/`sigusr1`/`10`, `usr2`/`sigusr2`/`12`, `alrm`/`sigalrm`/`14`, `pipe`/`sigpipe`/`13`, `chld`/`sigchld`/`17`, `winch`/`sigwinch`. Comparison is case-insensitive.

Unknown name → `unknown signal name \`X\``.

### `proc:signal(sig)`

Same as `:kill(sig)`. Read better when the signal is non-fatal (`proc:signal("usr1")`).

### `proc:pid()`

Returns the kernel PID of the process inside the guest. Fetched live via the agent's GetPid op.

### `proc:handle()`

Returns the opaque agent-side handle id (an integer). Distinct from `pid()` — the handle stays stable across forks; the PID can change if the process re-execs.

### `proc:status()`

Non-blocking poll. Returns the agent's view of the process's status without waiting. Useful for "is it still running" without blocking.

### `proc:stdin_write(data)`

Write `data` (Lua string) to the process's stdin. Returns whatever the agent reports (typically the byte count).

### `proc:close_stdin()`

Close the stdin pipe. After this, subsequent `:stdin_write` errors.

### `proc:stdout_stream()` / `proc:stderr_stream()`

Open a [Tail](/provium/reference/streams.md) stream subscribed to captured stdout / stderr. The agent polls the captured-output buffer and emits frames. Useful when the process produces a lot of output you want to consume incrementally.

The returned Tail's `StreamMeta` is pre-filled with `kind="proc_stdout_stream"` (or `"proc_stderr_stream"`) and the process handle id, so snapshot diagnostics show meaningful detail.

### `proc:close()`

Auto-close hook. If `:wait()` already happened, this is a no-op. Otherwise, sends SIGTERM, then waits up to 2 seconds for exit (the agent escalates to SIGKILL on its own timeout).

## Example: tail logs while the process runs

```lua
test("server emits ready then handles request", function(t)
    local vm = provium:vm("v", "peios"):boot()
    local proc = vm:run_async("python3", {args = {"server.py"}})
    local out  = proc:stdout_stream()

    -- Wait for the server to log "ready".
    out:expect("ready", "5s")

    -- Now hit it.
    local r = vm:run("curl -s http://localhost:8080/")
    r:assert_ok()
    t:assert(r.stdout:find("hello"))

    -- Tear down. Inside the test scope this happens automatically;
    -- here we just demonstrate it works explicitly too.
    proc:kill("term")
    local exit = proc:wait("2s")
    t:assert(exit.signal == 15 or exit.exit_code == 0)
end)
```

## See also

- [VM](/provium/reference/vm.md) — `vm:run_async` returns a Process.
- [Worker](/provium/reference/worker.md) — `worker:run_async` returns a Process.
- [Streams](/provium/reference/streams.md) — what `proc:stdout_stream` / `proc:stderr_stream` return.
- [Running commands](/provium/writing-tests/running-commands.md) — patterns for sync vs async, stdin, env.

---

# Worker

_Provium / Reference_

> A Worker userdata gives a test concurrent agent connections to the same guest VM, so you can drive multiple ops in parallel without spinning up another VM.

A Worker is what `vm:spawn_worker()` returns: a sub-agent connection to the same VM. It exposes the same VM-style API for running commands, opening files, and issuing syscalls — handles allocated under it live in the worker's own namespace on the agent side.

Workers are not a hard isolation boundary. They're bookkeeping namespaces — handles are routable from outside the worker, but the agent tracks per-worker membership for cleanup. Enforced per-worker isolation is not currently supported; treat workers as parallelism, not security.

## Constructing

| Source | Returns |
|---|---|
| `vm:spawn_worker()` | New worker on the VM's main agent. |
| `vm:spawn_worker({thread = false})` | Same. (`{thread = true}` is rejected with a future-work pointer.) |

## Methods

The worker mirrors the VM's run / file / syscall surface. Where semantics differ from the VM equivalent, it's noted explicitly.

### `worker:run(cmd, opts?)`

Synchronous exec. Same call shape as [`vm:run`](/provium/reference/vm.md#vmruncmd-or-args-opts). Returns a [RunResult](/provium/reference/vm.md#runresult).

### `worker:run_async(cmd, opts?)`

Async spawn. Returns a [Process](/provium/reference/process.md). Like `vm:run_async`, the `timeout` opt is rejected — pass it to `proc:wait` instead.

The returned Process is auto-registered with the test's resource registry, so the scope walker SIGTERMs and reaps it at scope end. Without this, a worker-spawned child would leak past the test boundary.

### `worker:open_file(path, mode_table)`

Open a guest-side file under the worker's namespace. Returns a [File](/provium/reference/file-handle.md). Same mode-table shape as `vm:open_file`.

The returned File is auto-registered with the test scope so `file:close()` fires automatically at scope end.

### `worker:syscall(nr, …)`

Direct syscall. Same call shape as [`vm:syscall`](/provium/reference/vm.md#vmsyscallnr-) — both the integer-only and table forms work. Returns the same `{ret, result, errno, out_bufs}` shape.

### `worker:kill(sig?)`

Broadcast a signal to every async process spawned under this worker. Argument shape mirrors `proc:kill`: nil → SIGTERM, int → that signal, string → friendly name.

A bad argument type (e.g. a Lua table) errors with the type-name in the message rather than silently defaulting to SIGTERM. This catches test-code typos.

### `worker:join()`

Wait for every in-flight worker child to exit. Returns whatever the agent reports as the joined exit summary.

### `worker:handle()`

Returns the worker's opaque handle id.

### `worker:close()`

Auto-close hook. SIGTERMs every in-flight worker child, then joins. The scope walker fires this at scope end.

## Concurrency pattern

Workers shine for tests that need two threads of control inside the same guest:

```lua
test("two concurrent writers don't tear", function(t)
    local vm = provium:vm("v", "peios"):boot()
    local w1 = vm:spawn_worker()
    local w2 = vm:spawn_worker()

    -- Each worker writes its own block of data, in parallel.
    local p1 = w1:run_async("dd", {args = {"if=/dev/urandom", "of=/tmp/a", "bs=1M", "count=8"}})
    local p2 = w2:run_async("dd", {args = {"if=/dev/urandom", "of=/tmp/b", "bs=1M", "count=8"}})

    p1:wait("10s"):assert_ok()
    p2:wait("10s"):assert_ok()

    t:assert_eq(vm:stat("/tmp/a").size, 8 * 1024 * 1024)
    t:assert_eq(vm:stat("/tmp/b").size, 8 * 1024 * 1024)
end)
```

For coordination between workers' guest processes, use a guest-side primitive (file, fifo, etc.). [`lab:barrier(name, count, timeout?)`](/provium/reference/lab.md#labbarriername-count-timeout) is a host-side rendezvous — a worker's guest processes can't call it. See [Labs and scope — Barriers](/provium/writing-tests/labs-and-scope.md#barriers) for what barriers can and can't synchronise today.

## See also

- [VM](/provium/reference/vm.md) — `vm:spawn_worker()` and the parent surface.
- [Process](/provium/reference/process.md), [File](/provium/reference/file-handle.md) — what worker ops return.

---

# Streams

_Provium / Reference_

> The three stream types — Tail, Capture, and ConsoleStream — and the shared surface of next, read_until, expect, drain, close, eof, and creation_site.

Provium has three concrete stream types. They expose a shared surface, so test code can `:next` / `:read_until` / `:expect` / `:drain` / `:close` / `:eof` / `:creation_site` against any of them without caring which kind it is.

| Type | Returned by | Backing |
|---|---|---|
| **Tail** | `vm:tail_file`, `vm:fd_stream`, `file:tail_stream`, `proc:stdout_stream`, `proc:stderr_stream` | Frame-based agent stream over vsock. |
| **Capture** | `bridge:capture`, `nic:capture` | `tcpdump` child stdout — pcap bytes, not framed. |
| **ConsoleStream** | `console:read` (returned by `vm:console():read()`) | Unix socket connected to QEMU's console chardev. |

All three buffer "leftover bytes" past the last `expect`/`read_until` match. A subsequent `:next` returns those leftover bytes as the next frame, so test code never silently loses data.

## Common methods

### `:next(timeout?)`

Returns the next chunk of bytes as a Lua string, or `nil` at EOF / timeout. The exact frame shape:

- **Tail**: one frame per agent-side write. Each frame is whatever bytes the guest had emitted since the previous frame.
- **Capture**: chunks of pcap bytes from tcpdump's pipe (variable size, up to 64 KiB).
- **ConsoleStream**: chunks of console bytes (variable size, up to 64 KiB).

The `timeout` argument accepts seconds (number) or a string with `ms`/`s`/`m`/`h` suffix. Default: 10 seconds.

### `:read_until(pattern, timeout?)`

Pull frames until `pattern` (a Lua string) is found in the accumulated buffer. Returns the prefix up to and including the matched bytes. Errors with a pattern-naming message on timeout (`read_until timed out waiting for \`X\``) or stream EOF (`stream EOF`).

Bytes past the matched suffix are kept as `pending` and replayed on the next call.

Default timeout: 10 seconds.

### `:expect(pattern, timeout?)`

Like `:read_until` but discards the matched prefix. Returns nothing. Useful as an assertion: "the stream produced this; I don't care about the bytes". Same error shape as `read_until`.

Default timeout: 10 seconds.

### `:drain(timeout?)`

Collect every frame currently available and return them as a Lua array. Stops when the next read would block past the timeout, when EOF is reached, or when no more data is available.

Frame-per-element semantics — each iteration of the inner read produces one Lua-string entry. Pending bytes from a prior `expect`/`read_until` come out as the first entry.

Default timeout: 0.5 seconds (much shorter than `next`/`read_until`/`expect` because drain's intent is "what's available right now, then return").

### `:close()`

Close the underlying transport. Subsequent `:next` returns `nil`; subsequent `:read_until`/`:expect` error with `stream closed` / `console closed` / `capture closed`. Idempotent.

Pending bytes are dropped on close so a `:next` after close can't drain stale data.

### `:eof()`

Returns `true` once the stream's transport has reported EOF AND no `pending` bytes remain. Pending bytes mean there's still data to deliver before EOF is honest, so `eof()` returns `false` until they've been consumed.

### `:creation_site()`

Returns `{file=string, line=int}` for the test-author frame that opened the stream, or `nil` if no `*.test.lua`/`*.fixture.lua` frame is on the stack at creation. Useful for snapshot diagnostics — when `vm:snapshot()` refuses because of a live stream, the error names the offending stream's creation site so you can fix it.

## Tail-only behaviour

`vm:tail_file(path, opts?)` accepts a `start` opt:

| Value | Effect |
|---|---|
| `"end"` (default) | Stream only bytes appended after the call. |
| `"beginning"` or `"start"` | Replay the whole file from byte 0, then continue tailing. |
| Non-negative integer | Start streaming from that exact byte offset. |
| Negative integer | Start N bytes before EOF — Provium stats the file and resolves to an absolute offset. If N exceeds the current file size, the stream starts at byte 0. |
| Finite float | Truncated toward zero, then treated as the integer cases above (negative floats count back from EOF). |

A closed Tail returns `nil` from `:next` — the closed stream is semantically EOF. This matches Capture and ConsoleStream, so `while s:next() do … end` loops terminate cleanly.

Tails carry per-frame metadata visible to scope-end diagnostics: `kind` (e.g. `tail_file`, `fd_stream`, `file_tail_stream`, `proc_stdout_stream`), `detail` (the path or fd), `creation_site`, and `test_name`.

## Capture-only notes

Capture spawns `tcpdump -i <iface> -U -w - -s 65535`. The output is pcap-formatted bytes — the standard pcap file format with global header followed by per-packet records. Feed it through `pcap-parser`, `tshark -r -`, or `pyshark` to interpret.

The Capture holds a guard against the bridge's `active_captures` counter so a `vm:snapshot()` while capture is live errors instead of silently producing a half-captured pcap.

Drop on close prevents `:next` from returning stale tcpdump bytes after the child has been killed.

## ConsoleStream-only notes

ConsoleStream connects via `UnixStream` to the VMM's console chardev path (`vm:console_socket_path()`). Reads come back as raw bytes from the chardev — typically the boot console + login prompt + anything the guest has written to `/dev/ttyS0` since the last read.

The connect-time read timeout is 50 ms. The `:next(timeout)` arg overrides it for the duration of the call and the previous timeout is restored afterwards, so a `next("5s")` followed by a bare `next()` doesn't inherit the 5s deadline forever.

Mid-stream `ConnectionReset` and `BrokenPipe` errors are mapped to "console EOF" — they happen when the QEMU chardev closes mid-read on a VM reset or shutdown. Semantically EOF for a console stream, not a real I/O error.

## Example patterns

### Wait for a log line

```lua
local f = vm:open_file("/var/log/messages", {read=true})
local stream = f:tail_stream()
stream:expect("ready", "30s")
```

### Race a process against a timeout

```lua
local proc = vm:run_async("slow-server")
local out  = proc:stdout_stream()
out:expect("listening", "10s")
-- now hit it
```

### Drain a capture and pass through pcap-parser

```lua
local cap = bridge:capture()
vm:run("ping -c 5 10.0.0.2")
local frames = cap:drain("2s")
local pcap = table.concat(frames)
-- write pcap to a file or feed it to a parser
```

## See also

- [VM](/provium/reference/vm.md) — `vm:tail_file`, `vm:fd_stream`.
- [Bridge](/provium/reference/bridge.md) — `bridge:capture`.
- [Nic](/provium/reference/nic.md) — `nic:capture`.
- [Console](/provium/reference/console.md) — `console:read` returns a ConsoleStream.
- [Process](/provium/reference/process.md) — `proc:stdout_stream` / `proc:stderr_stream` return Tails.
- [File](/provium/reference/file-handle.md) — `file:tail_stream` returns a Tail.
- [Streams and tails](/provium/writing-tests/streams-and-tails.md) — patterns and gotchas.

---

# Console

_Provium / Reference_

> The Console userdata — read the boot log, stream the chardev, expect a substring, write input, and how it interacts with snapshots.

A Console wraps the guest's serial console — the host writes to and reads from QEMU's chardev bound to the guest's serial port (`/dev/ttyS0`). Provium appends `console=ttyS0` to the kernel command line if no serial console is present, so kernel output always lands here.

## Constructing

| Source | Returns |
|---|---|
| `vm:console()` | The guest's console. Always succeeds; underlying chardev access is checked at the first `:read` / `:read_log` / `:write` call. |

## Methods

### `console:read()`

Returns a [ConsoleStream](/provium/reference/streams.md) backed by a `UnixStream` to the QEMU chardev socket. Use the stream's `:next` / `:read_until` / `:expect` / `:drain` to consume console output incrementally.

The stream is registered with the VM's resource graph, so an open `console:read()` blocks `vm:snapshot()` from succeeding silently — the snapshot precondition reports "live console stream" with creation site.

Errors with `console:read: VMM does not expose a console socket` if the VMM backend doesn't surface a chardev path (rare; the LocalAgent backend does not).

### `console:read_log()`

Returns the entire captured console log to date as a Lua string. Snapshot-style; not a stream. Useful for one-shot inspections:

```lua
local log = vm:console():read_log()
t:assert(log:find("Welcome to Peios"))
```

### `console:expect(pattern, timeout?)`

Poll the captured log every 50 ms until `pattern` (a Lua string) appears or `timeout` lapses. Default timeout: 30 seconds. Returns the matched substring on hit; raises with the pattern in the message on timeout.

This is the simple form for "wait for the kernel to log X." For long-running streams or when you also want to read past-the-match bytes, use `console:read():expect(...)` instead.

### `console:write(data, opts?)`

Write `data` (Lua string) to the chardev. Opts:

| Key | Type | Effect |
|---|---|---|
| `timeout` | number (seconds) or string `"500ms"` / `"5s"` | Bound the blocking write. `0` means no timeout. |

The legacy `timeout_ms` integer key is **rejected** with a clear error (`opts.timeout_ms is not supported (use timeout = "500ms" or timeout = 0.5)`).

Use this to feed input to interactive console programs, or to pre-populate a getty login.

### `console:close()`

No-op in v1. The chardev itself is owned by the VMM; closing the Console userdata is a Lua-level concept only.

## Example: drive a getty login

```lua
test("login as root via console", function(t)
    provium.timeout = "60s"
    local vm = provium:vm("v", "peios"):boot()
    local console = vm:console()
    local stream  = console:read()
    stream:expect("login:", "30s")
    console:write("root\n")
    stream:expect("Password:", "5s")
    console:write("toor\n")
    stream:expect("# ", "5s")
end)
```

## See also

- [VM](/provium/reference/vm.md) — `vm:console()`.
- [Streams](/provium/reference/streams.md) — what `console:read()` returns.

---

# Clock

_Provium / Reference_

> The Clock userdata — set, advance, sleep, and query the guest's wall clock in seconds or nanoseconds, including moving time backwards.

A Clock wraps the guest's wall clock. Operations dispatch to the agent, which adjusts the kernel's `CLOCK_REALTIME` (and equivalents) under the hood.

The clock is signed — you can move the guest backwards in time. This matters for testing time-dependent code: certificate expiry, timeout handling, leap-second handling, retry backoff.

## Constructing

| Source | Returns |
|---|---|
| `vm:clock()` | The guest's clock. |

## Methods

### `clock:get()`

Returns the current time as seconds-since-epoch (float). For sub-microsecond accuracy use `:get_ns()`.

```lua
local t = vm:clock():get()
-- t ≈ 1700000000.0
```

### `clock:get_ns()`

Returns nanoseconds-since-epoch as a 64-bit signed integer. Lua 5.4 integers are 64-bit so no precision is lost.

### `clock:set(seconds)`

Set the wall clock to `seconds` seconds since epoch. Accepts integers or floats. Negative values are allowed — the guest goes pre-epoch, useful for testing `time_t` boundary handling.

Errors on NaN or infinity (`clock:set: value must be a finite number (got NaN/inf)`). Without this guard, the integer cast would silently saturate or round to zero.

### `clock:set_ns(ns)`

Set the wall clock to `ns` nanoseconds since epoch (i64). Same semantics as `:set` but at full precision.

### `clock:sleep(duration)`

Sleep for `duration` seconds. Accepts:

- Integer / float — seconds.
- String — `"100ms"` / `"5s"` / `"5m"` / `"2h"`.

Errors on NaN, infinity, or negative.

### `clock:advance(duration)`

Advance the wall clock by `duration` seconds (signed). Unlike `:sleep`, this does not block — the kernel's `CLOCK_REALTIME` jumps forward (or backward).

Negative values are explicitly allowed (`clock:advance(-3600)` rolls the guest back an hour). Errors on NaN or infinity but not on negative.

## Example: certificate expiry test

```lua
test("client refuses an expired cert", function(t)
    local vm = provium:vm("v", "peios"):boot()
    -- Set the clock to before the cert was issued.
    vm:clock():set(1500000000)  -- 2017-07-14
    local r = vm:run("openssl s_client -connect server:443 < /dev/null")
    r:assert_ok()  -- baseline: cert is valid

    -- Jump past expiry.
    vm:clock():advance(20 * 365 * 24 * 3600)  -- +20 years
    local r2 = vm:run("openssl s_client -connect server:443 < /dev/null")
    t:assert(not r2:ok())  -- cert is now expired
    t:assert(r2.stderr:find("certificate has expired"))
end)
```

## Notes

- `clock:get()` (float) loses precision below ~250 ns at modern epoch values because of f64 mantissa width. Use `:get_ns()` if you need exact ns roundtrip.
- The clock is not a guest namespace — every process inside the guest sees the same time axis after `:set` / `:advance`.
- Boot-time initial clock is set via `boot_opts.initial_time` (see [VM boot opts](/provium/reference/vm.md#boot-opts)). The Clock methods are for adjustment after boot.

## See also

- [VM](/provium/reference/vm.md) — `vm:clock()` and the `boot_opts.initial_time` boot-time setter.

---

# Snapshot and LabSnapshot

_Provium / Reference_

> Snapshot and LabSnapshot userdata wrap the on-disk artefacts from vm:snapshot() and lab:snapshot() — path, size, and delete.

`vm:snapshot()` returns a **Snapshot** wrapping a single file. `lab:snapshot()` returns a **LabSnapshot** wrapping a directory containing per-VM snapshot files plus a `lab.json` index.

Both are thin handles over filesystem paths. They have no agent-side state — once the snapshot file or directory is on disk, the userdata just gives you ergonomic access to it.

## Snapshot

### Constructing

| Source | Returns |
|---|---|
| `vm:snapshot()` | New snapshot at a fresh tempfile. |
| `vm:snapshot("/explicit/path")` | Snapshot at that exact path. |

A fixture's chunk returns a Snapshot (single-VM fixture); the harness reads the path, compresses + sparsifies the file, and installs it into the cache.

### Methods

#### `snap:path()`

Returns the on-disk path as a Lua string.

#### `snap:size()`

Returns the file size in bytes. Returns `0` rather than ENOENT if the file has been moved or deleted (idempotent with `:delete()`).

#### `snap:delete()`

Best-effort delete. Idempotent — calling on a path that's already gone is fine. Returns nothing.

## LabSnapshot

### Constructing

| Source | Returns |
|---|---|
| `lab:snapshot()` | New lab snapshot in a fresh tempdir. |
| `provium:snapshot()` | Same, on the root lab. |

A lab fixture's chunk returns a LabSnapshot; the harness moves the directory into the cache as `<key>.lab/`.

### Methods

#### `lab_snap:path()`

Returns the on-disk directory path as a Lua string.

#### `lab_snap:size()`

Returns the sum of file sizes directly under the directory (one level deep, doesn't recurse). Useful for picking large fixtures out of the cache.

#### `lab_snap:delete()`

Recursively deletes the directory. Idempotent — calling on a path that's already gone is fine.

## Example: build a fixture

```lua
-- tests/fixtures/base.fixture.lua
local vm = provium:vm("base", "peios"):boot()
vm:run("seq 1000000 > /srv/corpus"):assert_ok()
return vm:snapshot()
```

```lua
-- tests/fixtures/cluster.fixture.lua
provium:bridge("lan")
local a = provium:vm("a", "peios"):boot()
local b = provium:vm("b", "peios"):boot()
provium.lan:attach({a, b})
a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")
return provium:snapshot()
```

```lua
-- tests/uses-base.test.lua
test("base fixture has the corpus pre-built", function(t)
    local vm = provium:vm_fixture("fixtures/base")
    vm:run("test -s /srv/corpus"):assert_ok()
end)

test("cluster fixture brings two VMs", function(t)
    local cluster = provium:lab_fixture("fixtures/cluster")
    cluster.a:run("ping -c 1 10.0.0.2"):assert_ok()
end)
```

## See also

- [VM](/provium/reference/vm.md) — `vm:snapshot()`, `vm:restore()`.
- [Lab](/provium/reference/lab.md) — `lab:snapshot()`, `lab:restore()`, `lab:vm_fixture()`, `lab:lab_fixture()`.
- [Fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) — how snapshots become cached fixtures.

---

# Test framework

_Provium / Reference_

> test() registers tests, the t context provides assertions and skip / fail / log, todo() skips a whole file, and wait_until() polls a predicate.

Provium's test framework is implemented mostly in Lua and installed onto every test file's state. This page documents every callable: `test`, `todo`, `wait_until`, the `t` context, and the per-test metadata table.

## `test(name, [meta,] fn)`

Register a test in declaration order. Two call shapes:

```lua
test("simple", function(t)
    t:assert(true)
end)

test("with metadata", {tags={"smoke"}, timeout="30s"}, function(t)
    t:assert(true)
end)
```

| Argument | Required | Type | Description |
|---|---|---|---|
| `name` | yes | string | Test name. Must be unique within the file (duplicate names raise at registration). |
| `meta` | no | table | Per-test metadata. See [meta tags](/provium/reference/meta-tags.md). |
| `fn` | yes | function | The test body, called with `(t)` — the test context. |

Tests run sequentially in declaration order. Each test gets a fresh `t` context. The harness wraps the call in `pcall` semantics — a Lua `error()` becomes a `Failed` outcome; clean return becomes `Passed`; explicit `t:skip()` becomes `Skipped`.

Duplicate names within a file error at registration time:

```
test: duplicate name `foo` in this file
```

## `todo(reason?)`

Declarative file-scope skip. When called at top level (before any `test()` body runs), every registered test is reported `Skipped` with the given `reason`. Test bodies are not executed.

```lua
todo("waiting on the spec")
test("a", function(t) … end)
test("b", function(t) … end)
-- Both report Skipped with reason "waiting on the spec".
```

Without an argument, defaults to `"todo"`.

Skipped tests still emit `TestSkipped` events with the meta intact, so coverage and dashboards see the same shape they would for an inline `t:skip()`.

## `wait_until(predicate, opts?)`

Call `predicate` repeatedly until it returns truthy (and return that value), or until the timeout lapses. Useful for polling guest-side conditions that don't have a stream interface.

```lua
local pid = wait_until(function()
    local r = vm:run("pidof nginx")
    if r:ok() then return r.stdout:match("%d+") end
end, {timeout = "30s", interval = "200ms", desc = "nginx running"})
```

Opts:

| Field | Default | Type | Description |
|---|---|---|---|
| `timeout` | `10` | number (seconds) or string `"30s"` / `"500ms"` / `"5m"` / `"2h"` | Total time to wait before giving up. |
| `interval` | `0.1` | number (seconds) | Time between predicate calls. |
| `desc` | `"condition"` | string | Used in the timeout error message. |

On timeout, errors with `wait_until: <desc> not met within <timeout>s`. The error includes the offending input (e.g. `bad timeout string ...`) when the timeout argument is malformed.

The predicate runs under `pcall` — if it raises, the error propagates immediately (with the raise location) rather than being treated as a "not yet" signal.

`wait_until` uses wall-clock seconds, not CPU time, so the deadline elapses while `_provium_sleep` is pausing.

## The `t` context

A fresh `t` table is built per test. It carries the test's name and metadata, and the runner keeps a sticky per-test failure record so a `pcall(t:assert(false))` is still classified as a failure (the inner pcall swallows the raise, but the failure was recorded before it).

### Fields

- `t.name` — the test's name.
- `t.meta` — the per-test metadata table (or `{}` if none was given).

### Assertions

Every assertion error includes the offending values in the message — assertion failures are self-explanatory without needing to read context.

#### `t:assert(cond, msg?)`

Raise if `cond` is falsy. `msg` is the failure message; defaults to `"assertion failed"`.

#### `t:assert_eq(a, b, msg?)`

Raise if `a ~= b`. The error message includes both values: `<msg>: <a> ~= <b>`.

#### `t:assert_neq(a, b, msg?)`

Raise if `a == b`. Mirror of `assert_eq`.

#### `t:assert_contains(haystack, needle, msg?)`

Raise unless `string.find(haystack, needle, 1, true)` returns non-nil. **Both arguments must be strings**; non-string args raise immediately with a "slice 2 limit" pointer.

#### `t:assert_raises(fn, msg?)`

Run `fn` under `pcall` and assert that it raised. Returns the error value if the raise happened. Errors with `<msg>: expected to raise` if `fn` returned cleanly.

The sticky failure record is saved before invoking `fn` and restored on a successful raise — that way an assertion *inside* `fn` (used to trigger the expected raise) doesn't poison the rest of the test.

### Skip and fail

#### `t:fail(msg?)`

Mark the test failed and raise. `msg` defaults to `"explicit failure (t:fail)"`.

#### `t:skip(reason)`

Mark the test skipped and raise via the internal sentinel. `reason` is recorded in the `TestOutcome` and emitted in the `TestSkipped` event.

```lua
test("only on btrfs", function(t)
    if vm:run("findmnt /"):ok() and not vm:run("findmnt -t btrfs /"):ok() then
        t:skip("not a btrfs root")
    end
    -- … btrfs-specific test body …
end)
```

### Logging

#### `t:log(msg)`

Append `msg` (any value, converted via `tostring`) to the test's log array. The log is included in the `FileOutcome.tests[i].log` and emitted alongside `TestPassed` / `TestFailed` events.

Useful for diagnostic output that should accompany failures. Unlike `print`, `t:log` is structured and tied to the specific test.

## Outcomes

A test ends in one of three states:

| Status | Triggered by |
|---|---|
| **Passed** | Test fn returned without raising AND no assertion failure was recorded. |
| **Failed** | Test fn raised (via `error()`, `t:assert*`, `t:fail`), or a sticky assertion failure was recorded before a pcall caught the raise. A Rust panic during the test also marks Failed and poisons every subsequent test in the file. |
| **Skipped** | `t:skip(reason)` was called, OR `meta.skip = true` / `meta.skip = "reason"`, OR `--tag` / `--no-tag` filtered the test out, OR `--include-slow` was off and `meta.slow = true`, OR file-scope `todo()` was called. |

A skipped test produces no event lifecycle other than `TestSkipped`. A failed test additionally captures the last 4 KiB of every booted VM's console log into the `TestFailed.console_excerpt` field — useful for diagnosing kernel-side regressions without reproducing the run.

## Time and timeouts

Per-test timeouts come from `meta.timeout`. File-default timeouts come from `provium.timeout`. Per-test wins.

The watchdog fires by tearing down the entire file's lab — there is no finer-grained cancellation in v1 because there is no cooperative cancel point in the AgentClient ops. **Practical implication:** for files using `provium.reset_between_tests = true` the lab is restored before the next test anyway, so the lab-wide tear-down is harmless. For files that share live state across tests, a per-test timeout invalidates the rest of the file. Don't put a `meta.timeout = N` on a flaky test in a state-sharing file expecting siblings to be unaffected.

## See also

- [Meta tags](/provium/reference/meta-tags.md) — full reference for `meta.slow`, `meta.tags`, `meta.skip`, `meta.timeout`, `meta.spec`.
- [provium global](/provium/reference/provium-module.md) — `provium.reset_between_tests`, `provium.timeout`.
- [The test function](/provium/writing-tests/the-test-function.md) — patterns for organising a test file.

---

# Meta tags

_Provium / Reference_

> The metadata table on test() — the well-known keys (slow, skip, tags, timeout, spec), arbitrary fields, and how --tag-meta filtering works.

Each `test(name, meta, fn)` call may include a metadata table. Provium's runner inspects a handful of well-known keys, but every key is preserved in the `MetaMap` it ships in `TestStarted` / `TestPassed` / `TestFailed` / `TestSkipped` events. Consumers like `provium-coverage` use this for spec linkage, classification, and filtering.

## Well-known keys

### `meta.slow`

```lua
test("big batch import", {slow = true}, function(t) … end)
```

When `--include-slow` is **not** passed to the CLI, tests with truthy `meta.slow` (`true` or any non-zero integer) are skipped with reason `filtered: slow tests skipped (pass --include-slow)`.

The default behaviour is "skip slow tests" so `provium tests/` is fast by default. CI and explicit slow runs pass `--include-slow`.

### `meta.skip`

```lua
test("not implemented yet", {skip = true}, function(t) … end)
test("waiting on RFC", {skip = "see issue #42"}, function(t) … end)
```

Declarative skip. Three accepted forms:

| Value | Skip reason |
|---|---|
| `true` (or any non-zero int) | `"skipped (meta.skip)"` |
| String `"<reason>"` | `"skipped: <reason>"` |
| `false` / `nil` / `0` | Not skipped. |

### `meta.tags`

```lua
test("dns lookup", {tags = {"net", "dns"}}, function(t) … end)
test("ipv6 only", {tags = "ipv6"}, function(t) … end)  -- single string also accepted
```

Tag filtering works through the CLI:

- `provium tests/ --tag net` — run only tests tagged `net`. Repeatable: `--tag a --tag b` is "a OR b".
- `provium tests/ --no-tag dns` — run every test EXCEPT those tagged `dns`. Repeatable; OR'd. Wins over `--tag` (intersect).

A single-string `tags = "smoke"` is accepted alongside the canonical array form. Without the single-string convenience, a one-tag test was silently filtered out by `--tag` with the misleading "did not match --tag" message instead of seeing its lone tag.

### `meta.subsystems` and other arbitrary fields

`meta.tags` is the de-facto field for orthogonal cross-cutting flags (`slow`, `flaky`, `perf`, `federation`). For declaring **what a test exercises** — particularly when test files are organised by user-visible scenario rather than by code subsystem — use a separate field with array values:

```lua
test("DNS service survives peinit restart", {
    subsystems = {"peinit", "loregd", "networking"},
    tags = {"dns", "services"},
    slow = true,
}, function(t) … end)
```

Filter via `--tag-meta KEY=VALUE`:

- `provium tests/ --tag-meta subsystems=peinit` — every test that touches peinit, regardless of where it lives in the directory tree.
- `provium tests/ --tag-meta subsystems=peinit --tag-meta subsystems=loregd` — touches peinit OR loregd (OR within key).
- `provium tests/ --tag-meta subsystems=peinit --tag-meta area=boot` — touches peinit AND is in the boot area (AND across keys).
- `provium tests/ --no-tag-meta flaky=true` — exclude anything tagged flaky=true.

`--tag-meta` accepts string-scalar (`{flaky = "true"}`) and array (`{subsystems = {"a", "b"}}`) meta values; nested-map and integer values don't match. The KEY can be any meta field name; the harness doesn't reserve `subsystems` or any other name.

The convention split:
- **`tags`** — orthogonal flags. "How does this test behave?" (slow, flaky, perf).
- **arbitrary keys (`subsystems`, `area`, `feature`)** — "What does this test exercise?" Filter via `--tag-meta`.

Why two mechanisms? `tags` is for things you'd cumulatively-enable ("run me everything tagged slow OR perf"). Arbitrary keys are for orthogonal axes you'd intersect ("run everything that touches peinit AND is in the boot area"). `--tag-meta`'s AND-across-keys is what makes intersection work.

### `meta.timeout`

```lua
test("flaky network probe", {timeout = "30s"}, function(t) … end)
test("quick", {timeout = 0.5}, function(t) … end)
```

Per-test wall-clock timeout. Numbers are seconds. Strings carry a suffix: `"500ms"` / `"5s"` / `"5m"` / `"2h"`. Per-test wins over the file-default `provium.timeout`.

When the timeout fires, the watchdog tears down the entire file's lab. See [test framework / Time and timeouts](/provium/reference/test-framework.md#time-and-timeouts) for the scope-limitation note.

### `meta.spec`

```lua
test("token issue follows §4.2.1", {spec = "PSD-KACS §4.2.1"}, function(t) … end)
```

Provium's runner does not interpret `meta.spec` — it ships the value through to consumers. `provium-coverage` reads it to map tests to spec-document sections for coverage reports.

By convention: a stable identifier that names the spec section a test exercises. Useful for cross-referencing tests against normative documents.

## Arbitrary keys

Any other key in the meta table is passed through to consumers as-is. Provium's `MetaValue` enum covers:

- `null` / `nil`
- bool
- int (i64)
- float (f64)
- string
- bytes (raw `Vec<u8>`)
- array of meta values
- string→meta map

Nested tables come through as either `Array` (when the table has integer keys 1..N) or `Map` (when string keys are present). The decision is made per nesting level; a mixed table is unusual and not specifically handled.

## Examples

### Multi-tag with skip-on-condition

```lua
test("federation handshake (multi-DC)", {
    tags = {"federation", "slow", "multi-dc"},
    timeout = "5m",
    spec = "PSD-FEDERATION §3.1",
}, function(t)
    if not have_two_dcs() then
        t:skip("requires two DCs")
    end
    -- … real test body …
end)
```

### Conditional gate via `wait_until`

```lua
test("sysctl knob takes effect", {
    spec = "PSD-PEINIT §6.5",
    timeout = "10s",
}, function(t)
    vm:run("sysctl -w kernel.shm_rmid_forced=1"):assert_ok()
    wait_until(function()
        return vm:read_file("/proc/sys/kernel/shm_rmid_forced") == "1\n"
    end, {timeout = "5s", desc = "sysctl knob propagated"})
end)
```

## See also

- [Test framework](/provium/reference/test-framework.md) — `test()`, the `t` context.
- [CLI](/provium/reference/cli.md) — `--tag`, `--no-tag`, `--tag-meta`, `--no-tag-meta`, `--include-slow`.
- [Events](/provium/reference/events.md) — how meta is emitted in test-lifecycle events.

---

# CLI

_Provium / Reference_

> Every flag, subcommand, environment variable, and exit code for the `provium` binary.

Provium's CLI surface is the single binary `provium`. By default it discovers `*.test.lua` files under the given paths and runs them; subcommands cover REPL, fixture management, and listing.

## Synopsis

```
provium [PATHS]... [FLAGS]
provium prepare [PROFILE]
provium repl <PROFILE> [--name <NAME>] [--fixture <PATH>]
provium console <PROFILE> [--mem <BYTES>] [--cpus <N>] [--cmdline <TEXT>] [--agent] [--qemu <PATH>] [--print-command] [-- QEMU_ARG...]
provium fixture list
provium fixture build <PATH>
provium fixture rebuild <PATH>
provium fixture clean
provium fixture stale
provium list [--fixtures]
provium lsp-setup [DIR] [--force]
```

If no paths are given, the current directory is scanned recursively.

## Top-level flags

### Discovery and selection

| Flag | Type | Default | Description |
|---|---|---|---|
| `--config <PATH>` | path | `provium.toml` | Config file location. |
| `--filter <STR>` | string | none | Only run files whose test-root-relative path contains this substring. |
| `--include-slow` | flag | off | Include `meta.slow` tests (default skips them). |
| `--tag <TAG>` | string, repeatable | none | Run only tests with one of these tags. OR'd. |
| `--no-tag <TAG>` | string, repeatable | none | Skip tests with any of these tags. Wins over `--tag`. |
| `--tag-meta <KEY=VALUE>` | string, repeatable | none | Run only tests where `meta[KEY]` contains `VALUE`. Multi-flag same KEY = OR within key; different KEYs = AND across keys. Useful for filtering on arbitrary fields like `subsystems`. |
| `--no-tag-meta <KEY=VALUE>` | string, repeatable | none | Skip tests where `meta[KEY]` contains `VALUE`. Same key/value semantics as `--tag-meta`. Wins over `--tag-meta`. |
| `--rerun-failed` | flag | off | Run only files that failed in the last run. Reads from `~/.cache/provium/rerun.json`. |
| `--since <PATH>` | path | none | Run only files whose mtime is newer than this reference file. |
| `--watch` | flag | off | Re-run on file change (poll every 500 ms). |

### VMM and resources

| Flag | Type | Default | Description |
|---|---|---|---|
| `--vmm <CHOICE>` | `qemu` / `local` | `qemu` | VMM backend. `local` is for dev runs without KVM — does not actually boot a kernel. |
| `--mem <BYTES>` | size string `"4G"` | 80 % of host RAM | Pool memory budget. |
| `--cpus <N>` | int | host online CPUs | Pool vCPU budget. |
| `--cpu-overcommit <F>` | float | `1.0` | Multiplier on `--cpus`. Clamped to [0.5, 8.0]. |
| `--no-preflight` | flag | off | Skip the startup `/dev/kvm` / iproute2 / nft / qemu / `CAP_NET_ADMIN` checks. |
| `--no-ksm` | flag | off | Skip Kernel Same-page Merging tuning at startup. |
| `--no-build` | flag | off | Skip every dynamic profile's `build` command for this run. Use when you know the artifacts are already current. Does not affect `provium prepare`, whose whole purpose is to build. |

### Lifecycle

| Flag | Type | Default | Description |
|---|---|---|---|
| `--timeout <DUR>` | seconds-int or duration string | `300` (5 min) | Per-file wall-clock timeout. `0` disables. Accepts `"500ms"`, `"30s"`, `"10m"`, `"2h"`. |
| `--fail-fast` | flag | off | Stop after the first failed file. |

### Output

| Flag | Type | Default | Description |
|---|---|---|---|
| `-v`, `--verbose` | flag | off | Show passing tests too. Mutually exclusive with `--quiet`. |
| `-q`, `--quiet` | flag | off | Show only failures. Mutually exclusive with `--verbose`. |
| `--json` | flag | off | Line-delimited JSON output, one object per file. Mutually exclusive with `--events-stdout`. |

### Observability

| Flag | Type | Description |
|---|---|---|
| `--save-events <PATH>` | path | Persist the event stream to PATH as length-prefixed msgpack frames. Compatible with `provium-coverage --from PATH`. |
| `--events-stdout` | flag | Emit msgpack event frames on stdout; the human/JSON renderer redirects to stderr. Mutually exclusive with `--json`. |
| `--events-socket <PATH>` | path | Multiplex the event stream over a Unix socket. The binary listens, accepts connections, fans out frames. Reused across `--watch` iterations. |
| `--coverage` | flag | Pipe the buffered event stream into `provium-coverage` (must be on `PATH`) after the run. |

## Subcommands

When a subcommand is given, the test-runner mode is suppressed.

### `provium prepare [PROFILE]`

Run dynamic profiles' [`build`](/provium/configuration/provium-toml.md#build) commands without booting anything. With a `PROFILE` argument, builds just that profile (and errors with `provium prepare: profile \`<name>\` not found in provium.toml` if it isn't declared); with no argument, builds every profile that declares a `build` command, in name order, stopping at the first failure.

Use it to pre-warm artifacts — `provium prepare && provium --no-build` builds once, then runs the suite without rebuilding — or to drive an image build from a machine that isn't a test host: `prepare` skips the pre-flight checks (`/dev/kvm`, iproute2, `CAP_NET_ADMIN`), so it runs anywhere `sh` and the builder do.

A profile with no `build` command is a silent no-op. A build that exits non-zero aborts with the profile name and exit status, and Provium never falls through to using whatever stale artifacts are on disk.

### `provium repl <PROFILE> [--name <NAME>] [--fixture <PATH>]`

Boot a VM and drop into an interactive Lua REPL against it.

| Arg / flag | Description |
|---|---|
| `<PROFILE>` | Profile name from `provium.toml`. Optional when `--fixture` is given. |
| `--name <NAME>` | VM name. Defaults to `repl`. |
| `--fixture <PATH>` | Resume from a fixture instead of cold-booting. |

If `--fixture` is set without a profile arg, the first profile in `provium.toml` (sorted by name) is used.

### `provium console <PROFILE> [flags] [-- QEMU_ARG...]`

Boot a profile's VM and attach your terminal to its serial console — no agent, no Lua. `Ctrl-A X` quits QEMU; `Ctrl-A C` toggles the QEMU monitor.

| Flag | Description |
|---|---|
| `--mem <BYTES>` | Override the per-VM memory default. |
| `--cpus <N>` | Override the per-VM vCPU default. |
| `--cmdline <TEXT>` | Override the profile's kernel command line. |
| `--agent` | Inject the agent overlay (off by default for console sessions). |
| `--qemu <PATH>` | Pick the QEMU binary. |
| `--print-command` | Print the assembled QEMU command line and exit. |

Anything after `--` is passed verbatim to QEMU.

### `provium fixture list`

Show every cached entry with size and key.

### `provium fixture build <PATH>`

Force a build for the named fixture (test-root-relative path, no `.fixture.lua` suffix). If the fixture is already cached, prints `already built: <path>` and exits.

### `provium fixture rebuild <PATH>`

Evict the existing cache entry and rebuild. Both single-VM (`.snap`) and lab (`.lab/`) layouts are evicted.

### `provium fixture clean`

Wipe the entire cache directory.

### `provium fixture stale`

List `.fixture.lua` files whose source hash (folded with kernel + initrd identifiers and any [external host-file deps](/provium/running-tests/fixtures-and-dependencies.md#external-host-file-deps) declared via `vm:push_file` / `lab:depends_on_file`) doesn't correspond to any cached entry. Useful for "what would `provium` rebuild on the next run?"

### `provium list [--fixtures]`

List discovered tests (default) or fixtures (`--fixtures`) without running anything. Useful for piping into `xargs` or for CI dry-runs.

### `provium lsp-setup [DIR] [--force]`

Drop Lua Language Server definitions into a test directory so `test`, `provium`, `wait_until`, `json`, and the rest of the harness globals stop showing up as undefined in your editor.

Writes:

- `<DIR>/.provium-meta/types.lua` — `---@meta` stubs covering the test framework, the `TestContext` (`t:assert_eq`, `t:fail`, …), the root `Lab`, and the `json` global. Less-trafficked methods are typed as `any` — the goal is silencing diagnostics, not full IDE intellisense.
- `<DIR>/.luarc.json` — config pointing the LSP at `.provium-meta/`.

`DIR` defaults to the current directory. If `.luarc.json` already exists, the command refuses to overwrite and prints the snippet to merge in by hand; pass `--force` to replace it.

Skips pre-flight checks (`/dev/kvm`, iproute2, nft) so it runs cleanly on machines without test prerequisites — useful from a dev laptop separate from the test host.

After running, reload your editor's Lua server. To extend the types further (e.g. richer completion on a specific userdata you use a lot), edit `.provium-meta/types.lua` directly.

## Exit codes

| Code | Meaning |
|---|---|
| `0` | Every file passed (or skipped). |
| `1` | At least one file did not finish cleanly (failed test, panicked runner, timed-out file). With `--coverage`, also reflects the coverage post-run's exit code if it failed. |
| `2` | Internal error — config load failure, pre-flight failure, a dynamic profile's `build` command failing, or a panic in the dispatcher. |

The exit code is intentionally clamped to `0/1/2` rather than reflecting the number of failed files, so shell scripts don't have to worry about overflow at the 256-file mark or accidentally interpret `failed_count == 2` as the internal-error sentinel.

## Environment variables

| Variable | Effect |
|---|---|
| `PROVIUM_TEST_FILTER` | JSON object set by the binary at startup carrying `--include-slow` / `--tag` / `--no-tag` / `--tag-meta` / `--no-tag-meta` so the runner picks them up. Test code does not set this directly. |
| `PROVIUM_OVERLAY` | Path to the agent-overlay cpio. Overrides the binary-relative path-walk; lower priority than the per-profile `agent_overlay_path` config field. Useful for distribution-installed Provium where the overlay isn't co-located with the binary. |
| `PROVIUM_RERUN_STATE` | Override the path of the rerun-state file (default: `~/.cache/provium/rerun.json`, or `/tmp/provium-rerun.json` if `$HOME` is unset). |
| `PROVIUM_COVERAGE_TMP` | Set by `--coverage` to point at the temp event file the post-run hook will read. Cleared after the hook completes. |
| `PROVIUM_COVERAGE_USER_FILE` | Set when `--coverage` reuses an explicit `--save-events` path so the post-run hook does NOT delete it. |
| `PROVIUM_COVERAGE_MARKER` | Marker file `--coverage` uses to recognise its own temp file (vs an externally-set `PROVIUM_COVERAGE_TMP` pointing at user data). |
| `HOME` | Used to compute the default rerun-state path. |
| `PATH` | Searched for `qemu-system-x86_64`, `ip`, `tc`, `nft`, `tcpdump`, and (on `--coverage`) `provium-coverage`. |

## Output shape

### Plain text (default)

```
PASS tests/smoke.test.lua (3 passed, 0 failed, 0 skipped)
FAIL tests/networking/partition.test.lua (2 passed, 1 failed, 0 skipped)
    FAIL split-brain
        client A could not reach client B after partition
TIME tests/long.test.lua (1 passed, 0 failed, 0 skipped)

3 file(s); 6 passed, 1 failed, 0 skipped; 12.34s
```

`-v` adds `PASS <name>` / `SKIP <name>` lines for each test. `-q` suppresses everything except the file-status line for files with failures and the summary.

### JSON (`--json`)

One object per file, line-delimited. Schema:

```json
{
  "path": "tests/smoke.test.lua",
  "timeout": "in_time",
  "passed": true,
  "chunk_error": null,
  "tests": [
    {"name": "boots", "status": "passed", "message": null, "log": []},
    {"name": "fails", "status": "failed", "message": "1 ~= 2", "log": []}
  ]
}
```

`status` is one of `"passed"`, `"failed"`, `"skipped"`. `timeout` is `"in_time"` or `"timed_out"`.

### Msgpack events (`--events-stdout`)

Length-prefixed msgpack frames, one per `EventFrame` (see [events](/provium/reference/events.md)). When `--events-stdout` is set, the human-readable / JSON renderer redirects to stderr so consumers piping `provium --events-stdout | provium-coverage` don't see human text interleaved into their msgpack parser.

## Filter precedence

Multiple selection flags compose:

1. Discovery: walk `<paths>` for `*.test.lua` files.
2. `--filter` substring match against test-root-relative path.
3. `--rerun-failed` intersect with the prior failed set (zero-result clean exit if no prior state).
4. `--since` mtime newer than the reference path.
5. Per-test (within each file): `meta.skip` → instant skip.
6. Per-test: `meta.slow` and `--include-slow`.
7. Per-test: `--no-tag` (skip if any match).
8. Per-test: `--tag` (run if any match).
9. Per-test: `--no-tag-meta KEY=VALUE` (skip if `meta[KEY]` contains VALUE).
10. Per-test: `--tag-meta KEY=VALUE` (run if all KEYs have at least one matching VALUE).

`--watch` mode rediscovers from scratch every iteration; `--rerun-failed` is automatically dropped under `--watch` so file-edit detection works as expected.

## See also

- [provium.toml reference](/provium/configuration/provium-toml.md) — config-file fields.
- [Events](/provium/reference/events.md) — wire format of the msgpack event stream.
- [Running tests](/provium/running-tests/the-cli.md) — patterns for the CLI.

---

# Events

_Provium / Reference_

> Every event variant and field in Provium's msgpack event stream — file lifecycle, test outcomes, VM spawns, pool state, claims, and fixture builds.

Provium's host-side scheduler and runners emit observability events that drive the human-readable summary, the `--json` output, `provium-coverage`, and any other consumer that wants to follow what the harness is doing.

The transport is **length-prefixed msgpack frames** (the same framing as the wire protocol). Get them via `--save-events <path>`, `--events-stdout`, or `--events-socket <path>`.

## Frame shape

Each frame is an `EventFrame`:

```
{
  "ts":      <i64>,    // ns since Unix epoch, host clock at emission
  "kind":    "<name>", // discriminator (snake_case)
  "payload": { … }     // variant-specific
}
```

The `Event` enum is flattened into the envelope so `kind` and `payload` appear at the top level of each frame.

`PROTOCOL_VERSION` (currently `1`) is bumped on any wire-shape change. Adding a new variant or field is a strict version bump; removing or renaming one is breaking. See [protocol version](/provium/reference/protocol-version.md) for the pinning policy.

## File lifecycle

### `file_discovered`

Emitted once per `*.test.lua` file before any dispatching begins.

| Field | Type | Description |
|---|---|---|
| `path` | string | Test-root-relative path. |
| `fixture_refs` | array of strings | Fixtures referenced by `vm_fixture(...)` / `lab_fixture(...)` (path with `.fixture.lua` stripped). |
| `declared_claim` | optional `ResourceAmount` | `provium:claim(...)` declared at file scope, if any. |

### `file_dispatched`

A test file was picked up by a runner thread.

| Field | Type | Description |
|---|---|---|
| `path` | string | File path. |
| `reservation` | `ResourceAmount` | Resources reserved at dispatch (sum of runner overhead + claim). |

### `file_blocked`

A test file is waiting on the resource pool or PSI pressure.

| Field | Type | Description |
|---|---|---|
| `path` | string | File path. |
| `waiting_for` | `ResourceAmount` | What the file is waiting on. |
| `reason` | string | `pool_full`, `psi_pressure`, etc. |

### `file_completed`

A test file finished — successfully, with failures, or terminated by timeout / panic.

| Field | Type | Description |
|---|---|---|
| `path` | string | File path. |
| `status` | enum | `passed`, `failed`, `timed_out`, `crashed`. |
| `duration_ns` | u64 | Wall-clock duration. |

`crashed` means the runner panicked or hit a test-infrastructure failure; remaining tests in the file are marked `failed-due-to-poisoned-state`.

## Test lifecycle

### `test_started`

A `test()` block within a file started.

| Field | Type | Description |
|---|---|---|
| `path` | string | Containing file's path. |
| `name` | string | `test()` block name. |
| `meta` | `MetaMap` | Per-test metadata. Always present; empty if none. |

### `test_skipped`

The test was filtered or marked Skipped (declarative `meta.skip`, inline `t:skip()`, file-scope `todo()`, or tag/slow filter).

| Field | Type | Description |
|---|---|---|
| `path` | string | Containing file's path. |
| `name` | string | Test name. |
| `reason` | string | Filter expression, `t:skip()` argument, or `todo()` argument. |
| `meta` | `MetaMap` | Per-test metadata. |

### `test_passed`

| Field | Type | Description |
|---|---|---|
| `path` | string | File path. |
| `name` | string | Test name. |
| `duration_ns` | u64 | Test wall-clock duration. |
| `meta` | `MetaMap` | Per-test metadata. |

### `test_failed`

| Field | Type | Description |
|---|---|---|
| `path` | string | File path. |
| `name` | string | Test name. |
| `duration_ns` | u64 | Wall-clock up to failure. |
| `reason` | string | Assertion message, exception text, or panic payload. |
| `console_excerpt` | string | Last 4 KiB from each booted VM's console log. May be empty. |
| `meta` | `MetaMap` | Per-test metadata. |

## VM lifecycle

### `vm_spawned`

| Field | Type | Description |
|---|---|---|
| `file` | string | File that owns this VM. |
| `vm_name` | string | Name as given to `lab:vm`. |
| `profile` | string | Profile used to boot. |
| `memory_bytes` | u64 | Memory cap. |
| `cid` | u32 | Assigned vsock CID. |

### `vm_shutdown`

| Field | Type | Description |
|---|---|---|
| `file` | string | File that owned this VM. |
| `vm_name` | string | VM name. |
| `duration_ns` | u64 | VM uptime. |

## Pool and claims

### `pool_state`

Periodic snapshot of resource-pool usage. Default cadence: 1 Hz.

| Field | Type | Description |
|---|---|---|
| `used` | `ResourceAmount` | Currently in use. |
| `available` | `ResourceAmount` | Currently free. |

### `claim_acquired`

A file successfully claimed resources via `provium:claim(...)`.

| Field | Type | Description |
|---|---|---|
| `path` | string | File path the claim belongs to. |
| `amount` | `ResourceAmount` | Amount claimed. |

### `claim_released`

A file released its claim. Pairs with `claim_acquired` on the same `path`.

| Field | Type | Description |
|---|---|---|
| `path` | string | File path. |
| `amount` | `ResourceAmount` | Amount released. |

## Fixtures

### `fixture_build_started`

| Field | Type | Description |
|---|---|---|
| `path` | string | Fixture path (test-root-relative, no `.fixture.lua`). |

### `fixture_build_done`

| Field | Type | Description |
|---|---|---|
| `path` | string | Fixture path. |
| `duration_ns` | u64 | Build duration. |
| `snapshot_bytes` | u64 | Cached snapshot size, post-compression. |

### `fixture_build_waiting`

A second file is waiting on the build lock another file holds.

| Field | Type | Description |
|---|---|---|
| `path` | string | Fixture path being waited on. |
| `held_by_file` | string | File currently holding the build lock. |

### `fixture_cache_hit`

A fixture was resumed from its cached snapshot — no rebuild required.

| Field | Type | Description |
|---|---|---|
| `path` | string | Fixture path. |

## Shared types

### `ResourceAmount`

```
{
  "memory_bytes": <u64>,
  "cpus":         <u32>
}
```

Used in pool / claim / reservation events.

### `MetaMap` and `MetaValue`

`MetaMap` is `BTreeMap<String, MetaValue>`. `MetaValue` is a tagged-by-type union over native msgpack types:

| Variant | msgpack mapping |
|---|---|
| `Null` | unit / nil |
| `Bool` | bool |
| `Int` | i64 |
| `Float` | f64 |
| `Str` | str |
| `Bytes` | bin (distinct from str) |
| `Array` | array of MetaValue |
| `Map` | map of String → MetaValue |

The deserialiser dispatches on the input type rather than relying on declaration-order fallthrough, so valid-UTF-8 byte sequences are not mis-classified as strings.

## Event guarantees

- `file_discovered` for every discovered file, before any dispatching.
- For each file: at most one `file_dispatched`, optional `file_blocked`(s) before it, exactly one `file_completed` after.
- For each test: exactly one `test_started`, then exactly one of `test_passed` / `test_failed` / `test_skipped`.
- `claim_acquired` and `claim_released` are paired per file.
- `fixture_build_started` and `fixture_build_done` are paired per build. A `fixture_build_waiting` may precede the `_done` if the file queued behind a peer.
- `fixture_cache_hit` fires AFTER successful restore — never on a corrupt entry that turns into a rebuild.

## Consuming the stream

The three output channels — `--save-events <path>` (file), `--events-stdout` (pipe), and `--events-socket <path>` (multi-consumer Unix socket) — and the patterns for choosing between them are covered in [events and coverage](/provium/running-tests/events-and-coverage.md#output-channels).

## See also

- [CLI](/provium/reference/cli.md) — flags that produce / consume events.
- [Protocol version](/provium/reference/protocol-version.md) — wire-shape pinning policy.
- [Events and coverage](/provium/running-tests/events-and-coverage.md) — patterns for using the event stream.

---

# Protocol version

_Provium / Reference_

> PROTOCOL_VERSION pins the host-agent wire shape and the event stream — what it pins, when it bumps, and how the pin tests catch silent breakage.

Provium has one wire-version constant: `PROTOCOL_VERSION`, currently `1`.

It pins:

- The host ↔ agent protocol — every op, every result variant, every error shape.
- The observability event stream — every `Event` variant and every payload field.
- The Hello / HelloOk / HelloErr handshake and the `OpenMode` field set.

## When it bumps

`PROTOCOL_VERSION` increments any time:

- A new wire variant is added (op or event).
- A field is added to an existing payload struct.
- A new `OpenMode` flag, `ExecResult` variant, or `OpResult` shape lands.
- The `EventFrame` envelope changes.

It is **not bumped** for changes that don't reach the wire — internal struct renames, host-side refactors, agent-side optimisation, additional Lua bindings that map onto existing wire ops.

## What it does NOT pin

- The Lua API surface (the userdata methods `vm`, `bridge`, etc.). That surface evolves freely; tests that use it are rebuilt against the same Provium version that runs them.
- Internal types like `Vm`, `Lab`, `Bridge`. Those are language-level objects, not wire-level contracts.
- The CLI flag set. New flags can be added without bumping the protocol; removing or renaming a flag is a separate compatibility concern documented in [the CLI reference](/provium/reference/cli.md).

## How the pin is enforced

The conformance suite locks every wire-facing struct against an explicit byte expectation. Examples of what it locks:

- `Hello.protocol_version` is a u32 currently encoded as `1`.
- `HelloOk` field tags (`agent_version`, `guest_os`, `agent_features`, …).
- `OpResult.outcome` is the `"ok"` / `"err"` discriminator key, with `value` for the payload.
- `SyscallResult` field set: `ret`, `errno`, optional `out_bufs` (omitted when empty).
- `OpenMode`'s six flags — `read`, `write`, `create`, `truncate`, `append`, `exclusive` — each serialise as their own named field (it is a struct of booleans, not a packed bitfield).

Any refactor that changes a tag name, reorders an enum, or shifts a bit fails the pin test. The fix is to bump `PROTOCOL_VERSION` and update the pin to match the new shape — explicit, not silent.

## How to add a new op or event

1. Add the new variant / payload type to the protocol crate.
2. Add a serialisation round-trip test alongside it.
3. Bump `PROTOCOL_VERSION`.
4. Update the conformance pin to lock the new shape.
5. (Op only) Wire the agent-side handler.
6. (Op only) Wire the host-side dispatcher and a Lua binding.

Skipping step 4 produces a protocol change without explicit pinning — exactly the regression the pin tests are designed to catch.

## Backwards compatibility

Provium does not commit to backwards compatibility on the host-agent wire. The host and agent are versioned together — they ship in lock-step out of the same workspace. The Hello handshake exchanges versions and `HelloErr`s the connection if they don't match, so a stale agent against a fresh host fails fast at connection time, not silently mid-op.

For the event stream, consumers (`provium-coverage`, dashboards) are versioned against `PROTOCOL_VERSION`. A consumer built against version N reading a stream from version N+1 will fail to deserialise the new fields; the typical recovery is to rebuild the consumer.

## See also

- [Events](/provium/reference/events.md) — what the event stream carries.
- [CLI](/provium/reference/cli.md) — flags that produce events.