# Getting started

---

# What is Provium

_Provium / Getting started_

> Provium is a KVM-backed test harness for kernel and system code. You write tests in Lua, and they run inside real virtual machines against a real Linux kernel.

Provium is a test harness for software that needs a real operating system underneath it. It runs each test file in its own Lua state, boots real VMs on demand, and exposes a Lua API for driving the guest (commands, files, syscalls, networking, faults), and reports results through the same `passed`/`failed`/`skipped` shape any other test runner would.

It is the runner Peios uses for kernel-side and system-service tests, but there is nothing Peios-specific in the harness — any guest that ships the Provium agent binary works.

## Why Provium exists

Most kernel-adjacent code is hard to test. Unit tests can verify pure logic, but they cannot exercise:

- Driver behaviour against real hardware abstractions (disks, NICs, clocks).
- Networking behaviour under partition, latency, packet loss, or bandwidth caps.
- Multi-host topologies — two guests on the same bridge, three guests in a cluster.
- The boot path itself.
- Failure injection (EIO, slow I/O, link-down, kernel panic recovery).
- Snapshots, restores, and time travel.

The alternatives — containers, mock kernels, in-process simulators — each give up something important. Provium chooses the path of "use a real VM, make it cheap to spawn, and put a high-leverage API on top."

A single test file looks like this:

```lua
test("guest can ping its bridge peer", function(t)
    local lan = provium:bridge("lan")
    local a = provium:vm("a", "peios"):boot()
    local b = provium:vm("b", "peios"):boot()
    lan:attach({a, b})
    a:run("ip addr add 10.0.0.1/24 dev eth0 && ip link set eth0 up")
    b:run("ip addr add 10.0.0.2/24 dev eth0 && ip link set eth0 up")
    a:run("ping -c 1 -W 1 10.0.0.2"):assert_ok()
end)
```

When the file runs, Provium boots two VMs, sets up a real Linux bridge with TAP interfaces, attaches both, runs the commands inside the guests, and tears everything down at the end. No mocks, no shims. The two `ip addr` lines are ordinary guest commands: Provium wires layer 2 and leaves IP addressing to the guest.

## What Provium gives you

| Capability | What it does |
|---|---|
| **VM lifecycle** | Boot, snapshot, restore, pause, resume, reset, power-button. Snapshots survive across files via the fixture cache. |
| **Layer-1 ops** | `vm:run("cmd")`, `vm:read_file`, `vm:write_file`, `vm:stat`, `vm:mkdir` — the things you'd normally do over SSH, but driven through a vsock agent so they stay fast and self-contained. |
| **Layer-0 ops** | `vm:syscall`, `vm:ioctl` — direct invocations against the guest kernel, with byte-buffer support for in/out parameters. |
| **File handles** | Open, read, write, seek, tell, close, tail. Mirrors POSIX semantics. |
| **Async processes** | `vm:run_async` returns a `Process` userdata you can `:wait`, `:kill`, `:signal`, write `stdin` into, and stream `stdout`/`stderr` from. |
| **Workers** | `vm:spawn_worker()` lets a test concurrently exercise the guest from multiple agent connections without spinning up another VM. |
| **Networking** | Real Linux bridges with TAP interfaces. `bridge:partition`, `bridge:add_latency`, `bridge:drop_rate`, `bridge:bandwidth_limit`, `bridge:isolate`, `bridge:capture` (pcap), and uplink/NAT for outbound traffic. |
| **Disks** | Attach images, read sectors, write sectors, inject `eio_read` / `eio_write` / `slow` faults. |
| **Console** | Read the boot log, stream the chardev, write input. Useful for tests that exercise early-boot behaviour or interactive prompts. |
| **Clock control** | `vm:clock():set`, `:advance`, `:sleep`. Tests that depend on time can move time deterministically. |
| **Streams** | `Tail`/`Capture`/`Console` streams all share `next` / `read_until` / `expect` / `drain` / `close` / `eof` so log-watching, pcap-watching, and console-watching all feel the same. |
| **Fixtures** | `provium:vm_fixture("base")` builds a snapshot once, caches it on disk, and restores it for every test that asks for it. Lab fixtures (`provium:lab_fixture(...)`) cache whole multi-VM topologies the same way. |
| **Resource pool** | A scheduler with memory and CPU budgets. Test files declare what they need with `provium:claim({memory="2G", cpus=4})`; the scheduler runs as many in parallel as the budget allows. |
| **Observability** | Every host-side action emits a structured msgpack event. Pipe it to `provium-coverage`, save to a file, multiplex over a Unix socket, or just watch the human-readable summary. |
| **Determinism aids** | `boot_opts.rng_seed`, `boot_opts.initial_time`, fixture-cache keying that folds in kernel + initrd identity. |

## How it compares

| | Provium | LXC / Docker | KUnit | Mocked I/O |
|---|---|---|---|---|
| **Real kernel** | Yes (per VM) | Shared with host | Yes (single test kernel) | No |
| **Driver-level testing** | Yes | Limited | Limited | No |
| **Network impairments** | Built-in | External tools | No | No |
| **Multi-host topologies** | Built-in | Compose / k8s | No | No |
| **Snapshot + restore** | Built-in (fixtures) | Manual | No | N/A |
| **Fault injection** | Built-in (`fault_inject`, `clock:advance`) | Limited | Limited | Yes |
| **Per-test isolation** | Fresh VM per file | Container per test | Test-binary boundary | Process |
| **Wire protocol exposed** | Yes (`vm:syscall`, `vm:ioctl`) | No | Direct in-kernel | N/A |
| **Test language** | Lua 5.4 | Shell / Go / Python | C | Any |
| **Dependencies** | QEMU, KVM, iproute2, nftables | Docker daemon, etc. | Kernel build | Test framework |

## How it works

```mermaid
flowchart LR
    cli[provium CLI] --> disp[Dispatcher]
    disp --> runner1[File runner #1]
    disp --> runner2[File runner #2]
    runner1 --> lua1[Lua state]
    runner2 --> lua2[Lua state]
    lua1 -->|host bindings| host1[VM, Bridge, Lab]
    host1 -->|QEMU launch| vm1[Guest VM]
    vm1 -->|vsock| agent1[provium-agent]
    agent1 -->|exec, syscall, file ops| guest_kernel[Guest kernel]
```

When you run `provium tests/`, the CLI:

1. Walks `tests/` for `*.test.lua` files.
2. Hands each file to the dispatcher, which acquires resources from the pool.
3. Spins up a Lua state per file, installs the `provium` global, and executes the file.
4. Inside the file, calls like `provium:vm("a", "peios"):boot()` launch real QEMU processes and hand back userdata wrappers around them.
5. Operations against those VMs (`vm:run`, `vm:read_file`, `vm:syscall`) are dispatched over vsock to the `provium-agent` running in the guest.
6. At file end (or per-test if `provium.reset_between_tests = true`), the resource graph is walked in reverse-dependency order and every stream, process, file, worker, VM, and bridge is closed.

The whole loop is observable: every VM spawn, every file dispatched, every fixture cache hit, every test pass/fail, every pool-state snapshot becomes an event on the wire.

## What's not here

- **No browser / GUI testing.** Provium drives guests through agent ops, not graphical interfaces.
- **No Windows guests.** The agent is Linux-only in v1.
- **No live cluster orchestration.** Provium is a test harness, not Terraform.
- **No mock VMM.** Tests run against real KVM. The `--vmm local` mode exists for ad-hoc dev runs without KVM but does not boot a kernel — it just gives the host bindings something to dispatch against.

---

# Quick start

_Provium / Getting started_

> Install Provium, configure a profile, write a first test, and run it. Five minutes from a clean checkout to a passing test.

This guide walks through the minimum setup needed to run one Provium test against a real VM. It assumes you have KVM, QEMU, iproute2, and nftables installed; see [project structure](/provium/getting-started/project-structure.md) for prerequisites detail.

## Install

Provium is a Cargo workspace. Build the host binary:

```
cd provium
cargo build --release --bin provium
```

The binary lands at `target/release/provium`. Optionally, install it onto your `PATH`:

```
cargo install --path provium-host --bin provium
```

Verify:

```
provium --help
```

## Pre-flight check

Provium runs a startup pre-flight on every invocation that can reach a VM (only `lsp-setup` and `prepare` skip it). There is no standalone pre-flight command, so any cheap run — `provium list`, say — exercises it before you start writing tests. To skip the checks (useful in containers without KVM):

```
provium tests/ --no-preflight
```

The pre-flight checks for:

| Check | Recovery |
|---|---|
| `/dev/kvm` exists and is openable | `sudo modprobe kvm-intel` (or `kvm-amd`); add yourself to the `kvm` group |
| `/dev/vhost-vsock` exists | `sudo modprobe vhost_vsock` |
| `ip` and `tc` on `PATH` | `apt install iproute2` / `pacman -S iproute2` |
| `nft` on `PATH` | `apt install nftables` / `pacman -S nftables` |
| `qemu-system-x86_64` on `PATH` | `apt install qemu-system-x86_64` / `pacman -S qemu-base` |
| Effective `CAP_NET_ADMIN` | `sudo setcap cap_net_admin,cap_net_raw=eip $(which provium)` |

## Provide a kernel and initrd

Provium boots VMs by direct kernel boot (`-kernel` / `-initrd`). You need:

- A bzImage-format kernel.
- An initramfs with a working `/init`. Almost any initrd works — a vanilla distro initramfs, a buildroot image, a from-scratch cpio with just `/init` and busybox.

Provium injects its own agent into your initrd at launch (concatenated cpio, content-hash cached) and chains to your `/init`. You don't need to bake `provium-agent` into your image. To opt out (e.g. when your image already bundles an agent), set `inject_agent = false` on the profile — see [provium.toml reference](/provium/configuration/provium-toml.md).

For Peios, the kernel + initrd are built by the Peios image-build pipeline. For ad-hoc use, see the [project structure](/provium/getting-started/project-structure.md) section.

## Create `provium.toml`

Provium loads `./provium.toml` (override with `--config`). The minimum useful config declares one profile:

```toml
[provium]
roots = ["tests"]

[profiles.peios]
kernel  = "/path/to/bzImage"
initrd  = "/path/to/provium-initrd.cpio.gz"
cmdline = "console=ttyS0 quiet"
```

`[profiles.<name>]` is the dictionary `provium:vm("name", "<profile>")` looks up. The `roots` setting is the list of directories scanned for `*.test.lua` and `*.fixture.lua` files; it also controls where `require("helper")` resolves.

See [provium.toml reference](/provium/configuration/provium-toml.md) for every field.

## Write your first test

Create `tests/smoke.test.lua`:

```lua
test("guest boots and runs commands", function(t)
    local vm = provium:vm("smoke", "peios"):boot()
    local r = vm:run("uname -a")
    r:assert_ok()
    t:assert(r.stdout:find("Linux"), "uname should report Linux: " .. r.stdout)
end)

test("filesystem ops round-trip", function(t)
    local vm = provium:vm("smoke", "peios"):boot()
    vm:write_file("/tmp/hello", "world")
    t:assert_eq(vm:read_file("/tmp/hello"), "world")
end)
```

Two things to know:

- Each `test(...)` body runs in its own scope. The two `provium:vm("smoke", "peios")` calls above each create a fresh VM in their respective test scope; the `"smoke"` name in test 1 and test 2 are independent and don't collide. Each VM is silently shut down at end of test.
- To share a VM across tests, declare it at file top-level (`local vm = provium:vm("v", "peios"):boot()`) and either capture the userdata as a Lua local or look it up from inside tests via the 1-arg form `provium:vm("v")`.

## Run it

```
provium tests/
```

You should see:

```
PASS tests/smoke.test.lua (2 passed, 0 failed, 0 skipped)

1 file(s); 2 passed, 0 failed, 0 skipped; 3.42s
```

Useful flags for the development loop:

- `provium tests/ --watch` — re-runs on file change (poll-based, 500 ms).
- `provium tests/smoke.test.lua` — run a single file.
- `provium tests/ --filter smoke` — substring match against the relative path.
- `provium tests/ --rerun-failed` — re-run only files that failed last time.
- `provium tests/ -v` — show passing tests too.
- `provium tests/ --json` — line-delimited JSON output, one object per file.

See [the CLI reference](/provium/reference/cli.md) for every flag.

## Add an assertion that should fail

Sanity-check the failure path:

```lua
test("fails on purpose", function(t)
    t:assert_eq(1, 2, "1 must equal 2")
end)
```

```
FAIL tests/smoke.test.lua (2 passed, 1 failed, 0 skipped)
    FAIL fails on purpose
        1 must equal 2: 1 ~= 2
```

Provium's exit code is `0` when every file passed (or skipped), `1` when one or more files had a failure or timed out, and `2` for internal errors (config load failure, pre-flight failure, etc.).

## What's next

- Read [project structure](/provium/getting-started/project-structure.md) to understand how `provium.toml`, `tests/`, fixtures, and the cache directory all fit together.
- Read the [test-framework reference](/provium/reference/test-framework.md) for everything the `test()` and `t` API expose.
- Read [VMs and profiles](/provium/writing-tests/vms-and-profiles.md) for the full set of `provium:vm(...)` options.
- Skim the [CLI reference](/provium/reference/cli.md) to see what's available for the running side.

---

# Project structure

_Provium / Getting started_

> A Provium project is a directory with provium.toml, test roots holding *.test.lua and *.fixture.lua files, and a per-user fixture cache. Nothing else.

Provium has no scaffolding command. A project is whatever directory `provium.toml` lives in. This page documents what each piece does.

## Directory layout

```
my-project/
  provium.toml                  # Required. Profiles + scan roots.
  tests/                        # One of the directories listed in `roots`.
    smoke.test.lua              # Run by `provium`.
    networking/
      partition.test.lua
      uplink.test.lua
    helpers/
      assert_pingable.lua       # `require("helpers.assert_pingable")`
    fixtures/
      booted-pair.fixture.lua   # Built once, restored per test.
  ~/.cache/provium/
    fixtures/                   # Per-user fixture cache.
      <hash>.snap               # Single-VM fixture snapshots.
      <hash>.lab/               # Multi-VM lab-fixture directories.
    rerun.json                  # `--rerun-failed` state.
```

The only required file is `provium.toml`. Tests, helpers, and fixtures all live under whatever directory you list in `[provium].roots`.

## `provium.toml`

The configuration file at the project root. Two sections:

```toml
[provium]
roots = ["tests", "vendor/upstream-tests"]
cache_dir       = "/var/cache/provium/fixtures"   # optional
cache_max_size  = "20G"                            # optional

[profiles.peios]
kernel   = "/build/peios/bzImage"
initrd   = "/build/peios/initrd.cpio.gz"
cmdline  = "console=ttyS0 quiet"
guest_os = "peios"

[profiles.peios-debug]
kernel   = "/build/peios-debug/bzImage"
initrd   = "/build/peios-debug/initrd.cpio.gz"
cmdline  = "console=ttyS0 debug loglevel=7"
```

`[profiles.<name>]` blocks declare named (kernel, initrd, cmdline) tuples. Test code looks them up by name: `provium:vm("a", "peios")` boots the VM using `[profiles.peios]`. You can have any number of profiles; tests pick whichever they need.

`[provium].roots` is the list of directories scanned for `*.test.lua` files. It also controls where `require("helper.module")` resolves; `helpers/foo.lua` under any root is reachable as `require("helpers.foo")`.

`cache_dir` and `cache_max_size` configure where fixture snapshots are stored and how big the cache is allowed to grow. Defaults: `~/.cache/provium/fixtures/`, `20 GiB`. See [provium.toml reference](/provium/configuration/provium-toml.md) for full detail.

## Test files

Files matching `*.test.lua` under any `roots` directory are picked up by `provium`. Each file is run in a fresh Lua state with a fresh root `Lab`.

A test file consists of `test(name, [meta,] fn)` calls. Tests run in declaration order, and each test runs sequentially (one at a time within a file). The harness builds a fresh `t` context for each test and passes it as the function's first argument.

```lua
test("simple", function(t)
    t:assert_eq(1 + 1, 2)
end)

test("with metadata", {tags = {"smoke"}, timeout = "30s"}, function(t)
    t:assert(true)
end)

test("declaratively skipped", {skip = "still figuring out the spec"}, function(t)
    -- never runs
end)
```

See [the test-framework reference](/provium/reference/test-framework.md) for `test()`, the `t` context, `todo()`, and `wait_until()`.

## Fixture files

Files matching `*.fixture.lua` are not picked up by the test runner directly. Instead, test files reference them by name:

```lua
test("uses a pre-booted VM", function(t)
    local vm = provium:vm_fixture("booted-base")
    -- vm is restored from a cached snapshot
end)
```

When the test runs, Provium:

1. Locates `<root>/booted-base.fixture.lua` (under any `roots` directory).
2. Hashes the file's source bytes plus every transitive dependency (other fixtures it references via `vm_fixture`/`lab_fixture`, every helper it `require`s, and any host file declared with `depends_on_file`) plus every profile's kernel and initrd identity and the wire-protocol version into a cache key.
3. Looks up `<cache_dir>/<key>.snap`. If present, restores it and hands the test a fresh VM.
4. If absent, builds the fixture by running its chunk under a build-time Lua state, takes the resulting snapshot, sparse-zstd-compresses it, and installs it into the cache.

A fixture file's chunk must end in a `return` statement that hands back either a `vm:snapshot()` (single VM) or a `provium:snapshot()` (whole lab). Example:

```lua
-- booted-base.fixture.lua
local vm = provium:vm("base", "peios"):boot()
vm:run("mkdir -p /srv/state"):assert_ok()
return vm:snapshot()
```

See [fixtures and dependencies](/provium/running-tests/fixtures-and-dependencies.md) for the cache lifecycle, eviction policy, and what triggers a rebuild.

## Helper files

Plain Lua files anywhere under a `roots` directory are reachable through `require`. The `roots` directories are prepended to `package.path` at file-runner setup, so `tests/helpers/assert_pingable.lua` can be loaded as:

```lua
local pingable = require("helpers.assert_pingable")
```

Helpers are not detected as tests (they don't end in `.test.lua`) and not detected as fixtures. Editing a helper invalidates the cache key of every fixture that transitively requires it.

## The fixture cache

```
~/.cache/provium/fixtures/
  c5e6f8….snap           # Single-VM fixture snapshot (sparse, zstd).
  c5e6f8….lock           # Per-key build lock.
  abc123….lab/           # Lab-fixture directory.
    lab.json             # Per-VM snapshot index.
    base.snap            # Per-VM snapshot.
    extra.snap
  abc123….lab.lock
```

The cache is per-user by default (`~/.cache/provium/fixtures/`). Override it system-wide with `[provium].cache_dir` in `provium.toml`. The cache is shared across runs and across files within a run — concurrent file runners building the same fixture coordinate via the per-key lock file.

LRU eviction runs at `provium` startup before tests dispatch: the harness sums file sizes under `cache_dir`, sorts entries by access time, and deletes the oldest until the total is under `cache_max_size`.

Inspect or manage the cache from the CLI:

```
provium fixture list      # Show every cached entry, with sizes.
provium fixture build P   # Force-build the named fixture.
provium fixture rebuild P # Evict and rebuild.
provium fixture clean     # Wipe the cache.
provium fixture stale     # List fixtures whose source hashes don't match any cache entry.
```

## The rerun state file

After every run that produced at least one failure, Provium writes the canonical paths of the failing files to `~/.cache/provium/rerun.json` (override with `$PROVIUM_RERUN_STATE`). The `--rerun-failed` flag intersects discovered files against this list, so:

```
provium tests/ --rerun-failed
```

re-runs only files that failed last time. A clean run leaves the file alone, so the failed set persists until you fix the failures.

## Required external binaries

| Binary | Used for | Provided by |
|---|---|---|
| `qemu-system-x86_64` | VMM backend | `qemu-system-x86` package |
| `ip` | Bridge / TAP / link operations | `iproute2` |
| `tc` | Latency / drop-rate / bandwidth qdiscs | `iproute2` |
| `nft` | Per-bridge partition rules, NAT for uplink | `nftables` |
| `tcpdump` | `bridge:capture()` and `nic:capture()` | `tcpdump` |

Provium's startup pre-flight checks for `/dev/kvm`, `/dev/vhost-vsock`, `ip`, `tc`, `nft`, `qemu-system-x86_64`, and effective `CAP_NET_ADMIN`. Missing pieces fail with an actionable message before any test runs. The exception is `tcpdump`: the pre-flight does not check for it, because it is only needed once a test calls `capture()`.

## What lives outside the project tree

- The fixture cache (default: `~/.cache/provium/fixtures/`).
- The rerun-state file (default: `~/.cache/provium/rerun.json`).
- Optional `--save-events` / `--events-socket` paths (your choice).
- The kernel and initrd files referenced by `[profiles.<name>]`.

Everything else — tests, fixtures, helpers, configuration — lives inside the project directory and is the project author's responsibility to keep version-controlled.