VM

21 min read

A VM userdata wraps one guest. You get one from lab:vm("name", "profile") (or the dot-sugar lab.<name> after creation), and every operation against the guest dispatches through it.

VM handles are cheap to clone — passing them around or storing them in tables doesn't duplicate state.

Constructing #

State machine #

Created ──:boot()──> Booted ──:pause()──> Paused
                       │                    │
                       ├─:reset()→Booted    ├─:resume()→Booted
                       ├─:wakeup()→Booted
                       ├─:power_button()──> Shutdown
                       └─:shutdown()─────> Shutdown

vm:state() returns "created", "booted", "paused", "shutdown", or "dead". Operations against a VM in the wrong state error cleanly with the offending state in the message.

Boot opts #

Boot opts split between the two call sites. The sizing keys go in the third arg to lab:vm(name, profile, opts) — the scheduler and QEMU need them when the VM is declared. The boot-shaping keys go to vm:boot(opts) and merge per field into the VM's pending boot options.

Example:

local vm = provium:vm("v", "peios", {memory = "2G", cpus = 4})
vm:boot({
    rng_seed = 0xdeadbeef,
    initial_time = 1700000000,
    files = {
        {path = "/etc/hostname", content = "v"},
    },
})

Injected files #

Each entry is written into the guest's root filesystem before its init runs, with the parent directories it needs. A path that already exists in the image is replaced, so a test can substitute a configuration file, a script, or a whole hook set without rebuilding the profile.

They arrive as one more cpio archive appended to the initrd, which is how the kernel is willing to receive them: an initrd is a sequence of archives unpacked in order into the same rootfs, and the last writer of a path wins. Injected files are appended after the agent overlay, so they win over both the profile's initrd and the agent's.

Two consequences follow from that mechanism, and neither is a limitation worth working around:

• They land in the initrd's rootfs, not in whatever filesystem the guest's init later mounts as its real root. For a profile whose initrd is the whole system this is the same thing. For one that pivots to a disk or a squashfs, an injected file is what the initramfs stage sees, which is usually exactly the point — it is how you hand an early boot stage a different configuration.
• They are placed before any userspace runs, so a guest cannot race them, and equally they cannot depend on anything a guest process does.

Paths are taken relative to the root whether or not they carry a leading slash. A path containing . or .. is rejected when the boot options are parsed, with the offending path named.

Boot disks #

Each entry becomes a virtio-blk-pci device present from the guest's first instruction, so the firmware and the initramfs can both see it — which is the difference between these and vm:attach_disk, whose attachment is created after the guest is already running.

Boot disks are attached after the profile's disks, so the profile's keep their device names: a test's first disk is /dev/vdb on a profile that already attaches a medium at /dev/vda. Ids share one namespace with the profile's, so a boot disk reusing a profile id fails the launch rather than silently displacing it.

Each entry names exactly one source, and the choice decides who owns the file:

A scratch or template image belongs to its VM: isolated from every other VM, removed when the VM goes, and surviving vm:reset() — a warm reboot against the same backing files — so a test can cut the power and ask what came back. Naming two sources, or none, fails the boot.

An entry may also set mediated = true, which changes not where the bytes come from but who serves them: Provium puts itself between the guest and the image and holds each write until the guest flushes it, so disk:power_cut() discards exactly what was never made durable. It combines with any of the three sources above, and is described in full under disks.

Once booted, vm:disk(id) returns a Disk handle for any disk attached this way, for sector access and fault injection.

local vm = provium:vm("v", "peios")
vm:boot({
    disks = {
        {path = "/tmp/blank.img", id = "target"},
    },
})
vm:disk("target"):fault_inject("eio_write")

Lifecycle methods #

vm:boot(opts?) #

Boots the guest. Returns self so calls chain (provium:vm("a", "peios"):boot()). The optional opts table (kernel_cmdline, kernel_cmdline_append, rng_seed, initial_time, files, disks, agent_timeout) merges per field into the VM's pending boot options. Errors if already booted. Under the dispatcher the boot reserves the VM's memory (plus 100 MiB of VMM overhead) and vCPUs first — the declared memory and cpus, or 512 MiB and one vCPU where the VM declared neither — from the file's claim when it has one, otherwise from the pool — and errors with cannot boot: … when the boot is past the claim, or when reserving it from the pool would deadlock the run.

Boots that are expected to fail are asserted through the error this raises. A guest whose init halts, or never starts, never reaches the agent, so vm:boot fails once the agent timeout lapses — and its message carries the tail of the guest console, which is the only record such a boot leaves. Wrap the call in pcall and assert on that text, and set agent_timeout (or the profile's agent_boot_timeout) low enough that waiting for it is not the bulk of the run.

Emits a vm_spawned event after the guest is up and the agent has handshaken. The event payload carries {file, vm_name, profile, memory_bytes, cid}.

vm:pause() #

Pauses the guest's vCPUs. Returns nothing. Errors if not in Booted.

vm:resume() #

Resumes a paused guest. Returns nothing. Errors if not in Paused.

vm:shutdown() #

Tears down the guest. Idempotent in the sense that Created, Booted, Paused, and Dead all transition to Shutdown. Emits a vm_shutdown event with {file, vm_name, duration_ns}.

vm:reset() #

Warm reboot — guest re-runs its boot path. Errors if not in Booted. The post-reset state is still Booted.

Returns once the guest is serving its agent again, so the VM is usable on return and the next vm:run needs no waiting of its own. Errors if the guest has not come back within two minutes.

The waiting is not a convenience. A reset severs the agent connection by definition — the guest it was talking to no longer exists — and a call that returned before the guest was back left the caller to meet that severed connection. Provium treats a connection-class failure as the VM having died, and refuses every later operation on that basis, so the first post-reset call would strand the VM permanently even though the guest was moments away from answering. Waiting here absorbs the gap where it belongs.

A consequence worth knowing when reading a failure: because the call waits, a vm:reset() that returns is a guest that rebooted and came back. One that errors names which of those did not happen.

vm:is_suspended() #

Returns whether QEMU reports that the guest is in ACPI S3, without waking it. Requires Booted, the same lifecycle state as wakeup; host-side Paused is a different state and is rejected. A running guest returns false. Backends without suspend-state support return an unsupported-operation error. Monitor failures propagate as errors, so polling cannot mistake a dead VM for a guest that has not suspended yet.

Use this query to wait for S3 when a test needs to leave the guest asleep for a measured interval before waking it. The query uses the monitor and remains available while the guest agent is suspended. Guest RPCs do not: a timer in another awake VM can supply the interval.

vm:wakeup() #

Wakes a guest that suspended itself — one that wrote mem to /sys/power/state and is sitting in ACPI S3 with its vCPUs halted. Sends QMP system_wakeup; the guest resumes where it stopped. Errors if the VM is not in Booted (a suspended guest is still Booted from the host's side: its vCPUs stopped, not its VMM), and errors if the guest is not actually suspended — QEMU refuses to wake a running guest. Retrying wakeup resumes the guest at the first opportunity; it does not guarantee any minimum time asleep.

The suspend itself is the guest's own act, and the syscall that performs it does not return until the wake — issue it through vm:syscall_async or a worker, wake from the host, then await it:

local pending = worker:syscall_async(sys.NR.write, ...)   -- "mem" → /sys/power/state
wait_until(function() return vm:is_suspended() end, {timeout = 5})
timer_vm:clock():sleep("2s")  -- a separate, awake VM
vm:wakeup()
pending:await()

vm:power_button() #

Sends ACPI power-button to the guest and waits for QEMU to confirm a new guest-initiated shutdown. Errors if not in Booted. On success, the state is Shutdown and the VM's scheduler reservation is released. The captured console remains readable, so a test can also inspect the guest's shutdown messages.

After the monitor accepts the request, Provium waits up to 60 seconds for the guest to finish shutting down. A guest may ignore ACPI: if that wait expires, the call errors, leaves the VM in Booted, and does not force it off. This is distinct from a failed monitor command or lost connection, which can mark the VM Dead. Booted after a timeout does not prove that the guest is responsive or that a late shutdown cannot still happen.

Only a fresh SHUTDOWN event with guest=true and reason="guest-shutdown" confirms completion. A previous event, guest reset or host-initiated termination is not confirmation of graceful poweroff.

vm:power_button()
assert(vm:state() == "shutdown")
local shutdown_log = vm:console():read_log()

vm:close() #

Auto-close hook used by the resource walker. Idempotent. Calls :shutdown() if not already in Shutdown. Test code rarely calls this directly — the harness fires it for every VM at file end (or per-test if provium.reset_between_tests = true).

Snapshot and restore #

vm:snapshot(path?) #

Take a snapshot. With an explicit path, writes there; without, writes to a tempfile. Returns a Snapshot userdata wrapping the path.

local s = vm:snapshot()
local s2 = vm:snapshot("/tmp/before-mutation.snap")

vm:restore(snap_or_path) #

Restore from a Snapshot userdata (preferred) or a bare path string. Errors if not in Created or Shutdown.

vm:shutdown()
vm:restore(s)

Layer-1 ops #

Layer-1 ops are the high-level "do it like SSH" primitives. They dispatch through the agent's exec / file / stat handlers.

vm:run(cmd_or_args, opts?) #

Run a command. Two forms:

• vm:run("echo hi") — runs through /bin/sh -c "echo hi". Shell metacharacters work.
• vm:run("echo", {"hi"}) — direct exec. No shell. The second arg is a positional-arg array.
• vm:run("echo", {args={"hi"}, env={K="v"}, cwd="/tmp", stdin="…", env_clear=true, timeout="5s"}) — opts form. Auto-detected by the presence of a recognised opts key (env, env_clear, cwd, stdin, timeout, timeout_ms, args); a table with none of them is treated as the legacy array. Mixed array entries with args= are not allowed; use one form or the other.

Returns a RunResult userdata.

vm:run_async(cmd, opts?) #

Like :run, but returns a Process userdata immediately. The agent does not auto-kill; you control the lifetime via proc:kill() / proc:wait(). Passing timeout here is rejected — use proc:wait(timeout) instead.

vm:read_file(path) #

Returns the file contents as a Lua string. Errors on agent-side read failure (ENOENT, EACCES, etc.).

vm:write_file(path, data) #

Replaces the file's contents with data. Creates the file if absent. Returns nothing.

vm:push_file(host_path, guest_path, opts?) #

Read a file from the host filesystem and write its bytes to guest_path in the guest. Inside a fixture, the host file is folded into the fixture's cache key automatically — rebuilding the host file (typical case: a binary under test) invalidates the snapshot. Relative host_path is resolved against the directory of the file containing the call. Pass {auto_dep = false} to skip the auto-fold for a single call. See files and handles and fixtures and dependencies.

vm:stat(path) #

Returns a table:

vm:listdir(path) #

Returns an array of {name=string, entry_type=string} tables.

vm:mkdir(path, opts?) #

Create a directory. Opts: {parents=bool, perm=int}. With parents=true, intermediate directories are created (mkdir -p). perm is the POSIX mode for the new directory.

vm:unlink(path) #

Remove a file or empty directory.

vm:rename(from, to) #

Atomic rename within the guest filesystem.

vm:open_file(path, mode_table) #

Returns a File userdata.

The mode table accepts:

At least one of read, write, or append must be true; an empty mode table errors at open time with a pointer.

vm:tail_file(path, opts?) #

Open a streaming subscription to a file. Returns a Tail userdata. Opts: {start = "beginning" | "end" | <offset>} ("start" is accepted as an alias for "beginning"). Default is "end" — only bytes appended after the call are streamed. <offset> may be a non-negative integer (absolute byte offset), a negative integer (N bytes before EOF — provium stats and resolves), or a finite float (truncated toward zero, same sign semantics).

vm:fd_stream(fd_or_file) #

Open a streaming subscription to an existing file handle. Accepts either an integer handle id (from file:fd()) or a File userdata directly. Returns a Tail.

Layer-0 ops #

Layer-0 ops are direct kernel-level primitives — syscall(2) and ioctl(2). Use these to exercise driver paths or to test syscall semantics directly.

vm:syscall(nr, ...) #

Two call shapes:

• Integer-only: vm:syscall(nr, a1, a2, a3, a4, a5, a6) — up to 6 integer arguments.
• Table form: vm:syscall(nr, {args={a1, a2, …}, bufs={"…", …}, ptrs={1, 3}, nested={…}}). Buffers are byte arrays; ptrs[i] says which arg slot (0-indexed) bufs[i] should be spliced into. The buffer's address is filled in for the kernel.

Nested pointers. nested is a list of {parent=N, child=M, offset=K} entries (parent/child are 1-based indices into bufs). Before the arg-slot ptrs are applied, the agent writes bufs[M]'s address into bufs[N] at byte offset (an 8-byte native-endian pointer). This is how you pass a struct argument that itself contains a pointer — a buffer pointing at another buffer. Use it for ioctls invoked via raw syscall(SYS_ioctl, fd, cmd, &struct) whose struct has an inner output-buffer pointer (a query/info ioctl), or for syscalls whose argument struct embeds pointers (kacs_access_check-style). Each pointed-at buffer is returned post-call in out_bufs at its own index.

-- ioctl(fd, QUERY, &args) where struct kacs_query_args { u32 class; u32 buf_len; u64 buf_ptr; }
-- buf_ptr (offset 8) must point at the output buffer.
local args = string.pack("<I4I4I8", PRIVS_CLASS, 4096, 0)  -- buf_ptr filled in by nested
local out  = string.rep("\0", 4096)
local r = vm:syscall(SYS_ioctl, {
    args   = { fd, QUERY_CMD, 0 },     -- arg slot 2 receives the struct pointer
    bufs   = { args, out },            -- buf 1 = struct, buf 2 = output
    ptrs   = { 2 },                    -- buf 1 → arg slot 2
    nested = { { parent = 1, child = 2, offset = 8 } },  -- buf 2's addr → buf 1 @ offset 8
})
local data = r.out_bufs[2]             -- the QUERY output buffer

Returns a table:

The same bufs / ptrs / nested options are available on worker:syscall.

vm:ioctl(fd, cmd, data?, opts?) #

Direct ioctl(2). fd and cmd are integers. data is an optional byte string passed as the third arg's pointer. opts.bufs and opts.ptr_offsets work like :syscall's bufs / ptrs.

Returns {ret, result, out_data, out_bufs}. out_data is the post-call value of data (for ioctls that write back through the same buffer).

Hypervisor resources #

vm:nic(name) #

Returns a Nic userdata for the named bridge attachment, or for a guest-style interface name (eth0, enp0s3, ens3). Guest-name lookup maps to the Nth attached bridge sorted by bridge name.

vm:disk(id) #

Returns a Disk userdata for an already-attached disk. Errors if no disk with that id.

vm:attach_disk(opts?) #

Attaches a disk to a running VM and returns a Disk userdata for sector access and fault injection against its backing file. Given an image on a booted or paused VM, the guest gets a real virtio-blk-pci device it can see; without an image, or on a VM that has not booted, the attachment is recorded host-side only, which is all sector access and fault injection need. What it cannot do is be there from the guest's first instruction — firmware and the initramfs look for block devices long before there is an agent to call anything — so a disk the guest must find at boot belongs in the profile's disks or in vm:boot({disks = …}). Opts: {id=string, size=int, image=string, readonly=bool}. Defaults: id = attached-<vm_name>, size = 4 GiB, image = no backing file, readonly = false.

Managed QEMU VMs have eight reusable runtime disk ports in addition to their launch disks. An occupied pool returns a capacity error; a detach request does not free a port until the guest completes removal. See the Disk reference for capacity, cleanup and retry behavior.

Console and clock #

vm:console() #

Returns a Console userdata.

vm:clock() #

Returns a Clock userdata.

Workers #

vm:spawn_worker(opts?) #

Spawns a sub-agent connection so test code can dispatch concurrent ops against the same guest. Returns a Worker userdata. Opts are reserved for future use; passing {thread=true} errors with a future-work pointer.

Batch operations #

vm:batch(fn) #

Collect multiple ops inside fn(b) and dispatch them as one wire round-trip. b exposes:

• b:run(...) (with the same shape as vm:run)
• b:read_file(path)
• b:write_file(path, data)
• b:stat(path)
• b:listdir(path)
• b:mkdir(path, opts)
• b:unlink(path)
• b:rename(from, to)
• b:syscall(nr, a, b, …) — integer-args form only; the table form is rejected here.

Returns a Lua array of {ok=value} or {err=msg} entries, one per op. A failure on op N does not short-circuit the rest of the batch.

local results = vm:batch(function(b)
    b:write_file("/tmp/a", "1")
    b:write_file("/tmp/b", "2")
    b:read_file("/tmp/a")
    b:stat("/tmp/missing")
end)
-- results = {{ok=nil}, {ok=nil}, {ok="1"}, {err="No such file or directory"}}

Accessors #

vm:name() — Returns the VM's name as given to lab:vm. #

vm:profile() — Returns the profile name. #

vm:state() — Returns "created", "booted", "paused", "shutdown", or "dead". #

vm:cid() — Returns the assigned vsock CID, or nil before boot. #

CIDs are unique across provium processes running side by side on one host: each process counts from a start scattered by its pid, and a CID is reserved by creating its per-VM scratch directory (/tmp/provium-qemu/vm-<cid>) atomically before QEMU launches, so two processes can never hand one guest CID to two VMs.

vm:is_quiescent() — true if no in-flight ops, no open files or streams. #

vm:open_file_count() — Number of files currently open through the agent. #

vm:open_stream_count() — Number of streams currently active (tails, fd-streams, console-reads, captures). #

RunResult #

The return value of vm:run(...) (and worker:run(...), proc:wait()).

Fields:

Methods:

• result:ok() — convenience for result.exit_code == 0.
• result:assert_ok() — raises if not OK; the error message includes status, stdout, and stderr (with signal name when signalled).

See also #

• Lab — what creates and contains VMs.
• Process — what vm:run_async returns.
• Worker — what vm:spawn_worker returns.
• Disk, Nic — hypervisor-side handles.
• Console, Clock — guest-side accessors.
• Streams — what vm:tail_file and vm:fd_stream return.

Edit this page