peinit
Technical Reference Manual — the init system and service manager: boot, the service model, identity, supervision, jobs and operations, and shutdown.
Single-page view · as markdown
1.1 Overview
Peios / Peios technical reference / peinit / Introduction
peinit is PID 1. It is the only service manager on a Peios system: every supervised process on the machine — a platform daemon, an application service, a startup hook, a health probe, a job some other service asked for on a user's behalf — is forked by peinit and watched by peinit until it exits.
It is a single-threaded Rust process. That is the constraint the rest of its design answers to. A blocking syscall in PID 1 stops everything: child reaping, watchdog expiry, shutdown signals, the control socket. So peinit keeps a complete in-memory model of every service it knows about, reads the registry synchronously only at boot and when it reloads — whether an administrator asked for the reload or a registry change notification set it off, which takes the same path (§10.4) — and pushes anything that could block off the main loop into a forked helper or a pollable descriptor.
1.1.1 What makes it not systemd #
Two things, and both run deep enough to change the shape of the daemon.
Services are securable objects. A service carries a Security Descriptor that says who may start it, stop it, query it, or reload it, and peinit evaluates that descriptor against the caller's KACS token on every control request. There is no "root can do anything" path, because there is no root — there is a token, and AccessCheck is the only thing that decides.
Service identity is a token, not a user. peinit never sets a UID, a GID, or a Linux capability on a service process. It obtains a KACS token — minted from its own for the platform daemons that start before an authority exists, requested from authd for everything else — restricts its privileges to what the definition asked for, and installs it on the child before exec. Every service also carries a per-service SID derived from its name, so two services sharing an identity are still distinguishable to an access check.
Configuration follows from the second one: service definitions live in
the registry, under Machine\System\Services\, where they are protected
by the registry's own descriptors rather than by file permissions. There
are no unit files, no generators, and no translation layer.
1.1.2 The shape of the daemon #
Boot is two phases with registryd as the boundary. Phase 1 is compiled in and has no registry dependency at all: it confirms the root is writable, mounts what is missing, restores entropy, settles the clock, and starts registryd. Phase 2 reads the service graph out of the registry and boots the system from it. When Phase 1 cannot complete, there is no Phase 2 to fall back to, and peinit drops into a recovery shell.
Once booted, peinit is an event loop over a handful of descriptors: a signalfd, the control socket, the jobs socket, the notify socket, one timerfd per armed timer, a pidfd per supervised process, a pipe pair per supervised process's output, and the registry's change-notification descriptor. Work arriving on most of them turns into an operation — a queued, observable request to move a service through its state machine — and executing an operation eventually forks a job. Work arriving on the jobs socket skips the operation: a submitted job has no service and no state machine to contend for, so a submission forks a job directly.
Those two objects are how peinit stays comprehensible under concurrency. Operations exist so that two administrators issuing conflicting commands get a defined answer instead of a race. Jobs exist so that "what actually ran" is a thing with an identifier, a token summary, an exit status and a log correlation key, rather than a PID that may already have been reused.
peinit keeps no history of either. A job or an operation that reaches a terminal state is emitted as a structured event into the KMES kernel ring buffer and dropped. eventd is the historian.
1.1.3 What peinit is not #
It does not assemble storage. The initramfs delivers a mounted, writable root, and peinit neither decrypts, nor assembles, nor checks it. It has no mount feature beyond the fixed Phase 1 set — mounting a data partition is a Oneshot service's job.
It does not store logs. It holds the pipes at birth, tags each line, and forwards it to eventd; before eventd exists it buffers, and when the buffer fills it drops the oldest.
It does not authenticate anyone. authd mints tokens; peinit installs them. It does not resolve identities, and it does not know or care whether a principal is local or from a domain.
And it does not support forking daemons. It tracks the process it spawned, through a pidfd obtained at fork, and there is no mechanism for a service to point supervision somewhere else.
1.2 What This Manual Covers
Peios / Peios technical reference / peinit / Introduction
This manual describes peinit as it is built: the boot sequence, the service model and its registry schema, how a service acquires its identity, the exact sequence between "start this" and "the binary is running", the state machine and its causes, dependencies, jobs and operations, timers, output handling, shutdown, and the security model.
1.2.1 What is a contract and what is not #
Three of peinit's interfaces are specified separately. PSPU §4 gives the control socket, spoken by administrative tools and by any program that manages services or observes jobs, and the notification socket, spoken by every service and every submitted job that reports readiness, status, keepalives, or a stored descriptor. PSPU §7 gives the jobs socket, spoken by any process that asks peinit to run and supervise a program on its behalf. Those are contracts. A third party implements one side of each, and what is written there binds both.
This manual covers the other side of that boundary — how peinit fulfils them, and everything that is not a contract at all:
- how a command becomes an operation, and what happens when two of them collide (§8.2, §8.3)
- what a command does to a service in each state (§10.3)
- how a notification is authenticated and applied (§10.5, §10.6)
- how a submission becomes a running job, and what it is checked against afterwards (§8.5, §10.7)
Where a chapter touches the contract it references PSPU §4 rather than restating it.
The service definition schema is here rather than in PSPU. It is registry configuration, protected by the registry's descriptors and administered by the same tools as the rest of the registry, and it is documented for the people who write service definitions rather than specified as a wire format. §3.2 is the reference.
1.2.2 What this manual does not cover #
- KACS — tokens, Security Descriptors, AccessCheck, and the per-service SID algorithm peinit reproduces. Peios Kernel TRM §3.
- LCS — the registry syscalls, watches, and layer resolution peinit reads through. Peios Kernel TRM §5.
- KMES — the ring buffer peinit emits its events into. Peios Kernel TRM §2.
- loregd — the storage behind registryd. Its own TRM.
- eventd — where the logs and events go. Its own manual.
- authd — token minting and identity routing. peinit's requirements of it are described in §4.3; its interface is authd's own.
- How a token is conveyed on a socket, and what the kernel checks
when it is — peer-token capture,
KACS_SCM_TOKEN, the impersonation gates and the level ratchet that job submission relies on. Peios Kernel TRM §3.5. - Using peinit — writing service definitions, the
svctlcommand surface includingsvctl job, and everyday administration are covered in Using Peios.
1.2.3 Constants and keys #
Registry keys and compiled-in constants are collected in the appendices, so a chapter's reference material does not interrupt the prose that explains it. Where an appendix defines a value the body references it rather than repeating it.
1.3 Terminology
Peios / Peios technical reference / peinit / Introduction
Terms defined where they are introduced — activation generation, boot generation, transition cause, cgroup generation, start generation — are not repeated here.
Service. A named, supervised unit of execution: a definition in the registry, a runtime state, a Security Descriptor, and at most one running main process. Services are the primary unit of management and the thing dependencies are expressed between.
Job. One supervised process execution. Every fork peinit performs is a job — a service's main binary, a pre-exec hook, a health check invocation, a submitted job — with a GUID, a lifecycle, a token summary and a log correlation key. Jobs are the observable unit of what actually ran.
Submitted job. A job whose definition arrived on the jobs socket rather than from a service definition. It has no service, no operation and no policy; it belongs to its submitter and is managed under its own Security Descriptor. §8.5.
Submitter. The identity a jobs socket connection carried when a
submit was read: recorded on the job, counted for the quota, and the
owner of the job's descriptor.
Job identity. The identity a submitted job's process runs as —
the submitter's own primary token, or a token the kernel attached to
the submit message. Never named by a field. §8.5.
Operation. A first-class object representing a requested state machine action on a service. Control commands do not mutate state directly; they create operations that are validated, queued, resolved against whatever else is in flight, and executed by the event loop.
Trigger. A rule in a service definition saying when the service starts automatically: at boot, once the boot set has settled, or on a schedule. A service with no triggers starts only when something asks for it.
Phase 1. The compiled-in bootstrap: root writability, the remaining virtual filesystems, the persisted random seed, the local machine ID, the clock, registryd, boot-time path provisioning, and infrastructure setup. No registry access happens before registryd is serving.
Phase 2. The registry-driven boot: read the service definitions, build and validate the dependency graph, and start the boot-triggered services in dependency order.
ErrorControl. The per-service policy for an irrecoverable failure. Normal leaves the service Failed; Critical syncs the filesystems and reboots.
Token. The per-thread KACS identity object — a user SID, group SIDs, a privilege bitmask, an integrity level, and metadata. Tokens are the sole identity mechanism: peinit sets no UIDs, GIDs, or Linux capabilities on service processes. Peios Kernel TRM §3.2.
Security Descriptor (SD). The KACS structure controlling access to a securable object. peinit evaluates three kinds: a ServiceSecurity descriptor per service, controlling who may manage it; its own control descriptor, controlling system-level operations; and a descriptor per submitted job, controlling who may observe, stop or signal it. Peios Kernel TRM §3 and PCDS §5.
AccessCheck. The KACS function that evaluates a token against a descriptor to produce a decision. peinit calls it for every control request. Peios Kernel TRM §3.8.
Per-service SID. A SID under authority S-1-5-80 derived
deterministically from the service name, carried in the group list of
every service token. Two services running as the same principal are
still distinguishable to an access check. §4.4.
registryd. The userspace registry source daemon serving the persistent hives. peinit starts it in Phase 1 from a compiled-in definition and treats it as opaque thereafter. Its implementation is loregd, a distinction visible only in recovery mode.
Registry. The configuration system LCS and its sources provide
together. peinit reads service definitions from
Machine\System\Services\, boot configuration from
Machine\System\Boot\, and its own parameters from
Machine\System\Init\.
KMES. The kernel-mediated event subsystem. peinit emits its structured events — job and operation lifecycle, audit records — into its ring buffer, where they survive eventd restarts and reboots. Peios Kernel TRM §2.
pidfd. A descriptor referring to one specific process, obtained
atomically at fork through clone3(CLONE_PIDFD). Every process peinit
supervises is tracked by one, which is what makes supervision immune to
PID reuse.
cgroup. peinit uses cgroups v2 for process tracking and clean kill
only — not for resource accounting or limits. Every service gets its own
tree under /sys/fs/cgroup/peinit/, and every submitted job one under
/sys/fs/cgroup/peinit/jobs/.
§5.1.
sd_notify. The datagram protocol services and submitted jobs use
to report readiness, status, progress, keepalives and stored
descriptors, over the socket named by NOTIFY_SOCKET. Specified in
PSPU §4.
TCB. The Trusted Computing Base: the kernel, KACS, LCS, KMES, peinit, registryd, authd, lpsd and eventd. A compromise of any of them compromises the system.
1.4 Compatibility and Prior Art
Peios / Peios technical reference / peinit / Introduction
peinit is not a port or a reimplementation of anything. The choices that give it its shape — identity as a token, configuration in the registry, operations as objects, jobs as observable executions — were made for Peios. But several interfaces deliberately match existing conventions, because the convention is good and breaking it buys nothing.
1.4.1 sd_notify #
peinit speaks systemd's sd_notify datagram protocol: a service reports
readiness, keepalives, status and stored descriptors by sending
KEY=VALUE lines to the socket named in NOTIFY_SOCKET. Existing
software that supports sd_notify works unmodified.
The compatibility is not total. MAINPID= is not supported,
because peinit does not supervise
forking daemons — it tracks the process it forked through a pidfd, and
there is no way to redirect supervision somewhere else. BUSERROR= is
not supported because Peios has no
D-Bus.
The protocol as peinit speaks it, including which fields it accepts and how a sender is authenticated, is specified in PSPU §4.
1.4.2 The fd store #
FDSTORE=1, FDNAME=, FDSTOREREMOVE=1 and FDPOLL=0 work as they do
under systemd, and descriptors come back to a restarted service at fd 3
onwards with LISTEN_FDS and LISTEN_FDNAMES set. A daemon written to
survive a restart without dropping its listening sockets keeps working.
1.4.3 Calendar expressions #
Timer schedules use systemd's OnCalendar format, including weekday
names, lists, ranges, repetition, the ~ last-day-of-month form, IANA
timezone suffixes and the named shortcuts. §9.1 gives the grammar peinit
actually parses; the one deliberate subtraction is sub-second precision,
which service scheduling
has no use for.
1.4.4 Windows Service Control Manager #
The service model owes its architecture to the Windows SCM: services as securable objects carrying their own descriptors, identity as a token rather than a user account, an access-controlled control interface, and a structured state machine. The debt is architectural. peinit implements none of the SCM's RPC protocol, none of its service types, and none of its control codes.
1.4.5 What is deliberately absent #
There is no systemd unit file support: no reader, no parser, no generator, no migration path. Service definitions are registry keys. Roles are how a package declares them.
There is no socket activation in the systemd sense — peinit does not listen on a service's behalf and hand it a connection. The fd store covers the case that matters, which is a service keeping its own listener across its own restart.
There is no resource control. peinit uses cgroups for tracking and for
clean kill, and sets RLIMIT_NOFILE and RLIMIT_CORE if a definition
asks, but it does
not do accounting, slices, or limits.
1.4.6 Features that belong to other components #
| Concern | Component |
|---|---|
| Authentication and token minting | authd |
| The local identity database | lpsd |
| Log storage, indexing and queries | eventd |
| Registry storage | registryd, over LCS |
| Packaging and installation | peipkg and the role system |
| Device management | eudev |
| Network configuration | a dedicated service |
| File access control enforcement | FACS |
1.5 Service Manager Boundaries
Peios / Peios technical reference / peinit / Introduction
These design notes explain the boundaries behind the operator workflows. The implementation chapters specify each mechanism in detail.
1.5.1 peinit in one sentence #
peinit is the sole service manager and PID 1 — it reads service definitions from the registry, starts them in dependency order under per-service identities, supervises them through a defined state machine, and tears them down in reverse at shutdown.
Everything in this topic is an elaboration of that sentence: what a service definition is, what dependency order means, what the state machine looks like, how identity is materialised, and how you drive the whole thing from a shell.
1.5.2 Three kinds of object #
peinit is easiest to understand through the three first-class objects it works with. Keeping them straight is most of the battle when you read a status output or an event log.
| Object | What it is | Lifetime |
|---|---|---|
| Service | A definition — a named unit of execution with identity, policy, and configuration, stored in the registry. It has a runtime state (the state machine) and, when running, a process. | Long-lived. Persists across restarts and reboots. |
| Job | A single process execution. Every time peinit forks — a service's main process, a hook, a health check, a submitted job — that is one job, with its own GUID and exit result. | Short-lived. A restart creates a new job. |
| Operation | A requested action on a service — start, stop, restart, reload, reset. Control commands create operations that are validated, queued, and executed. | Short-lived. Resolves to a terminal state, then is dropped. |
A service is the what; a job is what actually ran; an operation is what someone asked for. A single restart command, for example, is one operation, which stops the current job and starts a new one, all against one service. Jobs and operations both carry GUIDs and are emitted to eventd as they complete, which is how the history of a service is reconstructed after the fact. They get their own page: Jobs and operations.
Underneath all three sits the fourth thing peinit is always handling — the token, the identity a service runs under. Tokens are not peinit's invention; they are the system-wide identity object. peinit's role is to obtain the right token for each service and install it before the process runs. That is the subject of Service identity and privileges.
1.5.3 What peinit is not #
peinit borrows vocabulary from init systems you may already know, and the familiarity is a trap. Clearing three wrong mental models is most of understanding what peinit actually is.
It is not systemd. There are no unit files. peinit does not read, parse, or translate .service files, and there is no migration shim. A service definition is a key in the registry under Machine\System\Services\, made of typed registry values — not a text file under /etc. Some interfaces are deliberately compatible where that helps: peinit speaks the sd_notify readiness protocol, and timer schedules use systemd's OnCalendar calendar-expression format. The compatibility stops at those wire formats; the model underneath is different.
It is not Windows SCM. The influence is real — services as securable objects with per-service descriptors, token-based service identity, an access-controlled control interface, a structured state machine — but it is architectural, not interface-level. peinit does not implement the SCM RPC protocol, Windows service types, or Windows control codes.
It is not a logging system. peinit holds a service's stdout/stderr pipes at birth, so it controls where output goes — but storage, indexing, and queries are eventd's job. peinit forwards; eventd is the historian. The same split applies to job and operation history: peinit emits structured events and forgets them; eventd keeps them. See Service output and logging.
A fourth, smaller correction: peinit does not supervise forking daemons. A service that double-forks to "daemonise" is solving a problem that does not exist when the manager tracks the process it spawned. peinit tracks every process by pidfd; a legacy binary that insists on double-forking must be wrapped at the package layer.
1.5.4 The one operational invariant #
peinit is single-threaded PID 1, and that one fact explains a surprising amount of its design. In a single-threaded PID 1, a blocking system call blocks everything — child reaping, watchdog expiry, shutdown signals, every other event stops while the call is stuck. So peinit's hard rule is that its event loop never blocks on a userspace service.
Two consequences you will meet repeatedly:
- peinit operates on an in-memory snapshot of the registry, not a live view. It reads service definitions at boot and on configuration reload, and thereafter works from an in-memory model that it updates from change notifications. Supervision never waits on the registry being available. This is why a configuration change does not always take effect immediately — see Defining a service.
- Anything that could block is pushed off the main loop. Filesystem condition checks run in a short-lived forked helper rather than a
stat()on the event loop; token requests to authd are driven by a timed state machine, not a blocking call; log delivery is best-effort and never exerts back-pressure on peinit.
You do not need to think about the event loop to operate peinit, but it is the reason behind several behaviours that would otherwise look arbitrary.
1.5.5 Where peinit sits in the system #
peinit is part of the Trusted Computing Base: a compromise of peinit is a compromise of the whole machine. It runs as SYSTEM with all privileges, and it never drops that identity. It depends on, and is depended on by, the other TCB components:
| peinit relies on | For |
|---|---|
| KACS | Service identity (installing tokens on children), control-interface authentication (peer token + AccessCheck), and per-service access control. See Access decisions. |
| LCS / the registry | Service definitions, boot configuration, and its own parameters. See The registry. |
| registryd | The registry source daemon — the first service peinit starts. peinit treats it as an opaque dependency. (registryd is an interface; the default implementation is loregd — see Boot and boot modes.) |
| authd | Minting tokens for non-platform services. peinit requests; authd mints. |
| eventd | Service logs (forwarded over a socket) and job/operation/audit events (emitted through KMES). |
| KACS, again | Conveying a job identity: when a process submits a job over peinit's jobs socket and attaches a token, the kernel gates the attach — so peinit can run a program as a service's client without ever judging the identity itself. See Jobs and operations. |
The bootstrapping puzzle — authd needs the registry, the registry needs to be started by something, and that something is peinit running before any of them exist — is resolved by the boot model in Boot and boot modes.
1.5.6 How a token is materialised #
The Identity field decides where the token comes from:
Identity value | Token source |
|---|---|
SYSTEM | Minted by peinit from its own SYSTEM identity. |
| Any other principal name or SID | Requested from authd. |
| Absent or empty | authd, defaulting to LocalService. |
1.5.6.1 The SYSTEM path #
For a SYSTEM service, peinit mints an independent SYSTEM token using its own token as the template: it reads its own identity (user SID S-1-5-18, the group list, the privilege set) and mints a fresh primary token with the same identity, adding the service's per-service SID. The minted token is fully independent — the privilege trimming that follows affects only this token, never peinit's.
This path exists because of a bootstrapping problem: the platform services that make the normal token flow possible cannot use it, because they have to start before it exists. registryd and authd use this bootstrap path during boot. Standard eventd and lpsd deployments now use Identity=Service, take the authd path, and start after authd (§4.2).
1.5.6.2 The authd path #
For any other identity, peinit asks authd for a token: it sends the Identity value verbatim, and authd routes it to the right source —
- Well-known principals (
LocalService,NetworkService, …) → a built-in identity with a predefined minimal privilege set; - Local service accounts → lpsd, the local principal store;
- Domain accounts → the directory connector;
— resolves the principal's SIDs, mints a token, creates a logon session, and hands the token back to peinit to install. peinit neither knows nor cares whether the identity is local or domain; routing is authd's job.
authd records each request as an authd.service.attested event: the service (object.service.name), the identity it got (object.token.sid), its session and the privileges policy granted it. A refused request is recorded too, with the reason, such as account-restricted for an identity that may not be used for a service logon. The event is standard, so the emission policy can switch it off on a machine that restarts services often.
1.5.7 The per-service SID #
Every service token — minted or authd-issued — carries a per-service SID in its group list. It uses authority S-1-5-80 and is derived deterministically from the service name: the SHA-1 of the uppercased name (as UTF-16LE), with the digest split into five sub-authorities. authd adds it automatically for the tokens it mints; peinit computes and adds it itself for SYSTEM tokens, no authd involvement needed.
The point of the per-service SID is fine-grained access control even when services share an identity. A dozen services might all run as LocalService, but each has a unique service SID, so an ACL can grant access to exactly one of them without inventing a dedicated account. It also keeps services configured as SYSTEM distinguishable to AccessCheck. See Well-known principals for the SID landscape this fits into.
1.5.8 A clean context #
A service inherits only what peinit explicitly hands it — its standard streams and any stored file descriptors. Every other descriptor peinit holds — the control socket, the notify socket, the event-loop fd, the registry and authd and eventd connections — is opened close-on-exec, so it closes automatically at exec and can never leak into a service. peinit also resets the child's signal state to defaults (it runs with all signals blocked for its own signalfd; a service must not inherit that). The guarantee is that a service starts from a clean slate, not from peinit's privileged one.
1.5.9 The cgroup tree #
Every service runs in its own cgroup tree under /sys/fs/cgroup/peinit/, with separate sub-cgroups for the main process, hooks, and health checks:
/sys/fs/cgroup/peinit/<service>/
├── main/ the main process
├── hooks/ pre/post hooks
└── health/ health checks
peinit uses cgroups for two things only: tracking every process a service spawns (so a child that forks its own children is still accounted to the service), and clean kill (terminating the entire tree at once, so nothing is orphaned). It does not use cgroups for resource accounting or limits.
The split into sub-cgroups serves a real purpose: it satisfies cgroup v2's "no internal processes" rule, and it lets peinit kill a service's hooks or health checks without touching the main process.
When an old tree cannot be removed — because a process survived SIGKILL in D-state and leaked it — peinit creates a fresh generational tree (<service>.gen<N>) for the next start, so a stuck old instance never blocks a new one. Leaks are surfaced in the service's warnings; they are a sign of an underlying I/O fault, covered in Keeping services running.
2.1 The Initramfs Contract
Peios / Peios technical reference / peinit / Boot
peinit starts with the root filesystem already assembled. Everything that makes a root mountable — LUKS decryption, LVM activation, RAID assembly, any filesystem check — belongs to the initramfs and has happened before peinit exists. peinit performs no root assembly, no decryption, no repair, and no fsck, and it does not assume one has been done.
2.1.1 The handoff #
The initramfs transfers control by chroot-ing into the assembled root
and exec'ing peinit there.
Not switch_root, and not pivot_root: the
kernel refuses to relocate onto the initramfs rootfs, so that route is
closed. The consequence is that the initramfs rootfs does not go away.
It remains the mount-namespace root — emptied, and unreachable from
peinit's view, but present. peinit therefore never assumes a clean
single-root mount topology, and never attempts pivot_root.
peinit is installed in package storage at /usr/bin/peinit2 and reached
through the fixed runtime path /bin/peinit2. The boot-image tooling
sets the kernel init= to the runtime path.
Before the transfer, the
initramfs has assembled the base StrataFS topology, including the /bin
and /sbin views, because peinit reaches every binary it execs through
them.
At handoff:
- the real root is mounted read-write at
/; /proc,/sysand/devare mounted and have been moved into the real root;- the environment holds
TERMand nothing else, and argv is just peinit's own path.
The read-write requirement is registryd's, not peinit's. loregd's storage backend needs to write its write-ahead log and shared-memory files even to answer a read, so a read-only root cannot support Phase 2 at all. Delivering the root writable is the initramfs's job; peinit only confirms it.
peinit inherits nothing from that environment. It does not rely on having been given anything, and it does not pass its own near-empty startup environment through to services — the environment a service receives is constructed from scratch (§5.5).
2.1.2 What stays outside #
Non-root storage is not peinit's concern. Data partitions and additional
filesystems are mounted at the services layer, typically by a Oneshot
service that runs mount. peinit has no mount feature beyond the fixed
Phase 1 set.
2.2 Bootstrap Identity
Peios / Peios technical reference / peinit / Boot
The steady-state identity flow is: peinit asks authd for a token, authd mints it, peinit installs it on the child. That flow cannot start the system, because authd needs a token of its own, and registryd has to be running before it. The bootstrap model breaks the circle.
2.2.1 Platform services run as SYSTEM #
A service whose definition says Identity=SYSTEM gets a token peinit
mints from its own, with kacs_create_token (§4.2).
No authd interaction
is involved — which is the point, since authd does not exist when the
first of these services starts.
Two services use it:
| Service | Why |
|---|---|
| registryd | Starts before authd exists at all. |
| authd | Needs SeTcbPrivilege and SeCreateTokenPrivilege; it is the minter for everything else. |
lpsd and eventd used it too, until each moved to an account of its own
(Identity=Service). Both now require authd, start after it, and take
the authd path like any other service.
Nothing restricts which services may declare Identity=SYSTEM.
There is
no allowlist, because an allowlist would be enforcing a boundary that is
already enforced somewhere better: the Security Descriptor on
Machine\System\Services\. Anyone who can create a service definition
is by definition trusted to choose its identity, and adding a second
list to maintain would only create a way for the two to disagree.
Every SYSTEM token peinit mints carries the service's per-service SID in
its group list, computed by peinit itself from the service name (§4.4).
That is
what keeps platform services distinguishable to an access check despite
all of them running as S-1-5-18.
2.2.2 After authd #
Once authd is running, every subsequent service gets its token
through the ordinary authd flow (§4.3).
A definition with
no Identity field defaults to LocalService — a well-known principal
with a minimal privilege set
— and authd adds the per-service SID to the token it mints.
2.3 Phase 1
Peios / Peios technical reference / peinit / Boot
Phase 1 is compiled into peinit. It does not change at runtime and has no registry dependency, because its whole purpose is to reach the point where a registry exists. It does the minimum needed to make Phase 2 possible, and most of its failures are fatal to the boot.
2.3.1 Step 1: Confirm the root is writable #
The initramfs delivers the root mounted read-write. peinit does not remount it — mount flags belong to the initramfs, and a redundant remount of an already-writable or overlay root can fail for reasons that have nothing to do with the root being usable.
Instead peinit probes. It creates /.peinit/ if it is absent, writes a
uniquely named file there — the name is derived from peinit's own PID
and a namespace identifier, so two probes cannot collide — writes to it,
and removes it. If any part of that fails, the root is not usable for
Phase 2 and peinit enters recovery mode.
2.3.2 Step 2: Mount what is missing #
/proc, /sys and /dev are already mounted. peinit does not
blindly mount them again: a redundant mount stacks a second filesystem
over the populated one, and on some kernel and flag combinations returns
EBUSY instead.
peinit reads /proc/self/mountinfo to find out what is already there,
and mounts only what is not:
| Mount point | Filesystem | Flags | Provided by |
|---|---|---|---|
/proc | proc | nosuid, nodev, noexec | initramfs |
/sys | sysfs | nosuid, nodev, noexec | initramfs |
/dev | devtmpfs | nosuid | initramfs |
/dev/pts | devpts | nosuid, noexec | peinit |
/dev/shm | tmpfs | nosuid, nodev | peinit |
/run | tmpfs | nosuid, nodev | peinit |
/sys/fs/cgroup | cgroup2 | nosuid, nodev, noexec | peinit |
/sys/kernel/security | securityfs | nosuid, nodev, noexec | peinit |
Each mount(2) passes the filesystem name as both the source and the
filesystem type, passes only the listed flags, and passes null mount
data. Mount points that do not exist are created first.
/dev/pts additionally gets a KACS mount policy immediately after it is
mounted :
SYNTHESIZE_EPHEMERAL, with a template descriptor granting
SYSTEM and Administrators full control and Authenticated Users read,
write and traverse. devpts cannot store security descriptors, and its
slave nodes are materialised by the kernel when a terminal opens
/dev/ptmx — never through a create path that could stamp or inherit
one — so under the default DENY_MISSING every pseudo-terminal on the
machine is unopenable. The policy is set through
SYS_KACS_SET_MOUNT_POLICY on an O_PATH descriptor of the mount
point. The template applies to every SD-less inode of the mount alike,
so any authenticated principal may open any slave; per-owner slave
descriptors need kernel support for stamping the opener's identity at
materialisation, and are recorded as future work, not provided here.
/sys/kernel/security gets the same kind of policy, for the same
reason: securityfs cannot store descriptors either. Its template grants
SYSTEM full control and Authenticated Users read and traverse
:
O:SYG:SYD:(A;;GA;;;SY)(A;;GRGX;;;AU)
KACS publishes kacs/self and kacs/sessions there (see
Logon sessions), and
each makes its own check when read, so opening sessions is allowed to
anyone signed in while reading it stays with Administrators and SYSTEM.
Only SYSTEM may write: lockdown is the one writable file on the mount,
and raising the kernel's lockdown level is not routine administration.
There is a bootstrap wrinkle in reading mountinfo at all: the file lives
in /proc, which is one of the things being checked for. If the read
fails with ENOENT or ENOTDIR, peinit mounts /proc from the table
and retries. Any
other failure to read or parse mountinfo sends peinit to recovery.
For the three initramfs-provided rows, an already-mounted filesystem is
success, and so is an EBUSY from an attempted mount.
For the five
peinit owns, a mount failure sends peinit to recovery.
After successful mount setup, peinit ensures /run/lock exists as a
directory. Mounting /run
hides the directory shipped by fsbase, so peinit recreates it in the live
filesystem before starting services. A newly created directory inherits
/run's descriptor; an existing directory is preserved. This step also
runs when /run was already mounted. A creation failure enters recovery.
2.3.2.1 Seeding descriptors on the new filesystems #
Three of the five filesystems peinit mounts are fresh and empty:
/dev/shm, /run and /sys/fs/cgroup. Under KACS an inode with no
Security Descriptor is denied to every caller, and there is nothing on a
newly mounted tmpfs for a new inode to inherit from — so peinit stamps
the mount root with a descriptor that grants SYSTEM and Administrators
full control, inheritable by both containers and objects, and grants
Everyone read and execute, inheritable by containers only:
O:SY G:SY D:(A;OICI;GA;;;SY)(A;OICI;GA;;;BA)(A;CI;GRGX;;;WD)(A;OICIIO;GA;;;S-1-3-0)
Everything created underneath — the notify socket, per-service runtime
directories, the cgroup hierarchy — inherits from it; the two sockets
that need a different population are stamped explicitly in step 9. This is why peinit never sets mode bits on the sockets it creates;
under KACS they would mean nothing, and the descriptor is the thing that
does the work. It is the same descriptor the initramfs seeds onto the
root filesystem but for the inheritance flags on that last ACE, and the
two are kept in step on purpose: this one inheritable ACL is, in
practice, the access policy of everything under these mounts, so an ACE
missing here is missing from every per-service directory under
/run/services.
The Everyone ACE grants execute as well as read because on Peios execute
is traverse — FILE_TRAVERSE and FILE_EXECUTE are the same bit — and
an explicit chdir is the one traversal that does not get the
SeChangeNotifyPrivilege bypass. A service running as anything other
than SYSTEM cannot otherwise reach its own runtime directory. SYSTEM and
Administrators keep GenericAll rather than read/write/execute because
GenericAll carries WRITE_DAC, WRITE_OWNER and DELETE, and peinit
re-stamps descriptors throughout /run.
The last ACE is CREATOR OWNER, and it is what makes the container-only
Everyone ACE survivable. A file created under these mounts inherits the
SYSTEM and Administrators ACEs and nothing else — and that is a
non-empty DACL, so the creating token's default DACL is never
consulted. Without it a service would create runtime state it could not
read back. It is inherit-only, so it grants nothing on the directory it
sits on; it is resolved against whoever creates each object, and carried
onward down each container.
The Everyone ACE is inheritable by containers only, which is the one place this descriptor differs from the root filesystem's. What a service needs here is to walk to its own directory, and container inheritance gives it that by itself. Object inheritance would additionally put Everyone-read on every file anything creates beneath these mounts — a service's runtime state, and shared memory segments, readable by any principal through inheritance alone. The root filesystem needs object inheritance because a service has to read and execute the binaries beneath it; nothing under these three mounts is a binary.
This descriptor is not the one prelude's seed-sd writes by default.
That one is narrower, and stays narrower: it also stamps /dev, where
every ACE is inherited by the next hot-plugged block device, so a read
ACE for Everyone there would be every filesystem descriptor on the
machine bypassed by opening the raw disk.
Failure to apply the descriptor sends peinit to recovery. Without it every file peinit later creates on that filesystem would be unreachable to everything, including peinit.
/proc, /sys and /dev are not stamped: they arrive from the
initramfs already populated.
2.3.2.2 Device node policy #
/dev arrives with that same inheritable descriptor on its root and on
every node, which is the right default — whatever the root grants, a
disk hot-plugged later inherits, so the root must be acceptable on a raw
block device — but it leaves /dev/null unusable by anyone who is not
an administrator. A single inherited descriptor cannot say "/dev/null
for everyone, the disks for administrators", so peinit enumerates the
exceptions. Once the mounts are up it stamps each of these nodes with a
descriptor of its own:
| Node | DACL |
|---|---|
/dev/null, /dev/zero, /dev/full, /dev/random, /dev/urandom, /dev/tty, /dev/ptmx | D:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FRFW;;;WD) |
Everyone may open the node for reading and writing and may stat it;
nobody but SYSTEM and Administrators may change its descriptor. Only the
DACL is replaced — owner and group stay as the seed left them
— and the ACEs
carry no inheritance flags, because a device node has no children.
/dev/console is deliberately not on the list: it is the SYSTEM
console.
The step is advisory. A node that cannot be stamped is reported as a warning and stays on the inherited default, usable by administrators and denied to everyone else; a node that does not exist is noted and skipped. Neither sends peinit to recovery.
2.3.3 Step 3: Restore the persisted random seed #
peinit restores the seed at /var/state/peinit/random-seed once /dev
is available and before registryd starts. The seed is a machine-local
entropy cache for the kernel CSPRNG. It is not configuration, and
shipping one in a packaged image, live ISO or VM template would hand
every instance of that image the same starting entropy.
If the file is absent, that is an ordinary first boot or a stateless live boot, and peinit continues silently. When a seed is present peinit mixes it into the kernel pool, preferring the interface that credits entropy for a locally persisted seed; if that fails it mixes the bytes without crediting and records the failure. A seed file that is empty, or larger than 4096 bytes, is treated as an error.
Nothing in this step can send peinit to recovery. A system with no entropy cache still boots; it just starts with less entropy, which is a problem for the image builder to solve with a hardware or virtio RNG rather than with a seed baked into the image.
The initramfs may perform the same restore earlier, once the persistent root is mounted. peinit's restore stays as the fallback for initramfs images that do not participate and for boots that have no initramfs.
2.3.4 Step 4: Ensure the local machine ID #
/lcl/etc/machine-id holds a stable local install identifier used for
software compatibility, log correlation and instance identity. It is not
a security principal: not a credential, not a SID, not an account, and
not an input to any authorisation decision.
The format is 128 bits as exactly 32 lowercase hexadecimal characters followed by one newline. A valid existing file is left alone. A file that is absent, empty, all zeroes, the wrong length, not hexadecimal, or missing its trailing newline is replaced : peinit draws 128 bits from the kernel CSPRNG and writes a valid file atomically, through a temporary file and a rename, with the result flushed.
The write creates /lcl/etc/ if it is not there. This step runs before
the path-provisioning machinery of step 7, and /lcl is a StrataFS view
the initramfs assembled, so whether the directory exists is a property of
the image rather than something peinit can assume.
Only a CSPRNG failure sends peinit to recovery. If the kernel cannot produce random bytes, nothing else on the machine can be trusted either.
An unreadable file or an unwritable path does not. The boot continues with an identifier valid for this boot only, and says so :
peinit warning: machine-id not persisted (<reason>); using an identifier
for this boot only
A read failure takes the same path as finding no file, because it leaves peinit in the same position — unable to say whether a valid identifier is on disk. It is reported rather than treated as a clean generation, because it may have replaced something valid.
This is deliberately fail-soft. The identifier is a local opaque install ID and not an authorisation input, so failing a boot over it would be disproportionate — and an ephemeral ID is already the defined behaviour for a stateless live boot.
Images and templates are expected to ship with no machine ID, or with an empty file as a reset marker. Clone tooling that wants a new identity removes or truncates the file and lets the next boot generate one. Stateless live boots without a persistent overlay get an ephemeral ID for that boot.
2.3.5 Step 5: Set the clock from the hardware RTC #
peinit reads the hardware clock and calls clock_settime() before
registryd starts, so that timestamps on registry operations, log entries
and the boot attempt counter mean something.
It opens /dev/rtc, falling back to /dev/rtc0 if that device is
absent , and reads it with RTC_RD_TIME.
The returned struct rtc_time is interpreted as UTC and converted to
CLOCK_REALTIME seconds with zero nanoseconds.
Every failure in this step sends peinit to recovery: no openable RTC
device, a failed read, a value that is invalid or before the Unix epoch,
a failed clock_settime, and — deliberately — a failure to close the
descriptor after a successful read.
Leaking a descriptor in PID 1 during
bootstrap is a symptom of something being badly wrong, not a detail to
swallow.
2.3.6 Step 6: Start registryd #
peinit holds a compiled-in definition for registryd — the only compiled-in service definition there is :
| Field | Value |
|---|---|
| ImagePath | /sbin/registryd |
| Arguments | Machine=/var/state/loregd/Machine.hive, Users=/var/state/loregd/Users.hive |
| Identity | SYSTEM |
| Readiness | Notify |
| ErrorControl | Critical |
The hive paths are peinit's choice, not registryd's: where the machine registry lives is a boot-policy decision, and it is made here because this is the one service start that cannot consult configuration.
peinit mints a SYSTEM token including registryd's per-service SID,
creates the cgroup tree, forks with the token installed, and execs
/sbin/registryd through the runtime StrataFS view.
Two separate timeouts
bound the start, both 30 seconds: one on process setup, driven
synchronously because there is no event loop yet, and one on readiness.
registryd's READY=1 means "accepting and serving registry requests",
not "the process is alive" — it does not signal until its storage
backend is open, its schema is validated and it can answer a read.
2.3.6.1 The schema-version guard #
After readiness, peinit ensures the base registry structure exists and
then probes it. Ensuring comes first: peinit creates Machine\System,
Machine\System\Services and Machine\System\Init if they are absent
and stamps Machine\System\Services\SchemaVersion with the current
schema version, 1. Only then does it read the value back.
The read is what verifies registryd is genuinely serving. A value that
is present but not a REG_DWORD, or that is a REG_DWORD of the wrong
length, fails the probe.
A key or value that is absent reads as zero and passes
— the structure was just
created, so absence at this point means the write did not take effect,
and the failure that matters is the provisioning failure, which is
reported directly.
The consequence is that the guard is self-healing. An unprovisioned
first boot, or a registry cleared by the recovery tools, comes up with
an empty Machine\System\Services\ and boots into a Phase 2 with no
services rather than into recovery.
2.3.6.2 Keeping registryd #
When registryd passes readiness and the probe, peinit retains the activation as an ordinary runtime service instance: its state, its pidfd-tracked main process, its cgroup generation and paths, its output pipe ownership, its notify generation, its job identity, and any cleanup evidence. Ownership is not dropped at the Phase 2 boundary.
During Phase 2 the registry's own definition of registryd, if there is
one, is merged onto the retained activation. peinit does not create a
second inactive record and does not restart registryd because a
definition has appeared.
If
the registry definition is absent or invalid, the ordinary graph
validation rules apply.
If registryd fails to start, its readiness times out, or the probe fails, peinit enters recovery. There is no Phase 2 without a registry.
2.3.7 Step 7: Autorun scripts #
Between registryd starting and path provisioning, peinit runs every
non-directory entry in /lcl/policy/autorun.d, in sorted order, by
absolute path , with
the working directory / and PATH=/sbin:/bin
. Each runs under peinit's own
SYSTEM token.
The step is fail-open at every point: a missing directory, an unreadable directory, a spawn failure and a non-zero exit are all console warnings and none of them stops the boot. Its console output bypasses the quiet policy (§2.6), because a script that ran this early and went wrong needs to be visible.
2.3.8 Step 8: Provision boot-time paths #
Covered in §2.4.
2.3.9 Step 9: Infrastructure setup #
Three things, before Phase 2 begins:
- The control socket at
/run/services/peinit/control.sock, which serves every runtime command for the lifetime of the system, stamped so that any authenticated principal may connect and every command is then checked on its own (§10.1). - The jobs socket at
/run/services/peinit/jobs.sock, on which any authenticated principal may submit a job once Phase 2 runs, stamped so that connecting is the submission permission (§10.7). - Loopback: peinit brings up
loover netlink, because services that bind127.0.0.1need it.
Either socket failing to bind or to take its descriptor sends peinit to recovery — without the first there is no way to administer the system, and a jobs socket with the wrong descriptor is either unreachable or open to everything. A loopback bring-up failure is a warning that lets Phase 2 proceed.
2.3.10 Failure summary #
| Failure | Response |
|---|---|
| Root writability probe fails | Recovery |
| A mount point cannot be created | Recovery |
| A peinit-owned filesystem fails to mount | Recovery |
| A mounted filesystem cannot be stamped with its descriptor | Recovery |
/proc/self/mountinfo unreadable or unparseable | Recovery |
/proc, /sys or /dev already mounted, or EBUSY | Tolerated as success |
| A device node in the policy list cannot be stamped | Warning; node keeps the inherited default |
| A device node in the policy list does not exist | Noted; boot continues |
| Random seed absent, oversized, empty, or unrestorable | Warning; boot continues |
| Machine ID generation fails (CSPRNG) | Recovery |
| Machine ID unreadable, or unwritable at its path | Warning; boot continues with an identifier for this boot only |
| Machine ID absent, empty, or malformed | Regenerated; boot continues |
| Any RTC or clock failure | Recovery |
| registryd fails to start, or setup times out | Recovery |
| registryd readiness times out | Recovery |
| Base registry provisioning fails | Recovery |
| Schema-version probe returns a wrong type or length | Recovery |
| An autorun script is missing, unspawnable, or exits non-zero | Warning; boot continues |
| A provisioning entry is malformed | Warning; entry skipped |
| An optional provisioned path fails | Warning; boot continues |
| A required provisioned path fails | Recovery |
| Control socket creation or stamping fails | Recovery |
| Jobs socket creation or stamping fails | Recovery |
| Loopback bring-up fails | Warning; boot continues |
2.4 Path Provisioning
Peios / Peios technical reference / peinit / Boot
Some filesystem objects belong to no single service. A directory two packages both write into, a state file created before anything runs, a path that has to exist with a particular Security Descriptor before the first service that uses it starts — none of these has an owner in the service model, and creating them from a service's pre-exec hook makes their existence depend on start ordering.
Boot-time path provisioning is the registry-backed answer, and the equivalent of the tmpfiles.d role elsewhere. peinit applies it after registryd is serving and before any Phase 2 service is planned or started.
2.4.1 Entries #
Each child key under Machine\System\Init\ProvisionedPaths\ is one
entry. Unknown values on an entry
are ignored.
| Value | Type | Required | Default | Meaning |
|---|---|---|---|---|
Kind | string | yes | — | directory or file. |
Path | string | yes | — | Absolute path to create or verify. |
Security | binary | no | built-in | The Peios file Security Descriptor to apply. |
Required | dword | no | 0 | If 1, failing this entry prevents Phase 2. |
For Kind=directory peinit ensures the path exists as a directory; for
Kind=file it ensures the path exists as a regular file.
In either case a path that
exists with a different file type fails the entry.
An existing file is opened
rather than created, so provisioning never truncates one.
peinit does not create parent directories. The parent is checked, and has to already be a directory. A package that needs a hierarchy declares each directory explicitly, or depends on the package that owns the parent — which keeps the ownership of every directory traceable to a package rather than to whichever entry happened to run first.
2.4.2 Descriptors #
When Security is present, peinit applies the supplied binary
descriptor. A malformed or
rejected descriptor fails the entry.
When it is absent, peinit applies a built-in default :
O:SY G:SY D:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FR;;;BU)
SYSTEM and Administrators get full control; ordinary users get
FILE_GENERIC_READ.
2.4.3 Failure #
Entries with Required=0 are fail-soft: peinit logs and continues.
Entries with Required=1 are
fail-closed: peinit logs and enters recovery before Phase 2 starts.
An entry that is malformed — a missing or unrecognised Kind, a missing
or relative Path, a value of the wrong type — is logged as a warning
and skipped, regardless of Required.
Required marks a path as
essential to boot; it does not make a broken entry more dangerous than a
missing one.
2.5 Phase 2
Peios / Peios technical reference / peinit / Boot
With registryd serving, peinit reads the service graph and boots the system from it. Phase 2 is entirely registry-driven.
2.5.1 Reading the definitions #
peinit reads every key under Machine\System\Services\.
The reads are bounded by
LCS's request timeout: if registryd hangs mid-read, peinit receives
ETIMEDOUT and enters recovery.
Decoding is per key. A definition that fails to decode — an invalid
service name, a malformed trigger, an unclosed quote in a command, a
registry: check naming an uncacheable key, a duplicate known field, an
unrecognised value for an enumerated dword — fails that service, which
is marked Failed with cause ValidationError.
The boot
proceeds with every other definition, and anything that depended on the
failed service fails in turn through the ordinary dependency propagation.
peinit holds no definition for such a service, so the entry behind it
is a placeholder: status reports it Failed with cause
ValidationError, marked definition-removed (§3.8), with the decode
error as its description. start, restart, reload and reset are
refused with UNKNOWN_SERVICE, as for any definition-removed service.
The placeholder leaves with its key: a reload-config that re-reads a
repaired key restores the entry, and one that finds the key deleted
discards it, since Failed does not retain a definition-removed entry.
Only services carrying a boot trigger are root candidates.
A service with no
triggers is demand-only and is not a root, though it can still be pulled
into the boot transaction as somebody's dependency.
A service with
Disabled=1 is excluded from the boot graph entirely, but its definition
is still loaded into the in-memory model so it can be started by hand
later.
2.5.2 Building and validating the graph #
The boot graph is every boot-triggered root candidate plus the
transitive closure of their Requires, BindsTo, and existing
non-disabled Wants dependencies.
Requires and
BindsTo pull their target in even if the target has no boot trigger.
Wants
targets come in as best-effort members; missing or disabled ones are
ignored. A missing or
disabled Requires or BindsTo target blocks the dependent with cause
DependencyFailure, and that blocking propagates.
peinit topologically sorts the graph and validates it before starting anything. The rules are in §7.2; the outcomes that matter here are:
- A cycle fails every service in it. A cycle involving a Critical service downgrades the boot to Safe mode without rebooting.
- An unresolvable conflict fails both services. Same downgrade if either is Critical.
- A missing
Requirestarget fails the dependent. - Warnings — a
Readiness=Aliveservice with dependents that require it — are logged and do not prevent boot: a[ WARN ]peinit warning:line on the console and apeinit.graph.validation.warnedevent under phaseboot, whether or not the same graph also has findings.
2.5.3 Starting #
peinit walks the graph and starts services, parallelising wherever the graph allows, up to a configurable limit:
An absent key uses the default. A value of zero, a type mismatch, or a malformed payload is invalid boot configuration and sends peinit to recovery — running the scheduler with an effective limit of zero would hang the boot rather than fail it.
As each service reaches a dependent-satisfying state — Active for
Simple, Completed for Oneshot with or without RemainAfterExit, Skipped
for a service whose conditions did not hold — its dependents become
eligible and join the start queue.
A Oneshot without
RemainAfterExit passes through Completed, releasing its dependents, and
then goes Inactive.
Dependents blocked on a Requires or BindsTo target wait for that
target to reach a satisfying state.
Dependents
blocked on a Wants target wait only for it to reach any terminal
state, satisfying or not — which is what makes Wants ordering rather
than dependency.
2.5.3.1 The typical order #
Nothing below is hardcoded. It falls out of the dependency graph that the standard role definitions produce, and an administrator who changes a dependency gets a different order.
- eudev — device management. SYSTEM, with
RequiredPrivilegesstripped to the minimum it needs. Starts before authd exists. - lpsd — the local identity database. Depends only on registryd.
- authd — the identity authority. Depends on registryd and lpsd. Once it is ready, token minting is available.
- eventd — logging and audit. Services started before it log into peinit's pre-eventd buffer.
- Networking — the first service to receive a token from authd.
- Application services, in dependency order.
- Login services, last, so the system is operational before it accepts a session.
2.5.3.2 The bootstrap matrix #
| Service | Phase | Identity | Token from | Readiness | ErrorControl |
|---|---|---|---|---|---|
| registryd | 1 | SYSTEM | minted by peinit | Notify | Critical |
| eudev | 2 | SYSTEM | minted by peinit, privileges stripped | Alive | Normal |
| lpsd | 2 | SYSTEM | minted by peinit | Notify | Critical |
| authd | 2 | SYSTEM | minted by peinit | Notify | Critical |
| eventd | 2 | SYSTEM | minted by peinit | Notify | Critical |
| networking | 2 | service token | authd | Notify | Normal |
| sshd | 2 | service token | authd | Alive | Normal |
| application services | 2 | service token | authd | per-service | Normal |
2.5.4 Deferred starts #
A service whose trigger is boot:settled is not part of the boot plan
at all. It is not a root, it does not consume the parallel-start budget,
it is not counted towards boot success, and it cannot block or delay
anything. peinit starts it after the plan, once the boot has stopped
moving.
"Stopped moving" is precise: every service in the plan — those that were started and those that were blocked — is in a state it will not leave without help. Active, Completed, Failed, Skipped, Abandoned and Inactive all count as settled. Starting, Reloading, Stopping and Backoff do not, because a service between restart attempts is going to produce more output.
A deadline bounds the wait:
| Key | Default | Meaning |
|---|---|---|
Machine\System\Boot\SettleTimeout | 5 | Seconds before deferred services start regardless. |
The deadline is measured from the moment the plan was observed. An absent key uses the default; a type mismatch or a bad length sends peinit to recovery. Zero is legal and means "start on the next turn, settled or not".
Whichever comes first — the set settling or the deadline expiring — the deferred services start, once, each independently. A start that fails is recorded and dropped: one refusing service does not stop the others and does not affect the boot. The dispatch carries a flag saying whether the deadline expired rather than the set settling, so a service that cares whether the boot was still moving can be told.
2.5.5 Boot success #
A boot is successful once every Critical service has held a dependent-satisfying state continuously for a grace period:
| Key | Default | Meaning |
|---|---|---|
Machine\System\Boot\BootSuccessGrace | 30 | Seconds of held health before the boot counts. |
The criterion is satisfying, not Active. A Critical Oneshot reaches Completed and never reaches Active, so a test for Active would make such a service unable to ever mark a boot successful. Skipped counts too.
Success resets the boot attempt counter to zero (§2.7). The control
socket's boot command says where a boot stands: which Critical
services it is waiting for, or when it will count, or that it has
(§10.8).
2.5.6 Failure summary #
| Failure | Response |
|---|---|
| A service definition fails to decode | That service Failed with cause ValidationError; the rest of the boot proceeds |
| A registry read times out | Recovery |
Invalid MaxParallelStarts or SettleTimeout | Recovery |
| A dependency cycle | All services in it Failed; Safe mode if any is Critical (see below) |
| An unresolvable conflict | Both Failed; Safe mode if either is Critical (see below) |
A missing Requires target | The dependent Failed |
| A Critical service fails during boot | Restart budget, then reboot |
| A non-Critical service fails | Failed; its Requires dependents fail; the rest continue |
| authd unavailable when a service needs a token | That service Failed |
The two Safe-mode rows carry a caveat. When the downgrade fires, peinit rebuilds the graph in Safe mode and discards the Full-mode one, so the services that caused it are not marked Failed — they are never entered into the blocked set at all. What records them is the boot-level downgrade finding described in Boot modes.
2.6 Boot Modes
Peios / Peios technical reference / peinit / Boot
peinit boots in one of three modes, forming an escalation path from normal operation to last-resort maintenance.
Full boot ---+-- success ----------------> counter reset, operational
|
+-- cycle w/ Critical ------> Safe mode (no reboot)
|
+-- conflict w/ Critical ---> Safe mode (no reboot)
|
+-- Critical failure -------> sync + reboot --+
|
Safe boot ---+-- success ----------------> counter reset, |
| operational (reduced) |
+-- Critical failure -------> sync + reboot --+
|
counter increments <----------+
|
counter >= N ---------> Recovery mode
Phase 1 failure ------> Recovery mode
2.6.1 Full mode #
The default. Every boot-triggered service starts in dependency order, as §2.5 describes.
2.6.2 Safe mode #
Safe mode starts a reduced set. Eligibility is a filter within the
boot-triggered set, not a replacement for it: a service with no boot
trigger does not auto-start in Safe mode whatever its SafeMode or
ErrorControl says, and remains demand-only.
Within the
boot-triggered set, two categories are eligible:
- Critical services (
ErrorControl=Critical) start. If one fails, the ordinary Critical failure path applies — restart budget, reboot, counter increment, eventually recovery. SafeMode=1services are attempted best-effort. If one fails, Safe mode continues without it.
ErrorControl=Critical implies SafeMode, so a Critical service does
not have to declare both.
2.6.2.1 What caused the downgrade #
The rebuild discards the Full-mode graph, so a service that forced Safe
mode and is then excluded from the rebuilt graph is never entered into
the blocked set and never marked Failed. That is deliberate: Safe mode
was never going to start it, and a Failed state would say something
about its own health that is not true. status should keep meaning
"this service is broken".
A service that survives into the rebuilt graph gets no such protection.
Two Critical services that require each other are both eligible in
Safe mode, so the rebuild contains the same cycle and blocks them both,
exactly as it would anywhere else — and there is no order that could
have started either.
The reason is therefore recorded at boot level rather than per
service. Every finding that forced the downgrade — each critical cycle,
each critical boot conflict — is written to the console and emitted as a
peinit.boot.downgraded KMES event naming the services involved, an
essential type the emission policy cannot switch off.
All of them are reported, not just the first. A machine can be downgraded by a cycle and a conflict at once, and an operator who fixed only the one they were shown would reboot straight back into Safe mode.
They stay available for the rest of the boot: the control socket's
boot command reports the mode, why, and every finding (§10.8).
peinit rebuilds the dependency graph from scratch using only the eligible services. Dependencies on excluded services are dropped: if A depends on non-Critical B and B is excluded, A's dependency on B does not exist in the Safe mode graph. This is what makes Safe mode useful — it is a graph in which the broken parts of the configuration are simply not present, rather than a graph in which they are present and failing.
A successful Safe boot resets the boot attempt counter.
2.6.2.2 Entry #
- A cycle involving a Critical service at boot. Graph validation detects it and peinit downgrades in place, without rebooting — the cycle is a configuration error, and rebooting would find it again.
- An unresolvable conflict involving a Critical service at boot. Same reasoning.
peios.safemode=1on the kernel command line.
Safe mode is not entered because a Critical service crashed at runtime. That follows the ordinary path: restart budget, reboot, counter increment, recovery.
2.6.3 Console output #
peios.quiet=N bounds what peinit writes to the console:
The two rules are independent, and an error is never less visible at 2
than at 1. A terminal is matched by device rather than by path, since
/dev/console and /dev/ttyS<n> can name the same device
; where the device
cannot be determined, peinit treats the terminal as free and writes to
it. Guessing wrong
in that direction costs a scrambled line, which is recoverable;
guessing wrong the other way costs the operator their console output at
the moment the machine is least able to explain itself.
Suppressed messages are discarded rather than buffered.
The autorun step (§2.3) bypasses the policy: a script that ran that early and went wrong is worth interrupting a login prompt for.
2.6.4 Kernel command line #
A malformed value is ignored in favour of the default rather than failing the boot. Nothing exists this early to report a diagnostic to.
2.7 The Boot Attempt Counter
Peios / Peios technical reference / peinit / Boot
The counter is what turns a repeated failure into an escalation. peinit
keeps it at /.peinit/boot-attempts, as a plain decimal integer in a
file on the root filesystem
— deliberately not in the registry, because the registry may be the
reason the boot is failing.
2.7.1 The cycle #
peinit reads the counter at startup, before selecting a boot mode. The recovery threshold is evaluated against this pre-increment value, so a default threshold of 3 admits exactly three boot attempts before recovery.
- Absent file: treated as 0.
- Unreadable, empty, non-decimal, carrying trailing non-whitespace data, or overflowing the counter representation: recovery. A counter that cannot be read cannot be trusted to escalate.
peinit increments once per boot, after the root is known writable and before Phase 2 begins. Incrementing before the root is known writable would silently lose the increment on a read-only root and defeat escalation entirely.
The increment happens after the mount, seed, machine ID and clock steps
rather than immediately after the writability probe, because reading the
kernel command line requires /proc. One consequence is that a recovery
entered from a mount failure, a machine ID failure, or an unreadable
command line does not advance the counter.
A write failure — a full disk, say — is treated as a counter of 0 and the boot continues. A failure to record an attempt is not itself a reason to escalate.
The counter resets to 0 on a successful Full or Safe boot, after the grace period.
The control socket's boot command reports the count this boot began
with, the threshold, and whether this boot has reset the counter yet
(§10.8).
2.7.2 Recovery threshold #
The threshold is peios.bootattempts=N on the kernel command line,
defaulting to 3. It is a
command-line value rather than a registry one because the check runs in
Phase 1, before registryd is serving.
peios.bootattempts=0 disables the check entirely — the escape hatch
for a system whose counter is itself the fault.
peios.recovery=1 forces recovery without consulting the counter, but
does not suppress the increment.
2.8 Recovery Mode
Peios / Peios technical reference / peinit / Boot
Recovery mode is the last resort. There is no TCB guarantee and no degraded boot to speak of — it is a maintenance environment that hands the administrator an unrestricted SYSTEM shell on the console.
2.8.1 Entry #
- The boot attempt counter reaching the threshold (§2.7).
peios.recovery=1on the kernel command line.- Any of the Phase 1 failures listed in §2.3, most importantly a registryd that will not start or will not serve.
- A Phase 2 registry read that fails or times out, or invalid boot configuration.
- A required provisioned path that cannot be created or secured.
- The runtime loop failing on an error about supervision itself: the
event wait, event source registration, a listener, the event ring
refusing even the small
peinit.event.dropped, shutdown finalisation. An error peinit can attribute to one service is contained to that service instead and does not enter recovery (§8.2).
2.8.2 What peinit does #
peinit records the reason as a peinit.recovery.entered KMES event —
essential, so no emission policy switches it off — with the reason in
outcome.reason and what went wrong, in the error's own words, in
outcome.detail; a Phase 2 plan that would not build adds its finding
as a peinit.graph.validation.failed under phase phase2-boot
, then:
- Completes Phase 1 steps 1–5 if they have not been reached yet.
- Ensures the base registry structure exists, so the shell sees a normal layout even on a system that has never been provisioned.
- Attempts to start registryd. A failure here is ignored — recovery delivers a shell whatever registryd's state.
- Skips all Phase 2 services.
- Starts a shell on
/dev/consolefrom a compiled-in definition, with no registry dependency:/bin/recshif it is present and executable, otherwise/bin/sh, running as SYSTEM with a fixed environment ofPATH=/sbin:/bin,TERM=linuxandHOME=/. peinit does not care where either binary comes from. - Logs the failure reason to the console.
If the shell exits, peinit respawns it. Recovery never exits to an unmanaged PID 1.
If neither /bin/recsh nor /bin/sh can be exec'd, peinit cannot
deliver a shell at all. It logs the reason to the console, syncs, and
halts — PID 1 exiting would panic the kernel.
A missing shell is a
binary-integrity failure and sits outside the boot-attempt machinery's
remit.
The shell receives /dev/console duplicated onto its standard streams,
but peinit does not call setsid() or acquire a controlling terminal
for it. The shell is not a session leader, so job control is not
available in the recovery shell.
Step 1 runs on every entry. The steps are individually idempotent, so completing them and ignoring the failures is what "if they have not been reached yet" amounts to in practice — including the step that failed, which by the time the operator has a shell may well succeed.
Step 3 runs only if Phase 1 has not already reached its own registryd
start, and it runs at most once.
A recovery entered from a Phase 2,
provisioning or runtime failure starts no registryd: Phase 1 already
did, and its activation is retained for the runtime. A recovery entered
from the registryd start failing also starts none — the attempt has
been made, and a process may have forked before the failure was
reported. In both cases a second daemon would bind over the first's
notify socket, which succeeds rather than reporting EADDRINUSE
because the bind unlinks the path first, and would then open the same
hive files behind the first daemon's back.
Where recovery does start one, it uses the settings this boot parsed, so
peios.notifysocket= and peios.quiet apply to it.
2.8.3 Registry diagnosis and recovery limits #
Recovery provides a console maintenance shell when one is available as described above; it does not define an offline storage-repair interface for the registryd provider. Preserve the console failure reason and source startup output, then identify the deployed provider and version, its hive declarations, and the actual database paths. See LCS and sources.
Entering Recovery does not establish that the source is stopped. As explained above, a registryd started or attempted earlier in the boot can remain active. The loregd exit contract also permits a source to keep running after storage-operation failures. Do not launch a second source against the same hive files.
The source-checked loregd implementation
takes one or more HiveName=DatabasePath declarations and rejects arguments
without =. It has no offline inspection, backup-recovery or database-clearing
switches and does not create an automatic startup backup.
Startup WAL recovery and orphan cleanup are not backup restoration or a
database-corruption repair facility. The linked references pin these findings
to the implementation revision checked; identify the installed provider and
revision before choosing a recovery procedure.
Offline inspection, repair or database replacement therefore needs a procedure supported by the installed provider and version. Establish the source's state and verify the existence and coverage of a recovery copy before planning a replacement. This manual supplies no offline stop, repair or replacement sequence. Role definitions can re-supply service configuration; they do not constitute a backup of every registry value.
The ordinary Backup and restore workflow is separate: LCS coordinates operations through a working source. Backup requires a read-only snapshot transaction; restore replaces a target subtree, including descriptors, in a read-write transaction. Neither is a way to repair a source that cannot serve those operations.
2.8.4 Remote recovery #
Recovery requires console access: physical, IPMI or serial. Two post-v1 features address headless servers — rolling the registry back to a last-known-good state from the recovery shell, and an emergency sshd started without registry involvement.
3.1 Services
Peios / Peios technical reference / peinit / The Service Model
A service is the primary unit of management: a definition in the
registry, a runtime state, a Security Descriptor, and at most one
running main process. Definitions live under
Machine\System\Services\<name>, where the key name is the service
name. peinit reads them at Phase
2 boot and on an explicit reload-config.
3.1.1 Names #
A service name is 1 to 128 bytes drawn from [A-Za-z0-9._-]. Any other
byte makes the name invalid.
Two exclusions are deliberate. / is out because names map directly
onto cgroup identifiers (§5.1) and onto registry key names, and a name
containing a separator would mean something different in each. : is
reserved for peinit's own synthetic naming.
3.1.2 The two types #
3.1.2.1 Simple #
A long-running daemon. peinit forks, installs a token, and execs the binary; the process is the service. When it exits, the service has stopped. This is the default and covers nearly everything — registryd, authd, sshd, application services.
Readiness comes from the Readiness field. Notify, the default,
waits for READY=1. Alive treats the process as ready the moment it
exists.
The service goes Active on readiness and stays Active until the process exits or something stops it.
3.1.2.2 Oneshot #
A run-to-completion task: database initialisation, a schema migration, a directory that has to exist. peinit forks, installs a token, execs, and waits for the exit.
Readiness is ignored. A
Oneshot's readiness is always "it exited successfully", because
READY=1 is meaningless from a process whose job is to finish. Success
means exit code 0, or any code listed in SuccessExitCodes.
The differences from Simple are:
- A successful exit goes to Completed. With
RemainAfterExit=1it stays there; without, it passes through Completed to release dependents and then goes Inactive. - A non-zero exit goes to Failed.
ExecStartPostruns after the successful exit rather than after a readiness signal, and does not run at all if the Oneshot failed.StartTimeoutcovers the entire execution, from the first pre-hook to the process exiting.
RemainAfterExit matters when the Completed state itself is the useful
information — so a status query shows a migration as finished rather
than as inactive.
3.1.3 Forking daemons #
peinit does not support them. A service that double-forks to daemonise
itself is working around a problem that does not exist when the service
manager tracks the process it spawned, and peinit tracks its child
through a pidfd obtained at fork.
There is no MAINPID=, and
no way to point supervision at a different process.
A legacy binary that insists on double-forking is wrapped by whoever
packages it — a script with a --no-daemon flag, typically. That is a
packaging concern.
3.2 The Definition Schema
Peios / Peios technical reference / peinit / The Service Model
Every field a service definition can carry, with its registry type and its default. The semantics of each are in the section named alongside.
Value names are matched case-insensitively, so ImagePath and
imagepath are the same field
— and therefore a definition carrying both is a duplicate, not two
fields.
| Field | Type | Default | Meaning |
|---|---|---|---|
| ImagePath | string | required | Absolute path to the service binary. |
| Arguments | multi_string | — | Arguments passed to the binary. |
| Type | dword | 0 (Simple) | 0 Simple, 1 Oneshot. §3.1 |
| Triggers | multi_string | — | When the service starts automatically. §3.4 |
| Disabled | dword | 0 | If 1, no trigger activates the service. |
| SafeMode | dword | 0 | If 1, attempt this service in Safe mode. Implied by ErrorControl=Critical. §2.6 |
| Identity | string | LocalService | SYSTEM, shared LocalService/NetworkService, Service for the service’s own virtual account, or a designated principal. §4.1 |
| RequiredPrivileges | multi_string | — | Privileges to keep; all others are removed. §4.5 |
| Requires | multi_string | — | Hard dependencies. §7.1 |
| Wants | multi_string | — | Soft dependencies. §7.1 |
| BindsTo | multi_string | — | Runtime coupling. §7.1 |
| Conflicts | multi_string | — | Mutual exclusion. §7.1 |
| Provides | multi_string | — | Roles this service fills: virtual names a dependency may target (network), and that peinit derives edges to. A role is a bare name — the level in network:routed comes from the depending service's readiness requirement, not from this value. §7.6 |
| OnFailure | string | — | Service to start when this one fails. §6.3 |
| ErrorControl | dword | 0 (Normal) | 0 Normal, 1 Critical. |
| RemainAfterExit | dword | 0 | Oneshot only: stay Completed after a successful exit. |
| SuccessExitCodes | multi_string | — | Non-zero exit codes treated as success. |
| ExecStartPre | multi_string | — | Commands run before the main binary, sequentially. §5.3 |
| ExecStartPost | multi_string | — | Commands run after readiness or successful exit. §5.3 |
| HookIdentity | string | — | Principal for the hook processes. Falls back to Identity. §4.1 |
| ExecReload | string | — | Reload command, or signal:<NAME>. Absent means SIGHUP. §6.5 |
| PreStartCheckTimeout | dword | 5 | Seconds before a filesystem check helper is killed. §3.5 |
| StartTimeout | dword | 30 | Seconds for the entire start sequence. §5.3 |
| StopTimeout | dword | 10 | Seconds after SIGTERM before SIGKILL. |
| WatchdogTimeout | dword | 0 | Seconds between expected WATCHDOG=1 pings; 0 disables. §6.6 |
| HealthCheck | string | — | Command run periodically. Exit 0 is healthy. §5.6 |
| HealthCheckInterval | dword | 30 | Seconds between health checks. |
| HealthCheckTimeout | dword | 5 | Seconds before a health check is killed and counted failed. |
| HealthCheckRetries | dword | 3 | Consecutive failures before the service is unhealthy. |
| RestartPolicy | dword | 1 (OnFailure) | 0 Never, 1 OnFailure, 2 Always. §6.4 |
| RestartMaxRetries | dword | 5 | Consecutive restarts before Failed. §6.4 |
| RestartWindow | dword | 120 | Seconds of sustained health that reset the restart counter. |
| RestartDelay | dword | 1 | Seconds before a restart; doubles each consecutive failure, capped at 60. |
| Readiness | dword | 0 (Notify) | 0 Notify, 1 Alive. Ignored for Oneshot. |
| NotifyAccess | dword | 0 (Main) | Who may send notifications. Main is the only mode. §10.5 |
| FdStoreMax | dword | 0 | Maximum descriptors held for the service; 0 disables the store. §10.6 |
| TimerPersistent | dword | 1 | Catch up a missed timer run after a reboot. §9.3 |
| TimerJitter | dword | 0 | Maximum random delay added to each firing. §9.4 |
| Environment | multi_string | — | KEY=VALUE pairs added to the environment. §5.5 |
| WorkingDirectory | string | / | Working directory for the process. |
| TTYPath | string | — | Terminal to attach as the standard streams and controlling terminal. §5.4 |
| TTYPrecedence | dword | 0 | Who wins that terminal when several services want it at once. Higher wins. Refused without a TTYPath. §11.6 |
| RuntimeDirectories | multi_string | — | Private directories under /run, created before the main process. |
| LimitNOFILE | dword | — | RLIMIT_NOFILE. |
| LimitCORE | dword | — | RLIMIT_CORE, in bytes. |
| Conditions | multi_string | — | Start-time conditions; failure skips the service. §3.5 |
| Asserts | multi_string | — | Start-time assertions; failure fails the service. §3.5 |
| DisplayName | string | — | Human-readable name for status display. |
| Description | string | — | What the service does. |
| ServiceSecurity | binary | inherit | Descriptor controlling runtime operations on the service. §4.6 |
3.2.1 Registry types #
| Schema type | Registry type |
|---|---|
| string | REG_SZ, UTF-8 |
| multi_string | REG_MULTI_SZ, an ordered list |
| dword | REG_DWORD, 32-bit unsigned |
| binary | REG_BINARY |
A value whose registry type does not match the field's is a decode error. So is a dword carrying a value outside an enumerated field's range.
3.2.2 Schema version and forward compatibility #
Machine\System\Services\SchemaVersion is a dword, currently 1.
peinit creates it if it is
absent (§2.3).
Unknown values on a service key are ignored, which is what lets the schema grow additively: a definition written for a newer peinit still loads on an older one, minus the fields it does not understand. A newer schema version does not prevent boot.
Known fields are the opposite. A known field appearing more than once in
a collected definition is a decode error rather than a last-one-wins,
because a definition
that says two different things about the same field has no defensible
reading. Since names match case-insensitively, this catches ImagePath
and imagepath in the same key.
3.2.3 What a decode failure costs #
A definition that fails to decode fails that one service, and the answer differs by caller.
At boot the key is marked Failed with cause ValidationError and the
boot proceeds with every other definition. Anything that depended on the
failed service fails in turn through the ordinary dependency propagation
(§7.4), so the cost is bounded by what actually needed it.
On reload-config the same: that service is Failed with cause
ValidationError — or, if it is running, left running with its
definition marked removed — and every other definition loads (§10.4).
3.3 Field Formats
Peios / Peios technical reference / peinit / The Service Model
Rules that apply to how a field's value is written, as distinct from what it means.
3.3.1 Strings #
A string field that is present is non-empty, unless this section says otherwise for that field. Three fields treat the empty string as absence:
Identity— empty is the same as absent, and defaults toLocalService.HookIdentity— empty is the same as absent, and falls back to the service'sIdentity.DisplayNameandDescription— empty is the same as absent.
WorkingDirectory, when present, is a non-empty absolute path.
Whether it exists,
is a directory, and is reachable is checked when the service starts, not
when the definition is read.
TTYPath is checked for emptiness before absoluteness: an empty value
means no terminal, and a non-empty relative path is rejected.
3.3.2 Identity and HookIdentity #
Either a well-known principal name — SYSTEM, LocalService,
NetworkService — matched case-insensitively and canonicalised,
or a literal
SID string such as S-1-5-18. Anything else is passed to authd verbatim
to resolve (§4.3).
3.3.3 RequiredPrivileges #
Privilege names, matched case-sensitively against the published
privilege table. A name that does not match exactly fails token
materialisation and therefore the service start, so SeTCBPrivilege
does not start a service that SeTcbPrivilege would.
3.3.4 RuntimeDirectories #
Each entry names one directory directly under /run. Entries are
non-empty relative names; an entry equal to . or .. is rejected, as
is any entry containing /, \, a NUL, or a control character. A dot
inside a name is fine — app.sock.d is a valid entry.
For an entry foo, peinit creates /run/foo immediately before
launching the service's main process, with a descriptor granting full
access to SYSTEM, Administrators, and the service's own SID. Hook
processes inherit the service's environment and identity rules but do
not cause provisioning — the directories belong to the main start.
If creation or descriptor assignment fails, the start fails with
ParentSetupFailure.
peinit does not remove runtime directories when a service stops.
/run
is a boot-scoped tmpfs and the next boot clears it.
3.3.5 Environment #
Each entry is KEY=VALUE, with a non-empty key containing no NUL. An
entry that does not split into that shape is a decode error.
3.3.6 SuccessExitCodes #
Each entry is a decimal integer from 0 to 255 — a process exit code. Signal names and ranges are not accepted. Code 0 is always success and does not need listing. Duplicates are collapsed.
3.3.7 Dependency and handler names #
Entries in Requires, Wants, BindsTo and Conflicts, and the
value of OnFailure, are validated when the definition is read. A
dependency naming something outside [A-Za-z0-9._-] is a decode error
rather than an unresolved dependency discovered later
— the difference
being that a typo containing an illegal character is caught immediately,
while a typo that is still a legal name is caught at graph validation as
a missing target.
An entry in the four list fields may carry a readiness level, written
after a colon (netd:routed). The name and the level are validated
separately, so the colon is not one of the illegal characters there.
OnFailure takes a bare
service name and no level: it names a handler to run, not a state to
wait for.
3.3.8 Timeouts and intervals #
Every timeout and interval in the schema is in whole seconds unless the
field says otherwise.
The two sd_notify fields that carry durations,
WATCHDOG_USEC and EXTEND_TIMEOUT_USEC, are in microseconds because
that is what the protocol specifies.
3.3.9 Identifiers peinit generates #
Job identifiers, operation identifiers, and every other GUID peinit mints are UUIDv7. UUIDv7 is time-ordered, so identifiers sort by creation time — which is what keeps eventd's time-range and recency queries over jobs and operations cheap.
3.4 Triggers
Peios / Peios technical reference / peinit / The Service Model
A trigger says when a service starts by itself. Triggers are independent of service type: a Simple service can have a timer, and a Oneshot can start at boot.
Triggers is a multi_string, each entry either type or
type:argument.
| Trigger | Form | Meaning |
|---|---|---|
| Boot | boot | Start during the Phase 2 boot sequence. |
| Deferred boot | boot:settled | Start once the boot set has settled, or the deadline expires. §2.5 |
| Terminal handover | tty:released | Start when this service's TTYPath is let go of. §11.6 |
| Timer | timer:<schedule> | Start on a schedule. §9.1 |
A service with no triggers is demand-only: it starts only when something
asks for it, whether an administrator, a dependency, or an OnFailure
handler.
Multiple triggers of the same type are allowed.
A service with
["timer:*-*-* 02:00:00", "timer:*-*-* 14:00:00"] runs at 2am and 2pm,
and each trigger is independent — its own timerfd, its own next-firing
computation, its own history.
3.4.1 Arity is enforced #
boot takes no argument. boot:<something> accepts only the
sub-triggers in the table above, and any other value is malformed rather
than an unknown trigger type to be ignored.
tty carries the same
rule: the bare word names no event, and tty: accepts only released.
timer
with no schedule is malformed, as is timer: with an empty one.
That strictness is the point. Unknown values on a service key are
ignored for forward compatibility, and if unknown trigger types were
ignored too, boot:setled would be silently accepted and the service
would simply never start. Instead it fails to decode, loudly.
3.4.2 Triggers that need a field #
tty:released is about the terminal this service names, so a definition
carrying it without a TTYPath is refused rather than ignored
— it describes a
service that would silently never start, and starting is the entire
point of the trigger. TTYPrecedence is refused on the same grounds.
Emptying TTYPath
(§3.3) takes both down with it.
3.4.3 Disabled #
Disabled=1 suppresses automatic activation and nothing else. No
trigger fires: not boot, not boot:settled, not tty:released, not a
timer, and not any trigger type added later.
A disabled service's timers
are neither armed nor serviced.
The definition is still loaded into the in-memory model, and an explicit
start still works.
To stop a service being started at all, deny
SERVICE_START in its ServiceSecurity descriptor — that is an access
control question, not a trigger question, and answering it with the
Disabled flag would mean a flag that anyone who can write the key can
clear.
Enabling and disabling are registry writes performed by administrative tools, not peinit commands. peinit picks the change up through a registry change notification, or on the next reload-config.
3.4.4 Extensibility #
The type:argument shape is designed to grow. Path, device and event
triggers slot into the same array with no schema change,
because a trigger is a
string in a list rather than a field of its own.
3.5 Conditions and Asserts
Peios / Peios technical reference / peinit / The Service Model
Both are start-time checks in the same type:argument form, over the
same four check types. What differs is the
consequence of a failure.
| Check | Form | Passes when |
|---|---|---|
| path | path:<path> | The path exists, of any type. |
| file | file:<path> | A regular file exists there. |
| directory | directory:<path> | A directory exists there. |
| registry | registry:<key> | The registry key exists. |
Conditions describe when a service applies. A failed condition skips the service: it transitions to Skipped, which satisfies its dependents, because a service that does not apply has succeeded by not needing to run.
Asserts describe what a service needs. A failed assert fails the
service, with cause AssertionError
— the
service was expected to run and a precondition it depends on is missing.
All entries of a kind are AND'd. Conditions are evaluated first; asserts only if every condition passed. Both are evaluated before dependency resolution and before any pre-exec hook, and an entry with an empty argument or an unrecognised type is a decode error.
3.5.1 Evaluation without blocking #
peinit is single-threaded PID 1 and its event loop cannot block while a check runs. Two constraints follow, and they are why the check types behave differently from one another.
3.5.1.1 Registry checks are cache-only #
A registry: check is evaluated against the in-memory model, never by a
live registry read. Accepted paths are exactly Machine\System\Services,
Machine\System\Init, and Machine\System\Services\<name> with one
nonempty service-name component. Init descendants, deeper Services
paths, and trailing separators are decode errors, caught at load rather
than silently failing at start. The same rule applies to Conditions and
Asserts.
The two roots resolve as present. A
registry:Machine\System\Services\<name> check is true when that service
exists in the model; it does not read an arbitrary registry key. A
missing service is a valid check whose result is false.
3.5.1.2 Filesystem checks run in a helper #
stat() can block uninterruptibly on hung I/O — a dead NFS mount, a
failing disk controller — so peinit does not call it from the event
loop. It forks a short-lived helper into a dedicated checks/ cgroup
under the service's tree, using the same clone3 path as any other
child. The helper stats the paths and reports over a non-blocking pipe;
peinit waits on the pipe and the helper's pidfd through epoll, and never
blocks.
PreStartCheckTimeout, default 5 seconds, bounds the helper. A check
that does not report in time is treated as not satisfied
— the
fail-safe direction, so a condition skips the service and an assert
fails it. peinit then SIGKILLs the helper's cgroup and unregisters the
result descriptor, and the event loop is never held up by a hung check.
3.5.2 When results are computed #
Checks are evaluated once, before the service's dependencies start, and the result is cached for the rest of that activation. A service that waits a long time for a dependency starts on the answer that was true when the wait began, not on a fresh one.
3.6 Command Strings
Peios / Peios technical reference / peinit / The Service Model
Four fields hold executable commands: ExecStartPre, ExecStartPost,
the command form of ExecReload, and HealthCheck. All four are parsed
the same way.
3.6.1 Parsing #
The string is split on whitespace into an argv, with double quotes grouping. There is no shell — no expansion, no substitution, no globbing, and peinit never invokes one. A command that needs shell features is wrapped in a script.
Whitespace means exactly six characters: space, horizontal tab, line feed, carriage return, form feed and vertical tab. Every other Unicode whitespace code point is an ordinary argument character, which keeps the split independent of the Unicode version peinit was built against.
Double quotes group text into one argv element and are not retained in
the result. Grouping may happen inside an argument, so
--name="hello world" becomes the single entry --name=hello world.
An empty quoted string is
preserved as an empty argv entry.
Backslash has no escape semantics and is copied literally. A single quote is an ordinary character.
An empty or whitespace-only command is invalid, and so is an unclosed double quote.
3.6.2 The executable #
For all four fields, argv[0] is the executable path and begins with /.
Relative names, empty names and PATH-searched execution are decode
errors. Everything after
argv[0] is an opaque string and is not path validated.
3.6.3 ExecReload signals #
ExecReload may instead name a signal, as signal:<NAME>. The name is
an exact canonical Linux signal name. Numeric values, realtime signal
expressions, aliases, lowercase spellings and names with surrounding
whitespace are all invalid.
SIGKILL and SIGSTOP are invalid for reload,
because a service
cannot handle either as a request to re-read its configuration.
The accepted set is the standard non-realtime Linux signals other than those two:
SIGHUP, SIGINT, SIGQUIT, SIGILL, SIGTRAP, SIGABRT,
SIGBUS, SIGFPE, SIGUSR1, SIGSEGV, SIGUSR2, SIGPIPE,
SIGALRM, SIGTERM, SIGSTKFLT, SIGCHLD, SIGCONT, SIGTSTP,
SIGTTIN, SIGTTOU, SIGURG, SIGXCPU, SIGXFSZ, SIGVTALRM,
SIGPROF, SIGWINCH, SIGIO, SIGPWR, SIGSYS.
Names are validated when the definition is read and again before
delivery, and mapped to host signal numbers at that point. ExecReload
absent means SIGHUP.
3.7 Configuration Generations
Peios / Peios technical reference / peinit / The Service Model
peinit operates on snapshots, not on a live view of the registry. Two generation concepts govern when a registry change takes effect.
3.7.1 The in-memory model #
peinit maintains a full in-memory model of every service definition. The registry is read synchronously exactly twice: during Phase 2 boot, before meaningful supervision has started, and during a reload-config, which an administrator initiated and which is bounded. At all other times peinit works from the model.
Change notifications arrive as events on a pollable descriptor. peinit
subscribes to Machine\System\Services\ and Machine\System\Init\ at
boot, using the LCS watch
mechanism — a persistent subscription that
delivers change events on a key descriptor, with an OVERFLOW event when
the kernel-side queue is exceeded (Peios Kernel TRM §5).
Any drained watch event triggers a full reload of the configuration, not a targeted re-read of the changed key. That covers the OVERFLOW case by construction, and it is why an administrator writing one value causes every definition to be re-read.
3.7.2 The boot generation #
At the start of Phase 2, peinit reads all definitions, builds the dependency graph and validates it. The plan and the graph are fixed at that point, and the boot executes against them.
The watches are armed as the event loop starts, which is after the plan
is fixed but while boot-plan services are still starting. A registry
write during that window — from an install script, a post-hook, a
package transaction — triggers a reload like any other, and the reload
applies like any other, with one exception: a boot-plan service whose
launch has not yet been attempted keeps the definition the plan was
built from, and the new one is recorded as pending, exactly as for a
running service — the boot executes against its snapshot (§2.5). Such
services are listed under deferred in the reload's summary, said on
the console, and recorded as peinit.config.reload.deferred. Once every
planned launch has been attempted or decided against — each service
terminal, or in Backoff with its dependents held for it (§6.1) — peinit
runs one more reload that applies what was deferred, recorded as
peinit.config.reload.applied naming the deferred services in
graph.services; nothing is emitted when nothing was deferred.
A service already running keeps its pinned definition as always.
3.7.3 The activation generation #
When peinit starts a service, it snapshots that service's definition. The snapshot governs the whole start lifecycle: pre-exec hooks, the token request, the readiness timeout, the initial health checks. A field changed while the service is Starting does not take effect until the next start.
A service in Inactive or Failed has no activation snapshot, so starting it uses the current model. That gives the expected behaviour for the common edits:
- A new service entry is available once the change notification is processed.
- A timer change on an inactive service takes effect at the next trigger evaluation.
- A dependency change takes effect on the next start for an inactive service, and on the next restart for an active one.
3.7.4 Field mutability #
Which class a field falls into depends on when its value is consumed.
3.7.4.1 Pinned to the running definition #
A change takes effect only when the service is restarted:
ImagePath, Type, Identity, RequiredPrivileges, ErrorControl,
RemainAfterExit, Triggers, Disabled.
Triggers and Disabled are pinned only while the service is running.
On a service that is not running, both take effect as soon as the
notification is processed
— which is what arms a timer added to an inactive service.
3.7.4.2 Applied on the next start #
A change takes effect at the next start or explicit graph reload, not while services are running:
Requires, Wants, BindsTo, Conflicts, OnFailure, Conditions,
Asserts.
3.7.4.3 Reloaded at runtime #
A change takes effect at the next relevant event, with no restart:
Arguments, SuccessExitCodes, every timeout and retry value
(StartTimeout, StopTimeout, WatchdogTimeout, RestartDelay,
RestartMaxRetries, RestartWindow, PreStartCheckTimeout),
HealthCheck and its three parameters, RestartPolicy, Environment,
WorkingDirectory, ExecStartPre, ExecStartPost, ExecReload,
HookIdentity, Readiness, NotifyAccess, LimitNOFILE,
LimitCORE, FdStoreMax, TTYPath, TTYPrecedence,
RuntimeDirectories, Provides, TimerPersistent, TimerJitter,
SafeMode, DisplayName, Description, and ServiceSecurity.
ServiceSecurity is the one whose reload is immediately observable:
a change takes effect on the very next control request against that
service (§4.6).
3.8 Service Removal
Peios / Peios technical reference / peinit / The Service Model
When a definition disappears from Machine\System\Services\, peinit
learns of it through the ordinary change-notification path.
What happens next depends
on whether anything is running.
3.8.1 Not running #
An entry in Inactive, Failed, Completed, Skipped, Abandoned or Backoff is discarded immediately. There is no process to consider.
Backoff belongs in this list rather than the next one, which is not obvious: a service between restart attempts looks like it has something pending. It does not. It has no process, and once the definition is gone there is nothing left to restart it from, so there is neither an instance to supervise nor a restart to wait for.
3.8.2 Running #
An entry in Active, Starting, Reloading or Stopping is not killed. The running process is a job, and a job's lifecycle is independent of the definition that produced it — removing a definition stops future management, it does not terminate work in progress.
peinit marks the entry definition-removed and keeps the cached
definition, solely to go on supervising the instance it already has.
When that instance
exits it is not restarted — RestartPolicy is moot, because there is
nothing left to restart from — and peinit then discards the entry.
A definition-removed entry never has dependents. A read in which one
service Requires another the read does not define is refused whole,
with MissingHardDependency — so a removal that would leave a dangling
hard dependency does not land at all, and the administrator has to
remove the dependents first.
While an entry is definition-removed:
stopis accepted, so an administrator can drain it cleanly, using the cachedStopTimeout;start,restartandreloadare rejected withUNKNOWN_SERVICE— there is no definition to work from;statusreports the current runtime state withdefinition_removedset, so the draining instance is visible rather than silent.
definition-removed is a flag on the existing runtime state, not a
state of its own. The state machine (§6.1) and the command × state
matrix (§10.3) are unchanged by it.
A restart already in its stop phase is a case of its own. The stop
phase still drains the existing instance; the start phase does not
begin, because there is nothing to start from. When the instance exits,
the operation is aborted with the reason
definition_removed_during_restart_stop_leg and the entry takes the
ordinary removal discard.
This is the window a package upgrade opens —
remove the definition, then restart the service.
Once the instance exits or is stopped, the entry — including anything in its fd store (§10.6) — is discarded. A crash counts: the exit is routed to Failed rather than into a restart, because there is no policy left to apply. The fd store goes with it, which a crash would otherwise preserve for the restart that is not going to happen.
4.1 Token Materialisation
Peios / Peios technical reference / peinit / Service Identity
Every service process runs with a KACS token that determines its
identity and its access rights. peinit obtains or creates that token and
installs it on the child before exec. It never shares its own token —
even an Identity=SYSTEM service receives a separately materialised
token of its own.
Which route the token comes from depends on the Identity field:
peinit reads its own token with kacs_open_self_token requesting the
real token rather than any impersonation, and opens it query-only: it is
a template to copy from, never a thing to hand out.
4.1.1 Where a token is materialised #
Materialisation happens at the point of use, per launched process, not once per service. A service that runs a pre-exec hook, a main process, and then a health check materialises three tokens.
Health checks and reload commands deliberately do not honour
HookIdentity. A health check reports on the service's own health and
should see what the service sees; a reload command acts on the running
service. HookIdentity exists for setup work — creating directories in
privileged locations, running a migration — which is a different job
from either.
If materialisation fails at any point — authd unreachable, an identity
that cannot be resolved, a KACS error — no child exists yet, and the
start fails with ParentSetupFailure for the main process or
PreHookFailure for a hook.
4.2 The SYSTEM Path
Peios / Peios technical reference / peinit / Service Identity
For Identity=SYSTEM, peinit mints a token itself. This is what breaks
the bootstrap circle: registryd and authd need tokens, and authd — the
thing that mints tokens — is one of them. eventd and lpsd took this
path too until they moved to accounts of their own (Identity=Service);
they now take the authd path (§4.3) and start after authd.
4.2.1 Minting #
peinit reads its own token as a template and builds a new primary
token carrying the same identity: user SID S-1-5-18, the same group
list, the same privilege set, the same integrity level. The mint
requires SeCreateTokenPrivilege, which the boot SYSTEM token carries;
the kernel refuses the call with EPERM otherwise.
Two details of the copy matter.
The logon session comes from the token's statistics.
peinit takes
the auth_id from the source token's TokenStatistics — not from the
independent interactivity_scope field, and not from a hard-coded
well-known SYSTEM LUID. The minted token therefore stays associated
with the real SYSTEM logon session peinit was given at boot, while
carrying its own interactivity scope, which is zero for a platform
service. Substituting either of the other two values would associate
platform services with a session that does not exist.
The logon SID group is dropped from the copy. The kernel re-appends the session's logon SID when it creates the token, and rejects a create whose group list already contains it. So peinit filters that group out of the template before building.
Two groups are added that the template does not carry.
The
per-service SID (§4.4), which is what keeps platform daemons
distinguishable while they all run as SYSTEM; and the Service group
S-1-5-6, which every token authd mints for a service logon carries by
derivation from the logon type.
The second is load-bearing rather than cosmetic. S-1-5-6 is the
grantee on anything reachable by services as a class — peinit's own
notification socket first among them (§10.5) — so a SYSTEM service
minted here without it would be unable to report itself ready, while an
otherwise identical service on an authd-minted token could. A service is
no less a service for having been started before the authority was.
The minted token carries a default DACL the template cannot supply.
A token's
default DACL is the DACL an object gets when the token creates it with
no parent to inherit from — an abstract socket, a key under a container
written without inheritable ACEs. The kernel has no fallback of its
own: a token whose default DACL is empty leaves such an object with a
null DACL, which grants everything to everybody. The bootstrap SYSTEM
token carries D:(A;;GA;;;SY)(A;;GA;;;BA) (see
Bootstrap tokens),
but a token's default DACL is not a field peinit can read back from its
own handle, so peinit states the same value itself and sets it on every
SYSTEM token it mints. Non-SYSTEM services do not pass through here; their default DACL
is the authority's, from the DefaultDacl value of the principal's
policy record (§4.3).
peinit also asserts that its own token is a primary token and that its
user SID really is S-1-5-18 before minting, and fails the start with a
message naming what it found otherwise.
PID 1 minting from
something that is not the boot SYSTEM token is not a situation to
proceed from.
The minted token is fully independent. The privilege restriction that follows (§4.5) operates on it alone and cannot affect peinit's own.
4.3 The authd Path
Peios / Peios technical reference / peinit / Service Identity
For any identity other than SYSTEM, the token comes from authd.
peinit
does not resolve identities, does not know whether a principal is local
or from a domain, and does not want to: routing is authd's whole
purpose.
The division is not that peinit lacks the privilege to mint — it plainly
has it, and uses it for the SYSTEM path (§4.2). It is that the
privilege set and integrity level an identity carries are policy,
and policy is authd's. A second component deciding it in parallel is a
disagreement waiting to happen, and peinit has already shipped one: its
hand-written privilege bit table had four wrong entries, silently
stripping privileges a service had asked to keep.
4.3.1 The request #
peinit sends a ServiceAttest on /run/logon.sock, specified as PGSS
Logon §2.19. It carries two strings:
| Field | Value |
|---|---|
identity | The definition's Identity value, verbatim |
service | The name of the service being started |
authd answers with AccessGranted carrying the token as an SCM_RIGHTS
descriptor, or AccessDenied. There is no credential exchange, because
a service identity has no credential and must never acquire one — a
machine that could authenticate its own services unattended would have
to hold their secrets.
authd derives the per-service SID from service and adds it to the
token, so the SID that distinguishes one service from another comes from
the authority on this path rather than from peinit (§4.4).
Note that
authd cannot verify service: the process it names does not exist yet,
so there is no token to interrogate. peinit is trusted on it, which is
the reason for the restriction below.
4.3.2 This request must come from PID 1 #
authd authorises a ServiceAttest on two facts: that the peer's token
names SYSTEM, and that the peer is PID 1. The second is not
redundant. SYSTEM alone is every platform daemon on the machine, so
without it, compromising any one of them would yield a token for any
service identity authd would mint.
That makes it a standing constraint on this crate: the connection is opened by peinit's own process and must never be opened by a forked helper. Nothing in the code enforces it. If it is ever broken the symptom is every non-SYSTEM service failing to start at boot with an authorisation error, which reads like a fault in authd rather than like a refactor.
4.3.3 The token is weaker than a user's, deliberately #
An attested token is minted one rung lower on the impersonation ratchet
than a credentialled logon: Impersonation rather than Delegation.
The evidence behind it does not travel. A credential a principal source verified is something another machine could in principle have checked too; peinit's word is evidence on this machine only. A service token that could be forwarded off the box on the strength of it would let one system's PID 1 act as an arbitrary account on another.
Because the level is a ratchet, nothing derived from a service's token can exceed it either.
4.3.4 Failure is a failure #
Every non-SYSTEM service start depends on authd being reachable, and every one of them fails if it is not. Only services using the SYSTEM path are unaffected; platform services such as eventd and lpsd use the authority too.
peinit orders around that dependency rather than merely suffering it.
The authority declares Provides = ["authn"], and peinit derives a hard
dependency on whoever fills that role for every service whose Identity
— or whose HookIdentity, where it has hooks — is not SYSTEM
(§7.6).
Nothing is written on the consumer side: the edge comes from the same
field that creates the requirement, so a new service cannot forget it.
Until that existed the ordering was left to each definition, and all
three LocalService definitions in the tree omitted it — so a service
needing a token routinely reached this code before the socket was bound
and failed its launch with ENOENT, on most boots, recovering only
through its restart policy.
This is the intended behaviour rather than a limitation. The client that
preceded it returned a freshly minted SYSTEM token whenever it could not
do better, so a service declaring LocalService ran with peinit's entire
privilege set — and status output reported it as LocalService, which is
what kept the problem invisible from outside. A service that does not
start is visible; a service running as the wrong principal is not.
summarize_token now refuses to describe a token as an identity it does
not carry: where the declared identity predicts a user SID, a token
carrying a different one fails the launch rather than being reported
under the name that was asked for.
It
is only checked where the identity
predicts a SID — a principal name is authd's to resolve, and peinit has
nothing to compare it against.
4.3.5 What may be attested #
authd accepts the platform's own service identities — SYSTEM,
LocalService, NetworkService, and the per-service virtual account
selected by Service — which no principal source holds and
for which no credential could exist.
Anything else must be designated in the principal's own record as usable for a service logon, and until that designation exists authd refuses it. So a definition naming an ordinary principal does not start. That is the property the design exists to guarantee: without it, the service manager would be an oracle that mints a credential-free token for anybody on the machine.
4.3.6 The service’s own virtual account #
Identity=Service asks authd to use the per-service SID (§4.4) as the
user SID, as well as including it in the token’s groups. The spelling
Service is case-insensitive. Each service name therefore selects a
separate virtual account without creating an account in a principal source
or enabling a credentialled logon.
The derived groups include Everyone (S-1-1-0), Authenticated Users
(S-1-5-11), Local (S-1-2-0), and Service (S-1-5-6), together with the
kernel’s per-session logon SID. It does not acquire SYSTEM or LocalService
membership. Privileges, integrity, owner and default DACL still come from
authd’s policy evaluation; choosing a distinct user SID grants no extra
privileges by itself.
With Identity=LocalService, the user SID is the shared S-1-5-19 instead.
The service’s own SID remains a group, so ACLs can still distinguish two
services that share that account. Use Service when their user identities
should also be distinct.
Service takes the same derived hard dependency on the providers of the
authn role as other non-SYSTEM identities (§7.6). On the standard platform
that provider is authd; an explicit Requires=authd is unnecessary.
4.3.7 The interface is authd's #
peinit speaks the protocol through libauthd, which is its definition,
rather than encoding the message itself. A second implementation of a
wire format that mints identity is the kind of duplication that fails
silently and in the worst place.
4.4 Per-Service SIDs
Peios / Peios technical reference / peinit / Service Identity
Every service token carries a SID derived from the service's name, in its group list, alongside whatever the principal's own identity brings.
The derivation is the one KACS defines (Peios Kernel TRM §3.2). The
authority is S-1-5-80. The service name is uppercased and encoded as
UTF-16LE, SHA-1 is taken over those bytes, and the 20-byte digest is
split into five little-endian 32-bit sub-authorities:
S-1-5-80-<sub1>-<sub2>-<sub3>-<sub4>-<sub5>
peinit computes this itself, from the name alone, with no involvement from anything else. authd computes the same value independently when it mints a token, and the two implementations are pinned against each other by a shared test vector.
4.4.1 Why they exist #
Per-service SIDs are what make access control useful when services share
a principal. Every platform daemon runs as SYSTEM, and every service
with no Identity runs as LocalService; without something to tell
them apart, an ACL could grant a right to "LocalService" and thereby
grant it to a dozen unrelated services.
With a per-service SID, an ACE can name one specific service. It costs nothing — no account, no registry entry, no allocation — because it is a hash of a name that already has to be unique.
They are load-bearing in more than access checks. peinit uses the
service SID directly when it provisions a service's runtime directories,
stamping /run/<name> with a descriptor that grants full access to
SYSTEM, Administrators, and that service's SID and nothing else (§3.3).
4.4.2 The uppercasing rule #
The name is uppercased before encoding, using full Unicode case mapping
— the one that expands ß to SS and fi to FI rather than mapping
each code unit in place. peinit uppercases in the code-point domain,
before the UTF-16 encoding, so any expansion happens first.
For an ASCII service name, which is every real service, the choice is invisible. It becomes visible only for a name containing a character whose uppercase form is longer than itself.
4.5 Privilege Restriction
Peios / Peios technical reference / peinit / Service Identity
RequiredPrivileges is a list of the privileges a service needs.
Everything else is removed from its token before exec.
4.5.1 Subtractive only #
peinit removes privileges. It never adds one, on either path — there is
no code that constructs anything but a removal. A service cannot acquire
a privilege by naming it in RequiredPrivileges; if the token it was
given does not have it, listing it changes nothing.
The removal targets the token's present bitmask, through
KACS_IOC_ADJUST_PRIVS on the token descriptor — one kacs_priv_entry
per removed privilege, carrying the privilege-removed attribute. The
right required on the descriptor is KACS_TOKEN_ADJUST_PRIVS, which a
freshly minted token always has.
peinit does not use KACS_IOC_RESTRICT. That builds a restricted-SID
token — a different mechanism with different semantics — and does not
touch the privilege bitmask at all. It is available in the bindings and
would be a plausible-looking mistake.
Removing a privilege clears its present, enabled and enabled-by-default bits together, and irreversibly. The privileges that survive keep the enable state their source gave them: peinit does not enable, disable or re-order anything. Enable policy belongs to whoever minted the token.
peinit iterates all sixty-four privilege bits rather than only the ones it has names for, so a privilege this build does not know about is stripped along with the rest. The safe direction is to remove what was not asked for, including what cannot be named.
If RequiredPrivileges is absent, peinit does not query or adjust the
token at all, and the source's default privilege set stands unchanged.
4.5.2 Names #
Privilege names are matched case-sensitively against the published privilege table. A name that does not match exactly is not silently ignored: it fails token materialisation, and therefore fails the service start.
Two privileges KACS enforces are absent from the published table —
SeTakeOwnershipPrivilege and SeRelabelPrivilege — and so cannot be
named in RequiredPrivileges at all. A service that needs either
declares nothing and takes the source token's defaults, or fails to
start if it tries to name one.
4.6 Service Security Descriptors
Peios / Peios technical reference / peinit / Service Identity
A service carries two independent descriptors, and they answer different questions.
The registry key descriptor on Machine\System\Services\<name>
controls who may read and write the service's definition. It is
enforced by LCS at key-open time and is not peinit's concern — peinit
reads definitions as SYSTEM.
The ServiceSecurity descriptor controls who may perform runtime
operations on the service through the control interface. It is stored
as a binary ServiceSecurity value on the same registry key, but
enforced by peinit rather than by LCS.
The two are genuinely independent. An administrator might be able to query a service's status without being able to read its configuration, or the reverse. Runtime control and configuration access are separate concerns and there is no reason for one to imply the other.
4.6.1 Access rights #
Restart requires SERVICE_START and SERVICE_STOP together.
Reset
requires SERVICE_STOP, because clearing a Failed or Abandoned state is
the tail of stopping something rather than the head of starting it.
The generic mapping peinit passes to AccessCheck:
4.6.2 Inheritance and the default #
A service whose definition carries no ServiceSecurity value takes the
one on Machine\System\Services itself. The lookup is a single step to
that key, not a walk up the hierarchy, which is exact for the flat
layout definitions actually use. That includes the compiled-in registryd,
which takes it once Phase 2 has read the key and on every reload after;
a registry definition of registryd (§2.3) may give it a descriptor of its
own like any other service.
If that key has no ServiceSecurity either, peinit applies a built-in
default:
O:SY G:BA D:(A;;0x000F;;;SY)(A;;0x000F;;;BA)(A;;0x0001;;;AU) S:(AU;FA;0x000F;;;WD)
SYSTEM and Administrators both get SERVICE_ALL_ACCESS. The SACL audits
every refusal, for everyone, which is what records a denial (below). An
administrator who may stop any service and shut the machine down gains
nothing by being unable to start one, so the default does not attempt a
narrower grant; a service that wants one carries its own
ServiceSecurity value.
Every authenticated principal gets SERVICE_QUERY_STATUS: what a service
is doing is not, by default, a secret from the people using the machine,
and a service whose state is a secret carries a ServiceSecurity that says so.
4.6.3 The check #
When a control command arrives, peinit:
- Takes the caller's token, captured when the connection was accepted.
- Resolves the target service and its ServiceSecurity descriptor. A
command naming no definition and no addressable definition-removed
entry returns
UNKNOWN_SERVICE; peinit does not invent a descriptor to check against. - Calls AccessCheck with the caller's token, that descriptor, the
generic mapping above, the right the command needs, and an audit
context naming the service,
{kind: "service", service: {name}}. - On denial, returns
ACCESS_DENIED. The attempt is recorded by KACS, not by peinit: when the descriptor's SACL asks for it — and the built-in default's does — the check writes akacs.audit.access.checkedrecord withobject.kindservice,object.service.name, the caller's token, and the access requested and granted (§8.4). - On grant, proceeds.
4.6.4 The SACL is the audit policy, and who may drop it #
A service's SACL decides which of its decisions are recorded. A descriptor written to the registry is used as it is: one with no SACL records nothing, and one that narrows the default's audits less.
ServiceSecurity is a whole registry value. Whoever may write it — on
the service's key, or on Machine\System\Services, whose value every
service without its own takes — may replace the descriptor, SACL
included, without holding SeSecurityPrivilege, which is what changing
an object's SACL otherwise needs. So those keys must be writable only by
principals trusted to change audit policy. By default they are: both
inherit the Machine hive root, which grants write to SYSTEM and
Administrators alone, and Administrators hold SeSecurityPrivilege.
Delegate a service through its ServiceSecurity DACL, never by
loosening the key's own descriptor; and a tool that changes only the
DACL must write the SACL back unchanged. The same holds for the control
descriptor in Machine\System\Init (§10.2).
4.6.5 Hot reload #
ServiceSecurity changes take effect on the next control request, with
no restart. A registry change notification triggers a configuration
reload, and the reload re-reads every descriptor. There is no cached
decision to invalidate — the check runs against the current descriptor
every time.
4.6.6 Filtering, not denying #
list returns only the services the caller has SERVICE_QUERY_STATUS
on. Services the caller cannot query are omitted, not denied: a
caller with no query rights anywhere receives an empty list and a
successful response. The denials are not surfaced to the caller; each
service's SACL decides whether KACS records them.
What the filtering protects is a service's state, which is what
SERVICE_QUERY_STATUS grants. That a service exists is not protected:
a command naming one is answered UNKNOWN_SERVICE or ACCESS_DENIED
according to whether it exists, and the definitions under
Machine\System\Services are readable. A client that can read the
definitions may therefore show a service list left out as one whose
state the caller may not see.
4.7 The Control Descriptor
Peios / Peios technical reference / peinit / Service Identity
Three commands are not about any one service: shutting the system down,
re-reading the configuration, and asking how the machine booted. They
are checked against peinit's own descriptor, stored at
Machine\System\Init\ControlSecurity as a binary value.
4.7.1 Access rights #
The generic mapping:
| Generic right | Maps to |
|---|---|
GENERIC_READ | SYSTEM_QUERY_STATUS |
GENERIC_WRITE | SYSTEM_RELOAD_CONFIG |
GENERIC_EXECUTE | SYSTEM_SHUTDOWN |
GENERIC_ALL | all three |
GENERIC_READ maps to the one right that changes nothing, as it maps to
SERVICE_QUERY_STATUS on a service. Before boot existed the control
descriptor governed two actions and no queries, and GENERIC_READ on it
conveyed nothing; a descriptor written then that grants GENERIC_READ
now lets its holder ask how the machine booted, and nothing more.
4.7.2 The default #
Absent a value in the registry, peinit applies:
O:SY G:BA D:(A;;0x0007;;;SY)(A;;0x0007;;;BA)(A;;0x0004;;;AU)
SYSTEM and Administrators both get every right, as they both get every service right in the ServiceSecurity default (§4.6). An administrator who can stop services one at a time can already stop the system, so withholding shutdown would be theatre.
Authenticated Users get SYSTEM_QUERY_STATUS and nothing else.
How a machine booted is machine status, like what /proc shows, not a
secret; and it is what a person signed in to the machine needs to tell
that it came up in Safe mode, or is a failed boot or two from recovery.
The ServiceSecurity default lets the same group query every service for
the same reason.
The default applies only while ControlSecurity is absent or
malformed. A descriptor already written to the registry is used as it
stands: one written before SYSTEM_QUERY_STATUS existed grants it only
where it grants GENERIC_READ, GENERIC_ALL or the bit itself, and a
machine that wants every signed-in user to read its boot status adds an
ACE for them.
4.7.3 Loading #
peinit loads the descriptor during Phase 2 boot and hot-reloads it on registry change notification, on the same path as the service descriptors. Until it is loaded — during Phase 1 and the early part of Phase 2 — the built-in default applies, which matters because the control socket exists from Phase 1 infrastructure setup onwards.
5.1 The Cgroup Tree
Peios / Peios technical reference / peinit / Starting a Service
peinit uses cgroups v2 for exactly two things: knowing which processes belong to a service, and killing all of them at once. It does no resource accounting and sets no limits.
/sys/fs/cgroup/peinit/<cgroup-id>/ service root
/sys/fs/cgroup/peinit/<cgroup-id>/main/ the main process
/sys/fs/cgroup/peinit/<cgroup-id>/hooks/ hooks and reload commands
/sys/fs/cgroup/peinit/<cgroup-id>/health/ health check invocations
/sys/fs/cgroup/peinit/<cgroup-id>/checks/ pre-start filesystem check helpers
The sub-cgroups satisfy cgroups v2's "no internal processes" rule, which applies whenever controllers are enabled, and give hooks and probes containment of their own so that killing one does not touch the service.
They are not all created at once.
The root and hooks/ are
created when the first thing needs them — the first pre-exec hook, if
there is one — and main/ and health/ when the main process launches.
5.1.1 The cgroup id #
<cgroup-id> is the service name with every byte outside
[A-Za-z0-9._-] percent-encoded as % plus two uppercase hex digits.
Service names are already restricted to that set (§3.1), so in practice
the id equals the name.
The encoding is a defensive guarantee that distinct names always map to
distinct, cgroup-safe ids. % is not itself in the safe set, so it
escapes to %25 and the encoding is prefix-free. That is what makes it
injective, unlike a plain substitution of / for -, under which a/b
and a-b would collide.
The id is internal. The name a user sees is unchanged.
5.1.2 Generations #
A cgroup whose processes survived SIGKILL cannot be removed — rmdir on
it fails with EBUSY. peinit records the leak and moves the service to a
fresh tree:
/sys/fs/cgroup/peinit/<cgroup-id>%gen<N>/
Old leaked trees persist until the next reboot.
Two conditions record a leak, and both are needed because they answer
different questions.
cgroup.events reporting populated at the post-kill deadline says live
processes remain. rmdir returning EBUSY says the tree cannot be given
up, which covers that case and the ones where nothing is running but
the directory still will not go. Acting on only the first left the second
reusing a tree that already existed and was not empty.
The generation advances once per tree, not once per leak recorded
against it. A single
failed start records two cleanup deadlines — one for
hooks/, one for the service tree — and a tree cleanup can report
main/, hooks/, health/ and the root separately; all of those are
the same tree, so the first advances the generation and the rest find
themselves already behind it. Leaks are still deduplicated by path and
kind, so each is recorded once.
The separator is %gen, not .gen. . is in the safe set and service
names permit it, so a service literally named app.gen1 would have
shared a directory with generation 1 of a service named app. %
self-escapes and the encoder only ever emits it followed by two uppercase
hex digits, so %g cannot occur inside an encoded id — which makes the
whole path injective, not just the id.
5.2 Pre-Start Evaluation
Peios / Peios technical reference / peinit / Starting a Service
Before anything is forked, peinit evaluates the service's conditions and asserts (§3.5). The definition comes from the in-memory cache, so this never touches the registry.
- If the service names a
TTYPathand another service is holding that terminal, it goes to Skipped with causeTtyUnavailableand the start is abandoned (§11.6). Asked first because a terminal is a fact about the machine right now, and evaluating conditions ahead of it would fork a check helper for a start that was never going to happen. - Conditions are evaluated. Any failure transitions the service to Skipped and abandons the start. Skipped satisfies dependents.
- If every condition passed and the service has asserts, they are
evaluated. Any failure transitions the service to Failed with cause
AssertionErrorand abandons the start.
Only when all three pass does the pre-exec sequence continue.
5.2.1 Where the transition happens #
For a fresh start from Inactive, the evaluation gates the transition: the service becomes Starting only after the checks pass, so a skipped service goes straight Inactive → Skipped.
For an activation that is already Starting — the start leg of a restart, for instance — the transition has already happened, and the evaluation gates further progress instead. A restart whose conditions no longer hold therefore passes through Starting on its way to Skipped, where a fresh start would not.
An assert failing takes the same route to Failed on both paths, and on
both it propagates: dependents in the same graph context hear that the
dependency failed (§7.3). This matters more on the restart leg than
elsewhere, because an AssertionError is never restarted — where a
running dependent is deliberately unaffected by a dependency crashing
(§7.1), since a crash is transient and the restart policy handles it, a
failed assert means the dependency is not coming back without
intervention.
5.2.2 Two check kinds, two mechanisms #
registry: checks resolve against the in-memory model. Since a
non-cacheable key is rejected when the definition is read, this
evaluation never needs a live read.
Filesystem checks — path:, file:, directory: — run in a forked
helper. peinit clones it with CLONE_PIDFD | CLONE_INTO_CGROUP into the
service's checks/ sub-cgroup, exactly as it launches anything else.
The helper stats the paths and writes the results to a non-blocking
pipe; peinit watches the pipe and the helper's pidfd through epoll.
The terminal is re-checked when the helper's results come back, rather than the earlier answer being carried over : the helper ran in between, and a terminal can change hands while it did.
PreStartCheckTimeout bounds the helper, at 5 seconds by default. On
expiry every check still outstanding is marked not satisfied — the
fail-safe direction — and re-evaluated on that basis, so a condition
skips the service and an assert fails it. peinit then kills the helper's
cgroup and unregisters the result descriptor.
A helper that survives the kill, stuck in uninterruptible sleep, has its cgroup abandoned and recorded exactly as a leaked hook or health cgroup is (§5.7).
5.2.3 Caching #
The results are computed once, before the service's dependencies are started, and reused for the rest of that activation. A service that waits a long time on a dependency starts on the answer that was true when the wait began.
Filesystem checks are gathered in one helper run, from the conditions and the asserts together. There is only ever one run — the completion path evaluates both lists against the results it gets back, and there is no mechanism to ask for a second — so both lists' paths have to be stat'd before it. A path named in both is stat'd once.
A check with no result counts as not satisfied, which is the fail-safe direction for a timeout and is why the gather has to be complete: an omitted path is indistinguishable from a path that is not there.
5.3 The Pre-Exec Sequence
Peios / Peios technical reference / peinit / Starting a Service
Everything between "peinit decides to start service X" and "X's binary is running". The service is in Starting throughout.
5.3.1 Step 1: Arm the start timeout #
StartTimeout covers the entire remaining sequence — pre-hooks, fork,
exec, and the readiness wait. A single deadline, measured from the
operation's creation rather than from this point, bounds all of it.
Expiry aborts the start and kills the service's cgroup tree. The cause
recorded depends on where the deadline landed
: a timeout
during pre-hooks is PreHookFailure; one during the readiness wait or
the main process's execution is ReadinessTimeout.
5.3.2 Step 2: Provision runtime directories #
If the definition lists RuntimeDirectories, peinit creates each one
under /run and applies its descriptor (§3.3) before anything else in
the launch. Failure classifies as ParentSetupFailure.
5.3.3 Step 3: Run pre-exec hooks #
Each ExecStartPre command runs in sequence, forked into hooks/. A
token is materialised for each at the point of use, from HookIdentity
if set and Identity otherwise (§4.1); a failure there fails the hook
and the service with PreHookFailure.
Any hook exiting non-zero kills the entire service cgroup tree —
cleaning up whatever grandchildren the hook left — and fails the service
with PreHookFailure.
When every hook has succeeded, peinit kills the hooks/ sub-cgroup
before the main process starts, so a hook that forked something into the
background does not become part of the service.
5.3.4 Step 4: Materialise the service token #
The main process's token, per §4.1. A failure means no child exists and
the service fails with ParentSetupFailure.
If a later parent-side step fails before the fork, peinit closes the token descriptor exactly once. A failure to close is retained as cleanup evidence and does not change how the start is classified — the classification describes what went wrong with the start, and a failed close is a separate fact about the same failure.
5.3.5 Step 5: Create the cgroups and the error pipe #
The main/ and health/ sub-cgroups are created, and a
pipe2(O_CLOEXEC) error pipe. The parent keeps the read end,
non-blocking; the child will hold the write end.
The pipe is how the child reports a failure it hits after fork but
before exec. If exec succeeds, the write end closes automatically —
that is what O_CLOEXEC is for — and the parent reads EOF, which means
setup succeeded. If setup fails, the child writes a structured error
first.
The payload is exactly eight bytes, written with one write(2):
| Bytes | Content |
|---|---|
| 0–3 | little-endian u32, the child setup step identifier |
| 4–7 | little-endian i32, a positive Linux errno |
The child writes at most one payload — the first reportable failure —
and then exits. The parent treats a non-EOF payload that is not exactly
eight bytes, or carries an unknown step identifier, or an errno of zero
or less, as malformed evidence and fails closed with PreExecFailure.
| Id | Step |
|---|---|
| 1 | Close the read end of the error pipe |
| 2 | Set the standard streams |
| 3 | Reset the signal environment |
| 4 | Install the service token |
| 5 | Set the resource limits |
| 6 | Set oom_score_adj |
| 7 | Set the working directory |
| 8 | reserved |
| 9 | Confirm NOTIFY_SOCKET |
| 10 | Inject stored descriptors |
| 11 | Exec the binary |
| 12 | Create a session |
| 13 | Acquire the controlling terminal |
Identifier 8 is reserved and never emitted
: the environment is built in the
parent and applied by execve, so there is no step in the child that
could fail. Identifiers 12 and 13 are the two terminal steps, which
occur only for a service with a TTYPath.
If pipe2 fails, no child exists and the service fails with
ParentSetupFailure.
5.3.6 Step 6: Fork #
clone3(CLONE_PIDFD | CLONE_INTO_CGROUP), targeting main/. This does
two things atomically: it returns a pidfd for the child, and it places
the child directly into main/ at creation. There is no window in which
the child exists without a pidfd, and none in which it runs or execs in
peinit's own cgroup.
Placing the child at creation also avoids the alternative, which would
be having the child write its own PID into cgroup.procs — something
its post-installation token could not do.
A clone3 failure means no child exists: ParentSetupFailure, with
EMFILE/ENFILE, EAGAIN and ENOMEM the usual causes.
peinit does not take the atomicity on trust. If clone3 reports success
without setting the pidfd, or the pidfd cannot be made close-on-exec,
peinit writes to cgroup.kill through the main/ directory descriptor
before returning ParentSetupFailure — because by then a child does
exist, running in the service's cgroup with the service's token, and the
parent has no handle on it. Walking away would leave a process peinit
cannot supervise, cannot stop, and has no record of, which the
zero-delay cleanup that follows a ParentSetupFailure would then record
as a leaked tree rather than kill.
Immediately after clone3 returns, in the parent and on either outcome,
peinit closes the token descriptor and the main/ cgroup descriptor,
exactly once each. The child relies on close-on-exec and its own exit
instead. A failure to close is cleanup evidence and does not change how
the start is classified.
5.3.7 Step 7: The parent after the fork #
- Close the write end of the error pipe.
- Register the read end with the event loop. peinit does not block on it — signals, control traffic, timers and log pipes stay serviceable while the child's setup is pending.
- When the read end becomes readable or hangs up, read it
non-blocking:
- Would block: leave the source registered, the job stays in pending setup.
- EOF: exec succeeded. Record the pidfd on the job, emit
peinit.job.started, and only then apply readiness side effects such as Simple/Alive activation. - Data: setup failed. Parse the step and errno, log the specific
failure, fail the service with
PreExecFailure.
While setup is pending the job is not Running, and no dependent that waits on this service's readiness is released.
5.3.8 Steps 8 and 9 #
The child's own path is §5.4. What follows exec is the readiness wait and the post-readiness work:
- Simple with
Readiness=Notify: wait forREADY=1, then Active. - Simple with
Readiness=Alive: Active as soon as exec succeeds. - Oneshot: wait for the exit. Success is Completed; without
RemainAfterExitit then goes Inactive once dependents are released. A non-zero exit is Failed.
On readiness or successful exit, peinit runs ExecStartPost in sequence
into hooks/, releases the service's dependents, kills hooks/, and —
for a Simple service only — arms the watchdog if WatchdogTimeout is
non-zero and the health check timer if HealthCheck is set.
A post-hook that fails is logged and does not fail the service. The state transition to Active or Completed happens before the post-hooks run, so a service is already Active while its post-hooks are executing.
5.3.9 Failures before the fork #
Steps 2, 4, 5 and 6 can all fail with no child in existence. peinit
handles all four entirely in the parent: clean up whatever cgroups were
created, fail the service with ParentSetupFailure, and return the
error to the caller. These are system-level resource problems — file
descriptor limits, PID limits, memory, cgroup filesystem errors — rather
than anything about the service.
5.4 The Child Path
Peios / Peios technical reference / peinit / Starting a Service
Between clone3 returning in the child and execve, peinit runs a
straight line of setup. The path is kept minimal — no logging, no
complex library calls — because it runs after a fork, where very little
is safe to do.
Each step reports through the error pipe with the identifier from §5.3
if it fails, then exits: _exit(126) for a setup failure, _exit(127)
for a failed exec.
The steps, in the order the child runs them:
| # | Step | Id |
|---|---|---|
| 1 | Close the read end of the error pipe | 1 |
| 2 | setsid() — only with a TTYPath | 12 |
| 3 | Set the standard streams | 2 |
| 4 | ioctl(TIOCSCTTY) — only with a TTYPath | 13 |
| 5 | Reset the signal environment | 3 |
| 6 | Set RLIMIT_NOFILE and RLIMIT_CORE | 5 |
| 7 | Set oom_score_adj | 6 |
| 8 | Install the KACS token, then close its descriptor | 4 |
| 9 | Change the working directory | 7 |
| 10 | Confirm NOTIFY_SOCKET is present in the environment | 9 |
| 11 | Inject stored descriptors from fd 3 upward | 10 |
| 12 | execve | 11 |
5.4.1 The terminal steps #
setsid() comes first, and its position is load-bearing in two
directions. It has to precede the stream setup, because setsid() drops
any controlling terminal the child inherited — doing it afterwards would
throw away the terminal just attached. And it has to precede
TIOCSCTTY, which requires a session leader that does not already own a
controlling terminal.
TIOCSCTTY is passed a literal zero argument, which is what makes it
unable to steal a terminal already owned by another session.
Without a TTYPath neither step runs and the service stays in peinit's
session.
5.4.2 The standard streams #
Without a TTYPath: stdin from /dev/null, stdout and stderr onto the
write ends of the service's output pipes, and every inherited pipe end
that is no longer needed closed.
With a TTYPath: all three streams onto the opened terminal, and the
/dev/null descriptor and both pipe pairs closed.
A terminal-attached service's output is therefore not captured for
logging — it goes to the terminal, which is the point of asking for one.
5.4.3 The signal environment #
peinit blocks every signal for its signalfd (§12.3) and the child
inherits that mask across the fork. Step 5 empties the mask and resets
every resettable disposition to SIG_DFL. A service starting with
signals blocked, or with PID 1's handling in place, is one of the
classic ways for a daemon to behave inexplicably.
5.4.4 oom_score_adj #
-1000 — OOM-immune — for an ErrorControl=Critical service, and 0
for everything else. A Critical service is one whose loss reboots the
machine, so letting the OOM killer choose it would convert memory
pressure into a reboot.
5.4.5 Resources before the token #
The resource limits and oom_score_adj are set while the child still
holds peinit's credentials, and only then is the service token
installed. Both are peinit's policy rather than authority the service
holds: raising a hard limit or lowering oom_score_adj needs
CAP_SYS_RESOURCE, which KACS projects from SeIncreaseQuotaPrivilege.
Setting them after the install would make every Critical service,
and every service with a raised limit, depend on that privilege. A
RequiredPrivileges list that leaves it out would then stop the
service from starting at all.
The step identifiers are unchanged by this order. They name steps, not positions.
5.4.6 The environment #
There is no step that sets the environment, which is why identifier 8 is
reserved and never emitted. peinit builds the environment in the parent
(§5.5) and hands it to execve as envp, so it arrives with the exec
rather than being installed beforehand. Step 10 is a check rather than a
set: it confirms NOTIFY_SOCKET is present in the prebuilt environment
and fails with a synthetic EINVAL if it is not.
5.4.7 What the child does not inherit #
A service inherits only what peinit hands it: its standard streams and any descriptors injected from the fd store. Everything else peinit holds is created close-on-exec — the control socket and every accepted connection, the notification socket, the epoll instance, the signalfd, every timerfd, both pidfds, every pipe, and every stored descriptor until it is deliberately un-marked at injection.
The signal reset and the close-on-exec discipline together are what make a service start from a clean context rather than from PID 1's privileged one.
A descriptor opened through the Peios native file interface returns
without close-on-exec set, so it does not get the discipline for free.
peinit sets the flag by hand on each of those — the input devices it
opens for the power button, the main/ cgroup directory descriptor,
/dev/null, and the service and recovery consoles — immediately after
opening them. The child duplicates its intended streams onto descriptors
0, 1 and 2 without close-on-exec, then explicitly closes the original
descriptors. The streams survive exec; the original handles do not.
5.5 The Base Environment
Peios / Peios technical reference / peinit / Starting a Service
peinit constructs every service and hook process's environment from
scratch, in four layers, lowest precedence first. Nothing is inherited
: peinit's own startup
environment holds TERM and nothing else (§2.1), and none of it is
passed through.
5.5.1 Layer 1: the compiled-in base #
One variable:
| Variable | Value |
|---|---|
PATH | /sbin:/bin |
Executables are addressed through the root-level StrataFS runtime views.
Package storage paths under /usr are deliberately not on the default
search path.
5.5.2 Layer 2: global environment variables #
Each value under Machine\System\Init\EnvVars\ becomes a variable: the
value name is the variable name, the REG_SZ data is the value. An
EnvVars\PATH overrides the compiled-in PATH; every other name adds.
A malformed entry — an empty name, or a name containing = — fails the
whole layer, which at boot means recovery mode.
registryd does not receive this layer.
The exemption is a
trust rule, not an availability one. Write access to EnvVars\ is equivalent
to compromising every service peinit starts: LD_PRELOAD,
LD_LIBRARY_PATH and their relatives are not filtered, because the
key's Security Descriptor is meant to be the control boundary. A key
that could inject into the daemon that enforces who may write it would
make that boundary self-referential.
The exemption is narrow, and matches on two things at once: the job's
resolved identity is SYSTEM and its service name is registryd.
A
non-platform service that happens to be called registryd receives the
ordinary layering. It matches on the job, so a hook of registryd's
running under a non-SYSTEM HookIdentity would receive the layer; and
it matches the resolved identity string, so a definition naming
S-1-5-18 literally rather than SYSTEM would not be exempt.
registryd is launched in Phase 1, before EnvVars has been read at all,
so the exemption is only observable on a restart.
5.5.3 Layer 3: the service's own Environment #
The definition's Environment entries, overriding both layers below.
5.5.4 Layer 4: protocol variables #
NOTIFY_SOCKET, always. LISTEN_FDS, LISTEN_FDNAMES and
LISTEN_PID, only when descriptors are being injected — from the fd
store for a service, or from a submission's attachments for a submitted
job (§8.5).
These have the highest precedence. The first three are inserted after
both configurable layers, and all four names are dropped from those
layers before insertion, so a service cannot override NOTIFY_SOCKET
and break its own notification protocol, and an EnvVars\LISTEN_FDS=3
cannot point an fd-store-less service's sd_listen_fds at whatever
happens to sit at descriptor 3.
The name filter is what protects the LISTEN_* set when it is not
being set.
LISTEN_PID, which a conforming sd_listen_fds implementation checks
against its own PID before trusting LISTEN_FDS, is appended by the
child itself after the clone, because only then is the PID known; the
name filter keeps a configured value from shadowing it.
5.5.5 What peinit does not set #
Not HOME, USER, LOGNAME, SHELL or TERM. Peios identity is a
KACS token — a SID — rather than a passwd entry, so there is no
canonical home directory or login shell to populate. A service that
needs one supplies it through EnvVars\ or its own Environment; a
submitted job's submitter supplies it in the submission's
environment, which occupies layer 3 for a job.
5.5.6 Hooks and probes #
Hooks, health checks and reload commands are built through the same
path, so they receive the identical environment: the same layers, the
same NOTIFY_SOCKET, and the service's WorkingDirectory,
LimitNOFILE, LimitCORE and RequiredPrivileges. They never receive
LISTEN_FDS — stored descriptors go to the main process only.
5.5.7 When changes apply #
The global layer is a snapshot refreshed at boot and on reload-config,
and both it and the per-service Environment take effect at a service's
next start. Neither is applied to a running process.
5.6 Health Checks
Peios / Peios technical reference / peinit / Starting a Service
A watchdog tells peinit that a service is still ticking. A health check tells it that the service still works. They address different failures: a process can be alive and responsive to its own event loop while having lost its database connection, wedged in a bad state, or started returning errors to everyone.
5.6.1 Execution #
The health check command runs with the service's own token — never
HookIdentity — so it checks the service's health from the service's
own vantage point.
Each invocation runs in an ephemeral health/ sub-cgroup under the
service's tree, as a child of peinit rather than of the service. When
the check completes or times out, peinit kills the whole sub-cgroup,
which cleans up anything the check spawned.
5.6.2 Overlap #
If the previous check is still running when the next interval fires, the
new one is skipped and nothing is counted. A check exceeding
HealthCheckTimeout has its sub-cgroup killed and is counted as a
failure.
A launch failure — a token that could not be materialised, a fork that failed — is not counted. The invocation is recorded and the next interval schedules normally, and nothing about the service changes.
The distinction is between "the probe ran and said the service is
unhealthy" and "the probe could not be run". Only the first is evidence
about the service. Collapsing them meant a transient authd unavailability
could kill a service outright: with HealthCheckRetries=1, a reasonable
setting for a probe an operator trusts, one failed token materialisation
exhausted the budget and restarted it.
5.6.3 Failure #
HealthCheckRetries consecutive failures mark the service unhealthy.
An unhealthy service is restarted through the ordinary restart policy:
RestartPolicy, exponential backoff and throttling all apply, exactly
as for a crash. The failure count resets the moment a check succeeds.
Escalation kills the service's root cgroup rather than just main/,
so it takes hooks and probes with it.
5.6.4 The flap constraint #
Restart throttling is what stops a service flapping — failing checks,
restarting, passing initial checks, failing again. But it only works if
the failure cycle is shorter than RestartWindow, because otherwise the
service stays healthy long enough between failures to reset the restart
counter, RestartMaxRetries is never reached, and it restarts forever.
So this relationship has to hold:
HealthCheckRetries × HealthCheckInterval < RestartWindow
It is enforced as an error rather than a warning, in both places a
definition can arrive. At boot, a violating service is blocked with
cause ValidationError and never started.
On reload-config, it
is a validation finding, and a finding rejects the entire reload.
The constraint applies only where a check will actually run, which means Simple services. A Oneshot cannot flap through health checks because it never runs one.
A Oneshot that declares a HealthCheck is still rejected — but for
declaring one at all, not for its timing.
That is the
thing actually wrong with the definition; reporting it as timing
arithmetic let an operator adjust RestartWindow, see the definition
validate, and still have a HealthCheck that does nothing.
5.7 Leaked Sub-Cgroups
Peios / Peios technical reference / peinit / Starting a Service
A process in uninterruptible kernel sleep — D-state, typically a hung NFS mount or a failing disk controller — does not die when it is SIGKILLed. Its cgroup cannot be removed while it is there.
peinit detects this through cgroup.events: after sending the kill it
arms a post-kill deadline, 5 seconds by default, and checks whether
populated is still 1 when the deadline fires.
5.7.1 What happens depends on which cgroup it is #
For the main process, a survivor is fatal to supervision: the
service transitions to Abandoned with cause ProcessUnkillable and
peinit stops supervising it (§6.1).
For a health check or a hook, it is not. Those are diagnostic and setup processes; they hold no service resources — no ports, no file locks, no database connections — so a stuck one does not make the service unmanageable. peinit orphans the sub-cgroup instead:
- Marks it leaked, recording the path, the kind and the time.
- Increments the service's cgroup generation, so the next start builds a fresh tree (§5.1).
- Carries on supervising the service normally.
A leaked pre-start check helper is treated the same way: its
checks/ sub-cgroup is recorded and the generation bumped, which matters
because otherwise the next start would build into a tree that still
contains the unkillable process.
The leaked cgroup stays in the hierarchy until the next reboot.
A tree the service's next instance is already running in is not checked. A restart can relaunch before the post-kill deadline of the instance it replaced — the default restart delay is one second, the post-kill deadline five — and it relaunches into the same tree, since nothing has leaked yet to move the generation on. When the deadline fires, the tree is populated by the live instance, not by a survivor, so peinit drops the check rather than record a leak. A survivor of the killed instance that shares that tree goes unreported until the tree is next killed and checked.
5.7.2 Visibility #
Leaks are not silent. peinit both pushes one when it is detected and keeps it queryable afterwards.
The push happens once, on first detection — recording is idempotent, so a leak that is re-examined on a later cleanup pass is not announced again:
-
A
peinit.cgroup.leakedevent on the event stream, carrying the service, the sub-cgroup path (object.cgroup.path) and its kind (object.cgroup.type:service-tree,health,hooksorhelper). When it was found is the event's own time.
The pull side survives the moment of detection, for anyone who was not watching:
-
A status query includes a
warningsarray, one entry per leak, each an object with the sub-cgroup path, its kind, and the time of detection: -
A start command on a service with leaks returns a warning in its acknowledgement, saying the service has leaked sub-cgroups from a previous generation and that this indicates an I/O problem needing investigation.
The type is the same vocabulary everywhere
: health for a leaked
health/ sub-cgroup, hooks for a leaked hooks/ one, helper for a
pre-start check helper's checks/, and service_tree for a leaked
service root — the last being the most serious, since it means the whole
tree including main/ could not be reclaimed.
Whichever way it reaches you, what the leak means is the same: something underneath the service is not responding to the kernel, and no amount of restarting the service will fix it.
6.1 States
Peios / Peios technical reference / peinit / The State Machine
Every service is in exactly one state. There are ten.
| State | Process? | Satisfies dependents? | Meaning |
|---|---|---|---|
| Inactive | No | No | Not started, or stopped cleanly and not set to restart. |
| Starting | Maybe | No | Activation in progress: checks, hooks, fork, or the readiness wait. The process may not exist yet. |
| Active | Yes | Yes | Running and ready. |
| Reloading | Yes | Yes | Re-reading configuration. Still satisfies dependents. |
| Stopping | Briefly | No | SIGTERM sent, awaiting exit or SIGKILL escalation. |
| Completed | No | Yes | Oneshot only. Exited successfully. |
| Backoff | No | No | A restart is pending; waiting out the backoff delay. |
| Failed | No | No | Exited abnormally with the restart policy exhausted or absent. |
| Abandoned | Yes, unkillably | No | SIGKILL sent and processes survived. Supervision has stopped, the cgroup is leaked. |
| Skipped | No | Yes | Conditions were not met. The service does not apply. |
6.1.1 Dependent satisfaction #
Exactly four states satisfy dependents: Active, Reloading,
Completed and Skipped.
Nothing else does, and a dependent blocked on a Requires target in any
other state does not start.
Completed satisfies regardless of RemainAfterExit — a Oneshot without
it passes through Completed to release its dependents on the way to
Inactive, rather than skipping the state.
Skipped satisfies because a service whose conditions do not hold has succeeded by not needing to run. Treating it as a failure would make every conditional service a hazard to everything that depends on it.
Reloading satisfies because the process is still there and still
serving; a reload is a service telling itself to re-read a file, not an
outage. So a reload does not restart the RestartWindow clock (§6.4),
clear a readiness level the service has published (§7.5), or count as
its target going away to a BindsTo dependent (§7.1).
Backoff does not, and the distinction from Failed matters: a service in
Backoff is going to start again, and its dependents wait rather than
failing.
It is the state that makes a restart something other than a
transit through Failed. A dependent held this way looks exactly like a
start held on a readiness level — Inactive, its start operation Pending
(§8.2) — and is released when the service next reaches a
dependent-satisfying state. It fails with dependency_failure only when
the service gives up: its restart budget is exhausted, or it is stopped
or its definition withdrawn while in Backoff.
6.1.2 Invariants #
- A service is in exactly one state at any moment. There is one state field and exactly one place in the implementation that assigns to it.
- Only the transitions in §6.2 are performed. The assignment is gated by a central whitelist, so an unlisted transition is not merely avoided by convention — it cannot be written.
- Only peinit transitions a service. The control socket produces operations; nothing outside peinit writes state.
- A service object is securable independently of its process token. The ServiceSecurity descriptor governs who may manage the service; the process token governs what the service may reach. Neither implies anything about the other.
- Readiness is per start generation. The generation increments on every
transition into Starting,
and a
READY=1carrying a stale generation is rejected — so a notification from a previous incarnation can never be mistaken for this one's.
6.2 Transitions
Peios / Peios technical reference / peinit / The State Machine
Every transition peinit performs. Anything not listed here is not performed.
| From | To | Trigger |
|---|---|---|
| Inactive | Starting | Start command, dependency resolution, or a timer trigger. |
| Inactive | Skipped | A condition check failed. |
| Inactive | Failed | Validation, assertion, cycle or dependency failure — before any process existed. |
| Starting | Active | Readiness. Simple only: READY=1 received, or the process exists under Readiness=Alive. |
| Starting | Completed | Oneshot exited successfully. Without RemainAfterExit it passes through, releasing dependents, then goes Inactive. |
| Starting | Skipped | A condition failed after the activation had already entered Starting. |
| Starting | Failed | An assert failed after entering Starting; or a timeout, hook failure, setup failure, or an exit before readiness. |
| Starting | Backoff | A startup failure with the restart policy allowing a retry and budget remaining. |
| Starting | Stopping | An explicit stop cancelling an in-progress start. A restart on a Starting service is queued rather than cancelling. |
| Starting | Failed | The shutdown wave SIGKILLed a starting service. |
| Active | Reloading | ExecReload issued, or SIGHUP delivered. |
| Active | Stopping | Explicit stop, conflict eviction, a bound dependency stopping, or shutdown. |
| Active | Backoff | A crash, a watchdog timeout, or health check failure, with a restart allowed and budget remaining. |
| Active | Failed | The same three, with RestartPolicy=Never or the budget exhausted. |
| Active | Inactive | A Simple service exited successfully and RestartPolicy is not Always. Cause CleanExit. |
| Active | Backoff | A Simple service exited successfully and RestartPolicy=Always. Cause CleanExitRestart. |
| Reloading | Active | Reload resolved: READY=1, the detection window expiring, the extended wait expiring, or the reload command exiting — success or failure. |
| Reloading | Stopping | The same triggers as Active to Stopping. The reload is cancelled. |
| Reloading | Backoff | The main process exited during the reload, with a restart allowed. |
| Reloading | Failed | The same, with no restart available. |
| Stopping | Inactive | The process exited after an explicit stop or a shutdown. |
| Stopping | Failed | The process exited after a conflict eviction or a bound dependency stopping. |
| Stopping | Abandoned | SIGKILL sent and main/ still populated after the post-kill deadline. |
| Stopping | Starting | An explicit start queued against a service that is still stopping. Cause ExplicitStart. |
| Backoff | Starting | The backoff delay elapsed. Cause RestartPolicy. |
| Backoff | Inactive | An explicit stop cancelled the pending restart. |
| Backoff | Skipped | The backoff delay elapsed and the relaunch's pre-start checks skipped the service: its terminal is now held (§11.6), or a condition no longer holds. |
| Backoff | Failed | The backoff delay elapsed and peinit could not execute the relaunch. Cause InternalError. |
| Starting, Active, Reloading, Stopping | Failed | peinit could not carry out a step on the service and contained the error to it (§8.2). Cause InternalError. |
| Completed | Inactive | RemainAfterExit=0 and dependents released; or an explicit stop; or shutdown clearing it. |
| Completed | Starting | A start command or timer trigger re-running the Oneshot. |
| Failed | Starting | An explicit start, a bound dependency recovering, or a timer trigger. |
| Failed | Inactive | A reset command clearing the state. |
| Failed | Abandoned | A service SIGKILLed by the shutdown wave whose cgroup stayed populated past the post-kill deadline. |
| Abandoned | Inactive | A reset command. |
| Skipped | Inactive | A reset command, or an explicit start clearing Skipped before it re-evaluates the conditions. |
6.2.1 Things the table settles #
A restart never passes through Failed. A restart-eligible failure
goes to Backoff, waits, and goes to Starting.
Failed is reached only
when there will be no retry: RestartPolicy=Never, an invalid policy
for the cause, or an exhausted budget. This is why OnFailure (§6.3),
which fires on entry to Failed, does not fire on each retry — only when
the service finally fails out.
A clean exit is not a crash. A Simple service exiting zero goes to
Inactive under cause CleanExit, consulting neither the restart policy
nor the budget.
It goes to Backoff only under RestartPolicy=Always,
and then with the distinct cause CleanExitRestart, so status and
events say plainly that the process succeeded and was restarted by
policy rather than that anything went wrong.
A forced stop remembers why. Stopping to Failed carries the cause
from the transition that started the stop — ConflictEviction or
BindsToPropagation — rather than a generic failure.
A
service that lost a conflict and a service whose binding target went
away are distinguishable afterwards, which is what makes
bound-dependency recovery (§7.1) possible at all.
A restart detours through Inactive. The stop leg of an administrator's restart ends in Inactive, and the start leg begins from there. The two legs meet within one turn of the event loop, and peinit answers no request mid-turn, so no query ever sees the service Inactive in between.
A crash before readiness may be retried. A Simple process that exits
before signalling readiness is a ProcessCrash from Starting, and is
restart-eligible like any other, rather than a terminal startup failure.
A process can outlive its service's state. Most states are reached by the main process exiting, but not all of them: a watchdog timeout, a health check escalation and the post-kill give-up all move a service on while its process is still running. The exit that follows arrives for a service in Backoff, Abandoned or another state that no longer expects one.
peinit records that exit and performs no transition. The service keeps the state it had, any pending restart deadline stands, and the console reports it:
peinit: service <service> main process exited in state <state>; no action taken
This is deliberately not a fatal condition. A supervised service producing an unexpected sequence costs that service; it does not cost the machine. Treating it as a broken invariant of the runtime loop meant one crash-looping service dropped PID 1 into recovery mode and killed every session on the box.
6.2.2 Reset from Abandoned #
Resetting an Abandoned service re-checks its main/ sub-cgroup. If it
has finally emptied, peinit cleans up the whole service tree — main/,
hooks/, health/, then the root — and transitions to Inactive.
If it is still populated, peinit leaves the cgroup leaked, transitions to Inactive anyway, and returns this warning in the operation acknowledgement:
abandoned main cgroup for service <service> is still populated after
reset -- cgroup remains leaked; underlying D-state process requires
investigation
The warning is also written to the console, so a reset issued without reading the response still leaves a trace of the still-leaked cgroup.
The re-check targets main/. Note that the two paths into Abandoned
probe different cgroups: an explicit stop checks main/, while the
shutdown wave checks the service root.
6.3 Transition Causes
Peios / Peios technical reference / peinit / The State Machine
Every transition carries a cause recording why it happened. peinit
tracks both the current state and the cause of the most recent
transition, and the cause determines restart eligibility, OnFailure
behaviour, and what an administrator is told.
6.3.1 The taxonomy #
| Cause | Leads to | Meaning |
|---|---|---|
ExplicitStart | Starting | An administrator, an OnFailure handler, or a boot plan started it. |
ExplicitStart | Inactive | An explicit start on a Skipped service, clearing it so the conditions are re-evaluated. |
DependencyStart | Starting | Started to satisfy another service's dependency. |
RestartPolicy | Starting | An automatic restart after a backoff delay. |
BindsToRecovery | Starting | A bound dependency returned to Active. |
Timer | Starting | A timer trigger fired. |
ExplicitStop | Stopping | An administrator requested a stop. |
ExplicitReload | Reloading, Active | A reload was issued and resolved. |
ExplicitReset | Inactive | An administrator cleared Failed, Abandoned or Skipped. |
ConflictEviction | Stopping | A conflicting service started; this one lost. |
BindsToPropagation | Stopping | A bound dependency stopped. |
ShutdownWave | Stopping, Failed | The system is shutting down. Active and Reloading services go to Stopping; Starting services go straight to Failed. |
ProcessCrash | Failed, Backoff | The main process exited unexpectedly. |
CleanExit | Inactive | A Simple process exited successfully with a policy other than Always. |
CleanExitRestart | Backoff | A Simple process exited successfully under RestartPolicy=Always. |
ReadinessTimeout | Failed, Backoff | StartTimeout expired before readiness. |
WatchdogTimeout | Failed, Backoff | A keepalive did not arrive in time. |
HealthCheckFailure | Failed, Backoff | HealthCheckRetries consecutive failures. |
PreHookFailure | Failed, Backoff | An ExecStartPre hook exited non-zero, or its token failed, or the start timed out during hooks. |
ParentSetupFailure | Failed, Backoff | A parent-side failure before the fork. No child was created. |
PreExecFailure | Failed, Backoff | Post-fork setup failed before exec, reported through the error pipe. |
DependencyFailure | Failed | A Requires dependency entered Failed. |
RestartBudgetExhausted | Failed | RestartMaxRetries reached. |
CycleDetected | Failed | The service is part of a dependency cycle. |
ValidationError | Failed | The definition failed graph validation. |
AssertionError | Failed | A start-time assert failed. |
ConditionSkipped | Skipped | A start-time condition failed. |
TtyUnavailable | Skipped | Another service was holding the terminal this one names in TTYPath. §11.6 |
ProcessUnkillable | Abandoned | Processes survived SIGKILL. |
InternalError | Failed | peinit could not carry out a step on the service — a relaunch, a job terminal, a setup status, a health probe, a deadline action. The service did nothing wrong; its operation fails with the internal_error result (§8.2). |
6.3.2 Restart eligibility #
Causes fall into four classes, and the class decides whether the restart policy is consulted at all.
Restart-eligible. peinit consults RestartPolicy and the budget. If
a restart is allowed and the budget holds, the service goes to Backoff
and then to Starting; otherwise Failed.
ProcessCrash, WatchdogTimeout, HealthCheckFailure,
ReadinessTimeout, PreHookFailure, PreExecFailure,
ParentSetupFailure.
Always-only. CleanExitRestart applies when a Simple service exits
successfully and the policy is Always. It uses the same backoff and the
same budget as a failure,
which is what stops a daemon that exits
cleanly in a tight loop from bypassing throttling entirely. It is never
treated as a ProcessCrash, and both status and events make clear the
process succeeded and was restarted only because the policy says so.
CleanExit is its non-restarting counterpart: straight to Inactive,
consulting neither policy nor budget.
Budget-exempt. BindsToRecovery takes a service from Failed to
Starting when its binding target returns. It is not subject to the
policy or the budget,
because the service did not fail on its own — it
was stopped because its dependency went away.
Never-restart. peinit does not consult the policy at all:
ExplicitStop, ExplicitReset, ShutdownWave, ConflictEviction,
BindsToPropagation, ProcessUnkillable, RestartBudgetExhausted,
ValidationError, CycleDetected, DependencyFailure,
AssertionError, ConditionSkipped, TtyUnavailable, InternalError.
Retrying cannot help with any of them.
6.3.3 OnFailure #
When a service enters Failed and its definition names an OnFailure
service, peinit starts that service — with the exceptions below.
OnFailure fires on entry to Failed, and Failed to Failed is not a
transition, so it fires at most once per failure.
ShutdownWave— no new service starts during shutdown, so a fallback would be both impossible and pointless.ValidationError,CycleDetected,DependencyFailure,AssertionError— these are definition or graph breakage rather than runtime degradation. A broken definition cannot meaningfully trigger a fallback, and the fallback would probably sit in the same broken graph.
It fires for everything else, including ProcessCrash,
WatchdogTimeout, HealthCheckFailure, the startup failures, and
RestartBudgetExhausted on a non-Critical service.
For a Critical service exhausting its budget, the reboot takes
precedence and no fallback is started.
The suppression
asks whether a reboot is owed rather than inspecting ErrorControl
itself, which is the same question §6.4 puts.
OnFailure is for graceful degradation — the main web interface fails,
so start a minimal emergency endpoint. It is not for monitoring or
alerting, which is eventd's job.
6.3.3.1 A handler that is already running #
If the named handler is already Active — or Reloading, on its way back to Active — the failure starts nothing. A running service has no state to transition into on being started again, and it is already doing the job the failure would have asked of it, so the start resolves as satisfied: no operation is created, and no loop-guard chain entry is recorded for a cascade that did not happen.
This is the same answer the control interface gives an administrator who starts an already-running service, and it is deliberately the same test. A handler in any other state is started normally — so a Oneshot handler that ran for an earlier failure runs again for a new one, and a handler that has since failed is restarted.
6.3.3.2 The loop guard #
An OnFailure handler can fail and carry its own OnFailure, so a
misconfiguration where A's handler is B and B's is A could run forever.
peinit bounds the chain originating from one failure two ways: it tracks
the set of services already started as handlers for that failure and
will not start one already in the set,
and it will
not follow the chain past a fixed depth of 16.
When either
trips, peinit records a peinit.on-failure.suppressed event naming
which (outcome.reason cycle or max-depth), and stops.
A chain entry is retired two ways.
The handler stopped needing tracking. Completed, Inactive, Skipped or Abandoned — it ended, gave up, or became unkillable.
The handler arrived. It has been dependent-satisfying for its
RestartWindow — the same window that resets a restart budget (§6.4).
That is peinit's existing test for a service having genuinely taken over
rather than merely come up, and it is what ends the originating failure:
a later failure of that handler is a new one, starting a fresh chain.
The distinction matters in both directions. Without the second rule, a
handler that runs healthily for days holds a slot forever, so a
degradation path handing off through several healthy layers exhausts its
own depth budget by working. But retiring on Active alone would break
the guard entirely: Readiness=Alive reports Active the instant the
process spawns, so two services naming each other as handlers would hand
off forever, with no delay and no cost — an OnFailure start is not a
restart and spends no restart budget. Only a handler that held is
treated as having arrived.
6.3.4 The logging contract #
A transition into Failed, Skipped or Abandoned is written to the
console, naming the service and its cause: what failed, and why.
There
is no per-transition event — a transition is visible through the
peinit.job.* and peinit.operation.* events that carried it, not as a
record of its own.
Cryptic failure messages are a defect. A reboot loop caused by a configuration error with an opaque message is the worst outcome the system has, and the cause taxonomy exists so that the "why" is never a guess.
6.4 Restart
Peios / Peios technical reference / peinit / The State Machine
When a restart-eligible cause occurs, or a Simple clean exit produces
CleanExitRestart, peinit evaluates the restart policy. The outcome is
either Failed, or Backoff followed by Starting.
evaluate_restart(service, cause):
// 1. Policy.
if cause is never-restart:
return STAY_FAILED
if cause == CleanExitRestart and policy != Always:
return INVALID_CAUSE_FOR_POLICY
if policy == Never:
return STAY_FAILED
if policy == OnFailure:
// A termination counts as success only when the cause is a
// process exit whose code is in SuccessExitCodes. Only
// ProcessCrash and CleanExitRestart carry an exit code, and
// CleanExitRestart was handled above. Every other eligible
// cause has no exit code and is always a failure here.
if cause == ProcessCrash and exit_code in success_exit_codes:
return STAY_FAILED
// A clean terminal logout is not a failure.
if cause == CleanExitRestart and TTYPath is set:
consecutive_failures = 0
return RESTART_AFTER(min(restart_delay, 60))
// Other Always exits fall through.
// 2. Budget.
if consecutive_failures >= restart_max_retries:
cause = RestartBudgetExhausted
if error_control == Critical:
sync and reboot
return STAY_FAILED
// 3. Delay.
delay = min(restart_delay << consecutive_failures, 60)
// 4. Schedule.
return RESTART_AFTER(delay)
RESTART_AFTER puts the service in Backoff for the delay; when it
elapses the service transitions to Starting and the ordinary activation
sequence begins, with its own fresh StartTimeout.
The elapsed backoff timer is consumed when the relaunch is admitted. If a pre-start check needs an asynchronous filesystem helper, the service remains in Backoff until that check finishes, retaining its terminal reservation. The pending check drives progress; the expired timer cannot admit another restart while it waits.
The relaunch runs the pre-start checks like any other start, and the
machine can have changed while the service waited. If its terminal is
now held (§11.6) or a condition no longer holds, the service goes from
Backoff to Skipped, with the cause the check gives, and its pending
operation completes as a skip. A relaunch that peinit itself cannot
execute — its bookkeeping refuses the start — takes the service from
Backoff to Failed under InternalError (§6.3), fails the operation
waiting on the restart with the internal_error result, and reports
both on the console; it does not cost the machine.
The exit code is available only when peinit observed a process exit, and
it is carried into the evaluation wherever there was one — including the
pre-readiness exit, where a Simple service exits before signalling
READY=1. So SuccessExitCodes means the same thing on both sides of
readiness: a service that legitimately concludes "nothing to do" during
startup and exits with a listed code is not restarted.
Failures with no process exit behind them — a hook that never ran, a
readiness deadline, a dependency failure — carry no code, and the
SuccessExitCodes branch cannot apply to them.
That
is not a gap: a success code is a statement about how the service's own
process ended.
6.4.1 The policies #
| Policy | Value | Behaviour |
|---|---|---|
| Never | 0 | Never restart. The service stays Failed. |
| OnFailure | 1 | Restart on a non-zero exit or a runtime failure. An exit matching SuccessExitCodes is not restarted. |
| Always | 2 | Restart on any failure regardless of exit code, and for a Simple service also on a successful clean exit. |
For a Oneshot with RestartPolicy=Always, a successful exit is not
restart-eligible. RestartPolicy governs the response to failures; a
Oneshot that succeeds has done its job. It goes to Completed — and then
Inactive without RemainAfterExit — whatever the policy says, and only
a non-zero exit reaches the restart evaluation at all.
Timer triggers are the mechanism for re-running a Oneshot on a schedule.
6.4.2 Backoff #
The delay doubles on each consecutive failure, starting from
RestartDelay and capped at 60 seconds.
The arithmetic is
overflow-safe, so a large RestartDelay or a long failure run saturates
at the cap rather than wrapping.
While a service is in Backoff it is down and does not satisfy
dependents. An explicit start creates or merges into a deferred
start operation, which honours the remaining delay rather than
short-circuiting it;
a
stop cancels the pending restart and takes the service to Inactive.
For a terminal-attached service, a successful Always exit is a normal session
logout. It resets the failure counter and uses the base delay (capped at 60
seconds), without spending or exhausting the crash budget. Failed exits and
startup failures still use the ordinary budget and exponential backoff.
6.4.3 The budget, and when it resets #
consecutive_failures counts consecutive restart-eligible failures. It
resets to zero only after the service has stayed Active for
RestartWindow seconds.
It is not a
count of restarts within a trailing window, and the difference is what
the mechanism turns on.
peinit stamps the moment a service becomes dependent-satisfying, and
clears that stamp on any transition to a non-satisfying state — so a
crash restarts the clock.
The reset fires when the
stamp plus RestartWindow is reached with the service still Active.
A service that recovers and stays Active longer than RestartWindow
between crashes therefore never exhausts its budget: each crash starts
from a counter of zero.
Only failures recurring faster than the service can sustain a window of
health accumulate.
Two other events also zero the counter, both of which mean the service is no longer in a failure run: a clean exit to Inactive, and an administrative reset. An explicit stop while in Backoff does not — the accrued failures are preserved.
Because the reset requires the service to be Active, a service that happens to be Reloading when its window boundary passes misses that reset and gets it on returning to Active.
6.4.4 Exhaustion #
Once RestartMaxRetries restarts have happened without the service
sustaining a window of health, the next failure is not restarted: Failed
with cause RestartBudgetExhausted.
peinit then
applies ErrorControl:
- Normal — the service stays Failed.
- Critical — peinit syncs the filesystems and reboots immediately.
The reboot takes precedence over
OnFailure.
The reboot does not depend on how the budget was exhausted. The paths
that observe a terminal outcome for a running service — the main job
ending, a health check failing or timing out, the watchdog expiring —
raise it directly. Every other route reaches it through a check made
once per runtime turn, which asks a single question of the service
table: is a Critical service Failed with cause RestartBudgetExhausted?
That covers a budget exhausted purely by startup failures — repeated
ReadinessTimeout, PreHookFailure, ParentSetupFailure — where a
service that can never get as far as running would otherwise have
settled in Failed with neither escalation.
The OnFailure suppression asks the same question rather than
inspecting ErrorControl itself, so the handler is skipped exactly when
a reboot is owed and never on the strength of one that was not
scheduled.
A failure during a shutdown reboots nothing (§12.2 step 2), and the handler is not started either — the machine is already going down.
6.5 Reload
Peios / Peios technical reference / peinit / The State Machine
Reload tells a service to re-read its configuration without restarting. peinit issues the reload, moves the service to Reloading, and resolves it one of three ways.
A failed reload never takes a running service out of Active. And reload never gets stuck: every path has a timeout.
6.5.1 Choosing a path #
ExecReload absent means SIGHUP to the main process.
A signal:<NAME>
value means that signal instead.
Anything else is a command,
forked into the service's hooks/ sub-cgroup under the service's
own identity — never peinit's token, and HookIdentity does not
apply.
6.5.2 The signal path #
There is no command exit to observe, so completion is inferred from the main process's own notifications.
start the detection window (2 seconds)
on READY=1, at any time:
-> Active, "confirmed"
on RELOADING=1 within the window:
cancel the window
start the extended wait (StartTimeout)
on the window expiring with no RELOADING=1:
-> Active, "advisory"
on the extended wait expiring with no READY=1:
-> Active, "advisory"
The two-second window is a constant and is not configurable through the
registry, directly or otherwise.
It is not clamped
to the operation's own deadline: a service configured with a short
StartTimeout, which is a reasonable thing to do for something that
starts fast, still gets its full two seconds to answer a reload.
The
reload operation's own lifetime already bounds the whole thing.
The extended-wait expiry means the service announced a reload and never finished one. That is what the detection protocol exists to catch — the service has wedged mid-reload or lost its handler — so it is reported rather than only returned:
peinit: service <service> signalled RELOADING=1 but never completed reload
and audited as a peinit.service.reload.timed-out event. The outcome is
still carried in the operation's result for a wait=true caller,
but a
reload issued without waiting — the default — no longer resolves
silently.
A detection window expiring is a different thing and stays quiet. A service that does not implement the handshake lets it expire on every reload, and reporting that would drown the case above.
6.5.3 The command path #
The command's exit gates failure; the main process's READY=1 gates
confirmation.
on the command exiting non-zero:
-> Active, "failed"
on the command exceeding StartTimeout:
SIGKILL the hooks/ sub-cgroup, taking its descendants
-> Active, "failed"
on the command exiting zero:
-> Active, "confirmed" if the main process sent READY=1 during the
reload, otherwise "advisory"
6.5.4 Auto-detection #
The protocol needs no per-service configuration. A service that
implements the notification handshake — RELOADING=1 then READY=1 —
gets real lifecycle tracking. One that does not gets a brief Reloading
state that resolves itself when the detection window expires. Neither
has to declare which it is.
6.5.5 Interruptions #
The main process crashes while Reloading. That is a ProcessCrash,
and the restart policy is consulted: Reloading to Backoff if a restart
is allowed and the budget holds, Reloading to Failed otherwise.
Both reload timers are cancelled and any in-flight reload command is
killed.
This is a different event from an external reload command exiting non-zero, which is the "failed" outcome above and leaves the main process — and the service — running.
A stop arrives while Reloading. peinit cancels the reload immediately: it drops the reload deadlines, kills any reload command's cgroup, and sends SIGTERM in the same turn, without waiting out the window or the extended wait. The service goes to Stopping and the reload operation is aborted.
6.6 The Watchdog and Timeout Extension
Peios / Peios technical reference / peinit / The State Machine
Two notification fields let a running service adjust the deadlines it is held to. Both are authenticated exactly as any other notification (§10.5), and both carry microseconds.
6.6.1 The watchdog #
WatchdogTimeout sets the interval peinit expects WATCHDOG=1 pings
at. Zero, the default, disables it.
Missing a ping is a WatchdogTimeout cause and takes the ordinary
restart path.
A service may change the interval at runtime by sending
WATCHDOG_USEC=<value>:
- A value greater than zero updates the interval and re-arms immediately — the current timer is cancelled and a fresh one starts from the moment the message was received, rather than the new interval applying only from the next ping.
- A value of zero disables the watchdog entirely, equivalent to
WatchdogTimeout=0.
The runtime value does not persist. On a restart the interval reverts to
the definition's WatchdogTimeout converted to microseconds, and if
that is zero the watchdog starts disabled whatever the previous
incarnation had set.
WATCHDOG_USEC is honoured only while the service is Active. A service
that sends it while still Starting — before its own READY=1 — is
ignored and gets the definition's value.
6.6.2 Timeout extension #
A service may ask for more time during a start, stop or reload by
sending EXTEND_TIMEOUT_USEC=<value>.
peinit sets the current phase's deadline to expire that many microseconds from now. The extension replaces the deadline rather than adding to it — each message sets an absolute deadline computed from its own arrival — and may be sent repeatedly.
Because it replaces, a small value shortens the remaining time rather than being ignored, and a value of zero sets the deadline to now.
6.6.2.1 The caps #
The extended deadline cannot exceed four times the phase's base timeout:
| Phase | Base | Ceiling |
|---|---|---|
| Starting | StartTimeout | StartTimeout × 4 |
| Stopping | StopTimeout | StopTimeout × 4 |
| Reloading | StartTimeout | StartTimeout × 4 |
A value beyond the cap is clamped, not rejected — the message succeeds and the deadline becomes the maximum permitted. Because the cap is anchored to when the operation started rather than to the previous deadline, repeated messages cannot creep past it.
During shutdown an additional cap applies: the deadline cannot exceed
the time remaining in the global ShutdownTimeout, and where both caps
apply the stricter wins.
6.6.2.2 Where it does not apply #
A message arriving while the service is in a non-transitional state — Active, Completed, Failed — is ignored. There is no deadline to extend.
During shutdown, as at any other time, an extension moves only a deadline the service already has. A service still in a transition when the shutdown begins — starting, reloading, or stopping under an explicit stop — extends that transition's deadline, and one already stopping keeps its stop deadline, extension included, into its wave (§12.2). A service that is Active until its stop wave reaches it has no deadline to extend before then, so a request it sends earlier is ignored like any other in a non-transitional state. A service that expects its stop to take longer sends the request once its stop has begun.
7.1 Relationships
Peios / Peios technical reference / peinit / Dependencies
Four relationship types. Each says something different about how two services interact at start, at stop, and on failure.
7.1.1 Requires #
A hard dependency. If A Requires B:
- Start. B has to reach a dependent-satisfying state before A
starts.
If B is not running, peinit starts it with cause
DependencyStart. If B enters Failed, A goes to Failed with causeDependencyFailurewithout attempting to start. - Stop. Stopping B does not stop A. This is a start-ordering constraint, not a runtime coupling.
- Runtime failure. If B crashes while A is Active, A is unaffected and keeps running. B's own restart policy handles B.
- Missing target. A goes to Failed with
DependencyFailure, detected at graph validation.
7.1.2 Wants #
A soft dependency. If A Wants B, peinit starts B before A — with
cause DependencyStart — provided B exists and is not disabled.
If B
fails to start, or does not exist at all, A starts anyway.
There is no stop or failure effect in either direction.
The waiting rule is where the difference from Requires actually lives:
a dependent blocked on a Requires target waits for it to reach a
satisfying state, while a dependent blocked on a Wants target waits
only for it to reach any terminal state, satisfying or not.
That is
what makes Wants ordering rather than dependency — "start this first
if you can, but I will work without it".
7.1.3 BindsTo #
A runtime coupling. If A BindsTo B:
- Start. Identical to
Requires. - Stop. If B stops for any reason — explicit stop, conflict,
crash, shutdown — A stops too, transitioning to Stopping with cause
BindsToPropagation. - Recovery. When B returns to Active, peinit automatically restarts
anything sitting in Failed with cause
BindsToPropagationfrom B's stop. This is reactive rather than polled: peinit watches for the transition into Active from a non-satisfying state, and reacts to it. These restarts do not consume the restart budget — the dependent never failed on its own, it was stopped because its binding target went away.
BindsTo implies Requires. A definition may list both for clarity,
and if it does, the BindsTo semantics apply.
7.1.4 Conflicts #
Mutual exclusion. Starting A while B is Active creates a stop operation
for B — source ConflictResolution, cause ConflictEviction — and A
does not start until B has left every state in which it could still be
running.
Conflicts are symmetric. If A declares Conflicts = ["B"], starting
either one stops the other; B does not have to declare it reciprocally,
and peinit scans both a starting service's own conflicts and everything
that declares a conflict against it.
If both A and B are boot-triggered and they conflict, graph validation
detects an unresolvable conflict and fails both with ValidationError.
A missing conflict target is silently dropped — there is nothing to conflict with.
7.1.5 Ordering and self-reference #
Dependencies imply start ordering, and — except for BindsTo — imply
nothing about stopping. peinit starts dependencies before dependents;
during shutdown it reverses the same graph and stops dependents before
dependencies (§12.2), derived from the one graph rather than from a
separate stop-ordering configuration.
A service may not name itself in any dependency field. peinit rejects a
self-reference at graph validation as a CycleDetected failure, which
is what it is — a cycle of length one.
7.2 Graph Validation
Peios / Peios technical reference / peinit / Dependencies
peinit validates the dependency graph before executing it. Validation runs once per graph build — at boot, on an administrator-requested on-demand start's transitive closure, and on a reload-config — and is never incremental.
7.2.1 Cycles #
peinit topologically sorts the graph; if the sort fails, a cycle exists. Detection returns all cycles rather than stopping at the first: each detected cycle's members are removed and the search re-run until the graph is clean.
Every service in a cycle is failed with cause CycleDetected, and the
cycle path is logged so an administrator can see what to break.
If any service in a cycle is Critical, peinit downgrades to Safe mode (§2.6) rather than rebooting. The cycle is a configuration error, and rebooting would find it again.
7.2.2 Missing targets #
| Relationship | A missing target means |
|---|---|
Requires | The dependent is failed with DependencyFailure. |
BindsTo | The same — treated as a missing Requires. |
Wants | Silently dropped. |
Conflicts | Silently dropped. |
This holds in every mode. Safe mode drops a hard dependency on a service it excluded — that is its own rule (§2.6), and without it excluding a service would fail everything downstream and Safe mode could start almost nothing. Only that case drops. A target missing from the registry or disabled by an administrator is a configuration error rather than a Safe mode exclusion, and blocks the dependent in Safe mode exactly as in Full.
7.2.3 Validation errors #
A service that hits one is failed with cause ValidationError and never
started:
- The flap constraint,
HealthCheckRetries × HealthCheckInterval < RestartWindow(§5.6). - An invalid timer calendar expression (§9.1). This one is checked across every definition, not only those in the graph.
- Two boot-triggered services that conflict with each other. Both are failed. Safe mode applies if either is Critical.
7.2.4 Validation warnings #
Logged, and do not prevent boot:
- A service using
Readiness=Alivethat something else depends on hard.Alivereadiness means the process exists, which is no guarantee it is functional, so anything waiting on it is waiting on the wrong thing. - Services that need a role no service fills — in practice
authn, needed by every service whoseIdentityis notSYSTEM. Warned about rather than failed, because a missing hard dependency rejects the whole reload transaction, and that would prevent reloading the very definition that installs the missing provider. See §7.6.
7.2.5 Multiple findings #
A service can attract more than one finding in one pass — being both in
a cycle and missing a Requires target, say. The runtime state records
a single primary cause, chosen by precedence:
CycleDetectedValidationErrorDependencyFailure
The precedence affects only which cause is stored. Every other finding for the service is retained beside it, in discovery order, and the operation's failure message enumerates all of them:
CycleDetected: dependency cycle a -> b -> a (also: ValidationError:
health check interval exceeds the restart window) [2 findings]
The primary comes first and unqualified, so a service with one finding reads exactly as it always did. A higher-precedence finding arriving later demotes the previous primary rather than deleting it — breaking the cycle should not be what it takes to discover the second fault.
Every finding is also emitted as its own peinit.graph.validation.failed
KMES event, carrying graph.phase boot and the finding as
outcome.reason.
The console lines are for whoever is watching the boot; the events are
the account that survives it.
HardDependencyBlocked — blocked because a dependency is blocked —
gets its own reason, hard-dependency-blocked, rather than being reported as a missing
dependency, which would claim the target does not exist when it does.
It has no reload-path equivalent, because reload rejects wholesale
instead of propagating a block.
The reload path behaves differently, because its consequence is
different. Validation there accumulates every finding, encodes each as
its own event under graph.phase reload-config, and then rejects the
entire reload — the previous generation stays live and the findings
return to the caller (§10.4).
Boot marks individual services and continues; reload reports everything
and changes nothing.
Both use the same event type deliberately: one consumer filter catches
validation problems in either regime, and phase says which.
7.2.6 On-demand starts #
Before dispatching an administrator-requested on-demand start, peinit runs the shared definition validator over the
services the start plan would activate. An unschedulable health check,
an invalid health-check restart window or an invalid timer schedule
rejects the plan with INVALID_STATE; no service from that plan is
launched. This includes definitions reached through dependencies.
Definitions outside the plan, and dependencies whose running state already satisfies the request, do not make it fail. Missing or disabled hard dependencies retain the ordinary blocked-plan handling; dependency cycles retain the existing cycle refusal. Conflicting boot triggers are not themselves a reason to reject an explicit start.
This check also applies to an administrator restart when the service has
no running process and the request builds an on-demand start plan. A queued
administrator operation is checked when promoted into such a plan; rejection
fails that operation with an invalid_state reason and leaves other queued
operations eligible for promotion. It does not terminate the supervisor loop.
Internal starts from restart policy, timers, BindsTo recovery, OnFailure and TTY release retain their existing closure and cycle checks, without this additional definition-validation refusal, including when a queued internal operation is promoted. In particular, an invalid nonboot OnFailure target must not roll back the source service's transition to Failed. The stop/start leg of a restart of a running service and a deferred Backoff restart also retain their existing internal start behavior. Boot and reload-config validation remain as described above.
7.3 Graph Execution
Peios / Peios technical reference / peinit / Dependencies
Executing a graph means starting the services whose dependencies are all satisfied, and doing it again each time something becomes satisfying, until nothing is left.
execute_graph(graph, max_parallel):
ready = services with no unsatisfied dependencies
in_flight = 0
while ready is not empty or in_flight > 0:
while ready is not empty and in_flight < max_parallel:
begin_start(ready.dequeue())
in_flight += 1
match wait_for_event():
ServiceSatisfied(s):
in_flight -= 1
for each dependent of s:
if all its dependencies are satisfied:
ready.enqueue(dependent)
ServiceFailed(s):
in_flight -= 1
propagate_failure(s)
MaxParallelStarts bounds the concurrency, counted per context as the
members currently running.
The events are not only member completions. A dependency naming a
readiness level
is settled against the live service table rather than the graph's own
bookkeeping, and the arrival of a LEVEL= — or the publisher leaving a
dependent-satisfying state — re-runs the release for every context
holding an edge on that service.
7.3.1 Execution contexts #
A graph execution is a retained object, not a transient loop. peinit holds a context carrying its members, their dependencies and their status, and there are two kinds: one boot context built from the boot plan, and an on-demand context per explicit start, built from that service's validated transitive closure.
Both use the same scheduler, the same satisfaction rules, the same failure propagation and the same parallelism, but they are distinct runtime objects — which matters because they can overlap.
An operation is associated with every context that created or adopted it. One operation can belong to more than one active on-demand context: two administrators starting different services that share a dependency both end up merged into the same already-starting operation for it, and both contexts need to hear how it turns out.
So when a pre-start outcome is terminal for an operation, peinit dispatches the corresponding graph event once per associated context. An operation with no associated context completes or fails normally, and its waiters are notified, but no graph event is dispatched.
A member whose dependent resolves without ever needing it is pruned — a dormant sub-tree is cancelled rather than started, so an on-demand start that turns out not to need half its closure does not start that half.
A context is retired once every member has reached a terminal status — satisfied, failed or pruned — which is exactly when no further graph event can be dispatched through it. Its operation associations go with it, except where an operation is also associated with a context that is still live.
Retirement happens on the operation-maintenance sweep rather than at the moment the last member goes terminal. The callers of a terminal outcome walk the graph events it produced and release the contexts those events name, so dropping a context inline would pull it out from under the rest of the same turn's work.
Without this a context and its associations persisted for the lifetime of the process. The memory was the smaller half: dispatching a terminal outcome walks every context associated with the operation, so on a long-lived machine where services are started regularly the terminal path grew steadily slower, in PID 1's single thread.
7.3.2 Failure propagation #
When a service enters Failed during graph execution:
- Everything that
RequiresorBindsToit transitions to Failed with causeDependencyFailure. - Everything that
Wantsit is unaffected and starts normally. - Propagation is transitive: if A requires B and B requires C, and C fails, then B fails and then A fails.
7.3.3 On-demand start #
Starting a service explicitly resolves its dependencies first:
- Collect the transitive
RequiresandBindsToclosure. - Collect the transitive
Wantsclosure, best-effort. - Validate the sub-graph — cycles, missing targets.
- Resolve conflicts, stopping whatever has to stop.
- Start the sub-graph with the same parallel algorithm. Dependencies
started this way carry cause
DependencyStartand operation sourceDependencyPropagation.
A service already in a satisfying state — Active, Completed or Skipped — is not restarted. Its dependency is already met.
A disabled hard-dependency target blocks the dependent, on both
paths.
Disabled suppresses automatic activation and the service may
still be started explicitly (§3.2) — but nobody started this one
explicitly. Something that requires it did, and its administrator took it
out of service deliberately, so starting it to satisfy someone else's
dependency would defeat the flag by a route its description does not
consider.
Starting the disabled service itself still works. That is the escape hatch, and it is unchanged.
A disabled Wants target is skipped rather than blocking anything: a
soft dependency is advisory, so there is nothing to fail.
7.3.4 Shutdown ordering #
Shutdown reverses the graph: services with no dependents stop first, services that everything depends on stop last, derived by reverse topological sort from the same edges.
Only hard dependencies — Requires and BindsTo — contribute to the
ordering. A Wants dependent may therefore be stopped after its target.
The ordering is entirely emergent from what the definitions declare. There is no floor and no pinning, so where the TCB services end up in the wave order depends on their declared dependencies being right.
7.4 The Boot and On-Demand Paths
Peios / Peios technical reference / peinit / Dependencies
The two ways a graph gets built differ in more than scope, and the differences are worth having in one place.
| Boot | On-demand | |
|---|---|---|
| Members | Every boot-triggered root plus its closure | One requested service plus its closure |
| A missing hard-dependency target | Blocks the dependent | Blocks the dependent |
| A disabled hard-dependency target | Blocks the dependent | Blocks the dependent |
| A validation finding | The service is failed, others continue | The start fails |
| Multiple findings on one service | Only the highest-precedence one is reported | — |
| Conflicts | Two boot-triggered conflicting services fail both | The conflicting service is evicted |
| Context | One boot context | One context per explicit start |
| Failure of the whole build | Recovery mode | An error to the caller |
7.4.1 Where a definition error lands #
The three places a bad definition can be caught behave differently, and which one catches it depends on what kind of wrong it is:
- Decoding. A definition that does not parse — an invalid name, a
malformed trigger, an unclosed quote, a
registry:check naming an uncacheable key, a duplicate field — is caught when the registry is read. It fails that service withValidationErrorand the rest continue, at boot and on reload alike (§3.2). - Graph validation. A definition that parses but does not fit — a cycle, a missing target, a flap-constraint violation, an invalid calendar expression — is caught here, and fails that service at boot or the whole reload on a reload-config (§7.2).
- Start. A definition that parses and fits but whose preconditions do not hold — a failed assert, an unresolvable identity, a missing binary — is caught when the service actually starts, and fails that activation (§5.2, §5.3).
The dividing line between the first two is whether the problem is visible in one definition on its own. Decoding sees one key at a time; validation is the first place that can see two definitions together.
7.5 Readiness Levels
Peios / Peios technical reference / peinit / Dependencies
network-online.target on systemd is famously meaningless, because
"online" has no single definition. A service needing a default route and
a service needing only a loopback address want different things, and one
target cannot mean both.
peinit does not have an online target. A service says what it is waiting for, in the publisher's own vocabulary:
Requires = ["netd:routed"]
Wants = ["timed:synchronised"]
netd:routed waits for netd to be active and to have published the
level routed.
netd on its own is unchanged — the ordinary "that service must be
active" dependency.
7.5.1 The syntax #
<service>:<level>, split on the first colon.
It rides on the
existing Requires, Wants and BindsTo fields rather than a field of
its own, because a level dependency is a service dependency with a
stricter predicate: netd:routed subsumes netd. Each relationship
keeps exactly the semantics
it already has, applied to
the stricter test.
It cannot collide with a service name. A service name may contain only
letters, digits, ., _ and -, so a colon never appears in one, and
no existing definition can accidentally become a level dependency.
A trailing colon — "netd:" — reads as a plain dependency on netd.
It
is a typo, and treating it as a request for the empty level would produce
a condition nothing could ever satisfy.
7.5.2 Through a role #
The service side of the syntax may be a role instead of a name:
Requires = ["network:routed"]
network is filled by whichever service declares Provides = ["network"]
— netd, on a shipped image — and peinit rewrites the entry to
netd:routed before anything reads it, exactly as
§7.6
resolves a role. The level rides across unchanged.
Everything downstream
— validation, the graph, svctl status — sees a level dependency on a
service, and never learns a role was involved.
This is the form a definition should use for the network. The role is
"the executor of Peios Network Policy's interface layer", and the levels
are not the executor's own words but the specification's: PNP defines
readiness as link, addressed and routed, and whichever service
fills the role publishes those. A definition that says network:routed
survives netd being replaced; one that says netd:routed does not.
Several services may fill a role; the entry then resolves to one level
dependency per provider, and a Requires waits for all of them.
A
role no service fills is left as written and fails validation as the
missing hard dependency it is, naming the role the definition wrote.
7.5.3 How a level gets there #
The service publishes it on the notification channel it already has, as
LEVEL= beside READY=1 and STATUS=:
LEVEL=link
LEVEL=addressed
LEVEL=routed
All LEVEL lines in one datagram form one set: peinit atomically replaces
the sender's previous set with the distinct nonempty values, before
releasing any dependents, including when the same datagram carries
READY=1. A datagram with no LEVEL lines leaves the set unchanged.
LEVEL= on its own clears the set. Empty values mixed with nonempty
ones add no member; they do not clear other lines in the same datagram.
peinit records the set against the
sending service, and holds any dependent whose requested level is absent.
There is no subscription. peinit does not connect to netd's socket or timed's, does not need to find them before they exist, and carries no knowledge of their wire formats — which matters, because PID 1 is the one process on the machine that cannot afford to fail to start.
7.5.4 Levels match set membership #
Requires = ["netd:addressed"] is satisfied when the active publisher's
set contains addressed. Every member is an opaque, case-sensitive name;
commas do not split a name into several levels.
peinit has no ordering over another daemon's vocabulary. A publisher
that sends only LEVEL=routed does not satisfy an addressed dependent.
netd knows which conditions imply others, so when routed it publishes
link, addressed and routed together. A service should depend on the
condition it needs, and the publisher reports every condition that holds.
7.5.5 A level is never stale #
peinit clears a service's entire level set whenever that service leaves a state that satisfies dependents. A level is a claim about a process that is currently making it, and one that outlived its process would hold a dependent open on a promise nobody was keeping — the exact failure the mechanism exists to prevent.
Because peinit is the process that notices a service exiting, this needs no timeout and no liveness check.
7.5.6 How the gate holds #
A level is settled at dispatch, not at planning. The plan decides what to start; whether a level is met is a live question — it can arrive after the target's own start completed (netd is active long before DHCP finishes) and be retracted while a dependent is still waiting — so the execution graph re-checks it against the service table every time it considers releasing the dependent, and holds the start until the answer is yes.
That check runs even when there is nothing to start. An already-active
target is not part of the start plan, but the condition on it still is:
the graph keeps a level edge to a service it is not starting, which is
what makes Requires = ["netd:routed"] hold when netd is up and merely
not routed yet.
Two events re-open the question. A level-set update arriving from the
publisher releases every dependent whose requested level is now present
— this is the only thing that can open a Requires or BindsTo gate,
which take the exact level and nothing else. And the
publisher leaving a state that satisfies dependents
releases Wants waiters: a soft dependency waits only while someone is
running who could still publish the level, and proceeds once nobody is —
which keeps Wants failure-tolerant, the property that defines it.
A held start does not time out, and its StartTimeout runs from the
moment the level arrives, not from when the start was asked for.
Requires = ["netd:no-such-level"] holds the dependent indefinitely:
the service stays inactive and the start operation stays pending,
visible in svctl status as an operation that has not completed.
That is the declared semantics,
not a hang — the condition was never met, so the start never happened.
7.5.7 What happens on a drop #
Nothing, in this version. If a level falls away after a dependent has already started, the dependent keeps running and is not told.
That is a deliberate limit, not an oversight. The notification channel is one-way by specification (PSPU §4.16), so peinit has no way to tell a running service anything. A service that needs to react to a level changing while it runs should subscribe to the publisher directly — which is what resolvd does with netd today, and what any consumer that cares that much will want anyway, since it will need the detail rather than just the fact.
peinit's contribution is the start gate: the moment when the service does not exist yet and so cannot subscribe to anything at all.
7.5.8 Published levels #
| Publisher | Levels |
|---|---|
netd, as the role network | link → addressed → routed, defined by Peios Network Policy |
| timed | unsynchronised, settling, synchronised, spike |
Any service may publish a level; nothing about the mechanism is specific to these two. The vocabulary is the publisher's own — or, where the publisher fills a role a specification defines, the specification's — and needs no registration.
7.6 Derived Dependencies
Peios / Peios technical reference / peinit / Dependencies
Most dependencies are declared. One is not.
A service whose Identity is anything but SYSTEM cannot start until the
authority is listening, because peinit has to ask it for a token before
there is a process at all
(§4.3).
That is a hard dependency in every sense
Requires
means one.
It used to be nobody's. peinit hardcodes the socket the authority listens
on, so the requirement was real on every boot, but the ordering existed
only if a definition happened to declare it — and all three
LocalService definitions in the tree did not. Three of three is the
number that makes it a defect in the mechanism rather than in three
definitions: the requirement is created by Identity, so Identity is
what should produce the edge.
7.6.1 Provides #
A service names the roles it fills:
Provides = ["authn"]
A role is a virtual name — a capability, not a service. Virtual and
real names share one namespace, exactly as they do for packages (PSPU
§5.4): a dependency on authn is satisfied by a service literally called
authn, or by any service providing it.
The indirection is the point. peinit knows it needs an authority and knows which socket to speak to, but the authority's name is registry data and none of peinit's business — an image may ship a different one. So the derived edge names the role, and peinit resolves it to whoever declares it.
Provides entries use the service-name grammar and never carry a
readiness level. netd:routed names a condition on a service, and a
role is not a service, so there would be nothing for the level to
qualify; a level on a Provides entry is rejected rather than ignored.
The other side of a dependency may name a role, with or without a
level. Requires = ["network:routed"] is rewritten to one entry per
service providing network, the level carried across, before the
definition set becomes a service table — the same pass, at boot and on
every reload, that derives the edges below.
Requires, Wants and
BindsTo are rewritten;
a real service
name is never mistaken for a role,
a
provider never gains a dependency on its own role, and a role no
service fills is left as written for validation to report. See
§7.5 for
why the network is declared this way.
7.6.2 What peinit derives #
Before a definition set becomes a service table — at boot and on every
reload, through the same code, because an edge that held until the first
reload would be worse than no edge — peinit adds a Requires on each
provider of authn to every service that needs the authority to launch.
A service needs it when its Identity is not SYSTEM, or when it has
hooks and its HookIdentity is not SYSTEM.
The
second half matters: a
SYSTEM service whose hooks run as LocalService materialises a token
for them, and a definition that needs the authority for half its launch
needs the ordering for all of it.
Three rules keep the derivation from doing damage:
- A provider never gains a dependency on its own role. An authority declaring a non-SYSTEM identity would otherwise require itself, and a service that requires itself never starts.
- An edge already declared is not added again, whether it was
declared as
RequiresorBindsTo, and including one carrying a level.Requires = ["authd:ready"]already orders against the authority, and more strictly; adding a plain edge beside it would be a second edge to the same service saying less.BindsTorequires its target as well as binding to it, so the same holds for it. - Nothing is derived when no service fills the role. See below.
Because the result is an ordinary Requires, it needs no new kind of
edge and behaves exactly like a declared one everywhere else — in
graph validation,
in graph execution,
at shutdown, and in svctl status, where it appears as the dependency it
is rather than as a hidden ordering.
Several services MAY fill one role; each becomes its own Requires, so
the dependent is ordered after all of them.
For
authn this cannot
arise — PGSS Logon §2.1 requires at most one authority on a running
system — and where a future role allows several, ordering after all of
them is the conservative reading and needs nothing new. peinit
deliberately does not offer "wait for any one of them": that would change
the release rule from every edge satisfied to every group has one
satisfied, and would contradict Requires's failure rule, under which a
target entering Failed fails the dependent.
7.6.3 When nothing fills the role #
An image whose services need a token and which ships no authority is broken. A reload of it produces a validation warning naming every service that cannot start, and succeeds anyway.
It deliberately does not invent an edge to a name no service answers to. That would be a missing hard dependency, and a missing hard dependency rejects an entire reload transaction — so an operator could not reload the definition that installs the authority. The image would be unrecoverable by the one mechanism meant to recover it.
The individual launches still fail, loudly, where the fault actually is: at the service that asked for a token and could not have one.
7.6.4 A role is not an online target #
§7.5 opens
by rejecting network-online.target, and the objection there does not
reach this mechanism. It is worth being explicit about why, because the
two look alike from a distance.
The complaint about network-online.target is that "online" has no
single definition: a service needing a default route and a service
needing a loopback address want different things, and one name cannot
mean both. The abstraction is over a condition, and the condition is
ambiguous.
authn abstracts over no condition at all. It means "whoever answers
ServiceAttest on the logon socket" — a protocol role, specified by PGSS
Logon §2.19, which §2.1 says MUST have at most one occupant. There is
exactly one thing it can mean, and what a dependent gets is the same
thing it would get by naming the service directly.
network:routed passes the same test from the other direction. The role
network is a position too — the executor of Peios Network Policy's
interface layer — and routed is not peinit's word for "online" but the
specification's word for one exact condition, a default route. The
ambiguity that sinks network-online.target is absent because the
condition is named, not the target.
The rule the two share: peinit does not invent vocabulary for conditions inside another service. A readiness level is the publisher's own word for its own state, or its specification's; a role is a name for a position in a protocol peinit already speaks. Neither is a target.
8.1 Jobs
Peios / Peios technical reference / peinit / Jobs and Operations
A job is one supervised process execution. Every fork peinit performs is a job: a service's main binary, a pre-exec hook, a post-exec hook, a reload command, a health check invocation, a submitted job (§8.5).
Jobs are the observable unit of what actually ran. Services are definitions carrying identity, policy and configuration; jobs are instances. A restart creates a new job.
8.1.1 Lifecycle #
Created --> Running --> Completed
| |
| +------> Failed
| |
| +------> Abandoned
+------------------> Failed
| State | Meaning |
|---|---|
| Created | The job object exists but exec has not succeeded. The process may not have been forked, or it may be in pending post-fork setup with the error pipe unresolved. |
| Running | Exec succeeded and the process is alive. |
| Completed | The process exited successfully — code 0, or one in SuccessExitCodes. |
| Failed | The process failed, or peinit classified the job failed before the fork because parent setup failed. |
| Abandoned | The process survived SIGKILL. |
Job states are simpler than service states because they describe a process rather than a policy. A service has Starting, Reloading and Backoff because those are decisions; a job is running, or it finished, or it is stuck.
8.1.2 Fields #
Job {
id: GUID // UUIDv7
service: string? // null for a submitted job
job_type: enum // ServiceMain, PreExecHook, PostExecHook,
// ReloadHook, HealthCheck, Submitted
state: enum
pid: u32?
pidfd: fd?
resolved_identity: string // the resolved service, hook or submitter identity
token_summary: object // the resulting SID, groups, privileges
image_path: string
arguments: string[]
created_at_ns: u64
started_at_ns: u64?
ended_at_ns: u64?
exit_code: i32?
exit_signal: i32?
failure_cause: string?
cgroup_id: string
cgroup_generation: u32
activation_generation: u32
operation_id: GUID? // null for a submitted job
hook_index: u32?
}
The rules that govern when the nullable fields are populated are what make a job record trustworthy:
idis assigned before the fork, so a job that never forks still has an identity.pidandpidfdland on the record only once exec success is confirmed by EOF on the error pipe. Until then they are held in pending setup state and the job is Created.- An exit observed during that window is held, not applied.
peinit
resolves a reaped child to a job by PID, and inside this window no job
carries the PID yet — a short-lived process can be gone before the
error pipe is read. The exit is kept against its PID and replayed once
the job has been started and can receive it. Discarding it would be
unrecoverable: the job would be started against a PID that is already
dead and reaped, and no second
SIGCHLDis coming, so it would stay Running for ever. A reaped PID that matches no job and no pending setup is genuinely not peinit's — PID 1 reaps orphans it never started — and is ignored. - A setup failure takes the job straight from Created to Failed.
ended_at_nsrecords the classification time,failure_causerecords what went wrong, andpid,pidfd,started_at_ns,exit_codeandexit_signalall stay null. There was no process to have a PID or an exit status. exit_codeandexit_signalare populated only when peinit observed an exit — never both, since a process either exits or is killed.- For an Abandoned job,
ended_at_nsrecords when peinit stopped supervising, and the exit fields stay null. Nothing exited.
resolved_identity is the identity string — SYSTEM,
LocalService, a SID — that was resolved for the execution. For a
submitted job it is the job identity's user SID, since no name was ever
involved (§8.5).
token_summary is what the resulting token actually contains. They are
separate because they can differ, and the identity field exposed in
status views and job events is the former.
8.1.3 Retention #
peinit tracks active jobs in memory. When a job reaches a terminal state it emits a structured event carrying the full record and then drops the job. There is no job history in peinit, and no structure that could hold one.
eventd is the historian. It consumes those events from the KMES kernel ring buffer, and a query for a service's past jobs is a query to eventd.
A submitted job is the one partial exception: its record is dropped like any other, but the jobs system keeps its own entry for a further 60 seconds so that a submitter polling for the outcome can still read it (§8.5). That entry is a retained answer, not a history.
8.1.4 The launch queues are hints #
A job awaiting launch is queued by id, in one of six queues — service main, pre-start hook, post-start hook, control, health check and submitted. The queues hold ids; the store holds the records, and the store is the only truth.
They can therefore disagree. Any path that finishes a job in Created
drops the record, and a path that does not also clear the queue leaves an
id behind pointing at nothing. Requiring every such path to remember was
a poor trade: there are around a dozen of them, nothing enforced it, and
the cost of one omission was that peinit's runtime loop ended — which
ends PID 1.
So a drain that meets an id with no record discards it and moves on to the next, rather than treating it as an error. A live entry behind a stale one still launches, so one bad id cannot stall a queue. This is why the ordering rule is worth stating plainly: keeping the queue in step is a tidiness obligation, not a correctness one.
Discarded entries are counted and reported on the work pump's turn, so a bookkeeping fault shows up as a number rather than disappearing. A count that is not zero means some path finished a job without clearing its queue entry, and that path is a defect even though it is no longer fatal.
8.1.5 Ownership #
| Concern | Owner |
|---|---|
Restart policy, dependencies, health check schedule, ErrorControl | Service |
| Current state — Active, Failed, … | Service |
| PID, pidfd, exit code, exit signal | Job |
| Execution timestamps | Job |
| Identity and token | Job |
| cgroup assignment | Job |
| Log correlation | Job |
| Submitter, job descriptor, progress, retention | Submitted job entry (§8.5) |
A service tracks its current main job's identifier, and a status query returns it.
8.2 Operations
Peios / Peios technical reference / peinit / Jobs and Operations
An operation is a requested state machine action on a service, as a first-class object. Control commands do not mutate state directly: every one creates an operation that is validated, queued, resolved against whatever else is in flight, and executed by the event loop.
Operations exist because peinit serves concurrent callers — administrative tools, automated triggers, other services. Without them, two commands arriving together collide with whatever behaviour falls out; with them, the resolution is explicit and observable.
8.2.1 Lifecycle #
Pending --> Running --> Completed
| |
+-> Merged +-------> Failed
| |
+-> Cancelled +-----> Aborted
|
+-> Failed
| State | Meaning |
|---|---|
| Pending | Validated and queued, waiting on a precondition. |
| Running | Executing. |
| Completed | The goal was reached. Start: Active for Simple, Completed or Inactive for Oneshot. Stop: the service is no longer running. Reload: the reload resolved. |
| Failed | The goal was not reached — or the operation's maximum lifetime expired while it was still Pending (a start peinit is holding on a dependency has no lifetime, §7.5) — or peinit could not execute it (below). |
| Merged | Merged into an existing identical operation, whose identifier is recorded. |
| Cancelled | Terminated while Pending. It never executed. |
| Aborted | Terminated while Running. |
An operation can also fail because peinit could not execute it: the
matrix admitted it and then the boundary found nothing to act on — no
current main job, a transition the state machine does not permit. That
is peinit's fault rather than the service's, and it is contained to the
one operation. The operation fails with the result
internal_error: <cause>, the service keeps the state it had, a
[FAILED] console line names the service, the operation and the cause
(peinit: service <service>: <type> operation <id> failed before it began: <cause>), and the runtime loop carries on. A caller waiting on
the operation is answered INTERNAL_ERROR (§10.1).
The same rule covers every path that is about one service — a job
terminal, a setup status, a health probe, a lifecycle deadline, a notify
datagram, an event the ring refuses. An internal error there fails that
service under InternalError (§6.3): its job is retired, its operation
fails with internal_error: <step>: <cause>, a [FAILED] line
peinit: service <service>: internal error at <step>: <cause> and a
peinit.internal-error.contained event (§8.4) announce it, and the runtime loop
carries on. Only an error about supervision itself — the wait, event
source registration, the listeners, the event ring refusing even the
small peinit.event.dropped, shutdown finalisation — ends the loop and
enters recovery (§2.8). A bug peinit can attribute to one service costs
that service; it does not cost the machine its control socket.
A lifecycle deadline that raised is removed once its error is contained, so it is announced once, not again on every timer turn. One that peinit cannot remove is retried at most once a second, and a retry that raises the same error and changes nothing is not announced again.
Cancelled and Aborted are the same idea at different points: never ran versus was running. Why it happened is a property of the event, not of the state.
Two things reach Aborted. A conflicting operation superseding this one,
and a Restart whose target's definition is withdrawn while its stop leg
is draining — the stop still finishes, but there is nothing to start, so
the operation is aborted with the reason
definition_removed_during_restart_stop_leg (§3.8).
A start that is aborted or cancelled takes its readiness deadline with
it: the start is no longer waiting for READY=1, and ending its main
process is the superseding operation's job. A readiness deadline whose
operation has ended by the time it comes due is dropped, not acted on.
8.2.2 Fields #
Operation {
id: GUID
operation_type: enum // Start, Stop, Restart, Reload, Reset
service: string
state: enum
created_at_ns: u64
started_at_ns: u64?
completed_at_ns: u64?
source: enum
caller: token_summary? // admin-initiated only
result: string?
merged_into: GUID?
}
8.2.3 Sources #
Why peinit created the operation:
| Source | Meaning |
|---|---|
Admin | A control client asked for it. |
Boot | The Phase 2 boot plan. |
Shutdown | The shutdown lifecycle. |
DependencyPropagation | A start operation created one for an unsatisfied dependency. |
RestartPolicy | A restart policy generated a start. |
Timer | A timer trigger fired. |
BindsToRecovery | A bound target returned to Active. |
BindsToPropagation | A bound target stopped. |
ConflictResolution | A conflict evicted the running service. |
OnFailure | A failed service's fallback handler. |
TtyRelease | A service released a terminal another service was waiting for. |
Graceful shutdown creates a Stop operation with source Shutdown for
each participant in its stop waves (§12.2).
All are requested
when the plan is fixed; each starts when its wave becomes eligible.
These records observe the shutdown planner. They do not enter the
ordinary control queue or replace an existing administrator operation,
whose retained stopping deadline still applies. A later-wave observation
may fail its Pending lifetime without authorising an early signal or
cancelling the service's eventual stop.
A participant that exits before its wave can complete its observation without ever starting it; a crash fails it. Once stopping, reaching Inactive completes it, while a stop timeout, global timeout or forced reboot fails any observation still live. A failed observation is not completed again when the process is subsequently reaped.
8.2.4 The types #
Start creates a job for the target. Unsatisfied dependencies produce
their own start operations with source DependencyPropagation. It
completes when the service reaches Active, Completed or Inactive as
appropriate, or Skipped when pre-start conditions do not hold.
Stop sends SIGTERM, arms StopTimeout, escalates to SIGKILL. It
completes when the service reaches Inactive, or Failed after a conflict
eviction or bound-dependency propagation.
Restart is a stop then a start, tracked under one identifier across
both phases, and the type stays Restart throughout for observability.
Reload issues the reload command or signal (§6.5) and completes when the reload resolves. Unlike the other lifecycle commands it defaults to not waiting — the caller gets the identifier immediately.
Reset clears Failed, Abandoned or Skipped, taking the service to Inactive. It is synchronous.
8.2.5 Timeouts #
A start, reload or reset inherits the target's StartTimeout as its
maximum lifetime; a stop inherits StopTimeout.
A restart has
two legs, each enforced against its own timeout, with the sum as the
overall lifetime. A start the graph is holding has no lifetime while
every unsatisfied dependency it waits on is a fact with no clock of its
own — a readiness level not yet published (§7.5), a target in Backoff
(§6.1), or a target itself held only on such facts; it stays Pending for
as long as those facts are undecided, and a caller waiting on it waits
with it. A dependency that is itself starting is bounded by that
service's StartTimeout, and a dependent waiting on one keeps its own
lifetime, measured from creation as below. When a hold ends, the
released start's lifetime runs from the release. Queue time behind
another operation is not a hold and still counts.
This also applies to an on-demand start whose Requires or Wants target
is already in Backoff. The request is accepted and held; it does not
bypass the target's remaining restart delay. Repeated requests merge into
the pending operations. The target's deferred start receives its lifetime
when the restart deadline releases it. If the target is stopped instead,
hard dependents fail and soft dependents may proceed.
For a transitive hold, the required release point can precede the dependent's own dispatch: when its dependency becomes clocked, the dependent must receive its full lifetime even if it is still Pending, waiting for that dependency to finish. Its later dispatch must not restart that lifetime again.
The clock starts at creation, including queue time — except across
a hold (§7.5), during which it does not run. From the caller's point of
view they have been waiting since they sent the command, not since
peinit got round to it. A start that sits Pending behind a stop for
longer than StartTimeout fails without ever running.
8.2.6 Retention #
Pending and Running operations are held in memory. A terminal operation is emitted as an event and dropped after a grace period of 60 seconds — long enough for a polling client to collect the result.
peinit keeps no operation history, for the same reason it keeps no job history. eventd is the historian.
8.3 Conflict Resolution
Peios / Peios technical reference / peinit / Jobs and Operations
When a new operation is requested and one for the same service is already Pending or Running, peinit resolves the two.
8.3.1 Merging #
An operation of the same type merges. The new caller receives the existing operation's identifier, and from their point of view their request is in progress — they neither know nor need to know that it merged.
| Existing | New | Resolution |
|---|---|---|
| Start | Start | Merge |
| Stop | Stop | Merge |
| Reload | Reload | Merge |
| Restart | Start | Merge — a restart already includes a start |
Restart is not mergeable with itself. A second restart while one is in
progress is queued.
The one place that does not apply is a restart deferred by a backoff
(§10.3): it is Pending with nothing running for a second one to queue
behind, so a second restart in Backoff merges into it.
8.3.2 Cross-type #
| Existing | New | Resolution |
|---|---|---|
| Start (Pending) | Stop | Cancel the start, create the stop |
| Start (Running) | Stop | Abort the start, create the stop |
| Start (Pending) | Restart | Cancel the start, queue the restart |
| Start (Running) | Restart | Queue the restart |
| Stop (either) | Start | Queue the start |
| Stop (either) | Restart | Queue the restart |
| Restart (Pending) | Stop | Cancel the restart, create the stop |
| Restart (Running) | Stop | Abort the restart, create the stop |
| Restart (either) | Restart | Queue |
| Reload (Pending) | Stop | Cancel the reload, create the stop |
| Reload (Running) | Stop | Abort the reload, create the stop |
| Reload (Pending) | Restart | Cancel the reload, create the restart |
| Reload (Running) | Restart | Abort the reload, create the restart |
| Anything (either) | Reset | Reject |
Combinations outside this table are rejected: a new Reload while a Start, Stop or Restart is active. A new Start while a Reload is active is not rejected — a reloading service is already where a start would take it, so the command is answered with the service's status and creates no operation, before conflict resolution is reached at all.
8.3.3 The principles #
- Stop wins over start. An explicit stop always takes priority. The administrator said stop, so stop; a queued start can follow.
- Later supersedes earlier. Start then immediately stop means the stop wins and the start is cancelled, recorded as superseded.
- Merging is transparent. The merged caller gets the original identifier and blocks on the original operation's outcome.
Reset is rejected outright while anything is in flight, because reset means "clear a terminal state" and nothing in flight has one.
8.3.4 Dependency propagation #
When a start executes against a service with unsatisfied dependencies, peinit creates start operations for them:
Requires— sourceDependencyPropagation. If one fails, the parent operation fails withDependencyFailure.Wants— sourceDependencyPropagation. If one fails, the parent continues.BindsTorecovery — sourceBindsToRecovery, created when a bound target returns to Active, for dependents sitting in Failed with causeBindsToPropagation.
Dependency-created operations follow the same resolution rules as administrator-created ones. If a dependency is already starting because something else also depends on it, the operations merge — and both graph execution contexts are then associated with the one operation (§7.3).
BindsToRecovery restarts are not subject to the restart budget. They
are created because a dependency returned, not because anything failed.
8.3.5 Restart policy and timers #
A restart-eligible failure creates a start operation with source
RestartPolicy once the backoff delay elapses.
It goes through the ordinary validation and resolution: if an
administrator has already sent a stop, or the budget is exhausted, it is
rejected.
A timer firing creates an operation based on the service's current state:
| Type | State | Action |
|---|---|---|
| Oneshot | Inactive, Completed, Failed | Create a start, source Timer. |
| Oneshot | Active, Starting | Set the pending flag. One catch-up run, no operation. |
| Simple | Inactive, Failed | Create a start, source Timer. |
| Simple | Active, Starting | No-op. The firing is recorded. |
The Oneshot catch-up creates its start when the current run completes. Multiple missed firings collapse into one pending run.
8.3.6 Boot and shutdown are not operations #
Boot and shutdown are modes peinit enters, which then generate
per-service operations. There is no "shutdown operation" to observe or
cancel. Boot-generated starts use source Boot.
Shutdown-generated stops use source Shutdown, and they are the one
exception to the tables above. They are created outside conflict
resolution: a Shutdown Stop does not merge into, cancel or abort an
administrator's Stop or Restart already in flight, but exists beside it,
so that operation keeps its identity and its retained timing (§12.2).
8.4 Event Emission
Peios / Peios technical reference / peinit / Jobs and Operations
peinit emits a structured event at every job and operation lifecycle
transition, and for its own records. All of them go into the KMES
kernel ring buffer through kmes_emit, encoded as msgpack per the KMES
event-record format (Peios Kernel TRM §2).
There is no event socket. Structured events are not sent to eventd over any connection. eventd consumes them from the ring buffer, which is why they survive eventd being down, being restarted, or not existing yet. The only thing peinit sends eventd over a socket is service output (§11.4), which is a different path with different guarantees.
8.4.1 Names and shape #
Every type peinit writes is under its platform root, peinit, and every
payload is a nested map whose fields sit at their catalogue paths
(PGSS §6.4): the job is object.job.guid, the service
object.service.name, the client that asked subject.token.sid, the
result outcome.success. A value peinit does not have is an absent key,
never nil; SIDs are written in binary (bin.sid), job and operation
identifiers as binary GUIDs (bin.guid) that read back as the text
svctl shows, enumerated values in kebab-case (service-main,
explicit-start), and times as nanoseconds since the Unix epoch.
peinit's fragment, /usr/share/evman/peinit.evman, describes every type
and field, and the Events reference generates its pages from it.
No event carries a principal's name, peinit's own log wording, or Rust debug output. What a person reads is on the console; what a consumer queries is in the fields.
8.4.2 The emission policy #
Before peinit builds an event that is not essential, it asks the
emission policy, Machine\Generic\Events (PGSS §6.9): a type switched
off is never built, let alone written. With nothing set, a standard
type is on and a verbose type is off.
peinit opens one view of the policy for the life of PID 1; it needs no
registry to open, decides by tier until the registry is readable, and
applies a committed change to the next decision.
Four types are essential and never ask: peinit.boot.downgraded,
peinit.recovery.entered, peinit.critical-service.failed and
peinit.event.dropped. Each is rare, and each is the only record of
what it describes. The verbose types are peinit.job.created,
peinit.job.status.reported, peinit.operation.started,
peinit.operation.merged, peinit.graph.operation.ended,
peinit.config.reload.deferred, peinit.notify.status.reported and
peinit.notify.progress.reported; everything else is standard.
Setting Enabled = 1 on Machine\Generic\Events\peinit turns all of
them on.
8.4.3 Job events #
peinit.job.created — the job object exists, before its process
does. Carries the job, its type and state, its service and the
activation generation it serves, the operation it serves, the
executable, and the token it will run as.
peinit.job.started — exec succeeded. Adds the process ID
(object.process.pid) and the cgroup (object.cgroup.path).
peinit.job.ended — the process exited or was killed, or the job
ended without starting. Carries the whole record: the terminal state,
outcome.success (true exactly for completed) with the failure cause
as outcome.detail, the exit code or signal, the executable and
arguments, the times it was created and started, its duration, and its
cgroup and cgroup generation.
The token a job runs as is on each once peinit holds it: its user and
group SIDs, and its present and enabled privileges as flags. A service's
token is minted as its process is set up, so its peinit.job.created
carries at most the user SID of a well-known identity.
peinit.job.ended cuts object.job.arguments to 32 KiB of whole
arguments; object.job.arguments-truncated and
object.job.arguments-count say so.
An event the ring
refuses outright is dropped, counted, and replaced by a
peinit.event.dropped naming its type, its length, its service and
job, and the error number, and a [ WARN ] line reports it; only the
ring refusing that small event is fatal (§8.2). The count of drops is
the console's, and the trail's is the number of those records.
peinit.internal-error.contained records an error peinit contained to
one service (§8.2): the step, the service, the job and whether the
service was failed. The error's text stays on the console and in the
failed operation's result.
A submitted job's lifecycle rides the same three events with no service, no activation generation and no operation: the keys are absent. Two more are its own:
peinit.job.status.reported — the job reported STATUS or
PROGRESS. Carries the job, the submitter's SID, and notify.status
and notify.progress.* as retained after the datagram. Written at most
once per job per second; a datagram inside the window updates the view
and writes nothing (§8.5).
peinit.job.output.dropped — the submitter's output sink stopped
draining and peinit began dropping its copy of the job's lines. Once per
job, on the first drop; the record in eventd is unaffected (§11.1).
8.4.4 Operation events #
Every operation event carries the operation (object.operation.guid),
its type, source and state after the transition, and its service.
An operation a client's command created also carries the client as
subject.token.sid; one peinit created itself has no subject.
| Event | Adds |
|---|---|
peinit.operation.requested | — |
peinit.operation.started | — |
peinit.operation.ended | outcome.success, outcome.detail, object.operation.duration |
peinit.operation.merged | object.operation.merged-into.guid |
peinit.operation.ended is one type for the four ways an operation can
end other than merging — completed, failed, cancelled while pending,
aborted while running — and object.operation.state says which.
outcome.success is true exactly for completed. A merged operation
has no outcome of its own: the one it merged into decides it.
object.operation.duration is measured from the request, not from
the start of execution — the same reasoning as the operation timeout —
and is absent on a cancelled operation, which never ran.
What a
caller waited is what matters, and queue time is part of it.
The operation's result, in the words the control interface gives a
client (inactive, startup killed by shutdown), is
outcome.detail, whichever way it ended. The control interface's
operation view calls the same text result or error (PSPU §4).
8.4.5 Ordering #
When one runtime step produces several lifecycle events, peinit emits them in causal order before committing the retained state for that step. For a terminal pre-start graph dispatch, the terminal event for the operation whose outcome satisfied or failed the graph input precedes the events for the operations that dispatch releases, which preserve graph dispatch order.
Every operation in a graph context is requested when the context is
built rather than when its turn comes, so what a release emits is
peinit.operation.started.
8.4.6 Peinit's own records #
peinit's own records go through the same path:
peinit.on-failure.suppressedwhen the fallback chain guard trips;peinit.graph.validation.failedandpeinit.graph.validation.warnedfor validation findings, one per finding, andpeinit.boot.downgradedfor each finding that forced Safe mode;peinit.config.reload.appliedfor an explicitreload-configand for the reload that ends the boot window, andpeinit.config.reload.deferredfor a reload during it (§3.7);peinit.notify.rejectedfor a refused notification datagram, andpeinit.fd-store.rejectedfor a refused descriptor;peinit.notify.status.reported,.errno.reported,.exit-status.reported,.stopping.reportedand.progress.reportedfor the event-emitting notification fields, the last at most once a second for each activation (§10.5);peinit.cgroup.leakedthe first time a sub-cgroup is found still populated after its post-kill deadline (§5.7);peinit.graph.operation.endedfor a graph member's terminal outcome;peinit.recovery.enteredfor the reason peinit dropped to a recovery shell (§2.8);peinit.critical-service.failedandpeinit.service.abandonedfrom the shutdown path (§12);peinit.service.reload.timed-outfor a reload a service started and never confirmed (§6.5).
Records are events rather than logs, and that distinction is the point: the ring buffer persists from the moment PKM loads, so a recovery during Phase 1 is captured before the registry exists, let alone eventd.
8.4.7 Access decisions #
peinit writes no event of its own for a refused command. Every access
check it makes goes through KACS AccessCheck, and KACS records the
decision, as kacs.audit.access.checked, when the descriptor's SACL asks
for it (PGSS §6.7).
peinit's compiled-in descriptors — the default service descriptor, the
control descriptor and a submitted job's default — each carry a SACL
auditing every refusal, for everyone, so a denial is recorded unless a
descriptor says otherwise (§4.6, §13.4). A refusal also leaves a
debugging line on the console, written only at peios.quiet=0.
So that the record says what was decided on, each check passes KACS an
audit context naming its object, which the kernel copies into the record
beside fields.attestation.userspace:
| Checked | Context | The record carries |
|---|---|---|
| A service | {kind: "service", service: {name}} | object.kind service, object.service.name |
| A job, on either socket | {kind: "job", job: {guid}}, the GUID in binary | object.kind job, object.job.guid |
| The control door | {kind: "peinit-system"} | object.kind peinit-system |
The rights in access.requested and access.granted are decoded against
object.kind: the SERVICE_*, JOB_* and SYSTEM_* tables (§4.6,
§8.5, §10.2). job-list checks every job it might list against that
job's own descriptor, so each job it leaves out is one more decision for
the job's SACL.
peinit holds SeAuditPrivilege enabled for this: without it KMES
refuses its events, and the access-check path writes no record (§13.2).
8.5 Submitted Jobs
Peios / Peios technical reference / peinit / Jobs and Operations
A submitted job is a supervised process whose definition arrived on the jobs socket rather than from a service definition. It has no name, no policy and no operation: something asked for it, peinit ran it under an identity the asker was entitled to confer, and it is managed by whoever the job's own Security Descriptor admits. The protocol is PSPU §7; this chapter is how peinit implements its side.
Two shapes drive the design, and one mechanism serves both. A task — a backup, a migration — is submitted by a broker on behalf of the principal it is serving, runs to completion and reports progress on the way. A session — a per-logon host process — is submitted with a freshly issued logon token and one end of a socketpair, and lives until something stops it. Nothing in peinit distinguishes them.
8.5.1 Two doors #
The control socket is an administrator's door: its commands change the
state of the system's services, and its descriptor admits the
principals entitled to do that. Submitting a job is a different
permission held by a different population, so it has a different
socket, /run/services/peinit/jobs.sock (§10.7), with a different
descriptor. The filesystem says who may submit and who may administer,
separately, and peinit carries no policy of its own about either:
being able to connect to the jobs socket is the permission to submit,
and peinit performs no access check before a submit.
What a caller may do to a job is a third question, answered by the
job's own descriptor. A submitter manages its jobs on the jobs socket;
an administrator observes and stops them from the control socket with
job-list, job-status and job-stop (§10.2). Both doors check the
same descriptor with the same rights.
8.5.2 Two identities #
Every submission involves two identities, and peinit keeps them apart.
The submitter is the connection identity: the peer token captured once, at accept, exactly as the control socket captures it (§10.1). It is recorded on the job, it is what the quota counts against, and it is the owner of the job's descriptor. A submitter that connects while impersonating is recorded as the principal it was impersonating.
The job identity is what the process runs as. It is established in
exactly one of two ways, and the boundary that does it —
LinuxJobIdentityProvider — has no third:
- No token attached. peinit opens the primary token of the
connecting process through the kernel's handle on it. The handle
comes from
SO_PEERPIDFDat accept; the token comes frompeios_token_open_processon that pidfd withQUERY | DUPLICATE. The job then has the identity a child of the submitter would have had — its primary token, with no impersonation inherited — so a submitter that was impersonating when it connected produces a job running as itself while being recorded as a submitter under the impersonated identity. A submitter that wants the impersonated identity on the job attaches it. - A token attached. The kernel delivered a token with the
submitmessage throughKACS_SCM_TOKEN, having run the impersonation gates with the submitter as installer and recorded on the token the lower of the socket's level and the source's own (Peios Kernel TRM §3.5). peinit uses that token as it arrived, atQUERY | IMPERSONATE | DUPLICATE.
Either way the source is duplicated to a primary token at its own
impersonation level, with TOKEN_ALL_ACCESS, and the duplicate is
what the child installs. The level carries through: a job started from
an Impersonation-level token cannot convey Delegation, and a submitter
that needs it — for a session that will authenticate onward on its
user's behalf — sets its own jobs connection to Delegation before
attaching, because the attach is clamped to the socket's level.
peinit refuses, with BAD_TOKEN, a source whose impersonation level is
below Impersonation
—
an Identification-level token cannot pass an
access check and an Anonymous one is not an identity to run a process
as — and a source it cannot duplicate. It refuses nothing because of
who the token names. Whether the submitter may run a job as that
identity was the kernel's decision, made when the token was conveyed,
and a second decision inside peinit could only disagree with the first.
There is no identity field in the definition, and no route by which a
plain descriptor passed with SCM_RIGHTS becomes a job identity. A
descriptor passed that way is not gated by the kernel — whoever holds
an fd can pass it — and it would be the one path worth attacking.
The prepared token is a raw descriptor held on the job's entry until
the launch takes its own duplicate of it (materialize_prepared_token,
§4.1). A submission refused after the token was prepared closes it on
the way out.
8.5.3 The submission #
submit runs in this order, and a refusal at any step leaves nothing
behind — no record, no event, no quota consumed, and every descriptor
the message carried closed:
- Parse and validate the definition against the attachment count
(below). A malformed definition is
INVALID_ARGUMENTSbefore any identity work is done. - Refuse during shutdown with
INVALID_STATE. The other four commands keep working while the system goes down. - Establish the job identity, as above.
- Check the quota. peinit counts the submitter's live jobs — those
not yet terminal — by submitter SID against
Machine\System\Init\MaxJobsPerSubmitter, default 64. SYSTEM (S-1-5-18) is exempt. At the bound the answer isQUOTA_EXCEEDED. A job stops counting the moment it is terminal, not when its retained record is dropped. - Build the job's descriptor: the one the submission supplied in SDDL, used as given, or the default (below).
- Create the record, the entry and the launch queue entry, in one
SupervisorWorktransaction.
The job is two things in peinit's state. The job record is the same
JobRecord every fork gets (§8.1), with job_type: Submitted, no
service, no operation, the job identity's user SID as
resolved_identity, and a cgroup id under /sys/fs/cgroup/peinit/jobs/
named by the job's GUID. The entry is what the jobs system holds
beside it: the submitter, the job identity's user SID and logon
session, the definition, the descriptor, the prepared token, the
attached descriptors with their names, the output sink, readiness and
the retained STATUS and PROGRESS, any stop in progress, and once the
job is terminal its outcome and retention deadline. Records are dropped
when terminal, as every job's is; entries outlive them for the grace
period, which is how a terminal job still answers status.
The submit is answered when the job leaves created — on exec
confirmation or on launch failure — and not before.
The
submitter's connection holds a pending wait until then, reads nothing
further, and is not idle. A running job's answer carries a duplicate of
the job's pidfd as SCM_RIGHTS; a failed launch is still
"status": "ok", with a terminal view whose cause says why.
A pidfd that cannot be duplicated
is answered INTERNAL_ERROR rather than a view with the handle silently
missing — the submitter was promised one, and status gives it another
chance.
8.5.3.1 The definition #
| Field | Default | Validation |
|---|---|---|
image_path | required | Absolute, non-empty, no NUL. Not checked for existence: execve decides that, as the job's identity. |
arguments | [] | Strings without NUL. |
environment | {} | Keys non-empty, no =, no NUL; values no NUL. |
working_directory | / | Absolute. |
description | "" | |
timeout | 0, no limit | Non-negative integer, seconds. |
stop_timeout | 10 | Positive integer, seconds. |
readiness | none | none or notify. |
readiness_timeout | 30 | Non-negative integer, seconds. |
success_exit_codes | [] | Integers 0–255. |
descriptors | [] | One name per injected descriptor; non-empty, no :. |
output | false | Requires at least one attached descriptor. |
security_descriptor | the default | SDDL with an owner and a DACL. |
descriptors has to match the attachments exactly: its length plus one
if output is set equals the number of descriptors on the message.
The combined size of arguments and environment is bounded at 2 MiB,
below what execve accepts.
Nothing here
restarts, depends, probes or schedules; a program that needs any of that
is a service.
8.5.3.2 The descriptor #
Unless the submission supplied one, the descriptor is built by
PeiosSystemAccessChecker from the submitter SID: owner and group the
submitter; a DACL granting JOB_ALL_ACCESS to the submitter, to SYSTEM
and to Administrators; nothing else; and a SACL auditing every refusal,
for everyone, so a denied command on the job is recorded (§8.4).
The job identity is granted
nothing. A process cannot, by virtue of running as U, see or stop a job
that runs as U — a submitter that wants the principal to see its own
session says so in the descriptor it supplies.
A supplied descriptor is used as given, with no default entries added — its SACL included, so one with none has its denials recorded by no one. A submitter that omits itself has locked itself out of its own job, and peinit does not prevent that. The descriptor is fixed at submission.
| Right | Bit |
|---|---|
JOB_QUERY | 0x0001 |
JOB_STOP | 0x0002 |
JOB_SIGNAL | 0x0004 |
JOB_ALL_ACCESS | 0x0007 |
The generic mapping is read → JOB_QUERY, write and execute →
JOB_STOP | JOB_SIGNAL, all → JOB_ALL_ACCESS. Every command on either
door checks against this descriptor, the submitter included, naming the
job to KACS in the check's audit context; a denial is answered
ACCESS_DENIED, and the descriptor's SACL decides whether KACS records
it (§8.4).
8.5.4 Launch #
Queued submissions launch from the work pump, one per pump step, through the
ordinary child path (§5.3, §5.4) with three differences. The token
source is LaunchTokenSource::Prepared: the launch takes its own
duplicate of the prepared token through materialize_prepared_token,
and the entry's copy is closed once the launch has it, whatever the
outcome. The attached descriptors ride as inherited descriptors, placed
from 3 upward with close-on-exec cleared and LISTEN_FDS,
LISTEN_FDNAMES and LISTEN_PID set, exactly as the fd store hands
descriptors back to a service (§10.6).
And the environment is built as
a service's is — the compiled-in base, the global layer, then the
submission's environment in the place a definition's own variables
occupy, then the protocol variables last, so a submitter cannot
override NOTIFY_SOCKET or the LISTEN_* set (§5.5).
The cgroup is /sys/fs/cgroup/peinit/jobs/<guid>, a tree of its own
rather than a service's.
Exec confirmation and failure arrive through the setup pipe as for any
job. A parent-side failure — no token, no cgroup, no fork — takes the
record from Created straight to Failed with cause
parent_setup_failure; a child-side failure between fork and exec, or
a failed exec, is pre_exec_failure.
Both close everything the entry
still held and try to remove the cgroup. On success the record is
Running, the runtime registers the job's pipes with origin
jobs/<guid>, and adopts the output sink if there is one (§11.1).
A stop on a job still queued cancels it: the launch entry is removed,
the record fails before start with the stop's cause, and the answer is
the terminal view.
A stop that
lands while setup is pending kills the cgroup and lets setup completion
find the stop already recorded.
8.5.5 While it runs #
A submitted job speaks the notification socket like a service, with one difference in how it is believed: peinit routes a datagram to a service or to a submitted job by a pure PID lookup first — does any service's current main job carry this PID — and only then verifies the sender against that job's pidfd. A submitted job has no activation generation and no replacement, so the generation step does not apply. The sender is verified exactly once whichever way it was routed (§10.5).
What the fields do to a submitted job:
| Field | Effect |
|---|---|
READY=1 | For a notify job not yet ready: ready becomes true and the readiness timeout is disarmed. Otherwise ignored. |
STATUS= | Retained as status_text. |
PROGRESS= | Parsed as §4.19's three forms; retained as progress. A value outside the forms, T of zero or N above T is dropped, never repaired; the rest of the datagram is applied. |
PROGRESS_UNIT= | bytes, items or percent, retained; anything else dropped. |
STOPPING=1 | Recorded, so a later stop sends no termination signal. |
| Everything else | Ignored: a job has no reload, no watchdog, no fd store. |
A datagram that changed status_text, progress or the unit is due a
peinit.job.status.reported event carrying the retained values (a
verbose type, written only when the emission policy turns it on) —
but at most one per
job per second, measured from the last event emitted for that job, so
a sender updating a thousand times a second produces one event a
second while the view is current on every query.
Deadlines are held on the entry and folded into the one lifecycle
deadline timer the event loop arms, as SubmittedJob kinds:
| Deadline | Due | Action |
|---|---|---|
Timeout | started_at + timeout | Begin a stop with cause timeout. |
ReadinessTimeout | started_at + readiness_timeout, for a notify job not yet ready | Begin a stop with cause readiness_timeout. |
StopKill | stop requested + stop_timeout | SIGKILL the job's cgroup. |
PostKill | kill + PostKillTimeout | Abandon (below), or re-arm if the cgroup is empty but the exit not yet reaped. |
CgroupCleanup | terminal + PostKillTimeout | Retry removing a cgroup that was busy when the job ended. |
A job being stopped has only its stop deadline; a deadline that fires for a job that has already ended does nothing.
8.5.6 Stopping #
Every stop — a submitter's stop, an administrator's job-stop, a
timeout, shutdown — goes through begin_submitted_stop: record the
cause and the kill deadline, then SIGTERM the main process through its
pidfd unless the job has already sent STOPPING=1.
A
second stop on a job already stopping changes nothing and does not
restart the deadline; a stop on a terminal job is a no-op answered with
the unchanged view.
At the kill deadline the cgroup is killed.
At the post-kill
deadline, a cgroup still populated means the process survived SIGKILL:
the record is abandoned, the cause becomes process_unkillable, the
cgroup is leaked and reported as peinit.cgroup.leaked, naming the job
rather than a service, and supervision stops.
signal is the raw mechanism, deliberately: one signal, by number, to
the main process through its pidfd, on a running job only.
SIGKILL
this way leaves whatever the job spawned and produces a failed job with
exit_signal set and a null cause.
A
submitter that wants the job ended uses stop.
8.5.7 Ending #
On reap, exit code 0 or one in success_exit_codes completes the job;
any other code, or a signal, fails it.
Then
whatever the job left in
its cgroup is killed and the cgroup removed; a busy cgroup schedules
one CgroupCleanup retry, and a cgroup still busy after that is
reported leaked. The prepared token, any attached descriptor the launch
never took, and any sink the runtime never adopted are closed. The sink
the runtime did adopt closes when the job's last pipe closes.
cause records what peinit decided, and stays null when the process
ended of its own accord.
A stop
whose process handled SIGTERM and
exited 0 is completed with cause explicit_stop: peinit asked, the
process agreed, and both facts are recorded.
A
stopped job that died to the signal is failed with exit_signal set
and the same cause.
The record is dropped at once, as every terminal job's is (§8.1), and
peinit.job.ended carries it. The entry is retained for 60 seconds so a
submitter polling for the outcome finds it, and then purged by
operation maintenance; status on a purged or never-existent
identifier is UNKNOWN_JOB either way.
A
terminal job whose cgroup cleanup is still pending is not purged until
the retry has run.
8.5.8 Shutdown #
When a shutdown begins, every live submitted job is stopped at once,
with cause shutdown and no ordering between them or against the
service waves — a job has no dependencies to order by.
A job still
queued for launch is cancelled with the same cause. The shutdown is not
finished while a live job remains: a job's reap advances shutdown
progress exactly as a service's does, and the global timeout kills
every live job's cgroup along with the remaining services (§12.2).
submit is refused throughout; the other four commands keep answering,
because a submitter watching its jobs go down has a reason to keep
looking.
8.5.9 Clients #
svctl job list|status|stop are the control socket's job commands, and
svctl job submit|wait|signal speak the jobs socket, submitting as the
caller's own primary token with --fd NAME=FD for descriptors and
--output for a sink.
libpeinit exposes the same two halves:
peinit_job_status, peinit_job_list and peinit_job_stop on the
control client, and <peinit/jobs.h> — a peinit_jobs_t with
peinit_job_submit, peinit_jobs_status, peinit_jobs_wait,
peinit_jobs_stop and peinit_jobs_signal — with
peinit_response_take_pidfd to claim the handle a submit returned.
8.6 What a Caller Sees
Peios / Peios technical reference / peinit / Jobs and Operations
Operations are how a caller observes something taking effect. This article covers what the model looks like from the outside; the wire contract itself is PSPU §4.
8.6.1 Every lifecycle command returns an identifier #
A command that creates, merges into, queues, cancels, clears or executes an operation returns that operation's identifier. A caller can then poll it, or block on it.
Two cases return no identifier, because no operation exists: a command whose target is already in the state it asks for, and one that has no effect at all — a stop on an Inactive service. Both return the service's status instead of an acknowledgement, which is the honest answer, and which also says what else the caller may do to the service (§10.2).
Commands that never create an operation — status, list,
operation-status — have their own shapes. So do the three job
commands, which never create an operation either: a submitted job has
no state machine to contend for, so job-stop acts on it directly and
answers with the job view.
8.6.2 Waiting #
Lifecycle commands block by default until the operation is terminal.
The
exception is reload, which returns immediately unless asked otherwise,
because a reload's outcome is often advisory and a caller usually wants
the identifier rather than the wait.
A waiting connection is not idle and is never closed by the idle timeout, however long the operation runs. It is bounded by the operation's own lifetime instead.
8.6.3 Merging is invisible #
A caller whose command merged receives the surviving operation's
identifier and blocks on that operation's outcome.
Nothing tells them
they merged, because there is nothing they could usefully do about it.
The consequence worth knowing is that the identifier a caller gets back
may be older than their request, and its created_at will be earlier
than when they sent it — which is exactly right, because that is when
the work they are waiting on actually began.
8.6.4 What an operation's result carries #
A completed stop or reset carries the resulting service state; a
completed start carries the readiness that satisfied it — alive readiness: process started, notify readiness: READY=1, or the exit
code for a Oneshot — rather than a state name. A failed one carries the
failure reason; one that peinit could not execute (§8.2) carries
internal_error: <cause>. A merged one carries the survivor's
identifier. A cancelled or aborted one carries why.
For a reload, the result also determines the reload's mode — whether
the service confirmed the reload with a READY=1, whether peinit is
merely assuming it happened, or whether it outright failed.
8.6.5 The job view #
A submitted job is reported in one shape on both sockets, the job view
of PSPU §7.7: identifier, type of submitted, state, cause,
submitter and identity SIDs, logon session, description, image path,
pid while there is a process, ready for a notify job, exit code
or signal, the retained status_text and progress, and the three
timestamps. Every inapplicable
field is present and null, and pid is null once the job is terminal —
there is no process to name.
Abandoned is
the exception, and the reason for it: an abandoned job's processes
survived SIGKILL and are still running, so its pid is kept.
What a submitter sees of its job on the jobs socket is exactly what an
administrator sees of it in job-status, but for one field the control
socket adds: granted, the job rights that caller holds on it (§10.2).
And job-list
is a list of the same views, filtered by JOB_QUERY on each (§10.2).
A job-stop
with wait registers a wait on the connection like a lifecycle
command's, answered with the terminal view — or UNKNOWN_JOB, if the
record was purged before the answer could be sent.
8.7 Submission and Progress Examples
Peios / Peios technical reference / peinit / Jobs and Operations
This reference expands the submitted-job operator workflow. Job implementation is in §8.5; transport is in §10.7 and the submission protocol is PSPU §7.
8.7.1 What a job runs as #
There is no identity field. A submitter cannot name an identity for its job; it can only run the job as an identity it already holds:
- Nothing attached — the job runs as the submitting process's own primary token, exactly as a child of the submitter would. This is the common case for a tool or a service running something as itself.
- A token attached — the submitter attaches a token to the submit message, and the kernel (through KACS) gates the attach: it verifies the submitter could act as that identity, at that impersonation level, at that moment. peinit turns the attached token into the job's primary token. This is how
backupdruns a backup as its client, or a logon service starts a session as the user who just authenticated — the same impersonation machinery, with the kernel as the only judge. A job whose session must convey Delegation needs the submitter to set its jobs connection to Delegation level first; a job started at Impersonation cannot be raised later.
A token below Impersonation level, or one peinit cannot duplicate, is refused with BAD_TOKEN. peinit never refuses a token because of who it names — that decision was the kernel's.
Two identities are recorded on every job, and the view shows both: the submitter (who asked — the identity the connection was made under) and the identity (who it runs as), plus the identity's logon session.
8.7.2 The definition #
A submission carries the whole job in one message: image_path (absolute; required), arguments, environment, working_directory (default /), description, timeout (seconds; 0, the default, is no limit), stop_timeout (default 10 s), readiness (none or notify, to wait for READY=1), readiness_timeout (default 30 s), success_exit_codes, a list of named descriptors to hand the job, an output flag, and optionally a security_descriptor for the job itself. There are deliberately no policy fields — no restart, dependencies, health check, or trigger. A program that needs those is a service.
peinit does not check that the program exists or is executable when it accepts the submission; that is decided at execve, by the job's own token, and a program that cannot be run becomes a failed job rather than a refused submission. The submit is answered only once the job has left created — the process confirmed its exec, or starting it failed. A readiness: notify job is answered when it is running, not when it is ready; a submitter that wants to block until readiness follows with a wait.
Descriptors a submitter attaches are handed to the job from descriptor 3 with LISTEN_FDS, LISTEN_FDNAMES and LISTEN_PID set — the same convention a service uses to adopt stored descriptors, so a program written for one adopts them from the other unchanged. This is what makes a session job possible: the submitter keeps one end of a socketpair and attaches the other.
8.7.3 The job view #
Whether you ask on the jobs socket or with job status, you get the same view: the job id, type (submitted), state and cause, submitter and identity SIDs, logon_session, description, image_path, pid (while running), ready (null for a readiness: none job, otherwise whether READY=1 has arrived), status_text, progress, the exit code or signal, and the created_at/started_at/ended_at timestamps.
status_text and progress come from the job itself, through the same sd_notify socket a service uses. A job sends STATUS=Backing up /data/media for a human-readable line and PROGRESS= in one of three forms:
| Form | Meaning | Shown as |
|---|---|---|
PROGRESS=N | Counting, with no end in sight. | A rising count. |
PROGRESS=N/ | Counting towards an end that exists but is not yet known. | A rising count, awaiting a bound. |
PROGRESS=N/T | N of T. | A bar. |
PROGRESS_UNIT=bytes|items|percent says what the numbers are. peinit keeps the latest accepted values and emits a peinit.job.status.reported event when they change, rate-limited to at most one per job per second so a chatty job cannot flood the event stream — the view always has the latest value regardless.
9.1 Calendar Expressions
Peios / Peios technical reference / peinit / Timers
A timer schedule is a calendar expression in systemd's OnCalendar
format. The grammar below is what peinit parses.
DayOfWeek Year-Month-Day Hour:Minute:Second Timezone
Every field is optional, and which fields are present is worked out positionally from their shape.
| Field | Form | Default if omitted |
|---|---|---|
| DayOfWeek | weekday names | any day |
| Date | Year-Month-Day | *-*-* |
| Time | Hour:Minute:Second | 00:00:00 |
| Second | the :Second of Time | :00 |
| Timezone | an IANA name | system-local |
A time-only expression such as 02:00:00 implies the date *-*-*.
Hour:Minute alone defaults the seconds to 00.
Years range 0–9999, months 1–12, days 1–31.
9.1.1 Component syntax #
Every numeric component, and the weekday, accepts:
- Wildcard —
*matches anything. - List — comma-separated values:
1,15. - Range — two values around
.., inclusive:Mon..Fri,8..17. A reversed range is a parse error rather than a wrap-around. - Repetition — a value or range suffixed with
/and a step.value/stepmatches the value and every multiple of the step above it, so0/15in the minute field is 0, 15, 30 and 45.start..end/stepwalks from start to end inclusive.
Weekdays are English names, case-insensitive, abbreviated or full, and
accept lists and ranges.
Mon and Monday are the same day, as are Tue, Tues and Tuesday.
9.1.2 Last day of the month #
A ~ in place of the - between Month and Day counts the day from the
end of the month: ~01 is the last day, ~02 the second-to-last,
and so on.
*-*~01 is the last day of every month; *-02~03 is the
third-to-last day of February.
Repetition combines with it, and steps in the day-of-month direction —
which means it walks the offset downwards, towards the end of the
month.
Mon *-05~07/1 covers the last seven days of May, and combined
with Mon resolves to exactly one day: the last Monday in May.
Wildcards, ranges and lists are also accepted after ~, so ~* matches
every day and ~03..07 the third- to seventh-to-last.
9.1.3 Named shortcuts #
| Shortcut | Equivalent |
|---|---|
minutely | *-*-* *:*:00 |
hourly | *-*-* *:00:00 |
daily | *-*-* 00:00:00 |
weekly | Mon *-*-* 00:00:00 |
monthly | *-*-01 00:00:00 |
quarterly | *-01,04,07,10-01 00:00:00 |
semiannually | *-01,07-01 00:00:00 |
yearly, annually | *-01-01 00:00:00 |
Shortcut names are case-insensitive and accept a trailing timezone, so
daily UTC is valid.
9.1.4 Timezones #
Timezone specifiers are IANA database names — Europe/London,
US/Eastern, UTC. An expression with no timezone is interpreted in
system-local time.
A name is validated when the expression is parsed and resolved again at evaluation. An unrecognised zone is a hard parse error, not a silent fallback to UTC.
9.1.5 Daylight saving #
- Spring forward, where the clock skips an hour: a scheduled time
falling inside the skipped interval does not fire. peinit moves on to
the next scheduled second, then the next date.
A schedule of
*-03-31 01:30 Europe/Londonskips 2024 entirely. - Fall back, where an hour repeats: a scheduled time inside the repeated interval fires exactly once, on the first occurrence. On re-arming, the same civil time resolves to the same instant, which is not later than the firing that just happened, so the timer advances to the next day rather than firing again.
9.1.6 Precision #
Second-level. Unlike systemd, peinit does not accept fractional seconds — service scheduling has no use for finer granularity, and dropping it keeps the parser simpler. A fraction anywhere in the time component is a parse error.
9.1.7 Examples #
| Expression | Meaning |
|---|---|
*-*-* 02:00:00 | Every day at 2am, system-local. |
Mon *-*-* 00:00:00 | Every Monday at midnight. |
*-*-1,15 12:00:00 | The 1st and 15th, at noon. |
*-*~01 00:00:00 | The last day of each month, at midnight. |
Mon..Fri *-*-* 09:00:00 | Weekdays at 9am. |
*-*-* *:00/15:00 | Every fifteen minutes. |
*-*-* 02:00:00 Europe/London | Every day at 2am London time. |
9.2 Evaluation and Arming
Peios / Peios technical reference / peinit / Timers
A timer is a trigger, not a service type. A service with a
timer:<schedule> trigger is an ordinary Simple or Oneshot service that
peinit starts on a schedule.
9.2.1 Arming #
At boot, once the service graph is loaded, and whenever timer configuration changes, peinit computes the next firing time for every active trigger and arms a timerfd for it. Each trigger gets its own descriptor and its own computation.
A disabled service gets neither a registration nor a firing.
A schedule that fails to parse, or whose next occurrence cannot be
computed, fails that trigger. Every other timer arms normally, and what
did not arm is reported to the console and in the service's status.
This matches how graph
validation already treats an invalid schedule (§7.4), so the outcome no
longer depends on which of the two caught it.
The next-occurrence search looks ten years ahead and then gives up.
A
schedule can parse and still match nothing — *-02-30, or a fixed year
already past — and the horizon turns that into a prompt error against
the one service rather than a very long walk. Ten years clears the
sparsest schedule that is genuinely meaningful: *-02-29 skips a
century year not divisible by 400, so it can run eight years dry.
9.2.2 Firing #
handle_timer(service, trigger):
// 1. Decide what the firing means, from the service's state.
match (service.type, service.state):
(Oneshot, Active | Starting):
service.pending_timer = true // at most one
(Simple, Active | Starting):
record the firing; no action
(Oneshot, Inactive | Completed | Failed):
create_operation(Start, service, source = Timer)
(Simple, Inactive | Failed):
create_operation(Start, service, source = Timer)
// 2. Record when it fired.
write the last-run timestamp to the registry // asynchronously
// 3. Re-arm.
next = next_occurrence(trigger.schedule, now) + random(0, TimerJitter)
arm an absolute CLOCK_REALTIME timerfd for next
Every other state — Backoff, Stopping, Reloading, Abandoned, Skipped — records the firing and does nothing.
A timer can outlive its service until the next reload. A service whose definition is deleted while it runs carries on, and is discarded when it stops, and nothing re-plans the timers then. A firing for a service no longer known does nothing, records no last run, and is not armed again; the next reload leaves it out. It is never an error: an error out of a firing takes PID 1 into recovery.
No last run is recorded for a service whose definition has been deleted, whether it is still running or already gone: there is no key to record it in, and writing one would make the key again.
The last-run write happens in a forked child so that the event loop never waits on the registry — which matters because the registry is served by registryd, a service peinit supervises, so a synchronous write would let a wedged registryd stall PID 1.
The parent returns immediately with the child's pid, and remembers it. When the child is reaped, peinit matches its exit status back to the write; a failure is reported:
peinit warning: recording the last run of timer <schedule> for service
<service> failed; it will run catch-up again after a reboot
The write stays best-effort — nothing is retried and nothing is failed over it — but a persistent timer whose timestamp never lands runs its catch-up on every boot, and that is otherwise a symptom with no thread to pull. The outstanding-write table is bounded, so a child that somehow escapes reaping cannot grow it.
9.2.3 Oneshot pending runs #
A Oneshot that fires while it is already running sets a flag rather than queueing an operation. When it next reaches Inactive or Completed, peinit immediately creates a start operation and clears the flag.
Multiple firings during one run collapse into a single pending run. There is no queue, and the flag is per service rather than per trigger — a service with three timers that all fire during one long run still gets exactly one catch-up.
9.2.4 Multiple triggers #
Triggers on one service are independent: each has its own timerfd, its own next-firing computation, and its own last-run history. Only the Oneshot pending flag is shared.
9.2.5 Reporting #
status reports every trigger of a service that is not disabled, as it
stands: the occurrence armed, the time it will fire with its jitter, and
when it last fired; or, for a trigger that did not arm, why not (PSPU
§4.14). list carries the soonest firing of each service.
The figures are the armed ones, given to the supervisor whenever a trigger is armed: at boot, on a reload that adds or changes the timer (an unchanged timer keeps its armed deadline), after each firing and after a clock change. A client cannot compute them itself, because the jitter is drawn at random when the trigger is armed.
When a trigger last fired is held in memory as well as the registry. It is seeded at boot from the recorded timestamp of a persistent trigger, set by each firing, including a boot catch-up, and kept across a reload for the same service and schedule. A definition deleted and made again between two reloads is the same service and schedule, and keeps it too. A non-persistent trigger, which records nothing (§9.3), reports only its firings since peinit started.
9.3 Persistence
Peios / Peios technical reference / peinit / Timers
TimerPersistent, on by default, controls whether a run missed across a
reboot is caught up.
9.3.1 Where history lives #
Last-run timestamps are REG_QWORD values in the registry, written
after a firing.
A service with a single timer trigger stores its timestamp as
LastTimerRun on the service's own key:
Machine\System\Services\<name>\LastTimerRun
A service with multiple triggers stores one per trigger under a subkey, named by the schedule string:
Machine\System\Services\<name>\TimerState\<encoded-schedule>
A schedule contains characters — spaces, :, * — that are not valid
LCS value names, so the name is the schedule with every character
outside [A-Za-z0-9._-] percent-encoded, with uppercase hex digits.
This is the same encoding used for cgroup ids (§5.1). The schedule
*-*-* 02:00:00 is stored as:
%2A-%2A-%2A%2002%3A00%3A00
Two identical schedule strings on one service encode to the same name and therefore share one timestamp.
Timer firings are infrequent, so the write cost is negligible.
9.3.2 Catching up #
On boot, for each persistent trigger:
- Read the last-run timestamp.
- Compute the next scheduled firing after it.
- If that time has already passed, at least one run was missed: fire once, immediately.
- Compute the next future occurrence normally.
Catch-up is always a single run however many were missed. A daily timer that missed five days fires once on the next boot, not five times.
A trigger with no history at all is treated the same way, so its first boot produces one catch-up firing.
TimerPersistent=0 ignores history entirely — peinit does not even read
the registry for that trigger, and computes the next occurrence from
now.
9.3.3 When the timestamp is written #
The timestamp is written after the timer fires and the start is initiated, not after the service finishes. A service that crashes mid-run is not re-triggered on the next boot: the run was attempted, not missed.
A configuration reload keeps an already-armed timer when its service name,
schedule string, TimerPersistent, TimerJitter, and timestamp storage
layout are unchanged. Its scheduled occurrence, jittered firing time,
and last firing are retained, even if the occurrence has passed while
its jitter is still pending. Unrelated configuration edits and a timer's
own LastTimerRun writes therefore cannot postpone that firing.
New or changed timers are armed from the current time with no catch-up,
whatever TimerPersistent says. Removed or disabled triggers are
unregistered. History is consulted at boot only.
9.4 Jitter and Clocks
Peios / Peios technical reference / peinit / Timers
9.4.1 Jitter #
TimerJitter, zero by default, adds a random delay to each firing.
peinit draws a uniformly random whole number of seconds from zero to
TimerJitter inclusive, from the kernel's random source, and adds it to
the computed occurrence.
The delay is recomputed on every firing, so a daily timer with
TimerJitter=900 fires at a different moment between 00:00 and 00:15
each day.
With TimerJitter=0 no randomness is consulted at all.
Jitter is applied after the calendar expression is evaluated and is only ever added, so a timer never fires early — only late.
The boot catch-up firing is not jittered. It fires immediately, and jitter applies from the next armed occurrence onward.
9.4.2 Which clock #
The split is the point of this section, and it is not cosmetic.
Calendar timers are wall-clock schedules, so they are armed as
absolute CLOCK_REALTIME timers: timerfd_settime with
TFD_TIMER_ABSTIME | TFD_TIMER_CANCEL_ON_SET. CANCEL_ON_SET makes the
descriptor's read return ECANCELED whenever the realtime clock is
discontinuously changed — an NTP step, a manual set. peinit recomputes
the next occurrence against the new wall clock and re-arms, which is
what keeps *-*-* 02:00:00 anchored to 02:00 across clock corrections.
Interval timers are genuine relative durations and use
CLOCK_MONOTONIC: the watchdog, health check intervals and timeouts,
restart backoff, and the Start, Stop and Reload phase timeouts. "Wait
thirty seconds" means thirty elapsed seconds regardless of what happens
to the wall clock.
They are not armed with CANCEL_ON_SET, correctly —
a monotonic timer has no reason to be cancelled by a realtime set.
These are not separate descriptors. Every interval deadline is aggregated onto one monotonic timerfd armed to the earliest of them.
Last-run timestamps are recorded on CLOCK_REALTIME, since they
record when a timer actually fired in wall-clock terms.
The same firing
passes a monotonic timestamp into the operation machinery, because
operation timing is elapsed time.
9.4.3 Clock events #
- A realtime step at runtime. The armed timer is cancelled; peinit recomputes against the new wall clock and re-arms. A backward step pushes the next firing later; a forward step that crosses an occurrence fires it once. If the step lands inside a jitter window, the firing happens at the un-jittered scheduled time — later than the schedule, never earlier.
- Suspend and resume. An absolute deadline that elapsed while suspended fires once on resume. The expiration count is ignored, so a long suspend produces one firing, not one per occurrence.
- A missed occurrence within one uptime. Fire once, then compute the next future occurrence. peinit never replays every occurrence that elapsed during a gap — the same rule as the cross-reboot catch-up.
- A backward jump across a boot. If the last-run timestamp is in the future relative to the current wall clock at boot, peinit treats the history as unknown and fires the catch-up immediately. This check is boot-time only; there is no runtime equivalent.
- A wrong clock at boot. A system that boots with a badly wrong
clock and has NTP correct it later may fire a persistent catch-up
spuriously or not at all. The runtime half is covered by
CANCEL_ON_SET— once NTP corrects the clock, armed calendar timers are cancelled and recomputed — but the boot-time catch-up decision has already been made by then. Short of NTP-aware rescheduling, this remains an edge.
10.1 The Control Socket
Peios / Peios technical reference / peinit / The Control Interface
peinit serves every runtime command on a Unix stream socket at
/run/services/peinit/control.sock, created during Phase 1
infrastructure setup and existing for the lifetime of the system. Its
wire protocol is specified in PSPU §4; this chapter is how peinit
implements its side. A second socket beside it, the jobs socket, is the
door for submitting jobs rather than administering services; it is
§10.7.
10.1.1 Creation and protection #
The socket is created with SOCK_CLOEXEC | SOCK_NONBLOCK and a listen
backlog of 32, and unlinked when peinit drops it.
Accepted connections come from
accept4 with both flags, so no connection descriptor is ever inherited
by a service.
peinit sets no POSIX mode bits on the socket, on the notification socket, or on anything else it creates. Under KACS, mode bits are not what governs access — a Security Descriptor is — so setting them would be inert.
What governs access is the descriptor on the socket inode, which the
kernel checks at connect() before any peer identity is established.
peinit creates /run/services/peinit/ and stamps the control socket
explicitly, after binding and before anything can connect
:
O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU)
Every authenticated principal may connect. FW is the file-write generic
right, which is what connect() on a pathname socket needs. Reaching the
socket grants nothing on its own: every command is checked against the
control descriptor or the target service's descriptor, with the token
captured at accept (§10.2). Admitting only SYSTEM and Administrators would
make that check moot for everyone else, since a descriptor that grants a
principal something could never be reached by them.
The notification socket is stamped explicitly too, with a descriptor of its own that grants SYSTEM and the Service group rather than Administrators (§10.5). The jobs socket carries a broader descriptor again, because reaching it is a different permission (§10.7).
10.1.2 Connections #
peinit accepts a connection, obtains the peer's token, and only then admits it against the connection limits:
The per-user limit is what keeps one caller from taking the connections
everyone shares: with every authenticated principal able to connect, a
single overall limit would let anyone fill it and lock administrators
out. SYSTEM is exempt, as it is from MaxJobsPerSubmitter, because what
it does is the machine's.
A connection over either limit is closed at the socket level, before any request is read and without a response — there is no error code for it, because there is no protocol state in which to deliver one. A peer whose token cannot be obtained is closed the same way.
The jobs socket has its own three, under Machine\System\Init\:
MaxJobsConnections (64), MaxJobMessageSize (32768) and
JobsConnectionTimeout (30), plus a fourth bound that is the
submitter's rather than the connection's, MaxJobsPerSubmitter (64).
All four are read at boot and on reload-config, and are listed with the
other operational keys in the registry key reference.
10.1.3 The peer token #
The token is read once, when the connection is accepted, through
the kernel's peer-token socket option (getsockopt(SOL_KACS, KACS_SO_PEER_TOKEN); the Peios Kernel TRM §3.5). It is the identity the
peer thread was acting under when it connected, so a peer that was
impersonating is captured as the impersonated identity
— which is what makes access decisions reflect the identity a client is
actually operating under rather than its underlying service identity.
Because it is captured once, a peer that changes identity mid-connection is still evaluated against the identity it connected with.
10.1.4 Idle and waiting #
A connection is idle only when it has nothing in flight. One blocked on
a wait=true operation, on a job-stop with wait, or with output
still buffered, is never idle and is never closed by
ConnectionTimeout
— it stays open until the operation or job
resolves, bounded by the operation's own timeout rather than the
connection's. A job wait has no timeout of its own; it is bounded by
the job.
A wait is answered when its operation reaches a terminal state, with
the lifecycle acknowledgement carrying the service's state at that
moment. Two cases are answered differently. An operation that failed
because peinit could not execute it (§8.2) is answered with an
INTERNAL_ERROR whose message carries the operation's result, not with
an ok carrying an unchanged state. And an operation whose service has
been discarded by the time it resolves — a stop that drained a
definition-removed entry (§10.3) — is answered with the state the
service last had, since there is no entry left to ask. Neither answer
costs the connection or the listener.
peinit handles one frame per readiness turn, and processes no further frames from a connection while a wait is pending on it. Pipelined requests are therefore serialised behind a wait.
10.1.5 Timestamps #
Every timestamp peinit puts on the wire is derived by projecting a monotonic event stamp through the current offset between the realtime and monotonic clocks. Elapsed-time decisions stay monotonic; only the presentation is wall-clock.
10.2 Dispatch and Authorisation
Peios / Peios technical reference / peinit / The Control Interface
A parsed command runs a fixed sequence before it does anything.
- The shutdown gate. If peinit is shutting down, everything except
status,list,operation-status,boot,job-status,job-listandjob-stopis rejected.bootstays open because, like the service queries, it changes nothing. The three job commands stay open because a shutdown stops every submitted job anyway (§12.2), and an administrator watching that happen, or wanting a job gone sooner than the shutdown will manage, has a reason to ask. The gate runs before the access check, so during shutdown a caller who would have been denied is told the command is invalid for the current state rather than that they lack the right. - Resolve the target. A command naming no definition, and no
addressable definition-removed entry, returns
UNKNOWN_SERVICE; one naming no submitted job peinit holds returnsUNKNOWN_JOB. peinit does not synthesise a descriptor for something that does not exist. - AccessCheck. The caller's token, the target's descriptor, the generic mapping, the right the command requires (§4.6, §4.7), and an audit context naming the target: the service, the job, or the control door itself (§8.4). For a job the descriptor is the job's own and the mapping is the job mapping (§8.5).
- On denial, return
ACCESS_DENIED. The decision is recorded by KACS, askacs.audit.access.checkednaming the caller, the target and the access requested and granted, whenever the target's descriptor has a SACL that asks for it; every descriptor peinit builds itself audits every refusal. peinit writes no event of its own. Silent denial is not acceptable; a denial an administrator cannot see is indistinguishable from a bug, which is why the defaults audit, and why a descriptor written to the registry should keep a SACL that does (§4.6). - On grant, classify the command against the service's state (§10.3) and act.
10.2.1 Rights #
operation-status resolves the operation before it checks the right, so
an unknown identifier is reported as unknown regardless of who asked.
If the operation's service has since been discarded (§3.8), the right is
checked against the ServiceSecurity the service had when the operation
was created, which peinit records on the operation; a retained operation
therefore stays queryable for its retention window (a2) by exactly the
callers who could have queried the service.
job-status and job-stop resolve the job first too, and for them
that is the whole story: UNKNOWN_JOB for an identifier that names
nothing, ACCESS_DENIED for one that names a job the caller may not
touch. A job identifier is unguessable, so a caller that names one was
told it by something that had it, and there is no existence to hide.
job-stop creates no operation; it stops the job directly (§8.5) and,
with wait, registers a job wait on the connection answered by the
terminal view.
10.2.2 What the caller may do #
A status response carries granted, the service rights the caller
holds on the service, so that a client can offer only the commands that
will be allowed rather than learn the rest from ACCESS_DENIED (PSPU
§4.14). peinit finds them with step 3's AccessCheck — the caller's
token, the service's ServiceSecurity, the service mapping — asking
for MAXIMUM_ALLOWED instead of one right, once the query itself has
been allowed, and reports the service rights among what it grants.
A lifecycle command that
has nothing to do answers with the status (§10.3), and that carries
granted for its caller too.
Each job view on the control socket carries granted the same way:
the job rights the caller holds, from the AccessCheck against the job's
own descriptor with the job mapping (§8.5). job-stop with wait
finds them when it is taken and keeps them on the connection's wait,
because the terminal view is written by the run loop with no caller's
token to hand; the descriptor is fixed at submission, so they are the
same as at the answer. The jobs socket's view has no granted.
The MAXIMUM_ALLOWED check is a question, not a command. A right it
does not grant is left out of granted; nothing is refused, and peinit
answers the command as it would have without it.
10.2.3 Filtering #
list checks every service and partitions the result. Services the
caller can query are returned; services it cannot are omitted, and the
denials are left to each service's SACL rather than shown to the caller. A
caller with no query rights anywhere gets an empty list and a successful
response, not a denial . The
filtering keeps a service's state from whoever may not query it; it does
not hide that the service exists, which step 2 above already answers
for any command that names one (§4.6).
Definition-removed services are listed, and the list entry does not say
so. A status query on one does.
job-list is filtered the same way. The four request filters —
submitter, identity, logon_session, state — narrow the
candidates first; each survivor is then checked for JOB_QUERY against
its own descriptor, each denial is the job's SACL's to record, and the
response does not say whether a filter or a right removed an entry.
Terminal jobs still within their retention are listed; a caller that
wants only live ones filters by state.
10.3 The Command × State Matrix
Peios / Peios technical reference / peinit / The Control Interface
A command sent to a service in an unexpected state gets an answer, not a silent no-op. What the answer is depends on the pair.
ALREADY — the service is already where the command would take it and no operation of that type is in flight. peinit returns the current status rather than an error.
MERGE — an operation of that type is already running. The command merges into it; the caller receives that operation's identifier and, if waiting, blocks on its outcome.
DEFER — create a Pending start operation but do not execute it until the existing backoff deadline expires. A deferred start already present is merged into.
QUEUE — the operation is queued Pending and executes after the current one completes.
NOOP — the command has no effect. peinit returns the status.
ERROR — the command is invalid for the state.
Clear — reset to Inactive.
Cancel — abort the current operation, then proceed.
10.3.1 The Backoff column #
Backoff is the interesting one, because the service is down with an automatic restart already pending.
startcreates or merges into a deferred start operation and honours the remaining delay. It does not short-circuit the backoff. If the automatic restart later becomes due, it merges into the administrator's operation, so the identifier the caller holds is the one that executes.stopcancels both the pending restart and any deferred start, and the service goes Inactive. A subsequent automatic restart is refused, because the service is no longer in Backoff.restartreplaces the automatic restart with the administrator's. A Restart operation with sourceadminis queued Pending and honours the remaining delay exactly as a deferredstartdoes; when the delay expires it is that operation — keeping its type, for observability — that executes, so the identifier the caller holds is the one that runs. A deferredstartalready pending is cancelled with reasonsuperseded_by_restart, and a secondrestartmerges into the pending one rather than queueing behind it. Nothing is sent to the control boundary, because there is no process for it to act on.reloadandresetare invalid: there is no process to reload, and no terminal state to clear.
10.3.2 The Skipped column #
start and restart clear Skipped before they run. A Skipped service
is not in a state a start can proceed from — the state machine permits
Skipped -> Inactive and nothing else — so the activation performs that
transition first, then re-evaluates the conditions from scratch.
Both
outcomes are possible: the precondition that was missing at boot may now
hold, in which case the service starts; or it may still not, in which
case the service is skipped again, for whatever reason applies now.
The clear is reported like any other transition, so a console watching
the service sees it leave Skipped rather than appearing to jump:
peinit: service X left skipped: ExplicitStart, before the start's own
lines.
reset also clears Skipped, and differs only in stopping there.
10.3.3 The Abandoned column #
Every lifecycle command is invalid on an Abandoned service except
reset, which clears it (§6.2). Nothing else is meaningful while
processes that ignored SIGKILL are still in the cgroup.
10.3.4 Definition-removed services #
Independently of state, a service whose definition has been removed
(§3.8) rejects start, restart and reload with UNKNOWN_SERVICE,
accepts stop, and reports its state on status.
The accepted stop drains the instance with the cached StopTimeout,
and the exit that ends it also discards the entry. A caller waiting on
that stop is answered after the discard, with the state the service
last had — Inactive, cause ExplicitStop — rather than with a lookup
of an entry that no longer exists. A status sent after that is
UNKNOWN_SERVICE, as for any service peinit does not know.
10.4 reload-config
Peios / Peios technical reference / peinit / The Control Interface
reload-config takes a fresh snapshot of the configuration from the
registry. It does not live-update anything.
It is also the path a registry change notification takes: any drained watch event triggers the same full reload, rather than a targeted re-read of whatever changed.
10.4.1 Atomicity #
peinit reads everything first. Every registry read happens before any mutation, so a read failure returns an error with nothing touched. It then builds and validates a complete new graph in memory, and only swaps it in if validation succeeds.
If validation fails, the previous generation stays live and the findings are returned to the caller. This is where the reload path differs sharply from boot: boot marks individual services Failed and carries on, because it has to produce a running system; reload rejects the whole thing, because it has a running system already and a half-applied configuration would be worse than the one in place.
An explicit reload-config writes one peinit.config.reload.applied
event either way: outcome.success, and on success the counts of what
it added, updated, restored, marked removed, discarded and found
undecodable; on failure why, in words, and one
peinit.graph.validation.failed per finding under phase
reload-config (§8.4). A reload a registry watch starts writes its
findings and warnings but no peinit.config.reload.applied.
Decoding does not split: reload decodes per key exactly as boot does. A
key that will not decode fails that one service with ValidationError —
a placeholder entry, marked definition-removed, that status reports
and nothing can start — and every other definition in the batch loads; a
dependent of the failed key fails through ordinary propagation when it
is next started. A service that is running when its key stops decoding
keeps running with its definition marked removed, as if the key had been
deleted. The reload succeeds, lists the keys in summary.undecodable,
and names each key, the offending field and the problem in
undecodable; each is also a peinit.graph.validation.failed event
with outcome.reason validation-error under phase reload-config. A
key whose name is not a service name (§3.1) is
undecodable in the same way, reported under its raw name with field: name. Repairing the key restores the service on the next reload.
10.4.2 What changes #
- Every service definition is re-read.
- A new dependency graph is built and validated.
- Running services are unaffected and continue on their activation generation.
- New definitions take effect at the next start, restart, or trigger.
- New services become available for
startimmediately. - Timer triggers are re-evaluated. Unchanged calendar timers retain their armed occurrence and jittered deadline (§9.3); new or changed timers are armed from the current time with no catch-up.
A reload also refreshes things that are not service definitions: the control descriptor, the four control socket limits, the log configuration, the shutdown settings, the global environment layer, and the eventd log socket path. It also prunes the fd stores of services that no longer exist.
10.4.3 Removals and the compiled-in service #
A definition that has disappeared is handled by §3.8 — discarded if nothing is running, marked definition-removed if something is.
registryd is exempt. Its compiled-in definition survives a reload that does not mention it, and its provenance survives a registry entry that shadows it. peinit started registryd before the registry existed, and a reload finding no definition for it cannot conclude that it should stop being managed.
10.5 The Notification Socket
Peios / Peios technical reference / peinit / The Control Interface
peinit binds one Unix datagram socket for service notifications, by
default at /run/services/peinit/notify.sock. Its path is an
implementation detail: services receive it through NOTIFY_SOCKET and
nothing hardcodes it.
The kernel command line can override it with peios.notifysocket=.
peinit creates the socket's directory only when it lies under /run;
an overriding path anywhere else must have a parent that already
exists. /run/services/peinit itself is created whichever path is in
use.
There is one socket, not one per service, and the bind unlinks any stale path first.
10.5.1 The descriptor #
O:SYG:SYD:(A;;GA;;;SY)(A;;FW;;;S-1-5-6)
S-1-5-6 is the Service group, which every token minted for a service
logon carries.
It is written out rather than as its SU alias, and that is deliberate
rather than an oversight. The alias table lives in libpeios, which is a
separately versioned package that nothing here declares a minimum
version of — so an image pairing a peinit that writes SU with a
libpeios that predates the alias parses this descriptor, fails, and
takes PID 1 into recovery before a console exists to say why. That is
not hypothetical: it is what happened the first time this descriptor was
written. A literal SID has been understood by every version there has
ever been. That is the right grantee because it is the
population with something to say here: membership follows from having
been started as a service rather than from which account a process runs
under, so an ordinary user process cannot acquire it however it was
launched.
FW is the file-write generic right, which is what a write to a
pathname socket needs — the same reasoning as the jobs socket (§10.7).
Administrators are absent, unlike the control socket: an administrator
has no business asserting that a service is ready.
The descriptor is installed by path, immediately after bind —
before the socket is made non-blocking, given SO_PASSCRED, or polled.
It is by path because the fd form does not work on a socket. KACS
exposes no fd-targeted descriptor call of its own: fd_set_sd is the
path call with the target as dirfd, an empty path and AT_EMPTY_PATH,
and the kernel answers EOPNOTSUPP for a socket. Stamping by fd was
tried here and took PID 1 into recovery.
That leaves a window between bind and the stamp in which the socket
carries what it inherited. It is safe for the same reason it is on the
control socket: the parent grants services traverse and nothing else and
carries no inheritable ACE, so the inherited descriptor is narrower than
the one being installed. The failure direction is refusal, not
exposure.
The parent directory /run/services/peinit carries a descriptor of its
own, shared with the control socket that lands in it later in boot:
SYSTEM and Administrators in full, and S-1-5-6 traverse only.
One constant
serves both because ensure_directory re-stamps a directory that
already exists — two call sites installing different descriptors on one
path would silently leave whichever ran last, and since the notify
socket is bound first, that would be the control socket's, quietly
removing the traverse services need to reach notify.sock at all.
The protocol itself is PSPU §4. What follows is how peinit decides whether to believe a datagram.
10.5.2 Authenticating a sender #
SO_PASSCRED is enabled on the socket, so every datagram arrives with a
kernel-attested SCM_CREDENTIALS control message. A datagram without one
is rejected outright.
Descriptors for the fd store arrive alongside, as SCM_RIGHTS.
Authentication then runs five steps, and each closes a hole the previous one leaves:
- Find the sender. Scan for the current service-main job whose
PID equals the sender's. Only a main job is ever a candidate, which
is what makes
NotifyAccess=Mainthe only mode there is — a hook or a health check cannot notify on a service's behalf. If no service's main job carries the PID, the live submitted jobs (§8.5) are scanned instead. The routing question is asked purely by PID, before either door verifies anything, so that a sender is verified against its pidfd exactly once whichever way it is routed. - The job is Running. A job still in pending setup has not exec'd.
- The job has a pidfd.
- The pidfd still refers to that PID. This is the step that matters: PID matching alone is racy, because a PID can be recycled between the sender writing and peinit reading. The pidfd was obtained atomically at fork, so verifying the PID against it is what makes the match sound rather than probable.
- The generation matches. A job whose activation generation is
not the service's current one is a previous incarnation, and its
notifications are rejected.
This is
invariant 5 of §6.1 in force: a
READY=1from the process that just crashed cannot mark its replacement ready. A submitted job has no generation and no replacement, so for it the check stops at step 4.
Anything that fails is dropped and recorded as a peinit.notify.rejected
event carrying the sender's PID (subject.process.pid), the sender's
service, job and activation when peinit could attribute it, and the
reason as one of a closed set of words in outcome.reason —
unauthenticated-sender, generation-mismatch, malformed-line and so
on (§8.4).
The UID and GID in the credentials are parsed and never used. They are not policy inputs, and peinit does not consult them for anything — identity on Peios is a token, and the token here is established by which job the sender is, not by what UID it claims.
10.5.3 Applying a datagram #
A datagram may carry several newline-separated lines, and peinit applies every one. Parsing happens before application and is all-or-nothing: if any line is malformed, nothing from that datagram is applied, and any descriptors it carried are dropped and closed. Partial application of an ambiguous service-control message is structurally impossible rather than merely avoided.
A rejection is recorded after authentication, so the event can name the service where one could be established.
10.5.4 What the fields do #
Most are handled elsewhere: READY=1 and RELOADING=1 in §6.5,
WATCHDOG=1 and WATCHDOG_USEC and EXTEND_TIMEOUT_USEC in §6.6,
STOPPING=1 in §12.2, and the fd store fields in §10.6. What each does
to a submitted job — and which are ignored for one, since a job has no
reload, watchdog or store — is in §8.5.
Six are event-emitting. STATUS=, ERRNO= and EXIT_STATUS= are
authenticated and then emitted as KMES events —
peinit.notify.status.reported, peinit.notify.errno.reported,
peinit.notify.exit-status.reported — whose payloads carry the sender
as subject.service.name, subject.job.guid,
subject.job.activation-generation and subject.operation.guid,
alongside the value: notify.status; notify.errno, the number the
service sent, negated as every errno field is; notify.exit-status. A
value that is not a number is left out and the event still written.
They take the same
path as job and operation events, not a forward to eventd.
peinit.notify.status.reported is a verbose type, written only when
the emission policy turns it on (§8.4).
STOPPING=1 emits peinit.notify.stopping.reported, carrying the same
attribution and no value. It is there because the
field's only effect is the absence
of an action — peinit suppresses the SIGTERM (§12.2) — and an absence
cannot be inferred from what happened afterwards. Without the event, a
service that was stopping and correctly received no SIGTERM looks
identical to one that should have received it and did not.
PROGRESS= and PROGRESS_UNIT= emit peinit.notify.progress.reported
(a verbose type), carrying the same attribution and the progress as
retained after the datagram — notify.progress.current, .total,
.bounded and .unit, the fields of a submitted job's
peinit.job.status.reported. A
datagram carrying either yields one event, not one per line, and at
most one a second for each activation: a datagram inside the second
updates what a status query reports and emits nothing, so a service
updating a thousand times a second produces one event a second.
READY=1 and RELOADING=1 emit nothing, deliberately: both are
observable through the state transitions they cause.
STATUS= is additionally stored on the service's runtime state and
exposed as status_text in a status query.
It is cleared to null at the
start of every activation generation, in the same step that increments
the generation, so a status string cannot survive a restart and describe
a process that no longer exists.
PROGRESS= and PROGRESS_UNIT= are stored the same way and exposed
together as progress in a status query, in the form a submitted job's
view gives it: {current, total, bounded, unit}, or null until the
service sends a PROGRESS=.
Each is parsed as for a submitted job (§8.5): a value outside
PROGRESS='s three forms, a T of zero, an N above T, or a unit
other than bytes, items or percent is dropped, never repaired, and
the rest of the datagram is applied. Each is replaced only by a datagram
that carries it. Both are cleared with status_text, in the same step.
ERRNO= and EXIT_STATUS= are not stored. They are emitted and
otherwise not retained.
10.5.5 Bounds #
A datagram is read into a fixed 64 KiB buffer, and the control message
buffer is sized for 64 descriptors. Both MSG_TRUNC and MSG_CTRUNC
are inspected on the way in, and a datagram that overran either buffer
is refused as truncated rather than acted on in part: a larger datagram
is not applied from its surviving prefix, and a send carrying more than
sixty-four descriptors is not applied from the ones that fitted.
10.6 The Fd Store
Peios / Peios technical reference / peinit / The Control Interface
The fd store lets a service keep file descriptors across its own restart. It pushes them to peinit, peinit holds them, and the new process gets them back. That is what lets a stateful daemon — a web server holding a listening socket, say — restart without dropping connections it has already accepted.
FdStoreMax in the definition sets the maximum number of descriptors
peinit will hold. It defaults to 0, which disables the store
: most
services do not need it, and holding descriptors on behalf of a service
that will never ask for them back is pure cost.
10.6.1 Storing #
When an authenticated datagram carries FDSTORE=1 with descriptors
attached:
- If
FdStoreMaxis 0, peinit logs the rejection and closes the descriptor. - If the store already holds
FdStoreMaxentries, peinit logs the rejection and closes the descriptor. The existing store is not modified — a full store does not evict. FDNAME=<name>names it; an absent or empty name meansstored.FDPOLL=0marks the descriptor exempt from poll monitoring.
Either rejection emits a peinit.fd-store.rejected event carrying the
service, the descriptor's name and the reason (disabled or full), so
a service whose descriptors are being silently dropped can find out why.
Several descriptors may share a name. One FDSTORE=1 carrying N
descriptors creates N entries under the one name, each independently
subject to the limit — so the first few fit and the overflow is
rejected and closed.
peinit does not monitor stored descriptors. The FDPOLL flag is
recorded and nothing reads it, so no stored descriptor is evicted for
becoming invalid.
10.6.2 Removing #
FDSTOREREMOVE=1 with FDNAME=<name> removes every descriptor of that
name and closes them. A name matching nothing is a no-op rather than an
error.
FDSTOREREMOVE=1 without a name aborts the whole fd-store step for
that datagram — so a datagram carrying both an unnamed remove and an
FDSTORE=1 performs neither, and the attached descriptors are dropped
and closed.
10.6.3 Injecting #
When a service restarts, peinit injects the stored descriptors during the child's pre-exec path (§5.4):
- They are placed consecutively from
SD_LISTEN_FDS_START— descriptor 3 — upward, with close-on-exec cleared: the only sanctioned exception to the close-on-exec discipline. LISTEN_FDSis set to the count.LISTEN_FDNAMESis set to a colon-separated list of names in the same order as the descriptor numbers.- The store is cleared. peinit no longer holds them.
All three variables are omitted entirely when the store is empty.
LISTEN_PID, which a conforming client checks against its own PID
before trusting LISTEN_FDS, is appended by the child itself after the
clone, since only then is the PID known
; a value for it from
either configurable environment layer is discarded so that it cannot
shadow the real one (§5.5).
A submitted job receives its attached descriptors by exactly this path (§8.5).
A listening socket that comes back this way still conveys the identity
captured when the previous instance called listen() on it. peinit
does not touch that :
a service that wants connecting clients to see the
instance actually accepting restamps the returned listeners itself
(peios_socket_restamp, the Peios Kernel TRM §3.5) before it starts
accepting.
Injection happens for the main process only. Hooks and health checks never receive stored descriptors.
The store is cleared on a successful injection, at the top of the started-launch handling. A launch that fails does not clear it, so the descriptors survive a failed attempt and are available to the next one.
10.6.4 Clearing #
The store is cleared, and its descriptors closed, when:
- The service is stopped explicitly — by an administrator, or by shutdown. The distinction peinit draws is the operation's type and source together: an administrator's stop clears, and a restart-policy-sourced stop does not. The service is not coming back from an explicit stop, so the descriptors are no longer useful.
- The definition is removed and its entry finally discarded (§3.8), immediately if nothing was running and on the instance's exit otherwise.
It survives an automatic restart — crash, restart policy, new start — which is the entire point. The descriptors persist through exactly the restart the service did not choose and cannot prepare for.
10.7 The Jobs Socket
Peios / Peios technical reference / peinit / The Control Interface
peinit serves job submission on a second Unix socket at
/run/services/peinit/jobs.sock, created in Phase 1 infrastructure
setup beside the control socket (§2.3) and existing for the lifetime of
the system. Its wire protocol is PSPU §7; this article is how peinit
implements the transport. What a submission is — identities, the
definition, the job's life — is §8.5.
10.7.1 Why a second socket, and why sequenced packets #
The control socket's descriptor admits the principals allowed to change the system's services. Submitting a job is a different permission held by a different population, and putting the two on one socket would force one descriptor to serve both. Two sockets let the filesystem say who may submit and who may administer, separately, with no policy inside peinit.
The socket is SOCK_SEQPACKET rather than a stream
because a message
on it carries more than bytes: the token the job is to run as, and the
descriptors it is to be given, as ancillary data. The kernel ties
ancillary data to the record it was sent with, and a sequenced-packet
socket makes one send one message and one identity, so peinit never
has to decide which request an attached token or descriptor belongs
to. The cost is that a message has a maximum size and cannot be split;
a definition is small.
10.7.2 Creation and protection #
The listener is created with SOCK_CLOEXEC | SOCK_NONBLOCK and a
backlog of 32; a stale path is unlinked before the bind, and
connections are accepted with accept4 under the same flags.
After
binding, and before anything can connect, peinit stamps the socket
inode :
O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU)
FW is the file-write generic right, which is what a Unix connect()
on a pathname socket needs, so every authenticated principal may
connect and SYSTEM and Administrators may additionally change the
descriptor. Being able to connect is the permission to submit. The
kernel checks this descriptor at connect(), and peinit performs no
access check of its own before a submit
; an administrator who
wants a narrower or wider population changes the descriptor on the
socket, not anything in peinit. What a submitter may then do to a job is
decided by the job's own descriptor (§8.5), never by this one.
A failure to bind or stamp the socket sends peinit to recovery, as the control socket's does: a system whose jobs door is missing is administrable, but one whose door exists with the wrong descriptor is either unreachable or open to everything, and neither is a state to boot into.
10.7.3 Connections #
The listener is one event source of the runtime loop; each accepted
connection is another, keyed by its descriptor. On accept peinit
captures the peer's identity once — the peer token, as the control
socket does (§10.1), and the peer's process handle through
SO_PEERPIDFD — and only then admits the connection against the
limit :
A connection over the limit, or one whose peer cannot be identified,
is closed at the socket level without a response.
The
pidfd is held for the life of the connection, because a submit with
no token attached opens the job identity through it (§8.5).
A connection carries an identity and nothing else. Closing it does not affect any job submitted on it: a job belongs to its submitter's identity, and a submitter that reconnects finds its jobs where it left them.
10.7.4 Messages #
Every message is received with room for one attached token and for 64
descriptors, the output sink included.
One
token is all a message can
carry: the kernel refuses a second KACS_SCM_TOKEN control message at
send time, so the "more than one attached token" refusal in PSPU §7.5
is enforced before peinit ever sees the record. Two truncations are
told apart.
Content the kernel truncated (MSG_TRUNC) is REQUEST_TOO_LARGE, and
that closes the connection, since the transport has lost a record.
Ancillary data the kernel could not fit (MSG_CTRUNC) is
INVALID_ARGUMENTS, with the connection kept: the content is intact,
but a request whose attachments were partly lost does not describe the
job the submitter meant, and peinit does not act on it.
Every
descriptor a message carried that is not handed to a job or adopted as
a sink is closed, on every path.
A response is one compact JSON object, no terminator.
A submit
answered with a running job carries a duplicate of the job's pidfd as
SCM_RIGHTS on the response record; nothing else carries ancillary
data.
Only REQUEST_TOO_LARGE closes the connection after an error. Every
other error is answered and the connection kept.
10.7.5 The turn #
peinit handles one message per readiness turn on a connection, and reads nothing from a connection while a wait is pending on it, so pipelined messages serialise behind a wait. A turn either answers at once, or records a pending wait on the connection state, or does both in the case of a refusal. Three waits exist:
| Wait | Set by | Answered when |
|---|---|---|
| Submit | submit | The job leaves created: exec confirmed, or the launch failed. |
| Wait | wait | The condition holds — terminal, or ready-or-terminal. |
| Stop | stop with wait | The job is terminal. |
Waits are flushed after the supervisor's work has been committed each
turn, and again whenever a job's state moves — a launch, a reap, a
cancelled stop. The flush answers every satisfied wait with the job
view at that moment, and for a Submit whose job is running, the pidfd.
An answer that cannot be built — a job whose record was purged before
the flush, or a pidfd that cannot be duplicated — is that connection's
error record, UNKNOWN_JOB or INTERNAL_ERROR; it never aborts the
flush of every other wait.
A connection with a pending wait, or with a response still queued, is
not idle and is never closed by JobsConnectionTimeout.
A wait has no
timeout of its own; it is bounded by the job, and a submitter that
needs a bounded wait polls status or waits on the pidfd it was
given.
10.7.6 Commands #
submit, status, wait, stop and signal, exactly as PSPU §7.8
defines them and §8.5 implements them. Every one but submit names a
job and is checked against that job's descriptor with the connection's
token; a denial is answered ACCESS_DENIED, and recorded by KACS as
kacs.audit.access.checked naming the job when the job's descriptor
audits it, as a default descriptor does (§8.4).
10.7.7 Idle and shutdown #
Idle connections are closed before and after each wait of the event
loop, and the next idle deadline is folded into the loop's wait
timeout. During shutdown the socket
stays open: submit is refused
with INVALID_STATE, and the other four commands keep answering
(§12.2).
10.7.8 Timestamps #
As on the control socket (§10.1): monotonic stamps projected through the current realtime offset at the moment of answering.
10.8 The Boot Query
Peios / Peios technical reference / peinit / The Control Interface
boot tells a client how the machine booted, so that a settings app or
an administrator at a shell can see that it is in Safe mode, why, and
how many more failed boots stand between it and recovery. The wire shape
is PSPU §4.15. This page is where peinit gets each answer.
It takes no arguments and changes nothing. It is checked against the
control descriptor for SYSTEM_QUERY_STATUS, which the default grants
every authenticated principal (§4.7), and it is answered during
shutdown (§10.2).
10.8.1 Mode and reason #
mode is the mode Phase 2 booted in, after any downgrade: full or
safe (§2.6). A
Recovery boot never reaches the runtime — peinit runs the recovery shell
instead (§2.8) — so there is no control socket to ask, and boot never
answers recovery.
reason | When |
|---|---|
normal | A Full boot. |
requested | A Safe boot asked for with peios.safemode=1. |
safe_mode_downgrade | A Full boot that Phase 2 downgraded to Safe: a Critical service in a dependency cycle, or in an unresolvable conflict, at boot (§2.6). |
For a downgrade, downgrade carries every finding that forced it, in
the words the console line and the peinit.boot.downgraded event use
— all of them, not the first.
peinit keeps them from the boot plan, the one place they survive the
Safe-mode rebuild.
10.8.2 Attempts #
attempts is the boot attempt count (§2.7) as Phase 1 checked it
against the recovery threshold: the value in /.peinit/boot-attempts
before this boot's increment, or 0 if the increment could not be
written, which is how Phase 1 treats a counter it cannot advance.
It is the count
at the start of this boot, so it does not drop to 0 when this boot is
confirmed; confirmed says that.
max_attempts is the threshold: peios.bootattempts=N, or 3 when the
command line does not set it. 0 means the check is off.
10.8.3 Confirmation #
A boot is confirmed when it has counted as a success: every Critical
service has held a dependent-satisfying state for BootSuccessGrace
seconds without a break, and peinit has reset the counter to 0 (§2.5).
grace_seconds is that grace. The other fields say where the boot
stands.
confirmed | waiting_on | confirms_at | confirm_error | Meaning |
|---|---|---|---|---|
| false | the Critical services not holding | null | null | The boot is waiting for them. |
| false | empty | a time | null | Every Critical service holds; if they keep holding, the boot counts then. |
| true | empty | null | null | The boot has counted, and the counter is 0. |
| false | empty | null | why | The grace ended but the counter could not be reset. The next boot will count this one as a failure. |
confirmed is true only once the reset has been written.
A grace that ends
with the write failing — a full disk, say — is not a confirmed boot: the
file still holds this boot's increment, and the next boot will count it,
whatever its services did. The failed reset is not otherwise reported,
so this is where an administrator finds it.
A confirmed boot stays confirmed. A Critical service that fails later is handled by the ordinary Critical failure path — restart budget, then reboot (§2.6) — not by un-counting the boot.
A boot with no Critical services holds from the moment Phase 2 runs, so
confirms_at is known at once.
10.9 Control Client Examples
Peios / Peios technical reference / peinit / The Control Interface
These examples connect the svctl operator guide to the control interface. PSPU §4 owns the wire contract; the socket and dispatch mechanisms are in §10.1 and §10.2.
10.9.1 How the control interface works #
The socket speaks newline-delimited JSON: one request object per line, one response object per line. A request and its success response look like this:
Two properties are worth knowing even if you only ever use svctl:
- Every command is access-controlled. By default, anyone who is signed in may connect. When you connect, peinit captures your token from the kernel and runs AccessCheck against the target service's descriptor for every command. There is no "trust localhost," no override. Who may do what is the subject of Who can manage a service.
- Lifecycle commands create operations. A
start/stop/restart/reload/resetreturns anoperation_id— a GUID you can poll. Conflict resolution between concurrent commands happens at the operation layer, which is why two simultaneousstarts merge instead of colliding.
10.9.2 Reading status #
status returns the full picture for one service:
display_nameanddescriptionare the definition's, ornull.stateandcauseare the lifecycle pair — read them together.status_textis the latestSTATUS=string the service sent via sd_notify (nullif never sent; cleared on each restart).progressis how far the service last said it had got, fromPROGRESS=andPROGRESS_UNIT=:{"current": 3, "total": 10, "bounded": true, "unit": "items"}, withtotalandunitnullwhen it didn't say them.nullif it never sent one; cleared on each restart.current_jobandcurrent_operationare the job and operation GUIDs, ornull.healthishealthy,unhealthy,unknown, ornull(no health check).definition_removedistruewhen the definition was deleted but an instance is still draining.warningslists leaked sub-cgroups and other operator-relevant notices.timershas one entry for each timer trigger, described below.grantedis what you may do to this service, as peinit checks it: the rights the service's permissions give you, out ofquery_status,start,stopandinterrogate(reload). A program shows a control only where its right is listed, rather than offering one that will be refused.
Each entry in timers describes one schedule as peinit has it armed:
scheduled_atis the schedule's next occurrence.fires_atis when the timer will actually fire. It is later thanscheduled_atby the random delayTimerJitterdrew for this firing.last_fired_atis when it last fired, ornullif it has not fired since peinit started and has no recorded run.not_armedis set only for a schedule peinit refused, such as one that never comes round. It gives the reason, and the times are thennull.
Times are in UTC. svctl status prints them to the second:
$ svctl status logrotate
logrotate: inactive
cause: clean_exit
timers:
*-*-* 02:00:00
next: 2026-06-02T02:00:00Z, firing at 2026-06-02T02:07:12Z after jitter
last fired: 2026-06-01T02:03:40Z
list returns a compact summary of every service you can query — services you lack SERVICE_QUERY_STATUS on are simply omitted, not denied:
next_timer_at is the soonest fires_at of the service's timers. svctl list shows it in a NEXT TIMER column.
operation-status returns one operation by GUID; an unknown or expired GUID is the UNKNOWN_OPERATION error. (Operations are dropped after a short retention grace once terminal — long enough for a polling client to read the result, not forever.)
job status returns one submitted job's view under "job", and job list returns the views you can query under "jobs":
The fields are explained in Jobs and operations. A job is retained for 60 seconds after it ends; after that its GUID is the UNKNOWN_JOB error. Like list, job list omits the jobs you lack JOB_QUERY on rather than denying them.
11.1 Wiring
Peios / Peios technical reference / peinit / Output Handling
peinit is not a logging system — eventd stores, indexes and queries logs. But peinit holds the pipes at birth. It decides where a service's output goes, and it has to cover the window before eventd exists.
11.1.1 The pipes #
Before exec, peinit creates a pipe for stdout and one for stderr with
pipe2(O_CLOEXEC). The child's streams are redirected onto the write
ends; peinit keeps the read ends and watches them with epoll.
The blocking discipline is asymmetric, deliberately :
- The parent's read ends are non-blocking. PID 1 cannot afford to block on a read.
- The child's write ends keep ordinary blocking semantics.
That second half is what makes backpressure work. When a service
produces faster than peinit consumes, the kernel pipe buffer fills and
the service's own write() blocks, so the service slows down. Making
the write ends non-blocking would turn that into EAGAIN failures in
the service, converting a flow-control mechanism into an error the
service has to handle.
The child's stdin is redirected to /dev/null.
peinit provides no
interactive input channel; a service needing input obtains it explicitly
— through a socket, or a stored descriptor — never through inherited
stdin.
The epoll instance itself is close-on-exec and is never inherited.
11.1.2 Tagging #
peinit reads output line by line and tags each line with:
- the origin,
- the stream — stdout or stderr,
- a
CLOCK_REALTIMEtimestamp, - the job's identifier.
The origin is the service name for a main process. For a hook it is the
service name, the hook kind and its index — jellyfin/ExecStartPre[0].
Reload commands are <service>/ExecReload and health checks
<service>/HealthCheck. A submitted job, which has no service, is
jobs/<guid>.
Health check output is captured, which is usually the only way to find out why a health check failed.
11.1.3 The output sink #
A submitted job's output is recorded like any other's; that is
unconditional, and eventd is always the primary destination. A
submitter may additionally ask for a copy by attaching a descriptor
with output: true (§8.5). When the job's pipes are registered, the
runtime adopts that descriptor as the job's sink: it is put in
non-blocking mode, and every line read from either pipe is written to
it as the line followed by a newline, untagged, in the order the lines
were read. The two
streams interleave; the sink gets what the job wrote and nothing else.
The copy is best-effort and the record is not. A write that would
block drops the line for the sink only, counts it, and on the first
drop for that job emits one peinit.job.output.dropped event — one per job,
however many lines follow — so a submitter that sees a gap can learn
it caused one.
Any other write failure closes the sink at once. The
sink is otherwise closed when the job's last pipe closes, after the
last line has gone to it, and a sink attached to a job that never got
pipes is closed without a write.
11.1.4 Terminal-attached services #
A service with a TTYPath has all three streams on its terminal, and
both pipe pairs are closed in the child (§5.4). Its output is not
captured at all — it goes to the terminal, which is what asking for one
means.
peinit opens the terminal with read data, write data and read
attributes. The
last matters because the descriptor is inherited by
every process the service starts, and a KACS handle answers fstat
only if that right is on it: without it a session can read and write
its terminal but not name it — ttyname() fails, tty reports "not a
tty", and a program choosing TERM cannot tell a serial line from a
virtual console. isatty() is an ioctl and works either way, which is
why the gap is easy to miss.
11.2 The Pre-Eventd Buffer
Peios / Peios technical reference / peinit / Output Handling
Before eventd starts there is nowhere to send service output: eventd's log socket does not exist until eventd binds it. peinit buffers in memory until then. The buffer is bounded, and drops the oldest entries when it fills.
| Key | Default | Minimum | Meaning |
|---|---|---|---|
Machine\System\Init\PreEventdBuffer | 1048576 | 4096 | Bytes of output retained before eventd exists. |
A value below the minimum keeps the default and logs a warning, rather than being honoured or failing the boot (§11.3). Zero is the case that matters: a zero-capacity buffer rejects every record, so honouring it would silently discard the whole pre-eventd window.
The value is read from the registry at boot and refreshed on reload, and the buffer adopts the new capacity each time. Lowering it takes effect immediately, dropping the oldest entries until the contents fit — the same end of the buffer a steady-state overrun drops.
Dropping the oldest rather than the newest is the right way round for this buffer: the point of the window is the boot that is happening now, and the most recent output is what explains where it got to.
11.2.1 Audit records are not logs #
peinit's own audit records — access denials, critical failures, recovery mode entry, graph errors, security-relevant transitions — are events rather than logs. They go into the KMES kernel ring buffer, which persists from the moment PKM loads: before eventd, before the registry, before Phase 2.
So there is no pre-eventd buffer for them, and no window in which they could be lost. eventd picks them up from the ring buffer when it attaches, wherever in the boot they were emitted.
That distinction is the reason the two paths exist at all. Logs are best-effort and lossy by design; audit events are neither.
11.3 Flood Protection
Peios / Peios technical reference / peinit / Output Handling
A noisy service cannot be allowed to starve the event loop. Three bounds apply.
A value below the minimum is not honoured and does not fail the
boot: the compiled-in default is used and a warning is logged naming the
key, the configured value and what was used instead.
The same rule
covers PreEventdBuffer (§11.2), and it is the rule peinit already
applies to the equivalent kernel command-line knobs — a typo in a
logging knob must not decide how the machine boots, and must not be
silent either.
MaxLogBufferPerService's minimum is the one with an external cause:
F_SETPIPE_SZ will not go below one page and rounds up regardless, so a
smaller number in the registry would describe a pipe that does not
exist. The other minimums are engineering judgement — the point below
which the mechanism stops working rather than working differently.
A line exceeding MaxLogLineLength is truncated and marked
[truncated], with the content trimmed so that content and marker
together come to exactly the limit.
Everything
up to the next newline is then suppressed rather than emitted as a
second line.
MaxLogBufferPerService is applied as the pipe's own capacity through
F_SETPIPE_SZ ,
which is a literal reading of "bytes buffered per
service pipe before backpressure" — the kernel does the buffering and
the bound is where it belongs.
LogReadBytesPerEvent bounds one readable event rather than one loop
iteration. A turn
with several ready pipes reads up to the budget from each, bounded
overall by how many events one epoll wait returns.
11.3.1 Where loss is allowed and where it is not #
At the pipe stage, peinit does not drop. It keeps reading within its
budget and appends every complete line; only EAGAIN, end of file, or
an error stops the read. Backpressure through the pipe is the
flow-control mechanism between a service and peinit, and dropping there
would replace it with silence.
Downstream of the pipe, delivery is loss-tolerant by design. The pre-eventd buffer drops its oldest entries when full, and eventd's datagram socket may drop records under load. The no-silent-drop guarantee covers reading the pipe, not delivery to eventd.
Audit events are exempt from all of it. They go through KMES.
11.3.2 Event loop fairness #
No source may starve the loop. Signals are handled at the highest priority — SIGCHLD reaping and shutdown handling take precedence over everything else in every iteration — with the shutdown deadline timer next and every other source below that, ties broken by arrival order.
The power button shares the top priority with signals , on the reasoning that someone physically pressing it is asking for the same class of attention.
11.4 The eventd Handoff
Peios / Peios technical reference / peinit / Output Handling
When eventd reaches Active, peinit switches from buffering to forwarding.
- Start sending to eventd's log datagram socket, at the path from
Machine\System\eventd\LogSocketPath. - Replay the pre-eventd buffer, oldest first, preserving each line's timestamp and metadata. The replay is best-effort: these are datagrams on a loss-tolerant socket, so some may be dropped, and peinit does not block waiting to deliver them.
- Switch to real-time forwarding — new output sent as it arrives.
- Clear the buffer.
From there peinit is a relay: read from the pipes, tag each line, forward. Audit events continue to flow as KMES events, entirely separately.
11.4.1 Broker authentication #
eventd's log socket denies FILE_WRITE_DATA to the Service logon group
before allowing SYSTEM. peinit's bootstrap token is SYSTEM without that
group; every phase-2 service token carries it, including a token whose
user is SYSTEM. peinit is therefore the only service-output broker.
peinit does not attach a KACS token per log datagram. The socket
descriptor authenticates the broker once at send admission, and peinit
derives each origin from the pipe it is draining. Avoiding ancillary
token fds also preserves mixed-origin batching and the hot-path reuse
described below.
11.4.2 Datagram framing and the record #
peinit sends a msgpack array of one or more records in each datagram. It takes the largest ordered prefix that fits the smaller of the PSPU portable ceiling of 262144 encoded bytes and the log socket's send buffer. It deliberately does not read eventd's larger local ceiling: fixing the producer boundary avoids registry work and means an eventd configuration change cannot invalidate peinit's batches. peinit asks for a send buffer large enough for the portable ceiling when it connects, but the kernel caps what it grants, so the socket's own limit is the one that finally bounds a batch. A successful datagram advances the replay buffer by the whole array, while a failed datagram advances it by nothing. The one exception is a batch the socket refuses as too large: a single record that cannot fit one datagram is discarded rather than replayed forever, and peinit announces the discard on the console.
Each record is a msgpack map :
| Key | Type | Content |
|---|---|---|
origin | string | The service name, or the hook identifier. |
is_error | bool | True for stderr. |
message | string | The line. |
timestamp | uint | Nanoseconds, wall clock. |
job_id | bin | The job's 16-byte GUID in PCDS binary layout. Omitted when absent. |
The map has four or five entries depending on whether a job identifier applies.
PSPU §3.7 requires PCDS GUID layout: Data1, Data2 and Data3 are little-endian; the final eight bytes (Data4) retain their byte order. The payload is still exactly 16 bytes.
11.4.3 Lossy delivery #
eventd's log socket is a non-blocking Unix datagram socket. If eventd
cannot drain it fast enough its SO_RCVBUF fills and the kernel drops
further datagrams silently — log ingestion deliberately exerts no
backpressure on senders.
peinit therefore keeps no unbounded outbound write buffer. It sends bounded arrays of records and accepts that some datagrams may be dropped. It never blocks on a send, and never lets pending records grow without bound.
peinit creates one non-blocking datagram socket when forwarding begins, connects it to eventd, and reuses both that socket and its encoding allocation. It does not create a socket or allocate a fresh output buffer per record. When eventd becomes inactive, the connection is discarded so the next Active transition connects to the newly bound socket.
A drop and a transport failure are handled differently, and the difference is what keeps the lossy design working.
A drop is EAGAIN, EWOULDBLOCK or ENOBUFS — the receive buffer is
full and the datagram did not fit. That is the designed outcome under
load, so the connection stands and forwarding continues.
Live records in
the dropped datagram are gone; peinit counts them and sends the next
batch. Replaying the pre-eventd buffer is the one case where waiting
beats dropping — those records are already held and the buffer is
bounded, so the flush stops and the next turn tries again.
A transport failure is anything else: the socket path gone, the peer refusing. peinit clears the connection, keeps the unsent records in the pre-eventd buffer in order, and the end of the turn re-establishes forwarding by replaying them.
Treating a drop as a transport failure made peinit oscillate between forwarding and buffering under exactly the load the lossy socket exists to absorb, and replay records eventd may already have held — so the busier eventd got, the more work peinit made for it.
11.4.4 When eventd goes away #
peinit supervises eventd like any other service, so it sees the exit directly. It re-enables the pre-eventd buffer, and when eventd restarts and reaches Active the handoff repeats.
There is a log gap between eventd crashing and restarting, bounded by the buffer size. Events are unaffected: they land in the KMES ring buffer regardless of eventd's state, and eventd resumes consuming from the last persisted sequence when it comes back.
11.5 Console Output
Peios / Peios technical reference / peinit / Output Handling
peinit writes its own operational messages to /dev/console
:
- Phase 1 progress — mount results, registryd starting.
- Phase 2 progress — services starting and failing, dependency errors.
- Shutdown progress.
- Recovery mode entry.
- Critical service failures.
Service output is never echoed to the console.
The console is for
peinit's own messages; a service that wants a terminal asks for one with
TTYPath.
11.5.1 The line format #
Every line carries a fixed-width outcome tag, then the component name, then the message:
[ ] peinit: phase1 starting
[ OK ] peinit: service authd started
[ SKIP ] peinit: service resolvd skipped: DependencyFailure
[ WARN ] peinit warning: calendar timer not armed: Malformed
[FAILED] peinit: service netd failed to launch: token materialisation failed
[ CRIT ] peinit: entering recovery: BootAttempts(3)
The tag column is the point of the format. An operator scanning a boot reads down one column rather than reading sentences, and a failure stops looking exactly like the twenty lines around it.
An [ OK ] line reads "mounted", never "mounting": the tag already
claims the outcome, so the verb has to agree with it.
11.5.1.1 Tag and severity are different axes #
The tag is presentation. Severity decides who is
allowed to print. They correlate but neither determines the other — a
Normal service failing and a Critical service failing both render
[FAILED] while carrying different severities — and collapsing them
would quietly change what peios.quiet=2 suppresses.
11.5.1.2 Colour #
Tags are coloured with ANSI SGR escapes: green for OK, yellow for
SKIP and WARN, red for FAILED, and white-on-red for CRIT.
Untagged progress is never coloured.
Colour is decided once, from the kernel command line, following the same
rule systemd uses: on unless the command line says TERM=dumb.
peinit
is PID 1 and has no inherited TERM to consult, so the command line is
the only place an operator can say a console cannot render escapes.
Deliberately not a peios.* token — that set is kept small on purpose,
and TERM=dumb is a spelling people already know.
Colour changes the bytes and never the layout, so a serial log and a virtual terminal agree about where the message starts.
11.5.1.3 The stage banner #
Before Phase 1 does any work, peinit prints a banner naming the stage and the boot mode:
══════════════════════════════════════════════════════════════
peinit · real root · PID 1 · Full boot v0.0.1
══════════════════════════════════════════════════════════════
It is punctuation rather than decoration. prelude prints the matching one when the initramfs takes over, so the two mark the handover from kernel to initramfs to real root — the one thing a boot log otherwise never says out loud.
The mode is the one this boot starts in: Full boot, Safe mode, or
RECOVERY MODE.
A later downgrade to Safe is announced by its own message
rather than by reprinting a banner, because a second banner would read as
a second stage.
Naming the mode is why the kernel command line is read at the very top of
run_init, before any Phase 1 work.
That reverses an earlier ordering
which assumed mounting the virtual filesystems is what makes
/proc/cmdline readable. It is not: the mount step reads
/proc/self/mountinfo first, and /proc is provided by the initramfs. One
consequence — a machine with both an unreadable command line and another
Phase 1 fault now reports the command line as its recovery reason.
The banner carries ordinary status severity, so peios.quiet=2 drops it
along with every other kind of progress.
11.5.1.4 Who else writes to this console #
The format is shared by specification, not by shared code. peinit, prelude and the hook scripts each implement it separately, because prelude is a size-critical initramfs PID 1 with no dependencies and the hooks are shell. Changing a tag word or a width means changing it in all three.
Everything peinit runs that reaches the console is in this format, including output peinit did not write itself. Autorun scripts are captured, not left to inherit peinit's streams, and their lines are relayed under the script's own file name :
[ ] 10-apply-seeds.sh: applied 20 file(s), 92 key(s); 0 failed
[ OK ] peinit: ran 1 autorun script(s)
Relayed lines always carry the blank tag. peinit does not know whether a given line of somebody else's output is good news — the producer knows and peinit does not, and guessing from prose is how a rename silently turns an error green. The producer's outcome is reported separately, from its exit code, which peinit does know.
Relayed lines are sanitised first. A script sharing the console could otherwise steer it with an escape sequence, and peinit's own output shares the device. Three rules apply:
- Control characters become
?. - A line is cut at 512 characters.
- A producer is capped at 200 lines, with the remainder counted.
The only lines on a normal boot that are not in this format come from
outside peinit's reach entirely: the firmware's own messages before any
kernel runs, and a service that owns the terminal through TTYPath
writing to it directly.
11.5.2 Severity and quiet #
Each message carries a severity, and peios.quiet (§2.6) decides what
that means:
- At
0, everything is written. - At
1, the default, peinit stays out of a terminal held as the controlling terminal of a running service, except to announce loss of the system. Terminals are matched by device rather than by path, since/dev/consoleand/dev/ttyS<n>can be the same device; where the device cannot be determined peinit assumes the terminal is held. - At
2, ordinary progress is dropped everywhere while errors still get through.
Suppressed messages are discarded rather than buffered for later.
Shutdown progress carries ordinary status severity, so peios.quiet=2
suppresses it along with every other kind of progress.
The autorun step in Phase 1 (§2.3) bypasses the policy entirely, on the grounds that a script running that early and going wrong is worth interrupting anything for. Its lines are still tagged.
11.5.3 Phase 1 registryd failures #
One deliberate exception to "service output is never echoed to the
console". peinit gives every service capture pipes and drains them in the
runtime loop — but Phase 1 has no loop, so a registryd that printed
exactly why it could not serve was reported to the operator as nothing but
registryd readiness timeout expired before READY=1, with the reason
sitting unread in a pipe.
When Phase 1 registryd fails to become ready, or says READY=1 and then
cannot serve, peinit now drains those pipes and relays what it found
:
[FAILED] peinit: registryd failed; what it said follows
[ ] registryd(stderr): open hive Machine: permission denied
Failure path only. On a normal boot the descriptors are retained for the
runtime, which drains them into the pre-eventd buffer and on to eventd
like any other service's output — so registryd's startup lines are
queryable with evctl and are not on the console.
The drain is safe to run from PID 1 because the read ends are
non-blocking: a registryd that is alive and silent yields EAGAIN rather
than hanging the boot.
11.6 Terminal Ownership
Peios / Peios technical reference / peinit / Output Handling
A TTYPath (§3.2) names a device that has exactly one owner at a time.
Two services writing to the same terminal do not fail; they interleave.
On a console that produces a login prompt with a progress line through
the middle of it, and a password echoed into whatever redrew last. So
peinit hands the device to one service and makes the others wait.
11.6.1 Who is holding it #
A service holds its terminal in every state where it may still have a process on the device:
| Holding | Released |
|---|---|
| Starting, Active, Reloading, Stopping, Backoff | Inactive, Completed, Failed, Skipped, Abandoned |
The two emphasised entries are the ones worth stating outright.
Backoff holds. A service between restart
attempts is coming back,
and handing its console to somebody else during the gap produces two
owners a second later. This is deliberately not the "settled" rule
used by boot:settled (§2.5), which also treats Backoff as still
moving, but reaches it from the opposite direction.
Abandoned releases , and that one is a compromise. The process survived SIGKILL (§5.7) and may well still be writing. But peinit has no remaining way to reclaim the device from it, so treating it as held would mean the terminal is lost for the rest of the boot — and a console nobody can log in on is worse than a console with a stuck process on it.
Nothing is stored. The holder is derived from the service table, so there is no claim to leak, to double-release, or to fall out of step with the state machine.
11.6.2 Losing the terminal #
Every start path — boot, deferred, on-demand, restart, and the
completion leg after a filesystem check helper (§5.2) — asks whether the
terminal is free before it asks anything else. If another service holds
it, the start ends there: the service goes to Skipped with cause
TtyUnavailable (§6.3), its operation completes, and its dependents are
satisfied exactly as any other skip satisfies them.
Skipped rather than Failed, because nothing is wrong. Nothing is written to the console about it either — it is the mechanism working, and the message would land on the very terminal whose new owner is at that moment drawing on it.
The check runs ahead of the service's own conditions and asserts, which is why a service queued on a busy terminal never forks a check helper.
The graph paths — the Phase 2 boot plan, and any start that brings dependencies with it — ask twice. Their pre-start check runs for every service released together before any of them has moved, so two boot-plan services naming one terminal both pass it. The question is then asked again at the moment each one would enter Starting, and by then the first holds the device: the second is skipped there, with the same cause and the same completed operation, rather than launched into a terminal that already has a session.
A relaunch from Backoff asks too, and a service whose terminal was taken while it waited goes from Backoff to Skipped (§6.2).
11.6.3 Winning the terminal #
TTYPrecedence decides who wins when several services want the same
device at the same moment. Higher wins; equal precedence breaks on
service name , so an
operator who states no preference still gets the
same machine on every boot rather than whichever definition the registry
happened to enumerate first. The default is 0.
Precedence is consulted among simultaneous candidates only. A service that already holds a terminal is never preempted by a higher-precedence latecomer : taking a live terminal away from a process that is using it would lose whatever the operator was in the middle of typing.
Two places have simultaneous candidates. Deferred starts (§2.5) are dispatched highest-precedence first, so the winner takes the device and the rest are skipped. Handovers, below, pick one winner outright.
Within the Phase 2 boot plan the order is the dependency graph's, not
precedence's
— a boot-plan terminal service is being written over by
peinit's own progress output anyway, which is what boot:settled
exists to avoid.
11.6.4 Handing it on #
A service that names a terminal and carries the tty:released trigger
(§3.4) is started when that terminal comes free. peinit watches every
transition out of a holding state, so it does not matter whether the
holder exited cleanly, crashed, was stopped by an administrator or was
evicted by a conflict — all four free the device
, and the
waiter is
wanted after a crash more than after a clean exit, not less.
Which terminal was released is recorded on the transition, not looked
up afterwards, and that is load-bearing rather than tidy.
A service whose
definition has been removed loses its table entry the moment it reaches a
state that does not retain one (§3.8) — the same moment it releases its
terminal. Asking afterwards which TTYPath it had gets nothing.
That is not a corner case. It is precisely what a service that retires itself does: remove its own definition so it never runs again, then exit. First-boot setup is exactly that, and before the release rode on the transition it ended by telling the operator they could log in, on a console it had just vacated and nobody had been given.
One winner per release, chosen by precedence. Starting the whole queue would have every loser immediately skipped again by the rule above: the same outcome, reached noisily, with a skipped-looking service for each.
Two services are never offered a terminal:
- The one whose exit freed it.
It has the strongest claim on its own
console, so without this exclusion the highest-precedence holder
restarts itself on every exit — a restart policy with no budget, and
one no waiter behind it ever gets past. Relaunching a service that
stopped is
RestartPolicy's job (§6.4). - One whose definition has been removed (§3.8), since it cannot be started at all.
A waiter is usually sitting in Skipped when its turn comes, from the moment it lost the terminal. The handover starts it as an explicit start would, which is the transition that takes a service out of Skipped (§6.2) and re-evaluates its conditions.
11.6.5 Why not Conflicts, or a dependency #
Conflicts (§7.1) fails both services when they meet in one boot. The
services that want a terminal are the interactive ones — a login prompt,
an installer, a first-boot setup flow — so that answer is a machine with
no way into it.
A dependency says the wrong thing twice. A login prompt does not need first-boot setup to have run; it needs the console to be free, and a setup flow that failed frees it just as well as one that succeeded. And naming the holder would mean rewriting every waiter each time the set of things that might take the console changed.
What is left is a queue on a device, which is what this is.
11.6.6 Fixed system-console endpoints #
When the LCS boundary reads service definitions, /dev/console in TTYPath is
resolved from the last entry in /sys/class/tty/console/active. A tty0 endpoint
is pinned to /dev/tty1; an explicit /dev/tty0 is also pinned to /dev/tty1.
The selected system-console endpoint is cached for the boot so registry reloads
and foreground VT changes do not move existing sessions. Resolution failure
makes the affected definition undecodable, rather than silently opening a moving
alias. Arbitration and process launch use the same resolved path.
On terminal main-process exit the runtime kills the exiting job's cgroup before releasing the terminal or scheduling its successor. Descendants belong to the session that ended, not the next login. Other terminal services and ordinary daemon lifecycle semantics are unaffected. An error killing the cgroup fails terminal-event processing; a new prompt must not be scheduled through that error.
11.7 Service Output Flow
Peios / Peios technical reference / peinit / Output Handling
These notes collect the output-flow explanation behind finding service logs. The preceding sections define the precise transport and limits. Service-output delivery is best-effort; KMES events use a separate finite in-memory ring, and eventd supplies persistence.
11.7.1 Capturing output #
When peinit forks a service it creates pipes for stdout and stderr, redirects the child's streams to the write ends, and keeps the read ends — monitoring them in its event loop. (The child's stdin is /dev/null; see The execution environment.) It reads output line by line and tags each line with:
- the service name,
- the stream (stdout or stderr),
- a timestamp,
- the job GUID.
The job GUID is the important one: it is what lets a later query return exactly one execution's output, never interleaved with the run before or after it. Hook output is captured too, labelled with the hook (e.g. jellyfin/ExecStartPre[0]), and so is health-check output — invaluable for diagnosing why a check failed. A submitted job's output is captured the same way, labelled jobs/<id>; its submitter cannot opt out of that, but can ask for a copy by attaching an output sink, to which peinit writes each line as it reads it. The copy is best-effort — lines the sink cannot take are dropped for the sink only, while forwarding to eventd continues independently, and reported once per job as an output.dropped event — and the sink never slows the job or peinit.
11.7.2 The pre-eventd buffer #
Before eventd is running there is nowhere to send logs — its socket does not exist yet. peinit therefore buffers captured output in a fixed-size in-memory pre-eventd buffer (default 1 MB); when it fills, the oldest entries are dropped.
In practice the only services that run before eventd are registryd, lpsd, authd, and eudev. The first three are Peios-owned with controlled output; eudev is the real overrun risk (verbose device enumeration), and the restart budget naturally bounds output from a crash loop.
11.7.3 Flood protection #
A noisy service must never starve peinit's event loop. Several limits enforce that:
Key (Machine\System\Init\) | Default | Limits |
|---|---|---|
MaxLogLineLength | 8192 | Bytes per line; longer lines are truncated with a [truncated] marker. |
MaxLogBufferPerService | 65536 | Bytes buffered per service pipe before back-pressure. |
Two mechanisms back these up:
- A per-readable-event budget.
LogReadBytesPerEventbounds the bytes drained from one pipe per readable event. A loop turn with several ready pipes may read up to that budget from each; see §11.3. - Back-pressure, not dropping. If a service writes faster than peinit reads, the kernel pipe buffer fills and the service's own
write()calls block. This is deliberate: the service slows down. While reading the pipe, peinit does not drop output — the no-silent-drop guarantee covers reading the pipe. It is only downstream (the pre-eventd buffer, and eventd's datagram socket) that delivery becomes lossy.
11.7.4 The eventd handoff #
When eventd reaches Active, peinit switches from buffering to forwarding:
- It begins sending to eventd's log datagram socket (path from
Machine\System\eventd\LogSocketPath). - It replays the pre-eventd buffer, oldest first, preserving each line's timestamp and metadata. This replay is best-effort — the records are datagrams on a loss-tolerant socket and some may be dropped under load; peinit never blocks waiting to deliver them.
- It switches to real-time forwarding — new output goes out as it arrives.
- It clears the buffer.
From then on peinit is a pipe relay: read lines, tag them, and forward the largest ordered batch that fits the smaller of the portable 262144-byte ceiling and the log socket’s send buffer. peinit does not read eventd’s larger local ceiling; see §11.4. It keeps one connected socket and reuses its encoding buffer, so normal forwarding does not create a socket or output allocation for every line. The log socket is non-blocking and loss-tolerant — if eventd cannot drain it fast enough the kernel drops further datagrams silently, because log ingestion must never exert back-pressure on the whole system through PID 1. peinit keeps no unbounded outbound buffer and never blocks on a send to eventd.
If eventd itself crashes after starting, peinit (which supervises it like any service) notices, re-enables the pre-eventd buffer, and repeats the handoff when eventd comes back. There can be a log gap across the restart, bounded by the output buffer. Structured events use the independent KMES ring; eventd can resume reading the events that remain there. The ring can overwrite old events and does not survive reboot, so eventd supplies persistence rather than KMES. See KMES failure modes.
12.1 Triggers
Peios / Peios technical reference / peinit / Shutdown
Four paths initiate a shutdown.
12.1.1 The control socket #
A shutdown command naming a type, gated on SYSTEM_SHUTDOWN against
peinit's control descriptor (§4.7):
| Type | Effect |
|---|---|
poweroff | Stop everything, unmount, power off. |
reboot | Stop everything, unmount, reboot. |
halt | Stop everything, unmount, halt — the CPU stops, the system stays powered. |
12.1.2 Signals #
| Signal | Meaning |
|---|---|
| SIGINT | Reboot. The kernel sends it on Ctrl+Alt+Del. |
| SIGTERM | Poweroff. PID 1 cannot be killed by it but may choose to act on it. |
| SIGPWR | Poweroff. The compatibility path for environments that surface power failure or a power-button policy as a signal. |
Three SIGINTs within five seconds force an immediate shutdown: no graceful stop, no ordering, SIGKILL every service cgroup, sync, reboot. The window is a sliding five seconds and the press is recorded before the already-shutting-down check, so three presses still force even after a graceful reboot has begun. That is the point — someone pressing it three times has decided the graceful path is not working.
12.1.3 The power button #
An EV_KEY / KEY_POWER press from a readable /dev/input/event*
device is a graceful poweroff.
Only a press —
value 1 — initiates. Releases, key repeats, other keys and other event
types are ignored.
The path is fail-soft throughout: a missing /dev/input, a device that
cannot be opened or registered, and a registered descriptor that later
fails to read are all survivable, and a failing descriptor is removed
from the event loop so repeated failures cannot spin PID 1.
Losing it
degrades only direct power-button handling; the socket and signal paths
remain.
It is deliberately minimal. It is not a power-management policy engine and does not replace a future daemon that would translate richer policy into control socket commands.
12.1.4 Critical service failure #
A Critical service entering Failed with its restart budget exhausted means peinit syncs the filesystems and reboots immediately. This is not a graceful shutdown: there is no stop ordering, no seed save, and no unmount. The system is in an undefined state and the fastest path to a defined one is a reboot.
12.2 The Graceful Sequence
Peios / Peios technical reference / peinit / Shutdown
12.2.1 Step 1: Enter the shutdown state #
peinit sets an internal flag. While it is set, no new service starts.
Control commands
other than status, list, operation-status, job-status,
job-list and job-stop are rejected as invalid for the current
state.
Every live submitted job (§8.5) is stopped at this point too: SIGTERM
to each — unless it has already sent STOPPING=1 — with cause
shutdown, its own stop_timeout to the kill, and no ordering between
jobs or against the service waves, because a job has no dependencies
to order by.
A job still queued for launch is cancelled with the same
cause.
submit is refused for the rest of the shutdown; status,
wait, stop and signal keep answering, as do the control socket's
job-status, job-list and job-stop.
Timer triggers are disarmed. The timerfds stay armed and keep firing — peinit gates the handler rather than the fd, matching how every other event source is handled — but a firing during the shutdown window is classified as a no-op and nothing is started or queued.
The gate matters more than the rule reads. The states a timer firing would start from are Inactive, Completed and Failed, precisely the states step 3 classifies as not participating, so a firing would create a service after the plan was fixed: in no wave, not waited for, and not reached by the global-timeout sweep, still holding files open when peinit reaches the unmount step.
12.2.2 Step 2: Suspend Critical failure semantics #
A Critical service failing during shutdown is recorded but does not trigger a reboot. The system is already going down, and rebooting from here would loop.
12.2.3 Step 3: Classify #
Completed services — Oneshots with RemainAfterExit — have no process
and are transitioned to Inactive, releasing the dependency
relationships they were holding so their dependents can be stopped
cleanly.
The rest are classified for stop eligibility:
- Active and Reloading are graceful-stop eligible and enter the waves.
- Stopping services are already on a stop path. They join the waves for ordering and timeout purposes, but do not receive another SIGTERM.
- Starting services are not eligible. peinit cancels the startup,
SIGKILLs the service cgroup if one exists, and transitions them to
Failed with cause
ShutdownWavewhile post-kill verification is pending. A cgroup still populated after the post-kill timeout takes the service to Abandoned with causeProcessUnkillableand the cgroup is leaked. A Starting service whose job never forked skips the check and goes straight to Failed. - Inactive, Failed, Skipped, Backoff and Abandoned do not participate. Abandoned cgroups stay leaked and shutdown continues.
12.2.4 Step 4: Stop in reverse dependency order #
peinit builds the reverse dependency graph over hard dependencies and stops in waves:
- Each eligible service receives SIGTERM. Already-stopping ones do not.
- Each has
StopTimeoutto exit. - On expiry, SIGKILL to the service's entire cgroup.
- No service is stopped until everything depending on it has stopped.
The compiled-in Phase 1 service, registryd, stops in a wave of its own after every other participant, though no definition declares a dependency on it. Every service defined in the registry rests on it, as it started before any of them; stopped by the graph alone it would land in the first wave and go while the services still running needed it. The rule follows the definition's compiled-in provenance, not its name.
A participant the service table no longer has counts as stopped. A service whose definition is withdrawn while it runs is kept until it stops and then discarded (first-boot setup withdraws its own service this way), and the plan fixed at step 3 still names it. Waiting for it would hold its wave open for good.
A service that sent STOPPING=1 does not receive a SIGTERM at all: it
has already said it is shutting down, and peinit goes straight to the
stop deadline.
The notification is acknowledged as a peinit.notify.stopping.reported
event (§10.5), which is what makes the suppressed SIGTERM legible
afterwards rather than looking like one peinit failed to send.
12.2.4.1 Observing the waves #
Every stop-wave participant receives its own Shutdown-sourced Stop
record when the shutdown plan is fixed. Its requested event records the
intent; its started event records when the participant's wave becomes
eligible, including when an existing stop is adopted without another
SIGTERM. The records and their identifiers are visible through the
ordinary operation queries and peinit.operation.* events.
A service that exits cleanly before its wave completes its Pending observation without a started event. A crash before the wave fails it. For a service already stopping, reaching Inactive completes the observation; a stop timeout, global timeout, or forced reboot fails it. Post-kill abandonment does not produce a second terminal result for an observation already failed at the kill. Starting services retain their existing startup-failure records, and Completed services cleared without a process do not receive stop observations.
An administrator Stop or Restart already in flight keeps its identity and its original timing evidence. The shutdown observation is separate; it neither supersedes that record nor queues a second control command.
12.2.4.2 Timing an already-stopping service #
peinit does not reset an already-Stopping service's clock, either when shutdown begins or when its wave becomes eligible. It uses the timing evidence retained from the stop path that put the service in Stopping:
- If a Stop operation, or a Restart executing its stop leg, is in flight, that operation's retained timing governs.
- Otherwise peinit uses service-level evidence, which carries the cause
that initiated the transition —
ExplicitStop,ShutdownWave,ConflictEvictionorBindsToPropagation.
peinit requires the evidence to be present, to belong to a service actually in Stopping, to carry a cause matching the service's current cause, to name one of those four causes, and not to describe a deadline earlier than its own start.
Evidence that is missing, stale or ambiguous fails closed rather
than being guessed at.
A stop that began before the shutdown has already
spent part of its own clock, and inventing a fresh budget here would
hand the service a second full StopTimeout.
Failing closed costs that one participant, not the shutdown. A service whose evidence cannot substantiate a deadline is given no graceful period at all: its deadline is already due, so the timeout scan escalates it to SIGKILL on its wave, and the console says which way the evidence was unusable:
peinit: shutdown cannot substantiate a stop timeout for <service>
(<reason>); killing it without a graceful period
Every other participant keeps its full StopTimeout, and the shutdown
runs to completion.
An operation whose lifetime expires while it is still Pending — a later-wave stop waiting for its dependencies — fails that operation and its waiters. It does not authorise signalling the service before its wave is eligible. Shutdown owns the signalling; the operation object is an observation of it.
12.2.5 Step 5: Global timeout #
| Key | Default | Meaning |
|---|---|---|
Machine\System\Boot\ShutdownTimeout | 90 | Seconds for the whole sequence. |
Machine\System\Boot\PostKillTimeout | 5 | Seconds for a cgroup to drain after SIGKILL. |
On expiry, every remaining participant's cgroup is killed, anything that does not drain within the post-kill timeout is marked Abandoned with its cgroup leaked, and shutdown continues regardless. Live submitted jobs are killed in the same sweep, whatever phase their stop had reached, and abandoned on the same terms.
The global timeout fires once. What its sweep killed is then governed by the post-kill timeout alone: sweeping again would kill the survivors again and push their post-kill checks back each time, so nothing would ever be abandoned and the sequence would never reach step 6.
The console says what the sequence was waiting for when the timeout fired, and, after any later timeout turn that leaves it still waiting, what remains: participants with their state and wave, pending stop and post-kill checks, and live jobs.
peinit: shutdown global timeout expired waiting for lpsd (Active, wave 1), job <id>
peinit: shutdown still waiting for post-kill check of lpsd
Submitted jobs are named on the console as they are stopped, killed and
abandoned: peinit: shutdown stopping job <id>, killing job <id> and
abandoned job <id>.
The sequence does not move to step 6 while a submitted job is live.
A
job's reap advances shutdown progress exactly as a service's does, so a
session that exits promptly on SIGTERM costs the shutdown nothing, and
one that does not costs it stop_timeout and the post-kill grace.
12.2.6 Step 6: Save the random seed #
After every service has stopped and before any unmount, peinit writes a
fresh seed to /var/state/peinit/random-seed for the next boot: 512
bytes from the kernel CSPRNG on this machine, never copied from an
image, protected so that only SYSTEM-equivalent authority can read or
replace it.
The write is crash-conscious: a temporary file on the same filesystem, written, flushed, atomically renamed over the old seed, and the containing directory flushed. A failure is recorded and shutdown continues.
The forced and Critical-reboot paths skip it, along with the unmount step, and go directly to sync and the final action.
12.2.7 Step 7: Unmount #
- Snapshot the mount table first, from
/proc/self/mountinfo. - Attempt to unmount every remaining non-root mount in the namespace — not only what peinit mounted, so the Phase 1 set is covered by construction.
- Process in descending path depth, so children go before parents.
- A mount point already gone is a successful no-op.
- On failure, attempt a read-only remount. If that fails too, record it and continue.
- The root is never unmounted, but is remounted read-only at the end. A failure there is recorded and does not stop step 8.
12.2.8 Step 8: Sync and the final action #
This step is irreversible. Cleanup failures retained from steps 6 and 7 affect diagnostics only and never block it.
sync(). Called on all three paths — graceful, forced and Critical reboot.reboot(2)withRB_POWER_OFF,RB_AUTOBOOTorRB_HALT_SYSTEM.- If
reboot(2)returns, the final action failed. peinit enters a minimal failed-shutdown state, keeps PID 1 alive, records the failure, and retries the same action no more than once a second. It does not restart services and does not enter recovery mode — finalisation has begun and there is nothing to go back to.
12.2.9 Shutdown during boot #
A shutdown requested while Phase 2 is still running takes effect immediately, through the same classification: Starting services are SIGKILLed, services that reached Active are stopped gracefully, and the boot is abandoned.
12.3 Signals
Peios / Peios technical reference / peinit / Shutdown
PID 1 handles every signal through a signalfd. All signals are blocked and read from the event loop. So peinit installs no signal handlers of its own, and nothing peinit writes has to be async-signal-safe.
/proc/1/status does report two caught signals, SIGSEGV and SIGBUS.
They are the Rust runtime's stack-overflow guard, which runs on an
alternate stack and touches nothing peinit owns, so the reasoning above
is unaffected.
12.3.1 Setup #
peinit builds a mask containing every blockable signal in the supported
range — SIGKILL and SIGSTOP are not blockable and are never delivered
through a signalfd — and installs it with
rt_sigprocmask(SIG_BLOCK, ...) before entering the main event
loop. It then creates the
descriptor with signalfd4(-1, mask, ...),
using the same mask, with SFD_CLOEXEC | SFD_NONBLOCK.
If any part of that fails — blocking, creating the descriptor, retaining it, registering it with the event loop — peinit fails closed. There is no fallback to asynchronous handlers, because a PID 1 with handlers installed where it expected a signalfd is a PID 1 whose assumptions about what can interrupt it are wrong.
The mask is inherited across fork, which is why every child resets it before exec (§5.4).
12.3.2 The signals #
| Signal | Behaviour |
|---|---|
| SIGCHLD | Reap children with waitpid. Match them to tracked jobs; also reap orphans belonging to nothing. |
| SIGINT | Reboot. Three within five seconds forces one. |
| SIGTERM | Poweroff. |
| SIGPWR | Poweroff. |
| SIGHUP | Ignored. PID 1 has no controlling terminal. |
| SIGPIPE | Ignored. A broken pipe on the control socket cannot be allowed to kill PID 1. |
Everything else is ignored. The kernel protects PID 1 from fatal signals, so no signal can kill it.
12.3.3 Reaping #
peinit reaps with waitpid(-1, ..., WNOHANG) in a drain loop, and
normalises the wait status before any service or job policy sees it:
- An exited child carries its exact exit code, 0–255.
- A signalled child carries the terminating signal number and whether the core-dump bit was set.
- A stopped or continued status is invalid on this path, because peinit never asks for them. Observing one fails closed rather than being interpreted.
That last rule matters more than it looks. A stopped child reported as an exit would be read as a service that had terminated, and peinit would act on a process that is merely paused.
As PID 1, peinit also reaps processes nobody is tracking — orphans reparented to it — and reports them as untracked rather than trying to attribute them to a service.
12.4 Finalisation
Peios / Peios technical reference / peinit / Shutdown
Three shutdown paths reach the kernel, and they do different amounts of work on the way.
| Path | Stop waves | Seed save | Unmount | Sync | Final action |
|---|---|---|---|---|---|
| Graceful | Yes | Yes | Yes | Yes | Yes |
| Forced — three SIGINTs | No, SIGKILL everything | No | No | Yes | Yes |
| Critical service failure | No | No | No | Yes | Yes |
The two abrupt paths are required to reach sync() and the final kernel
action with minimal additional work, and skipping the seed and the
unmounts is what "minimal" means.
A machine that
is rebooting because
its audit daemon died has nothing to gain from a tidy unmount and
something to lose from the time it takes.
12.4.1 What survives a failure #
Steps 6 and 7 — the seed and the unmounts — retain their failures as evidence rather than acting on them. Every one is recorded, none of them blocks step 8, and none of them enters recovery mode. By this point there is nothing to recover to: services have stopped and the filesystems are on their way down.
12.4.2 If the final action returns #
reboot(2) does not return on success. If it does, the final action
failed, and peinit enters a minimal failed-shutdown state:
- PID 1 stays alive. It has to — PID 1 exiting panics the kernel.
- The failure is recorded.
- The same action is retried, no more than once a second.
- Services are not restarted, and recovery mode is not entered.
There is no way back from here. Finalisation has begun, the services are gone and the filesystems are read-only; the only correct behaviour is to keep trying the one thing that would end it.
RB_HALT_SYSTEM does not return either, so this state is only reachable
for a genuine failure rather than for the halt case.
12.4.3 A note on the mount table #
The unmount step re-reads /proc/self/mountinfo when checking whether a
mount point that returned ENOENT or EINVAL is really gone. Depth
ordering puts the depth-one mounts last, alphabetically — /dev,
/proc, /run, /sys — so /proc is unmounted before /run and
/sys are attempted.
Neither of those answers ENOENT or EINVAL in practice — /sys
unmounts, and /run either unmounts or is busy — so the check never
runs for them. It is a latent hazard rather than a live one: if either
ever did answer that way, the check could not read the file it needs,
and the shutdown would record a cleanup failure and attempt a pointless
read-only remount.
13.1 Trust Boundaries
Peios / Peios technical reference / peinit / Security
peinit sits at two.
13.1.1 Kernel to peinit #
peinit is the first userspace process, and the kernel gives it a SYSTEM
token — S-1-5-18, every privilege. This is the root of trust for all
userspace identity on the system.
peinit does not drop that token and does not authenticate to anything. Its identity is axiomatic: there is no authority above it in userspace that could vouch for it, and the kernel handing it the boot token is the vouching.
13.1.2 peinit to services #
peinit creates service processes with specific identities and reduced privileges. The trust runs one way. peinit trusts the kernel because it has no alternative; services trust peinit because peinit gave them their identity. Services do not trust each other — KACS mediates every access between them, and nothing peinit does creates a relationship between two services beyond the ordering their definitions asked for.
13.1.3 The TCB #
peinit is part of the Trusted Computing Base, alongside the kernel, KACS, LCS, KMES, registryd, authd, lpsd and eventd. A compromise of any of them compromises the system.
That list is not decoration. It is why registryd is exempt from the global environment layer (§5.5) — a component in the TCB cannot be configurable by a mechanism it is itself the enforcement point for — and it is why eventd is Critical, since a TCB whose audit trail can be silently stopped is not one.
13.1.4 Filesystem enforcement #
KACS enforces Security Descriptors on filesystem access. peinit's Phase
1 descriptor seeding (§2.3) exists precisely because of it: a freshly
mounted tmpfs carries no descriptor, and under DENY_MISSING every
inode on it would be unreachable to everything — including peinit —
until something stamps a descriptor it can inherit from.
That enforcement has no bypass. There is no owner exemption, no privilege that overrides a missing descriptor, and no root escape , which is what makes the seeding step fatal on failure rather than advisory.
FACS extends the model further, and until it lands the filesystem layer still relies partly on conventional trust — correct packaging, controlled binary paths — for the objects nothing has stamped.
13.2 peinit's Privileges
Peios / Peios technical reference / peinit / Security
peinit runs as SYSTEM with every privilege for the lifetime of the system. What it actually exercises is narrower.
| Privilege or capability | Used for |
|---|---|
SeCreateTokenPrivilege | Minting SYSTEM tokens for platform services during bootstrap, before authd exists (§4.2). |
SeTcbPrivilege | Requesting tokens from authd on a service's behalf, and installing a primary token on a child whose identity differs from peinit's own — a service's, or a submitted job's. |
SeAuditPrivilege | Writing events: KMES refuses an emit from a token without it enabled. And recording access decisions: an AccessCheck peinit runs writes the kacs.audit.access.checked record its descriptor's SACL asks for only while the caller holds it enabled (§8.4). |
| Token duplication | Turning a job identity into the primary token a submitted job runs as (§8.5): opening the submitter's own primary through its pidfd, or taking the token the kernel attached, and duplicating either to a primary. SYSTEM holds full access on every token's default descriptor, which is what makes the duplicate permitted. |
| Process creation | Fork and exec, inherent to PID 1. |
| cgroup management | Creating and destroying trees under /sys/fs/cgroup/peinit/. |
| Signal delivery | SIGTERM and SIGKILL to managed processes. |
| Mount operations | The Phase 1 virtual filesystems. |
peinit verifies the three privileges at startup, before Phase 1 does anything that needs them, and fails to recovery naming which is missing :
peinit: required privileges are not held: Token("peinit is missing
required privilege(s): SeCreateTokenPrivilege")
Present and enabled — a privilege the token carries but has not
enabled is not usable, and would fail identically to being absent.
SeAuditPrivilege is the exception: held but disabled, peinit enables
it before checking, rather than run with no events and no record of who
was refused what. The boot SYSTEM token PID 1 runs as holds every
privilege enabled, so on a token the kernel made the check passes and
changes nothing.
The check earns its place by what it replaces. A missing
SeCreateTokenPrivilege used to surface as an EPERM from the first
token mint, which is the first service start — registryd, in Phase 1
step 6. So a peinit that could not mint tokens entered recovery
reporting what read as a registryd problem, and nothing anywhere named
the privilege.
SeImpersonatePrivilege is not required and not checked, even
though it appears in the requirement list peinit was specified against.
It is not used. peinit
passes the peer's token descriptor to AccessCheck directly rather than
impersonating the caller and evaluating as them, so the privilege that
would be needed to impersonate is not needed at all. The same holds on
the jobs socket: peinit never impersonates a submitter, and the identity
a job runs as is one the kernel already verified the submitter could
convey.
For non-platform services peinit creates no tokens. It installs the ones authd minted. It mints only for the SYSTEM platform services it starts during bootstrap, before authd is available.
13.3 The Attack Surface
Peios / Peios technical reference / peinit / Security
| Surface | Reachable by | Controls | Protected by |
|---|---|---|---|
| Control socket | Every authenticated principal, by default | Service lifecycle, shutdown, observing and stopping jobs | The peer token and AccessCheck against the target's descriptor for every command; a per-user connection limit |
| Jobs socket | Every authenticated principal, by default | Running a program under supervision as an identity the submitter holds; managing the jobs it submitted | The socket inode's descriptor, which is the whole of the submission permission; the kernel's gating of the attached token; the job's own descriptor for every later command; a per-submitter quota |
| Notification socket | Anything that can connect | Service readiness, watchdog, stored descriptors | The socket inode's descriptor, then PID matching verified through a pidfd, plus the start generation |
| Registry keys | Anything with registry access | Definitions, triggers, configuration | Registry key descriptors, enforced by LCS |
Machine\System\Init\EnvVars\ | Anything with registry access | The environment of every service | That key's descriptor, and nothing else — variable names are not filtered |
| Service cgroups | peinit | Process tracking and clean kill | Ownership of the hierarchy |
| Phase 1 mounts | peinit | Virtual filesystem availability | Hardcoded, no external input |
/lcl/policy/autorun.d | Whatever can write it | Arbitrary code as SYSTEM in early boot | The directory's descriptor |
| Boot attempt counter | Whatever can write /.peinit/ | Recovery mode entry | That file's descriptor |
13.3.1 What the socket descriptors mean in practice #
The control socket admits every authenticated principal (§10.1), and
what each may then do is decided per command by the control descriptor
and each service's own. The ACCESS_DENIED path and the KACS audit
record it produces are therefore reachable by anyone who can log on: a
caller can make as many kacs.audit.access.checked records as it has
refused commands to send, because the default SACLs ask for one per
refusal; a descriptor whose SACL audits less makes fewer. So is peinit's request
parser, which bounds every request by MaxRequestSize before parsing
it. What one caller can hold open is bounded by
MaxControlConnectionsPerUser, so no one caller can fill the
connections everyone shares.
The notification socket is stamped for SYSTEM and for the Service group
S-1-5-6 (§10.5) — every process started as a service, and nothing
else. Administrators are deliberately absent from it: an administrator
has no business asserting that a service is ready, and the two sockets
have different populations however alike their paths look.
A connection to the notification socket from any other principal is
refused at connect(), by the filesystem, before peinit ever obtains a
peer token — which means no audit event records it.
The jobs socket is deliberately wider: FILE_WRITE_DATA for
Authenticated Users, full control for SYSTEM and Administrators
(§10.7). Connecting is the submission permission, and peinit adds no
check of its own behind it. What bounds the population that can reach
it is that descriptor, and what bounds what any one submitter can do
is the quota and the rule that a job runs only as an identity the
kernel verified the submitter held. The ACCESS_DENIED path is
reachable here — it is how a submitter learns it may not touch another
submitter's job — and every such denial is the job's SACL's to record:
under a job's default descriptor, KACS records each as
kacs.audit.access.checked naming the job (§8.4).
These moved together with §4.3, and had to. A service resolved to a
non-SYSTEM identity could not have reached the notification socket to
report that it had started, so making identity real while the socket
stayed SYSTEM-only would have broken every non-SYSTEM service's
readiness protocol at once. What makes the pair work is that the tokens
peinit mints for itself now carry S-1-5-6 as well, so a SYSTEM
service and a LocalService one are on the same footing here.
13.3.2 The autorun directory #
The Phase 1 autorun step (§2.3) executes every file in
/lcl/policy/autorun.d as SYSTEM, before path provisioning and before
any service. It is fail-open by design, so nothing about a script going
wrong stops the boot.
Its only protection is the descriptor on that directory. Anything that can write there executes as SYSTEM at the earliest point in userspace that exists.
13.4 Security Invariants
Peios / Peios technical reference / peinit / Security
Properties peinit does not violate.
1. peinit does not grant privileges it was not asked to grant.
RequiredPrivileges is subtractive. peinit removes privileges and never
adds one — there is no code path that constructs anything but a removal
(§4.5).
2. peinit does not bypass AccessCheck for control operations. Every control command reaches a check against the appropriate descriptor. There is no backdoor, no override flag, and no "trust localhost". Every command on a submitted job, on either socket, is checked against the job's own descriptor — the submitter included, and the job identity excluded unless the descriptor says otherwise (§8.5).
3. peinit does not expose one service's state to another without
access control. list returns only what the caller may query, and
status is checked per service; job-list and job-status are
checked per job (§10.2).
4. No access denial is silent by default. Every check names its
object to KACS in an audit context, and every descriptor peinit builds
itself — the default service descriptor, the control descriptor, a
submitted job's default — carries a SACL auditing every refusal, so a
failed AccessCheck is recorded by KACS as kacs.audit.access.checked
with the caller, the object and the access requested and granted
(§8.4). A descriptor written to the registry or supplied with a job is
honoured as it is, SACL included; whoever may write one is trusted
with that object's audit policy (§4.6). The MAXIMUM_ALLOWED check
that fills a response's granted refuses nothing (§10.2).
5. The control descriptor, the ServiceSecurity descriptors and the per-job descriptors are the only policy inputs for runtime access control. peinit consults no configuration file, no environment variable and no hardcoded principal list. The only inputs to AccessCheck are those descriptors — the first two sourced from the registry with a compiled-in default, the third fixed at submission from the submitter's SID or the SDDL it supplied. Who may submit is not a policy input at all: it is the jobs socket's file descriptor, decided by the filesystem before peinit sees the connection.
6. peinit does not share its SYSTEM token. It opens its own token
query-only, as a template, and never installs it on a child. Even an
Identity=SYSTEM service gets a separately minted token.
7. peinit does not drop its SYSTEM identity. PID 1 runs as SYSTEM for the lifetime of the system. Only the forked child installs a token; peinit never installs one on itself.
8. Identity is deterministic, and the dangerous case is never
implicit. Every service runs with a known identity. SYSTEM has to be
declared explicitly; an absent or empty Identity means LocalService.
The declaration logic upholds the eighth: an empty value resolves to
LocalService, and SYSTEM is reached only by naming it. What a
service actually receives depends on materialisation, and the two can
no longer diverge: the authd client that returned a minted SYSTEM token
for whatever it was asked for is gone, and a token whose user SID is
not the one the declared identity predicts fails the launch rather than
being reported under the name that was asked for (§4.3).
9. A submitted job runs only as an identity the kernel verified its submitter could convey. There is no identity field, no route from a plain descriptor to a job identity, and no token opened through a PID or descriptor the submitter named. The job identity is the connecting process's own primary — reached through the kernel's handle on that process — or the token the kernel attached to the message after running the impersonation gates with the submitter as installer. peinit never re-derives that decision and never refuses a token for who it names (§8.5).
14.1 Losing the Registry
Peios / Peios technical reference / peinit / Failure Modes
peinit depends on the registry once, hard, and then never again in the same way. The difference between those two situations is most of what this section is about.
14.1.1 During Phase 1 #
registryd failing to start, failing readiness, or failing the schema-version probe sends peinit to recovery mode immediately. There is no Phase 2 without a registry and nothing useful to degrade to. The counter is incremented, so a persistently broken registryd burns boot attempts even though every one of them fails the same way.
The recovery shell reached from this failure does not have registryd running, and the offline tools (§2.8) are what an administrator has. A Phase 1 failure earlier than the registryd step is different: recovery starts a registryd of its own there (§2.8).
14.1.2 During Phase 2 #
A registry read that fails or times out during the definition read sends peinit to recovery, for the same reason: there is no graph to boot.
14.1.3 At runtime #
This is where the design pays off. peinit holds a complete in-memory model and does not read the registry during normal supervision, so registryd going away does not stop peinit supervising anything. Services keep running, restarts keep working, the control socket keeps answering, and timers keep firing.
What stops working is anything that needs new configuration: reload-config fails. Change notifications stop arriving, and a timer's last-run timestamp cannot be written — so a persistent timer may produce a spurious catch-up on the next boot.
registryd itself is a Critical service , so its failure takes the ordinary Critical path: restart budget, then sync and reboot. peinit does not have to handle a permanently absent registryd at runtime, because the system reboots first.
14.1.4 A definition that will not decode #
One definition that fails to decode fails that definition (§3.2). At
boot the key is marked Failed with cause ValidationError and every
other service starts normally; on a reload the key is failed the same
way and every other definition loads. One malformed key no longer stops
unrelated definitions in the same batch from taking effect, and a boot
has no previous generation to fall back to anyway, so refusing everything
would mean not booting at all over a single malformed key — which is
what it used to do.
14.2 Losing a Dependency
Peios / Peios technical reference / peinit / Failure Modes
14.2.1 authd #
Without authd, no non-SYSTEM service can obtain a token, so every such
start fails with ParentSetupFailure. Platform services are unaffected,
since they never take that path.
authd is Critical , so its own failure eventually reboots the system rather than leaving it in a state where no user-facing service can start.
14.2.2 eventd #
peinit supervises eventd like anything else, and eventd is Critical. While it is down, peinit re-enables the pre-eventd buffer (§11.2) and keeps collecting output; when eventd returns the handoff repeats.
There is a log gap bounded by the buffer size. There is no event gap — audit events land in the KMES ring buffer regardless of eventd's state, and eventd resumes from the last persisted sequence. A submitter's output sink (§11.1) is fed regardless, since it is written from the pipe read rather than from the forwarder.
14.2.3 A bound dependency #
A service that BindsTo something is stopped when that something stops,
with cause BindsToPropagation, and restarted when it returns, with
cause BindsToRecovery and no charge against the restart budget
(§7.1). This is the one dependency relationship that recovers by itself.
A Requires dependency crashing does not affect a running dependent at
all. The dependent was ordered after it, not coupled to it.
14.2.4 A dependency that never becomes satisfying #
A dependent blocked on a hard dependency waits until its own operation
lifetime expires, and then fails.
The timeout is measured from when the operation was created rather than
from when it started running, so a service queued behind a slow
dependency can exhaust its StartTimeout without ever having attempted
to start
—
which is the honest answer, in that the caller really has been waiting
that long.
14.3 Unkillable Processes
Peios / Peios technical reference / peinit / Failure Modes
A process in uninterruptible kernel sleep does not respond to SIGKILL. Nothing peinit can do will make it exit, and the design consequence is that peinit stops trying rather than blocking on it.
Detection is uniform: after sending the kill, arm a post-kill deadline (5 seconds by default) and check whether the cgroup still reports as populated when it fires.
For a service stop, the post-kill deadline belongs to that stop operation,
including the stop leg of a Restart. When the main exits and the stop ends,
the deadline is cancelled. A retained obsolete deadline is ignored before
probing or removing the cgroup: the same pathname may already belong to the
next instance. A later stop that escalates to SIGKILL gets its own full
PostKillTimeout; it cannot inherit the earlier instance's remaining grace.
Re-recording a deadline for the same stop does not extend it.
What follows depends on which cgroup it was:
| Cgroup | Consequence |
|---|---|
main/ | The service goes to Abandoned with cause ProcessUnkillable. Supervision stops; the cgroup is leaked. |
health/ or hooks/ | The sub-cgroup is orphaned and recorded as a leak. The service carries on normally. |
checks/ | The sub-cgroup is recorded as a leak of type helper, as any other leaked sub-cgroup is (§5.7). |
The distinction is about what the stuck process is holding. A main process holds the service's ports, locks and connections, so a service whose main process cannot be killed cannot be restarted into a working state. A health check or a hook holds nothing, so a stuck one is a nuisance rather than a blocker.
14.3.1 What Abandoned means #
peinit has given up. The service is not restarted, does not satisfy
dependents, and every lifecycle command against it is invalid except
reset (§10.3).
A reset re-checks main/. If it has finally emptied — the I/O that
was hung completed, or the device came back — peinit cleans up the whole
tree and the service returns to Inactive. If it is still populated, the
service returns to Inactive anyway, the cgroup stays leaked, and both the
acknowledgement and the console carry a warning saying so.
14.3.2 When the process finally dies #
Abandoning a service does not close its main job. The process is still there, so there is still something to reap, and peinit keeps the job open against the possibility that it eventually exits — which it often does, once the I/O it was blocked on completes.
That exit arrives for a service already in Abandoned, a state that has stopped expecting a process. peinit records it and does nothing else: no transition, no restart, no operation update. The service stays Abandoned, the stop stays failed , and the console reports the exit:
peinit: service <service> main process exited in state Abandoned; no action taken
The exit is news about the process, not a retroactive success for the
stop that gave up on it. Recovering the service still takes a reset,
which is also what cleans up the cgroup the process was holding.
The same handling covers a late exit in any state that no longer expects a process — see §6.2.
14.3.3 The generational escape #
A leaked cgroup cannot be removed, so the next start would collide with
it. Recording a leak increments the service's cgroup generation, and the
next start builds a fresh tree at %gen<N> (§5.1). Old trees persist
until reboot.
That is what lets a service recover from a leak at all: peinit cannot clean up the old tree, so it stops trying to and uses a new one.
14.3.4 What it actually means #
Every path here has the same underlying cause. Something below the service — a hung mount, a failing controller, a device that stopped answering — is not responding to the kernel, and no amount of restarting the service will change that. The leak record exists to say so, because the alternative is a service that mysteriously will not restart.
14.4 Resource Exhaustion
Peios / Peios technical reference / peinit / Failure Modes
peinit is single-threaded PID 1, so most resource pressure reaches it as a failure at a syscall rather than as slowness.
14.4.1 Descriptors #
peinit holds a descriptor per supervised process (a pidfd), two per
supervised process for output pipes, one per armed timer, one per
control connection, two per jobs connection (the socket and the peer's
pidfd), one per submitted job's output sink, plus the sockets, the
epoll instance and the signalfd.
Control connections are bounded
in all by MaxControlConnections (256), which is set for descriptors
rather than for fairness: what keeps one caller from taking them is
MaxControlConnectionsPerUser (§10.1). A submission also holds its
prepared token and attached descriptors from acceptance until the launch
takes them, which is bounded by MaxJobsPerSubmitter times the
descriptor cap per message.
EMFILE or ENFILE from pipe2 or clone3 fails the start with
ParentSetupFailure — a restart-eligible cause, so a service that
failed because the system was momentarily out of descriptors gets
another go.
The pre-start check helper's result descriptor and pidfd are both unregistered from the event loop and closed, so a service using filesystem conditions costs no descriptors beyond the start itself.
14.4.2 Processes #
EAGAIN from clone3 — the PID limit — is also ParentSetupFailure
and restart-eligible.
The one path that forks outside the launch machinery is the timer
last-run write, one child per firing of a persistent timer (§9.2).
The children _exit as soon as the write returns and peinit matches
each one's exit status back to its write, so a failure is reported
rather than discarded — but the fork itself is real and happens on every
firing.
It is a fork of PID 1, so the child briefly inherits everything peinit holds: every pidfd, the epoll instance, both sockets, every service's pipe read end. It uses none of them and exits immediately. Driving the write through an asynchronous LCS path instead would remove the fork entirely; there is no such path today.
14.4.3 Memory #
ENOMEM from clone3 behaves like the others.
peinit's own memory is bounded by design in the places that could otherwise grow without limit: the pre-eventd buffer has a fixed size and drops its oldest, there is no outbound queue for log delivery, terminal jobs and operations are dropped rather than retained, and neither has a history structure.
Nor do the two structures that once did. A graph execution context and
its operation associations are retired on the maintenance turn after
every member reaches a terminal state (§7.3), so a boot or an on-demand
start costs nothing lasting.
And an OnFailure chain
entry is cleared once its handler has held for a RestartWindow (§6.3),
so a resident handler does not permanently occupy a slot of that
failure's depth budget.
14.4.4 Disk #
A full root filesystem shows up in three places. The boot attempt counter cannot be written, which peinit treats as a counter of zero and continues — a failure to record an attempt is not itself worth escalating. The random seed cannot be saved at shutdown, which is recorded and does not block the shutdown. And registryd cannot write, which is registryd's problem and reaches peinit as a Critical service failing.
14.4.5 The OOM killer #
An ErrorControl=Critical service is marked OOM-immune, with
oom_score_adj at -1000; everything else is left at the default
(§5.4). A Critical service is one whose loss reboots the machine, so
letting the OOM killer pick it would turn memory pressure into a reboot.
peinit itself is PID 1 and the kernel will not choose it.
14.5 Power Loss and Corruption
Peios / Peios technical reference / peinit / Failure Modes
14.5.1 What peinit writes #
Three things, and all three are written to survive an unexpected loss of power:
| What | Where | How |
|---|---|---|
| Boot attempt counter | /.peinit/boot-attempts | A plain integer, rewritten each boot. |
| Local machine ID | /lcl/etc/machine-id | Temporary file, flush, rename. |
| Random seed | /var/state/peinit/random-seed | Temporary file on the same filesystem, flush, atomic rename, directory flush. |
The seed and the machine ID are atomic in the sense that matters: a reader sees either the old value or the new one, never a partial write.
14.5.2 Reading a damaged file #
The three behave differently on finding something they cannot use, and the differences track how bad each situation is.
The counter is the strictest. Absent means zero. But unreadable, empty, non-decimal, carrying trailing data, or overflowing all send peinit to recovery mode. A counter that cannot be trusted cannot escalate, and the whole point of it is escalation — so failing to read it is treated as though it had already escalated.
The machine ID is regenerated. Absent, empty, all zeroes, the wrong length, not hexadecimal, or missing its trailing newline all produce a fresh identifier, recorded as a warning. It is an opaque install identifier, not a security principal, and a new one is a smaller problem than no boot. An unreadable file or an unwritable path is fail-soft too — the boot continues on an ephemeral identifier, recorded as a warning. Only a failure to generate one sends peinit to recovery, because at that point there is no identifier to continue with (§2.3).
The seed is entirely fail-soft. Absent, empty, oversized, or unrestorable all continue the boot. An empty or oversized seed is recorded as an error on the way past (§2.3); none of them stops the boot. A system with no entropy cache still boots; it starts with less entropy, which is the image builder's problem to solve with a hardware or virtio RNG.
14.5.3 The registry #
peinit does not write service state to the registry, apart from timer last-run timestamps. Everything else about a service's runtime state lives in memory and is rebuilt from the definitions on the next boot, which means a power loss cannot leave peinit's own state inconsistent — there is no state on disk to be inconsistent with.
Registry consistency across power loss is loregd's concern, and its recovery paths are what the recovery shell offers (§2.8).
14.5.4 Timers across a loss #
A persistent timer's last-run timestamp is written after the firing and after the start is initiated, not after the service completes (§9.3). A power loss mid-run therefore does not re-trigger on the next boot — the run was attempted, not missed.
A power loss between the firing and the write does re-trigger, which is the right way round: one extra run is better than a silently skipped one.
14.5.5 Shutdown that never finishes #
If power is lost during the graceful sequence, the effect depends on how far it got. Before step 7, the filesystems are still mounted read-write and the next boot behaves like any unclean shutdown. After step 7, the root has been remounted read-only and everything else unmounted, so there is nothing outstanding to lose.
The counter was incremented at the start of the boot that is now ending, and a successful boot resets it during that boot, once the success grace period has been held. So a power loss during the shutdown of a boot that succeeded leaves the counter at zero; one during the shutdown of a boot that never reached success leaves it advanced. Enough of those in a row reach the recovery threshold, which is the intended behaviour: a machine that keeps failing to reach a successful boot is a machine an administrator should be looking at.
Appendix A Registry Key Reference
Peios / Peios technical reference / peinit
Every registry key peinit reads or writes. Semantics are in the sections referenced.
A.1 Service definitions #
| Key | Purpose | Defined in |
|---|---|---|
Machine\System\Services\ | Parent key. Each child key is one service. | §3.2 |
Machine\System\Services\SchemaVersion | Schema version guard. REG_DWORD, currently 1. Created by peinit if absent. | §2.3, §3.2 |
Machine\System\Services\ServiceSecurity | The descriptor inherited by definitions that carry none. REG_BINARY. | §4.6 |
Machine\System\Services\<name> | One service definition. | §3.2 |
Machine\System\Services\<name>\Provides | Roles this service fills, which dependencies may name in place of the service. REG_MULTI_SZ. Read at boot and on reload. | §7.6 |
Machine\System\Services\<name>\LastTimerRun | Last-run timestamp for a single-trigger persistent timer. REG_QWORD, written by peinit. | §9.3 |
Machine\System\Services\<name>\TimerState\ | Per-trigger timestamps for a multi-trigger service. Each value is named by the percent-encoded schedule and holds a REG_QWORD. | §9.3 |
A.2 Boot configuration #
| Key | Type | Default | Purpose | Defined in |
|---|---|---|---|---|
Machine\System\Boot\MaxParallelStarts | dword, > 0 | 10 | Services starting concurrently during boot. Zero, a type mismatch, or a malformed payload is invalid and enters recovery. | §2.5 |
Machine\System\Boot\BootSuccessGrace | dword | 30 | Seconds every Critical service has to hold a dependent-satisfying state before the boot counts as successful. | §2.5 |
Machine\System\Boot\SettleTimeout | dword | 5 | Seconds to wait for the boot set to settle before starting boot:settled services regardless. Zero is legal. | §2.5 |
Machine\System\Boot\ShutdownTimeout | dword | 90 | Seconds for the entire graceful shutdown. | §12.2 |
Machine\System\Boot\PostKillTimeout | dword | 5 | Seconds for a cgroup to drain after SIGKILL before it is treated as stuck. Bounds one service's final stop, where ShutdownTimeout bounds the sequence. | §12.2 |
A.3 Operational parameters #
| Key | Type | Default | Purpose | Defined in |
|---|---|---|---|---|
Machine\System\Init\ControlSecurity | binary | SYSTEM and Administrators, every right; Authenticated Users, SYSTEM_QUERY_STATUS | The descriptor for system-level control operations and the boot query. | §4.7 |
Machine\System\Init\MaxControlConnectionsPerUser | dword | 16 | Concurrent control socket connections one caller's user SID may hold. SYSTEM is exempt. | §10.1 |
Machine\System\Init\MaxControlConnections | dword | 256 | Concurrent control socket connections in all. | §10.1 |
Machine\System\Init\MaxRequestSize | dword | 65536 | Maximum control request size, in bytes. | §10.1 |
Machine\System\Init\ConnectionTimeout | dword | 30 | Seconds before an idle control connection is closed. | §10.1 |
Machine\System\Init\MaxJobsConnections | dword | 64 | Concurrent jobs socket connections. | §10.7 |
Machine\System\Init\MaxJobMessageSize | dword | 32768 | Maximum jobs socket message content, in bytes. | §10.7 |
Machine\System\Init\JobsConnectionTimeout | dword | 30 | Seconds before an idle jobs connection is closed. | §10.7 |
Machine\System\Init\MaxJobsPerSubmitter | dword | 64 | Live submitted jobs one submitter SID may hold. SYSTEM is exempt. | §8.5 |
Machine\System\Init\MaxLogLineLength | dword | 8192 | Bytes per output line before truncation. Minimum 256; below that the default is used and a warning logged. | §11.3 |
Machine\System\Init\MaxLogBufferPerService | dword | 65536 | Pipe capacity per service, applied with F_SETPIPE_SZ. Minimum 4096 (one page, the kernel's own floor). | §11.3 |
Machine\System\Init\LogReadBytesPerEvent | dword | 16384 | Bytes drained from one output pipe per readable event. Minimum 512. | §11.3 |
Machine\System\Init\PreEventdBuffer | dword | 1048576 | Bytes of output retained before eventd is available. Applied at boot and on reload. Minimum 4096. | §11.2 |
Machine\System\Init\EnvVars\ | parent key | empty | Variables injected into every service but registryd. Value name is the variable name; REG_SZ data is the value. Its descriptor is security-critical. | §5.5 |
Machine\System\Init\ProvisionedPaths\ | parent key | empty | Boot-time path provisioning entries. | §2.4 |
Machine\System\Init\ProvisionedPaths\<name>\Kind | string | — | directory or file. Required. | §2.4 |
Machine\System\Init\ProvisionedPaths\<name>\Path | string | — | The absolute path. Required. | §2.4 |
Machine\System\Init\ProvisionedPaths\<name>\Security | binary | built-in | The descriptor to apply. | §2.4 |
Machine\System\Init\ProvisionedPaths\<name>\Required | dword | 0 | If 1, a failure enters recovery before Phase 2. | §2.4 |
A.4 Other subsystems #
| Key | Type | Purpose | Defined in |
|---|---|---|---|
Machine\System\eventd\LogSocketPath | string | Where peinit forwards service output. | §11.4 |
A.5 Watches #
peinit subscribes to Machine\System\Services\ and
Machine\System\Init\ at boot. Any drained event triggers a full
reload-config, which also covers the OVERFLOW case (§3.7).
Appendix B Constants and Paths
Peios / Peios technical reference / peinit
Values compiled into peinit, which no registry key changes.
B.1 Filesystem paths #
| Path | Purpose |
|---|---|
/usr/bin/peinit2 | Where peinit is installed in package storage. |
/bin/peinit2 | The runtime path the kernel init= names. |
/sbin/registryd | The compiled-in registryd image path. |
/var/state/loregd/Machine.hive | The machine hive, passed to registryd. |
/var/state/loregd/Users.hive | The users hive, passed to registryd. |
/.peinit/ | peinit's own state directory on the root filesystem. |
/.peinit/boot-attempts | The boot attempt counter. |
/lcl/etc/machine-id | The local machine identifier. |
/var/state/peinit/random-seed | The persisted entropy seed. |
/lcl/policy/autorun.d | Phase 1 autorun scripts. |
/run/services/peinit/control.sock | The control socket. |
/run/services/peinit/notify.sock | The notification socket, by default. |
/run/services/peinit/jobs.sock | The jobs socket. |
/sys/fs/cgroup/peinit/ | The root of every service cgroup tree. |
/sys/fs/cgroup/peinit/jobs/<guid> | A submitted job's cgroup. |
/dev/rtc, /dev/rtc0 | The hardware clock, in that order of preference. |
/dev/console | Where peinit writes its own messages. |
/bin/recsh, /bin/sh | The recovery shell, in that order of preference. |
B.2 Timeouts and limits #
| Constant | Value | Meaning |
|---|---|---|
| registryd setup timeout | 30 s | Process setup, driven synchronously in Phase 1. |
| registryd readiness timeout | 30 s | Waiting for READY=1 in Phase 1. |
| Reload detection window | 2 s | Waiting for RELOADING=1 after a reload signal. Bounded above by the operation deadline. |
| Restart delay cap | 60 s | The ceiling on exponential backoff. |
| Timeout extension cap | ×4 | The multiple of a phase's base timeout an extension may reach. |
| Operation retention | 60 s | How long a terminal operation is kept before being dropped. |
| Submitted job retention | 60 s | How long a terminal submitted job's entry answers status before being dropped. |
peinit.job.status.reported interval | 1 s | The least spacing between two peinit.job.status.reported events for one job. |
Submitted job stop_timeout | 10 s | The default, per submission. |
Submitted job readiness_timeout | 30 s | The default, per submission. |
| Submitted job argv and environment | 2 MiB | The bound on a definition's arguments and environment together. |
| Boot attempt threshold | 3 | The default, overridden by peios.bootattempts=. |
OnFailure chain depth | 16 | The maximum handler chain from one originating failure. |
| Pre-eventd buffer | 1 MiB | The compiled-in buffer capacity. |
| Notification datagram | 64 KiB | The receive buffer size. |
| Descriptors per datagram | 64 | The control message buffer capacity. |
| Random seed | 512 bytes | Written at shutdown. Restoring accepts 1–4096 bytes. |
| Control listen backlog | 32 | |
| Jobs listen backlog | 32 | |
| Descriptors per jobs message | 64 | Including an output sink. |
| Tokens per jobs message | 1 | |
| First injected descriptor | 3 | Where stored descriptors are placed. |
| Final action retry | 1 s | The minimum interval between reboot(2) retries. |
B.3 Environment #
| Variable | Value |
|---|---|
PATH | /sbin:/bin — the compiled-in base for every service. |
The recovery shell additionally receives TERM=linux and HOME=/.
B.4 Kernel command line #
| Parameter | Effect |
|---|---|
peios.safemode=1 | Force Safe mode. |
peios.recovery=1 | Force recovery mode. |
peios.bootattempts=N | The recovery threshold; 0 disables the check. |
peios.quiet=N | Console verbosity: 0, 1 or 2. |
peios.notifysocket=PATH | Override the notification socket path. |
B.5 Descriptors peinit applies #
| Where | SDDL |
|---|---|
/dev/shm, /run, /sys/fs/cgroup after mounting | O:SYG:SYD:(A;OICI;GA;;;SY)(A;OICI;GA;;;BA)(A;CI;GRGX;;;WD)(A;OICIIO;GA;;;S-1-3-0) |
/dev/null, /dev/zero, /dev/full, /dev/random, /dev/urandom, /dev/tty, /dev/ptmx (DACL only) | D:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FRFW;;;WD) |
A provisioned path with no Security | O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FR;;;BU) |
A service's /run/<name> runtime directory | O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;GA;;;<service SID>) |
| The random seed file | O:SYG:SYD:(A;;GA;;;SY) |
| The default ServiceSecurity | O:SYG:BAD:(A;;0x000F;;;SY)(A;;0x000F;;;BA)(A;;0x0001;;;AU) |
| The default ControlSecurity | O:SYG:BAD:(A;;0x0007;;;SY)(A;;0x0007;;;BA)(A;;0x0004;;;AU) |
| The control socket inode | O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU) |
| The jobs socket inode | O:SYG:SYD:(A;;GA;;;SY)(A;;GA;;;BA)(A;;FW;;;AU) |
A submitted job with no security_descriptor | Owner and group the submitter; (A;;0x0007;;;<submitter>)(A;;0x0007;;;SY)(A;;0x0007;;;BA) |
B.6 Access rights #
| Right | Bit |
|---|---|
SERVICE_QUERY_STATUS | 0x0001 |
SERVICE_START | 0x0002 |
SERVICE_STOP | 0x0004 |
SERVICE_INTERROGATE | 0x0008 |
SERVICE_ALL_ACCESS | 0x000F |
SYSTEM_SHUTDOWN | 0x0001 |
SYSTEM_RELOAD_CONFIG | 0x0002 |
SYSTEM_QUERY_STATUS | 0x0004 |
JOB_QUERY | 0x0001 |
JOB_STOP | 0x0002 |
JOB_SIGNAL | 0x0004 |
JOB_ALL_ACCESS | 0x0007 |
B.7 Identifiers #
Every GUID peinit generates — jobs, operations — is UUIDv7, so identifiers sort by creation time.