Services & jobs

Use Services Manager on the desktop or svctl in a terminal to see what is running and control it. Both talk to peinit, the service manager that runs as PID 1. Start with a status check; change the definition only when the running service needs different configuration.

$ svctl list
$ svctl status jellyfin
$ svctl boot

Replace jellyfin with the service name you want to inspect. list shows only services you have permission to query.

Where to start #

For exact field types and defaults, use the Registry key reference. For implementation and protocol details, use the peinit Technical Reference Manual.

Three kinds of object #

See Jobs and operations to follow a run or submit a one-off job.

peinit in one sentence #

peinit starts services from registry definitions, runs each under its configured identity, supervises it, and stops services in dependency order at shutdown. You use svctl or Services Manager to request those actions.

What peinit is not #

Service definitions live under Machine\System\Services\ in the registry. There are no systemd unit files or Windows SCM commands to edit or translate. Use Defining a service for configuration and Service types for programs that need to stay in the foreground.

Logs and durable history belong to eventd. Open Logs… for a service in Services Manager, or follow Service output and logging. The TRM compatibility notes describe the interface boundaries.

The one operational invariant #

peinit uses a cached service configuration. A saved change may wait for the next start or another relevant operation; it does not necessarily change a running process. Check when configuration changes apply. The event-loop design explains why.

Where peinit sits in the system #

The service’s Identity controls what its process may access. Its ServiceSecurity permissions control who may manage it. Permission to edit its registry definition is separate again. See Service identity and Who can manage a service. The TRM system map covers the components behind these checks.

Use svctl to inspect and control services from a terminal, or Services Manager on the GXWI desktop. Start by checking the service before issuing a command that interrupts it.

$ svctl list
$ svctl status jellyfin

Use the name from the list in place of jellyfin. The command form is svctl <command> [service] [flags]. To investigate a failure, keep the state, cause, job ID, and warnings, then open the service’s logs.

Reading status #

Read state and cause together, then check health and warnings:

$ svctl status jellyfin
$ svctl list

An Active service has met its configured readiness check. With Readiness=Alive, that only confirms the process exists; it does not prove the application is serving requests.

For a scheduled service:

$ svctl status logrotate
logrotate: inactive
cause: clean_exit
timers:
  *-*-* 02:00:00
    next: 2026-06-02T02:00:00Z, firing at 2026-06-02T02:07:12Z after jitter
    last fired: 2026-06-01T02:03:40Z

svctl prints timer times in UTC, to the second. Services Manager shows them in local time. svctl list has a NEXT TIMER column for each service’s soonest firing. Jitter is already included in the actual firing time.

list omits services you cannot query. job list does the same for jobs. An operation ID expires after its terminal retention grace; a submitted job is retained for 60 seconds after it ends. For older results, look in eventd. The TRM client examples retain the full JSON views and field formats.

From the desktop: Services Manager #

On a GXWI desktop, Services Manager does the same from a window. It lists every service, shows what the selected one is doing (its state and why, its process, the account it runs as, how long it has been running, anything it has reported), and offers Start, Stop, Restart, Reload and Reset. The same commands are on each row's right-click menu. Type in Find a service to narrow the list by name or description.

Services Manager is another client of the same control socket, so it has exactly your authority and no more:

• A command is offered only where it will act. A button is available only if you hold the right the command needs (see the table below) and the service is in a state where the command does something (see the command × state matrix). Otherwise the button is unavailable. The details pane says which commands you may use on the selected service.
• Commands do not block the window. A command is sent without waiting. Services Manager follows its operation and reports the result, such as "SSH server was restarted." or the reason it failed.
• Hidden services are listed, not silently dropped. list omits the services you cannot query. If you can read the service definitions in the registry, Services Manager still lists those services, marked Not yours to see, and says how many there are.
• If peinit does not let you connect at all, Services Manager says so. By default the control socket admits everyone who is signed in, but it can be locked down. Any definitions you can read are still listed, and no command is offered.
• Its definition opens in a window of its own, from Edit definition… in the details pane or on a row's menu. New service… on the bar defines one. Both are covered in Defining a service, with svctl definition, which does the same from a terminal.
• Who may control a service, and who may change its definition, open from the details pane in the permissions editor. They are covered in Who can manage a service.
• What it has logged opens in Event Viewer, from Logs… in the details pane or on a row's menu. The window shows the service's lines and those of its hooks and health checks, newest first and as they are written. See Event Viewer.
• Timers show when they run. The Next run column shows when each service's soonest timer fires. The details pane lists each timer under its schedule, with its next run and its last run, in the machine's local time. A timer with jitter says that its run is put back by a random delay, and the next run shown already includes that delay. A timer peinit could not arm says why it never runs. These are peinit's own figures, read from status, not worked out from the definition.

peinit doesn't announce state changes, so Services Manager asks it for the services' state every two seconds. Press F5, or select Refresh, to ask straight away. What the registry holds is different. That covers which services are defined, who may control them, and what you may change. Services Manager watches the Services key for changes and reads it again only when something changes, so a change made elsewhere, from svctl or another window, shows at once.

Before changing a service #

• Start begins the full start sequence and may start dependencies.
• Stop interrupts the service. A BindsTo dependent stops with its target; a Requires dependent that is already running does not. See Dependencies.
• Restart stops and starts as one operation, replacing the current process. Use it when a definition change needs a new process.
• Reload asks the running program to re-read its own configuration. reload-config instead asks peinit to re-read service definitions. Neither promises to apply launch-only settings to an existing process.
• Reset clears Failed, Abandoned, or Skipped to Inactive; it does not start the service or repair the cause.

After the action finishes, run svctl status <service> again. If it failed or only returned an advisory reload result, follow the troubleshooting guide before repeating it.

Service commands #

$ svctl start jellyfin
$ svctl restart jellyfin
$ svctl reload nginx
$ svctl reset failed-migration     # clear a Failed state without starting

Wait semantics #

By default, a lifecycle command blocks until its operation reaches a terminal state — start waits for Active (or Failed), stop waits for Inactive, and so on. Pass --no-wait to get the operation_id back immediately and poll with operation-status instead.

reload is the exception — it returns immediately by default, because a reload may have no observable completion. With --wait, the response carries a mode:

The detection window is a fixed 2 seconds — it is built in and is not configurable via the registry. If the service says nothing within that window, the reload resolves as advisory and the service stays Active.

System commands #

$ svctl shutdown poweroff
$ svctl reload-config
$ svctl operation-status a1b2c3d4-...

Job commands #

Submitted jobs are reachable from two places, and svctl job covers both. Querying and stopping a job is administration and goes over the control socket; submitting a job, waiting on it, and signalling it belong to its submitter and go over the jobs socket at /run/services/peinit/jobs.sock (--jobs-socket overrides the path). Every job command is checked against the job's own descriptor, which by default admits the submitter, SYSTEM, and Administrators.

$ svctl job list --state running
JOB       STATE    SUBMITTER              IDENTITY               PROGRESS   DESCRIPTION
5f2a...   running  S-1-5-21-...-1001      S-1-5-21-...-1001      3/5 items  nightly backup
$ svctl job status 5f2a...
$ svctl job stop 5f2a...
$ svctl job submit --description "nightly backup" --timeout 3600 -- /usr/bin/backup --full
$ svctl --wait job submit --env MODE=full /usr/bin/backup   # exits 0 only if the job completed
$ svctl job wait --for ready 5f2a...
$ svctl job signal 5f2a... SIGHUP

job submit takes the definition fields as options: --description TEXT, --cwd DIR, --env NAME=VALUE (repeatable), --timeout SECS, --stop-timeout SECS, --readiness none|notify, --readiness-timeout SECS, --success-exit-code N (repeatable), --security-descriptor SDDL. Two hand the job something of yours: --fd NAME=FD passes a descriptor of the svctl process to the job under NAME (from descriptor 3, with LISTEN_FDS/LISTEN_FDNAMES set), and --output attaches svctl's standard output as the job's output sink, so the job's lines appear on your terminal as well as in eventd. Everything after IMAGE belongs to the job, options included; use -- before an image path that starts with a dash.

The CLI has no way to attach a token, so a svctl-submitted job always runs as the caller. Running a job as someone else is a programmatic act — a service attaching the token of the client it is impersonating — and is described in Jobs and operations.

The command × state matrix #

A command sent to a service in an unexpected state returns an error, not a silent no-op. This matrix is the authority on what each command does in each state:

Legend:

• MERGE — an operation of this type is already running; your command merges into it and you get its GUID.
• DEFER — your start is accepted and creates a Pending start operation, but it does not run yet: it waits out the service's existing backoff deadline and then starts. If a deferred start is already waiting, you merge into it and get its GUID.
• ALREADY — already in the target state; returns the current status, not an error.
• QUEUE — queued as Pending; runs after the current operation finishes.
• NOOP — no effect; returns the current status.
• Clear / Cancel — clear the state to Inactive / abort the current operation, then proceed.
• ERROR — invalid for this state; returns an error with an explanation.

The Backoff column is the subtle one: the service is down with an automatic restart pending, so start is deferred — it creates a Pending start that honours the remaining backoff delay and only runs once that deadline expires (a second start merges into the one already waiting), stop cancels the pending restart and any deferred start (the service goes Inactive), restart cancels the automatic one and does an admin restart, and reload/reset are invalid because no process exists.

Error codes #

An error response is {"status": "error", "code": "...", "message": "..."}. The code is one of:

The jobs socket uses the same envelope and codes, plus two of its own:

The message is human-readable and non-normative — read it for context, key off the code.

Exit status #

How the control interface works #

Both tools use /run/services/peinit/control.sock. Connection admission and each service action are access-controlled. The default socket admits signed-in users; the target’s permissions decide which commands they may run.

Lifecycle commands create an operation ID you can follow. Concurrent identical requests can merge into one operation. For JSON framing and complete response examples, see the control-client reference.

Connection limits #

peinit enforces hard limits on the control socket, all tunable under Machine\System\Init\:

The jobs socket has its own set, under the same key:

Where to start #

To understand the states the matrix refers to, read The service lifecycle.

To understand the operations every lifecycle command creates and how to poll them, read Jobs and operations.

To configure who may run each command, read Who can manage a service.

In Services Manager, select the service and choose Logs…. It opens Event Viewer with the service’s lines and the output of its hooks and health checks, newest first and updating as they arrive.

For terminal queries, use evctl. peinit forwards output; eventd stores and queries the records it receives.

Find the failed run #

1. Run svctl status <service> and note its state, cause, current job ID, and warnings.
2. Open Logs… and inspect the time around the failure. Hook and health-check output can explain a start or health failure even when the main process wrote nothing.
3. Match the job GUID to isolate one execution. A restart has a new job ID.
4. For earlier attempts, look for peinit.job.started, peinit.job.ended, and the operation events in eventd. The live job or operation may already have expired.

Log origins identify the producer: the service name, a hook such as jellyfin/ExecStartPre[0], or jobs/<id> for a submitted job. A log’s stream tells you stdout or stderr, not whether the line necessarily describes an error.

Capturing output #

peinit captures stdout and stderr, including hooks and health checks, and tags lines with the origin, stream, timestamp, and job GUID. A submitted job cannot opt out of forwarding; --output requests an additional live copy.

Pipe wiring and tagging are described in the output-flow reference.

Logs and events have different limits #

Service logs and structured events take separate paths:

Do not assume an event has survived a reboot merely because it was emitted. The kernel ring is not persistent, and it can overwrite older events if consumers fall behind. See KMES failure modes for those limits and Auditing for event storage and queries.

The pre-eventd buffer #

Before eventd is available, peinit holds output in memory, by default up to PreEventdBuffer=1048576 bytes. Overflow drops the oldest lines. The buffer is replayed when eventd is ready, but it is not persistent storage and cannot guarantee a complete boot log.

The buffer reference explains the early-boot path.

Flood protection #

A [truncated] marker means a line exceeded MaxLogLineLength (8192 bytes by default). Output delivered faster than peinit can read can block the service’s writes at its pipe; downstream delivery to eventd is loss-tolerant.

If a submitted job’s live output has gaps, look for peinit.job.output.dropped. That reports loss in the extra output copy, not necessarily loss in eventd; query eventd separately. Ordinary log delivery remains best-effort there too.

See the registry limits before changing buffer sizes and the TRM flood-control detail for the implementation.

The eventd handoff #

When eventd reaches Active, peinit replays buffered output and forwards new lines. If eventd exits, peinit resumes buffering and repeats the handoff when it returns. Missing log lines around that interval can reflect buffer overflow or lossy delivery; an empty query alone does not prove the service wrote nothing.

Access policy can also hide records from a query. Check who may read what if the expected origin is absent. The handoff reference describes transport behaviour.

Console output #

peinit writes boot progress, shutdown progress, recovery entry, and critical failures to /dev/console. Service output is not echoed there by default. A terminal-attached service is the exception because it writes directly to its terminal.

peios.quiet can suppress peinit’s console messages, and suppressed messages are dropped rather than saved for later. See console noise settings when investigating a boot problem.

Where to start #

For symptom-based next steps, use Troubleshooting. For finding older executions or actions, use Jobs and operations. For query tools, use Event Viewer or evctl.

Collect the current state and cause before changing anything:

$ svctl status <service>
$ svctl list
$ svctl boot

Keep the job or operation ID and any warnings. Open Logs… for the service in Services Manager to inspect its main process, hooks, and health checks. For older attempts, query logs and events; live status is not a permanent history.

Use the symptom below to choose the next check. Avoid repeating a restart or changing permissions until you know which layer failed.

A service won't start #

status shows it Failed or Skipped. The cause says why:

A service keeps restarting #

The service flaps — up, down, up, down. This is restart policy doing its job, but it points at an unstable service.

• It eventually settles in Failed with RestartBudgetExhausted. It crashed RestartMaxRetries times faster than it could stay healthy for RestartWindow. The fix is the service, not the policy — read its logs for the crash. Raising the budget only delays the inevitable.
• It flaps forever and never exhausts the budget. Check whether it stays Active for the full RestartWindow between failures. Reaching Active briefly does not reset the budget. Inspect the health-check output too; a faulty check can restart a working service.
• It restarts on a clean exit you did not expect. RestartPolicy=Always restarts even successful exits (cause CleanExitRestart). If the service is meant to exit, use OnFailure instead.

The machine keeps rebooting #

Preserve the last console error and any service records before attributing repeated reboots to a service. If peinit reports a Critical service exhausting its restart budget, its ErrorControl=Critical policy makes peinit sync and reboot. Repeating that failure can produce a reboot loop; the symptom alone does not establish this cause.

The boot-attempt counter can break a sequence of boots that reach peinit's counter increment but never become confirmed: after N attempts (default 3), peinit enters Recovery mode. If each boot reaches the success grace before the Critical failure, its counter is reset and the reboot loop can continue without automatic Recovery. Check svctl boot while the normal control interface is available; it is unavailable in Recovery. Do not assume every reboot loop will stop after three attempts, or that peinit can deliver Recovery if it cannot start.

In Recovery, start with the failure reason on the console. Phase 2 services are skipped, so stored eventd history is not automatically queryable from that shell. Where a supported running eventd query service is available, use Find service logs and Find missing records to investigate earlier attempts. evctl queries that service; it does not open stored databases offline. Otherwise, retain the console evidence for the administrator or provider's recovery procedure.

Once the evidence identifies a Critical service failure, repair its cause or restore known-good configuration through a supported recovery path before rebooting. Disabling a required platform service can leave its dependents unable to start. If you need to intervene before the counter trips, boot with peios.recovery=1 on the kernel command line.

A service is Abandoned #

Abandoned means peinit SIGKILLed the process but it survived — stuck in uninterruptible kernel sleep (D-state), the signature of hung I/O (a dead NFS mount, a failing disk). Nothing in userspace can kill a D-state process.

• The underlying I/O fault is the real problem — investigate the storage or mount, not peinit.
• The service's cgroup is leaked and shows in warnings. A later start uses a fresh generational cgroup, so the new instance is unaffected.
• svctl reset <service> clears the Abandoned state and re-checks the cgroup: if the stuck process finally died, peinit cleans up; if not, it stays leaked and warns you. Leaked cgroups clear fully only on reboot.

I changed the config and nothing happened #

peinit works from an in-memory snapshot, and each field has a mutability class that decides when a change applies:

• Immutable at runtime (ImagePath, Type, Identity, RequiredPrivileges, ErrorControl) → takes effect only on restart.
• Apply on next start (dependencies, conditions, OnFailure) → next start or graph reload.
• Reloadable at runtime → next relevant operation. Timeouts and health-check policy can affect later checks; environment, arguments, working directory and process limits apply to a new process, not one already running.
• Hot-reloaded (ServiceSecurity) → next control request.

If a change is not landing, check its class first. For a wholesale re-read of every definition, run svctl reload-config — it rebuilds and validates the whole graph atomically and swaps it in only if validation passes (running services are untouched). And remember peinit pulls changes from registry notifications rather than having them pushed, so there can be a brief lag.

ACCESS_DENIED #

A control command returns ACCESS_DENIED. peinit ran AccessCheck on your token against the target's descriptor and the right was not granted:

• A service command needs the matching right in the service's ServiceSecurity descriptor (start needs SERVICE_START, restart needs both stop and start, …).
• shutdown and reload-config need SYSTEM_SHUTDOWN / SYSTEM_RELOAD_CONFIG in peinit's control descriptor.
• The check uses your effective identity at connect time — if you are impersonating, that is what is checked.

Under the default descriptors every denial is recorded by the kernel as a kacs.audit.access.checked event naming the caller, the target (its object kind is service, job or peinit-system) and the rights requested and granted. Walk it through Debugging a denial.

A job submission is refused #

A job submit — from svctl or from a service — comes back with an error rather than a job. The code says which door it hit:

A job that was accepted but never ran is not an error response: it is an ok response whose view is failed with a cause — parent_setup_failure (peinit could not prepare the launch) or pre_exec_failure (the program could not be executed as that identity, which is where "no such file" and "permission denied" land, because peinit does not check the image before accepting a submission).

A job's output has gaps #

If a submitter attached an output sink and sees gaps, look for a peinit.job.output.dropped event for the job in eventd: the sink was not being drained fast enough, and peinit dropped lines for the sink only rather than let it slow the job. Query eventd under the job’s GUID for the independently forwarded record. Sink loss does not itself drop that record, but eventd’s log path is also best-effort, so completeness is not guaranteed.

UNKNOWN_JOB #

The GUID is unknown or its retention has elapsed. A submitted job is retained for 60 seconds after it ends, then dropped; check eventd for any stored record. A known job that you lack permission to query returns ACCESS_DENIED. job list instead omits jobs you cannot query.

A dependent never started #

You started (or booted) a service, but something that depends on it never came up:

• A Requires/BindsTo dependent stays blocked until its target reaches a satisfying state (Active, Reloading, Completed, or Skipped). If the target is stuck in Starting or Failed, so is the dependent — fix the target.
• If a dependent got connection refused on boot, check the target's readiness setting and startup logs. With Readiness=Alive, peinit waited only for the process to exist, not to be serving. Use Readiness=Notify only if the program supports sending READY=1 from its tracked main process when ready; changing the setting does not add that support and can cause ReadinessTimeout. See the notification contract. If the program cannot signal readiness, have its service or package owner resolve the startup race. peinit's boot validation warning can identify an Alive-readiness dependency; it does not prove the cause of a refused connection.
• A Wants dependent waits for the optional target’s startup attempt to resolve, but starts even if the target failed. Use Requires when successful readiness is necessary.

Booted into Safe mode #

Safe mode selects Critical and SafeMode=1 services within the boot-triggered set. Those flags do not give a demand-only service a boot trigger. You land here when graph validation finds a cycle or unresolvable conflict involving a Critical service, or when peios.safemode=1 is on the kernel command line. The TCB is healthy; the configuration is broken.

Fix the graph: the logs name the cycle path (A → B → C → A) or the conflicting pair. Once the cycle is broken or the conflict resolved, a normal boot returns. You can start any service by hand from Safe mode in the meantime — it is only auto-start that is restricted.

Booted into Recovery mode #

Recovery mode gives you a SYSTEM shell on the console and skips Phase 2 services. It starts registryd only if the boot has not already attempted that start. You reach it when the boot counter hits N, when peios.recovery=1 is set, or when registryd failed in Phase 1 (Recovery is immediate; this boot attempt has already incremented the counter).

If the registry itself is failing:

1. Preserve peinit's console failure reason and the source's startup diagnostics. A failed registry operation does not prove the source process has exited.
2. Identify the installed registryd provider and version, its hive declarations and actual database paths. LCS and sources explains the default loregd provider and its diagnostics.
3. Establish whether the source is already active before any storage work. Recovery may retain a registryd started or attempted earlier in the boot. Do not launch a second source against the same hive files.
4. Obtain the installed provider's supported recovery procedure before attempting offline inspection or repair. The source-checked loregd implementation accepts hive declarations and has no offline repair switches. It does not create an automatic startup backup; its orphan cleanup is not corruption repair or backup restoration. Those references identify the checked revision. Verify what recovery copy actually exists and what it contains before planning a replacement.

When the source is serving the required registry operations, Backup and restore describes the supported privileged subtree workflow. That workflow depends on a working source; it does not repair an unavailable source. Restore replaces the target contents and descendants, including security descriptors, so review the target and backup before using it. Role definitions can re-supply service configuration but do not restore every registry value.

Recovery offers no TCB guarantee. Treat the unrestricted SYSTEM shell accordingly, and remember it needs console access (physical, IPMI, or serial); there is no remote recovery yet.

Where to start #

To read states and causes fluently, keep The service lifecycle handy.

For restart, backoff, and Critical-reboot behaviour, see Keeping services running.

For the commands and their outputs, see Controlling services.

Run svctl status <service> and read state and cause together. State tells you where the service is now; cause tells you why it got there. Failed with ValidationError calls for a definition fix, while Failed with RestartBudgetExhausted calls for the logs from repeated attempts.

Use the state table below for a quick interpretation, then the troubleshooting guide for the next action. You do not need the full state machine before running a status check.

The states #

Four of these — Active, Reloading, Completed, Skipped — satisfy dependents. Everything else blocks them. That single column is the rule the whole dependency system turns on: a service waiting on a Requires target does not move until that target reaches one of those four.

Choose the next check #

• Starting: inspect the current operation, readiness configuration, and pre-hook output. The main process may not exist yet.
• Backoff: an automatic retry is already pending. Another start honours the existing delay; stop cancels the pending restart.
• Failed: read the cause and the failed run’s logs before starting again. Definition and dependency faults are not fixed by increasing retries.
• Skipped: inspect the condition or terminal owner. Skipped satisfies dependents and is not itself a failure.
• Abandoned: investigate the underlying I/O fault. reset clears the state but cannot kill a process stuck in D-state.

Controlling services lists which actions are valid in each state.

Transition causes #

Every transition carries a cause. peinit keeps the cause of the most recent transition on the service, and emits all of them to eventd. The full taxonomy, grouped by what kind of thing happened:

Why a service started

Why a service stopped

Why a service failed or restarted — the diagnostic causes

Why a service was rejected — definition and graph problems

The grouping matters for what peinit does next: the diagnostic causes are mostly restart-eligible (peinit consults restart policy), whereas the definition/graph causes are never restarted — retrying a broken definition cannot help. The distinction is spelled out in Keeping services running.

The Abandoned state #

Abandoned is the one state that reflects a kernel-level problem rather than a service-level one. peinit reaches it when it has sent SIGKILL to a service's process group but the processes are still there after a grace period — the post-kill timeout, 5 seconds by default. If the service's cgroup has not emptied within it, the processes are wedged in uninterruptible kernel sleep (D-state), typically behind a hung mount or a broken storage controller, and the service is marked Abandoned (its cgroup leaked).

peinit cannot kill a D-state process; nothing in userspace can. So it stops trying: it marks the service Abandoned, leaks the cgroup (it cannot be removed while populated), and moves on rather than hanging. The leak is never silent — it shows up in the service's warnings and, on a later start, peinit creates a fresh "generational" cgroup so the new instance is unaffected by the stuck old one.

An Abandoned service is cleared with reset, which re-checks the cgroup: if it finally emptied, peinit cleans up; if it is still populated, peinit leaves it leaked and warns you. An Abandoned service is a sign of an underlying I/O fault that needs investigating, not something to paper over with a restart.

The common path #

Most of a service's life is a small loop. The diagram below shows the states a healthy Simple service moves through, plus the two ways it leaves Active.

stateDiagram-v2
    [*] --> Inactive
    Inactive --> Starting: start / trigger / dependency
    Starting --> Active: ready (READY=1 or alive)
    Active --> Stopping: stop / shutdown / conflict
    Stopping --> Inactive: exited (clean stop / shutdown)
    Stopping --> Failed: exited (conflict / BindsTo eviction)
    Active --> Backoff: crash + restart allowed
    Backoff --> Starting: backoff delay elapsed
    Active --> Failed: crash + no restart
    Starting --> Failed: timeout / hook / setup failure
    Failed --> Starting: explicit start

A few things this picture makes concrete:

• An automatic restart always routes through Backoff, never through Failed. Backoff → Starting is the retry; Failed is reached only when restarts are exhausted or disabled. This is why OnFailure fires once at the end, not on every retry.
• Starting can fail before any process exists — a parent-side setup error, a failed pre-hook, a condition or assert. The cause records which.
• Failed → Starting is how a manual start (or a recovery path) revives a dead service; automatic restarts never originate from Failed.
• Stopping has two exits. A clean stop or a shutdown lands in Inactive. But a service stopped because it lost a Conflicts race (ConflictEviction) or because its BindsTo target went away (BindsToPropagation) comes to rest in Failed, carrying that cause — so an evicted or bound-out service shows as Failed in status and needs a reset (or, for a bound service, its target returning) before it starts again. It was not shut down on purpose, so peinit does not treat it as cleanly Inactive.

Oneshot services follow a parallel path through Completed instead of Active: Starting → Completed, and then either staying there (RemainAfterExit=1) or passing through to Inactive. See Simple and Oneshot services.

Readiness gating during boot #

The dependent-satisfaction rule is what makes boot orderly. A service's dependents do not start until it satisfies them:

• A Simple service satisfies dependents when it signals READY=1 (Notify) or when its process exists (Alive).
• A Oneshot service satisfies dependents when it exits successfully and reaches Completed.
• A Skipped service satisfies dependents immediately — it succeeded by not needing to run.

If a service does not reach readiness within its StartTimeout, its dependents diverge by relationship: Requires dependents transition to Failed (DependencyFailure); Wants dependents start anyway. That difference is the whole point of the two relationship types — see Dependencies and ordering.

Invariants #

A handful of rules hold without exception:

1. A service is in exactly one state at any moment.
2. Only the transitions peinit defines are valid — there are no others.
3. Only peinit changes a service's runtime state. No external process can reach in and set it.
4. A service's securable identity (its ServiceSecurity descriptor — who can manage it) is independent of its process identity (its token — what it can access).
5. Readiness is per start generation — a READY=1 from a previous incarnation is never honoured for the current start.

Where to start #

To understand what happens after a service crashes — restart policy, backoff, health checks, watchdogs — read Keeping services running.

To understand how dependents block on and are released by these states, read Dependencies and ordering.

To see which commands are valid in which state, read the command × state matrix in Controlling services.

A job is one execution; an operation is one action requested on a service. Use the IDs from svctl status to follow what is happening. A service restart creates a new job, so its job ID also distinguishes its output from the previous run.

$ svctl status jellyfin
$ svctl operation-status <operation-id>
$ svctl job list --state running
$ svctl job status <job-id>

job list and job status here are for submitted jobs. For service-run history, follow the job ID into logs and events. Live status is short-lived; eventd is where to look after an ID expires.

Run a one-off job #

Use job submit when you want one supervised execution under your own identity. For example, if /usr/bin/backup is installed:

$ svctl job submit --description "nightly backup" --timeout 3600 -- /usr/bin/backup --full

Keep the returned ID, then query or wait for it:

$ svctl job status <job-id>
$ svctl job wait <job-id>

To end the whole job, use svctl job stop <job-id>. job signal sends a signal only to its main process and is not a substitute for stopping its cgroup. Check the final state and cause: a job that handles a requested stop and exits successfully is Completed with cause explicit_stop.

A submitted job remains queryable for 60 seconds after it ends. After that, use its GUID in eventd. See job command options and permissions before passing descriptors, changing the job descriptor, or attaching live output.

How they surface #

You meet jobs and operations in four places:

• status — current_job and current_operation give the GUIDs of the service's active execution and active action (or null).
• operation-status <id> — polls one operation to a terminal state; a --wait lifecycle command does this for you under the hood.
• job list / job status <id> — the submitted jobs you may query, and one job's view.
• eventd — the durable history. Every job and operation lifecycle transition lands there as a structured event, which is how you reconstruct what happened after the objects themselves are gone from peinit's memory.

Jobs: what actually ran #

Every time peinit forks a process — a service's main binary, a pre- or post-hook, a health check, or a submitted job — that single execution is a job, with its own GUID and exit result. If a service is the what, a job is what actually ran. A restart does not reuse a job; it creates a new one. The service always tracks its current job GUID, and a status query reports it.

Jobs come in five types:

Their lifecycle is simpler than a service's, because a job only tracks a process — not policy:

A job record carries the things you would want for forensics: the resolved identity and a token summary, the image path and arguments, created/started/ended timestamps, the exit code or signal, the cgroup, the failure cause, and the operation_id that created it (null for submitted jobs). The clean division of ownership is: the service owns policy (restart, dependencies, health schedule, current state); the job owns the execution facts (PID, exit, timestamps, identity, cgroup, log correlation).

Retention and log correlation #

For service executions, peinit keeps only active job records in memory. When a job reaches a terminal state it emits a structured event (through KMES) carrying the full record, then drops the job. Submitted jobs retain a terminal view for 60 seconds, as described below; peinit keeps no durable job history — eventd is the historian, consuming those events from the kernel ring buffer. The lifecycle events are peinit.job.created, peinit.job.started, and peinit.job.ended; a submitted job also emits peinit.job.status.reported as its progress changes. peinit.job.created and peinit.job.status.reported are verbose events, off unless the event emission policy (Machine\Generic\Events, PGSS §6.9) turns them on.

Separately, captured job stdout/stderr is forwarded to eventd tagged with the job's GUID. A service with TTYPath writes to its terminal instead; see the terminal-attached service exception. The GUID lets a query like "show me the logs for job X" distinguish that execution's records from the run before or after it.

Operations: what was requested #

An operation is a first-class object representing a requested state-machine action on a service. Rather than letting control commands mutate state directly, peinit turns each one into an operation that is validated, queued, and executed by its event loop. This is what gives concurrent callers — admin tools, automated triggers, dependency propagation — explicit, observable conflict resolution instead of races.

There are five operation types — Start, Stop, Restart, Reload, Reset — and each carries a source recording why peinit created it:

The source is why a status showing current_operation.source: "restart_policy" tells you the service is being auto-restarted, while "admin" tells you a person asked.

Operation lifecycle #

flowchart LR
    P["Pending"] --> R["Running"]
    R --> C["Completed"]
    R --> F["Failed"]
    R --> A["Aborted"]
    P --> M["Merged"]
    P --> X["Cancelled"]
    P --> F

Cancelled and Aborted are the same idea at two points: cancelled never ran, aborted was running. The reason (admin action, supersession) is a property of the event, not the state.

Conflict resolution and merging #

When a command arrives for a service that already has an operation in flight, peinit resolves the collision deterministically — the same logic the command × state matrix summarises:

• Same type (start over start, stop over stop, reload over reload) → merge. The new caller transparently receives the existing operation's GUID; from their side, their request is in progress.
• Stop wins over start. An explicit stop cancels a pending start or aborts a running one. The admin said stop.
• Later supersedes earlier, and a start while a stop is in flight is queued to run after the stop.

Timeout and retention #

An operation inherits its target's timeout as its maximum lifetime — StartTimeout for start/reload/reset, StopTimeout for stop, and the sum of both legs for restart. Crucially, the clock starts at operation creation, including queue time — from the caller's perspective they have been waiting since they sent the command, so a long queue can time an operation out before it even runs.

Operations are emitted as events (peinit.operation.requested, .started, .ended and .merged; one .ended covers completed, failed, cancelled and aborted, and says which) and terminal ones are dropped from memory after a short grace (default 60 s — long enough for a polling client to read the result). As with jobs, peinit keeps no operation history; eventd does.

Submitted jobs #

A submitted job is a program peinit runs once, under supervision, because a process asked it to — not because a definition in the registry said so. Two situations motivate it:

• A task on a client's behalf. backupd takes a request from a user, and wants the backup to run as that user, in its own cgroup, with a timeout, reporting progress, visible in job list — but supervised by peinit rather than by backupd itself.
• A session. A logon service (say, a web front-end) has authenticated a user and wants a long-lived per-logon process running as them, with a socket it can talk to, that it can stop when the session ends.

Both are the same mechanism: a jobs socket at /run/services/peinit/jobs.sock, separate from the control socket. A submitter connects, sends a job definition, and gets back the job's view and — while the job runs — a handle to its process.

Who may submit #

Being able to connect is the permission. peinit performs no access check of its own on a submission; the check is the one the kernel performs against the socket file's security descriptor when a process connects. The default descriptor gives SYSTEM and Administrators full control and Authenticated Users the write right that connecting requires — so out of the box, any logged-on principal may submit, and an administrator narrows or widens that by changing the socket's descriptor, not by configuring peinit.

What limits an over-eager submitter is a quota: at most MaxJobsPerSubmitter live jobs per submitting SID (default 64; SYSTEM is exempt). A submission that would exceed it is refused with QUOTA_EXCEEDED and creates nothing. A job stops counting the moment it ends.

What a job runs as #

A job submitted by svctl runs as your process’s own primary token. There is no CLI identity switch. A service can submit with a token it is authorised to convey; the kernel checks that authority, and a token below Impersonation level is refused with BAD_TOKEN. A job needing Delegation must be submitted through a connection already at that level; it cannot be raised later.

The view records both the submitter and the identity the job runs as, plus its logon session. The TRM identity-conveyance reference describes attached tokens.

The definition #

Choose an absolute image path, its arguments and environment, a working directory, and any timeout or readiness settings. A submitted job has no restart policy, dependencies, health check, or trigger; use a service definition if you need those.

Submission returns once the job is running or has failed to start, not when it has finished or signalled readiness. Use --wait to wait for completion or job wait --for ready for a job configured with notify readiness. An accepted submission can still fail at exec, for example because its identity cannot execute the image.

See svctl job options for the CLI and the submission reference for complete fields and descriptor passing.

Who may manage a job #

Every job carries its own security descriptor from the moment it exists. By default it is owned by the submitter and grants JOB_ALL_ACCESS to the submitter, SYSTEM, and Administrators — and to nobody else. In particular the job identity gets nothing: a process running as U cannot see or stop a job that happens to run as U, unless the submitter's descriptor says so. A submitter may supply its own descriptor (in SDDL) and peinit uses it exactly as given — including a descriptor that locks the submitter out of its own job.

These rights govern both the jobs socket's own commands (status, wait, stop, signal) and the control socket's job-status, job-list, job-stop. As with services, job list filters rather than denies: you see the jobs you hold JOB_QUERY on, while a command naming an existing job you cannot query returns ACCESS_DENIED. UNKNOWN_JOB means the ID is unknown or no longer retained.

The job view #

svctl job status <id> reports the job’s state and cause, submitter and execution identity, PID, readiness, exit code or signal, and timestamps. Its status text and progress are reported by the program itself; an absent value does not establish that it is stuck.

The TRM job-view reference describes the progress message forms and update limits.

Stopping, timeouts, and the end of a job #

A job ends when its process exits, when its timeout elapses, when its readiness_timeout elapses without READY=1, when someone stops it, or at shutdown. A stop — whichever of those triggers it — is a service stop in miniature: SIGTERM to the main process (skipped if the job already said STOPPING=1), then after stop_timeout a SIGKILL of everything in the job's cgroup, and if that survives, the job is Abandoned after the post-kill grace with cause process_unkillable.

cause records why peinit ended a job, and is null when the process ended of its own accord:

State follows the exit, not the cause: a stopped job whose process handled SIGTERM and exited 0 is Completed with cause explicit_stop — peinit asked, the process agreed, and both facts are recorded. One that died to the signal is Failed with exit_signal set. A signal command is the raw mechanism, by contrast: SIGKILL sent that way produces a Failed job with a null cause, because peinit did not decide to end it.

A terminal job is retained for 60 seconds — long enough for a submitter polling once a second to read the result — then dropped; after that its GUID answers UNKNOWN_JOB, and the durable record is in eventd.

Shutdown #

At shutdown, every live submitted job is stopped at once, in no particular order — jobs have no dependencies, so there is nothing to order — and a job still queued to launch is cancelled with cause shutdown. Shutdown does not complete while a live job remains, bounded by the global ShutdownTimeout like everything else. Submissions during shutdown are refused with INVALID_STATE.

Output #

A job's stdout and stderr are captured and forwarded to eventd tagged with the job's GUID, unconditionally — a submitter cannot turn that off. A submitter that wants a live copy attaches one extra descriptor and sets output: true: peinit then writes each line it reads to that output sink as well. The copy is best-effort — a sink that is not being drained never slows the job or peinit; lines that would block are dropped for the sink only while the independent forwarding to eventd continues, and the drop is reported once per job as a peinit.job.output.dropped event.

Submitted jobs bypass the operation model entirely — there is no service to "start," so the submission creates a job directly. The job is the whole lifecycle.

Where to start #

To create and poll operations from the command line, read Controlling services.

To understand the service states an operation drives a service through, read The service lifecycle.

To query the durable job and operation history, read Auditing.

Use svctl definition or Services Manager to change a service’s configuration. Both check field values before writing them. You need permission to change the service’s registry key; permission to start or stop the service is separate.

Start by reading the current definition and runtime state:

$ svctl definition show sshd
$ svctl status sshd

A definition lives at Machine\System\Services\<name>. Its typed registry values describe the program, identity, dependencies, triggers, and supervision policy. The name of the key is the name you use in svctl.

Change and verify #

1. Read the current definition. Decide which field needs to change and when it will apply.
2. Use svctl definition edit or Edit definition… in Services Manager. Check the reported changes before restarting a running service.
3. Validate the saved definition with svctl definition validate <service>. This checks fields; identity, privilege names, and dependency targets also need their runtime checks.
4. If launch settings require it, restart the service during an appropriate interruption window. svctl reload-config re-reads definitions but does not replace a running process.
5. Run svctl status <service> again. If it fails, inspect the cause and logs.

Creating and changing a definition #

A definition is a registry key, so reg can write one. Two tools also know what each field means, and check a definition before writing it.

From a terminal: svctl definition #

$ svctl definition show sshd
$ svctl definition validate sshd
$ svctl definition create web --set ImagePath=/usr/bin/web --set Requires=lpsd --set Requires=netd:routed
$ svctl definition edit web --set StopTimeout=20 --unset Wants

def is short for definition.

• Fields go by their registry names, in any case.
• Values are given as text:
  • a number as digits;
  • yes or no;
  • a choice by its name (Type=Oneshot);
  • a list with one --set for each item, in order.
• A change is checked first, as peinit checks it. It goes through peinit's own decoder, and every timer: schedule is parsed. If peinit would reject it, nothing is written and you are told why.
• Only the values that change are written, in one registry transaction. If one of them has changed since svctl read it, nothing is written.
• Values that are not fields are kept as they are.
• edit names any changed fields that wait for a restart on a running service. See the mutability classes.

Some things can't be checked until later:

• privilege names, when the token is made;
• whether the services a definition names exist, when the graph is built;
• the identity, by authd.

svctl status after the change is the final word.

From the desktop: Services Manager #

In Services Manager, Edit definition… opens the selected service's definition in a window of its own. It's in the details pane and on each row's right-click menu. It reads Definition… when you may only read the definition. New service… on the bar defines a new one. A service's definition opens in one window at a time: while it is open, Edit definition… is unavailable for that service.

• Every field is shown, by group. Each has its name in words with the registry name beside it, its default, and when a change takes effect.

• What you type is checked when you leave the field, as peinit checks it. Anything wrong is said beside the field, and Save stays unavailable until it is put right.

• Each timer's schedule says when it next comes round, under Triggers, in local time. A schedule that never comes round, such as *-02-30, is marked as one that never starts the service. peinit takes such a definition and arms its other timers, so it does not stop you saving. These times are the schedule's, before any jitter. When the timer will really fire is in the details pane, once it is saved (see Controlling services).

• Save writes as svctl does, from the button or with Ctrl+S. Only the changes are written, in one transaction, and it is refused if someone else changed a value meanwhile. Revert reads the definition again. If the service is running, Save says which changes wait for a restart.

• Delete… asks first. A running service carries on until it stops.

• What you may do is asked of the registry:

  • changing needs the right to set values on the service's key;
  • defining a service needs the right to create keys under Machine\System\Services;
  • deleting needs DELETE on the key.

  Without those rights, the definition is shown with every field fixed and the reason said.

• Who may control the service (ServiceSecurity) is changed with Who may control it…, not here. See Who can manage a service.

Field mutability: when a change takes effect #

Every field falls into one of four mutability classes. This table is the one to keep handy.

On a service that is not running, Triggers and Disabled take effect as soon as peinit has read the change: that is what arms a timer added to an inactive service.

The practical reading: changing what a process is or runs as (ImagePath, Identity, Type, privileges, ErrorControl) is fundamental enough that it only applies when a fresh process starts — you must restart. Changing policy that peinit consults each time it acts (timeouts, restart behaviour, health checks) is picked up the next time peinit acts. And ServiceSecurity is special: it is re-read on every control request, so an access-control change takes effect on the very next command without touching the running service.

Environment, Arguments, WorkingDirectory, and process limits do not alter an already-running process. Even though peinit can reload those fields, the process receives them only on its next start.

For inactive services the rules are simpler, because there is no running process to protect: a new service entry becomes available once the change notification is processed; a changed timer arms on the next evaluation; a changed dependency takes effect on the next start.

Removing a service definition #

Use svctl definition delete web only when you intend to remove the definition of web. To stop its current work, use svctl stop web instead.

Deleting a definition from the registry does not kill a running instance. peinit learns of the removal through the same notification path and behaves according to whether the service is running:

• Not running (Inactive, Failed, Completed, Skipped, Abandoned, Backoff): peinit discards the in-memory entry immediately. There is nothing to supervise.
• Running (Active, Starting, Reloading, Stopping): the running process is a job, and a job outlives its definition. peinit marks the entry definition-removed, keeps the cached definition only to finish supervising the existing instance, and does not restart it when it exits. Once it exits, the entry — and any stored fds — are discarded.

A removal that leaves a dangling hard dependency does not apply: peinit refuses the configuration read with MissingHardDependency. Remove or update the dependent definitions first. A definition-removed entry therefore has no dependents.

For a definition-removed entry, stop still works so you can drain it cleanly, but start, restart, and reload are rejected with UNKNOWN_SERVICE — there is no definition to start from. A status query reports it with a definition_removed: true flag so the draining instance is never invisible.

Service names #

A service name must be made of characters from [A-Za-z0-9._-] and be 1–128 bytes long. Anything else is a validation error. Two characters are pointedly excluded:

• / — names map directly onto cgroup ids and registry key names, and a slash would be ambiguous in both.
• : — reserved for peinit-internal synthetic names (for example, the way a hook job is labelled).

The name is how you refer to the service everywhere: in svctl commands, in another service's dependency list, in a ServiceSecurity descriptor, and in the per-service SID derived from it.

The definition schema #

A definition is a set of typed registry values. Rather than list all of them in one wall of rows, the tables below group the fields by what they are for, with a pointer to the page that explains each group in depth. Every field is optional unless noted; the only hard requirement is ImagePath. The full type-and-default catalog lives in the Registry key reference.

What to run — the binary and its execution context.

What kind of service — type and readiness. See Simple and Oneshot services.

When to start — triggers and conditions. See Triggers and timers.

Who it runs as — identity and privileges. See Service identity and privileges.

How it relates to other services — dependencies. See Dependencies and ordering.

How it is kept alive — supervision and health. See Keeping services running.

The transition phases — hooks and timeouts. See The execution environment and The service lifecycle.

The remaining knobs — scheduling, notify, fds, metadata.

Forward compatibility #

The schema version lives at Machine\System\Services\SchemaVersion (currently 1). peinit is deliberately forward-compatible:

• Unknown values are ignored. A definition written for a newer peinit does not break an older one.
• A newer schema version does not block boot. peinit logs a warning and continues.
• The schema only grows. New capability arrives as new optional fields, never as a breaking change to an existing one.

One defensive rule cuts the other way: a known field must not appear more than once in a collected definition. A duplicated known field is a validation error, even though the registry would ordinarily give you at most one value per name.

peinit works from a snapshot, not the live registry #

Here is the idea that explains most surprises. peinit does not re-read the registry every time it touches a service. It reads definitions at well-defined moments and operates on an in-memory model in between.

Two layers of snapshotting stack on top of each other:

• Boot generation. At the start of boot, peinit reads all definitions, builds the dependency graph, and validates it. The whole boot runs against that one snapshot. Changes made during boot — by an install script, a post-hook — do not perturb the boot in progress.
• Activation generation. When peinit starts a specific service, it snapshots that service's definition for the entire start. Pre-exec hooks, the token request, the readiness timeout, the first health checks — all use the values captured at activation. Edit a field while the service is Starting, and the edit waits for the next start.

peinit learns about registry edits through change notifications: it subscribes to Machine\System\Services\ and Machine\System\Init\ at boot, and processes events in its event loop at a time of its choosing. If the notification queue overflows — a bulk admin operation, a flurry of scripted writes — peinit detects the overflow marker and does a full reload-config to resynchronise. You never have to think about the queue; you do have to know that a change is picked up, not pushed, and that when it takes effect depends on the field.

Where to start #

To understand the Type and Readiness fields and the Simple/Oneshot split, read Simple and Oneshot services.

To understand how a definition becomes a running, supervised process — and what each state in a status output means — read The service lifecycle.

For the complete type-and-default catalog of every registry key peinit reads, see the Registry key reference.

Choose Simple for a program that stays running and Oneshot for a task that should finish. Then check how it reports success: a Simple service becomes ready through Notify or Alive; a Oneshot succeeds by exiting successfully.

The default is Type=Simple with Readiness=Notify. Before starting a new definition, make sure the program can send READY=1 from its main process. A program that never sends it will time out even if its process is alive. Keep daemons in the foreground; peinit does not follow a double-forked replacement process.

Simple vs Oneshot at a glance #

Simple services #

A Simple service is a long-running daemon. peinit forks, installs the token, and execs the binary. The process is the service: while it runs, the service is running; when it exits, the service has stopped. This is the default and covers the large majority of services — registryd, authd, sshd, application daemons.

The interesting question for a Simple service is when does it count as started — the moment its dependents are allowed to begin. That is the readiness model, set by the Readiness field.

Readiness: Notify vs Alive #

The distinction matters because it is a promise to dependents. With Notify, a service that declares itself a Requires target is telling peinit "do not start anything that needs me until I say READY=1." With Alive, peinit can only promise that the process was forked — not that it is functional.

Once ready, a Simple service transitions to Active and stays there until its process exits or it is stopped. What happens on exit — clean exit, crash, restart — is the subject of The service lifecycle and Keeping services running.

Oneshot services #

A Oneshot service is a run-to-completion task. peinit forks, installs the token, execs the binary, and waits for it to exit. It is the right type for setup work: creating directories, initialising a database, running a schema migration, applying a one-time fixup at boot.

For a Oneshot, readiness is exit, not a signal. The Readiness field is ignored entirely — READY=1 is meaningless for a process whose whole job is to finish. Success and failure are decided by the exit code:

• Exit 0 (or any code listed in SuccessExitCodes) → the service succeeded.
• Any other exit, or death by signal → the service failed, and the failure policy applies.

SuccessExitCodes is for binaries that signal a meaningful outcome with a non-zero code — for example, a tool that exits 2 to mean "nothing to do." Each entry is a decimal 0–255; code 0 is always success and need not be listed.

Completed, and RemainAfterExit #

A successful Oneshot transitions to Completed, and what happens next depends on RemainAfterExit:

Both forms satisfy dependents; the only difference is what a later status query shows. Use RemainAfterExit=1 when "this task is done" is a state worth seeing — a boot-time migration you want to confirm ran, say — rather than a service that quietly returns to Inactive.

Oneshot timing and hooks #

Two details distinguish a Oneshot's start sequence from a Simple one:

• StartTimeout covers the entire run. For a Simple service the timeout covers startup until readiness; for a Oneshot it covers everything from the first pre-hook to process exit. A long-running Oneshot must raise StartTimeout accordingly (or send EXTEND_TIMEOUT_USEC as it works — see Keeping services running).
• ExecStartPost runs only on success. Post-hooks fire after the successful exit. If the Oneshot fails, ExecStartPost does not run.

Restarting a Oneshot #

A Oneshot is not restarted on success, regardless of RestartPolicy — even Always. A successful exit is the goal, not a failure to retry. RestartPolicy governs only the response to failure (a non-zero exit). To re-run a Oneshot on a schedule, give it a timer trigger; that is the intended mechanism.

No forking daemons #

peinit does not support services that double-fork to background themselves. The whole reason a traditional daemon double-forks — to detach from the launcher — is moot when the launcher tracks the process it spawned. peinit obtains a pidfd for every process at fork time, which is a kernel handle to that exact process, immune to PID-reuse races. There is no MAINPID= mechanism for a service to redirect supervision onto a different process; peinit supervises what it forked.

If a legacy binary insists on daemonising, the fix lives at the packaging layer — wrap it so it stays in the foreground (commonly a --no-daemon/--foreground flag) — not at the init layer.

The sd_notify contract #

A Notify-readiness service signals readiness by sending READY=1 over the sd_notify protocol — a datagram to the socket named in the NOTIFY_SOCKET environment variable, which peinit sets for every service. READY=1 is only the most visible message; the same channel carries watchdog keepalives (WATCHDOG=1), graceful-stop notice (STOPPING=1), timeout extensions (EXTEND_TIMEOUT_USEC), status strings (STATUS=), and the fd store. These are covered where they belong — readiness here, the rest in Keeping services running and Controlling services.

Readiness is always per start generation. A READY=1 left over from a previous incarnation of the process is never valid for the current start — peinit ties every notify message to the specific process it is currently supervising, verified by pidfd.

Who is allowed to send these messages is governed by NotifyAccess, which currently has a single value: Main (0, the default). Under Main, only the tracked main PID — the process peinit forked and supervises — is authorised to send sd_notify readiness and status messages; a message arriving from any other PID (a child, a helper, an unrelated process) is ignored. So if a worker your service spawns sends READY=1, peinit will not act on it — the readiness signal must come from the main process itself.

Where to start #

To control when either type of service starts, read Triggers and timers.

To follow a service from definition to running process and read its states, read The service lifecycle.

To configure restart, health checks, and watchdogs for a Simple service, read Keeping services running.

To see when a scheduled service will run, use svctl status <service> or the Next run column in Services Manager. To change automatic starts, edit the definition’s Triggers field. A saved schedule is not evidence that a timer is armed; check status after saving it.

A trigger decides when the service starts on its own. It is independent of service type: boot and timer triggers can start either Simple or Oneshot services. Each entry in Triggers is written type or type:argument.

Without triggers, the service is demand-only: it starts through a control command or because another service needs it. To prevent automatic activation while keeping manual start, use Disabled.

Seeing when a timer runs #

peinit reports each timer as it has it armed, in the service's status:

• the schedule's next occurrence;
• when the timer will actually fire, with its jitter added;
• when it last fired.

A schedule peinit refused, such as *-02-30, which never comes round, is reported with the reason, rather than left out.

$ svctl status logrotate
...
timers:
  *-*-* 02:00:00
    next: 2026-06-02T02:00:00Z, firing at 2026-06-02T02:07:12Z after jitter
    last fired: 2026-06-01T02:03:40Z

svctl list has a NEXT TIMER column, with each service's soonest firing. On the desktop, Services Manager shows the same things in local time: a Next run column, and each timer in the details pane. See Controlling services for the fields.

These times are peinit's own. Jitter is drawn at random each time a timer is armed, so the time a timer fires can't be worked out from its definition.

The Disabled flag #

Disabled=1 suppresses automatic activation. A disabled service will not be started by any trigger — boot, timer, terminal handover, or anything added later — but it can still be started by hand through the control interface. It is the switch for "configured, but not running on its own right now."

Two related controls are easy to confuse with it:

Enabling and disabling are registry writes performed by admin tooling, not peinit commands. peinit picks up a Disabled change through a registry notification or on reload-config. A disabled service's definition is still loaded into peinit's model, so it is ready for an on-demand start the instant you ask.

Calendar expressions #

Timer schedules are written in systemd's OnCalendar format. peinit's grammar is normative and self-contained — what follows is the whole language. The general shape is:

DayOfWeek Year-Month-Day Hour:Minute:Second Timezone

Every field is optional and has a default, so most real schedules are short.

So 02:00:00 is a complete expression — date defaults to every day, and it means "every day at 02:00." Hour:Minute with no seconds defaults the seconds to 00.

Component syntax #

Each numeric component — year, month, day, hour, minute, second — and the weekday accept the same operators:

Weekdays are English names, case-insensitive, abbreviated (Mon) or full (Monday), and take lists and ranges.

Last day of the month #

A ~ in place of the - between Month and Day counts the day from the end of the month: ~01 is the last day, ~02 the second-to-last, and so on. So *-*~01 is the last day of every month. Repetition combines with it — Mon *-05~07/1 is "the last Monday in May."

Named shortcuts #

These stand in for the expression shown:

Timezones and precision #

A timezone specifier follows the IANA database — Europe/London, US/Eastern, UTC. With none, the expression is interpreted in system-local time.

Precision is second-level. Unlike systemd, peinit does not accept sub-second fractions; a fractional second is a parse error. Service scheduling has no use for finer granularity.

Examples #

What happens when a timer fires #

A timer firing does not blindly start the service — peinit considers the service's current state first, so a firing never collides with a run already in progress.

The Oneshot "pending" flag is deliberately not a queue: many firings that land while a Oneshot is busy collapse into one catch-up run, taken when the current run ends. A service cannot stampede itself.

That single flag is shared across all of a Oneshot's timers, not one per timer. If a Oneshot has several timer triggers and two of them fire while it is busy, at most one run is still pending when the current one finishes. The timers are otherwise fully independent — each keeps its own next-firing time and its own last-run timestamp — but they can only ever queue up a single shared catch-up between them.

Each timer firing creates a normal start operation, so a timer-started service is observable exactly like an admin-started one — same GUIDs, same events.

Persistent timers and catch-up #

TimerPersistent (default 1) controls whether runs missed while the machine was off are caught up after a reboot.

peinit records the last-run timestamp of each timer in the registry. On boot, for each persistent timer it reads that timestamp, computes the next firing after it, and — if that time is already in the past — fires once, immediately, then resumes the normal schedule.

The catch-up is always a single run, never one-per-missed-occurrence. A daily timer that was off for five days fires once on the next boot, not five times. A non-persistent timer (TimerPersistent=0) ignores history entirely and simply computes its next future firing from now.

One subtlety worth knowing: the last-run timestamp is keyed by the schedule string. If you change a timer's schedule, the old timestamp is orphaned and the new schedule has no history, which causes exactly one spurious catch-up run. Schedule changes are rare and the cost is one extra run, so peinit accepts it rather than tracking trigger identity.

Jitter #

TimerJitter (default 0) adds a random delay of 0 to TimerJitter seconds to each firing, recomputed every time. A timer scheduled for midnight with TimerJitter=900 fires at a slightly different moment between 00:00 and 00:15 each day. Jitter only ever delays — a timer never fires before its scheduled time. It is the standard remedy for a fleet of machines all hammering the same backend at exactly midnight.

When the clock jumps #

Calendar timers are wall-clock schedules: *-*-* 02:00:00 means "02:00 on the wall," not "every 86,400 seconds." peinit arms them as absolute real-time timers that are cancelled and recomputed whenever the system clock is stepped, so the schedule stays anchored to the wall clock across corrections.

Daylight-saving is the case most likely to surprise you, and it only affects timers that name an IANA zone that observes DST (a bare 02:30:00 in system-local time follows whatever the system does). For a scheduled local time inside the affected hour:

• Spring forward — that time does not exist on the transition day. The timer does not fire for that occurrence; it resumes at the next valid occurrence.
• Fall back — that time occurs twice. The timer fires exactly once, on the first occurrence.

This is distinct from peinit's interval timers — the watchdog, health-check intervals, restart backoff, and the start/stop/reload phase timeouts. Those are genuine durations measured on the monotonic clock, so "wait 30 seconds" means 30 elapsed seconds and never lurches when the wall clock is set.

There is one boot-time case worth calling out on its own. When peinit reads a timer's stored last-run timestamp at boot and finds it is in the future relative to the current wall clock — meaning the clock has gone backwards since that run — it can't trust the timestamp, so it treats the timer as having an unknown last run and fires the persistent catch-up immediately. This only applies to the boot-time check; the runtime handling above is unaffected.

Waiting for the boot to settle #

boot:settled is boot with one difference: the service starts once the Phase 2 boot set has stopped moving — every service in it reached a state it will not leave on its own — rather than during it.

It exists for services that write to a terminal. peinit reports its progress to /dev/console, and a service holding that same terminal writes there too, so a prompt started mid-boot gets written over: login emits Username: with no newline and waits, peinit appends a service line, and the result is unreadable until you press Enter and login redraws.

The wait is capped (Machine\System\Boot\SettleTimeout, 5 seconds by default). A service stuck in Starting therefore costs a deferred service a short delay rather than preventing it — which matters most exactly when something is broken and a console prompt is what you need.

A boot:settled service is deliberately not part of the boot plan. It does not consume the parallel-start budget, is not counted towards boot success, and cannot block another service. Waiting for quiet is a preference; it must not change what the boot means.

Queueing for a terminal #

A terminal has one owner at a time. Two services writing to the same TTYPath do not fail — they interleave, which on a console means a login prompt with a progress line through the middle of it. So peinit gives the device to one of them and skips the rest, and tty:released is how the ones that were skipped get their turn.

The whole arrangement is three fields. A first-boot setup flow and a login prompt, both on the console:

On a first boot both are deferred, oobe outranks the login prompt on TTYPrecedence, so oobe takes the console and login-console is skipped with cause TtyUnavailable. When oobe finishes — or crashes, or is stopped; all three free the device — tty:released starts the login prompt. On every later boot oobe is gone, nothing is holding the console, and the login prompt starts straight away.

A few rules worth knowing:

• Higher TTYPrecedence wins, and equal precedence breaks on service name, so the outcome is the same on every boot. The default is 0.
• Nobody is preempted. Precedence decides between services starting at the same moment; a service already using a terminal keeps it.
• A service that stops is not offered its own terminal back. Relaunching it is RestartPolicy's job, not this.
• Both fields need a TTYPath. tty:released or TTYPrecedence without one is rejected when the definition is read, rather than quietly doing nothing.
• tty:released on its own starts nothing until somebody takes the terminal and lets it go. Give a service that must come up either way a boot or boot:settled trigger as well, as login-console has above — otherwise a machine with no oobe installed never gets a login prompt.

Being skipped is not a failure: a skipped service satisfies anything that depends on it, exactly as a service skipped by a Conditions entry does.

You can list more than one trigger, including more than one of the same type. ["timer:*-*-* 02:00:00", "timer:*-*-* 14:00:00"] runs at 2 am and 2 pm. The model is built to grow — future trigger types (path, device, event) will slot into the same list without a schema change.

Where to start #

To see how a timer-started run appears as a job and an operation, read Jobs and operations.

To understand the states a timer-started service moves through, read The service lifecycle.

For the exact registry keys that store last-run timestamps, see the Registry key reference.

When a service keeps restarting, check its state, cause, and logs before changing the restart policy. Backoff means peinit is waiting to retry; Failed with RestartBudgetExhausted means the retry budget ran out.

Use the policy reference below to distinguish a program failure from a failing health check or watchdog. Preserve the underlying error while you investigate. Increasing a timeout or retry count can change when the failure appears without fixing it.

Restart policy #

When a service fails in a way that might be transient, peinit consults RestartPolicy:

Not every failure is restart-eligible. peinit splits causes into three groups:

• Restart-eligible — peinit consults the policy and budget: ProcessCrash, WatchdogTimeout, HealthCheckFailure, ReadinessTimeout, PreHookFailure, PreExecFailure, ParentSetupFailure. (The startup failures are eligible on purpose — a pre-hook that failed on a momentarily-missing mount deserves another try.)
• Never restarted — retrying cannot help: an explicit stop or reset, shutdown, conflict eviction, a broken definition (ValidationError, CycleDetected, DependencyFailure, AssertionError), an already-exhausted budget, or an unkillable process.
• Budget-exempt — BindsToRecovery, where a bound service came back and its dependents are revived. The dependent did not fail on its own, so this restart does not count against its budget.

The budget and backoff #

Restarting an instantly-crashing service in a tight loop is worse than useless, so every restart is throttled by two mechanisms working together.

Exponential backoff. Before each restart the service sits in the Backoff state for a delay that doubles on each consecutive failure, starting from RestartDelay (default 1 s) and capped at 60 s. So a crash-looping service backs off 1 s, 2 s, 4 s, 8 s, … up to a minute between attempts.

The budget. A consecutive-failure counter tracks how many times the service has failed in a row. Once it reaches RestartMaxRetries (default 5), the next failure is not restarted: the service transitions to Failed with cause RestartBudgetExhausted.

The reset. The counter resets to zero once the service stays Active for RestartWindow seconds (default 120) without failing. This is the crucial detail: the budget is not "N restarts ever" — it is "N restarts faster than the service can sustain RestartWindow of health." A service that crashes once a day and recovers cleanly each time never exhausts its budget, because every crash starts from a counter of zero. Only failures recurring faster than the service can stay healthy accumulate.

flowchart LR
    A["Active"] -->|crash| B{"budget left?"}
    B -->|yes| C["Backoff<br/>delay doubles"]
    C -->|delay elapsed| D["Starting"]
    D -->|ready, stays Active<br/>RestartWindow| A
    B -->|no| E["Failed<br/>RestartBudgetExhausted"]

Clean exits #

For a Simple service, exiting with code 0 (or a SuccessExitCodes match) is success, not a crash — a daemon chose to quit. What peinit does next depends on the policy:

• Under Never or OnFailure, the cause is CleanExit and the service goes straight to Inactive, with no restart-policy consultation at all.
• Under Always, the cause is CleanExitRestart and the service is restarted — but through the same backoff and budget as a failure, so a daemon that exits cleanly in a tight loop cannot bypass throttling. Logs and status make clear it exited successfully and was restarted only because the policy is Always.

A terminal-attached Simple service is an exception: a successful exit with RestartPolicy=Always is treated as a normal logout. It resets the failure counter and restarts at the base delay, capped at 60 seconds, without using the crash budget. Failures still use the ordinary budget.

(A Oneshot clean exit is never restart-eligible regardless of policy — see service types.)

Health checks #

A live process is not always a working one — it can lose its database connection, wedge in a bad state, or return errors while still "up." An active health check closes that gap by running a command periodically and treating its exit code as a verdict: 0 is healthy, non-zero is a failure.

Health checks apply to Simple services only. peinit starts a service's health-check timer once the service becomes Active; a Oneshot has no long-running process to probe, so it never has one. HealthCheck and its companion fields are inert on a Oneshot — no timer is started — so don't set them there expecting supervision.

HealthCheckRetries consecutive failures mark the service unhealthy and restart it through the normal restart policy — backoff and budget included. A single success resets the failure count. If a check is still running when the next interval fires, the new one is skipped rather than stacked.

The health field in a status response reflects this: healthy, unhealthy (failing but not yet failed out), unknown (configured, no result yet), or null (no health check).

The flap guardrail. Health-check throttling only works if a failure cycle is shorter than the restart window — otherwise the counter resets between failures and the service restarts forever. peinit enforces this at validation time as a hard rule:

HealthCheckRetries × HealthCheckInterval < RestartWindow

A definition that violates it is a validation error, not a warning — the service is marked Failed with ValidationError rather than allowed to restart indefinitely.

A health check stuck in D-state is handled like any other stuck helper: its sub-cgroup is leaked and surfaced as a warning, but — unlike a stuck main process — it does not push the service to Abandoned. A health check holds no service resources; it is only a probe.

The watchdog #

The watchdog inverts the health check: instead of peinit asking the service if it is alive, the service must periodically tell peinit. Set WatchdogTimeout (seconds; 0 disables) and the service must send WATCHDOG=1 via sd_notify at least that often. Miss the deadline and peinit treats it as a failure (cause WatchdogTimeout) and applies the restart policy.

Like health checks, the watchdog is a Simple-only supervision mechanism. peinit arms the watchdog timer only once a Simple service becomes Active; on a Oneshot WatchdogTimeout is inert — no timer is started, and setting it changes nothing.

A service can adjust its own watchdog at runtime by sending WATCHDOG_USEC=<microseconds>: a positive value re-arms the timer with the new interval immediately; 0 disables the watchdog. This is for services whose phases have different latencies — a database might run a tight 5 s watchdog normally but widen it to 60 s during a compaction pass, because it knows its own phases better than the schema author did. The runtime value does not persist: a restart reverts to the schema's WatchdogTimeout.

Timeout extension #

A service that does genuinely slow work during a transition can ask for more time by sending EXTEND_TIMEOUT_USEC=<microseconds> while it is Starting, Stopping, or Reloading. Each message replaces the current phase deadline (it is not additive); a service that keeps sending them keeps proving it is making progress, and if it stops, the last deadline fires and peinit escalates normally.

The extension is capped to prevent a buggy service from extending forever:

During shutdown a second, global cap also applies — the deadline can never exceed the remaining time in the global shutdown timeout, and the stricter of the two caps wins. A message received outside a transition (the service is Active, Failed, …) is ignored — there is no timeout to extend.

OnFailure #

When a service enters Failed, an OnFailure field names another service to start in response — a fallback for graceful degradation (the main web UI failed; bring up a minimal emergency endpoint). It is not a monitoring or alerting hook; that is eventd's job.

OnFailure fires for runtime failures — ProcessCrash, WatchdogTimeout, HealthCheckFailure, the startup-failure causes, and non-Critical budget exhaustion. It deliberately does not fire for:

• ShutdownWave — nothing new starts during shutdown.
• ValidationError, CycleDetected, DependencyFailure, AssertionError — these are definition or graph breakage, not a running service degrading. The fallback would likely sit in the same broken graph anyway.

Because a fallback can itself fail and carry its own OnFailure, peinit bounds the chain: it tracks which services it has already started for one originating failure, refuses to start one twice, and never follows the chain beyond a depth of 16. When the guard trips, it logs the loop and stops.

ErrorControl: when failure is not an option #

ErrorControl decides how serious an irrecoverable failure is:

A Critical service is one the system genuinely cannot run without — the platform daemons registryd, authd, lpsd, eventd are all Critical. "Irrecoverable" means the restart budget was exhausted: peinit tried, backed off, retried up to RestartMaxRetries, and the service still would not stay healthy. At that point the fastest path to a known-good system is a reboot.

A few consequences worth holding onto:

• Critical failure outranks OnFailure. When a Critical service exhausts its budget, peinit reboots; it does not start the OnFailure handler.
• ErrorControl=Critical implies SafeMode. A Critical service is always eligible to start in Safe mode.
• Critical services are OOM-protected. peinit sets their oom_score_adj to -1000, so the kernel will not pick them as out-of-memory victims.
• A runtime Critical crash does not enter Safe mode — it follows the reboot path. Safe mode is only for boot-time configuration errors (a cycle or conflict involving a Critical service). The escalation between reboots, the boot-attempt counter, and Recovery mode are covered in Boot and boot modes.

Where to start #

To see exactly which states these policies move a service through, read The service lifecycle.

To understand how a failed service propagates to the ones that depend on it, read Dependencies and ordering.

When a service will not stay up, work backward from the symptom in Troubleshooting peinit.

Check a service’s dependencies before stopping a shared component or diagnosing a blocked start. Requires waits for successful startup but leaves an already-running dependent alone if the target later stops. BindsTo also stops the dependent when its target goes away.

Relationships are set on the service that needs something, naming the target it needs. Wants attempts an optional target first; Conflicts makes two services mutually exclusive and is symmetric, so one side declaring it is enough.

Before changing a dependency #

1. Read the dependent’s definition with svctl definition show <service> and inspect each target with svctl status <target>.
2. Decide whether the requirement is successful startup (Requires), optional startup ordering (Wants), runtime coupling (BindsTo), or mutual exclusion (Conflicts).
3. Use a readiness level when the target must have reached a specific condition, such as a routed network.
4. After saving, validate the definition and check the next start. Dependency edits apply on a new start or graph reload; they do not rewrite a running process’s history.

A DependencyFailure usually points to the target to investigate first. An on-demand start can pull in other services and evict conflicting ones, so its effects can extend beyond the service you named.

The four relationships #

Requires — a hard dependency #

If A Requires B:

• Start. B must reach a satisfying state before A starts. If B is not running, peinit starts it first (cause DependencyStart). If B fails, A is marked Failed with DependencyFailure and never even attempts to start.
• Stop. Stopping B does not stop A. Requires is a start-ordering constraint, not a runtime leash — A keeps running.
• Runtime failure. If B crashes while A is already Active, A is unaffected. B's own restart policy handles B's recovery.

Requires is the workhorse: "I need this to have started, and if it can't, neither can I."

Wants — a soft dependency #

If A Wants B:

• Start. peinit starts B before A if B exists and is not Disabled — but if B fails to start, or does not exist, A starts anyway.
• Stop / failure. No effect either way. A and B are runtime-independent.

Wants is best-effort ordering: "start this first if you can, but I work without it."

BindsTo — a runtime coupling #

If A BindsTo B:

• Start. Identical to Requires — B must satisfy before A starts.
• Stop. If B stops for any reason — explicit stop, conflict, crash, shutdown — A is stopped too, transitioning to Stopping with cause BindsToPropagation. When A finishes stopping it comes to rest in Failed (carrying BindsToPropagation), not Inactive — so a status query shows the bound service as Failed, making it clear it was taken down by its target rather than shut down cleanly.
• Recovery. When B returns to Active, peinit automatically restarts A from that Failed state (cause BindsToRecovery). This is reactive — peinit watches B's transitions — and it is budget-exempt: A did not fail on its own, so the restart does not count against its budget. If B never comes back, A stays Failed until you reset (or start) it.

BindsTo is Requires plus a runtime leash: "I need this to start, and I should not outlive it." It is the right choice for a sidecar that is meaningless without its principal. BindsTo implies Requires; listing both for the same target is harmless, and BindsTo semantics win.

Conflicts — mutual exclusion #

If A Conflicts with B:

• Start. Starting A while B is Active creates a stop operation for B (source ConflictResolution), evicting it (cause ConflictEviction) before A starts — and vice versa. If the loser will not stop within its StopTimeout, SIGKILL escalation applies. The evicted loser does not land in Inactive: it comes to rest in Failed (carrying ConflictEviction), so it shows as Failed in status and needs a reset before it will start again. That is deliberate — an evicted service is not a clean stop, and leaving it Failed stops it from quietly restarting straight back into the conflict.
• Symmetry. Conflicts is two-way. If A declares Conflicts=["B"], starting either stops the other; B need not declare it back.

Conflicts is for true mutual exclusion — two services binding the same port, or two implementations of one role where exactly one must run — not for ordinary resource contention.

At a glance #

Waiting for a condition, not just a service #

A target may carry a readiness level after a colon. Requires = ["network:routed"] waits not merely for the network manager to be active but for it to have published the level routed — a default route in place — and holds the start until it does. Publishers report a set of currently true levels, and peinit checks membership. A routed network publishes link, addressed and routed, so a service can ask for the condition it actually needs. peinit clears the set the moment its publisher leaves a satisfying state.

network there is a role, not a service: the service that fills it declares Provides = ["network"], and peinit rewrites the entry to that service before anything else looks at it. Write the role, not the daemon. The levels link, addressed and routed are defined by network policy, so the definition keeps meaning the same thing whichever executor an image ships. svctl status shows the resolved dependency, netd:routed on a standard image.

On-demand starts #

When you start a service explicitly rather than at boot, peinit does the same graph work on a smaller scope — the requested service's transitive closure:

1. Collect all transitive Requires and BindsTo dependencies (and best-effort Wants).
2. Validate that sub-graph (cycles, missing targets).
3. Resolve Conflicts — stop anything that conflicts.
4. Start the sub-graph with the same parallel scheduler.

Anything already in a satisfying state (Active, Reloading, Completed, Skipped) is left alone — its dependency is already met, so there is no needless restart. Dependencies pulled in this way start with cause DependencyStart. If two on-demand starts need the same dependency at once, their start operations merge rather than racing.

Failure propagation #

When a service enters Failed during graph execution, the failure spreads along Requires and BindsTo edges — and only those:

1. Every service that Requires the failed one transitions to Failed with DependencyFailure.
2. Every service that merely Wants it is unaffected and starts normally.
3. Propagation is transitive: if A requires B and B requires C, and C fails, then B fails, then A fails — each with DependencyFailure.

This is the payoff of the Requires/Wants distinction. A hard dependency failing takes its dependents down with it; a soft one failing is shrugged off. Choosing the right relationship is choosing how far a failure is allowed to travel.

Graph validation #

Before peinit starts anything, it builds the dependency graph and validates it. Validation runs once per graph build — at boot for the whole boot graph, and per request for an on-demand start's transitive closure. It is not incremental.

Cycles. peinit topologically sorts the graph; if the sort fails, there is a cycle. Every service in the cycle is marked Failed with CycleDetected, and peinit logs the full path (dependency cycle: A → B → C → A) so you can find and break it. A service may not depend on itself — a self-reference is rejected as a cycle. If any service in the cycle is ErrorControl=Critical, peinit downgrades to Safe mode rather than rebooting, because a reboot would just hit the same cycle.

Missing targets. What a missing target means depends on the relationship:

Unresolvable conflicts. If two boot-triggered services conflict with each other, there is no way to honour both, so both are marked Failed with ValidationError. As with cycles, if either is Critical, Safe mode applies.

Warnings. Some conditions are logged but do not block boot — most notably a Readiness=Alive service that others Requires (see service types). Warnings appear in the logs and do not change state.

Validation errors. Some conditions do fail the service — for example a health-check configuration that violates the flap constraint. These mark the service Failed with ValidationError.

When more than one finding applies to the same service, peinit records a single primary cause by precedence — CycleDetected > ValidationError > DependencyFailure — but it logs all findings, so the primary cause never hides the others.

Parallel start #

After validation, peinit starts services whose dependencies are all satisfied, and it does so in parallel up to a configurable limit:

The scheduler is simple: every service with no unsatisfied dependencies is eligible; peinit starts up to MaxParallelStarts of them; as each one reaches a satisfying state its dependents become eligible and join the queue. The result is that independent subtrees of the graph come up at the same time, while ordering constraints are still honoured exactly.

Shutdown reverses the graph #

peinit does not need a separate stop-ordering configuration. Shutdown simply reverses the dependency graph: services with no dependents stop first, and services that others depend on stop last. A service is never stopped until everything that Requires or BindsTo it has already stopped.

Shared platform services stop after their dependents, with registryd last. The exact order comes from the installed graph; do not rely on a fixed list of daemon names. The full shutdown sequence is in Shutdown.

Where to start #

To see the states services move through as they start and stop in order, read The service lifecycle.

To understand how a target's failure interacts with the dependent's own restart policy, read Keeping services running.

To follow how dependency-driven starts appear as operations, read Jobs and operations.

A service’s Identity decides what its process can access. Its ServiceSecurity descriptor decides who may start, stop, or query it; the registry key descriptor decides who may edit the definition. Check the right boundary before changing permissions.

Read the definition with svctl definition show <service>. Prefer the identity and privileges the service actually needs. An absent or empty Identity means LocalService; SYSTEM must be explicit. peinit gives each service a separate token and never shares its own SYSTEM token.

A distinct virtual service account #

Set Identity to Service to run under the service's own S-1-5-80-… SID as its user identity. authd derives it from the attested service name, using the same case-insensitive derivation as the existing service group. Only SYSTEM PID 1 may request the attestation. No local principal record or stored credential is created, and this does not provide an interactive logon path.

Unlike shared LocalService, two differently named virtual services have distinct file and process owners. Grant storage access and policy privileges to the individual service SID. Renaming the service changes its identity and requires coordinated updates to those grants. The default remains LocalService for existing definitions; choosing Service is explicit.

The standard timed, resolvd and trustd deployments also use distinct Service identities, retaining their existing service-SID grants. Their SYSTEM preparation hooks, where present, remain separate from their main processes.

The standard eventd deployment uses this identity with SeChangeNotifyPrivilege, SeSecurityPrivilege (KMES consumption), and SeAuditPrivilege (event emission). Its System integrity level allows reception of higher-integrity client tokens; it does not grant SYSTEM account membership or SeTcbPrivilege. A short SYSTEM pre-start hook initializes registry authorization defaults, then exits before the long-running daemon starts. Private stores name eventd's own SID.

Services that need token minting/installation, raw network administration, device management or pre-authd bootstrap still require separate assessment. Granting SeTcbPrivilege solely to make an identity change work defeats the purpose of moving a service out of the machine-wide trust boundary.

The local principal source lpsd also runs as Service, with only SeChangeNotifyPrivilege and System integrity for handling client tokens. Its package provisions its private state scope; credential files are owned by lpsd and admit only lpsd and SYSTEM. Its administrative API still admits SYSTEM and Administrators. /run/psi.sock admits service connections, but authd registers a source only when its enabled service SID matches the configured source allowlist. Transport admission does not confer source authority.

Privilege restriction #

A token comes from its source (authd or the SYSTEM mint) with a default set of privileges. RequiredPrivileges lets a definition trim that set down to only what the service needs:

• peinit reads the RequiredPrivileges allow-list and removes every privilege not on it from the token before exec.
• Restriction is purely subtractive. peinit removes privileges; it never adds them. A service cannot acquire a privilege its token's source did not grant.
• If RequiredPrivileges is absent, the token's default privilege set is used unchanged.

This is least-privilege made concrete: eudev, for instance, runs as SYSTEM (it needs device access during early boot) but with its privilege set stripped to the minimum it actually uses, so a compromise of eudev does not hand an attacker the full SYSTEM privilege set. Removal is permanent for the life of that token — it is not a disable that the service can re-enable. See Privileges for what the individual privileges grant.

Which identity runs what #

A service is not just its main process — it has hooks, health checks, and a reload command, and each runs under a defined identity:

Token materialisation for hooks follows the same rules: a HookIdentity of SYSTEM is minted; any other principal is requested from authd, at the point the hook runs.

HookIdentity exists so hooks can run with different — usually higher — privileges than the service itself: creating directories in privileged locations, or running a database migration as an admin identity, before the long-running daemon drops to a lesser identity.

The per-service SID #

Every service token carries a SID derived from its service name. Granting access to that SID can distinguish two services even when they share LocalService. With Identity=Service, it is also the token’s user identity.

Renaming a service changes its SID. Update grants that name the old SID as part of the same planned change. The SID derivation reference gives the algorithm.

How a token is materialised #

Identity=SYSTEM uses a separate token minted by peinit. Other identities use authd; if authd cannot issue the required identity, the start fails rather than falling back to SYSTEM.

The SYSTEM path #

Use SYSTEM only when the service requires that authority, such as the services needed before authd is available. registryd and authd use this bootstrap path. Standard eventd and lpsd deployments now use their own Service identities and start through authd.

The authd path #

For a non-SYSTEM identity, check that authd is available and the account is permitted for service logon. The authd.service.attested event records the service, issued identity and privileges, or a refusal reason. peinit derives an authority dependency for a non-SYSTEM service or hook identity.

The token-materialisation reference retains the minting and authority flow. The current authd request is specified in the TRM authd path.

Security invariants #

The identity model rests on a few rules peinit never breaks:

1. peinit never shares its SYSTEM token. Even Identity=SYSTEM services get a separately minted token.
2. peinit never drops its own SYSTEM identity. PID 1 runs as SYSTEM for the life of the system — its identity is axiomatic, granted by the kernel at boot.
3. Privileges are subtractive only. RequiredPrivileges removes; it can never add.
4. Identity is deterministic. Every service runs as a known principal. SYSTEM must be declared explicitly; an empty Identity is the minimal LocalService, never SYSTEM.
5. A submitted job never runs as an identity its submitter could not act as. peinit installs only the submitter's own primary token or a token the kernel gated on the way in; it never opens a token by a PID or descriptor a submitter names, and there is no identity field to name one.

Where to start #

Identity decides what a service can do; the ServiceSecurity descriptor decides who can manage it — two independent concerns. Read Who can manage a service for the other half.

To see how the token is installed alongside the rest of the process setup, read The execution environment.

For the identity primitives themselves — tokens, SIDs, privileges — start at Tokens.

If a program works in your shell but fails as a service, compare its identity, environment, working directory, and standard streams. peinit does not inherit your login session. It launches the program with the context in its definition.

Start with svctl definition show <service>, then inspect svctl status <service> and the service’s logs. Check hook output as well as the main program. Environment and working-directory changes reach the next process start; they do not change a process already running.

The environment #

peinit constructs each service's environment in layers, lowest precedence first — it does not pass through its own near-empty startup environment.

1. Compiled-in base. A fixed floor peinit always provides — currently just PATH: /bin.
2. Global EnvVars. Each value under Machine\System\Init\EnvVars\ becomes a variable (value name → variable name). EnvVars\PATH overrides the base PATH; other names add. Every service gets this layer except registryd, which serves the key itself — see the warning below for why that exemption exists.
3. Per-service Environment. The definition's own KEY=VALUE entries, which override layers 1–2.
4. Protocol variables. NOTIFY_SOCKET (always) and LISTEN_FDS / LISTEN_FDNAMES (only when stored fds are injected). These have the highest precedence and cannot be overridden — a service that clobbered NOTIFY_SOCKET would break sd_notify.

A change to EnvVars\ takes effect on a service's next start, like the per-service Environment field — it is not pushed into running services.

peinit deliberately does not set HOME, USER, LOGNAME, SHELL, or TERM. Peios identity is a token (a SID), not a passwd entry, so there is no canonical home directory or login shell to fill in. A service that needs any of these supplies it through EnvVars\ or its own Environment.

Working directory and limits #

peinit also sets oom_score_adj: -1000 (OOM-immune) for ErrorControl=Critical services, and the default 0 for everyone else — so the kernel never picks a Critical platform daemon as an out-of-memory victim.

Runtime directories #

RuntimeDirectories gives a service private scratch space under /run without an ExecStartPre that has to mkdir it by hand. Each name you list becomes a directory created directly under /run — RuntimeDirectories=myapp yields /run/myapp — set up immediately before the main process starts. Each one gets a security descriptor granting full access to SYSTEM, Administrators, and the service's own SID, so the service can write to its directory while other principals cannot. If a directory can't be created, or its descriptor can't be applied, the start fails with ParentSetupFailure.

peinit does not remove these directories when the service stops. /run is a boot-scoped tmpfs, so they simply disappear at the next boot rather than being torn down on each stop — a restarting service finds its runtime directory (though not necessarily its contents) already in place.

For the exact naming rules on each entry and the field's type and default, see the Registry key reference.

Standard streams #

peinit wires a service's standard streams before exec:

• stdin is redirected to /dev/null. peinit never gives a service an interactive input channel; a service that needs input must obtain it explicitly (a socket, a stored fd).
• stdout and stderr are redirected to pipes peinit holds, so it can capture and forward every line. This is the subject of Service output and logging.

Attaching a terminal #

A service that sets TTYPath to an absolute terminal path — /dev/console, /dev/tty1, a serial line — gets that terminal on all three standard streams instead of the wiring above, and becomes a session leader owning it as its controlling terminal. That last part is what makes job control work: Ctrl-C interrupts, fg/bg behave, and a hangup reaches the process group. It is what a boot shell or a login prompt needs.

It carries a path rather than a flag because which terminal is a per-service question: a maintenance shell on tty1 while the kernel console is a serial line is an ordinary thing to want.

Two services pointed at the same terminal do not fight over it. peinit gives the device to one of them and skips the rest — highest TTYPrecedence wins — and a skipped service can ask to be started when the terminal frees, with the tty:released trigger. See Queueing for a terminal.

Pre-exec and post-exec hooks #

Hooks let a service do work around its main process without baking it into the binary.

• ExecStartPre runs before the main binary. Each command runs in sequence, in the hooks/ sub-cgroup; any hook exiting non-zero aborts the start (the whole cgroup is killed and the service Fails with PreHookFailure). Use it for prerequisites that must hold before the service starts — creating a runtime directory, waiting on a precondition.
• ExecStartPost runs after readiness (Simple) or after a successful exit (Oneshot). A post-hook failure is logged but does not fail the service — by the time it runs, the service is already up. (A Oneshot that fails never runs its post-hooks.)

Hooks run under HookIdentity if set, otherwise the service's own identity. Their output is captured and tagged like the main process's, labelled with the hook (e.g. jellyfin/ExecStartPre[0]).

Command string parsing #

The command fields — ExecStartPre, ExecStartPost, ExecReload, HealthCheck — are parsed into an argument list with simple, predictable rules. peinit never invokes a shell.

• Whitespace splits the command into arguments; double quotes group text containing spaces into one argument and are not retained (--name="hello world" → one argument --name=hello world).
• There is no shell expansion, variable substitution, or globbing. Backslash is literal; a single quote is an ordinary character.
• The first argument is the executable and must be an absolute path beginning with /. Relative names and PATH search are validation errors.
• An empty or whitespace-only command, or an unclosed quote, is a validation error.

If a command genuinely needs shell features, wrap it in a script and point the field at the script.

ExecReload can also be a signal, written signal:<NAME> — for example signal:SIGHUP. The name must be an exact, canonical Linux signal name (uppercase, no aliases, no numbers); SIGKILL and SIGSTOP are rejected because they cannot be handled as a reload. With no ExecReload at all, reload sends SIGHUP. The reload lifecycle is covered in Controlling services.

Conditions and asserts #

Before peinit runs any hook or the main process, it evaluates conditions and asserts — start-time checks in the same type:argument form. The difference is what failure means:

• A failed Condition skips the service — it transitions to Skipped, which satisfies its dependents (it succeeded by not needing to run). Conditions express "only run here if it makes sense."
• A failed Assert fails the service — it transitions to Failed with AssertionError. Asserts express "this was expected to run, and a precondition is missing."

Conditions are checked first; only if all pass are asserts checked. All entries are AND-ed. The check types:

Two constraints follow from peinit being single-threaded PID 1, which must never block:

• registry: checks are cache-only. Accepted paths are exactly Machine\System\Services, Machine\System\Init, and Machine\System\Services\<name> with one nonempty service-name component. The roots resolve as present; the last form checks whether the service exists in the in-memory model. Init descendants, deeper Services paths, trailing separators, and other keys are load-time decode errors. No live registry read is made.
• Filesystem checks run off the main loop. path:/file:/directory: checks are performed by a short-lived forked helper, not a stat() on the event loop, because stat() can wedge on a hung mount.

That timeout is the PreStartCheckTimeout field, which defaults to 5 seconds. A filesystem check that does not finish within it — say a stat() wedged on a hung mount — is treated as not satisfied, fail-safe: peinit kills the helper and treats the check as unmet, so a Condition skips the service and an Assert fails it, rather than letting the event loop hang.

The fd store #

The fd store lets a service preserve file descriptors across restarts — most usefully a listening socket, so a stateful daemon can restart without dropping connections. It is opt-in: FdStoreMax (default 0) sets the maximum number of fds peinit will hold; 0 disables it.

The mechanics ride on sd_notify:

• A service stores an fd by sending FDSTORE=1 with the fd attached (and optionally FDNAME=<name>; the default name is stored). peinit holds it.
• It removes one with FDSTOREREMOVE=1 and FDNAME=<name>.
• On the next start, peinit injects the stored fds starting at fd 3, sets LISTEN_FDS to the count and LISTEN_FDNAMES to the colon-separated names, then clears the store. (This is the same LISTEN_FDS convention systemd uses, so software with existing fd-passing support works unmodified.)

The store survives an automatic restart (crash → restart policy → new start) — that is the whole point. It is cleared on an explicit stop or shutdown (the service is not coming back), and when a removed definition is finally discarded.

A clean context #

A service receives its standard streams and explicitly supplied descriptors. It does not inherit peinit’s privileged connections or signal handling. The launch-isolation reference describes the descriptor and signal setup.

The cgroup tree #

peinit tracks a service’s main process, hooks, and health checks in separate cgroups beneath /sys/fs/cgroup/peinit/. It uses those groups for tracking and clean termination, not resource limits.

A leaked cgroup warning means a process survived cleanup, often because I/O is stuck. A later start can use a fresh <service>.gen<N> tree; that does not repair or kill the stuck old process. Investigate the Abandoned state. The cgroup reference retains the layout and rationale.

Where to start #

For the identity half of the launch — how the token is chosen and installed — read Service identity and privileges.

For what happens to the stdout/stderr pipes peinit holds, read Service output and logging.

For how conditions, asserts, and hooks fit into the start sequence and its states, read The service lifecycle.

System-console aliases and logout #

Peinit pins TTYPath=/dev/console to the kernel-selected terminal when reading service definitions. A display's moving tty0 alias becomes tty1, so switching the foreground console does not move the session. Both arbitration and launch use this fixed endpoint. When a terminal service's main process exits, peinit kills its remaining session processes before handing the terminal on. A clean exit under RestartPolicy=Always restarts the prompt at the base delay without spending the crash budget.

To let someone start or stop a service, change its ServiceSecurity permissions. Do not give them write access to the service definition just to grant control: definition writes can change the program and the identity it runs as.

In Services Manager, select the service and inspect the commands you may use. Who may control it… edits runtime control; Who may change its definition… edits the registry key’s permissions. If a command returns ACCESS_DENIED, check its required right and the caller’s identity before widening either descriptor.

Two descriptors, two questions #

A service is associated with two descriptors that are easy to conflate but answer different questions and are enforced by different components:

The registry key SD is ordinary registry access control and not peinit's concern — peinit reads definitions as SYSTEM, which has full access. The ServiceSecurity SD is peinit's domain, and the rest of this page is about it.

They are genuinely independent. An administrator might be able to query a service's status (ServiceSecurity grants SERVICE_QUERY_STATUS) but not read its configuration (the registry key SD denies read) — or the reverse. Runtime control and configuration access are separate concerns, and both combinations are valid.

Changing them from the desktop #

On a GXWI desktop, select the service in Services Manager. Below what you may do with it, the details pane says where its ServiceSecurity comes from: set for this service, the default set for every service, or peinit's built-in default. Two buttons open the permissions editor, and both are also on each row's right-click menu.

• Who may control it… opens the ServiceSecurity descriptor. Its boxes are the service rights in words: Full control, Start, Stop, Reload (SERVICE_INTERROGATE) and See its state (SERVICE_QUERY_STATUS). Restart needs Start and Stop together, and Reset needs Stop. When you apply, Services Manager writes the result as the service's own ServiceSecurity value. From then on, a change to the default no longer reaches that service. peinit applies the new descriptor on the next command, and the window's buttons change to match.
• Use the default appears beside it once the service has its own value. After you confirm, it deletes that value, so the service takes the default again.
• Who may change its definition… opens the descriptor of the service's registry key. Its boxes are Full control, Read and Write on the key. Entries you add there apply to the key and to any keys under it.

Under Every service, the pane says what a service without its own value takes:

• Default permissions… opens the ServiceSecurity on Machine\System\Services. If there isn't one, it starts from peinit's built-in default. Applying writes it to that key, and it reaches every service without its own value at the next command.
• Use the built-in default appears once that key has a value. After you confirm, it deletes the value, so services without their own go back to peinit's built-in default.

Each of these is offered for changing only if you are allowed to change it:

• ServiceSecurity is a value on a registry key: the service's key, or Machine\System\Services for the default. Changing it, or deleting it, needs the right to set values on that key (KEY_SET_VALUE).
• The service key's own descriptor needs WRITE_DAC, and WRITE_OWNER for its owner.

If you are not allowed, the editor still shows who may do what and says why it can't be changed, and the two "use the default" buttons are unavailable with the reason. Services Manager finds this out by asking the registry for those rights, not from your group memberships.

The same can be done from a terminal with reg. For example, this makes sshd take the default again:

$ reg del 'Machine\System\Services\sshd' ServiceSecurity

Service access rights #

The ServiceSecurity descriptor grants these rights:

restart requires both SERVICE_STOP and SERVICE_START, since it is a stop followed by a start. reset requires SERVICE_STOP.

When peinit evaluates the descriptor it maps the generic rights as follows, so a descriptor written with generic rights behaves sensibly:

The default descriptor #

If a service has no ServiceSecurity value, it inherits the one on Machine\System\Services itself. If that key has none either, peinit applies a built-in default:

• SYSTEM (S-1-5-18) — full access.
• Administrators (S-1-5-32-544) — full access.
• Authenticated Users (S-1-5-11) — query only.
• Everyone (S-1-1-0) — every refusal audited, in its SACL.

So out of the box, administrators can do anything with a service, everyone who is signed in can see what it is doing, and every refused command is recorded. To let someone else start or stop a particular service, give that service a ServiceSecurity that grants it; anyone signed in can reach the control socket, so a grant to them takes effect.

ServiceSecurity is hot-reloaded: a change to the value in the registry takes effect on the next control request, with no service restart. peinit picks the change up through a registry notification. This is why ServiceSecurity is in its own mutability class — access policy should be able to change without disturbing a running service.

How a command is authorised #

Every control command runs the same gate:

1. peinit captures the caller's token from the kernel (the KACS_SO_PEER_TOKEN socket option) — the caller's effective identity at connection time, so if the caller is impersonating, the impersonated identity is what is checked.
2. peinit resolves the target service and its ServiceSecurity descriptor.
3. peinit runs AccessCheck: the caller's token against the descriptor, for the right the command needs.
4. Denied → return ACCESS_DENIED. The kernel records the attempt — caller, target service, rights requested and granted — as a kacs.audit.access.checked event, because the descriptor's SACL asks it to.
5. Granted → execute the command.

The system control descriptor #

Some operations are not about any one service — shutdown and reload-config act on the whole system, and boot reports on it. These are checked against peinit's own descriptor, stored at Machine\System\Init\ControlSecurity:

The default grants SYSTEM and Administrators all three rights, and Authenticated Users SYSTEM_QUERY_STATUS, as the default service descriptor lets everyone signed in query a service. peinit loads this descriptor at boot and hot-reloads it on registry change, exactly like ServiceSecurity. A descriptor written before SYSTEM_QUERY_STATUS existed grants it only through GENERIC_READ, GENERIC_ALL or the bit itself.

Jobs have descriptors too #

A submitted job is a securable object in the same way. It carries its own descriptor from the moment it exists — by default owned by the submitter and granting JOB_QUERY, JOB_STOP, and JOB_SIGNAL to the submitter, SYSTEM, and Administrators, and nothing to the identity the job runs as — and every job command, on either socket, is checked against it. The one thing no descriptor inside peinit governs is who may submit: that is the file descriptor on the jobs socket, checked by the kernel when a process connects.

The list command filters, it does not deny #

list is access-control-aware in a quieter way: it returns only the services the caller has SERVICE_QUERY_STATUS on, and simply omits the rest. A caller with no query rights gets an empty list, not a denial. What this protects is a service's state; that a service exists is not a secret. A command naming a service is answered UNKNOWN_SERVICE only if it does not exist, and the definitions under Machine\System\Services are readable.

The boundaries that hold #

A few invariants are worth stating outright, because they are what make this trustworthy:

• peinit never bypasses AccessCheck for a control operation. No backdoor, no override flag, no "trust localhost."
• The descriptors are the only policy inputs. peinit consults the ServiceSecurity and ControlSecurity descriptors and nothing else — not config files, not environment variables, not hardcoded lists.
• One service's state is never exposed to another without a check. status is per-service access-controlled; list filters. The same holds for jobs: job status is checked, job list filters.

Where to start #

For the other half of the security model — what a service can reach once it is running — read Service identity and privileges.

For the commands these rights gate, read Controlling services.

For the descriptor and AccessCheck machinery itself, read Security descriptors and Access decisions.

On a running machine, use svctl boot or System Settings › Startup & Shutdown to see the mode, why it was chosen, and whether the boot has been confirmed successful.

• Full starts the normal boot-triggered set.
• Safe starts a reduced boot-triggered set. Use the reported graph error or requested-mode reason to decide what to repair.
• Recovery provides an unrestricted SYSTEM shell on the console and skips Phase 2 services. svctl boot is not available there.

How this boot went #

svctl boot asks peinit, and so does System Settings, at the top of its Startup & Shutdown section:

$ svctl boot
boot: full
reason: normal
unconfirmed boots before this one: 0 (recovery at 3)
confirmed: yes
grace: 30s

• boot is the mode, and reason why: normal, requested (peios.safemode=1), or a Safe mode peinit chose itself because a full boot couldn't be planned, with what stopped it.
• unconfirmed boots before this one is the boot-attempt counter as this boot found it, and the threshold at which the machine starts in Recovery.
• confirmed says whether this boot has counted as a success yet: every Critical service holding for BootSuccessGrace, and the counter put back to 0. Until then it names what it is waiting for.

Recovery mode never answers: it runs a recovery shell and no Phase 2 services or peinit control interface. Registryd may already be running, but there is no control socket to ask. Anyone signed in may ask; see who can manage a service for the descriptor.

The three boot modes #

The three modes form an escalation path from normal operation to last-resort maintenance.

flowchart TD
    F["Full boot"] -->|success| OK["operational<br/>counter reset"]
    F -->|cycle/conflict w/ Critical| S["Safe mode<br/>(no reboot)"]
    F -->|Critical failure| RB["sync + reboot"]
    S -->|success| OKS["operational (reduced)<br/>counter reset"]
    S -->|Critical failure| RB
    RB --> CNT["counter increments"]
    CNT -->|counter ≥ N| REC["Recovery mode"]
    F -.->|Phase 1 failure| REC

Full mode #

The default. All boot-triggered services start in dependency order. A boot is successful — and the boot-attempt counter resets to 0 — when every Critical service has reached and held a dependent-satisfying state (Active, Completed, or Skipped) for a grace period (BootSuccessGrace, default 30 s). The criterion is "satisfying," not "Active," so that a Critical Oneshot that reaches Completed can still mark the boot good.

Safe mode #

Safe mode starts a reduced set of boot-triggered services and rebuilds the graph from scratch using only those, dropping dependencies on excluded services. Two categories are eligible:

• Critical services (ErrorControl=Critical) — must start; a Critical failure here still follows the normal reboot path.
• SafeMode services (SafeMode=1) — best-effort; if they fail, Safe mode carries on without them.

Everything else is skipped. Safe mode is entered when:

• graph validation finds a cycle involving a Critical service, or
• an unresolvable conflict involving a Critical service, or
• the kernel command line says peios.safemode=1.

The first two are configuration errors — rebooting would just hit them again — so peinit downgrades without rebooting. Safe mode is not entered by a Critical service crashing at runtime; that follows the reboot path. And it is purely a boot-sequencing concern: once booted, you can start any service by hand exactly as in Full mode. It is the mode for "the configuration is broken but the TCB is healthy."

On a machine with the dynamic-boot feature, System Settings › Startup & Shutdown › Always Start in Safe Mode puts peios.safemode=1 on the command line the next boot image is made from. Its next-boot effect depends on a successful rebuild to the intended boot partition; verify the activation chain. Once present in the booted image it holds until changed there, and while the machine is in Safe mode because of it, System Settings says so.

Recovery mode #

Recovery mode is the last resort, and it offers no TCB guarantee — the administrator gets an unrestricted SYSTEM shell on the console and must treat it with corresponding care. It is a maintenance environment, not a degraded boot.

In Recovery, peinit completes the Phase 1 basics and starts registryd only if that boot has not already attempted it; a failed registryd start is not repeated. Registryd failure does not prevent the shell. peinit skips all Phase 2 services and execs a SYSTEM shell on /dev/console (/bin/recsh if present, else /bin/sh), respawning it if it exits. It is entered when:

• the boot-attempt counter reaches N (default 3),
• the kernel command line says peios.recovery=1, or
• registryd fails during Phase 1 — entered immediately, without rebooting, because there is no Phase 2 to attempt. The counter has already been incremented for this attempt.

Preserve the console failure reason and any source startup diagnostics first. Identify the installed registryd provider, its version, hive declarations and database paths. Recovery does not establish that the registry source is stopped: a source started or attempted earlier in the boot may still be active. Do not start a second source against the same hive files.

Offline inspection or repair needs a procedure supported by that provider and version. The source-checked loregd implementation has no offline repair switches and does not create an automatic startup backup. Its orphan cleanup is not corruption repair or backup restoration; the references identify the checked revision. Verify the existence and coverage of any recovery copy before planning to replace data.

Follow Recovery troubleshooting for the evidence to collect and the point to involve the administrator or provider. If the source can serve the required operations, the supported Backup and restore workflow can recover a subtree; it depends on that working source and is not offline source repair. Restore replaces data and permissions. Role definitions can re-supply service configuration but do not restore all registry data.

The boot-attempt counter #

The counter can escalate repeated unconfirmed boots to Recovery when peinit reaches the increment and can persist it. It is a plain integer in a file at /.peinit/boot-attempts — not in the registry, because the registry may be the very thing that is broken. A confirmed boot resets the count; the counter does not guarantee that every reboot loop ends in Recovery.

• peinit reads it at startup, before choosing a mode. The Recovery threshold (counter ≥ N) is checked against this pre-increment value, so the default N of 3 admits exactly three attempts before Recovery. A missing file counts as 0; a corrupt or unreadable one → Recovery. Override N with peios.bootattempts=N on the kernel command line, or set it to 0 to disable the check when the counter is itself the fault.
• peinit increments it once per boot after the root, mount, seed, machine-ID and clock steps, before registryd starts. A registryd startup failure therefore still consumes an attempt. An earlier failure that enters Recovery before this point does not increment it.
• peios.recovery=1 forces Recovery but does not suppress the increment once that point is reached.
• The counter is reset to 0 on a successful Full or Safe boot (after the grace period).
• If the counter file cannot be written (disk full), peinit treats it as 0 and continues — a write failure must not by itself trigger Recovery.

A Critical service exhausting its restart budget — at boot or at runtime — triggers a sync and reboot. Repeated boots that never complete the success grace accumulate attempts until the counter reaches N and peinit enters Recovery. A failure after a confirmed boot still reboots the machine, but that boot already reset the counter: repeated late failures can therefore continue without reaching the Recovery threshold. Use svctl boot to distinguish these cases. The counter deliberately does not try to catch a peinit too broken to reach its own increment; that is a binary-integrity problem, not a boot-loop problem. See the TRM counter cycle for the exact increment boundary.

Boot configuration #

All but ShutdownTimeout are read at boot, so a change applies at the next one; ShutdownTimeout applies the next time peinit re-reads its configuration (svctl reload-config). System Settings shows and changes them in its Startup & Shutdown section, under Timeouts: Apply and Undo appear once one has been edited. Changing them needs write access to Machine\System\Boot — as shipped, Administrators.

The kernel command line #

peinit reads the following peios.* tokens. They are deliberately few: everything peinit can read after registryd is serving belongs in the registry instead, where it can be inspected, secured and changed without editing a boot entry. What is left is either a per-boot mode decision or a Phase 1 value — one peinit needs before there is a registry to ask.

Console noise: peios.quiet #

peinit reports its progress to /dev/console, and so does any service that claims that terminal with TTYPath. Two writers, one device: a login prompt started while peinit is still narrating gets written over, and the reader cannot tell input from log. peios.quiet decides who yields.

It sets two independent rules, which is why it is a level and not a flag:

The rules stack rather than scale: errors are never less visible at 2 than at 1. An error overrides a requested blackout — silence was a preference, and an error is news — but it does not override terminal ownership, which is not peinit's to override. Only losing the machine (dropping to Recovery, halting with no shell, a Critical service about to force a reboot) is worth one corrupted line of somebody else's session.

prelude honours peios.quiet=2 as well, since it writes to the same console. The other two levels mean nothing there — prelude exits before the first service starts, so no terminal has an owner yet.

A few lines escape all of this: whatever peinit and prelude print before they have read the command line. Neither can honour a preference it has not seen yet, and those lines are also the only evidence either started at all.

Seeing and changing it #

System Settings shows the command line this boot was started with, under Kernel Command Line in the Startup & Shutdown section, and the one the next boot image will be made from, /lcl/etc/boot/cmdline, where they differ.

On an installed machine the command line is part of the boot image, so editing that file changes nothing on its own. Dynamic Boot supplies mkuki-watch, but package delivery or feature enablement alone does not mean it is running or rebuilding successfully. Check its selected input, output filesystem and completed rebuild using the activation checklist. With the feature installed, System Settings offers peios.bootattempts and peios.quiet as choices — how many boots may fail before Recovery, and what peinit writes on the console — and writes them into the file with Apply at Next Boot, which appears once one has been changed. Nothing else on the line is offered: a mistake there can leave a machine that doesn't boot. Writing the file needs write access to it, which as shipped only Administrators have.

Unknown peios.* tokens are ignored, as is a malformed value on a valued token — this parser runs before anything exists to report a diagnostic to, and refusing to boot over a typo in a tuning knob is the worse outcome.

Where peinit takes over #

peinit is PID 1, but it is not the first thing that runs. The initramfs assembles and mounts the real root — decryption, RAID/LVM, the root filesystem itself — and then hands control to peinit. The contract peinit relies on is narrow:

• The real root is already mounted read-write, and /proc, /sys, /dev are mounted and moved into it.
• peinit is exec'd as PID 1 from a fixed path on the real root.

peinit does not assemble, decrypt, repair, or even re-mount the root — those need tools and configuration that belong to the initramfs. It also does not fsck the root or mount non-root storage (a data partition is mounted by an ordinary Oneshot service, not by peinit). For the trust and identity side of this handoff — signatures, the SYSTEM token peinit inherits — see peinit at PID 1.

Phase 1: the hardcoded bootstrap #

Phase 1 is compiled into peinit. It cannot change at runtime and touches no registry. It does the minimum to make Phase 2 possible:

1. Confirm the root is writable with a single probe write. registryd's storage needs a writable root even for reads, so a read-only root cannot support Phase 2 → Recovery.
2. Mount the remaining virtual filesystems — /dev/pts, /dev/shm, /run, /sys/fs/cgroup — mounting each only if absent. A failure here → Recovery.
3. Restore the persisted random seed from /var/state/peinit/random-seed, mixing it into the kernel's entropy pool early so anything that needs randomness during boot gets it. A missing seed is normal — first boots and stateless live boots have none — so peinit just carries on; a seed problem is never fatal and never sends boot to Recovery.
4. Establish the machine-id from /lcl/etc/machine-id — a stable, opaque identifier for this install (used for log correlation, instance identity, and software compatibility). It is not a security principal: it is not a SID, an account, or a credential, and no authorisation decision depends on it. If the file is missing, empty, or malformed, peinit generates a fresh 128-bit ID. If it cannot persist the ID, boot continues with a warning and an ID valid only for this boot; failure to obtain random bytes sends boot to Recovery.
5. Set the clock from the hardware RTC, so early timestamps and the boot counter are meaningful. A failure here → Recovery.
6. Start registryd and wait for it to signal readiness, then probe-read the schema-version key to confirm it is actually serving reads. Any failure → Recovery — there is no Phase 2 without a registry.
7. Provision boot-time paths. With registryd up and before Phase 2 starts, peinit applies the entries under Machine\System\Init\ProvisionedPaths\ — the registry-driven equivalent of tmpfiles.d, creating directories and files (with Peios security descriptors) that no single service owns. Best-effort entries that fail are logged and skipped, but an entry marked Required=1 that cannot be provisioned sends boot → Recovery. The individual keys are cataloged in the registry key reference.
8. Infrastructure setup — create the control socket and the jobs socket, and bring up the loopback interface. A control-socket or jobs-socket failure → Recovery; a loopback failure is logged as a warning and boot continues.

Most Phase 1 failures are fatal to a normal boot, because none of the later machinery can run without this foundation — the only outcome is Recovery mode. The exceptions are the fail-soft steps called out above: a missing or unusable random seed, a regenerated machine-id, and best-effort provisioned paths all let boot continue.

registryd and loregd #

registryd is an interface, not a specific program. It is the path peinit execs to get a registry source daemon — the component that implements the registry's persistent storage and answers peinit's reads. The implementation behind that interface can vary; by default it is loregd.

The distinction matters when diagnosing source failures: peinit's registryd role does not define a storage-repair interface. Check which provider and version the installation uses before choosing recovery instructions. The default provider's database paths come from its startup hive declarations, not registry settings. See LCS and sources for source availability checks and the Recovery-mode limits before direct storage work.

Phase 2: the registry-driven boot #

With registryd serving reads, peinit boots the rest of the system from the registry:

1. Read all definitions under Machine\System\Services\. (A registry read timing out here → Recovery.)
2. Build and validate the dependency graph from the boot-triggered services and their transitive dependency closure. Validation runs before anything starts.
3. Start services in dependency order, in parallel up to MaxParallelStarts, with readiness gating releasing each service's dependents as it becomes satisfied.

Only services with a boot trigger are start candidates; demand-only services are pulled in only if something boot-triggered depends on them, and Disabled services are excluded from the graph (but kept in the model for on-demand start). The whole boot runs against one snapshot — mid-boot registry edits do not perturb it.

The installed service graph determines the platform order. registryd and authd use the SYSTEM bootstrap path; standard eventd and lpsd deployments use Identity=Service and obtain tokens through authd. See Service identity. A non-SYSTEM service’s authority dependency is derived from its identity rather than left to an administrator to remember.

Do not infer readiness or shutdown order from a fixed list of daemon names. Check the installed definitions and the service status. The boot implementation describes graph construction and validation.

Where to start #

To understand the graph validation and parallel start that drive Phase 2, read Dependencies and ordering.

To understand the Critical-failure reboot path that feeds the boot counter, read Keeping services running.

For the trust and token side of early boot, read peinit at PID 1.

Use svctl shutdown when you intend to stop the machine. Choose one action:

$ svctl shutdown poweroff
$ svctl shutdown reboot
$ svctl shutdown halt

These are alternatives, not a sequence. Each interrupts running services and jobs and requires SYSTEM_SHUTDOWN. Save work and account for other users before issuing one. halt stops the CPU but leaves power on.

A normal shutdown stops dependents before their dependencies, gives running services time to exit, then unmounts, syncs, and performs the selected action. Forced shutdown skips that opportunity to finish work.

How shutdown is triggered #

There are four paths into shutdown, and they are not equal — three are graceful, one is not.

A control command. An administrator with SYSTEM_SHUTDOWN runs svctl shutdown <type> (the familiar reboot, poweroff and halt commands, and shutdown -r now, are the same client under other names):

Signals. As PID 1, peinit treats a handful of signals as shutdown requests:

The power button. On Linux, a physical power-button press is a graceful poweroff — peinit watches the machine's input devices under /dev/input and treats a KEY_POWER press (the key going down, not its release or auto-repeat) as a shutdown request. This path is deliberately minimal and fail-soft: if /dev/input is missing, an input device cannot be opened, or a device stops responding, peinit just keeps running and drops that device — the button quietly stops working, but the machine stays up and you can still shut down over the control socket or with a signal. It is not a full power-management policy engine; a future acpid/logind-style daemon can layer richer policy on top by sending control-socket commands.

A Critical service failure — the ungraceful path. When an ErrorControl=Critical service exhausts its restart budget, peinit syncs and reboots immediately. There is no service-stop ordering: the system is in an undefined state and the fastest route to a known one is a reboot. This is what feeds the boot-attempt counter, and a Critical service that fails identically every boot becomes a reboot loop that the counter eventually breaks by escalating to Recovery.

The graceful sequence #

For a poweroff, reboot, or halt, peinit runs an ordered sequence:

1. Enter the shutdown state. A flag is set, and while it holds: no new services may start, timer triggers are disarmed, and new lifecycle commands are rejected with INVALID_STATE. Status, list, operation-status, boot, and job query/stop commands remain available so you can watch progress or stop a job sooner.
2. Suspend Critical-failure semantics. If a Critical service fails during shutdown, it is logged but does not trigger a reboot — the system is already going down, and rebooting would loop.
3. Clear Completed services. Oneshot services sitting in Completed (with RemainAfterExit) have no process; peinit moves them to Inactive so their dependents can be stopped cleanly.
4. Stop every live submitted job at once — SIGTERM, then SIGKILL of the job's cgroup after its own stop_timeout — and cancel any job still queued to launch, both with cause shutdown. Jobs have no dependencies, so there is no order among them and nothing waits for them to finish before the service waves begin; but shutdown does not complete while a live job remains, within the same global timeout.
5. Stop services in reverse dependency order, in waves. Each graceful-stop-eligible service (those in Active or Reloading) gets SIGTERM and StopTimeout seconds to exit; if it does not, peinit SIGKILLs its whole cgroup. A service is never stopped until everything that Requires or BindsTo it has already stopped. Services that are Starting are not stopped gracefully — their startup is cancelled and the cgroup SIGKILLed.
6. Enforce the global timeout. The whole sequence is bounded by ShutdownTimeout (default 90 s). When it expires, all remaining services are SIGKILLed; any whose cgroups do not empty within the post-kill timeout (default 5 s) become Abandoned (leaked), and shutdown continues regardless.
7. Save the random seed. With every service stopped and before any filesystem is touched, peinit writes a fresh seed drawn from the kernel's CSPRNG to /var/state/peinit/random-seed — this is the entropy the next boot starts from. The write is crash-conscious (written to a temporary file, flushed, then atomically swapped in), and it is best-effort: if it fails, peinit logs it and carries on. Shutdown never stalls on it, and the forced and Critical-reboot paths skip it entirely in favour of getting the machine down fast.
8. Unmount filesystems. peinit snapshots the mount table and tries to unmount every remaining non-root mount in its namespace — not just the ones it mounted itself — working from the deepest mount points outward. That includes the Phase 1 set (/proc, /sys, /dev, /dev/pts, /dev/shm, /run, /sys/fs/cgroup) wherever those are still mounted. A mount that is already gone counts as done. If one won't unmount because it is busy, peinit falls back to remounting it read-only; anything it still can't clean up is logged and left for the final sync() and the kernel, never blocking shutdown. The root filesystem (/) is never unmounted — once the rest are handled, peinit remounts root read-only.
9. Sync and finish. sync() to flush pending writes, then the final action — power off, reboot, or halt.

The reverse ordering falls straight out of the dependency graph; there is no separate stop-order configuration. Shared platform services stop after their dependents, with registryd last. The exact order is derived from the installed graph.

STOPPING=1 and timeout extension #

A service that begins shutting down on its own — say, in response to an internal error — can tell peinit by sending STOPPING=1 via sd_notify. peinit acknowledges it and, importantly, stops sending SIGTERM to that service: it is already on its way down, and a redundant signal could interfere.

STOPPING=1 is a courtesy, not a timeout extension. The StopTimeout keeps running; if the process has not exited when it expires, peinit escalates to SIGKILL regardless. A service that genuinely needs longer must say so with EXTEND_TIMEOUT_USEC (see Keeping services running) — and during shutdown those extensions are capped not only by the per-service StopTimeout × 4 but also by the remaining global ShutdownTimeout, whichever is smaller.

Forced shutdown #

When the machine must be stopped despite a stuck graceful shutdown, repeated SIGINT forces it: three Ctrl-Alt-Del presses within five seconds skip the graceful sequence entirely — SIGKILL everything, sync, reboot. It is the escape hatch for a graceful shutdown that is itself stuck behind an unresponsive service.

Shutdown during boot #

A shutdown requested while peinit is still in Phase 2 boot takes effect immediately: services still Starting are SIGKILLed (they never reached Active, so there is nothing to stop gracefully), services that did reach Active are stopped per the normal sequence, and the boot is abandoned.

How peinit handles signals #

peinit handles all signals through a signalfd read from its event loop — every signal is blocked and read as data, so there are no async signal handlers and no async-safety hazards. PID 1 cannot be killed by any signal; the kernel protects it.

All other signals are ignored.

Where to start #

To understand the reverse ordering and which relationships gate it, read Dependencies and ordering.

To understand the Critical-failure path that bypasses graceful shutdown, read Keeping services running.

For the boot side of the lifecycle and the reboot/recovery escalation, read Boot and boot modes.

Use this catalog when you already know which setting you need. For a guided change, start with Defining a service, then use svctl definition show and svctl definition validate to inspect and check it.

The tables retain the service-definition schema and every registry key peinit reads or writes. Follow each row’s link for validation rules, safety notes, and when a change takes effect. Saving a correctly typed value does not mean a running process has adopted it.

Registry value types #

Service definition fields map onto registry value types as follows:

The service definition schema #

Every service is a key under Machine\System\Services\<name>; these are the values inside it. Only ImagePath is required. Fields are grouped by purpose; for full semantics see Defining a service and the linked pages.

Execution — see The execution environment

Type and readiness — see Simple and Oneshot services

Activation — see Triggers and timers

Identity — see Service identity and privileges

Dependencies — see Dependencies and ordering

Supervision and health — see Keeping services running

Transition phases — see The service lifecycle and Controlling services

Notify, fds, security, metadata

Service definition keys #

Boot configuration #

Under Machine\System\Boot\:

Operational parameters #

Under Machine\System\Init\:

Keys peinit reads from other subsystems #

LogSocketPath is read by peinit. MaxLogDatagramBytes is included for related eventd configuration; it is not a peinit input.

Where to start #

For how these fields combine into a working definition and when changes take effect, read Defining a service.

To look up the meaning, valid range, and effect-timing of any key on a live system, use regman.