Peios System Protocols Userspace

This document defines the Peios System Protocols Userspace (PSPU): the protocols and interchange formats by which the foundational userspace components of a Peios system agree with one another.

An interface belongs in this document when both of these hold:

• it is a contract between userspace parties, at least one of which is a component the system is built from rather than an application running on it; and
• the interface is public — a third party is expected to implement one side of it.

The parties need not exist at the same moment. A live protocol has two processes in conversation; an interchange format has a producer and a consumer that never meet, and the artifact between them carries the contract. Both are in scope, because what makes something belong here is that two independently written parties must agree on it.

1.1.1 These protocols are not conformance requirements #

A system that does not offer a protocol in this document is still Peios. The components that speak these protocols are one answer to a problem, not the definition of the platform; a system that solves the same problem with different components conforms exactly as well.

They are specified because they are public even so. A third party writing a component to plug into one side of one of these protocols needs the contract written down, and needs it to stay put. What they are not is a bar anyone must clear.

For each interface, this document covers:

• for a live protocol: the channel, its direction, which party connects to which, message framing and encoding, the messages exchanged, and the shape of a conversation
• for an interchange format: the layout of the artifact, how it is identified and versioned, and how a consumer validates one it receives
• how a party announces itself or is identified, and how its counterpart establishes what it is and what it may speak for
• the rules under which the format may be extended
• what each party must declare about itself, and what its counterpart validates rather than believes
• the conformance requirements for each role

This document does not cover:

• Standards a system MUST implement to be Peios — defined in PGSS
• Protocols spoken across the kernel boundary — defined in PSPK
• The binary structures these interfaces carry — defined in PCDS
• How a component stores its data, reaches the answers it gives, or produces the artifacts it emits — its own design
• Which counterparts a system is configured to trust, and how that configuration is expressed — the consuming component's own design
• Administering a component's contents — its own design

The fourth of those is the point of the whole document. A component is asked a question and gives an answer, or is asked for an artifact and produces one; how it arrives there is exactly what different components exist to do differently.

1.1.2 Stability #

Publication here is a commitment that the contract is written down and will not change out from under an implementation. Each specification states its own rules for extending its wire or file format; those rules are the supported way for an interface to grow.

The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY in this document are to be interpreted as described in RFC 2119. Text set off as a note is informative, not normative.

Everything else — roles, byte order, sizes, layout tables, notation, strings, timestamps, citation, and the external standards this anthology depends on — is defined in the Conventions book and is not restated here. PSPU departs from none of it.

Where a chapter needs a convention of its own, that chapter states it.

This chapter specifies the Principal Source Interface (PSI): the protocol by which an authentication authority federates identity to separate processes that hold it.

Two roles participate.

The authority is the process that mints tokens and creates logon sessions. It is the party asking. On PSI it listens; it never dials out (§2.3). It is also the party that speaks PGSS Logon to its clients, and PSI exists so that it can answer them.

The source, in full a principal source, is a process that is authoritative for some set of principals: it verifies their credentials and says who they are. The source role is publicly implementable: any process an authority has been configured to accept MAY register as a source, and a third party writing one for a directory, a hardware token service or an identity provider is the case this chapter is written for. A conforming source is the subject of the source obligations in §2.21.

A source is not a store, necessarily. A local source owns its bytes; a directory-backed source owns nothing and forwards the question. The interface deliberately does not distinguish them, which is why the term is "source" rather than "store".

What a source is emphatically not is a component of the authority. It runs as a separate process, at lower trust, and it cannot mint anything (§2.4).

This chapter covers:

• the channel, its direction, and why sources connect inward (§2.3, §2.6)
• message framing, the conversation identifier, and the rules under which the format may be extended (§2.7)
• registration: how a source announces itself, how the authority establishes what it is, and what domain it may speak for (§2.8 to §2.10)
• the relayed interrogation, and its relationship to PGSS Logon (§2.5, §2.12)
• assertion and refusal, the terminal messages of a source conversation (§2.13)
• querying a source outside a logon, so that an authority can serve PGSS Logon's identity lookup (§2.15, §2.16)
• what a source must declare about itself before an authority may cache its answers (§2.8, §2.17)
• scope: what a source may claim about identity, separately about membership, and separately again about POSIX identifiers (§2.18 to §2.20)
• the obligations binding on each role (§2.21)

This chapter does not cover:

• The logon protocol itself, specified in PGSS.
• Tokens, SIDs, sessions and privileges — described in the Peios Kernel TRM, with SIDs, security descriptors and claim attributes specified in PCDS.
• How a source stores identity, or verifies a credential.
• Derivation — what a token ends up containing — which is the authority's, applying local policy (PGSS §2.1).
• Which sources a machine trusts, and how that is configured.
• Which identifier range a source is given, and how that is configured. The rules the assignment must satisfy are §2.20.
• Administering a source's contents.

The third of those is the point of the whole interface. A source is asked a question and gives an answer; how it reaches the answer is exactly what different sources exist to do differently.

2.1.1 PSI is not a conformance requirement #

PGSS Logon is a Peios Generic System Standard: a system that does not offer it is not Peios. PSI is not. It is the interface an authority uses to reach the processes that know who exists, and a system running entirely different authentication infrastructure is still Peios.

It is specified because it is a public interface even so. A third party writing a principal source needs the contract written down, and needs it to be stable. What it is not is a bar anyone must clear.

Terms defined in the Peios Kernel TRM (token, logon session, privilege), in PCDS (SID, security descriptor, claim attribute) and in PGSS (authority, client, principal, conversation, round, credential material, prompt, derivation) are used here with the same meaning and are not redefined.

Source. A process that is authoritative for some set of principals: it verifies their credentials and says who they are. Called a principal source in full.

Registration. The exchange in which a source announces itself and the authority decides whether to accept it. Precedes any conversation.

Domain. The SID namespace a source is authoritative for. Every principal a source may assert lives under it. See §2.10.

Source conversation. One logon's exchange between the authority and a source, distinguished from other concurrent ones by a conversation identifier. Not to be confused with a PGSS Logon conversation, which is between a client and the authority; one of each exists per logon.

Assertion. A source's terminal message stating who a principal is. The only successful outcome a source can produce.

Originator. The verified identity of the process that requested a logon, as established by the authority from the client's connection. Relayed to the source, which cannot learn it any other way.

Service SID. A SID derived from a service's name, placed in that service's token by the init system, and unforgeable by anything else. How a source's identity is established (§2.9).

Membership scope. The constraint on which groups a source may assert. Separate from identity scope, which constrains whose identity it may assert at all. Sections 2.18 to 2.20 exist because these are different questions with different answers.

Relative identifier. In this chapter, a POSIX identifier as a source states it: an offset within the range the authority assigned that source, never an absolute number (§2.20). Where the SID sense is meant — the last sub-authority of a SID — the text says so.

The authority listens. Sources connect to it. The authority never initiates a connection to a source.

This is the most consequential shape decision in this chapter, and it is worth being explicit about what it buys.

2.3.1 Why the direction matters #

The authority holds the privilege to mint tokens. It is the most privileged userspace process on the system. An authority that dialled out would need, in its configuration, a list of paths to connect to — and a process holding that privilege having a configurable list of things to go and talk to is a liability out of proportion to the convenience.

Because sources connect inward, the authority's sockets are accept()-only. It never opens an outbound connection to anything, for any reason.

2.3.2 What follows #

Restart is the source's problem. A source whose connection drops reconnects. The authority does not retry, does not queue, and does not track sources it has not heard from. A source that has gone away is simply not registered.

A source is not required to exist. An authority with no registered sources cannot authenticate anybody, and that is a coherent state rather than an error — it means no identity has been made available to it yet.

Ordering is the init system's problem. The authority must be listening before a source can register, and a service that depends on authentication must start after a source has. Expressing that is a service-ordering question, not a protocol one, and this chapter says nothing about it.

A source says who somebody is. It cannot say anything else, and the protocol is built so that this is structural rather than a rule anyone has to remember.

2.4.1 The success terminal #

PGSS Logon's success terminal is AccessGranted, carrying a session identifier and a token descriptor. If PSI reused it, sources would be minting sessions.

PSI's success terminal is Assertion (§2.13), which carries an identity: a SID, a canonical name, and group memberships. There is no session identifier and no descriptor to attach a token to. A source could not mint one if it wanted to, because there is no message in which to say so.

That is the whole of the mechanism. No capability check, no trust level, no configuration flag — a source cannot mint because the protocol gives it no way to express minting.

2.4.2 What the authority keeps #

Everything else:

• Derivation. What the token actually contains — its privileges, its integrity level, its derived group memberships, its projected identifiers — is the authority's, applying local policy (PGSS §2.1).
• Session creation. The logon session, and the record of which source vouched for it.
• Validation. Every SID a source sends is bytes until the authority has checked it (§2.13).
• Scope enforcement. What a source is permitted to claim (§2.18 to §2.20).
• Peer verification. On every connection it accepts.
• Rate and round limits, and the policing of what a source may ask a client for (§2.12).

2.4.3 Why a compromised source is bounded #

A source that is entirely compromised can lie about the principals in its own domain. It cannot mint a token, cannot elevate anyone's privileges, cannot claim identities outside its domain (§2.18), and — unless configured otherwise — cannot assert memberships outside it either (§2.19).

That bound is the reason for the process boundary. It is not that sources are expected to be malicious; it is that a source is the component parsing credentials from the outside world, and therefore the one most likely to be wrong.

A principal source is an authentication authority for its slice of the world. The authority that federates is an authority over authorities. Once that is seen, most of PSI writes itself.

2.5.1 The relationship #

PSI's interrogation phase is PGSS Logon's, with identical message bodies. CredentialRequest and CredentialResponse carry exactly the bytes PGSS §2.8 defines, and the authority relays them nearly verbatim in both directions.

The consequences are worth stating plainly:

• The source decides what to ask for. Not the authority. The authority does not know what credentials a source requires, and does not need to.
• The authority becomes a relay in the interrogation phase. It is a shorter path than synthesising its own prompts, not a longer one.
• Adding a credential type is a change to sources, not to the authority and not to clients, which already render what they are given (PGSS §2.3).
• A source could be tested in isolation by pointing a PGSS Logon client at it, for the interrogation phase at least.

2.5.2 Where they diverge, deliberately #

Three differences, each for a stated reason.

The success terminal. Assertion rather than AccessGranted, so that a source cannot mint. See §2.4. This is the divergence that matters.

Multiplexing. PGSS Logon is one conversation per connection; the connection is the conversation. PSI carries many concurrent logons over one long-lived connection, so its header adds a conversation identifier (§2.7). The alternative — serialising every logon behind one connection — would make any slow logon a system-wide login stall.

Distinct magic. PPSI rather than PGSL. Two protocols this similar sharing a codec is a cross-protocol hazard: a socket plugged into the wrong daemon would partially work, which is far worse than failing outright. The magic makes it a hard error on byte zero.

The full accounting of what is shared, what is added and what differs is §2.C.

2.5.3 "Just relaying" is loose #

The authority is a relay in the interrogation phase only, and even there it is not passive. It polices what a source may ask a client for (§2.12), it validates what a source asserts (§2.13), and it enforces scope (§2.18 to §2.20). Everything before and after the interrogation is entirely its own.

2.6.1 Socket #

An authority that federates over PSI MUST listen on a SOCK_STREAM Unix domain socket.

Unlike PGSS Logon's path, this one is not normative. PSI is not a conformance requirement (§2.1), and an authority that offers it may put it where it likes provided its sources are told. Mainline's is /run/psi.sock.

2.6.2 Access control #

The socket SHOULD carry a security descriptor. It is DoS protection and nothing more, and an implementation MUST be written as though it were absent.

The reason is that the socket cannot be the boundary. What establishes a source's identity is the peer's token (§2.9), which is checked on every connection. A descriptor that kept casual traffic away would be a convenience; a descriptor relied upon would be a second, weaker access control that someone will eventually assume is doing the work.

An authority MUST therefore bound the number of unregistered connections it will hold open, and the time it will wait for a registration, independently of any descriptor.

2.6.3 Long-lived connections #

A source's connection persists for the life of the source and carries every logon routed to it.

An authority MUST bound the number of registered sources and the number of concurrent conversations per source. A source MUST bound the conversations it will track, and MUST NOT depend on the authority's bookkeeping to do it — a source that trusted the authority's limit would be trusting a bound it cannot verify.

2.6.4 Failure #

A framing error is fatal to the connection, not to a conversation. Once a message has failed to parse there is no way to know where the next one starts, so both parties MUST tear the connection down rather than attempt resynchronisation.

A failed write is likewise fatal. A partial write desynchronises the stream just as a bad frame does, and treating it as a per-conversation error would leave a corrupt connection in use.

Ordinary semantic failures — an unknown principal, a bad credential, a refused logon — are not connection failures. They are Refusal messages (§2.13) and the connection continues.

2.7.1 Header #

Every message begins with a 20-byte header:

The first twelve bytes are PGSS Logon's header, unchanged and at the same offsets. total_len in particular sits where PGSS §2.6 puts it, which is what lets one transport implementation frame either protocol off a stream.

2.7.2 Magic #

PPSI, checked on every message, fatal to the connection when wrong.

This matters more here than it would for an unrelated protocol, because PSI and PGSS Logon share message bodies (§2.5). A CredentialRequest from one is byte-identical to the other's. Without distinct magic, a socket plugged into the wrong daemon would decode several fields correctly before going wrong — which is the failure mode hardest to diagnose and easiest to miss.

2.7.3 Conversation identifier #

The conversation field distinguishes concurrent logons on one connection.

• Conversation 0 is reserved for connection-level messages: Register, Registered and Changed (§2.17). It MUST NOT be used for a logon or a query.
• Logon and query conversations use identifiers from 1 upward, drawn from one space.
• The authority allocates them. A source MUST NOT invent one, and MUST reply on the identifier it was given.
• An identifier is unique among live conversations on one connection. An authority MAY reuse one after a conversation has reached a terminal state.

A source MUST reject a message on a conversation it does not know, and MUST NOT treat it as opening a new one. Only Authenticate (§2.11), Query (§2.15) and EnumerateSource (§2.16) open one, and an authority MUST NOT open one with an identifier already live.

Rejecting means declining to act on it. A source MUST NOT reply on a conversation it does not know: an authority MAY have reused the identifier after a terminal state, so a reply could arrive as a second terminal message for a conversation that has already ended. Discarding it, and recording that it happened, is the whole of the obligation.

A source MUST likewise refuse an Authenticate, Query or EnumerateSource arriving on conversation 0, which is reserved.

2.7.4 Size limit #

A message MUST NOT exceed 81920 bytes — larger than PGSS Logon's ceiling, because a PSI message wraps one.

2.7.5 Message direction #

The high bit of msg_type marks a message sent by the source. This follows PGSS Logon's convention that the bit marks the authority for the matter at hand: on this interface the source is the authority for its own principals, and the logon authority is the one asking.

2.7.6 Encoding #

PSI shares its codec with PGSS Logon, and PGSS §2.6's encoding rules apply here unchanged: little-endian multi-byte integers; UTF-8 strings, length-prefixed and never NUL-terminated; length-framed structures and array elements, skipped to their declared end; u32 element counts on arrays, with stated maxima binding on encoder and decoder alike.

2.7.6.1 SIDs on the wire #

SIDs are carried as opaque bytes, in the binary self-relative form PCDS specifies, never as text.

They are opaque to the codec, which has no business knowing what a SID is. They are emphatically not opaque to the authority, which MUST validate every SID it receives before treating it as identity (§2.13). The obligation to check sits in the process that mints tokens, not in the layer that moves bytes.

A SID MUST NOT exceed 68 bytes — the eight-byte prelude plus fifteen sub-authorities, which is the most the encoding's one-byte count admits.

2.7.7 Body extensibility #

The extensibility rules of PGSS §2.6 apply unchanged: fields are appended only, a new field is optional with a safe default, and a new enumeration value is a breaking change requiring a version bump. The one exception is the capability bitmask of §2.8, for the reason given in §2.B.

One PSI-specific application deserves stating. Authenticate (§2.11) nests a whole LogonStart inside its own length frame rather than inlining its fields. The obvious encoding — LogonStart's fields, then PSI's — is wrong: LogonStart belongs to PGSS Logon and grows on PGSS Logon's schedule, so a field appended there would silently displace the field after it. Nesting lets the two evolve independently.

The same reasoning applies to every shared body PSI carries, and §2.C lists them.

A connection opens with registration. Nothing else may precede it.

2.8.1 Register #

msg_type = 0x8001. Source to authority, on conversation 0.

2.8.1.1 source_name #

What the source calls itself. Bounded at 32 bytes because it becomes the authentication-package name on every session the source authenticates — so a token's provenance answers which source vouched for this? rather than merely the authority minted it.

The name is a claim. It MUST be cross-checked against the identity the authority established for itself (§2.9), and it MUST NOT be used for anything else. A mismatch MUST be refused rather than quietly corrected: a service registering under another's name is worth failing on, not normalising.

2.8.1.2 domain #

The SID namespace this source is authoritative for (§2.10).

2.8.1.3 capabilities #

What the source can do beyond authenticating.

An authority MUST NOT send a message a source did not declare it answers, and MUST NOT set a field bit gating a capability the source did not declare. A source declaring nothing authenticates and does nothing else, which is what a source predating these fields is saying by omission — and is the only reading that keeps such a source working.

A source that declares no QUERIES cannot be asked about its principals outside a logon, so they are unresolvable through PGSS Logon's identity channel: they can sign in and will appear as bare numbers everywhere else. That is a coherent configuration and this chapter permits it, but it is almost never what an administrator intended, and an authority SHOULD report it where one will see it — as it reports a source registering with no identifier range.

This is a declaration of capability, not of willingness. A source declaring QUERIES may still answer Refused to any particular question (§2.15); a source declaring ENUMERATES may still refuse a cursor it can no longer honour (§2.16).

2.8.1.4 entry_ttl #

How long, in seconds, the authority may hold an answer from this source before asking again.

Zero means do not cache. A source declaring neither PUSHES_CHANGES nor a non-zero entry_ttl has said its answers must not be held at all, and an authority MUST honour that — see §2.17, where the reasoning for reading silence that way is set out.

entry_ttl and PUSHES_CHANGES are not exclusive, and a source declaring the second SHOULD declare a non-zero first as well. See §2.17: the TTL is the backstop against a notification that was never sent.

2.8.1.5 max_batch #

The largest number of keys the source will accept in one Query (§2.15). Zero means one.

An authority MUST NOT exceed it, and MUST NOT send more than 64 keys whatever the source declared: an encoder MUST NOT declare more, and a decoder MUST read a larger declaration as 64.

A source MUST still validate what it receives. The field is a hint the authority is required to respect, not a guarantee about what will arrive.

2.8.2 Registered #

msg_type = 0x0001. Authority to source, on conversation 0.

Load-bearing beyond its contents. A source SHOULD report itself ready only once it has received this, so that anything ordered after the source finds a system that can actually authenticate rather than merely a process that exists (§2.3).

2.8.2.1 unix_id_base, unix_id_count #

The POSIX identifier range the authority has assigned this source (§2.20). A base of 0 means no range was assigned, and every principal the source asserts will project as unmapped.

Informational. A source counts within its range and asserts relative identifiers; the authority applies the base. A source MUST NOT apply it — see §2.20, where the reasoning is set out in full.

It is sent so that a source's administration tools can show an operator the identifier a principal will really project to, rather than the relative number the source stores. Without it that arithmetic falls to the operator.

An authority that predates these fields sends neither, and a source MUST read their absence as no range assigned — which is what such an authority means, since it has no ranges to assign.

2.8.3 Rules #

1. A connection MUST open with Register on conversation 0. An authority MUST refuse any connection that opens otherwise.
2. An authority MUST bound the time it waits for the opening Register, and close the connection on expiry. A peer that connects and says nothing MUST NOT be able to hold resources indefinitely.
3. An authority MUST NOT admit two sources under one name at the same time.
4. An authority MUST send Registered only after the source is routable, so that a logon racing the acknowledgement cannot find a source that is registered but not yet reachable.
5. A source MUST NOT send any other message before receiving Registered.
6. An authority SHOULD report, where an administrator will see it, that a source registered with no identifier range — its principals will all project as unmapped, and the cause is a configuration omission rather than anything the source did.

The authority MUST establish a connecting source's identity for itself, from the kernel, and MUST NOT take it from anything the source sends.

2.9.1 A source proves nothing #

The mechanism worth recommending is that a source proves nothing at all, because the init system already did.

Where the init system places a service SID in each service's token — a SID derived from the service's name, which only the init system can mint — the authority can:

1. take its list of permitted source names from its own configuration;
2. derive the service SID each of those names implies;
3. read the connecting peer's token and ask which of those SIDs it carries.

The resulting identity is assembled entirely from the authority's configuration and the kernel. Nothing is contributed by the process on the other end. There is no shared secret, nothing to provision, nothing to rotate, and nothing to steal — the derivation is a pure function of a name that only the init system can act on.

2.9.2 What is deliberately not checked #

That the peer is SYSTEM. The service SID subsumes it: the user SID could never distinguish one platform service from another, since they all run as the same principal. Requiring SYSTEM as well would needlessly forbid a future source running under a lesser account, which is a direction worth keeping open.

2.9.3 The allowlist #

An authority MUST NOT accept a source it has not been configured to accept.

An empty configuration MUST mean no source may register, not any source may. An allowlist that fails open is not an allowlist. The visible cost is that a system configured with no sources cannot authenticate anyone, which is the correct way for that mistake to present — loudly, at the first logon attempt, rather than silently at the first compromise.

A source declares the domain it is authoritative for. Every principal it may assert lives under it (§2.18).

2.10.1 It is a claim #

A source generates or is given its own domain, so nothing about the SID can prove the claim is honest. What gives it weight is entirely what the authority does with it.

An authority MUST apply all of the following.

2.10.1.1 1. A domain MUST be declared #

A source that declares no domain MUST be refused. There would be nothing to confine its assertions to, which is the whole purpose of collecting one.

2.10.1.2 2. The shape MUST be checked #

A domain MUST be a well-formed, locally-issued domain SID: revision 1, the NT authority (5), the non-unique prefix 21, and three further sub-authorities — S-1-5-21-A-B-C, four sub-authorities in total.

The shape is the entire check, and it is enough. A source cannot claim S-1-5-32 (BUILTIN), or the NT authority's well-known range, or S-1-1-0, because none of them has that shape. There is no list of forbidden domains to keep in step with the SID catalogue — the permitted shape excludes every one of them by construction.

2.10.1.3 3. Domains MUST be disjoint #

No two concurrently registered sources may claim the same domain. Two authorities for one namespace means whichever answers first decides who a name belongs to, and the other's principals become impersonable by the first.

2.10.1.4 4. A source MUST NOT change domain #

A source that re-registers MUST declare what it declared before. An authority MUST refuse a change.

This is the check that survives a source restarting, and it is worth its cost: every other check passes for a source that is killed and comes back compromised. It still holds the right service SID, its new domain is still a claimable shape, and with itself deregistered there is nothing left to collide with.

An authority MAY hold this record only for its own lifetime. Persisting it means the authority writing state, which is a larger commitment than the guarantee justifies.

2.10.1.5 5. An administrator MAY pin #

An authority SHOULD allow an administrator to configure the exact domain a named source must declare, and MUST refuse a source declaring anything else when one is configured.

A configured pin that cannot be parsed MUST NOT be treated as absent. Absence means no pin; an unparseable value means an administrator tried to apply the control and got it wrong, and silently downgrading that to "unconstrained" removes the control at the moment it was being applied. An authority MUST fail towards refusing the source.

2.10.2 What remains uncovered #

With no pin configured, and another source not currently registered, a compromised source could declare that source's domain and assert its identities. Disjointness catches it only while both are registered.

This is stated rather than solved. Closing it requires someone to write the pin down, and an authority cannot invent that authority for itself — writing it automatically would mean the process holding the token-minting privilege also holding a configuration write handle, which is a worse trade than the gap it closes.

msg_type = 0x0002. Authority to source. Opens a conversation.

2.11.1 start #

The client's LogonStart, nested whole (§2.7). Its fields and their meanings are PGSS §2.7's, unchanged — including that identifier is an unverified claim and supported_credential_types binds what may be prompted for.

2.11.2 originator #

The verified identity of the process that requested this logon, taken by the authority from the client's connected socket and never from a message body.

A source cannot learn this for itself: it is not party to the client's connection, and there is nothing it could ask. The authority relays it because a source may legitimately refuse a logon on the strength of it — an account restricted to console logons needs to know what asked — and that decision needs a trustworthy input.

A source MUST treat originator as established fact and MUST NOT treat any other field of this message the same way.

2.11.3 Routing #

Before sending Authenticate, an authority MUST decide which single source answers.

The credential MUST NOT be offered to more than one source. Trying each in turn with the password hands every source the credentials of every other source's users, including on typos — the failure PAM stacking exemplifies (§2.D).

Resolution therefore happens on the identifier, before any credential exists. Asking several sources "do you own this name?" is a resolution step with no secret in it and is permitted; offering them the answer is not.

An authority SHOULD resolve a qualified name to its owning source and MUST NOT fall back to another source when the owning one is unreachable. A name that can fall through lets anyone who can break a network choose which authority answers for a principal.

2.11.4 Conversation limits #

An authority MUST bound the conversations it opens against one source. A source MUST bound what it will track, and MUST refuse beyond its own limit with AuthorityUnavailable (§2.13) rather than dropping the conversation silently.

Two messages, both carrying PGSS Logon bodies verbatim.

2.12.1 CredentialRequest #

msg_type = 0x8002. Source to authority. Body is PGSS §2.8's CredentialRequest, byte-for-byte.

The source decides what to ask for, in what order, and over how many rounds. The authority relays it to the client.

2.12.2 CredentialResponse #

msg_type = 0x0003. Authority to source. Body is PGSS §2.8's CredentialResponse, byte-for-byte.

2.12.3 What the authority polices #

The authority relays, but does not relay anything.

An authority MUST refuse to relay a prompt whose credential type is absent from the client's supported_credential_types. PGSS §2.8 makes this the authority's obligation towards the client, and it holds however the authority reached the prompt — a prompt originating in a source is still the authority's to police.

Relaying it would force the client to hard-fail, and a client that guessed instead might echo a secret to the screen. The authority MUST terminate the logon instead.

An authority MUST also enforce its own round and time limits on the relayed exchange (PGSS §2.3), independently of any the source applies. A source that never terminates a conversation MUST NOT be able to hold a client's logon open indefinitely.

2.12.4 Credential handling #

The obligations of PGSS §2.12 bind both parties on this leg as they do on the client's. Credential material reaching a source has been decoded and re-encoded once more than it would have been without federation, and every buffer it passed through on the way is one the obligation covers.

2.12.5 What the authority does not do #

It does not interpret prompts, rewrite messages, reorder anything, or synthesise a request of its own. A source's prompt reaches the client as the source wrote it, which is the property that makes adding a credential type a change to sources alone.

Exactly one terminal message ends a source conversation.

2.13.1 Assertion #

msg_type = 0x8003. Source to authority. The only successful outcome a source can produce.

Note what is absent: no session, no token, no privileges, no integrity level. A source has no way to express them (§2.4).

Every field after groups is optional in the way §2.7 requires: a source that does not write one has said nothing about it, and the authority substitutes the default named below rather than failing.

2.13.1.1 canonical_name #

The source's own spelling of the principal's name. A client may have typed JACK; this is what the principal is actually called.

Carrying it is what makes case-insensitive matching safe: the authority records the canonical form rather than whatever was typed, so a session's records do not vary with a caller's shift key.

A source MUST NOT assert a name that PGSS §2.15 forbids — one carrying a reserved character, a byte outside the printable ASCII range, or a leading or trailing space. The authority MUST refuse one anyway (§2.21), because a name from a source reaches a passwd-format record and an audit line, and by then the damage is the reader's to do.

The obligation is on what a source asserts, not on what it creates. A source that validates a name when an administrator adds it, and not when it reads one back from storage, has enforced nothing against a store it did not itself write.

2.13.1.2 groups #

Each entry is a separate length-framed structure:

A SID and a number. No attributes.

A source asserts which groups a principal belongs to. Whether a group entry is enabled, owner-marked, or deny-only is a decision about how to build a token, and building tokens is the authority's (§2.4). A source saying "this principal is an administrator" is identity; a source saying "and mark that group deny-only" would be reaching into derivation.

The per-entry framing is what allowed unix_id to be added here without breaking a decoder that predates it, and it will allow the next field the same way.

A unix_id of 0 means the source does not number this group — the honest answer for a group it does not own. A source naming a well-known group is stating a membership, not claiming authority over what that group projects to; see §2.20.

2.13.1.3 unix_id #

The principal's POSIX identifier, relative to the range the authority assigned this source (§2.20). Zero means the source has no number for this principal.

A source MUST NOT apply its own base. It counts within its range and the authority rebases; a source that added the base itself would have it added twice.

2.13.1.4 primary_group #

Which of the principal's groups projects to the POSIX group id, and becomes the default group of objects the token creates. Empty means the source did not say, and the authority chooses.

It need not appear in groups. The authority is required to place it on the token regardless (§2.21), because a token's primary group must be a group the token carries — so naming a group here is a membership claim, and it is subject to membership scope exactly as a listed group is (§2.19).

That applies to a primary group the source asserted. Where the field is empty and the authority substitutes one of its own, the substituted value is the authority's choice and MUST NOT be tested against the source's membership scope. Testing it would deny every logon from a source that declined to name a primary group, on the strength of a claim that source never made — and an authority's own default is very unlikely to be a sibling of the principal's domain, so the test all but always fails.

2.13.1.5 profile #

PGSS Logon's profile structure (PGSS §2.9), relayed onward to the client unchanged. It is not identity, it decides no access, and the authority does not interpret it.

The one thing the authority does check is the one PGSS §2.9 requires of it: home and shell, when non-empty, are absolute paths. That obligation binds the authority towards its client whatever the value's provenance, so a relayed profile is not exempt from it.

2.13.1.6 claims #

Named, typed attributes fed to conditional ACE evaluation, in the claim attribute format PCDS §5.9 specifies. Each entry is a separate length-framed structure:

A claim is the one field here that is a trusted input to access decisions rather than a statement of identity: a conditional ACE can turn a claim into a grant. Which claim names a source may assert is therefore the same kind of question as which groups it may assert, and belongs with membership scope (§2.19).

An authority MUST reject an assertion carrying a claim it cannot carry to a token — an unsupported value type, a name containing an interior NUL, a value exceeding its limit — rather than dropping the claim. The reasoning is rule 3 below: a dropped claim signs the principal in against a policy nobody stated.

2.13.1.7 What the authority MUST do with an assertion #

1. Validate every SID with a structural check before treating it as identity. Bytes from another process are bytes until checked, and the check belongs in the process that mints tokens rather than in the codec that moved them (§2.7). This includes SIDs carried inside a claim value.
2. Enforce identity scope (§2.18), membership scope (§2.19) — including over primary_group — and numeric scope (§2.20).
3. Fail the logon on a malformed group SID, an unusable claim, an invalid canonical_name, or a unix_id outside the source's range, rather than dropping it. Dropping would sign the principal in with authority the source did not state — a confusing way to be wrong at best, and for a claim, a silent change of the policy that will be applied to them. A source that cannot encode a SID is broken.
4. Drop a logon SID from the asserted groups, loudly. No source is authoritative for one: the kernel mints them per session, and this session's did not exist when the source answered. A source asserting one is either buggy or reaching for a different session's SID, which would forge membership of somebody else's logon.
5. Drop duplicates, first mention winning. A source asserting a group the authority also derives is redundant, not wrong.

Rules 4 and 5 drop rather than refuse because the token still ends up correct, and refusing would punish a principal for a source's defect without making anything safer. Rule 3 refuses because the token would not end up correct.

Rules 4 and 5 run before rule 2. A logon SID is by construction outside the principal's domain, so an authority that applied membership scope first would refuse the logon that rule 4 says to survive by dropping. Duplicates are the same shape of problem. The drops are about what a source should never have sent; the scope tests are about what it is permitted to claim, and only what survives the first is subject to the second.

An authority MUST also place primary_group on the token even when the source did not list it among groups, since a token's primary group must be a group the token carries.

2.13.2 Refusal #

msg_type = 0x8004. Source to authority.

Reuses PGSS Logon's denial vocabulary rather than inventing a parallel one, so that relaying a refusal outward needs no lossy translation.

A source MUST NOT distinguish an unknown principal from a bad credential — by code, by reason, or by timing (PGSS §2.10, §2.12). The obligation is the source's here, because the source is where the distinction exists to be leaked.

An authority MAY relay reason to the client and MAY replace it. It MUST NOT relay a reason that reveals a distinction the source was required not to make.

msg_type = 0x0004. Authority to source. No fields beyond the header.

Tells a source that a conversation will not continue: the client hung up, a limit was reached, or the authority terminated the logon for reasons of its own.

2.14.1 Why it exists #

Without it, a client that disconnects mid-prompt leaves the source holding conversation state forever. A source cannot detect this for itself — it is not party to the client's connection — so the authority has to say.

2.14.2 Rules #

1. An authority MUST send Abandon for any conversation it opened that will not reach a terminal state, unless the connection itself is being torn down.
2. A source MUST discard all state for the conversation on receipt, and MUST NOT reply.
3. Abandon is not a terminal message from the source, and no Assertion or Refusal follows it. If one arrives anyway, the authority MUST ignore it — the conversation is gone, and a late answer to an abandoned question is at best stale.
4. A source that receives Abandon for a conversation it does not know MUST ignore it. It has already cleaned up, which is the outcome the message wanted.

An authority serving PGSS Logon's identity lookup must be able to ask a source about a principal outside a logon: to render a name for a SID, or a POSIX record for a number.

A query is a conversation like any other. The authority allocates the identifier, Query opens it, and one terminal message closes it.

2.15.1 Query #

msg_type = 0x0005. Authority to source. Opens a conversation.

A key entry is a length-framed structure:

Exactly one of name, sid and relative_id is meaningful, selected by key_type. An encoder MUST leave the others empty or zero.

2.15.1.1 Keys are never absolute #

There is no key type carrying an absolute POSIX identifier, and this is the load-bearing property of the message.

PGSS Logon's lookup accepts one, because that is what getpwuid hands a name resolver. The authority resolves it: it locates the range containing the number, subtracts the base, and asks the owning source by relative identifier.

A source is therefore never asked an absolute number, exactly as it never asserts one during a logon (§2.20).

The reason is not that an absolute number would let a source escape its range — it could not, because the authority refuses a relative identifier at or past the count before adding anything to it (§2.20). The reason is that the arithmetic must exist in exactly one place. A source asked an absolute number would have to subtract its own base to answer, which is the operation §2.20 forbids it, and an authority that asked would have taught it that its stored numbers and the system's are the same numbers. Every subsequent bug in that source would be an off-by-a-base.

2.15.1.2 Batching #

keys is an array so that an authority may ask several questions in one exchange. An authority MAY send a single key, and one that always does is conforming.

The array is here from the outset because adding it later would break every source written against a single-key message. A source with a cheap local store gains little; a source backed by a remote directory gains the difference between one query and a hundred.

A source MUST answer every key it is sent, in order, and MUST NOT reorder, merge or omit results. A source that cannot serve a whole batch MUST refuse the conversation rather than answer part of it.

2.15.2 QueryResult #

msg_type = 0x8005. Source to authority. Terminal.

One result per key, in the order the keys were sent.

A result entry is a length-framed structure:

Everything from present onward is PGSS §2.16's structure, unchanged — the same reuse of message bodies as the interrogation phase (§2.5), and for the same reason: an authority relaying a source's answer outward should not have to translate it.

Where outcome is not Found, everything after it MUST be empty or zero.

2.15.2.1 canonical_name, not qualified #

A source returns its own spelling of the name, as it does in an Assertion (§2.13). It does not qualify it.

Qualification names which source answered, and a source cannot know what it is called in another authority's search order. PGSS Logon requires a qualified name on the way out (PGSS §2.15); producing it is the authority's.

2.15.2.2 Identifiers are relative #

Every identifier in a result — a UNIX_ID field, a reference's unix_id — is relative, exactly as in an Assertion (§2.20). A source MUST NOT apply a base, and the authority rebases before the number leaves it.

2.15.2.3 Outcomes #

A source may send only:

A source MUST NOT send Unavailable: it is answering, so nothing was unavailable to it. The authority produces that outcome when a source does not answer (PGSS §2.18), and a source claiming it would let a working source be recorded as a broken one.

A source MUST NOT send Malformed in a result. A message it cannot parse is a Refusal for the whole conversation (§2.13).

A Refused result is about the object, not about the caller, and an authority MUST NOT relay it outward as PGSS Logon's Refused outcome, which is reserved for a caller that may not make the request (PGSS §2.18). What a source declining to expose an object means to a client is the authority's to decide, and it is not "you lack permission".

A source that declared no QUERIES is a third case again. It has not answered and cannot be asked, so an authority MUST NOT record it as having answered NotFound: a source that was never consulted is not evidence that an object does not exist, and PGSS §2.18 forbids reporting NotFound on the strength of one. It contributes nothing to the search, and §2.8 says what an authority should do about the configuration that produced it.

2.15.3 Scope #

An authority MUST apply identity confinement (§2.18), membership scope (§2.19) and numeric scope (§2.20) to a QueryResult exactly as to an Assertion, and MUST validate every SID in one structurally before using it — including the SIDs of references inside a PRIMARY_GROUP, GROUPS or MEMBERS value, not only the sid of the result itself.

"Exactly as to an Assertion" is meant literally, and it is the sentence an implementation is most likely to satisfy by halves. An authority that confines the object a result names, while relaying the group references beside it unchecked, has left the whole of membership scope unenforced on this channel — and a source with no permission to assert a foreign membership can then report one through a name lookup that it could not report through a logon.

A query is not a weaker channel than a logon. A source that could name a principal outside its domain here would be able to make ls -l display another source's principals as its own — and, worse, could then be believed the next time something compared that name to a SID.

Query asks about objects the authority can already name. Enumeration asks a source to produce them: to fill a POSIX passwd or group table, or to page through a group whose membership will not fit in one answer.

A source is never required to enumerate. A source that declines is fully conforming, and this section exists as much to make declining safe as to make enumerating possible.

2.16.1 EnumerateSource #

msg_type = 0x0006. Authority to source. Opens a conversation.

kind MUST NOT be Any.

An empty of enumerates every object of kind the source holds. A non-empty of MUST name a group, and enumerates that group's members.

cursor is empty on the first request, and otherwise carries the next from the source's immediately preceding reply.

2.16.2 EnumerateResult #

msg_type = 0x8006. Source to authority. Terminal.

An empty next ends the enumeration. A non-empty next means there is more, even where entries is empty.

A source MUST size a page against the smaller of this chapter's message ceiling and PGSS Logon's, because the authority re-encodes what it returns into the latter (§2.A). Neither the 256-entry bound nor the message ceiling here prevents a page nobody can deliver.

A source that will not enumerate replies Refused, with entries and next empty. An authority MUST record it as a source that did not contribute, and MUST NOT retry it for the remainder of that enumeration — across pages as well as within one (PGSS §2.17).

Refused and an empty Found are different answers and MUST NOT be conflated. Found with entries and next both empty says there are none: the group exists and has no recorded members, or the source holds no objects of that kind. Refused says this source is not answering. A source that returns an empty Found where it means the second has told the authority a falsehood it cannot detect, and the authority will go on asking it — the non-retry rule above has nothing to attach to.

The cases most often got wrong, all of which are Refused and not an empty Found: a group whose membership the source will not expose, a key that names an object the source does not hold, a key that names a principal where a group was required, and a cursor the source can no longer honour.

2.16.3 Cursors belong to the source #

A cursor is opaque to the authority. The authority MUST NOT construct, parse or modify one; it relays what it was given.

A source MAY encode anything into a cursor, and MAY refuse one it no longer honours — a store rewritten underneath a half-finished walk is the ordinary case, not an exceptional one. A refused cursor is Refused, and the authority MUST NOT restart the enumeration on the source's behalf.

A source MUST NOT assume a cursor comes back on the same conversation, or on the same connection, and MUST NOT hold per-cursor state it is unwilling to discard.

The authority's own cursor, the one it hands its client, is its to construct — PGSS §2.17 requires it to reject one it did not issue, which it can only do for a cursor it made. A source's cursor travels inside it, not as it.

2.16.4 Why members are here and not only in Query #

MEMBERS is a field of Query (§2.15), so the common case — a small group, whose members fit alongside the rest of the record — costs one exchange.

A group whose membership will not fit is reported through PGSS Logon's TooLarge (PGSS §2.16), and this is where the caller is sent. Paging a membership through the mechanism that already pages is cheaper than a third message, and considerably cheaper than the alternative of a partial member list, which is a wrong answer rather than a smaller one.

2.16.5 Enumeration is not existence #

An authority MUST NOT use enumeration to determine whether a principal exists, and MUST NOT infer from a source declining to enumerate that the source holds nothing.

A directory-backed source able to answer any single question while quite unable to answer all of them is the expected case, not a degraded one.

An authority answering a name lookup for every process on the system will cache. This is the message that lets it.

2.17.1 Changed #

msg_type = 0x8007. Source to authority, on conversation 0. Unsolicited, and never answered.

sid is meaningful only for Object, and MUST be within the source's declared domain (§2.18).

Object scope MUST be used for a deletion as well as a change, and for a creation — an authority may be holding a cached NotFound for a name that now exists.

2.17.2 Obligations #

A source declaring PUSHES_CHANGES (§2.8) MUST send Changed before, or at the same time as, the altered answer becomes observable through Query.

Sending it afterwards leaves a window in which the authority's cache and the source disagree while both believe themselves current — which is indistinguishable, from the authority's side, from the notification never arriving.

A source MAY send All where it could have sent Object. Over-invalidation costs a query; under-invalidation costs correctness.

An authority MUST accept Changed at any time after Registered, including while conversations are open on the same connection, and MUST NOT reply to it.

An authority MUST treat the loss of a source's connection as All for that source. It does not know what changed while it was not listening.

A source declaring PUSHES_CHANGES SHOULD declare a non-zero entry_ttl as well (§2.8). The two are not alternatives: the TTL is the backstop against a notification that was never sent, or was sent and failed to write. Declaring PUSHES_CHANGES with a TTL of zero means a single lost notification leaves the authority holding a stale answer until the connection drops — and a source that tolerated a failed Changed write without tearing the connection down would have made that outcome reachable, which is one of the reasons §2.6 makes a failed write fatal.

2.17.3 Sources that do not push #

A source that does not declare PUSHES_CHANGES is conforming, and many cannot: a remote directory has no way to tell this machine that an account was renamed.

Such a source declares entry_ttl instead (§2.8), and an authority MUST NOT hold its answers beyond it.

A source declaring neither has said it cannot support caching, and an authority MUST NOT cache its answers at all. That is the safe reading of silence, and it is what a source predating this message says by omission.

An authority that does not cache at all satisfies this section trivially, and is conforming. The obligations here bind what an authority may hold, not whether it must hold anything.

2.17.4 What this does not carry #

Changed says that something changed. It does not say what it changed to.

Carrying the new value would make this a second, unsolicited path by which a source could assert identity — one arriving outside any conversation, with no key to check it against, and no logon in progress to refuse. An authority that believed it would have accepted an identity assertion it never asked for.

The authority discards what it holds and asks again through Query, where every scope rule in §2.18 to §2.20 applies.

A source may assert only principals within its declared domain (§2.10). No configuration lifts this.

An authority MUST refuse an Assertion whose user_sid does not lie within the domain the source registered for, and MUST terminate the logon. It MUST apply the same test to a QueryResult (§2.15) and to the sid of a Changed (§2.17).

2.18.1 Containment #

A principal SID lies within a domain when it is the domain's SID plus exactly one relative identifier.

Exactly one, deliberately. S-1-5-21-A-B-C-1000-1 is not a principal of S-1-5-21-A-B-C, and admitting it would let a source that owns one domain mint names in a nested namespace nobody agreed it owned. A domain SID is likewise not a principal of itself.

2.18.2 Why nothing lifts it #

A source that could assert identities outside its domain could hand out another authority's principals to anyone who satisfied its credential check.

The concrete case: a local source holds no domain credential. If it were unconfined, it could produce a domain administrator's identity for anyone who knew a local password. The domain's own authority would never be consulted and would have no way to know.

Confinement keeps a compromised source at "authority over its own domain" — which is what it already was — rather than "authority over everyone".

This is why identity scope and membership scope (§2.19) are separate settings rather than one. They are different questions, and only one of them has a legitimate exception.

Group membership is the question with a legitimate exception, and it needs one.

2.19.1 The default #

By default, an authority MUST refuse an Assertion carrying a group outside the domain of the principal being asserted.

A directory vouches for its own users and its own groups, and nothing else. Without this rule, a directory-backed source could declare its users members of BUILTIN\Administrators — making a remote authority the arbiter of who administers this machine.

The test is relative: are the group and the principal in the same domain? It needs no configuration and no knowledge of which domain belongs to whom, which is why membership scope is enforceable before anything else about scope is settled.

2.19.2 The exception #

An authority SHOULD allow a source to be configured as permitted to assert memberships outside the asserted principal's domain.

A local source needs it. Local group membership of any principal is a local decision: "CORP\Domain Admins is in BUILTIN\Administrators" is a record this machine keeps, not something a domain controller asserts at it. Without the exception, a local source could not express the one thing it is most authoritative about.

An authority MAY grant this to more than one source. Nothing about it is exclusive.

2.19.3 Memberships only, never identity #

A source holding this permission remains fully confined on identity (§2.18).

The two are separate because the risks are not symmetric. A source asserting a foreign membership is making a claim about what a principal may do on this machine, which is a local matter and is the local source's business. A source asserting a foreign identity is claiming to be the authority for somebody else's principal, which is never anyone's business but that authority's.

2.19.4 The primary group is a membership #

primary_group (§2.13) is subject to this section exactly as a listed group is, and an authority MUST apply the test to it.

It would otherwise be a way round: the authority is required to place the primary group on the token whether or not the source listed it, so a source that named BUILTIN\Administrators there and nowhere else would obtain a membership the group array denies it.

An authority that adds an unlisted primary_group to the membership set MUST do so before applying this section, not after. Adding it afterwards reintroduces exactly the route the rule closes.

This binds a primary group the source asserted. A default the authority substituted for an empty field is not the source's claim and MUST NOT be tested against the source's scope (§2.13); an authority that tested its own default would refuse every logon from a source that simply left the field empty.

2.19.5 Claims are the same question, unanswered #

A claim (§2.13) is a trusted input to conditional ACE evaluation, so asserting one can produce a grant just as asserting a membership can. Which claim names a source may assert is therefore the same shape of control as this section — and it is not yet specified.

The gap is stated rather than papered over. An authority federating to a source it does not fully trust should consider claims as it considers foreign memberships, and a future revision is expected to define the control here.

A source's POSIX identifiers are relative. The authority assigns the range and applies the base; a source is told its range but MUST NOT act on it.

Every unix_id in an Assertion — the principal's and each group's — and every identifier in a QueryResult is an offset within a range the authority assigned that source. The authority adds the base before the number reaches a token or a caller.

This is the numeric counterpart of identity confinement (§2.18). That section stops a source naming principals outside its namespace; this one stops it numbering them outside its namespace.

2.20.1 The range #

An authority MUST assign each source a range, as a base and a count, spanning [base, base + count).

An authority MUST reserve a band below every source's base for identifiers of its own — well-known SIDs, service SIDs, confinement SIDs — none of which come from any directory. The band has to be generous, because the last of those categories has no bound.

2.20.2 Rebasing #

Given a relative identifier r from a source with range (base, count), the authority computes base + r, and MUST refuse to produce a number at all when:

• r is 0. Zero is not an identifier; it is how a source says it has no number for something. It MUST NOT become base.
• r is at or past count. The source has reached outside the range it was given.

A number the authority declines to produce projects as unmapped, which the Peios Kernel TRM §3.10.1 defines.

2.20.3 Refuse, never clamp #

An out-of-range identifier MUST be refused. It MUST NOT be clamped to the top of the range, and it MUST NOT be reduced modulo the count.

Both alternatives look like graceful degradation and are worse than a refusal:

• Clamping puts two principals on one number, so a filesystem cannot tell them apart.
• Wrapping lands inside somebody else's range, so one source's principal projects as another source's.

The range is a boundary, not an offset, and the count is what makes it one.

2.20.4 Two numbers a source can never reach #

Because the base sits above the reserved band and a relative identifier cannot escape the count:

1. uid 0. It belongs to the authority's own table and is not reachable by adding a base to anything a source can send.
2. Another source's numbers. Whatever a source asserts, arithmetic confines it to its own range.

Neither depends on the source behaving. They hold because of what the source is able to express.

2.20.5 Well-known SIDs are not the source's to number #

An authority MUST use its own identifier for any SID in its reserved band, and MUST ignore whatever unix_id a source sent alongside it.

A source asserting BUILTIN\Administrators is stating a membership. It is not claiming authority over what that group projects to, and honouring a relative identifier there would place a well-known group inside that source's range — where a second source could number something else identically.

2.20.6 The authority tells the source its range #

A source is told its base and count at registration (§2.8). This is informational and exists so an administration tool can show an operator the identifier a principal will really project to, rather than the relative number on disk.

A source MUST NOT apply the base to what it asserts. It is told the range so it can explain itself, not so it can do the arithmetic. A source that applied its own base would have it applied twice.

Disclosure costs nothing that matters. What confines a source is not ignorance of the base but the authority's refusal to accept a relative identifier at or past the count — a check the authority performs on every number it rebases, whatever the source knows.

2.20.7 Uniqueness within a source #

SIDs are one namespace; POSIX user and group identifiers are two. A source MUST therefore allocate from a single counter across every kind of object it holds, so that a number issued to a principal is never issued again to a group.

An authority cannot check this — it sees one assertion at a time — so it is stated as an obligation on the source (§2.21) rather than as something enforced.

A conforming implementation MUST satisfy every requirement in this chapter. This section collects them by role.

2.21.1 Authority obligations #

An authority federating over PSI MUST satisfy all of the following.

2.21.1.1 Channel #

1. Listen; never dial out to a source (§2.3).
2. Never rely on the socket's descriptor as the access control, and bound unregistered connections and registration time independently of it (§2.6).
3. Bound registered sources, and conversations per source (§2.6).
4. Tear down the connection on a framing error or a failed write, rather than attempting resynchronisation (§2.6).

2.21.1.2 Registration #

5. Require Register on conversation 0 as the first message (§2.8).
6. Establish the source's identity from the peer's token, never from source_name, and refuse a mismatch rather than correcting it (§2.8, §2.9).
7. Accept only configured sources, treating an empty configuration as no source may register (§2.9).
8. Refuse a source that declares no domain (§2.10).
9. Refuse a domain that is not a well-formed locally-issued domain SID (§2.10).
10. Refuse a domain another registered source claims (§2.10).
11. Refuse a source declaring a different domain from the one it declared before (§2.10).
12. Refuse a source whose declared domain contradicts a configured pin, and treat an unparseable pin as refusing rather than as absent (§2.10).
13. Send Registered only once the source is routable (§2.8).
14. Report, where an administrator will see it, a source registering with no identifier range, and a source registering without QUERIES (§2.8).

2.21.1.3 Conversations #

15. Allocate conversation identifiers, never accepting one from a source, never opening one with an identifier already live, and reserve 0 for registration (§2.7).
16. Route each logon to exactly one source, resolved before any credential is collected (§2.11).
17. Never fall back to another source when the owning source is unreachable (§2.11).
18. Relay the verified originator, taken from the client's socket (§2.11).
19. Refuse to relay a prompt for a credential type the client did not advertise (§2.12).
20. Enforce its own round and time limits on the relayed exchange (§2.12).
21. Send Abandon for any conversation that will not reach a terminal state (§2.14).

2.21.1.4 Assertions #

22. Validate every SID structurally before treating it as identity, including SIDs carried inside claim values (§2.13).
23. Drop an asserted logon SID, and drop duplicates first-mention-wins, before applying any scope test (§2.13).
24. Enforce identity confinement, with no configuration lifting it, on an Assertion, on a QueryResult, and on the sid of a Changed (§2.18).
25. Enforce membership scope, subject only to a per-source permission covering memberships alone, and apply it to a source-asserted primary_group as well as to listed groups — promoting an unlisted one into the membership set before the test (§2.19, §2.13).
26. Never apply membership scope to a primary_group it substituted itself for an empty field (§2.13, §2.19).
27. Fail the logon on a malformed group SID, an unusable claim, a canonical_name PGSS §2.15 forbids, or an out-of-range identifier (§2.13).
28. Place primary_group on the token even when the source did not list it among groups (§2.13).
29. Perform derivation itself, and never accept privileges, integrity, or a token from a source (§2.4).
30. Never relay a refusal reason revealing a distinction the source was required not to make (§2.13).

2.21.1.5 Identifiers #

31. Apply the source's base to every relative identifier it accepts, and never accept one already rebased (§2.20).
32. Refuse a relative identifier of 0 or one at or past the source's count, rather than clamping or wrapping it (§2.20).
33. Reserve a band of identifiers below every source's base for its own, and use its own value for any SID within that band regardless of what the source sent (§2.20).

2.21.1.6 Queries #

34. Never send a message a source did not declare it answers, and never set a field bit gating a capability it did not declare (§2.8).
35. Never send more keys in one Query than the source's max_batch, nor more than 64 whatever it declared (§2.8).
36. Resolve an absolute POSIX identifier to a source and a relative identifier itself, and never send an absolute one to a source (§2.15).
37. Apply identity confinement, membership scope and numeric scope to a QueryResult exactly as to an Assertion, and validate every SID in one structurally — including those of references inside a value (§2.15).
38. Rebase every identifier in a result, under the rules of §2.20 (§2.15).
39. Qualify a source's canonical_name itself; never require a source to (§2.15).
40. Consult no further source on a Refused result, treat only NotFound as leave to continue, and never relay a source's Refused outward as PGSS Logon's Refused outcome (§2.15).
41. Never record a source it may not ask — one that declared no QUERIES — as having answered NotFound (§2.15).
42. Relay cursors opaquely, never construct or modify a source's, and never restart an enumeration on a source's behalf (§2.16).
43. Record a source that declined or could not be reached, and not retry it for the remainder of that enumeration, across pages as well as within one (§2.16).
44. Never infer from a source declining to enumerate that it holds nothing (§2.16).

2.21.1.7 Caching #

45. Not cache a source's answers at all unless it declared PUSHES_CHANGES or a non-zero entry_ttl (§2.8, §2.17).
46. Not hold an answer beyond a declared entry_ttl (§2.8).
47. Accept Changed at any time after Registered, and never reply to it (§2.17).
48. Re-read through Query after an invalidation, and never take a new value from Changed (§2.17).
49. Treat the loss of a source's connection as All for that source (§2.17).

An authority that holds nothing satisfies 45 to 49 trivially.

2.21.2 Source obligations #

A principal source MUST satisfy all of the following.

2.21.2.1 Connection #

1. Connect to the authority; never listen for it (§2.3).
2. Open with Register on conversation 0, carrying its name and its domain (§2.8).
3. Send nothing else before receiving Registered (§2.8).
4. Report itself ready — to an init system or equivalent — only after Registered (§2.3).
5. Declare the same domain on every registration, for the life of the machine's configuration (§2.10).
6. Tear down the connection on a framing error or a failed write, on every path including an unsolicited Changed (§2.6, §2.17).

2.21.2.2 Conversations #

7. Reply on the conversation identifier it was given, and never invent one (§2.7).
8. Decline to act on a message on a conversation it does not know, and never treat it as opening one — without replying on it, since the identifier may since have been reused (§2.7).
9. Refuse an Authenticate, Query or EnumerateSource arriving on conversation 0 (§2.7).
10. Bound the conversations it tracks itself, rather than relying on the authority's limit (§2.6).
11. Refuse beyond that bound with AuthorityUnavailable, rather than dropping silently (§2.11).
12. Discard conversation state on Abandon, and not reply (§2.14).

2.21.2.3 Answering #

13. Send exactly one terminal message — Assertion or Refusal — per conversation (§2.13).
14. Assert only principals within its declared domain (§2.18).
15. Assert group SIDs and identifiers only, never attributes (§2.13).
16. Carry the canonical spelling of the principal's name in canonical_name (§2.13), and never assert a name that PGSS §2.15 forbids — validating what it asserts, not only what it creates.
17. Never distinguish an unknown principal from a bad credential — by denial code, by reason, or by timing (§2.13).

2.21.2.4 Identifiers #

18. Assert relative identifiers only, and never apply its own base (§2.20).
19. Allocate from a single counter across every kind of object it holds, so that no number is issued twice (§2.20).
20. Send 0 for any object it does not number, including every group it does not own (§2.20).
21. Never issue an identifier at or past the count it was given (§2.20).

2.21.2.5 Queries #

A source declaring no capabilities (§2.8) is exempt from this section entirely.

22. Declare only capabilities it implements, and answer every message type it declared (§2.8).
23. Answer every key of a Query, in order, without reordering, merging or omitting — or refuse the conversation whole (§2.15).
24. Send Found, NotFound or Refused in a result, and never Unavailable or Malformed (§2.15).
25. Return its own canonical spelling of a name, unqualified (§2.15).
26. Return relative identifiers in a result, exactly as in an Assertion (§2.15, §2.20).
27. Answer only for principals within its declared domain, on a query as on a logon (§2.18).
28. Answer Refused, never an empty Found, wherever it is declining rather than reporting an absence — including a membership it will not expose, a key naming an object it does not hold, and a key of the wrong kind (§2.16).
29. Refuse a cursor it can no longer honour, rather than restarting or answering from a changed store (§2.16).
30. Hold no per-cursor state it is unwilling to discard unasked (§2.16).
31. Size a page against the smaller of PSI's message ceiling and PGSS Logon's, since the authority must re-encode it into the latter (§2.16, §2.A).
32. Send Changed before the altered answer becomes observable, if it declared PUSHES_CHANGES (§2.17).
33. Declare a non-zero entry_ttl if it does not push changes and can tolerate its answers being held, and SHOULD declare one even if it does (§2.8, §2.17).

2.21.2.6 Credentials #

34. Store verifiers that are not usable as credential material — nothing a challenge could be recomputed from (PGSS §2.11).
35. Erase credential material, and every buffer it was decoded through, before that memory is released (PGSS §2.12).
36. Never write credential material to a log, audit record, or diagnostic (PGSS §2.12).

2.21.3 What a source is not required to do #

A source is not required to store anything, to be local, to be persistent, or to know what a token is. It answers one question: given this identifier and whatever it chose to ask for, who is this?

Nor is it required to trust the authority beyond the connection. A source that refuses logons on the strength of originator, or declines to answer for principals it holds but does not wish to expose, is conforming.

2.A.1 Messages #

The high bit marks a message sent by the source, which is the authority for its own principals (§2.7).

2.A.2 Protocol constants #

2.A.3 Field limits #

68 bytes is the largest a SID can be: an eight-byte prelude plus fifteen sub-authorities (§2.7).

A claim name is bounded at 255 bytes of UTF-8 while PCDS §5.9 bounds it at 255 UTF-16 code units. A string's UTF-16 length never exceeds its UTF-8 byte length, so the byte bound is the stricter of the two and satisfies PCDS without transcoding to find out.

The claim limits are otherwise tighter than PCDS §5.9 permits — it allows 1024 values per claim. These bound the work an authority does decoding a message it has not yet decided to believe, and nothing needs a thousand-valued claim from a principal source.

Fields inside a nested LogonStart, CredentialRequest, CredentialResponse or profile keep PGSS Logon's limits (PGSS §2.A).

2.A.4 The ceiling that actually binds a page #

None of the entry counts above is the constraint on how much a source may return. An entry that fits a PSI message need not fit the PGSS Logon message the authority must re-encode it into: this chapter's ceiling is 81920 bytes and PGSS Logon's is 65536, and a QueryResult or EnumerateResult entry travels outward inside the smaller one.

A source MUST therefore bound a reply by the smaller of the two ceilings, not by this one, and MUST page rather than fill a PSI message it knows an authority cannot forward. A page that fits here and not there is a page nobody can deliver, and the entry-count bounds do not prevent one — 256 entries of a few hundred bytes each exceeds both.

The margin an implementation leaves for the authority's own framing is its own choice; leaving none is a defect.

Values PSI defines for itself. Everything else it carries is PGSS Logon's — see PGSS §2.B.

Adding a value to any enumeration here is a breaking change requiring a version bump (§2.7). The two exceptions are the capability bitmask below, and PGSS Logon's field bitmask, which PSI carries unchanged and which may gain bits without one.

2.B.1 Key types #

Carried in a Query key entry's key_type (§2.15) as a u8.

Zero is not a key. It is how an EnumerateSource encodes an empty of (§2.16), which is the only place it may appear; an encoder MUST NOT send it in a Query key and a decoder MUST reject one that arrives there.

PGSS Logon's key types (PGSS §2.B) share the first two values and differ in the third, where it carries an absolute POSIX identifier. The values are not interchangeable and the tables are deliberately separate: 3 means a rebased number on one side of the authority and a relative one on the other, which is the whole of §2.20 expressed as a number.

2.B.2 Result outcomes #

Carried in a result entry's outcome (§2.15) and in EnumerateResult.outcome (§2.16) as a u8.

The values are PGSS Logon's (PGSS §2.B). A source may send only these three:

Unavailable (3) and Malformed (5) are the authority's to produce and MUST NOT be sent by a source — see §2.15. A decoder MUST reject either arriving from a source, rather than leaving the check to a caller.

2.B.3 Change scopes #

Carried in Changed.scope (§2.17) as a u8.

2.B.4 Capabilities #

Carried in Register.capabilities (§2.8) as a u32 bitmask.

Unlike the enumerations above, a bit MAY be added here without a version bump. A source that does not set a bit has not declared the capability, and an authority MUST NOT send a message the source did not declare it answers (§2.8) — so an authority that predates a bit simply never uses it, and a source that predates one never sets it. Both are the safe reading.

MEMBERS gates a field rather than a message: an authority MUST NOT set the MEMBERS field bit of a Query (§2.15) against a source that did not declare it.

2.B.5 Claim value types and flags #

A claim's value_type and flags (§2.13) are PCDS §5.9's, and are not restated here.

They are nonetheless closed on this interface: a decoder MUST reject a value type or a flag bit it does not recognise, rather than carrying it through to an authority that will put it on a token. Adding one is therefore a breaking change to PSI by the rule above, even though the values themselves belong to PCDS.

PSI is a superset of PGSS Logon (§2.5). This appendix consolidates exactly what is shared, what is added, and what differs — as a checklist for an implementer building both, and as the list to re-examine whenever either specification changes.

2.C.1 Shared unchanged #

An implementation that reimplements any of these rather than sharing one definition has taken on the job of keeping two copies in step. The sharing is the point: a translation layer between two byte-identical formats is a place for them to drift.

2.C.2 Added by PSI #

2.C.3 Differs #

The success terminal is what makes minting structurally impossible for a source (§2.4); the socket path is normative in PGSS because it is a conformance bar and not here because PSI is not one (§2.1).

The message ceiling is the row most likely to catch an implementer out, because the larger number is the one that does not bind a reply — see §2.A.

2.C.4 When either specification changes #

A field appended to LogonStart, CredentialRequest, CredentialResponse or profile in PGSS appears here automatically, because the bodies are shared. That is the intended behaviour and needs no change to this chapter.

The profile is the one shared body that travels in the opposite direction to the others: the interrogation bodies pass from the authority outward to the client, while the profile originates at the source and is relayed outward through the authority. It is shared for the same reason regardless — one definition, so the value a source states and the value a client reads cannot drift apart.

A new Denial value, a new CredentialType, or any change to the header's first twelve bytes is a breaking change to both and requires a coordinated version bump. An implementer maintaining both MUST NOT bump one alone.

2.D.1 What this exists to avoid #

The design is shaped more by rejected approaches than adopted ones.

LSA authentication packages. Windows loads authentication packages as DLLs into the LSA process. A defect in any package is a defect in the most privileged process on the system, and the packages are exactly the components most likely to parse hostile input. PSI puts that boundary at a process, permanently: there is no in-process extension point and no message that could create one.

PAM modules. The same objection, plus stacking — offering a credential to each module in turn until one accepts, which hands every module the credentials of every other module's users. PSI resolves which source answers before any credential is collected (§2.11).

NSS. Name service switch modules answer "who is this?" as a library call in whatever process asked, with no boundary at all. PSI's answer is a message from a process that was separately identified.

2.D.2 What is adopted #

RSI's shape, specified in PSPK. A long-lived connection, sources that dial in and register, multiplexed requests tagged with an identifier, and the authority tearing down a connection it cannot parse. PSI is recognisably the same family, and deliberately so — an implementer who has written a registry source will find little surprising here.

The differences are worth naming, because they follow from what is being federated. A registry source is trusted with the correctness of a subtree and the kernel validates its structure; a principal source is trusted with identity, so the checks it faces are about scope — which principals, which memberships, which numbers — rather than about well-formedness alone. And the kernel assigns a registry source no numeric range, because there is nothing to project.

PGSS Logon's interrogation, wholesale. Rather than inventing a parallel vocabulary for prompts and answers, PSI relays PGSS Logon's messages with identical bodies (§2.5). The gain is not brevity but correctness: there is no translation layer to be lossy, and a source's prompt reaches the client exactly as written.

2.D.3 Design influences #

Sources connect inward. The authority never dials out. See §2.3 — this is the single most consequential shape decision in the chapter.

Assertion, not minting. The success terminal deliberately cannot express a session or a token (§2.4). The separation is structural rather than a rule an implementer must remember.

Domains claimed, not assumed. A source states what it is authoritative for and the authority confines it to that (§2.10, §2.18). The alternative — an authority that trusts whatever a source says about anyone — makes every source as dangerous as the most dangerous one.

Relative numbers. POSIX identifiers are the one thing a source states that has no namespace of its own to be confined by, so the range supplies one (§2.20). Nothing else in the protocol needed inventing; a SID already carries its domain.

This chapter specifies the observability interfaces: the three channels by which the programs on a Peios system deposit logs and metrics with an observability service, and by which anything on the system asks that service what it holds.

There are three interfaces and they are specified together because they are one contract from the service's side and because two of the three share their encoding, their validation posture and their loss model:

• the Log Ingestion Interface, on which a producer submits log records (§3.6 to §3.8)
• the Metric Ingestion Interface, on which a producer submits metric samples (§3.9 to §3.13)
• the Query Interface, on which a client asks for stored events, logs and metrics and receives records (§3.14 to §3.28)

Three roles participate.

The collector is the process that accepts ingestion on the two datagram channels, serves the query channel, and holds the data in between. There is one collector. It is the party being asked, on all three interfaces — which is why the obligations in this chapter fall mostly on it, and why the producer and client roles are so thin.

A producer is any process that submits log records or metric samples. The producer role is unrestricted by design: the point of a system log is that everything on the system can write to it. A producer declares what it is (§3.7, §3.11) and the collector does not verify the declaration (§3.4).

A client is a process that issues a query and reads the result. A client's identity, unlike a producer's, is established by the collector and determines what it may see (§3.28).

One program is commonly all three at once.

This chapter covers:

• the shape of the three channels and why two are datagram and one is a stream (§3.3)
• the loss model, which is the load-bearing decision of the whole ingestion design (§3.4)
• encoding, timestamps and the timestamp domain (§3.5)
• the log record, its fields, and exactly which malformations cost the record and which are merely ignored (§3.6 to §3.8)
• the metric data model, the three metric types, the sample record, and what makes two samples the same time series (§3.9 to §3.13)
• the query channel, its framing, and the four response messages (§3.14 to §3.17)
• the query language: its shape, its lexis, its operators, its ordering and grouping semantics, and the three modes (§3.18 to §3.25)
• cross-type filtering and streaming (§3.26, §3.27)
• what a client is and is not told about data it may not read (§3.28)
• how these interfaces may be extended (§3.29)
• the obligations binding on each role (§3.30)

This chapter does not cover:

• Event emission. Events reach a collector through KMES, not through any interface here; the consumer side of that is specified in PSPK, and emission is a kernel interface offered to privileged callers.
• Event type vocabulary and payload schemas, which belong to whichever subsystem emits the event.
• How a collector stores, indexes, retains or accelerates anything — its own design. The mainline collector's is described in the eventd TRMP.
• Which producers a system permits to reach the ingestion channels, and how that is configured.
• Administering a collector's contents.

The third of those is the point of the whole document. A collector is handed records and asked questions; how it gets from one to the other is exactly what different collectors exist to do differently.

3.1.1 These interfaces are not a conformance requirement #

A system that offers none of these is still Peios. Observability is not in the definition of the platform, and a system that ships a different collector, or none, conforms exactly as well.

They are specified because they are public. Every service on the system is a log producer, every collection agent is a metric producer, and every dashboard, alerting tool and command-line viewer is a query client. All three of those are third-party positions, and all three need a contract that stays put.

Terms defined in PSPK for the KMES event stream — event, header, payload, stamp, sequence number, origin class — are used here with the same meaning and are not redefined. Terms defined in PCDS — GUID, SID, Security Descriptor, ACL, ACE — likewise.

The following terms are specific to this chapter.

Collector: the process that accepts log and metric ingestion and serves queries. The role, not the program: the mainline collector is eventd, and this chapter never requires that it be.

Producer: a process that submits log records or metric samples.

Client: a process that issues a query.

Log record: one line of output from a program, with the light metadata of §3.7 attached. A log record is text; the collector does not parse it.

Metric sample: one measurement of one quantity at one moment, belonging to a time series (§3.13).

Time series: the sequence of samples sharing a name, a label set, and — for histograms — a set of bucket boundaries. Identity is defined in §3.13.

Boot ID: a GUID identifying one boot of the system. Every record a collector stores carries one, so that records from different boots are never interleaved and per-CPU event sequence numbers, which restart each boot, remain unambiguous. It is assigned outside this interface and is visible to a client only as a queryable field.

Datagram: one message on an ingestion channel, carrying either one record or a batch of them (§3.7, §3.11).

Effective query range: the half-open interval a query examines, [SINCE, UNTIL), with the bounds resolved as §3.19 defines.

Concrete identifier: the event type, log origin or metric name that a stored record actually carries — as distinct from the pattern a query or a Security Descriptor uses to match one. Access control resolves per concrete identifier (§3.28).

A collector listens on three AF_UNIX sockets. Two carry data inward, one carries it out.

The pathnames are configuration and this chapter does not fix them. A collector MUST serve each interface on a distinct socket; it MUST NOT multiplex two of them onto one.

3.3.1 Why ingestion is datagram #

Each submission is an independent message. A datagram either arrives whole or does not arrive, so there is no framing to get wrong, no length prefix to parse, no partial read to reassemble, and no connection state to keep for a producer that submits one line an hour. The record boundary is the datagram boundary.

The property that matters more is that a datagram socket cannot exert backpressure. When the receive queue is full the kernel discards the datagram and the sender proceeds. That is the behaviour §3.4 requires, and choosing a stream socket would make it unachievable: a full stream buffer blocks the writer, which is precisely the outcome this design forbids.

3.3.2 Why the query channel is a stream #

A query result is arbitrarily large, is delivered in several messages, and must not be silently truncated — the opposite requirements. It also needs a caller identity, and a peer token can be obtained from a connected stream socket. Both push the same way.

3.3.3 Separation #

The three channels are separated for the same two reasons.

The first is admission. Each listening socket has its own receive queue. Log volume is orders of magnitude above query volume on any normal system, and a burst of either must not delay the other. Three sockets means three populations of caller that cannot starve one another, whatever load any of them is under.

The second is access control. The set of processes that may write logs is every process on the system; the set that may read them is not. Those want different Security Descriptors, and a descriptor is a property of a socket.

The separation is not for isolation. One collector serves all three, so a defect or a hang in any of them reaches the others regardless, and this chapter does not pretend otherwise.

3.3.4 Protecting the channels #

A collector MUST protect each socket with a Security Descriptor.

This is the whole of the access control on the two ingestion channels: there is no per-record write authorization anywhere in this chapter (§3.4), so the descriptor on the socket is the only thing standing between a process and the ability to write a log line under any name it likes.

A collector MUST NOT rely on the socket's POSIX mode bits for this. On Peios an access decision is routed through the object's Security Descriptor, not through mode bits, so a mode set on a socket pathname does not restrict anything; and an inode created without a descriptor is denied to every caller, so a collector that binds a socket into a directory carrying no inheritable ACEs produces a socket nothing can reach. A collector MUST establish the descriptor on each socket before it begins accepting on it.

The single decision that shapes both ingestion interfaces is this: a producer is never slowed down, and never told that a record was lost.

3.4.1 The obligation #

A collector MUST NOT exert backpressure on a producer. A producer MUST NOT stall, block, retry or otherwise change its behaviour because a collector is slow, busy, or absent.

The consequence is accepted openly. When a collector cannot drain an ingestion socket as fast as producers fill it, the kernel discards datagrams. Neither party is notified. A collector MUST NOT report the loss to the producer, because there is no reply message on a datagram channel and adding one would reintroduce the coupling this rule exists to prevent.

3.4.2 Why loss is acceptable here #

A lost log line is an inconvenience. A lost metric sample is a visible gap in a chart. Neither is a failure of the system, and neither is worth the cost of the alternative — which is either blocking the producer or buffering without bound, and the second is only the first with a delay.

Events are the counter-example, and the reason the boundary between events and logs matters. An event may be a security audit record whose absence is itself the finding, so events do not travel on either interface in this chapter: they travel through KMES, where loss is detected, bounded and recorded. A program with data that must not be lost emits an event; a program with output for a human to read writes a log.

3.4.3 What a collector must not do about it #

A collector MUST NOT emit an event, write a log entry, or perform any other work proportional to the volume of malformed or unwanted input it receives.

Ingestion input is unauthenticated and arrives from arbitrary local processes (§3.3). A collector that reacted to bad input — by logging it, by counting it in a way a client can observe, or by emitting a diagnostic event — would hand every process on the system an amplification primitive: a cheap malformed datagram producing an expensive durable record. Silence is the defence.

The rule binds only on responses to input. A collector MAY record its own internal conditions, and the mainline collector records several (eventd TRMP §2.6).

3.4.4 Ordering and duplication #

A collector MUST NOT assume that datagrams arrive in the order they were sent, and MUST NOT reorder or deduplicate the records inside one. A producer MUST NOT assume that submitting two datagrams in order causes them to be stored in that order; the timestamp field (§3.7, §3.11) is the only ordering a producer controls.

Records are not deduplicated. A producer that submits the same record twice has produced two records.

3.5.1 MessagePack #

Every structured value on all three interfaces — log records, metric samples, query requests, query responses — is encoded as MessagePack. Strings are UTF-8.

The choice is inherited rather than made here: KMES event payloads are a single MessagePack value, so a collector already carries a decoder and a query result can carry an event payload outward without re-encoding it.

A decoder MUST accept any valid MessagePack encoding of a value it is given. In particular a producer MAY use any length-prefix width that can represent the value, and a collector MUST NOT require the shortest.

3.5.2 Canonical MessagePack #

Where this chapter requires a canonical encoding, the value MUST be encoded as follows:

• Nil and booleans use the fixed singleton encodings.
• Integers use the shortest encoding that preserves signedness: non-negative values use positive fixint, uint8, uint16, uint32 or uint64; negative values use negative fixint, int8, int16, int32 or int64.
• Floats are encoded as float64. Finite values use their IEEE-754 binary64 representation; the infinities use the normal binary64 encodings; any NaN is encoded as the single quiet NaN bit pattern 0x7ff8000000000000.
• Strings, binary values, arrays and maps use the shortest length-prefix form capable of representing the length.
• Arrays encode each element recursively, in order.
• Maps encode keys and values recursively, with entries sorted by the canonical encoded key bytes; ties are broken by the canonical encoded value bytes.

Canonical encoding exists so that two values that are equal are also byte-identical, which is what makes them comparable and orderable without decoding. It is required in exactly two places: histogram sample storage, where it makes a sample map a stable value (§3.11), and array comparison in query ordering and grouping (§3.21).

It does not constrain what a producer sends. Ingestion accepts any valid encoding.

3.5.3 Timestamps #

A timestamp is wall-clock time in nanoseconds since the Unix epoch, UTC, as a signed 64-bit value.

The timestamp domain is 0 to 9223372036854775807 inclusive. A value outside it is invalid wherever it appears: as a producer-supplied timestamp (§3.8, §3.12), as a query time literal (§3.19), or as a value in a result record.

The domain has no negative half. A collector MUST reject a negative timestamp rather than storing a time before 1970, and a query whose time arithmetic lands below zero — SINCE 100000d ago, for example — MUST produce an error rather than clamping.

Wall-clock time is not monotonic. A collector MUST store the timestamp it is given or derives without correcting it, and MUST NOT assume that timestamps within one time series increase (§3.13). A clock step backwards produces records that are out of order with respect to their arrival, and every ordering rule in this chapter is defined to remain total and deterministic when that happens.

A collector MUST expose an AF_UNIX SOCK_DGRAM socket for log ingestion, protected by a Security Descriptor as §3.3 requires.

3.6.1 The datagram ceiling #

A collector declares a maximum accepted datagram size, the log datagram ceiling. A collector MUST receive log datagrams into a buffer of at least that size, and MUST discard a datagram the kernel reports as truncated rather than storing the prefix that fitted.

A producer MUST NOT send a log datagram larger than the ceiling. One that does is discarded whole, taking every record in it, and the producer is not told (§3.4).

There is no mechanism by which a producer can learn the ceiling. A datagram channel has no reply, so a producer either knows the value out of band or assumes the mainline default. A collector that lowers the ceiling below the mainline value MUST expect producers to keep sending at the old one, and silently losing what they send. Raising it is safe; lowering it is a change to the contract with every producer on the system.

3.6.2 The receive queue is the buffer #

A collector MAY enlarge the socket receive queue, to at most four times the datagram ceiling. It MUST NOT buffer beyond it.

That queue is the only cushion between a producer and the collector's storage. While a collector is committing a batch it is not draining the socket, and datagrams arriving in that window occupy the queue; when the queue fills, they are discarded. This is the designed degradation (§3.4), not a failure to be tuned away — a larger buffer moves the threshold without changing what happens at it, and an unbounded one converts data loss into memory exhaustion.

3.6.3 Reachability #

Every process that produces output is a log producer, including processes that have not been written with a collector in mind.

The mainline arrangement is that the service manager holds each service's standard output and standard error at fork and forwards what it reads (peinit TRM). It is not a privileged producer: it uses this socket, this record format and these rules like anything else, and its role is to bridge programs that write to a file descriptor into an interface that expects datagrams.

A producer that wants control over its own metadata MAY write to the socket directly instead, with no registration, negotiation or setup of any kind. Direct submission and forwarded submission are the same interface; nothing distinguishes them on the wire, and a collector MUST NOT treat them differently.

A log datagram carries either one record, encoded as a MessagePack map, or several, encoded as a MessagePack array of maps. A collector MUST accept both forms. A producer MAY use either at any time; there is no mode and no negotiation.

3.7.1 Fields #

A collector MUST ignore fields it does not recognise (§3.29).

3.7.2 origin #

origin is what the producer says it is. A collector MUST NOT verify it, because it has no way to: the channel is a datagram socket and carries no peer identity (§3.4). Two producers MAY use the same origin, and one producer MAY use several.

An origin is nonetheless the unit that read access is granted on (§3.28), and a collector matches it against patterns using dot-delimited prefix semantics: the pattern svc matches the origin svc and any origin beginning svc., and matches neither svc_daemon nor svcfoo.

A producer therefore SHOULD choose an origin that names it stably and distinguishably, and SHOULD use dots for hierarchy, because an administrator writing an access rule has nothing else to write it against.

An origin MUST match the identifier grammar of §3.19:

[A-Za-z_][A-Za-z0-9_.-]*

A collector MUST discard a record whose origin does not.

The constraint exists because an origin is not merely a label. It is matched against patterns in which * is the wildcard, so an origin containing * could not be selected exactly and could match a rule its producer was never meant to satisfy; and it is the name an access rule is stored under, so an origin carrying a path separator or a quoting character could land somewhere other than where the administrator who wrote the rule believes it is. Constraining the producer is the only point at which either can be prevented.

Quoted forms remain valid syntax everywhere an origin may be written (§3.24). A conforming origin never needs them, but a pattern may, and a collector holding origins stored before this rule applied must still be able to return and select them.

3.7.3 is_error #

is_error is a boolean and deliberately not a severity level.

A forwarding producer can distinguish standard output from standard error and nothing more; inventing five levels out of two file descriptors would be a guess presented as data. A producer with real severity levels either writes them into the message text, where they are text and are searched as text, or emits events, which have types.

3.7.4 timestamp #

A producer SHOULD supply the timestamp it captured when the line was produced, not when it submitted it. A producer that batches (§3.8) and omits the field attributes every line in the batch to the moment the collector happened to read it, which discards the timing information the batch was accumulated over.

3.7.5 job_id #

job_id correlates a line to a single execution rather than to a program. A forwarding producer sets it so that the output of one run of a service can be separated from the run before and the run after; a producer with no such notion omits it.

A collector MUST treat it as an opaque 16-byte value. Nothing in this chapter interprets it, and a producer MAY use it for any correlation of its own, provided the value is a GUID.

Every failure on this channel is silent (§3.4). What differs between failures is how much is lost: the whole datagram, one record, or only one field.

3.8.1 The three scopes of failure #

The datagram is discarded when it cannot be resolved into records at all:

• it is not valid MessagePack
• it decodes to something that is neither a map nor an array of maps
• the kernel reported it truncated (§3.6)

One record is discarded, and the others in the same datagram are still processed, when the record itself is unusable:

• a required field is absent
• a required field has the wrong type — origin an integer, say
• origin is the empty string
• the map contains a duplicate top-level key

One field is ignored, and the record is still stored, when an optional field is unusable:

• timestamp is not an integer, is negative, or is outside the timestamp domain (§3.5)
• job_id is not binary, or is binary of a length other than 16

A collector MUST implement all three scopes as stated. In particular it MUST NOT discard a record because an optional field was malformed: a producer with a broken clock or a mangled correlation key still has a log line worth keeping, and the field it got wrong is the field of least value in the record.

3.8.2 Duplicate keys #

A record map carrying the same top-level key twice MUST be discarded.

A collector MUST NOT resolve the duplicate by taking the first or the last. MessagePack decoders differ on which they keep, and the fields here are entirely producer-controlled, so a rule that depended on decoder behaviour would let a producer choose which of two origin values a given collector saw. Discarding is the only answer that is the same everywhere.

3.8.3 Batches #

A batch is validated per record. A malformed record in a batch MUST NOT cost the valid records beside it.

A producer SHOULD batch under sustained load. Batching amortises the syscall over many records, and the ceiling (§3.6) is per datagram, so a batch is also the only way to use the channel's capacity efficiently.

The encoded datagram, batched or not, MUST NOT exceed the ceiling. A producer that batches without bounding the encoded size will eventually build a datagram that is discarded whole — which is the one case where batching loses more than sending singly would have.

3.8.4 What a collector adds #

A collector supplies the boot ID (§3.2) and, when the record omitted timestamp, its own clock reading at receipt. It MUST NOT alter any other field, and MUST store message byte-for-byte as given.

A collector MUST NOT parse message. If the text happens to be JSON, or logfmt, or anything else structured, that is the producer's business: this interface carries lines, and a producer with structured data to record emits an event instead (§3.4).

A collector MUST expose an AF_UNIX SOCK_DGRAM socket for metric ingestion, protected by a Security Descriptor as §3.3 requires, separate from the log socket.

The channel works exactly as the log channel does, for the reasons given there: a declared datagram ceiling, truncated datagrams discarded whole, a receive queue of at most four times the ceiling, no backpressure, no notification, and no way for a producer to discover the ceiling (§3.6).

3.9.1 A sink, not a collector of its own #

The collector is pushed to. It MUST NOT scrape an endpoint, read a kernel interface, or poll anything to obtain metrics; every sample it holds arrived on this socket because a producer sent it.

What gathers the measurements is a separate concern and a separate program. A collection agent that reads system counters and submits them is an ordinary producer here, with no privileged position and no interface of its own.

3.9.2 Batching #

Batching matters more here than it does for logs. A collection sweep produces many samples at once — every CPU core, every disk, every interface — and they share a moment, so a producer SHOULD submit a sweep as one batched datagram rather than as one datagram per sample.

The rules are the log rules: one map or an array of maps, per-record validation, and the encoded datagram bounded by the ceiling (§3.8, §3.12).

A metric is a quantity that varies and is worth watching over time: a utilisation, a queue depth, a running total, a distribution of latencies. Metrics are dense where events and logs are sparse — many measurements of the same thing rather than a record of a thing that happened — and the model reflects that.

A sample is one measurement, of one time series, at one moment. It carries:

• a name, identifying what is measured
• a label set, identifying which instance of it
• a type, fixing how the value is to be read
• a timestamp
• a value, whose shape depends on the type

Name and labels together identify the series (§3.13). The type is a property of the series, not of the sample.

3.10.1 Names #

A name MUST match the identifier grammar of §3.19:

[A-Za-z_][A-Za-z0-9_.-]*

A collector MUST discard a record whose name does not, for the same two reasons an origin is constrained (§3.7): names are matched against patterns in which * is the wildcard, and names are what read access is granted on.

Beyond that, naming is convention and a collector MUST NOT enforce any. The conventions in use are a dot-separated hierarchy from general to specific, the unit as the last component, and a cumulative name for a cumulative quantity:

system.cpu.usage
disk.read.bytes
request.duration.seconds
http.requests.total

3.10.2 Labels #

Labels are the dimensions of a measurement: which core, which device, which method. cpu.usage with core="0" and cpu.usage with core="1" are two series, not two samples of one.

Label keys and values MUST be non-empty UTF-8 strings. A key MUST match the identifier grammar above; a value MUST NOT contain = (0x3D) or , (0x2C), which are reserved as delimiters in the collector's canonical representation of a label set (§3.13). A key MUST NOT be repeated within one sample, and MUST NOT be any of the five fixed field names a metric result carries — timestamp, boot_id, name, type, value — because labels and fixed fields share one flat namespace in a result record (§3.22) and a collision would make the record ambiguous.

A record violating any of these is discarded (§3.12).

Label cardinality is the producer's responsibility. Each distinct combination of label values is a distinct series, so labels whose values are unbounded — request identifiers, user-supplied strings, timestamps — produce series without limit, and a collector is required neither to cap them nor to degrade gracefully when they arrive. A producer SHOULD use labels whose value sets are small and known. A dimension that is not bounded belongs in an event payload, where it costs one field, not in a label, where it costs a series.

3.10.3 Types #

The type is fixed when the series is first seen and is immutable. A sample that resolves to an existing series but declares a different type is discarded (§3.12), permanently and without notification. A producer that changes the type of a metric it already emits has stopped emitting it, and the only visible symptom is that the series stops advancing.

3.10.3.1 Counter #

A value that only increases, and resets to zero when the producer restarts. Used for cumulative quantities: requests served, bytes transmitted, errors encountered.

A counter value MUST be a finite, non-negative binary64 value.

The raw value is rarely what a reader wants; the rate of change is (§3.25). A decrease is read as a restart rather than as a negative change, which is why the type must be declared: the same number sequence means something different for a gauge.

3.10.3.2 Gauge #

A value that may move in either direction. Used for current state: a utilisation, an amount in use, a depth, a temperature.

A gauge value MUST be a finite binary64 value, and MAY be negative.

3.10.3.3 Histogram #

A distribution of observations across buckets the producer chose. Used where the shape matters more than the mean — latencies above all.

A histogram value carries:

• boundaries: a non-empty array of bucket upper bounds, strictly increasing in the order given
• counts: one cumulative count per boundary, each being the number of observations less than or equal to that boundary; non-decreasing, and each no greater than the total
• total_count: the number of observations
• sum: the finite sum of the observations

Boundaries are part of the series identity (§3.13). A collector MUST NOT sort or reinterpret them; a producer that changes them has started a new series, and SHOULD therefore keep them fixed for the life of a metric.

The final count MAY be less than the total: observations above the highest boundary are the difference between them, and are not otherwise represented. A total of zero is a valid empty sample, in which case every count and the sum MUST be zero.

3.10.4 Values are floating point #

Numeric input MAY be a MessagePack integer or a MessagePack float; both are converted to binary64 with round-to-nearest, ties-to-even. Every value a collector stores and every value a query returns is a finite binary64.

Non-finite values are refused rather than stored: a record whose value converts to NaN or to either infinity is discarded (§3.12). There is no representation for a missing measurement — a producer with nothing to report sends nothing, and the gap is the answer (§3.13).

A metric datagram carries one record, encoded as a MessagePack map, or several, encoded as an array of maps. A collector MUST accept both (§3.9). Each map is one sample of one series.

3.11.1 Fields #

A collector MUST ignore fields it does not recognise (§3.29).

3.11.2 The histogram value #

For a histogram, value is a map:

Counts and total_count MUST be MessagePack unsigned integers, or non-negative signed integers. boundaries and sum MAY be integers or floats and are converted as §3.10 requires.

3.11.3 type is per record, not per series #

Every record declares its type, including the second and every subsequent sample of a series that already exists.

This is redundant on the wire and deliberately so. A producer holds no state about what a collector already knows, has no way to ask, and MUST NOT be required to establish a series before sampling it: the first sample of a series and the millionth are the same message. The redundancy is what makes a producer stateless, and the cost is one short string per sample.

The collector uses the declaration only on first sight. Afterwards it is a consistency check, and a record that fails it is discarded (§3.10).

3.11.4 Timestamps need not increase #

A collector MUST store a valid sample whose timestamp is older than samples it already holds for that series.

Producers batch, clocks step, and a collection sweep may be submitted out of order or retried. A collector that refused late samples would turn any of those into silent data loss, so it accepts them and defines every ordering it performs over timestamp rather than over arrival (§3.21, §3.25). Two samples of one series MAY share a timestamp; the collector orders them deterministically and a client MUST NOT depend on which comes first.

Failures on this channel are silent, exactly as on the log channel (§3.4, §3.8). The scopes are the same, with one difference that matters: a metric record has no ignorable field. Every field a metric record carries participates either in the series identity or in the measurement, so there is nothing whose loss leaves a usable record behind. A malformed timestamp costs the log line nothing and costs the sample everything.

3.12.1 The datagram is discarded #

• it is not valid MessagePack
• it decodes to something that is neither a map nor an array of maps
• the kernel reported it truncated (§3.9)

3.12.2 The record is discarded #

A collector MUST discard a record, leaving the rest of its batch untouched, for any of the following.

Structure

• a required field is absent, or has the wrong type
• the map contains a duplicate top-level key
• type is not exactly "counter", "gauge" or "histogram"

Name and labels (§3.10)

• name is empty or does not match the identifier grammar
• a label key or value is not a string, or is empty
• a label key does not match the identifier grammar
• a label value contains = or ,
• a label key is repeated within the record
• a label key is one of timestamp, boot_id, name, type, value

Timestamp

• timestamp is present and is not an integer, is negative, or is outside the timestamp domain (§3.5)

Counter and gauge values

• the value is not a number
• it converts to a non-finite binary64
• it is negative and the type is counter

Histogram values

• the value is not a map, or its map has a duplicate key
• a field is absent or has the wrong type
• boundaries is empty
• a boundary or sum converts to a non-finite binary64
• counts and boundaries differ in length
• the converted boundaries are not strictly increasing
• a count is negative, the counts are not non-decreasing, or a count exceeds total_count
• total_count is zero and any count or sum is non-zero

Series consistency

• the record resolves to an existing series whose type differs (§3.10)

3.12.3 Why the timestamp rule differs from logs #

On the log channel a malformed timestamp is ignored and the record kept; here it discards the record.

The asymmetry is not an inconsistency. A log line with the wrong time is still the line, and reading it is still worth doing. A sample is a (time, value) pair and nothing else: attaching the collector's receipt time to a measurement taken at an unknown moment does not recover the sample, it fabricates one, and it fabricates one that will be charted next to real ones. Discarding leaves a gap, which is honest (§3.4).

Absence is different from malformation. A record that simply omits timestamp is asserting "now", and the collector's clock is the right answer to that.

3.12.4 Silence, again #

A collector MUST NOT emit an event, log an error, or increment anything a client can observe in response to any failure in this article — with no exception for the series-consistency failure, which is the one that most looks like it deserves one.

A producer that changes a metric's type is misconfigured, and the misconfiguration is permanent and invisible: every sample is discarded for as long as the series exists. The only signal available to an operator is that the series stopped advancing while the producer reported no error, and the only diagnosis is to query the series and read its type (§3.25).

Two samples belong to the same time series when they agree on:

• the name, compared exactly, byte for byte; and
• the label set, compared as an unordered set of key-value pairs, each compared exactly; and
• for histograms, the bucket boundaries, compared as an ordered sequence of binary64 values.

Nothing else participates. Type does not: a record whose type disagrees resolves to the series and is then discarded for disagreeing (§3.10). Boot ID does not: a series continues across a reboot, and a client that wants one boot's worth filters for it (§3.25). Time does not.

3.13.1 Order does not distinguish a label set #

{core: "0", host: "a"} and {host: "a", core: "0"} are the same series. A collector MUST compare label sets as sets.

To do so it needs a canonical form, and the form in use is the reason label values may not contain = or , (§3.10): pairs are sorted by key in unsigned UTF-8 byte order, each written key=value, and joined with commas. Because neither delimiter can occur inside a key or a value, no escaping is needed and no two distinct label sets can produce the same string.

The byte form itself is the collector's business and this chapter does not require it. What it requires is the property: a label set has exactly one identity, independent of the order the producer wrote it in. The delimiter reservation is stated normatively because it binds the producer, and a producer cannot see the encoding that motivates it.

3.13.2 Absent labels and empty labels #

A record with no labels field, a record with an empty labels map, and a record whose labels were all discarded are the same series: the one with no labels. There is no distinction between "unlabelled" and "labelled with nothing".

3.13.3 Boundaries are identity, not metadata #

Two histogram samples with different boundaries are different series even when name and labels agree.

This is the consequence that surprises producers, and it is unavoidable: cumulative counts against one set of bucket edges cannot be compared with counts against another, so calling them one series would mean computing percentiles across incommensurable distributions. A producer that re-tunes its buckets each collection cycle creates a series each cycle, each holding a single sample and each surviving until retention removes it.

A collector MUST NOT defend against this. It is a producer defect, it is indistinguishable at the interface from legitimately introducing a new metric, and every defence available — capping series, merging near-equal boundary sets, rejecting a second boundary set for a name — would break a correct producer to inconvenience an incorrect one.

3.13.4 Series are created, never announced #

A series comes into existence when its first sample arrives. There is no registration message, no schema, and no way to declare a series in advance or to retire one.

A collector MUST NOT require a series to be known before a sample of it is accepted, and MUST NOT limit how many series exist. A series with no remaining samples ceases to exist when retention removes the last of them; nothing else removes one.

3.13.5 Gaps are preserved #

A collector MUST NOT interpolate, backfill, or synthesise a sample that a producer did not send.

A missing sample is a real fact about the system — the producer was down, the datagram was dropped, the sweep was late — and it is a fact the metric is often being watched for. A series with a hole in it is returned with a hole in it, and a client that wants a value across the hole computes one itself.

A collector MUST expose an AF_UNIX SOCK_STREAM socket for queries, protected by a Security Descriptor as §3.3 requires.

One socket serves all three data types. The mode a query runs in — events, logs or metrics — is determined by parsing the query string (§3.18), never by the transport, so a client needs no connection setup, no mode selection, and no separate endpoint per data type.

3.14.1 One query per connection #

A connection carries exactly one query. A client that wants two concurrent queries opens two connections.

A collector MUST close the connection after the terminal message of a non-streaming query (§3.16), and MUST treat a client disconnect as cancellation of a streaming one.

3.14.2 Identity #

A collector MUST establish the client's identity from the connected socket, before executing anything, by obtaining the peer's token. This is possible here and not on the ingestion channels, and it is the whole reason the query channel is a stream (§3.3).

The token is captured once, at connection time, and is a snapshot. A client whose privileges change while a query runs — and in particular while a streaming query runs, which may be indefinitely — is evaluated throughout against the token it connected with.

If a collector cannot obtain the peer token, it MUST refuse the query. It MUST NOT execute a query for an unidentified caller, and MUST NOT fall back to any other means of identifying one.

3.14.3 No credentials cross this channel #

There is no message with which a client offers a credential and none with which a collector asks for one. Identity is established from the connection and from nothing else.

3.14.4 Concurrency #

A collector MUST bound the number of queries it will execute at once, and MUST reject a query beyond the bound with an error rather than queueing it behind the others.

The streaming bound is the lower of the two and is enforced separately, because a streaming query holds its resources for as long as its client stays connected while an ordinary one holds them for at most a timeout (§3.16).

Both bounds are global. Neither is per-client, because a collector cannot attribute connections to a caller beyond the token it has, and one client MAY therefore occupy every slot. A collector MUST NOT allow that to affect ingestion: queries and ingestion are separate channels precisely so that exhausting one cannot exhaust the other (§3.3).

Every message in either direction is a length-prefixed MessagePack value:

length is little-endian, as PSPU §1.2 requires of every integer field in this document. It counts the payload only; the four bytes of the prefix are not included.

There is no magic value and no version field. The channel is a Unix socket at a configured path, so there is no possibility of reaching the wrong service by accident in the way a shared header guards against (PSPU §2.7), and versioning is handled as §3.29 describes.

3.15.1 The message ceiling #

A collector declares a maximum payload size, the query message ceiling, which bounds messages in both directions.

Inbound. A collector MUST refuse a request whose length exceeds the ceiling. It MUST do so without reading the payload — the point of checking the prefix is to avoid allocating for a request that a malicious or broken client has declared too large — and it MUST send an error response (§3.16) before closing the connection. A collector MUST NOT close on an oversized request silently: a bare close is indistinguishable from a crash, and leaves a client unable to tell that shortening its query is the remedy.

Outbound. A collector MUST ensure every response payload it sends is within the ceiling, chunking result records across messages as §3.16 describes.

3.15.2 The ceiling must admit every record #

A collector MUST NOT operate with a query message ceiling smaller than the largest record it can store.

A single result record is never split across messages (§3.16), so a record larger than the ceiling cannot be returned at all — and it cannot be skipped either, because skipping it would silently misreport what the store holds. It fails the query, and it fails every query whose range covers it, for as long as retention keeps it. One oversized record renders a span of history unreadable.

The two are related by configuration and nothing enforces the relation automatically: the ingestion ceilings (§3.6, §3.9) bound the largest record a producer can deposit, and the query message ceiling bounds the largest that can be handed back. An administrator who raises one MUST raise the other.

3.15.3 Requests #

A request is a MessagePack map:

A collector MUST send an error response and close the connection if the payload is not valid MessagePack, is not a map, omits query, gives query a non-string value, or contains a duplicate top-level key. It MUST ignore unrecognised fields that are not duplicates (§3.29).

Unlike the ingestion channels, nothing here is silent. A query client is identified (§3.14), is one of a bounded number, and is asking a question, so telling it what went wrong is neither an amplification vector nor an information leak — with the one exception §3.28 sets out.

A response is a MessagePack map whose status field names its kind. There are four.

3.16.1 Result messages #

An "ok" message carries records, an array of flat maps (§3.22).

Records are chunked at record boundaries: a successful query sends one or more "ok" messages, each within the message ceiling (§3.15). A collector MUST NOT split one record across two messages. A record too large to fit in a message alone MUST fail the query with an error rather than being truncated, partially sent, or skipped.

Each record is self-describing and records in one response MAY carry different sets of keys — event payload fields vary by event type, metric labels vary by series. A client MUST NOT assume a uniform schema across a result set, and MUST NOT infer that a key absent from one record is absent from the data.

A successful query with no matching records sends exactly one "ok" message with an empty records array, then its terminal message. A collector MUST NOT omit it: "no records" and "the query has not yet produced records" are different states and a client must be able to tell them apart.

3.16.2 The two terminal messages #

"end" terminates a non-streaming query. "watch" marks the point in a streaming query where the stored result set ends and live delivery begins (§3.27). A query sends exactly one of them, never both.

Until one has arrived, the query has not succeeded. A collector that fails partway through MUST send "error", and a client that receives "error" before either terminal message MUST discard every "ok" message it received for that query. Partial results are not results: they are an arbitrary prefix of an ordering that was never completed, and a client that kept them would silently under-report.

An "error" after "watch" is different. It terminates the stream, and the records already delivered remain valid — they were complete when they were sent, and the ordering they belonged to had already closed.

3.16.3 Errors #

An "error" message carries error, a human-readable string.

There is no error code and no machine-readable classification. This is a deliberate limit on the interface: an error here is a parse failure, a type mismatch, a timeout, a limit, or a refusal, and a client's response to all of them is the same — show it to whoever wrote the query. A client MUST NOT parse the string, and a collector MAY change the wording of any error at any time.

A collector MUST NOT include in an error message any value the client was not authorized to read (§3.28).

3.16.4 Timeouts #

A collector MUST bound the time a query may take to reach its terminal message.

The clock starts once the request has been decoded and the caller's token obtained, and it covers everything that follows: parsing, access checks, cross-type pre-computation, execution, merging, aggregation, pagination, projection and transmission. A collector MUST send "end", or for a streaming query "watch", before it expires.

The timeout bounds the initial result set only. Once a streaming query has sent "watch" its watch phase is not time-limited; what bounds it instead is the streaming concurrency limit (§3.14), the distinct-value limit (§3.27), and the client's own ability to keep up (§3.27).

On expiry a collector MUST cancel the query and send "error". Any "ok" messages already sent are discarded by the client under the rule above.

Every value in a result record is encoded as follows.

A GUID is rendered as a string rather than as sixteen bytes because a result record is read by people as often as by programs, and a raw GUID in a terminal is unreadable. The canonical form is lowercase 8-4-4-4-12 hexadecimal within braces, as PCDS defines it. A query comparing against a GUID accepts either braced or unbraced input, and compares case-insensitively (§3.19); a result always uses the canonical form.

3.17.1 Maps do not appear as values #

An event payload is a MessagePack map, and result records are flat (§3.22). A map in a stored payload is therefore a container to be flattened, not a value to be emitted: its entries become top-level keys of the record, joined by dots, and the map itself never appears.

Arrays are different. An array is emitted as an array value at its flattened path, and a collector MUST NOT traverse into it. Maps nested inside an array are preserved as that array's contents, unflattened and unqueryable.

The asymmetry is deliberate. A map has keys, so its entries have names that can be addressed, granted access to and indexed. An array has positions, and a path like hops.3.address would mean something different in every record — so an array is carried across whole and treated as one value.

Binary values in a payload stay binary. A collector MUST NOT render bin as a string, in either direction.

3.17.2 Missing and null are the same value #

A field absent from an event payload or from a metric's label set encodes as nil, exactly as an explicitly null one does, and the two are indistinguishable in a result record.

This is consistent throughout: they compare equal, they sort together, and they group together (§3.20, §3.21). A collector MUST NOT distinguish them anywhere in the query surface, and a client MUST NOT attempt to.

A query is one string. It names a mode, narrows to some data, and says what to do with it.

EVENTS kacs.* SINCE 1h ago WHERE process_guid == "550e8400-e29b-41d4-a716-446655440000" TAKE 100
LOGS FROM loregd ERROR ONLY CONTAINING "connection refused" SINCE 1d ago
METRIC cpu.usage[core="0"] SINCE 1h ago AVG_OVER 5m

3.18.1 Three modes #

The first token selects the mode, and a collector MUST reject a query whose first token is not one of them.

• EVENTS searches structured event records, primarily by event type (§3.23).
• LOGS searches log output, primarily by origin (§3.24).
• METRIC evaluates measurements, primarily by name and labels (§3.25).

Events and logs are record-oriented: collections you search, returning the records that matched. Metrics are value-oriented: measurements you evaluate, returning numbers computed from samples. The modes differ because the data differs, and forcing all three through one shape would serve none of them.

3.18.2 The primary selector #

Immediately after the mode comes an optional primary selector, specific to the mode: an event type pattern, FROM with one or more log origins, or a metric name with an optional label selector. It narrows the data before anything else runs.

A primary selector MUST NOT be repeated unless its mode defines a list form — LOGS FROM a, b is one selector naming two origins, not two selectors.

3.18.3 Clauses #

Everything after the primary selector is a clause, and clauses MAY appear in any order. EVENTS SINCE 1h ago TAKE 10 and EVENTS TAKE 10 SINCE 1h ago are the same query.

Order of appearance never affects meaning. Execution follows the fixed sequence below regardless of how the string was written, so a collector MUST NOT derive semantics from clause position.

These clauses work identically in all three modes:

3.18.4 Execution order #

Whatever order the clauses were written in, a collector MUST evaluate them in this sequence:

Two positions in that list are load-bearing.

Access control is third, before every filter, aggregate, sort and limit. It is part of the query's logical execution and not a filter applied to the output (§3.28).

SELECT is last. It shapes the output and nothing else; a field it omits is still available to every earlier phase (§3.22).

3.18.5 Repetition #

A clause MUST appear at most once, and a collector MUST reject a repeat as a parse error, with two exceptions:

• WHERE is repeatable. Multiple WHERE clauses are combined with AND, each treated as a parenthesised group: WHERE a == 1 OR b == 2 followed by WHERE c == 3 means (a == 1 OR b == 2) AND c == 3.
• SELECT is repeatable where it is valid at all, and is additive: SELECT timestamp SELECT event_type names both fields.

Both exist so that a query can be built up in pieces — by a tool appending a filter, or by a person adding one to a query they already have — without rewriting what is already there.

3.18.6 Counts #

TAKE, SKIP and the N of TOP N BY are unsigned decimal integers that MUST fit in 64 bits. A negative, hexadecimal, floating-point or missing count is a parse error.

SKIP defaults to 0. TAKE omitted means no limit. TAKE 0 and TOP 0 BY are valid, and return no records after every earlier phase has run — which is not the same as not running the query, because a TOP 0 BY still counts and a TAKE 0 still enforces access control.

3.18.7 Case #

Keywords are matched case-insensitively, using ASCII case folding, in grammar positions where a keyword is expected. This document writes them in uppercase by convention only.

Identifiers are case-sensitive, except where the language defines a named alias for a value (§3.23).

A word spelled like a keyword MAY be used where the grammar expects an identifier or a value: LOGS FROM stream selects the origin stream, while LOGS STREAM enables streaming. A collector MUST resolve the ambiguity by grammar position and MUST NOT reserve keywords globally.

A query string is UTF-8. Whitespace separates tokens outside quoted strings and is otherwise insignificant.

3.19.1 Identifiers #

An unquoted identifier is ASCII and matches:

[A-Za-z_][A-Za-z0-9_.-]*

Identifiers name fields, payload paths, metric names, label keys, event type patterns, log origins and value aliases. ., _ and - are permitted inside one; /, :, whitespace, quotes, brackets, parentheses, commas and the comparison operators are not.

This grammar is the same one that constrains a log origin, a metric name and a metric label key at ingestion (§3.7, §3.10), which is what makes every stored identifier writable here without quoting.

A value that cannot be written as an identifier MUST be written as a quoted string. Quoted forms are accepted anywhere an identifier is — they are never required for a conforming identifier, but a pattern may need one, and a collector holding identifiers stored under an earlier revision must still be able to select them.

3.19.2 Strings #

A string literal is double-quoted UTF-8. The escapes are \", \\, \n, \r, \t, and \uXXXX for a scalar value in U+0000 to U+FFFF written as four hexadecimal digits.

A collector MUST reject any other backslash escape as a parse error, and MUST reject \uXXXX naming a surrogate code point in U+D800 to U+DFFF. Surrogate pairs are not decoded: a character outside the basic multilingual plane is written directly as UTF-8, not as two escapes.

3.19.3 Binary #

A binary literal is a lowercase x, a double quote, an even number of hexadecimal digits, and a closing quote:

WHERE target_sid == x"010500000000000515000000"

Hexadecimal digits inside are case-insensitive. x"" is valid and is the empty byte string. Whitespace inside the payload, an odd digit count, and any non-hexadecimal character are parse errors.

A binary literal compares only against MessagePack bin values. A collector MUST NOT coerce one to a string or a string to one: x"6162" and "ab" are different values and never compare equal (§3.20).

3.19.4 Integers #

An integer literal is decimal or hexadecimal.

WHERE origin_class == 2
WHERE granted_access == 0x1F01FF

A decimal integer MAY carry a leading -, in which case it MUST fit in signed 64 bits; without one it MUST fit in unsigned 64 bits. A hexadecimal integer is 0x followed by one or more digits, is always non-negative, and MUST fit in unsigned 64 bits. A leading + is not valid. An out-of-range literal is a parse error.

3.19.5 Floats #

A float literal is a finite decimal number with an optional leading - and either a fractional part or an exponent: 42.0, 0.001, 1e6, -1.25e-3. A token that looks like an integer, such as 42, is an integer literal and not a float.

Float literals are binary64 and MUST be finite. NaN, Infinity, -Infinity and any literal that overflows to infinity are parse errors. A leading + is not valid.

3.19.6 Booleans and null #

true and false are matched case-insensitively with ASCII folding.

NULL, likewise folded, is valid only in IS NULL and IS NOT NULL. A collector MUST reject field == NULL and field != NULL as parse errors rather than evaluating them.

3.19.7 Durations #

A duration is an unsigned decimal integer followed immediately by s, m, h or d — seconds, minutes, hours or days. A zero duration is a parse error.

3.19.8 Times #

Absolute literals are a fixed UTC subset. Components MUST be zero-padded exactly as shown, the date MUST be a valid Gregorian date, hours are 00–23, minutes and seconds 00–59. Leap seconds are not accepted. Timezone suffixes and fractional seconds are not part of this revision and MUST produce a parse error.

3.19.9 The evaluation time #

A collector MUST capture the evaluation time once, before execution begins, and MUST use that one reading for every ago, every hence, and for an omitted UNTIL, throughout the query — including throughout the watch phase of a streaming query.

A query that read the clock more than once could produce a range whose end preceded its start, or a window that grew while it was being scanned. One reading makes the effective query range a fixed interval for the life of the query.

SINCE is inclusive, UNTIL is exclusive: the effective query range is [SINCE, UNTIL). If SINCE is greater than or equal to UNTIL the query returns no records — which is a successful query with an empty result (§3.16), not an error. A time literal that evaluates outside the timestamp domain MUST produce an error (§3.5).

3.20.1 Operators #

IN and NOT_IN take a non-empty parenthesised, comma-separated list of literals. An empty list is a parse error.

WHERE origin IN ("loregd", "peinit")
WHERE origin_class NOT_IN (kacs, lcs)

= is not a comparison operator and MUST produce a parse error, with one exception: inside a metric label selector, where = and == are both equality (§3.25).

3.20.2 Strings fold case #

Every string comparison — ==, !=, STARTS_WITH, ENDS_WITH, CONTAINS, IN, NOT_IN — is case-insensitive, using ASCII-only folding: bytes A–Z compare equal to a–z, and every non-ASCII byte compares exactly.

This applies uniformly: to event header fields, to payload fields, to log messages, to metric label values, and to the pattern matching of primary selectors. Integers, floats, GUIDs, timestamps and binary values are unaffected.

3.20.3 Numbers compare mathematically #

Integers and floats compare by mathematical value, not by casting both to one storage type.

An integer equals a finite float only when the float represents exactly that value. Ordering between an integer and a float MUST be exact, including for integers outside the range binary64 can represent exactly. A collector MUST NOT resolve 9007199254740993 > 9007199254740992.0 by converting the left operand to a float, which would make it false.

3.20.4 Types do not coerce #

Values of different non-numeric types are never equal. The string "1" is not the integer 1, and != between them is true.

An ordering operator applied to a field whose runtime value is non-numeric evaluates false for that record — not an error, because a payload field's type varies from record to record and a query cannot know in advance.

An ordering operator applied to a known fixed field whose declared type cannot be ordered is different: a collector MUST reject the query during parsing or planning rather than executing a predicate that can never match. WHERE message > 5 is a mistake the collector can see, and returning zero records for it would be a wrong answer that looks like a right one.

Binary values compare by exact byte equality under ==, !=, IN and NOT_IN. Ordering is not defined for binary values, and a predicate applying an ordering operator to a binary literal MUST produce a parse error.

3.20.5 Absent fields #

A field absent from an event payload or from a metric's label set resolves to null (§3.17).

Every comparison against null evaluates false, except IS NULL, which is true, and IS NOT NULL, which is false. In particular WHERE field != "x" does not match records lacking the field: a record with no opinion is not a record with a different opinion.

3.20.6 Combining predicates #

Predicates within one WHERE combine with AND and OR. AND binds tighter than OR. Parentheses override.

Multiple WHERE clauses combine with AND, each parenthesised as a group (§3.18).

There is no NOT. Negation is written with the negative operators — !=, NOT_IN, IS NOT NULL — and a collector MUST reject NOT as a parse error rather than silently treating it as an identifier.

Two rules govern this article, and both exist for the same reason.

Ordering MUST be total and deterministic. For a fixed set of stored records, one query MUST produce one order. Without that, SKIP and TAKE are meaningless: a client paging through results would see records twice and never see others, and would have no way to tell.

Equality here MUST be the query language's, not the storage engine's. A collector that grouped by whatever its database considers equal would group differently depending on how it was built.

3.21.1 SORT #

SORT orders by one or more fields. Each defaults to ascending; ASC may be written, DESC reverses that field.

SORT timestamp DESC
SORT origin ASC, timestamp DESC

If the named fields do not uniquely order two records, a collector MUST append internal tiebreakers until the order is total. The tiebreakers are not query-language fields, are never emitted in a result record, and a client MUST NOT depend on their identity — only on their effect, which is that the order is stable.

When no SORT is present:

• Events and logs are ordered by timestamp descending, most recent first — the order a person reading a log wants.
• Metrics are ordered by timestamp ascending, the order a chart wants.

3.21.2 Value ordering #

SORT uses the query language's ordering, not the storage engine's. Missing fields and explicit nulls are equivalent. Ascending order sorts by type first, in this order:

1. Null
2. Boolean, false before true
3. Numeric, integers and floats compared mathematically (§3.20)
4. String and GUID, ASCII-folded
5. Binary, unsigned lexicographic
6. Array

DESC reverses the whole ordering, type order included.

Strings and GUIDs compare with the same ASCII folding as predicates. Two strings equal under folding are ordered by their original UTF-8 bytes, so that folding never costs totality. Binary values compare as unsigned bytes. Arrays compare by their canonical MessagePack encoding (§3.5).

Maps do not appear as result values (§3.17) and MUST NOT appear as sort keys.

3.21.3 Grouping #

COUNT BY, TOP N BY, GROUP and DISTINCT use query-language equality:

• missing and null are one group
• integers and floats that are numerically equal are one group
• strings and GUIDs group under ASCII folding
• binary values group by exact bytes

3.21.4 The canonical representative #

When a group's members are equal under those rules but not byte-identical — "Loregd" and "loregd", or 1 and 1.0 — the value emitted for the group MUST be its canonical representative:

Choosing the smallest rather than the first makes the representative a property of the set, independent of the order records were read in — which matters because a collector may read them from several places at once and merge (eventd TRMP §6.4).

3.21.5 Ordering of aggregates #

COUNT BY results are ordered by count descending. Ties are broken by the group key under the value ordering above, then by the representative's encoded bytes.

TOP N BY is exactly COUNT BY with TAKE N applied after that ordering.

DISTINCT results are ordered by the distinct value under the value ordering, unless an explicit SORT overrides it.

Every result record is a flat MessagePack map. There is no nesting in a result, in any mode.

Flatness is what makes one set of rules — for access control, for ordering, for grouping, for projection — apply uniformly to a header field, a payload field and a metric label alike. A nested result would need a path language, and a path language would need to be reproduced identically by every SD author and every client.

3.22.1 Event fields #

These names resolve to header fields:

timestamp, cpu_id, sequence, origin_class, event_type, effective_token_guid, true_token_guid, process_guid, boot_id

Every other name resolves to a payload field.

3.22.1.1 Header names are reserved #

Header field names are reserved in the query language and in result maps. If a payload carries a top-level key with a header field's name, the header wins.

The colliding payload value is stored unchanged, and is retrievable as part of the raw payload by whatever holds it, but it is not exposed through field resolution, SELECT, WHERE, aggregation, access control or result maps. Suppression is applied before descendants are flattened, so a payload key named timestamp removes its entire subtree from the query surface, not just itself.

An emitter SHOULD avoid payload keys that collide with header names.

3.22.1.2 Flattening #

Payload maps are flattened recursively, path segments joined with .: a payload {source: {name: "x"}} exposes the field source.name.

Each map key on a queryable path MUST be a MessagePack string matching:

[A-Za-z_][A-Za-z0-9_-]*

Note that . is not permitted in a segment, though it is permitted in an identifier generally (§3.19) — a key containing a dot could not be distinguished from a path through two maps.

A key that is not a string, contains ., or does not match the grammar is stored unchanged and is not queryable: it does not resolve, does not appear in a result map, and has no field identity for access control. An empty map produces no field at all.

Maps are containers; every non-map value, arrays included, is emitted at its flattened path (§3.17).

If two payload entries flatten to the same path, the first in MessagePack map order wins and later duplicates are suppressed.

3.22.2 Log fields #

timestamp, origin, is_error, message, boot_id, job_id

The set is closed. There are no payload fields and no flattening, and a collector MUST reject any other log field name as a parse error rather than resolving it to null. A log record has a fixed shape, so a name outside it is a mistake the collector can see — unlike an event payload field, which may legitimately be absent from a given record.

is_error is a boolean in the query language, and compares against true/false or against 1/0.

3.22.3 Metric fields #

timestamp, boot_id, name, type, value

Every other name resolves to a label. Ingestion refuses labels colliding with these five (§3.10), so the flat namespace is unambiguous by construction rather than by a precedence rule.

type is the series type as a string: "counter", "gauge" or "histogram".

3.22.4 What a record contains #

Event records carry the header fields plus every non-suppressed flattened payload field, as top-level keys.

Log records carry the log fields.

Raw metric sample records carry timestamp, boot_id, name, type, value, and the series' labels as top-level keys.

Aggregated metric results carry name, type and value, plus labels when the result belongs to one label set. They carry boot_id only when the query restricted the samples to exactly one boot by a boot_id equality predicate, in which case the value is that boot ID; a result that could span boots omits it rather than picking one.

Aggregation results in event and log mode carry the group key fields and the aggregate output, with the fixed schemas of §3.23.

3.22.5 SELECT #

SELECT narrows a result record to the named fields. It is valid only for non-aggregating event and log queries.

A collector MUST reject SELECT combined with COUNT BY, TOP N BY, DISTINCT or GROUP, and MUST reject it in metric mode: all of those have fixed output schemas, and a clause that reshapes a fixed schema is a contradiction rather than a refinement.

SELECT is applied last, after every other phase (§3.23). It controls the shape of the output and nothing else: a field not selected is still available to WHERE, to SORT, and to grouping. Narrowing what is displayed MUST NOT narrow what is filtered on.

EVENTS [type_pattern] [clauses…]

3.23.1 The type pattern #

The primary selector is an optional event type pattern, placed immediately after EVENTS.

EVENTS kacs.access_denied      -- exactly that type
EVENTS kacs.*                  -- every type beginning "kacs."
EVENTS *.denied                -- every type ending ".denied"
EVENTS kacs.*.denied           -- kacs.access.denied, kacs.token.denied, …
EVENTS                         -- every type

* is the only metacharacter, and matches zero or more of any character, dots included. ?, [ and { have no special meaning and a collector MUST NOT treat them as any. Matching folds case, like every string comparison (§3.20).

A pattern with no * is exactly WHERE event_type == "…". A pattern whose only * is trailing is exactly WHERE event_type STARTS_WITH "…". Anything else is a glob.

3.23.2 Origin class aliases #

origin_class accepts named aliases as well as its integer values:

EVENTS WHERE origin_class == kacs SINCE 1h ago

These are the only aliased values in the language. A collector MUST accept both forms and MUST treat them as identical.

3.23.3 Aggregation #

Grouping equality, canonical representatives and tie ordering are defined in §3.21. Every aggregation below has a fixed output schema, and rejects SELECT (§3.22).

3.23.3.1 COUNT BY #

Counts records grouped by one field, ordered by count descending.

EVENTS SINCE 24h ago COUNT BY event_type

Output: {<field>: representative, count: <unsigned integer>}.

3.23.3.2 TOP N BY #

COUNT BY with a limit — the N most frequent values.

EVENTS SINCE 1h ago TOP 10 BY process_guid

Output: the COUNT BY schema.

3.23.3.3 DISTINCT #

The distinct values of one field.

EVENTS SINCE 24h ago DISTINCT event_type

Output: {<field>: representative}.

3.23.3.4 GROUP #

Groups by one or more fields, followed by an aggregation function: COUNT, or SUM, AVG, MIN, MAX with a field argument.

EVENTS SINCE 1h ago GROUP origin_class COUNT
EVENTS SINCE 1h ago GROUP origin_class, event_type COUNT
EVENTS SINCE 1h ago GROUP event_type AVG queue_depth

Output, for GROUP a, b:

Group-key fields carry canonical representatives (§3.21).

3.23.3.5 What is aggregated #

For SUM, AVG, MIN and MAX, records whose field is null or non-numeric are excluded from the aggregate — not treated as zero. COUNT counts every record regardless. If no record in a group contributes a numeric value, the group's aggregate is null and the group is still present, because COUNT of it is still meaningful.

3.23.3.6 Result types #

• COUNT returns an unsigned integer.
• SUM over integers returns an integer when the exact mathematical sum fits in signed or unsigned 64 bits. If it does not, or if any input was a float, it returns a float64. If that would be non-finite, the query MUST fail with an error rather than returning an infinity.
• AVG returns a float64 whenever at least one numeric value contributed.
• MIN and MAX return the winning value itself, under exact numeric comparison. When an integer and a float tie, the integer wins.

3.23.4 Ordering #

Without SORT, results are ordered by timestamp descending, ties broken as §3.21 requires.

3.23.5 INDEX #

EVENTS INDEX target_sid

INDEX asks the collector to prioritise a field for query acceleration immediately, rather than waiting for it to be observed often enough to be prioritised automatically. It exists for incident response, where the field that suddenly matters has never been queried before.

INDEX is an administrative operation, not a query. It returns no records. A collector MUST check the caller's token against a Security Descriptor governing administration of the collector — one distinct from the read-path descriptors of §3.28 — and MUST refuse a caller that does not hold it. A collector without such a descriptor MUST refuse INDEX outright.

A collector MAY treat INDEX as advisory and MAY decline the request, shed the acceleration later, or do nothing at all. It is a hint about priority; the accelerations a collector maintains are its own business, and a conforming collector that maintains none accepts INDEX and has nothing to do.

There is no command to undo it, because there is nothing to undo: a collector reconsiders its own accelerations continuously and the hint decays with disuse.

LOGS [FROM origin[, origin…]] [ERROR ONLY] [CONTAINING "text"] [clauses…]

Log mode has three primary selectors rather than one, all optional and all combinable. Each is sugar for a WHERE predicate, and each exists because it is the thing a person actually types.

3.24.1 FROM #

Selects by origin. Several may be listed, comma-separated.

LOGS FROM loregd
LOGS FROM loregd, peinit
LOGS

FROM is exactly WHERE origin == "…" for one origin and WHERE origin IN ("…", "…") for several.

Origins are written as identifiers (§3.19) or as quoted strings. A conforming origin is always an identifier (§3.7).

3.24.2 ERROR ONLY #

Selects lines that came from standard error.

LOGS ERROR ONLY
LOGS FROM loregd SINCE 1h ago ERROR ONLY

It is exactly WHERE is_error == true, and like every clause it may appear anywhere after LOGS without changing the meaning (§3.18).

3.24.3 CONTAINING #

Selects lines whose message contains the given text — a substring match, folding case like every string comparison (§3.20).

LOGS CONTAINING "connection refused"
LOGS FROM loregd CONTAINING "failed to open"

It is exactly WHERE message CONTAINS "…".

CONTAINING is a log-specific keyword because searching text is the primary operation on log data, and the primary operation deserves the shortest spelling. It is a substring scan, not an indexed text search: a collector MUST NOT restrict what it matches, and combining it with SINCE is what keeps it affordable.

3.24.4 Projection and aggregation #

SELECT narrows non-aggregating results to named log fields, and is additive across clauses (§3.22).

COUNT BY, TOP N BY, DISTINCT and GROUP work exactly as in event mode (§3.23), with the same fixed output schemas, the same result types, and the same prohibition on combining them with SELECT.

LOGS SINCE 1h ago COUNT BY origin
LOGS SINCE 1h ago TOP 5 BY origin

3.24.5 Ordering #

Without SORT, results are ordered by timestamp descending, ties broken as §3.21 requires.

3.24.6 No payload fields #

Log mode has a closed field set (§3.22). A collector MUST reject an unknown log field name as a parse error, in a WHERE, a SORT, a SELECT or a grouping clause alike.

This differs from event mode, where an unknown name is a payload field that resolves to null. The difference is that a log record's shape is fixed and known: a name outside it cannot be a field that this record happens to lack, so treating it as null would answer a question the client did not ask.

METRIC name[label_selector] [transform] [aggregation] [clauses…]

Metric mode evaluates rather than searches. A collector MUST reject SELECT in metric mode: the result schemas are fixed (§3.22).

3.25.1 Selecting series #

The primary selector is a metric name, optionally followed by a label selector in brackets. The name supports * with the same glob semantics as an event type pattern (§3.23).

METRIC cpu.usage
METRIC cpu.*

The brackets — present, absent, or present and empty — decide how multiple matching series are handled, and this is the distinction that governs the rest of the mode.

No brackets — aggregate. Every matching series is combined into one result.

METRIC cpu.usage                    -- average across all cores
METRIC cpu.usage MAX                -- maximum across all cores

Empty brackets — break out. Each series is returned separately.

METRIC cpu.usage[]                  -- latest value per core
METRIC cpu.usage[] SINCE 1h ago     -- a time series per core

Filled brackets — select. Only series matching the label predicates.

METRIC cpu.usage[core="0"]
METRIC cpu.usage[core="0", host="srv1"]
METRIC disk.usage[device STARTS_WITH "sd"]

Label predicates are comma-separated and combined with AND. They use the operators of §3.20, and within a label selector = is accepted as equality alongside ==. Label keys are identifiers; values are identifiers or quoted strings. An absent label resolves to null, so [device IS NULL] selects the series that carry no device label.

3.25.2 Homogeneity #

After the name, the label selector, WHERE predicates and access filtering have been applied, the remaining series MUST be of one type. A collector MUST reject a selection spanning more than one type at execution time, with an error asking for a narrower name or an explicit WHERE type == ….

A selection resolving to zero series returns no records — a successful query with an empty result, not an error.

The rule exists because every function below is defined on one type. A selection mixing counters and gauges has no meaningful rate, and a selection mixing either with histograms has no meaningful value at all.

3.25.3 Function stages #

Function keywords execute in fixed stages regardless of where they were written:

1. Transform — RATE, DELTA, P50, P95 or P99. Operates within each series independently and produces scalars. At most one per query.
2. Terminal aggregation — either a scalar aggregation (AVG, MIN, MAX, SUM) or a window aggregation (AVG_OVER, MIN_OVER, MAX_OVER, SUM_OVER). At most one per query; specifying both a scalar and a window aggregation is a parse error.

The pipeline operates on scalars throughout. Counter and gauge samples are already scalar; a histogram sample is not scalar until a percentile function has been applied. A collector MUST therefore reject, at execution time when the type is known, a query that resolves to a histogram series without a percentile function, or that applies RATE, DELTA, or any scalar or window aggregation directly to one.

Every output is a finite binary64. If any computation would produce NaN or an infinity, the query MUST fail with an error rather than returning it.

3.25.4 Transforms #

3.25.4.1 RATE and DELTA #

DELTA is the change between consecutive samples; RATE is that change per second. Both apply only to counter series, and a collector MUST reject them on a gauge or histogram at execution time.

Both use the same pair construction. Samples of one series are taken in ascending timestamp order, with a deterministic tiebreaker among samples sharing a timestamp. Each consecutive pair (s1, s2) whose s2 falls inside the effective query range, and where s2 is later than s1, produces one scalar at s2's timestamp. The immediately preceding sample before the first in-range one MUST be used as s1 for the first pair, when such a sample exists — without it the first point of every range would be missing, and a chart would show a notch at the start of every window.

The adjusted delta is s2 - s1 when the value rose, and s2 alone when it fell, because a fall means the counter restarted from zero. RATE is that adjusted delta divided by the elapsed seconds. A pair with non-positive elapsed time contributes nothing.

METRIC http.requests.total SINCE 1h ago RATE
METRIC http.requests.total SINCE 1h ago DELTA

3.25.4.2 P50, P95, P99 #

Percentiles of histogram series only; a collector MUST reject them on a counter or gauge at execution time. Each histogram sample yields one value.

Evaluation is nearest-rank over the sample's cumulative counts: for percentile q, compute rank = ceil(q × total_count), and take the first boundary whose cumulative count is at least rank.

A sample with total_count == 0 yields no value. A sample whose rank falls above the final cumulative count — meaning the percentile lies in the overflow region above the highest boundary — also yields no value, because the distribution does not record where in that region it lies.

METRIC request.duration P95
METRIC request.duration[origin="loregd"] SINCE 1h ago P99

3.25.5 Scalar aggregations #

AVG, MIN, MAX and SUM reduce scalars to one value. They MUST NOT be applied to a histogram series directly.

What they aggregate over depends on the brackets:

• Bracketed, so one result per series: over time, within each series.
• Unbracketed without SINCE: over the latest transformed value of each matching series. The result timestamp is the greatest of the contributing timestamps. A series that cannot produce a value — a RATE with fewer than two samples, say — contributes nothing.
• Unbracketed with SINCE: valid only when the selector resolves to zero or one series. More than one MUST be rejected with an error asking for a window aggregation.

METRIC cpu.usage AVG
METRIC http.requests.total RATE SUM
METRIC cpu.usage[] SINCE 1d ago AVG
METRIC cpu.usage[core="0"] SINCE 1h ago MIN

The unbracketed default aggregation, when no SINCE and no explicit function is given, is AVG. No implicit scalar aggregation is added when a window aggregation is present.

If nothing contributes to an aggregation, the query returns no record for that output group. Otherwise the output timestamp is the greatest contributing timestamp — for RATE and DELTA, the later sample of the contributing pair.

3.25.5.1 Why unbracketed plus SINCE needs a window #

A collector MUST NOT synthesise a merged time series from samples that do not share timestamps.

Two series sampled at unrelated moments cannot be averaged point by point without inventing values between the points, and interpolation would make the collector responsible for a number nobody measured. A window aggregation supplies the common time grid explicitly, which is why it is required rather than assumed.

3.25.6 Window aggregations #

AVG_OVER, MIN_OVER, MAX_OVER and SUM_OVER take a duration and produce one value per window. They require SINCE; a collector MUST reject a window aggregation without one as a parse error.

Windows are fixed and aligned to Unix-epoch multiples of the duration — not to the query's start — so that the same window boundaries fall in the same places for every query. The output timestamp is the window start. Windows with nothing in them are omitted rather than emitted as null.

METRIC cpu.usage SINCE 1d ago AVG_OVER 1h
METRIC cpu.usage[] SINCE 1d ago AVG_OVER 5m
METRIC http.requests.total SINCE 1h ago RATE SUM_OVER 5m
METRIC request.duration P95 SINCE 1h ago AVG_OVER 5m

AVG and AVG_OVER are different keywords and a collector MUST NOT treat them as synonyms: AVG produces one value for the range, AVG_OVER one per window.

For raw and percentile-transformed values, a window contains the scalars whose timestamps fall inside it, and the function is applied to those. No interpolation is performed.

For RATE and DELTA with a window aggregation, each series first produces at most one scalar per window: the window DELTA is the sum of reset-adjusted deltas for pairs whose later sample is in the window, and the window RATE is that divided by the elapsed seconds those pairs covered. The preceding-sample rule applies to the first pair of each window. The terminal aggregation then combines the per-series window values — so RATE SUM_OVER 5m sums the series' five-minute rates, and RATE AVG_OVER 5m averages them. Where the selector resolves to exactly one series, all four window functions return that series' window value.

Bracketed window queries keep labels in the result rows. Unbracketed ones omit them, unless the selector resolved to exactly one series.

3.25.7 Without SINCE #

With no SINCE, the query returns the latest value.

METRIC cpu.usage[core="0"]
METRIC cpu.usage[]
METRIC cpu.usage

"Latest" is per series, by timestamp with the deterministic tiebreaker. For RATE and DELTA, it is the latest valid consecutive pair with positive elapsed time; a series with no such pair returns nothing.

3.25.8 Boot filtering #

Samples carry boot_id but series continue across boots (§3.13). A query MAY restrict to one boot:

METRIC cpu.usage[] WHERE boot_id == "550e8400-e29b-41d4-a716-446655440000"

A boot-filtered metric query MUST be evaluated from raw samples. A collector MUST NOT serve one from any pre-computed aggregate that is not itself partitioned by boot.

3.25.9 Results #

One record per raw sample; one per valid pair for RATE and DELTA, timestamped at the later sample; one per histogram sample that yields a percentile; one per window for window aggregations; one for a scalar aggregation.

A histogram result carries only the percentile in value. The boundaries, counts, total and sum are not returned by the query language in this revision, in any mode.

{timestamp: 1714000000000000000, boot_id: "{550e8400-…}", name: "cpu.usage", type: "gauge", core: "0", value: 42.7}
{timestamp: 1714000300000000000, name: "cpu.usage", type: "gauge", core: "0", value: 39.8}

Without SORT, metric results are ordered by timestamp ascending (§3.21) — the opposite of events and logs, because a metric result is read as a series rather than as a list of occurrences.

A cross-type filter narrows one data type by a condition on another. It is the only correlation mechanism in the language; there is no join.

EVENTS kacs.* SINCE 1h ago WHERE METRIC cpu.usage[core="0"] > 80
LOGS FROM loregd SINCE 1h ago WHERE EVENT kacs.access_denied EXISTS
METRIC cpu.usage[] SINCE 1h ago WHERE EVENT synthetic.storage_error EXISTS
EVENTS kacs.* SINCE 1h ago WHERE LOG loregd CONTAINING "error" EXISTS

3.26.1 How it is evaluated #

A collector MUST evaluate the cross-type condition first, producing the set of time ranges over which it holds, and then apply those ranges as additional timestamp bounds on the primary source.

The condition is evaluated against the referenced data's own resolution — the metric's sample interval, or the density of matching events — and not once per record of the primary source. It is computed once for the query.

3.26.2 Metric conditions #

WHERE METRIC operates on raw scalar samples of counter and gauge series only. Transform, scalar aggregation and window aggregation keywords are not valid in one, and a condition resolving to a histogram series MUST be rejected.

The selector MUST resolve to zero or one series. Zero produces no true ranges. More than one MUST be rejected with an error asking for a bracketed or narrower selector.

Within the effective query range, a sample's value is treated as active over [sample.timestamp, next_sample.timestamp), clipped to the range, and the final sample stays active through the upper bound. A collector MUST include the latest sample before SINCE as the initial state when one exists; without it the condition would be false from the start of every range until the first sample inside it, which for a fifteen-second sampling interval is fifteen seconds of wrongly excluded records. If no earlier sample exists, the condition is false until the first in-range sample.

Samples sharing a timestamp are ordered deterministically; the earlier ones create zero-width intervals and the last at that timestamp is the active value.

This is interpolation of a kind, and it should be understood as such: it assumes the condition held continuously between two samples. A metric that crossed a threshold and crossed back between samples is invisible.

3.26.3 Existence conditions #

WHERE EVENT … EXISTS and WHERE LOG … EXISTS are true when at least one matching record lies near the primary record in time. The event type supports * globbing (§3.23); the log form names an origin and optionally a CONTAINING text.

"Near" is a centred half-open window of a configured width W. With lower = floor(W / 2) and upper = W - lower, the condition is true for a primary timestamp t when a matching record exists with:

timestamp >= t - lower
timestamp <  t + upper

Equivalently, a matching record at e contributes the true range [e - lower, e + upper). When W is odd the extra nanosecond falls on the upper side, so the width is exactly W and never W ± 1.

3.26.4 The lookback limit #

A collector MUST bound how far back a cross-type filter may scan.

If the effective query range exceeds the limit, a collector MUST reject the cross-type filter with an error saying the range is too large, and the error SHOULD suggest narrowing it with SINCE or UNTIL.

A query with a cross-type filter and no SINCE MUST be rejected. An unbounded cross-type scan is never permitted, in any mode, at any configured limit.

The reason is that a cross-type filter reads a second store in full before the first query begins. Its cost is set by the referenced data's density, which the client did not select and cannot see, so a query that looks cheap can scan a hundred times more than it returns.

3.26.5 Cost #

A cross-type filter is efficient when it is selective — narrow true ranges eliminating most of the primary source — and expensive when it is broadly true, which is the case where it also eliminates nothing. A condition that holds across the whole range costs the full scan of both stores and returns exactly what the query would have returned without it.

STREAM turns a query into a live tail. It is a flag, may appear anywhere in the string, and takes no argument.

Streaming is available for event and log queries only. A collector MUST reject STREAM in metric mode.

3.27.1 The shape of a streaming query #

1. The collector executes the query normally and sends the initial result set as "ok" messages.
2. It sends "watch" (§3.16). The query is established at this point and not before.
3. It stays open. As records are committed, it evaluates them against the query and sends those that match.
4. It continues until the client disconnects, an error terminates it, or the collector shuts down.

There is no "end" message for a streaming query, ever.

3.27.2 What may be streamed #

Raw record queries and DISTINCT queries. A collector MUST reject STREAM combined with COUNT BY, TOP N BY or GROUP as a parse error — those produce one answer about a set, and a set that is still growing has no answer yet.

A collector MUST reject STREAM combined with UNTIL. An upper time bound and an unbounded live tail are contradictory requests.

SINCE is permitted and applies to both phases, resolved against the evaluation time captured at query start (§3.19).

3.27.3 What still applies during the watch phase #

Access control, the primary selector, the SINCE bound and every WHERE predicate — cross-type conditions included — are evaluated against each new record.

SORT, TAKE and SKIP apply to the initial result set only. Streamed records are delivered in commit order and a collector MUST NOT reorder, limit or skip them: there is no total order over records that have not arrived, and applying TAKE to a stream would silently end it.

SELECT applies to streamed records as it does to initial ones.

3.27.4 DISTINCT streaming #

EVENTS kacs.* DISTINCT process_guid STREAM
LOGS DISTINCT origin STREAM

A DISTINCT stream emits a value the first time it is seen, and never again. The output schema is DISTINCT's fixed one (§3.23) in both phases.

The initial result set is the complete distinct set visible at query start, after access control and every filter. The collector then holds a seen set initialised from it. Each newly committed record that passes access control and the filters is reduced to its value for the field, and emitted only if that value is not already in the seen set under the grouping equality of §3.21; emitted values are then added.

A collector MUST bound the seen set.

If initialising the set or inserting a value would exceed the bound, the collector MUST terminate the query with an error. It MUST NOT evict: "not seen before" is the entire meaning of the output, and a set that forgets would re-emit values it had already reported, which is worse than stopping.

A collector MUST reject DISTINCT … STREAM combined with SORT, TAKE or SKIP, so that the seen set always corresponds to the complete initial visible set. SELECT is already invalid with DISTINCT (§3.22).

3.27.5 Cross-type conditions during the watch phase #

The pre-computed time ranges of §3.26 describe the past. A collector MUST NOT reuse them for streamed records.

For a metric condition, the selector has already been required to resolve to exactly one series (§3.26). For each committed batch, the collector finds that series' active sample at the batch's latest candidate timestamp under §3.26's interval rules and evaluates the condition against it. If no sample is active there, the condition is false. A false condition filters out the whole batch; a true one leaves the batch to be filtered by the remaining predicates as usual.

For an existence condition, the collector applies §3.26's centred window to each candidate record's own timestamp. These are evaluated per record, not per batch, because a matching record may be near some of a batch and not the rest.

3.27.6 Backpressure #

If a client cannot keep up, the collector MUST drop the query rather than buffer for it.

Backpressure is detected on the socket send buffer: when a result message cannot be sent because the buffer is full, the collector MUST terminate the query immediately and MUST NOT block on the send. It sends an error if the socket will still take one, and closes otherwise.

Streaming MUST NOT slow or block ingestion. A streaming client is the lowest-priority consumer of a collector's time, and a slow one is disconnected rather than accommodated — the same principle as §3.4, applied on the way out.

3.27.7 Latency #

Delivery latency is bounded below by the collector's commit interval for the store concerned, because a record is only streamable once it is committed. A client that needs lower latency than that is not served by this interface: the KMES ring buffer is the lower-latency path and is specified in PSPK.

A collector MUST enforce read access on every query, against the token captured when the client connected (§3.14).

How it does so is its own design, and the mechanism the mainline collector uses is described in the eventd TRMP. What this chapter fixes is the part a client can observe: which results it gets, and what it is told about the ones it does not.

3.28.1 The unit of access is the concrete identifier #

Access is resolved per concrete identifier — the event type, log origin or metric name a stored record actually carries (§3.2) — and not per query, per store, or per pattern the query happened to write.

A collector MUST resolve each identifier that a query's data could touch independently. A broad selector authorizes nothing by itself: EVENTS with no pattern, EVENTS kacs.*, LOGS with no FROM, and METRIC cpu.* are all resolved identifier by identifier, and a client permitted to read one matching identifier and not another sees only the first.

Identifiers are matched to rules by dot-delimited prefix, most specific first, falling back to a wildcard default: for kacs.access_denied, a rule for kacs.access_denied, then one for kacs, then the default.

A collector MUST fail closed. If no rule resolves — including because the default is missing — access is denied.

3.28.2 Filtering is silent #

Records and fields removed by access control are removed without comment. A collector MUST NOT indicate in a response that anything was withheld, and a client MUST NOT assume a result set is complete.

The consequences are precise and a client needs all of them:

• A record whose identifier the client may not read is absent, not redacted.
• A field the client may not read is absent from the record, and is indistinguishable from a field the record never carried (§3.17).
• COUNT, COUNT BY, TOP N BY, DISTINCT and every aggregation reflect only authorized records. A count is a count of what the client may see.
• A cross-type condition referencing data the client may not read evaluates as though no matching data exists (§3.26). It does not fail the query.
• TAKE and SKIP page over the authorized records only.

3.28.3 Access control runs before everything #

A collector MUST remove unauthorized records from the logical row set before predicates, transforms, grouping, aggregation, sorting, pagination and projection (§3.18).

This is not tidiness. Counting, ordering or paginating over records a client may not read leaks them through the count, through the ordering, and through the gaps in pagination — a client could establish how many records of a type it cannot read exist, and roughly when, without ever seeing one.

A collector MAY reach the result however it likes: pushing the authorization down into its storage engine, or reading candidates and discarding them before aggregating. What it MUST NOT do is produce a different answer from the one filtering-first produces.

3.28.4 Denied fields do not fail the query #

When a query references a field in a predicate, a grouping, a sort or an aggregation, and some matching identifier does not grant that field, the records under that identifier contribute nothing — exactly as if their identifier had been denied outright.

A collector MUST NOT reject the query.

Authorization for a field is resolved from the field as written, against each concrete identifier, and does not depend on whether any record of that identifier actually carries it. Payload fields vary between records of the same type, so a rule that turned on presence would be undecidable before the scan it was meant to authorize.

3.28.5 What is not a field #

Derived aggregate outputs — count, sum, avg, min, max — are not source fields, have no access identity of their own, and are visible whenever the client is authorized for the records and the source fields they were computed from.

Values internal to preserving query semantics — row identifiers, series identifiers, ordering tiebreakers, series type checks — are likewise not query-language fields (§3.21). A metric result's value is a source field, because it is a raw sample or a scalar derived from raw samples.

3.28.6 Errors say nothing #

A collector MUST NOT include a value the client is not authorized to read in any error message (§3.16), including in errors raised by internal consistency checks.

3.28.7 Streaming #

Access decisions made for the initial result set are reused during the watch phase, but a collector MUST resolve any new concrete identifier that appears in a streamed batch and check it before using the record or its distinct value — a new event type or a new log origin appearing mid-stream has never been authorized.

If a rule changes during a streaming query, a collector MUST re-check subsequent batches against the new rule.

The token does not change. It was captured at connection (§3.14), so a client whose group memberships change mid-stream continues to be evaluated against what it connected with, and a client whose access is revoked keeps receiving records until it disconnects.

3.28.8 The write path is not access-controlled #

Nothing on either ingestion channel is authorized per record (§3.4). Access control here is a read-path mechanism only, and the Security Descriptor on each ingestion socket is the whole of the write-path control (§3.3).

The consequence is that origin and metric name are self-asserted (§3.7, §3.10). Any process that can reach an ingestion socket may write under any origin or metric name it likes, including one belonging to another program — which permits fabricating a plausible operational record, or burying a real one under noise attributed elsewhere.

Read-path rules limit who can see data written under a given identifier; they do nothing about who wrote it. A collector MUST NOT present a stored origin or metric name as evidence of provenance, and a client MUST NOT treat one as authenticated.

There is no version number on any of the three interfaces. No datagram carries one, no query message carries one, and there is no exchange in which either party could state or discover what the other speaks.

That is a deliberate consequence of the shapes chosen, and it is worth being explicit about, because it means every rule below is the only mechanism available.

Ingestion is one-way over a datagram socket: there is no reply in which a collector could announce a version and no state in which a producer could remember one. The query channel could carry a version — it is a stream, and it has a request message — and does not, because a version field is only useful if a party may then behave differently, and a client cannot usefully vary: it either asks a question the collector understands or does not.

What replaces negotiation is a set of rules under which both sides may change without either being told.

3.29.1 Unknown fields are ignored #

A collector MUST ignore fields it does not recognise in a log record (§3.7), in a metric record (§3.11), and in a query request (§3.15).

This is what allows a field to be added. A producer built against a later revision may send a field this collector has never heard of, and the record is still stored; a producer built against an earlier one omits a field that has since been added, and the record is still stored because everything added is optional.

A field added to any of these three maps MUST therefore be optional, and a collector MUST NOT require one to be present.

3.29.2 Unknown values are refused, not ignored #

The rule does not extend to values.

An unrecognised type in a metric record discards the record (§3.12); a first token that is not a mode fails the query (§3.18); an unrecognised keyword is a parse error. A collector MUST NOT guess at an unrecognised value, and MUST NOT skip a field it recognised but could not interpret.

The asymmetry is the point. An unknown field is something the sender knows about and this collector does not, and ignoring it loses only what was never understood. An unknown value in a known field is the sender saying something specific about this record, and proceeding without understanding it stores something other than what was sent.

3.29.3 Response statuses #

A client MUST treat a response whose status it does not recognise as an error terminating the query, and MUST discard the "ok" messages it has received for that query unless "end" or "watch" had already arrived (§3.16).

A status is the control flow of the response stream, so there is no ignoring one: a client that skipped an unknown status would be waiting for a terminal message that had already been sent, or treating an incomplete result as complete. Failing is the only safe reading.

A collector MUST NOT introduce a new status for a condition that the four existing ones can express.

3.29.4 What may change without notice #

• New optional fields in a log record, a metric record or a query request.
• New fields in result records. A client MUST tolerate a key it does not recognise, and MUST NOT reject a record for carrying one.
• New query keywords, clauses and functions. A client sending one the collector does not know receives a parse error, which is the correct answer.
• Wording of any error string (§3.16).
• New event types, log origins and metric names. These are data, not interface; nothing enumerates the valid set of any of them.

3.29.5 What may not change #

• The meaning of an existing field, in either direction. A field is added or it is left alone.
• The type of an existing field.
• The four response statuses, or the rule that exactly one terminal message ends a query.
• The framing of §3.15, which has no version field and therefore no way to change compatibly.
• A required field becoming optional, or an optional one becoming required.

3.29.6 Limits are not the interface #

The declared bounds — the datagram ceilings (§3.6, §3.9), the query message ceiling (§3.15), the concurrency and timeout bounds (§3.14, §3.16), the existence window and lookback limit (§3.26) — are configuration, and an administrator may change any of them.

A collector MUST behave identically at any value in its supported range. A producer or client MUST NOT infer a bound from having exceeded one, or from not having exceeded one, and MUST NOT depend on the mainline defaults quoted in this chapter.

The one place this bites is the log and metric datagram ceilings, which a producer cannot discover and which silently discard what exceeds them (§3.6). Lowering either is a change to the contract with every producer on the system, and there is no mechanism by which any of them will find out.

A conforming implementation of any role MUST satisfy every requirement in this chapter. This section collects the obligations that are not tied to one message.

3.30.1 A collector #

Serve three separate channels. Two SOCK_DGRAM for ingestion, one SOCK_STREAM for queries, each on its own socket, each protected by a Security Descriptor established before it accepts anything (§3.3).

Never exert backpressure. No producer stalls because of a collector, under any load, in any failure state (§3.4).

Never react to input. No event, no log entry, no client-observable counter, in response to a malformed, unwanted or excessive submission (§3.4).

Validate at the stated scope. Datagram, record, or field — as §3.8 and §3.12 set out, and no more broadly. In particular a malformed record MUST NOT cost the valid records batched with it, and a malformed optional field MUST NOT cost a log record.

Store what you were given. A log message byte-for-byte, an event payload unmodified, a timestamp uncorrected, a histogram's boundaries in the order sent (§3.8, §3.10, §3.5).

Preserve gaps. No interpolation, no backfill, no synthesised sample (§3.13).

Identify every query client from the connection, before executing anything, and refuse the query if you cannot (§3.14).

Order totally and deterministically. Every result, for a fixed set of stored records, in one order — so that SKIP and TAKE mean something (§3.21).

Use query-language semantics, not your storage engine's, for every comparison, ordering, grouping and equality test the language defines (§3.20, §3.21).

Enforce access before you compute, per concrete identifier, failing closed, and silently (§3.28).

Bound everything a client can consume: concurrent queries, streaming queries, message size, query time, distinct-stream values, cross-type lookback (§3.14, §3.15, §3.16, §3.26, §3.27).

Behave identically across your configured ranges (§3.29).

3.30.2 A producer #

Send well-formed records and accept that malformed ones vanish without notice (§3.8, §3.12).

Stay within the datagram ceiling, batched or not — and know that you cannot discover it (§3.6).

Choose a stable, conforming identifier. An origin or metric name matching the identifier grammar, naming you distinguishably, and using dots for hierarchy — because it is what access rules are written against and what queries select on (§3.7, §3.10).

Timestamp at production, not at submission (§3.7).

Bound your label cardinality, and keep histogram boundaries fixed for the life of a metric (§3.10, §3.13).

Never assume delivery. No acknowledgement exists, none is coming, and a record that mattered should have been an event (§3.4).

Never change a metric's type. Doing so ends the series silently and permanently (§3.10).

3.30.3 A client #

Tolerate unknown keys in result records, and unknown statuses as errors (§3.29).

Discard partial results. An error before "end" or "watch" means every "ok" message for that query is void (§3.16).

Assume nothing about completeness. Results are silently filtered by access, counts count only what you may see, and an absent field is indistinguishable from a denied one (§3.28).

Assume nothing about provenance. An origin and a metric name are what the producer claimed (§3.28).

Do not parse error strings (§3.16).

Open one connection per query (§3.14).

3.30.4 What this chapter does not require of a collector #

A conforming collector need not accelerate anything, pre-compute anything, shard anything, or retain anything for any particular period. It need not honour INDEX beyond accepting it (§3.23). Its storage, indexing, retention and query planning are entirely its own, and every requirement above is stated about the answer rather than about how the answer is reached.

Every bound this chapter requires a collector to enforce, with the value and adjustable range of the mainline collector. The mainline values are informative: a conforming collector chooses its own, and a producer or client MUST NOT depend on any of them (§3.29).

The mainline configuration key names are those of eventd, whose configuration is catalogued in the eventd TRMP §A.

3.A.1 Ingestion #

3.A.2 Queries #

3.A.3 Cross-type filtering #

3.A.4 Fixed by this chapter #

These are not configuration and a collector MUST NOT vary them.

3.A.5 The relation between two of them #

The query message ceiling MUST NOT be smaller than the largest record a collector can store, because a record that will not fit in a response fails every query that reaches it (§3.15). The ingestion ceilings bound what a producer can deposit; the query message ceiling bounds what can be handed back. Nothing enforces the relation automatically, and the mainline defaults do not satisfy it.

An index of the language, and of where each construct is valid. The normative definitions are in §3.18 to §3.27; nothing here adds a rule.

3.B.1 Shape #

EVENTS [type_pattern]                             [clauses…]
LOGS   [FROM o[, o…]] [ERROR ONLY] [CONTAINING s] [clauses…]
METRIC name[label_selector] [transform] [aggregation] [clauses…]

3.B.2 Clause validity #

WHERE and SELECT are the only repeatable clauses (§3.18).

3.B.3 Combinations that are rejected #

"Parse" failures need no data. "Execution" failures depend on what the store holds, so the same query string may succeed on one system and fail on another.

3.B.4 Metric functions #

Transforms feed aggregations; a query may have at most one of each (§3.25).

3.B.5 Operators #

== != > >= < <= STARTS_WITH ENDS_WITH CONTAINS IN NOT_IN IS NULL IS NOT NULL, combined with AND and OR (§3.20).

There is no NOT and no =.

3.B.6 Literals #

3.B.7 Fields #

3.B.8 Aliases #

origin_class accepts userspace (0), kmes (1), kacs (2), lcs (3). These are the only aliased values in the language (§3.23).

3.B.9 Default ordering #

All ties are broken to a total order (§3.21).

The three interfaces here are not novel, and each has a well-known counterpart whose shape informed it. What follows compares the contracts — this appendix is about wire shapes and the obligations they place on either side. The eventd TRMP §1.4 compares the systems.

3.C.1 Log ingestion #

The closest relative is journald's native socket: a Unix datagram socket, world-writable, accepting a self-describing record from any local process, with no acknowledgement and no notification of loss. The agreements are substantive — datagram rather than stream, self-asserted identity, silent drop under pressure, a forwarder bridging programs that only know standard output.

The differences are three. The record here is MessagePack rather than a line-oriented key-value text format, because the collector already carries a MessagePack decoder for event payloads and a second parser would be a second thing to get wrong. Severity is a boolean rather than a syslog priority, because a forwarder can distinguish two file descriptors and inventing eight levels from two would be a guess presented as data (§3.7). And a batch is a first-class datagram shape rather than a stream of records, which is what lets a forwarder amortise the syscall without giving up the datagram's all-or-nothing property.

Classic syslog over /dev/log is the older relative, and the departure from it is the same one journald made: a record with named fields rather than a formatted line that every consumer re-parses with a regular expression.

3.C.2 Metric ingestion #

The shape is StatsD's: push, datagram, fire-and-forget, no registration, sender-named series. It is the opposite of Prometheus's, where the collector pulls from endpoints it has been configured to know about.

The choice follows from the loss model rather than from taste (§3.9). A pulling collector must reach every producer on a schedule, which makes it responsible for their availability; pushing keeps a slow or dead producer invisible except for the gap it leaves.

What is taken from the Prometheus data model rather than from StatsD is the identity of a series: a name plus a set of labels, with each distinct label combination a distinct series, and the cardinality warning that comes with it (§3.10). The histogram is Prometheus's cumulative-bucket form, including the property that the top bucket is an overflow whose contents are counted but not located.

Two things are deliberately absent. There is no text exposition format, because nothing scrapes. And there is no summary type — a producer that has already computed its own quantiles cannot submit them, because quantiles do not aggregate and a stored one could not be combined with another (§3.25).

3.C.3 The query interface #

The unusual choice here is having a query language at all.

journald exposes a cursor and a set of field matchers, and computation belongs to the client. The Windows Event Log exposes XPath over an XML representation. Prometheus exposes PromQL, a genuine language, but only for metrics. This interface puts one language over all three data types, with a shared clause vocabulary and per-type modes (§3.18).

The reason is access control. Filtering, grouping and aggregation must happen on the side that knows what the caller may see, because a count computed by a client is a count of what the client was given and a count computed by the collector can be a count of what the client is entitled to (§3.28). A cursor interface pushes the computation across the trust boundary and takes the enforcement point with it.

The framing — a length-prefixed MessagePack request, a sequence of chunked result messages, one terminal message — is unremarkable and deliberately so. What it does not have is more interesting: no version field (§3.29), no error codes (§3.16), no multiplexing (§3.14), and no cursor. A query is one connection, and paging is SKIP and TAKE over a total order (§3.21) rather than an opaque token the collector must keep state for.

3.C.4 Where these interfaces sit #

This chapter defines the two interfaces a Peios service manager offers: the control channel, by which a program manages services, and the notification channel, by which a supervised service reports on itself.

Both are Unix-domain sockets between userspace parties, and both have a publicly implementable side. A monitoring tool, an orchestration agent, a shell utility, or a privileged action broker implements the client side of the control channel. Every supervised service that reports readiness, sends keepalives, or preserves file descriptors across a restart implements the producer side of the notification channel.

4.1.1 The roles #

The manager is the process that supervises services. It listens on both channels. On Peios this is peinit, running as PID 1, but nothing here depends on that beyond the manager being a single process holding both sockets.

A client connects to the control channel to issue commands and read answers. A client is any process; it holds no special relationship with the manager beyond the one its token establishes.

A service is a process the manager started, and speaks the notification channel about itself. A service does not connect to the control channel in that capacity — a program that does both is acting in two roles.

Requirements are stated against the role, not the program.

4.1.2 What this chapter covers #

• the two channels, their addressing, and how each is reached
• message framing and encoding on both
• how a client's identity is established, and how a command is authorised
• the command set, the response shapes, and the error vocabulary
• what a command does to a service in each of its states
• how a service's notification is authenticated, and what a service may say
• the file-descriptor store
• the rules under which either channel may be extended
• the conformance requirements for each role

4.1.3 What this chapter does not cover #

• How the manager supervises anything. Dependency resolution, restart policy, timers, cgroups, the boot sequence and shutdown are the manager's own design. This chapter defines what a client can ask for and what it is told, not how the answer comes about.
• How service definitions are expressed. On Peios they are registry keys, administered like any other registry data. That is the service manager's own design.
• What a service state means. The vocabulary is fixed here (§4.B) because it appears on the wire; what causes a service to be in one of those states is not.
• Kernel interfaces. Establishing a peer's identity and evaluating an access decision are kernel operations, specified in PSPK and in the kernel's own reference manual.

Service. A named unit of execution the manager supervises. Service names are opaque to this chapter except for the character restriction in §4.8.

Job. One process execution. A service that has been restarted has had more than one job.

Operation. A requested state machine action on a service, with an identity and a lifecycle of its own. Lifecycle commands do not act directly; they create operations, and an operation is what a client observes and waits on.

Activation generation. A counter the manager increments each time a service begins starting. It distinguishes one incarnation of a service from the next.

Right. A named permission on a service or on the manager itself, represented as a bit in an access mask and evaluated against a Security Descriptor. §4.7.

Dependent-satisfying state. A service state in which the services that depend on the service may proceed. Which states these are is the manager's design; that a state is or is not one of them is observable through the state vocabulary.

Terminal state. For an operation, one of completed, failed, cancelled, merged or aborted. An operation in a terminal state does not change again.

Frame. One newline-terminated line on the control channel, carrying exactly one JSON object.

Datagram. One message on the notification channel, carrying zero or more KEY=VALUE lines and optionally file descriptors.

The two channels differ in almost every respect, and the differences are deliberate.

4.3.1 Why the notification channel is a datagram socket #

A service reporting on itself must not be able to block the manager, and must not block itself. A stream socket gives both parties a queue that fills, and a service writing into a full queue either blocks — hanging a service on the manager's scheduling — or gets an error it has to handle in the middle of doing something else.

A datagram socket has neither problem. A send either goes or is dropped, and the manager can drain at whatever rate it manages. The cost is that a notification can be lost, which is why nothing in §4.19 is a transaction: every field is either idempotent or a statement of current condition, and a service that needs a lost keepalive to have arrived sends another one.

4.3.2 Why the control channel is a stream socket #

A command has an answer, and a client waiting for one needs to know it did not arrive rather than assuming. It also needs framing: a request can be large, and a response certainly can.

4.3.3 Reaching either socket #

Both sockets are protected by the Security Descriptor on the socket's own inode, and a party that may not reach the socket is refused when it connects or sends, before any content is exchanged.

The manager MUST NOT rely on POSIX mode bits for this. On a Peios system access to a filesystem object is routed through its Security Descriptor, mode bits are not consulted, and a chmod on either socket has no effect whatever.

The manager MUST ensure that each socket, and each directory containing one, carries a Security Descriptor that admits the parties intended to use it. A socket created where nothing inheritable applies acquires no descriptor, and an object with no descriptor is denied to every caller — so a manager that leaves this to chance produces a socket nobody can reach, including principals its own default policy grants access to.

The manager MUST listen on a Unix SOCK_STREAM socket at a well-known path. On Peios that path is:

/run/services/peinit/control.sock

The socket MUST exist for as long as the manager is serving, and the manager MUST unlink it when it stops.

The manager MUST create the listening socket and every accepted connection with close-on-exec set, so that no connection descriptor is inherited by a process the manager starts.

4.4.1 A connection #

A client connects, issues one or more commands, and closes. The manager MUST NOT require a client to issue any command before another, and MUST NOT hold state across connections: a connection carries an identity (§4.6) and nothing else.

Requests on one connection MUST be answered in the order they were received. The manager MAY read no further frames from a connection while a response on it is outstanding.

4.4.2 Limits #

The manager MUST enforce three limits, and MUST make their values discoverable to an administrator through the same configuration surface that sets them. The values a Peios service manager uses by default are in §4.A.

Concurrent connections. A connection accepted while the manager is already at its limit MUST be closed at the socket level, without a response. There is no error code for this condition: the manager has declined to enter the protocol at all, and a client MUST treat an immediate close with no response as a refusal rather than as a protocol error.

Request size. A request frame whose content exceeds the limit MUST be answered with REQUEST_TOO_LARGE and the connection MUST then be closed. The limit applies to the frame's content and MUST NOT count the terminating newline, so a request of exactly the limit plus its newline is within bounds.

Idle timeout. A connection with no request outstanding MAY be closed once it has been idle for the configured period. The manager MUST NOT treat a connection as idle while a request on it is outstanding — in particular a connection blocked on a wait=true operation (§4.13) is not idle, however long the operation runs, and MUST be held open until the operation resolves. Such a connection is bounded by the operation's own timeout, not by the idle timeout.

A connection closed for idleness MUST be closed without a response.

4.5.1 Frames #

Every message in both directions is one frame: a single JSON object, serialised compactly, followed by one 0x0A byte. This applies to requests and to responses alike, and the manager MUST terminate every response with a newline.

Framing is byte-oriented and is performed before any JSON is parsed. A 0x0A byte ends the frame wherever it appears, so a raw newline inside what a sender intended as a JSON string does not produce one frame with an embedded newline — it produces two malformed ones. (A raw 0x0A inside a JSON string is not valid JSON in any case; the \n escape sequence is unaffected and is the way to carry a newline in a value.)

The manager MUST NOT emit pretty-printed JSON, and MUST NOT emit more than one object per frame.

4.5.2 Encoding #

Frames are UTF-8. The manager MUST reject a frame that is not well-formed UTF-8 with MALFORMED_REQUEST.

4.5.3 What is malformed #

The manager MUST answer with MALFORMED_REQUEST when a frame:

• is empty — a bare newline with no content;
• is not well-formed UTF-8;
• is not valid JSON;
• is valid JSON but not an object. An array, a string, a number, true, false and null are all malformed requests.

4.5.4 Closing after an error #

The manager MUST distinguish two classes of failure, because they say different things about the connection.

A frame-level failure means the manager cannot trust the stream's framing any more: it does not know where the next frame begins. MALFORMED_REQUEST for an empty frame and REQUEST_TOO_LARGE are both frame-level. The manager MUST send the error response, discard any buffered input, and close the connection.

A command-level failure means the frame was well-formed and the command in it could not be carried out: unparseable JSON content, an unknown command, missing arguments, a denied access check, an unknown service. The manager MUST send the error response and MUST keep the connection open.

A client MUST NOT assume a connection survives an error response, and MUST be prepared for either.

4.5.5 Timestamps #

Every timestamp field the manager emits MUST be a UTC RFC 3339 string with exactly nine fractional-second digits and the literal offset marker Z:

"2026-06-01T12:34:56.123456789Z"

The manager MUST NOT emit a numeric offset in place of Z, and MUST NOT vary the number of fractional digits.

These are wall-clock instants, presented for a reader. The manager MUST NOT derive elapsed-time decisions — timeouts, retries, ordering — from wall-clock differences, and a client MUST NOT assume that two timestamps in the same response were taken from a clock that did not move between them.

The manager MUST establish the identity of every client from the kernel. There is no credential exchange in this protocol, and a client MUST NOT be able to assert who it is.

4.6.1 Obtaining the identity #

On accepting a connection, the manager MUST obtain the peer's token from the kernel. On Peios this is kacs_open_peer_token, which returns a token descriptor for the peer.

The token obtained is the peer thread's effective token at the moment of the call. A client that is impersonating another principal is therefore captured as the principal it is impersonating, not as its own service identity — which is the intended behaviour: access decisions reflect the identity a client is actually acting under.

4.6.2 When it is captured #

The manager MUST capture the identity once, when the connection is accepted, and MUST use that identity for every command on the connection.

A client MUST NOT expect a change of identity mid-connection to affect authorisation. A client that needs to act under a different identity MUST open a new connection.

4.6.3 Failure #

If the manager cannot obtain the peer's identity, it MUST close the connection without a response. There is no error code, because the manager has no basis on which to decide whether this caller may be told anything at all.

A client MUST treat an immediate close with no response as a refusal. This is the same observable outcome as exceeding the connection limit (§4.4), and a client cannot distinguish the two — deliberately, since distinguishing them would tell an unauthenticated caller about the manager's state.

4.6.4 The identity is not a UID #

The manager MUST NOT use the peer's UID or GID as an authorisation input. Identity on a Peios system is a token, and the token is what the kernel attests.

Every command is authorised against a Security Descriptor, using the peer's token. There is no command the manager performs without a check, and no principal exempt from one.

4.7.1 Rights #

Commands acting on a service are checked against that service's descriptor:

Commands acting on the system are checked against the manager's own descriptor:

4.7.2 Generic mappings #

The manager MUST use these generic mappings when evaluating a descriptor, so that a descriptor written in generic terms means the same thing to every implementation.

For a service descriptor:

For the manager's descriptor:

GENERIC_READ maps to nothing on the manager's descriptor because it governs two actions and no queries.

4.7.3 Per command #

reset requires SERVICE_STOP because clearing a terminal state is the tail of stopping something rather than the head of starting it.

4.7.4 The sequence #

1. If the manager is shutting down, apply §4.15's restriction. The shutdown restriction is evaluated before the access check, so a caller who would have been denied is told the command is invalid for the current state. A client MUST NOT infer anything about its own rights from an INVALID_STATE received during shutdown.
2. Resolve the target. A command naming no service the manager knows of MUST be answered UNKNOWN_SERVICE. The manager MUST NOT synthesise a descriptor for a service that does not exist.
3. Evaluate the access check with the peer's token, the target's descriptor, the appropriate generic mapping, and the required right.
4. On denial, answer ACCESS_DENIED, and record the attempt with at least the caller's SID, the target, and the right requested. The manager MUST NOT deny silently.
5. On grant, proceed.

4.7.5 Filtering rather than denying #

list MUST return only the services the caller may query, and MUST omit the rest rather than denying the command. A caller with no query rights on anything receives an empty list and a successful response.

The manager MUST NOT reveal, through the response, that services were omitted. Reporting the omissions would answer the question the filtering exists to leave unanswered.

4.7.6 Not revealing what a caller may not see #

Where a command names an object the caller may not query, the manager MUST NOT let the answer distinguish "this does not exist" from "you may not see this".

For operation-status this means the authorisation check MUST be evaluated before the operation's existence is reported: a caller lacking SERVICE_QUERY_STATUS on an operation's target MUST receive ACCESS_DENIED whether or not the identifier names a real operation, and MUST NOT receive UNKNOWN_OPERATION for one that exists.

Where the caller's rights cannot be established because the target cannot be resolved, UNKNOWN_OPERATION is the correct answer.

A request is one JSON object.

{"command": "start", "service": "jellyfin", "wait": true}

4.8.1 Fields #

command MUST be present and MUST be a string naming a command the manager implements. A request whose command is absent, is not a string, or names no known command MUST be answered INVALID_COMMAND.

service MUST be present and a string for start, stop, restart, reload, reset and status. Its absence, or a non-string value, MUST be answered INVALID_ARGUMENTS.

wait MUST be a boolean when present. A non-boolean MUST be answered INVALID_ARGUMENTS. Its default is per command (§4.13).

type MUST be present and MUST be exactly one of the three values for shutdown. Anything else MUST be answered INVALID_ARGUMENTS.

operation_id MUST be present and a string for operation-status. A value that is not a well-formed identifier MUST be answered INVALID_ARGUMENTS.

4.8.2 Fields that do not apply #

A field the command does not use MUST be ignored, not rejected. A service on a list, or a wait on a status, is accepted and has no effect.

This is what makes the request shape extensible: a client written against a later revision may send a field an earlier manager does not know, and the earlier manager ignores it. §4.21.

4.8.3 Service names #

A service name is 1 to 128 bytes drawn from [A-Za-z0-9._-]. The manager MUST NOT accept a name outside that set, and a client MUST NOT send one.

The restriction exists because service names are used as path components and as configuration key names by managers that store their definitions in a hierarchy. / and : are excluded specifically: the first because it is a separator wherever the name is used as a path component, and the second because it is conventionally reserved for a manager's own synthetic naming.

Every response carries a status field, which MUST be exactly "ok" or "error". What else it carries depends on which of four shapes it is.

4.9.1 The acknowledgement shape #

Returned by a lifecycle command that created, merged into, queued, cancelled, cleared or executed an operation.

{"status": "ok", "operation_id": "a1b2c3d4-…", "service": "jellyfin",
 "state": "active", "cause": "explicit_start", "warnings": []}

warnings here is an array of strings. The status response uses the same field name for an array of objects (§4.14); a client MUST distinguish them by which command it sent, not by inspecting the array.

4.9.2 The status shape #

Returned by status, and also by a lifecycle command that had no effect — see §4.12. §4.14 gives it in full.

4.9.3 The system shape #

Returned by shutdown:

{"status": "ok"}

Nothing else. A shutdown has no operation to observe and no service to report on. reload-config has its own shape (§4.15).

4.9.4 The error shape #

{"status": "error", "code": "ACCESS_DENIED",
 "message": "caller lacks SERVICE_START on jellyfin"}

code MUST be one of the values in §4.10. message is human-readable and is not normative: a client MUST NOT parse it, match on it, or branch on its content. Two managers answering the same request with the same code MAY word the message differently.

4.9.5 Nullability #

A field that does not apply to the current state MUST be present and null rather than omitted, except where this chapter says otherwise. A client MUST accept null for any field this chapter marks nullable, and MUST NOT treat a null as an error.

The two exceptions are mode, which appears only on a reload response, and job_id in the notification event payloads, which is omitted when there is no job.

The code field of an error response MUST be one of these values. The manager MUST NOT emit any other code, and a client MUST treat a code it does not recognise as an unrecoverable error for that request (§4.21).

4.10.1 Distinctions a client can rely on #

UNKNOWN_SERVICE versus ACCESS_DENIED. A caller that may not query a service still receives UNKNOWN_SERVICE for a name that does not exist and ACCESS_DENIED for one that does but which it may not touch. This chapter does not attempt to hide the existence of services from a caller that can name them: the list filtering (§4.7) hides them from a caller that cannot.

INVALID_STATE versus ACCESS_DENIED during shutdown. During shutdown the state restriction is evaluated first, so a caller who would have been denied receives INVALID_STATE instead. A client MUST NOT infer that it holds a right from receiving INVALID_STATE.

OPERATION_TIMEOUT does not cancel anything. It reports that the client's wait ended, not that the operation did. The operation continues and can still be observed with operation-status.

4.10.2 Codes are not extensible without a version #

The manager MUST NOT introduce a new code without the version negotiation in §4.21. A client written against this revision will not recognise one, and the only safe thing it can do with an unrecognised code is fail the request — so a new code silently converts a handled condition into an unhandled one.

Five commands move a service through its state machine. None of them acts directly: each creates, merges into, queues or cancels an operation, and the operation is what actually happens.

reload defaults to not waiting because a reload's outcome is often advisory, and a client usually wants the identifier rather than the block. reset is synchronous and completes before the response is sent, so waiting on it would mean nothing.

4.11.1 Operations #

The manager MUST return an operation identifier from any lifecycle command that created, merged into, queued, cancelled, cleared or executed an operation. The client uses it to poll (§4.14) or to correlate.

The manager MUST NOT invent an operation solely so that it has an identifier to return. Where a command had no effect, or the service was already in the state asked for, the manager MUST return the status shape instead of an acknowledgement (§4.12).

4.11.2 Merging #

Where an operation of the same type is already in flight for the same service, the manager MUST merge the new request into it and MUST return the existing operation's identifier.

A merged caller therefore receives an identifier that may be older than its own request, whose requested_at precedes the moment it sent the command. This is correct — that is when the work being waited on began — and a client MUST NOT treat an identifier older than its request as an error.

The manager MUST NOT tell the caller that a merge occurred. A merge is not a distinguishable outcome, and a client cannot do anything with the knowledge.

4.11.3 What completion means #

4.11.4 Timeouts #

Every operation has a maximum lifetime, derived from the target service's own configured timeouts.

The lifetime is measured from the operation's creation, including any time it spent queued. From the caller's point of view they have been waiting since they sent the command, and an operation that sat behind another for longer than its lifetime MUST fail rather than begin.

An operation whose lifetime expires while it is still queued MUST fail, and MUST fail its waiters. Expiry of the operation object MUST NOT by itself authorise the manager to act on the service — a stop operation that timed out while waiting its turn does not license signalling the service ahead of that turn.

A command sent to a service in an unexpected state MUST receive a defined answer. The manager MUST NOT silently do nothing.

4.12.1 The outcomes #

act — create an operation and execute it. The manager returns the acknowledgement shape.

merge — an operation of this type is in flight. The command merges into it (§4.11) and the caller receives that operation's identifier.

queue — the operation is created and left pending; it executes once the operation ahead of it completes. The caller receives the new operation's identifier.

defer — an automatic restart is already pending for this service. The manager MUST create a pending start operation, or merge into a deferred one that already exists, and MUST NOT execute it until the existing delay has elapsed. A start MUST NOT shorten a pending restart's delay.

already — the service is in the state the command would take it to and no operation of this type is in flight. The manager MUST return the status shape, not an error and not an acknowledgement.

noop — the command has no effect. The manager MUST return the status shape.

clear — the service returns to inactive. This is a synchronous outcome; the manager returns an acknowledgement.

cancel — abort or cancel the operation in flight, then proceed.

invalid — the command is not valid for this state. The manager MUST answer INVALID_STATE.

ok — status is answered from any state.

4.12.2 The backoff column #

A service in backoff is down with an automatic restart pending, and the four lifecycle commands mean different things there:

• start defers, as above, and honours the remaining delay.
• stop cancels both the pending restart and any deferred start, and the service becomes inactive.
• restart cancels the automatic restart and performs a caller-initiated one.
• reload and reset are invalid: there is no process to reload and no terminal state to clear.

4.12.3 The abandoned column #

Every lifecycle command except reset is invalid on an abandoned service. reset clears it. Nothing else is meaningful while processes the manager could not terminate are still present.

4.12.4 A service being withdrawn #

A manager MAY keep supervising a service whose definition has been removed while an instance of it is still running. In that condition the manager MUST answer start, restart and reload with UNKNOWN_SERVICE, MUST accept stop, and MUST report the condition in the status shape (§4.14). This holds whatever the service's state.

wait decides whether a lifecycle command's response is sent immediately or held until the operation resolves.

With wait: false, the manager MUST respond as soon as it has accepted the operation, with the acknowledgement shape and the service's state at that moment.

With wait: true, the manager MUST hold the connection open and respond when the operation reaches a terminal state, with the same shape and the service's state, cause and warnings observed at that time.

A connection blocked on a wait is not idle (§4.4). The manager MUST NOT close it for idleness however long the operation runs.

4.13.1 When a wait ends #

OPERATION_TIMEOUT ends the client's wait, not the operation. The operation continues, and the client MAY still observe it with operation-status using the identifier it never received — which it does not have. A client that needs to survive a timeout SHOULD issue the command with wait: false, keep the identifier, and poll.

4.13.2 Reload mode #

A response to a reload command MUST carry a mode field saying how the reload resolved:

mode MUST be present on every response to a reload, including one sent with wait: false — in which case it MUST be advisory, since nothing has been observed yet.

A client MUST treat these three as an exhaustive set and MUST NOT expect a fourth. A manager MUST NOT introduce one without §4.21.

The distinction between confirmed and advisory is the whole value of the field: a service that implements the reload handshake (§4.19) can be known to have reloaded, and one that does not cannot. failed does not mean the service stopped — a failed reload leaves a running service running.

Three commands read state and change nothing.

4.14.1 status #

Returns everything the manager knows about one service.

{
    "status": "ok",
    "service": "jellyfin",
    "state": "active",
    "cause": "explicit_start",
    "status_text": "Listening on port 8096",
    "current_job": {
        "id": "a1b2c3d4-…",
        "type": "service_main",
        "pid": 1234,
        "started_at": "2026-06-01T12:34:56.123456789Z",
        "identity": "jellyfin-svc"
    },
    "current_operation": {
        "id": "e5f6g7h8-…",
        "type": "start",
        "source": "admin"
    },
    "health": "healthy",
    "uptime_seconds": 86400,
    "definition_removed": false,
    "warnings": []
}