Peios System Protocols Userspace

This document defines the Peios System Protocols Userspace (PSPU): the protocols and interchange formats by which the foundational userspace components of a Peios system agree with one another.

An interface belongs in this document when both of these hold:

• it is a contract between userspace parties, at least one of which is a component the system is built from rather than an application running on it; and
• the interface is public — a third party is expected to implement one side of it.

The parties need not exist at the same moment. A live protocol has two processes in conversation; an interchange format has a producer and a consumer that never meet, and the artifact between them carries the contract. Both are in scope, because what makes something belong here is that two independently written parties must agree on it.

1.1.1 These protocols are not conformance requirements #

A system that does not offer a protocol in this document is still Peios. The components that speak these protocols are one answer to a problem, not the definition of the platform; a system that solves the same problem with different components conforms exactly as well.

They are specified because they are public even so. A third party writing a component to plug into one side of one of these protocols needs the contract written down, and needs it to stay put. What they are not is a bar anyone must clear.

For each interface, this document covers:

• for a live protocol: the channel, its direction, which party connects to which, message framing and encoding, the messages exchanged, and the shape of a conversation
• for an interchange format: the layout of the artifact, how it is identified and versioned, and how a consumer validates one it receives
• how a party announces itself or is identified, and how its counterpart establishes what it is and what it may speak for
• the rules under which the format may be extended
• what each party must declare about itself, and what its counterpart validates rather than believes
• the conformance requirements for each role

This document does not cover:

• Standards a system MUST implement to be Peios — defined in PGSS
• Protocols spoken across the kernel boundary — defined in PSPK
• The binary structures these interfaces carry — defined in PCDS
• How a component stores its data, reaches the answers it gives, or produces the artifacts it emits — its own design
• Which counterparts a system is configured to trust, and how that configuration is expressed — the consuming component's own design
• Administering a component's contents — its own design, unless the component publishes a protocol for other programs to administer it with, as the local principal store does (§10)

The fourth of those is the point of the whole document. A component is asked a question and gives an answer, or is asked for an artifact and produces one; how it arrives there is exactly what different components exist to do differently.

1.1.2 Stability #

Publication here is a commitment that the contract is written down and will not change out from under an implementation. Each specification states its own rules for extending its wire or file format; those rules are the supported way for an interface to grow.

The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY in this document are to be interpreted as described in RFC 2119. Text set off as a note is informative, not normative.

Everything else — roles, byte order, sizes, layout tables, notation, strings, timestamps, citation, and the external standards this anthology depends on — is defined in the Conventions book and is not restated here. PSPU departs from none of it.

Where a chapter needs a convention of its own, that chapter states it.

This chapter specifies the Principal Source Interface (PSI): the protocol by which an authentication authority federates identity to separate processes that hold it.

Two roles participate.

The authority is the process that mints tokens and creates logon sessions. It is the party asking. On PSI it listens; it never dials out (§2.3). It is also the party that speaks PGSS Logon to its clients, and PSI exists so that it can answer them.

The source, in full a principal source, is a process that is authoritative for some set of principals: it verifies their credentials and says who they are. The source role is publicly implementable: any process an authority has been configured to accept MAY register as a source, and a third party writing one for a directory, a hardware token service or an identity provider is the case this chapter is written for. A conforming source is the subject of the source obligations in §2.22.

A source is not a store, necessarily. A local source owns its bytes; a directory-backed source owns nothing and forwards the question. The interface deliberately does not distinguish them, which is why the term is "source" rather than "store".

What a source is emphatically not is a component of the authority. It runs as a separate process, at lower trust, and it cannot mint anything (§2.4).

This chapter covers:

• the channel, its direction, and why sources connect inward (§2.3, §2.6)
• message framing, the conversation identifier, and the rules under which the format may be extended (§2.7)
• registration: how a source announces itself, how the authority establishes what it is, and what domain it may speak for (§2.8 to §2.10)
• the relayed interrogation, and its relationship to PGSS Logon (§2.5, §2.12)
• assertion and refusal, the terminal messages of a source conversation (§2.13)
• querying a source outside a logon, so that an authority can serve PGSS Logon's identity lookup (§2.15, §2.16)
• what a source must declare about itself before an authority may cache its answers (§2.8, §2.17)
• scope: what a source may claim about identity, separately about membership, and separately again about POSIX identifiers (§2.18 to §2.20)
• relaying a principal's change of their own credential to the source that holds it (§2.21)
• the obligations binding on each role (§2.22)
• relaying a principal's addition or removal of one of their own credentials, such as an SSH public key, to the source that holds it (§2.23)

This chapter does not cover:

• The logon protocol itself, specified in PGSS.
• Tokens, SIDs, sessions and privileges — described in the Peios Kernel TRM, with SIDs, security descriptors and claim attributes specified in PCDS.
• How a source stores identity, or verifies a credential.
• Derivation — what a token ends up containing — which is the authority's, applying local policy (PGSS §2.1).
• Which sources a machine trusts, and how that is configured.
• Which identifier range a source is given, and how that is configured. The rules the assignment must satisfy are §2.20.
• Administering a source's contents.

The third of those is the point of the whole interface. A source is asked a question and gives an answer; how it reaches the answer is exactly what different sources exist to do differently.

2.1.1 PSI is not a conformance requirement #

PGSS Logon is a Peios Generic System Standard: a system that does not offer it is not Peios. PSI is not. It is the interface an authority uses to reach the processes that know who exists, and a system running entirely different authentication infrastructure is still Peios.

It is specified because it is a public interface even so. A third party writing a principal source needs the contract written down, and needs it to be stable. What it is not is a bar anyone must clear.

Terms defined in the Peios Kernel TRM (token, logon session, privilege), in PCDS (SID, security descriptor, claim attribute) and in PGSS (authority, client, principal, conversation, round, credential material, prompt, derivation) are used here with the same meaning and are not redefined.

Source. A process that is authoritative for some set of principals: it verifies their credentials and says who they are. Called a principal source in full.

Registration. The exchange in which a source announces itself and the authority decides whether to accept it. Precedes any conversation.

Domain. The SID namespace a source is authoritative for. Every principal a source may assert lives under it. See §2.10.

Source conversation. One logon's exchange between the authority and a source, distinguished from other concurrent ones by a conversation identifier. Not to be confused with a PGSS Logon conversation, which is between a client and the authority; one of each exists per logon.

Assertion. A source's terminal message stating who a principal is. The only successful outcome a source can produce.

Originator. The verified identity of the process that requested a logon, as established by the authority from the client's connection. Relayed to the source, which cannot learn it any other way.

Service SID. A SID derived from a service's name, placed in that service's token by the init system, and unforgeable by anything else. How a source's identity is established (§2.9).

Membership scope. The constraint on which groups a source may assert. Separate from identity scope, which constrains whose identity it may assert at all. Sections 2.18 to 2.20 exist because these are different questions with different answers.

Relative identifier. In this chapter, a POSIX identifier as a source states it: an offset within the range the authority assigned that source, never an absolute number (§2.20). Where the SID sense is meant — the last sub-authority of a SID — the text says so.

The authority listens. Sources connect to it. The authority never initiates a connection to a source.

This is the most consequential shape decision in this chapter, and it is worth being explicit about what it buys.

2.3.1 Why the direction matters #

The authority holds the privilege to mint tokens. It is the most privileged userspace process on the system. An authority that dialled out would need, in its configuration, a list of paths to connect to — and a process holding that privilege having a configurable list of things to go and talk to is a liability out of proportion to the convenience.

Because sources connect inward, the authority's sockets are accept()-only. It never opens an outbound connection to anything, for any reason.

2.3.2 What follows #

Restart is the source's problem. A source whose connection drops reconnects. The authority does not retry, does not queue, and does not track sources it has not heard from. A source that has gone away is simply not registered.

A source is not required to exist. An authority with no registered sources cannot authenticate anybody, and that is a coherent state rather than an error — it means no identity has been made available to it yet.

Ordering is the init system's problem. The authority must be listening before a source can register, and a service that depends on authentication must start after a source has. Expressing that is a service-ordering question, not a protocol one, and this chapter says nothing about it.

A source says who somebody is. It cannot say anything else, and the protocol is built so that this is structural rather than a rule anyone has to remember.

2.4.1 The success terminal #

PGSS Logon's success terminal is AccessGranted, carrying a session identifier and a token descriptor. If PSI reused it, sources would be minting sessions.

PSI's success terminal is Assertion (§2.13), which carries an identity: a SID, a canonical name, and group memberships. There is no session identifier and no descriptor to attach a token to. A source could not mint one if it wanted to, because there is no message in which to say so.

That is the whole of the mechanism. No capability check, no trust level, no configuration flag — a source cannot mint because the protocol gives it no way to express minting.

2.4.2 What the authority keeps #

Everything else:

• Derivation. What the token actually contains — its privileges, its integrity level, its derived group memberships, its projected identifiers — is the authority's, applying local policy (PGSS §2.1).
• Session creation. The logon session, and the record of which source vouched for it.
• Validation. Every SID a source sends is bytes until the authority has checked it (§2.13).
• Scope enforcement. What a source is permitted to claim (§2.18 to §2.20).
• Peer verification. On every connection it accepts.
• Rate and round limits, and the policing of what a source may ask a client for (§2.12).

2.4.3 Why a compromised source is bounded #

A source that is entirely compromised can lie about the principals in its own domain. It cannot mint a token, cannot elevate anyone's privileges, cannot claim identities outside its domain (§2.18), and — unless configured otherwise — cannot assert memberships outside it either (§2.19).

That bound is the reason for the process boundary. It is not that sources are expected to be malicious; it is that a source is the component parsing credentials from the outside world, and therefore the one most likely to be wrong.

A principal source is an authentication authority for its slice of the world. The authority that federates is an authority over authorities. Once that is seen, most of PSI writes itself.

2.5.1 The relationship #

PSI's interrogation phase is PGSS Logon's, with identical message bodies. CredentialRequest and CredentialResponse carry exactly the bytes PGSS §2.8 defines, and the authority relays them nearly verbatim in both directions.

The consequences are worth stating plainly:

• The source decides what to ask for. Not the authority. The authority does not know what credentials a source requires, and does not need to.
• The authority becomes a relay in the interrogation phase. It is a shorter path than synthesising its own prompts, not a longer one.
• Adding a credential type is a change to sources, not to the authority and not to clients, which already render what they are given (PGSS §2.3).
• A source could be tested in isolation by pointing a PGSS Logon client at it, for the interrogation phase at least.

2.5.2 Where they diverge, deliberately #

Three differences, each for a stated reason.

The success terminal. Assertion rather than AccessGranted, so that a source cannot mint. See §2.4. This is the divergence that matters.

Multiplexing. PGSS Logon is one conversation per connection; the connection is the conversation. PSI carries many concurrent logons over one long-lived connection, so its header adds a conversation identifier (§2.7). The alternative — serialising every logon behind one connection — would make any slow logon a system-wide login stall.

Distinct magic. PPSI rather than PGSL. Two protocols this similar sharing a codec is a cross-protocol hazard: a socket plugged into the wrong daemon would partially work, which is far worse than failing outright. The magic makes it a hard error on byte zero.

The full accounting of what is shared, what is added and what differs is §2.C.

2.5.3 "Just relaying" is loose #

The authority is a relay in the interrogation phase only, and even there it is not passive. It polices what a source may ask a client for (§2.12), it validates what a source asserts (§2.13), and it enforces scope (§2.18 to §2.20). Everything before and after the interrogation is entirely its own.

2.6.1 Socket #

An authority that federates over PSI MUST listen on a SOCK_STREAM Unix domain socket.

Unlike PGSS Logon's path, this one is not normative. PSI is not a conformance requirement (§2.1), and an authority that offers it may put it where it likes provided its sources are told. Mainline's is /run/psi.sock.

2.6.2 Access control #

The socket SHOULD carry a security descriptor. It is DoS protection and nothing more, and an implementation MUST be written as though it were absent.

The reason is that the socket cannot be the boundary. What establishes a source's identity is the peer's token (§2.9), which is checked on every connection. A descriptor that kept casual traffic away would be a convenience; a descriptor relied upon would be a second, weaker access control that someone will eventually assume is doing the work.

An authority MUST therefore bound the number of unregistered connections it will hold open, and the time it will wait for a registration, independently of any descriptor.

2.6.3 Long-lived connections #

A source's connection persists for the life of the source and carries every logon routed to it.

An authority MUST bound the number of registered sources and the number of concurrent conversations per source. A source MUST bound the conversations it will track, and MUST NOT depend on the authority's bookkeeping to do it — a source that trusted the authority's limit would be trusting a bound it cannot verify.

2.6.4 Failure #

A framing error is fatal to the connection, not to a conversation. Once a message has failed to parse there is no way to know where the next one starts, so both parties MUST tear the connection down rather than attempt resynchronisation.

A failed write is likewise fatal. A partial write desynchronises the stream just as a bad frame does, and treating it as a per-conversation error would leave a corrupt connection in use.

Ordinary semantic failures — an unknown principal, a bad credential, a refused logon — are not connection failures. They are Refusal messages (§2.13) and the connection continues.

2.7.1 Header #

Every message begins with a 20-byte header:

The first twelve bytes are PGSS Logon's header, unchanged and at the same offsets. total_len in particular sits where PGSS §2.6 puts it, which is what lets one transport implementation frame either protocol off a stream.

2.7.2 Magic #

PPSI, checked on every message, fatal to the connection when wrong.

This matters more here than it would for an unrelated protocol, because PSI and PGSS Logon share message bodies (§2.5). A CredentialRequest from one is byte-identical to the other's. Without distinct magic, a socket plugged into the wrong daemon would decode several fields correctly before going wrong — which is the failure mode hardest to diagnose and easiest to miss.

2.7.3 Conversation identifier #

The conversation field distinguishes concurrent logons on one connection.

• Conversation 0 is reserved for connection-level messages: Register, Registered and Changed (§2.17). It MUST NOT be used for a logon or a query.
• Logon and query conversations use identifiers from 1 upward, drawn from one space.
• The authority allocates them. A source MUST NOT invent one, and MUST reply on the identifier it was given.
• An identifier is unique among live conversations on one connection. An authority MAY reuse one after a conversation has reached a terminal state.

A source MUST reject a message on a conversation it does not know, and MUST NOT treat it as opening a new one. Only Authenticate (§2.11), Query (§2.15) and EnumerateSource (§2.16) open one, and an authority MUST NOT open one with an identifier already live.

Rejecting means declining to act on it. A source MUST NOT reply on a conversation it does not know: an authority MAY have reused the identifier after a terminal state, so a reply could arrive as a second terminal message for a conversation that has already ended. Discarding it, and recording that it happened, is the whole of the obligation.

A source MUST likewise refuse an Authenticate, Query or EnumerateSource arriving on conversation 0, which is reserved.

2.7.4 Size limit #

A message MUST NOT exceed 81920 bytes — larger than PGSS Logon's ceiling, because a PSI message wraps one.

2.7.5 Message direction #

The high bit of msg_type marks a message sent by the source. This follows PGSS Logon's convention that the bit marks the authority for the matter at hand: on this interface the source is the authority for its own principals, and the logon authority is the one asking.

2.7.6 Encoding #

PSI shares its codec with PGSS Logon, and PGSS §2.6's encoding rules apply here unchanged: little-endian multi-byte integers; UTF-8 strings, length-prefixed and never NUL-terminated; length-framed structures and array elements, skipped to their declared end; u32 element counts on arrays, with stated maxima binding on encoder and decoder alike.

2.7.6.1 SIDs on the wire #

SIDs are carried as opaque bytes, in the binary self-relative form PCDS specifies, never as text.

They are opaque to the codec, which has no business knowing what a SID is. They are emphatically not opaque to the authority, which MUST validate every SID it receives before treating it as identity (§2.13). The obligation to check sits in the process that mints tokens, not in the layer that moves bytes.

A SID MUST NOT exceed 68 bytes — the eight-byte prelude plus fifteen sub-authorities, which is the most the encoding's one-byte count admits.

2.7.7 Body extensibility #

The extensibility rules of PGSS §2.6 apply unchanged: fields are appended only, a new field is optional with a safe default, and a new enumeration value is a breaking change requiring a version bump. The one exception is the capability bitmask of §2.8, for the reason given in §2.B.

One PSI-specific application deserves stating. Authenticate (§2.11) nests a whole LogonStart inside its own length frame rather than inlining its fields. The obvious encoding — LogonStart's fields, then PSI's — is wrong: LogonStart belongs to PGSS Logon and grows on PGSS Logon's schedule, so a field appended there would silently displace the field after it. Nesting lets the two evolve independently.

The same reasoning applies to every shared body PSI carries, and §2.C lists them.

2.7.8 Initial unpublished layout replacement #

The coordinated SSH credential extension replaced the unpublished version-1 layout without retaining older codecs. Current LogonStart capability, required-method and binding fields, Prompt parameters, and all current Assertion fields are mandatory, including explicit empty/zero values. Omitted mandatory fields fail decoding. The compatibility rules above apply to future extensions of this replacement contract; they do not require accepting the superseded development layout. See PGSS Logon §2.D.

A connection opens with registration. Nothing else may precede it.

2.8.1 Register #

msg_type = 0x8001. Source to authority, on conversation 0.

2.8.1.1 source_name #

What the source calls itself. Bounded at 32 bytes because it becomes the authentication-package name on every session the source authenticates — so a token's provenance answers which source vouched for this? rather than merely the authority minted it.

The name is a claim. It MUST be cross-checked against the identity the authority established for itself (§2.9), and it MUST NOT be used for anything else. A mismatch MUST be refused rather than quietly corrected: a service registering under another's name is worth failing on, not normalising.

2.8.1.2 domain #

The SID namespace this source is authoritative for (§2.10).

2.8.1.3 capabilities #

What the source can do beyond authenticating.

An authority MUST NOT send a message a source did not declare it answers, and MUST NOT set a field bit gating a capability the source did not declare. A source declaring nothing authenticates and does nothing else, which is what a source predating these fields is saying by omission — and is the only reading that keeps such a source working.

A source that declares no QUERIES cannot be asked about its principals outside a logon, so they are unresolvable through PGSS Logon's identity channel: they can sign in and will appear as bare numbers everywhere else. That is a coherent configuration and this chapter permits it, but it is almost never what an administrator intended, and an authority SHOULD report it where one will see it — as it reports a source registering with no identifier range.

This is a declaration of capability, not of willingness. A source declaring QUERIES may still answer Refused to any particular question (§2.15); a source declaring ENUMERATES may still refuse a cursor it can no longer honour (§2.16).

2.8.1.4 entry_ttl #

How long, in seconds, the authority may hold an answer from this source before asking again.

Zero means do not cache. A source declaring neither PUSHES_CHANGES nor a non-zero entry_ttl has said its answers must not be held at all, and an authority MUST honour that — see §2.17, where the reasoning for reading silence that way is set out.

entry_ttl and PUSHES_CHANGES are not exclusive, and a source declaring the second SHOULD declare a non-zero first as well. See §2.17: the TTL is the backstop against a notification that was never sent.

2.8.1.5 max_batch #

The largest number of keys the source will accept in one Query (§2.15). Zero means one.

An authority MUST NOT exceed it, and MUST NOT send more than 64 keys whatever the source declared: an encoder MUST NOT declare more, and a decoder MUST read a larger declaration as 64.

A source MUST still validate what it receives. The field is a hint the authority is required to respect, not a guarantee about what will arrive.

2.8.2 Registered #

msg_type = 0x0001. Authority to source, on conversation 0.

Load-bearing beyond its contents. A source SHOULD report itself ready only once it has received this, so that anything ordered after the source finds a system that can actually authenticate rather than merely a process that exists (§2.3).

2.8.2.1 unix_id_base, unix_id_count #

The POSIX identifier range the authority has assigned this source (§2.20). A base of 0 means no range was assigned, and every principal the source asserts will project as unmapped.

Informational. A source counts within its range and asserts relative identifiers; the authority applies the base. A source MUST NOT apply it — see §2.20, where the reasoning is set out in full.

It is sent so that a source's administration tools can show an operator the identifier a principal will really project to, rather than the relative number the source stores. Without it that arithmetic falls to the operator.

An authority that predates these fields sends neither, and a source MUST read their absence as no range assigned — which is what such an authority means, since it has no ranges to assign.

2.8.3 Rules #

1. A connection MUST open with Register on conversation 0. An authority MUST refuse any connection that opens otherwise.
2. An authority MUST bound the time it waits for the opening Register, and close the connection on expiry. A peer that connects and says nothing MUST NOT be able to hold resources indefinitely.
3. An authority MUST NOT admit two sources under one name at the same time.
4. An authority MUST send Registered only after the source is routable, so that a logon racing the acknowledgement cannot find a source that is registered but not yet reachable.
5. A source MUST NOT send any other message before receiving Registered.
6. An authority SHOULD report, where an administrator will see it, that a source registered with no identifier range — its principals will all project as unmapped, and the cause is a configuration omission rather than anything the source did.

The authority MUST establish a connecting source's identity for itself, from the kernel, and MUST NOT take it from anything the source sends.

2.9.1 A source proves nothing #

The mechanism worth recommending is that a source proves nothing at all, because the init system already did.

Where the init system places a service SID in each service's token — a SID derived from the service's name, which only the init system can mint — the authority can:

1. take its list of permitted source names from its own configuration;
2. derive the service SID each of those names implies;
3. read the connecting peer's token and ask which of those SIDs it carries.

The resulting identity is assembled entirely from the authority's configuration and the kernel. Nothing is contributed by the process on the other end. There is no shared secret, nothing to provision, nothing to rotate, and nothing to steal — the derivation is a pure function of a name that only the init system can act on.

2.9.2 What is deliberately not checked #

That the peer is SYSTEM. The service SID subsumes it: the user SID could never distinguish one platform service from another, since they all run as the same principal. Requiring SYSTEM as well would needlessly forbid a future source running under a lesser account, which is a direction worth keeping open.

2.9.3 The allowlist #

An authority MUST NOT accept a source it has not been configured to accept.

An empty configuration MUST mean no source may register, not any source may. An allowlist that fails open is not an allowlist. The visible cost is that a system configured with no sources cannot authenticate anyone, which is the correct way for that mistake to present — loudly, at the first logon attempt, rather than silently at the first compromise.

A source declares the domain it is authoritative for. Every principal it may assert lives under it (§2.18).

2.10.1 It is a claim #

A source generates or is given its own domain, so nothing about the SID can prove the claim is honest. What gives it weight is entirely what the authority does with it.

An authority MUST apply all of the following.

2.10.1.1 1. A domain MUST be declared #

A source that declares no domain MUST be refused. There would be nothing to confine its assertions to, which is the whole purpose of collecting one.

2.10.1.2 2. The shape MUST be checked #

A domain MUST be a well-formed, locally-issued domain SID: revision 1, the NT authority (5), the non-unique prefix 21, and three further sub-authorities — S-1-5-21-A-B-C, four sub-authorities in total.

The shape is the entire check, and it is enough. A source cannot claim S-1-5-32 (BUILTIN), or the NT authority's well-known range, or S-1-1-0, because none of them has that shape. There is no list of forbidden domains to keep in step with the SID catalogue — the permitted shape excludes every one of them by construction.

2.10.1.3 3. Domains MUST be disjoint #

No two concurrently registered sources may claim the same domain. Two authorities for one namespace means whichever answers first decides who a name belongs to, and the other's principals become impersonable by the first.

2.10.1.4 4. A source MUST NOT change domain #

A source that re-registers MUST declare what it declared before. An authority MUST refuse a change.

This is the check that survives a source restarting, and it is worth its cost: every other check passes for a source that is killed and comes back compromised. It still holds the right service SID, its new domain is still a claimable shape, and with itself deregistered there is nothing left to collide with.

An authority MAY hold this record only for its own lifetime. Persisting it means the authority writing state, which is a larger commitment than the guarantee justifies.

2.10.1.5 5. An administrator MAY pin #

An authority SHOULD allow an administrator to configure the exact domain a named source must declare, and MUST refuse a source declaring anything else when one is configured.

A configured pin that cannot be parsed MUST NOT be treated as absent. Absence means no pin; an unparseable value means an administrator tried to apply the control and got it wrong, and silently downgrading that to "unconstrained" removes the control at the moment it was being applied. An authority MUST fail towards refusing the source.

2.10.2 What remains uncovered #

With no pin configured, and another source not currently registered, a compromised source could declare that source's domain and assert its identities. Disjointness catches it only while both are registered.

This is stated rather than solved. Closing it requires someone to write the pin down, and an authority cannot invent that authority for itself — writing it automatically would mean the process holding the token-minting privilege also holding a configuration write handle, which is a worse trade than the gap it closes.

msg_type = 0x0002. Authority to source. Opens a conversation.

2.11.1 start #

The client's LogonStart, nested whole (§2.7). Its fields and their meanings are PGSS §2.7's, unchanged — including that identifier is an unverified claim and supported_credential_types binds what may be prompted for.

2.11.2 originator #

The verified identity of the process that requested this logon, taken by the authority from the client's connected socket and never from a message body.

A source cannot learn this for itself: it is not party to the client's connection, and there is nothing it could ask. The authority relays it because a source may legitimately refuse a logon on the strength of it — an account restricted to console logons needs to know what asked — and that decision needs a trustworthy input.

A source MUST treat originator as established fact and MUST NOT treat any other field of this message the same way.

2.11.3 Routing #

Before sending Authenticate, an authority MUST decide which single source answers.

The credential MUST NOT be offered to more than one source. Trying each in turn with the password hands every source the credentials of every other source's users, including on typos — the failure PAM stacking exemplifies (§2.D).

Resolution therefore happens on the identifier, before any credential exists. Asking several sources "do you own this name?" is a resolution step with no secret in it and is permitted; offering them the answer is not.

An authority SHOULD resolve a qualified name to its owning source and MUST NOT fall back to another source when the owning one is unreachable. A name that can fall through lets anyone who can break a network choose which authority answers for a principal.

2.11.4 Conversation limits #

An authority MUST bound the conversations it opens against one source. A source MUST bound what it will track, and MUST refuse beyond its own limit with AuthorityUnavailable (§2.13) rather than dropping the conversation silently.

Two messages, both carrying PGSS Logon bodies verbatim. They serve a credential change (§2.21) exactly as they serve an authentication, and everything below applies to both.

2.12.1 CredentialRequest #

msg_type = 0x8002. Source to authority. Body is PGSS §2.8's CredentialRequest, byte-for-byte.

The source decides what to ask for, in what order, and over how many rounds. The authority relays it to the client.

2.12.2 CredentialResponse #

msg_type = 0x0003. Authority to source. Body is PGSS §2.8's CredentialResponse, byte-for-byte.

2.12.3 What the authority polices #

The authority relays, but does not relay anything.

An authority MUST refuse to relay a prompt whose credential type is absent from the client's supported_credential_types. PGSS §2.8 makes this the authority's obligation towards the client, and it holds however the authority reached the prompt — a prompt originating in a source is still the authority's to police.

Relaying it would force the client to hard-fail, and a client that guessed instead might echo a secret to the screen. The authority MUST terminate the logon instead.

An authority MUST also enforce its own round and time limits on the relayed exchange (PGSS §2.3), independently of any the source applies. A source that never terminates a conversation MUST NOT be able to hold a client's logon open indefinitely.

2.12.4 Credential handling #

The obligations of PGSS §2.12 bind both parties on this leg as they do on the client's. Credential material reaching a source has been decoded and re-encoded once more than it would have been without federation, and every buffer it passed through on the way is one the obligation covers.

2.12.5 What the authority does not do #

It does not interpret prompts, rewrite messages, reorder anything, or synthesise a request of its own. A source's prompt reaches the client as the source wrote it, which is the property that makes adding a credential type a change to sources alone.

Exactly one terminal message ends a source conversation. An authentication ends in Assertion or Refusal. A credential change ends in CredentialChanged or Refusal, never in Assertion (§2.21).

2.13.1 Assertion #

msg_type = 0x8003. Source to authority. The only successful outcome a source can produce.

Note what is absent: no session, no token, no privileges, no integrity level. A source has no way to express them (§2.4).

All listed fields are mandatory in the coordinated replacement layout. Empty arrays/strings and zero values represent absent assertions; omitted fields are malformed. The authority checks the authenticated method against the client's capabilities and any required method before granting a token.

2.13.1.1 canonical_name #

The source's own spelling of the principal's name. A client may have typed JACK; this is what the principal is actually called.

Carrying it is what makes case-insensitive matching safe: the authority records the canonical form rather than whatever was typed, so a session's records do not vary with a caller's shift key.

A source MUST NOT assert a name that PGSS §2.15 forbids — one carrying a reserved character, a byte outside the printable ASCII range, or a leading or trailing space. The authority MUST refuse one anyway (§2.22), because a name from a source reaches a passwd-format record and an audit line, and by then the damage is the reader's to do.

The obligation is on what a source asserts, not on what it creates. A source that validates a name when an administrator adds it, and not when it reads one back from storage, has enforced nothing against a store it did not itself write.

2.13.1.2 permitted_logon_types #

Which kinds of sign-on this principal may be used for, as the bitmask PGSS §2.16 defines for the LOGON_TYPES field.

A source states the property and MUST NOT act on it. Deciding what to do about a principal's restrictions is derivation, and derivation is the authority's (§2.4) — a source that refused an authentication on these grounds would be making an access decision it has no standing to make, and would do it differently from the authority that also has to.

Zero means the source states nothing, and a source that does not hold the property MUST send zero rather than inventing a value. In particular it MUST NOT send "everything", which would assert that a principal may be used for a credential-free service logon (PGSS §2.19) on no evidence at all.

Sources that state no restriction send zero explicitly.

2.13.1.3 groups #

Each entry is a separate length-framed structure:

A SID and a number. No attributes.

A source asserts which groups a principal belongs to. Whether a group entry is enabled, owner-marked, or deny-only is a decision about how to build a token, and building tokens is the authority's (§2.4). A source saying "this principal is an administrator" is identity; a source saying "and mark that group deny-only" would be reaching into derivation.

Both current group fields are mandatory. The per-entry framing preserves the boundary of any future trailing extension.

A unix_id of 0 means the source does not number this group — the honest answer for a group it does not own. A source naming a well-known group is stating a membership, not claiming authority over what that group projects to; see §2.20.

2.13.1.4 unix_id #

The principal's POSIX identifier, relative to the range the authority assigned this source (§2.20). Zero means the source has no number for this principal.

A source MUST NOT apply its own base. It counts within its range and the authority rebases; a source that added the base itself would have it added twice.

2.13.1.5 primary_group #

Which of the principal's groups projects to the POSIX group id, and becomes the default group of objects the token creates. Empty means the source did not say, and the authority chooses.

It need not appear in groups. The authority is required to place it on the token regardless (§2.22), because a token's primary group must be a group the token carries — so naming a group here is a membership claim, and it is subject to membership scope exactly as a listed group is (§2.19).

That applies to a primary group the source asserted. Where the field is empty and the authority substitutes one of its own, the substituted value is the authority's choice and MUST NOT be tested against the source's membership scope. Testing it would deny every logon from a source that declined to name a primary group, on the strength of a claim that source never made — and an authority's own default is very unlikely to be a sibling of the principal's domain, so the test all but always fails.

2.13.1.6 profile #

PGSS Logon's profile structure (PGSS §2.9), relayed onward to the client unchanged. It is not identity, it decides no access, and the authority does not interpret it.

The one thing the authority does check is the one PGSS §2.9 requires of it: home and shell, when non-empty, are absolute paths. That obligation binds the authority towards its client whatever the value's provenance, so a relayed profile is not exempt from it.

2.13.1.7 claims #

Named, typed attributes fed to conditional ACE evaluation, in the claim attribute format PCDS §5.9 specifies. Each entry is a separate length-framed structure:

A claim is the one field here that is a trusted input to access decisions rather than a statement of identity: a conditional ACE can turn a claim into a grant. Which claim names a source may assert is therefore the same kind of question as which groups it may assert, and belongs with membership scope (§2.19).

An authority MUST reject an assertion carrying a claim it cannot carry to a token — an unsupported value type, a name containing an interior NUL, a value exceeding its limit — rather than dropping the claim. The reasoning is rule 3 below: a dropped claim signs the principal in against a policy nobody stated.

2.13.1.8 What the authority MUST do with an assertion #

1. Validate every SID with a structural check before treating it as identity. Bytes from another process are bytes until checked, and the check belongs in the process that mints tokens rather than in the codec that moved them (§2.7). This includes SIDs carried inside a claim value.
2. Enforce identity scope (§2.18), membership scope (§2.19) — including over primary_group — and numeric scope (§2.20).
3. Fail the logon on a malformed group SID, an unusable claim, an invalid canonical_name, or a unix_id outside the source's range, rather than dropping it. Dropping would sign the principal in with authority the source did not state — a confusing way to be wrong at best, and for a claim, a silent change of the policy that will be applied to them. A source that cannot encode a SID is broken.
4. Drop a logon SID from the asserted groups, loudly. No source is authoritative for one: the kernel mints them per session, and this session's did not exist when the source answered. A source asserting one is either buggy or reaching for a different session's SID, which would forge membership of somebody else's logon.
5. Drop duplicates, first mention winning. A source asserting a group the authority also derives is redundant, not wrong.

Rules 4 and 5 drop rather than refuse because the token still ends up correct, and refusing would punish a principal for a source's defect without making anything safer. Rule 3 refuses because the token would not end up correct.

Rules 4 and 5 run before rule 2. A logon SID is by construction outside the principal's domain, so an authority that applied membership scope first would refuse the logon that rule 4 says to survive by dropping. Duplicates are the same shape of problem. The drops are about what a source should never have sent; the scope tests are about what it is permitted to claim, and only what survives the first is subject to the second.

An authority MUST also place primary_group on the token even when the source did not list it among groups, since a token's primary group must be a group the token carries.

2.13.2 Refusal #

msg_type = 0x8004. Source to authority.

Reuses PGSS Logon's denial vocabulary rather than inventing a parallel one, so that relaying a refusal outward needs no lossy translation.

A source MUST NOT refuse with NoSuchSession (PGSS §2.B). It answers only a request to end a logon session (PGSS §2.22), which the authority decides alone and never puts to a source, so in a refusal it can only be a mistake. An authority that receives it MUST NOT relay it, and SHOULD end the conversation with Internal instead.

CredentialRejected is the same kind of code: it answers only an enrolment (PGSS §2.23), so a source MUST NOT send it in any other conversation, and an authority that receives it elsewhere MUST NOT relay it (§2.23).

A source MUST NOT distinguish an unknown principal from a bad credential — by code, by reason, or by timing (PGSS §2.10, §2.12). The obligation is the source's here, because the source is where the distinction exists to be leaked.

An authority MAY relay reason to the client and MAY replace it. It MUST NOT relay a reason that reveals a distinction the source was required not to make.

msg_type = 0x0004. Authority to source. No fields beyond the header.

Tells a source that a conversation will not continue: the client hung up, a limit was reached, or the authority terminated the logon for reasons of its own.

2.14.1 Why it exists #

Without it, a client that disconnects mid-prompt leaves the source holding conversation state forever. A source cannot detect this for itself — it is not party to the client's connection — so the authority has to say.

2.14.2 Rules #

1. An authority MUST send Abandon for any conversation it opened that will not reach a terminal state, unless the connection itself is being torn down.
2. A source MUST discard all state for the conversation on receipt, and MUST NOT reply.
3. Abandon is not a terminal message from the source, and no Assertion or Refusal follows it. If one arrives anyway, the authority MUST ignore it — the conversation is gone, and a late answer to an abandoned question is at best stale.
4. A source that receives Abandon for a conversation it does not know MUST ignore it. It has already cleaned up, which is the outcome the message wanted.

An authority serving PGSS Logon's identity lookup must be able to ask a source about a principal outside a logon: to render a name for a SID, or a POSIX record for a number.

A query is a conversation like any other. The authority allocates the identifier, Query opens it, and one terminal message closes it.

2.15.1 Query #

msg_type = 0x0005. Authority to source. Opens a conversation.

A key entry is a length-framed structure:

Exactly one of name, sid and relative_id is meaningful, selected by key_type. An encoder MUST leave the others empty or zero.

2.15.1.1 Keys are never absolute #

There is no key type carrying an absolute POSIX identifier, and this is the load-bearing property of the message.

PGSS Logon's lookup accepts one, because that is what getpwuid hands a name resolver. The authority resolves it: it locates the range containing the number, subtracts the base, and asks the owning source by relative identifier.

A source is therefore never asked an absolute number, exactly as it never asserts one during a logon (§2.20).

The reason is not that an absolute number would let a source escape its range — it could not, because the authority refuses a relative identifier at or past the count before adding anything to it (§2.20). The reason is that the arithmetic must exist in exactly one place. A source asked an absolute number would have to subtract its own base to answer, which is the operation §2.20 forbids it, and an authority that asked would have taught it that its stored numbers and the system's are the same numbers. Every subsequent bug in that source would be an off-by-a-base.

2.15.1.2 Batching #

keys is an array so that an authority may ask several questions in one exchange. An authority MAY send a single key, and one that always does is conforming.

The array is here from the outset because adding it later would break every source written against a single-key message. A source with a cheap local store gains little; a source backed by a remote directory gains the difference between one query and a hundred.

A source MUST answer every key it is sent, in order, and MUST NOT reorder, merge or omit results. A source that cannot serve a whole batch MUST refuse the conversation rather than answer part of it.

2.15.2 QueryResult #

msg_type = 0x8005. Source to authority. Terminal.

One result per key, in the order the keys were sent.

A result entry is a length-framed structure:

Everything from present onward is PGSS §2.16's structure, unchanged — the same reuse of message bodies as the interrogation phase (§2.5), and for the same reason: an authority relaying a source's answer outward should not have to translate it.

Where outcome is not Found, everything after it MUST be empty or zero.

2.15.2.1 canonical_name, not qualified #

A source returns its own spelling of the name, as it does in an Assertion (§2.13). It does not qualify it.

Qualification names which source answered, and a source cannot know what it is called in another authority's search order. PGSS Logon requires a qualified name on the way out (PGSS §2.15); producing it is the authority's.

2.15.2.2 Identifiers are relative #

Every identifier in a result — a UNIX_ID field, a reference's unix_id — is relative, exactly as in an Assertion (§2.20). A source MUST NOT apply a base, and the authority rebases before the number leaves it.

2.15.2.3 Outcomes #

A source may send only:

A source MUST NOT send Unavailable: it is answering, so nothing was unavailable to it. The authority produces that outcome when a source does not answer (PGSS §2.18), and a source claiming it would let a working source be recorded as a broken one.

A source MUST NOT send Malformed in a result. A message it cannot parse is a Refusal for the whole conversation (§2.13).

A Refused result is about the object, not about the caller, and an authority MUST NOT relay it outward as PGSS Logon's Refused outcome, which is reserved for a caller that may not make the request (PGSS §2.18). What a source declining to expose an object means to a client is the authority's to decide, and it is not "you lack permission".

A source that declared no QUERIES is a third case again. It has not answered and cannot be asked, so an authority MUST NOT record it as having answered NotFound: a source that was never consulted is not evidence that an object does not exist, and PGSS §2.18 forbids reporting NotFound on the strength of one. It contributes nothing to the search, and §2.8 says what an authority should do about the configuration that produced it.

2.15.3 Scope #

An authority MUST apply identity confinement (§2.18), membership scope (§2.19) and numeric scope (§2.20) to a QueryResult exactly as to an Assertion, and MUST validate every SID in one structurally before using it — including the SIDs of references inside a PRIMARY_GROUP, GROUPS or MEMBERS value, not only the sid of the result itself.

"Exactly as to an Assertion" is meant literally, and it is the sentence an implementation is most likely to satisfy by halves. An authority that confines the object a result names, while relaying the group references beside it unchecked, has left the whole of membership scope unenforced on this channel — and a source with no permission to assert a foreign membership can then report one through a name lookup that it could not report through a logon.

A query is not a weaker channel than a logon. A source that could name a principal outside its domain here would be able to make ls -l display another source's principals as its own — and, worse, could then be believed the next time something compared that name to a SID.

Query asks about objects the authority can already name. Enumeration asks a source to produce them: to fill a POSIX passwd or group table, or to page through a group whose membership will not fit in one answer.

A source is never required to enumerate. A source that declines is fully conforming, and this section exists as much to make declining safe as to make enumerating possible.

2.16.1 EnumerateSource #

msg_type = 0x0006. Authority to source. Opens a conversation.

kind MUST NOT be Any.

An empty of enumerates every object of kind the source holds. A non-empty of MUST name a group, and enumerates that group's members.

That group is one the source holds, or a BUILTIN group (S-1-5-32-…) named by its SID. A BUILTIN group is no source's object, but a source permitted to assert memberships outside its domain records who is in it (§2.19), and the authority asks every such source for its members (PGSS §2.16). Such a source MUST answer with the principals it records in the group — by a listed membership or as their primary group — and with an empty Found where it records none.

cursor is empty on the first request, and otherwise carries the next from the source's immediately preceding reply.

2.16.2 EnumerateResult #

msg_type = 0x8006. Source to authority. Terminal.

An empty next ends the enumeration. A non-empty next means there is more, even where entries is empty.

A source MUST size a page against the smaller of this chapter's message ceiling and PGSS Logon's, because the authority re-encodes what it returns into the latter (§2.A). Neither the 256-entry bound nor the message ceiling here prevents a page nobody can deliver.

A source that will not enumerate replies Refused, with entries and next empty. An authority MUST record it as a source that did not contribute, and MUST NOT retry it for the remainder of that enumeration — across pages as well as within one (PGSS §2.17).

Refused and an empty Found are different answers and MUST NOT be conflated. Found with entries and next both empty says there are none: the group exists and has no recorded members, or the source holds no objects of that kind. Refused says this source is not answering. A source that returns an empty Found where it means the second has told the authority a falsehood it cannot detect, and the authority will go on asking it — the non-retry rule above has nothing to attach to.

The cases most often got wrong, all of which are Refused and not an empty Found: a group whose membership the source will not expose, a key that names an object the source does not hold, a key that names a principal where a group was required, and a cursor the source can no longer honour.

2.16.3 Cursors belong to the source #

A cursor is opaque to the authority. The authority MUST NOT construct, parse or modify one; it relays what it was given.

A source MAY encode anything into a cursor, and MAY refuse one it no longer honours — a store rewritten underneath a half-finished walk is the ordinary case, not an exceptional one. A refused cursor is Refused, and the authority MUST NOT restart the enumeration on the source's behalf.

A source MUST NOT assume a cursor comes back on the same conversation, or on the same connection, and MUST NOT hold per-cursor state it is unwilling to discard.

The authority's own cursor, the one it hands its client, is its to construct — PGSS §2.17 requires it to reject one it did not issue, which it can only do for a cursor it made. A source's cursor travels inside it, not as it.

2.16.4 Why members are here and not only in Query #

MEMBERS is a field of Query (§2.15), so the common case — a small group, whose members fit alongside the rest of the record — costs one exchange.

A group whose membership will not fit is reported through PGSS Logon's TooLarge (PGSS §2.16), and this is where the caller is sent. Paging a membership through the mechanism that already pages is cheaper than a third message, and considerably cheaper than the alternative of a partial member list, which is a wrong answer rather than a smaller one.

2.16.5 Enumeration is not existence #

An authority MUST NOT use enumeration to determine whether a principal exists, and MUST NOT infer from a source declining to enumerate that the source holds nothing.

A directory-backed source able to answer any single question while quite unable to answer all of them is the expected case, not a degraded one.

An authority answering a name lookup for every process on the system will cache. This is the message that lets it.

2.17.1 Changed #

msg_type = 0x8007. Source to authority, on conversation 0. Unsolicited, and never answered.

sid is meaningful only for Object, and MUST be within the source's declared domain (§2.18).

Object scope MUST be used for a deletion as well as a change, and for a creation — an authority may be holding a cached NotFound for a name that now exists.

2.17.2 Obligations #

A source declaring PUSHES_CHANGES (§2.8) MUST send Changed before, or at the same time as, the altered answer becomes observable through Query.

Sending it afterwards leaves a window in which the authority's cache and the source disagree while both believe themselves current — which is indistinguishable, from the authority's side, from the notification never arriving.

A source MAY send All where it could have sent Object. Over-invalidation costs a query; under-invalidation costs correctness.

An authority MUST accept Changed at any time after Registered, including while conversations are open on the same connection, and MUST NOT reply to it.

An authority MUST treat the loss of a source's connection as All for that source. It does not know what changed while it was not listening.

A source declaring PUSHES_CHANGES SHOULD declare a non-zero entry_ttl as well (§2.8). The two are not alternatives: the TTL is the backstop against a notification that was never sent, or was sent and failed to write. Declaring PUSHES_CHANGES with a TTL of zero means a single lost notification leaves the authority holding a stale answer until the connection drops — and a source that tolerated a failed Changed write without tearing the connection down would have made that outcome reachable, which is one of the reasons §2.6 makes a failed write fatal.

2.17.3 Sources that do not push #

A source that does not declare PUSHES_CHANGES is conforming, and many cannot: a remote directory has no way to tell this machine that an account was renamed.

Such a source declares entry_ttl instead (§2.8), and an authority MUST NOT hold its answers beyond it.

A source declaring neither has said it cannot support caching, and an authority MUST NOT cache its answers at all. That is the safe reading of silence, and it is what a source predating this message says by omission.

An authority that does not cache at all satisfies this section trivially, and is conforming. The obligations here bind what an authority may hold, not whether it must hold anything.

2.17.4 What this does not carry #

Changed says that something changed. It does not say what it changed to.

Carrying the new value would make this a second, unsolicited path by which a source could assert identity — one arriving outside any conversation, with no key to check it against, and no logon in progress to refuse. An authority that believed it would have accepted an identity assertion it never asked for.

The authority discards what it holds and asks again through Query, where every scope rule in §2.18 to §2.20 applies.

A source may assert only principals within its declared domain (§2.10). No configuration lifts this.

An authority MUST refuse an Assertion whose user_sid does not lie within the domain the source registered for, and MUST terminate the logon. It MUST apply the same test to a QueryResult (§2.15) and to the sid of a Changed (§2.17).

2.18.1 Containment #

A principal SID lies within a domain when it is the domain's SID plus exactly one relative identifier.

Exactly one, deliberately. S-1-5-21-A-B-C-1000-1 is not a principal of S-1-5-21-A-B-C, and admitting it would let a source that owns one domain mint names in a nested namespace nobody agreed it owned. A domain SID is likewise not a principal of itself.

2.18.2 Why nothing lifts it #

A source that could assert identities outside its domain could hand out another authority's principals to anyone who satisfied its credential check.

The concrete case: a local source holds no domain credential. If it were unconfined, it could produce a domain administrator's identity for anyone who knew a local password. The domain's own authority would never be consulted and would have no way to know.

Confinement keeps a compromised source at "authority over its own domain" — which is what it already was — rather than "authority over everyone".

This is why identity scope and membership scope (§2.19) are separate settings rather than one. They are different questions, and only one of them has a legitimate exception.

Group membership is the question with a legitimate exception, and it needs one.

2.19.1 The default #

By default, an authority MUST refuse an Assertion carrying a group outside the domain of the principal being asserted.

A directory vouches for its own users and its own groups, and nothing else. Without this rule, a directory-backed source could declare its users members of BUILTIN\Administrators — making a remote authority the arbiter of who administers this machine.

The test is relative: are the group and the principal in the same domain? It needs no configuration and no knowledge of which domain belongs to whom, which is why membership scope is enforceable before anything else about scope is settled.

2.19.2 The exception #

An authority SHOULD allow a source to be configured as permitted to assert memberships outside the asserted principal's domain.

A local source needs it. Local group membership of any principal is a local decision: "CORP\Domain Admins is in BUILTIN\Administrators" is a record this machine keeps, not something a domain controller asserts at it. Without the exception, a local source could not express the one thing it is most authoritative about.

An authority MAY grant this to more than one source. Nothing about it is exclusive.

2.19.3 Memberships only, never identity #

A source holding this permission remains fully confined on identity (§2.18).

The two are separate because the risks are not symmetric. A source asserting a foreign membership is making a claim about what a principal may do on this machine, which is a local matter and is the local source's business. A source asserting a foreign identity is claiming to be the authority for somebody else's principal, which is never anyone's business but that authority's.

2.19.4 The primary group is a membership #

primary_group (§2.13) is subject to this section exactly as a listed group is, and an authority MUST apply the test to it.

It would otherwise be a way round: the authority is required to place the primary group on the token whether or not the source listed it, so a source that named BUILTIN\Administrators there and nowhere else would obtain a membership the group array denies it.

An authority that adds an unlisted primary_group to the membership set MUST do so before applying this section, not after. Adding it afterwards reintroduces exactly the route the rule closes.

This binds a primary group the source asserted. A default the authority substituted for an empty field is not the source's claim and MUST NOT be tested against the source's scope (§2.13); an authority that tested its own default would refuse every logon from a source that simply left the field empty.

2.19.5 Claims are the same question, unanswered #

A claim (§2.13) is a trusted input to conditional ACE evaluation, so asserting one can produce a grant just as asserting a membership can. Which claim names a source may assert is therefore the same shape of control as this section — and it is not yet specified.

The gap is stated rather than papered over. An authority federating to a source it does not fully trust should consider claims as it considers foreign memberships, and a future revision is expected to define the control here.

A source's POSIX identifiers are relative. The authority assigns the range and applies the base; a source is told its range but MUST NOT act on it.

Every unix_id in an Assertion — the principal's and each group's — and every identifier in a QueryResult is an offset within a range the authority assigned that source. The authority adds the base before the number reaches a token or a caller.

This is the numeric counterpart of identity confinement (§2.18). That section stops a source naming principals outside its namespace; this one stops it numbering them outside its namespace.

2.20.1 The range #

An authority MUST assign each source a range, as a base and a count, spanning [base, base + count).

An authority MUST reserve a band below every source's base for identifiers of its own — well-known SIDs, service SIDs, confinement SIDs — none of which come from any directory. The band has to be generous, because the last of those categories has no bound.

2.20.2 Rebasing #

Given a relative identifier r from a source with range (base, count), the authority computes base + r, and MUST refuse to produce a number at all when:

• r is 0. Zero is not an identifier; it is how a source says it has no number for something. It MUST NOT become base.
• r is at or past count. The source has reached outside the range it was given.

A number the authority declines to produce projects as unmapped, which the Peios Kernel TRM §3.10.1 defines.

2.20.3 Refuse, never clamp #

An out-of-range identifier MUST be refused. It MUST NOT be clamped to the top of the range, and it MUST NOT be reduced modulo the count.

Both alternatives look like graceful degradation and are worse than a refusal:

• Clamping puts two principals on one number, so a filesystem cannot tell them apart.
• Wrapping lands inside somebody else's range, so one source's principal projects as another source's.

The range is a boundary, not an offset, and the count is what makes it one.

2.20.4 Two numbers a source can never reach #

Because the base sits above the reserved band and a relative identifier cannot escape the count:

1. uid 0. It belongs to the authority's own table and is not reachable by adding a base to anything a source can send.
2. Another source's numbers. Whatever a source asserts, arithmetic confines it to its own range.

Neither depends on the source behaving. They hold because of what the source is able to express.

2.20.5 Well-known SIDs are not the source's to number #

An authority MUST use its own identifier for any SID in its reserved band, and MUST ignore whatever unix_id a source sent alongside it.

A source asserting BUILTIN\Administrators is stating a membership. It is not claiming authority over what that group projects to, and honouring a relative identifier there would place a well-known group inside that source's range — where a second source could number something else identically.

2.20.6 The authority tells the source its range #

A source is told its base and count at registration (§2.8). This is informational and exists so an administration tool can show an operator the identifier a principal will really project to, rather than the relative number on disk.

A source MUST NOT apply the base to what it asserts. It is told the range so it can explain itself, not so it can do the arithmetic. A source that applied its own base would have it applied twice.

Disclosure costs nothing that matters. What confines a source is not ignorance of the base but the authority's refusal to accept a relative identifier at or past the count — a check the authority performs on every number it rebases, whatever the source knows.

2.20.7 Uniqueness within a source #

SIDs are one namespace; POSIX user and group identifiers are two. A source MUST therefore allocate from a single counter across every kind of object it holds, so that a number issued to a principal is never issued again to a group.

An authority cannot check this — it sees one assertion at a time — so it is stated as an obligation on the source (§2.22) rather than as something enforced.

A PGSS Logon credential change (PGSS §2.20) reaches the source that holds the principal as a conversation of its own kind. It opens with ChangeCredential, runs the relayed interrogation of §2.12, and ends in CredentialChanged or Refusal.

  authority                              source
    |------------ ChangeCredential ---------->|   conversation N
    |<----------- CredentialRequest ----------|   the current credential
    |------------ CredentialResponse -------->|
    |<----------- CredentialRequest ----------|   the new one
    |------------ CredentialResponse -------->|
    |<------ CredentialChanged | Refusal -----|

It is not an Authenticate that ends differently. A change asserts nobody, so it carries nothing an authority could mint from, and keeping the two apart by message type keeps the path that mints unreachable from this one (§2.4).

2.21.1 ChangeCredential #

msg_type = 0x0007. Authority to source. Opens a conversation.

2.21.1.1 start #

The client's CredentialChangeStart, nested whole (§2.7), for the reason Authenticate nests LogonStart (§2.11): it grows on PGSS Logon's schedule, and a field appended there must not displace principal. Its supported_credential_types binds what may be prompted for, exactly as in a logon.

2.21.1.2 principal #

The principal whose credential changes: the user of the client's token, established by the authority from the client's connected socket and never from a message body.

It names both who asked and who is being changed, because PGSS Logon gives a client no way to name anybody else. A source MUST treat it as established fact, MUST NOT change the credential of any other principal in this conversation, and MUST NOT take a principal from anything the client supplies in it.

2.21.1.3 Routing #

An authority MUST send ChangeCredential only to the source authoritative for principal's domain (§2.18), and MUST NOT offer the change to any other source. The rule and its reason are those of §2.11: a credential offered to the wrong source is a credential handed to it.

Where no registered source is authoritative for principal and a configured source is absent, the authority MUST answer the client AuthorityUnavailable rather than refusing the account. The absent source may be the one that holds it, and "this account has nothing to change" would turn an outage into a statement about the account.

An authority MUST NOT send ChangeCredential to a source that did not declare CHANGES_CREDENTIALS (§2.8).

2.21.2 The interrogation #

The rounds are those of §2.12, unchanged: CredentialRequest and CredentialResponse carrying PGSS Logon's bodies, relayed and policed by the authority as they are in a logon. The source decides what to ask for.

A source MUST establish that the principal can present their current credential before changing it (PGSS §2.20), and MUST refuse with AuthenticationFailed where they cannot. The token the authority read principal from is not that proof.

A source that proves the current credential in one round and collects the new one in a later round MUST NOT change the credential if the current credential has itself changed in between. Otherwise an administrator's reset between the two rounds would be overwritten on the strength of the credential it replaced.

2.21.3 CredentialChanged #

msg_type = 0x8008. Source to authority. The only successful outcome of a change conversation. No fields.

A source MUST NOT send it until the new credential is the one it will verify at its next Authenticate for this principal — until the change is durable. The authority relays it to the client as PGSS Logon's CredentialChanged, and PGSS §2.20 forbids reporting a change that a subsequent logon would not see.

2.21.4 Refusal #

A change that does not end in CredentialChanged ends in Refusal (§2.13). A source MUST refuse:

• with AccountRestricted, a principal it holds no changeable credential for: one that authenticates with no credential, one designated only for service logons, or one that is disabled;
• with AuthenticationFailed, a current credential that does not verify;
• with ConversationLimit, a change it gives up on for want of an acceptable new credential.

2.21.5 Terminals out of place #

An authority MUST treat an Assertion in a change conversation, and a CredentialChanged in any other conversation, as a source fault. It MUST end the client's conversation with Internal, MUST NOT mint from the assertion, and MUST NOT report the change.

Abandon (§2.14) applies unchanged. A source discards everything it held for the conversation, including what a completed round established.

A conforming implementation MUST satisfy every requirement in this chapter. This section collects them by role.

2.22.1 Authority obligations #

An authority federating over PSI MUST satisfy all of the following.

2.22.1.1 Channel #

1. Listen; never dial out to a source (§2.3).
2. Never rely on the socket's descriptor as the access control, and bound unregistered connections and registration time independently of it (§2.6).
3. Bound registered sources, and conversations per source (§2.6).
4. Tear down the connection on a framing error or a failed write, rather than attempting resynchronisation (§2.6).

2.22.1.2 Registration #

5. Require Register on conversation 0 as the first message (§2.8).
6. Establish the source's identity from the peer's token, never from source_name, and refuse a mismatch rather than correcting it (§2.8, §2.9).
7. Accept only configured sources, treating an empty configuration as no source may register (§2.9).
8. Refuse a source that declares no domain (§2.10).
9. Refuse a domain that is not a well-formed locally-issued domain SID (§2.10).
10. Refuse a domain another registered source claims (§2.10).
11. Refuse a source declaring a different domain from the one it declared before (§2.10).
12. Refuse a source whose declared domain contradicts a configured pin, and treat an unparseable pin as refusing rather than as absent (§2.10).
13. Send Registered only once the source is routable (§2.8).
14. Report, where an administrator will see it, a source registering with no identifier range, and a source registering without QUERIES (§2.8).

2.22.1.3 Conversations #

15. Allocate conversation identifiers, never accepting one from a source, never opening one with an identifier already live, and reserve 0 for registration (§2.7).
16. Route each logon to exactly one source, resolved before any credential is collected (§2.11).
17. Never fall back to another source when the owning source is unreachable (§2.11).
18. Relay the verified originator, taken from the client's socket (§2.11).
19. Refuse to relay a prompt for a credential type the client did not advertise (§2.12).
20. Enforce its own round and time limits on the relayed exchange (§2.12).
21. Send Abandon for any conversation that will not reach a terminal state (§2.14).

2.22.1.4 Assertions #

22. Validate every SID structurally before treating it as identity, including SIDs carried inside claim values (§2.13).
23. Drop an asserted logon SID, and drop duplicates first-mention-wins, before applying any scope test (§2.13).
24. Enforce identity confinement, with no configuration lifting it, on an Assertion, on a QueryResult, and on the sid of a Changed (§2.18).
25. Enforce membership scope, subject only to a per-source permission covering memberships alone, and apply it to a source-asserted primary_group as well as to listed groups — promoting an unlisted one into the membership set before the test (§2.19, §2.13).
26. Never apply membership scope to a primary_group it substituted itself for an empty field (§2.13, §2.19).
27. Fail the logon on a malformed group SID, an unusable claim, a canonical_name PGSS §2.15 forbids, or an out-of-range identifier (§2.13).
28. Place primary_group on the token even when the source did not list it among groups (§2.13).
29. Perform derivation itself, and never accept privileges, integrity, or a token from a source (§2.4).
30. Never relay a refusal reason revealing a distinction the source was required not to make (§2.13).

2.22.1.5 Identifiers #

31. Apply the source's base to every relative identifier it accepts, and never accept one already rebased (§2.20).
32. Refuse a relative identifier of 0 or one at or past the source's count, rather than clamping or wrapping it (§2.20).
33. Reserve a band of identifiers below every source's base for its own, and use its own value for any SID within that band regardless of what the source sent (§2.20).

2.22.1.6 Queries #

34. Never send a message a source did not declare it answers, and never set a field bit gating a capability it did not declare (§2.8).
35. Never send more keys in one Query than the source's max_batch, nor more than 64 whatever it declared (§2.8).
36. Resolve an absolute POSIX identifier to a source and a relative identifier itself, and never send an absolute one to a source (§2.15).
37. Apply identity confinement, membership scope and numeric scope to a QueryResult exactly as to an Assertion, and validate every SID in one structurally — including those of references inside a value (§2.15).
38. Rebase every identifier in a result, under the rules of §2.20 (§2.15).
39. Qualify a source's canonical_name itself; never require a source to (§2.15).
40. Consult no further source on a Refused result, treat only NotFound as leave to continue, and never relay a source's Refused outward as PGSS Logon's Refused outcome (§2.15).
41. Never record a source it may not ask — one that declared no QUERIES — as having answered NotFound (§2.15).
42. Relay cursors opaquely, never construct or modify a source's, and never restart an enumeration on a source's behalf (§2.16).
43. Record a source that declined or could not be reached, and not retry it for the remainder of that enumeration, across pages as well as within one (§2.16).
44. Never infer from a source declining to enumerate that it holds nothing (§2.16).

2.22.1.7 Caching #

45. Not cache a source's answers at all unless it declared PUSHES_CHANGES or a non-zero entry_ttl (§2.8, §2.17).
46. Not hold an answer beyond a declared entry_ttl (§2.8).
47. Accept Changed at any time after Registered, and never reply to it (§2.17).
48. Re-read through Query after an invalidation, and never take a new value from Changed (§2.17).
49. Treat the loss of a source's connection as All for that source (§2.17).

An authority that holds nothing satisfies 45 to 49 trivially.

2.22.1.8 Credential change #

An authority that relays PGSS Logon's credential change (PGSS §2.20) over PSI MUST also satisfy the following. Obligations 19 to 21 bind a change conversation as they bind an authentication.

50. Take principal from the client's connected socket, never from a message body (§2.21).
51. Send ChangeCredential only to the source authoritative for principal's domain, never offer the change to another source, and never send it to a source that did not declare CHANGES_CREDENTIALS (§2.21, §2.8).
52. Answer the client AuthorityUnavailable, rather than refusing the account, where no registered source is authoritative for principal and a configured source is absent (§2.21).
53. End the client's conversation with Internal on an Assertion in a change conversation or a CredentialChanged in any other, minting nothing and reporting no change (§2.21).

2.22.1.9 Credential enrolment #

An authority that relays PGSS Logon's enrolment (PGSS §2.23) over PSI MUST also satisfy the following. Obligations 19 to 21 and 50 to 53 bind an enrolment conversation as they bind a change, reading EnrollCredential for ChangeCredential.

54. Never send EnrollCredential to a source that did not declare ENROLLS_CREDENTIALS, and answer the client PermissionDenied where the source holding the principal did not (§2.23, §2.8).
55. Relay CredentialRejected only from an enrolment conversation, and end any other conversation carrying it with Internal (§2.23, §2.13).

2.22.2 Source obligations #

A principal source MUST satisfy all of the following.

2.22.2.1 Connection #

1. Connect to the authority; never listen for it (§2.3).
2. Open with Register on conversation 0, carrying its name and its domain (§2.8).
3. Send nothing else before receiving Registered (§2.8).
4. Report itself ready — to an init system or equivalent — only after Registered (§2.3).
5. Declare the same domain on every registration, for the life of the machine's configuration (§2.10).
6. Tear down the connection on a framing error or a failed write, on every path including an unsolicited Changed (§2.6, §2.17).

2.22.2.2 Conversations #

7. Reply on the conversation identifier it was given, and never invent one (§2.7).
8. Decline to act on a message on a conversation it does not know, and never treat it as opening one — without replying on it, since the identifier may since have been reused (§2.7).
9. Refuse an Authenticate, ChangeCredential, Query or EnumerateSource arriving on conversation 0 (§2.7).
10. Bound the conversations it tracks itself, rather than relying on the authority's limit (§2.6).
11. Refuse beyond that bound with AuthorityUnavailable, rather than dropping silently (§2.11).
12. Discard conversation state on Abandon, and not reply (§2.14).

2.22.2.3 Answering #

13. Send exactly one terminal message per conversation: Assertion or Refusal for an authentication, CredentialChanged or Refusal for a credential change (§2.13, §2.21).
14. Assert only principals within its declared domain (§2.18).
15. Assert group SIDs and identifiers only, never attributes (§2.13).
16. Carry the canonical spelling of the principal's name in canonical_name (§2.13), and never assert a name that PGSS §2.15 forbids — validating what it asserts, not only what it creates.
17. Never distinguish an unknown principal from a bad credential — by denial code, by reason, or by timing (§2.13).

2.22.2.4 Identifiers #

18. Assert relative identifiers only, and never apply its own base (§2.20).
19. Allocate from a single counter across every kind of object it holds, so that no number is issued twice (§2.20).
20. Send 0 for any object it does not number, including every group it does not own (§2.20).
21. Never issue an identifier at or past the count it was given (§2.20).

2.22.2.5 Queries #

A source declaring no capabilities (§2.8) is exempt from this section entirely.

22. Declare only capabilities it implements, and answer every message type it declared (§2.8).
23. Answer every key of a Query, in order, without reordering, merging or omitting — or refuse the conversation whole (§2.15).
24. Send Found, NotFound or Refused in a result, and never Unavailable or Malformed (§2.15).
25. Return its own canonical spelling of a name, unqualified (§2.15).
26. Return relative identifiers in a result, exactly as in an Assertion (§2.15, §2.20).
27. Answer only for principals within its declared domain, on a query as on a logon (§2.18).
28. Answer Refused, never an empty Found, wherever it is declining rather than reporting an absence — including a membership it will not expose, a key naming an object it does not hold, and a key of the wrong kind (§2.16).
29. Refuse a cursor it can no longer honour, rather than restarting or answering from a changed store (§2.16).
30. Hold no per-cursor state it is unwilling to discard unasked (§2.16).
31. Size a page against the smaller of PSI's message ceiling and PGSS Logon's, since the authority must re-encode it into the latter (§2.16, §2.A).
32. Send Changed before the altered answer becomes observable, if it declared PUSHES_CHANGES (§2.17).
33. Declare a non-zero entry_ttl if it does not push changes and can tolerate its answers being held, and SHOULD declare one even if it does (§2.8, §2.17).

2.22.2.6 Credentials #

34. Store verifiers that are not usable as credential material — nothing a challenge could be recomputed from (PGSS §2.11).
35. Erase credential material, and every buffer it was decoded through, before that memory is released (PGSS §2.12).
36. Never write credential material to a log, audit record, or diagnostic (PGSS §2.12).

2.22.2.7 Logon type restrictions #

37. State a principal's permitted_logon_types where it holds the property, and send zero where it does not (§2.13).
38. Never send "everything" for a principal whose restrictions it does not hold — zero is the honest answer, and "everything" asserts that the principal may be used for a credential-free service logon (§2.13, PGSS §2.19).
39. Never refuse an authentication on the strength of the restrictions it states. A source states; the authority decides (§2.4, §2.13).
40. Answer LOGON_TYPES from a Query as stored, including "not stated", and never substitute a default of its own — what silence means is the authority's reading, and two components with their own defaults can disagree (§2.15, PGSS §2.16).

2.22.2.8 Credential change #

A source that declares CHANGES_CREDENTIALS MUST also satisfy the following.

41. Change only the credential of principal, and never take a principal from anything the client supplied (§2.21).
42. Establish that the principal can present their current credential before changing it, and refuse with AuthenticationFailed where they cannot (§2.21).
43. Never change a credential on the strength of a current credential that has changed since it was proved (§2.21).
44. Refuse with AccountRestricted a principal it holds no changeable credential for (§2.21).
45. Send CredentialChanged only once the new credential is the one its next Authenticate for that principal will verify (§2.21).

2.22.2.9 Credential enrolment #

A source that declares ENROLLS_CREDENTIALS MUST also satisfy the following.

46. Add or remove credentials only of principal, and never take a principal from anything the client supplied (§2.23).
47. For both adding and removing, establish that the principal can present a credential they already have before changing anything, refuse with AuthenticationFailed where they cannot, and refuse with AccountRestricted, before asking, a principal it can ask for no such credential (§2.23).
48. Check the material again when applying the change, refuse material it will not accept with CredentialRejected, and send that code in no other conversation (§2.23).
49. Send CredentialChanged only once the change is durable, and, where it declares PUSHES_CHANGES, only after Changed (§2.23, §2.17).

2.22.3 What a source is not required to do #

A source is not required to store anything, to be local, to be persistent, or to know what a token is. It answers one question: given this identifier and whatever it chose to ask for, who is this?

Nor is it required to trust the authority beyond the connection. A source that refuses logons on the strength of originator, or declines to answer for principals it holds but does not wish to expose, is conforming.

A PGSS Logon enrolment (PGSS §2.23) reaches the source that holds the principal as a conversation of its own kind. It opens with EnrollCredential, runs the relayed interrogation of §2.12, and ends in CredentialChanged or Refusal.

  authority                              source
    |------------ EnrollCredential ---------->|   conversation N
    |<----------- CredentialRequest ----------|   the current credential
    |------------ CredentialResponse -------->|
    |<------ CredentialChanged | Refusal -----|

It is the sibling of the credential change (§2.21), and every rule that section states binds here unless this section says otherwise: routing to the source authoritative for the principal, AuthorityUnavailable while a configured source is absent, the principal as established fact, the terminals out of place, and Abandon.

2.23.1 EnrollCredential #

msg_type = 0x0008. Authority to source. Opens a conversation.

start is the client's CredentialEnrollStart, nested whole for the reason ChangeCredential nests its start (§2.21). principal is the user of the client's token, established by the authority from the client's socket, exactly as in ChangeCredential. A source MUST NOT add a credential to, or remove one from, any other principal in this conversation.

2.23.2 The capability #

An authority MUST NOT send EnrollCredential to a source that did not declare ENROLLS_CREDENTIALS (§2.8). Where the source that holds the principal did not declare it, the authority MUST answer the client PermissionDenied: for that principal it does not offer enrolment, and PGSS §2.23 gives that answer to an authority that does not.

ENROLLS_CREDENTIALS and CHANGES_CREDENTIALS are independent. A source may change passwords and not enrol keys, or the reverse.

2.23.3 The interrogation #

The source MUST establish that the principal can present a credential they already have, other than the one being enrolled, before it adds or removes anything, for Add and Remove alike, and MUST refuse with AuthenticationFailed where they cannot. The token the authority read principal from is not that proof. A source that cannot ask for such a credential for this principal — the principal has none, or signs in only with the kind being enrolled — MUST refuse with AccountRestricted before asking anything.

The source decides which credential types it enrols and what material it accepts. It SHOULD check the material before asking for proof, MUST check it again when it applies the change, and MUST refuse material it will not accept with CredentialRejected. A source MUST NOT send CredentialRejected in any conversation but this one, and an authority receiving it in any other MUST end the client's conversation with Internal (§2.13).

2.23.4 CredentialChanged #

The successful terminal is CredentialChanged (§2.21), sent only once the change is durable: an added key is one the source's next Authenticate will accept where the principal's policy allows keys, and a removed key is one it will not.

A source that declares PUSHES_CHANGES MUST send Changed (§2.17) before CredentialChanged, as for any other change to what it holds.

Enrolment does not change which credentials the principal may sign in with; that remains the source's administration.

2.A.1 Messages #

The high bit marks a message sent by the source, which is the authority for its own principals (§2.7).

2.A.2 Protocol constants #

2.A.3 Field limits #

68 bytes is the largest a SID can be: an eight-byte prelude plus fifteen sub-authorities (§2.7).

A claim name is bounded at 255 bytes of UTF-8 while PCDS §5.9 bounds it at 255 UTF-16 code units. A string's UTF-16 length never exceeds its UTF-8 byte length, so the byte bound is the stricter of the two and satisfies PCDS without transcoding to find out.

The claim limits are otherwise tighter than PCDS §5.9 permits — it allows 1024 values per claim. These bound the work an authority does decoding a message it has not yet decided to believe, and nothing needs a thousand-valued claim from a principal source.

Fields inside a nested LogonStart, CredentialChangeStart, CredentialRequest, CredentialResponse or profile keep PGSS Logon's limits (PGSS §2.A).

2.A.4 The ceiling that actually binds a page #

None of the entry counts above is the constraint on how much a source may return. An entry that fits a PSI message need not fit the PGSS Logon message the authority must re-encode it into: this chapter's ceiling is 81920 bytes and PGSS Logon's is 65536, and a QueryResult or EnumerateResult entry travels outward inside the smaller one.

A source MUST therefore bound a reply by the smaller of the two ceilings, not by this one, and MUST page rather than fill a PSI message it knows an authority cannot forward. A page that fits here and not there is a page nobody can deliver, and the entry-count bounds do not prevent one — 256 entries of a few hundred bytes each exceeds both.

The margin an implementation leaves for the authority's own framing is its own choice; leaving none is a defect.

Values PSI defines for itself. Everything else it carries is PGSS Logon's — see PGSS §2.B.

Adding a value to any enumeration here is a breaking change requiring a version bump (§2.7). The two exceptions are the capability bitmask below, and PGSS Logon's field bitmask, which PSI carries unchanged and which may gain bits without one.

2.B.1 Key types #

Carried in a Query key entry's key_type (§2.15) as a u8.

Zero is not a key. It is how an EnumerateSource encodes an empty of (§2.16), which is the only place it may appear; an encoder MUST NOT send it in a Query key and a decoder MUST reject one that arrives there.

PGSS Logon's key types (PGSS §2.B) share the first two values and differ in the third, where it carries an absolute POSIX identifier. The values are not interchangeable and the tables are deliberately separate: 3 means a rebased number on one side of the authority and a relative one on the other, which is the whole of §2.20 expressed as a number.

2.B.2 Result outcomes #

Carried in a result entry's outcome (§2.15) and in EnumerateResult.outcome (§2.16) as a u8.

The values are PGSS Logon's (PGSS §2.B). A source may send only these three:

Unavailable (3) and Malformed (5) are the authority's to produce and MUST NOT be sent by a source — see §2.15. A decoder MUST reject either arriving from a source, rather than leaving the check to a caller.

2.B.3 Change scopes #

Carried in Changed.scope (§2.17) as a u8.

2.B.4 Capabilities #

Carried in Register.capabilities (§2.8) as a u32 bitmask.

Unlike the enumerations above, a bit MAY be added here without a version bump. A source that does not set a bit has not declared the capability, and an authority MUST NOT send a message the source did not declare it answers (§2.8) — so an authority that predates a bit simply never uses it, and a source that predates one never sets it. Both are the safe reading.

MEMBERS gates a field rather than a message: an authority MUST NOT set the MEMBERS field bit of a Query (§2.15) against a source that did not declare it.

2.B.5 Claim value types and flags #

A claim's value_type and flags (§2.13) are PCDS §5.9's, and are not restated here.

They are nonetheless closed on this interface: a decoder MUST reject a value type or a flag bit it does not recognise, rather than carrying it through to an authority that will put it on a token. Adding one is therefore a breaking change to PSI by the rule above, even though the values themselves belong to PCDS.

PSI is a superset of PGSS Logon (§2.5). This appendix consolidates exactly what is shared, what is added, and what differs — as a checklist for an implementer building both, and as the list to re-examine whenever either specification changes.

2.C.1 Shared unchanged #

An implementation that reimplements any of these rather than sharing one definition has taken on the job of keeping two copies in step. The sharing is the point: a translation layer between two byte-identical formats is a place for them to drift.

2.C.2 Added by PSI #

2.C.3 Differs #

The success terminal is what makes minting structurally impossible for a source (§2.4); the socket path is normative in PGSS because it is a conformance bar and not here because PSI is not one (§2.1).

The message ceiling is the row most likely to catch an implementer out, because the larger number is the one that does not bind a reply — see §2.A.

2.C.4 When either specification changes #

A field appended to LogonStart, CredentialRequest, CredentialResponse or profile in PGSS appears here automatically, because the bodies are shared. That is the intended behaviour and needs no change to this chapter.

The profile is the one shared body that travels in the opposite direction to the others: the interrogation bodies pass from the authority outward to the client, while the profile originates at the source and is relayed outward through the authority. It is shared for the same reason regardless — one definition, so the value a source states and the value a client reads cannot drift apart.

A new Denial value, a new CredentialType, or any change to the header's first twelve bytes is a breaking change to both and requires a coordinated version bump. An implementer maintaining both MUST NOT bump one alone.

The one exception is a denial added under PGSS §2.6's rule for codes that only newer messages draw. NoSuchSession is one: it answers only requests a source is never party to, a source never sends it (§2.13), and so no PSI peer of either age meets it.

2.D.1 What this exists to avoid #

The design is shaped more by rejected approaches than adopted ones.

LSA authentication packages. Windows loads authentication packages as DLLs into the LSA process. A defect in any package is a defect in the most privileged process on the system, and the packages are exactly the components most likely to parse hostile input. PSI puts that boundary at a process, permanently: there is no in-process extension point and no message that could create one.

PAM modules. The same objection, plus stacking — offering a credential to each module in turn until one accepts, which hands every module the credentials of every other module's users. PSI resolves which source answers before any credential is collected (§2.11).

NSS. Name service switch modules answer "who is this?" as a library call in whatever process asked, with no boundary at all. PSI's answer is a message from a process that was separately identified.

2.D.2 What is adopted #

RSI's shape, specified in PSPK. A long-lived connection, sources that dial in and register, multiplexed requests tagged with an identifier, and the authority tearing down a connection it cannot parse. PSI is recognisably the same family, and deliberately so — an implementer who has written a registry source will find little surprising here.

The differences are worth naming, because they follow from what is being federated. A registry source is trusted with the correctness of a subtree and the kernel validates its structure; a principal source is trusted with identity, so the checks it faces are about scope — which principals, which memberships, which numbers — rather than about well-formedness alone. And the kernel assigns a registry source no numeric range, because there is nothing to project.

PGSS Logon's interrogation, wholesale. Rather than inventing a parallel vocabulary for prompts and answers, PSI relays PGSS Logon's messages with identical bodies (§2.5). The gain is not brevity but correctness: there is no translation layer to be lossy, and a source's prompt reaches the client exactly as written.

2.D.3 Design influences #

Sources connect inward. The authority never dials out. See §2.3 — this is the single most consequential shape decision in the chapter.

Assertion, not minting. The success terminal deliberately cannot express a session or a token (§2.4). The separation is structural rather than a rule an implementer must remember.

Domains claimed, not assumed. A source states what it is authoritative for and the authority confines it to that (§2.10, §2.18). The alternative — an authority that trusts whatever a source says about anyone — makes every source as dangerous as the most dangerous one.

Relative numbers. POSIX identifiers are the one thing a source states that has no namespace of its own to be confined by, so the range supplies one (§2.20). Nothing else in the protocol needed inventing; a SID already carries its domain.

This chapter specifies the observability interfaces: the three channels by which the programs on a Peios system deposit logs and metrics with an observability service, and by which anything on the system asks that service what it holds.

There are three interfaces and they are specified together because they are one contract from the service's side and because two of the three share their encoding, their validation posture and their loss model:

• the Log Ingestion Interface, on which a producer submits log records (§3.6 to §3.8)
• the Metric Ingestion Interface, on which a producer submits metric samples (§3.9 to §3.13)
• the Query Interface, on which a client asks for stored events, logs and metrics and receives records (§3.14 to §3.28)

Three roles participate.

The collector is the process that accepts ingestion on the two datagram channels, serves the query channel, and holds the data in between. There is one collector. It is the party being asked, on all three interfaces — which is why the obligations in this chapter fall mostly on it, and why the producer and client roles are so thin.

A producer is a process that submits metric samples. Its KACS token is conveyed with every datagram, and the collector verifies that it may publish each metric name (§3.9). Service output reaches the log channel through the service manager instead: the service manager authenticates the service origin by the pipe it owns, then acts as the sole log broker (§3.6).

A client is a process that issues a query and reads the result. A client's identity, unlike a producer's, is established by the collector and determines what it may see (§3.28).

One program is commonly all three at once.

This chapter covers:

• the shape of the three channels and why two are datagram and one is a stream (§3.3)
• the loss model, which is the load-bearing decision of the whole ingestion design (§3.4)
• encoding, timestamps and the timestamp domain (§3.5)
• the log record, its fields, and exactly which malformations cost the record and which are merely ignored (§3.6 to §3.8)
• the metric data model, the three metric types, the sample record, and what makes two samples the same time series (§3.9 to §3.13)
• the query channel, its framing, and the four response messages (§3.14 to §3.17)
• the query language: its shape, its lexis, its operators, its ordering and grouping semantics, and the three modes (§3.18 to §3.25)
• cross-type filtering and streaming (§3.26, §3.27)
• what a client is and is not told about data it may not read (§3.28)
• how these interfaces may be extended (§3.29)
• the obligations binding on each role (§3.30)

This chapter does not cover:

• Event emission. Events reach a collector through KMES, not through any interface here; the consumer side of that is specified in PSPK, and emission is a kernel interface offered to privileged callers.
• Event type vocabulary and payload schemas, which belong to whichever subsystem emits the event.
• How a collector stores, indexes, retains or accelerates anything — its own design. The mainline collector's is described in the eventd TRMP.
• Which principals receive publication rights for individual metric names, and how those rights are administered.
• Administering a collector's contents.

The third of those is the point of the whole document. A collector is handed records and asked questions; how it gets from one to the other is exactly what different collectors exist to do differently.

3.1.1 These interfaces are not a conformance requirement #

A system that offers none of these is still Peios. Observability is not in the definition of the platform, and a system that ships a different collector, or none, conforms exactly as well.

They are specified because they are public. Every service has its output brokered into logs, every collection agent is a metric producer, and every dashboard, alerting tool and command-line viewer is a query client. All three of those are third-party positions, and all three need a contract that stays put.

Terms defined in PSPK for the KMES event stream — event, header, payload, stamp, sequence number, origin class — are used here with the same meaning and are not redefined. Terms defined in PCDS — GUID, SID, Security Descriptor, ACL, ACE — likewise.

The following terms are specific to this chapter.

Collector: the process that accepts log and metric ingestion and serves queries. The role, not the program: the mainline collector is eventd, and this chapter never requires that it be.

Producer: a process that submits log records or metric samples.

Client: a process that issues a query.

Log record: one line of output from a program, with the light metadata of §3.7 attached. A log record is text; the collector does not parse it.

Metric sample: one measurement of one quantity at one moment, belonging to a time series (§3.13).

Time series: the sequence of samples sharing a name, a label set, and — for histograms — a set of bucket boundaries. Identity is defined in §3.13.

Boot ID: a GUID identifying one boot of the system. Every record a collector stores carries one, so that records from different boots are never interleaved and per-CPU event sequence numbers, which restart each boot, remain unambiguous. It is assigned outside this interface and is visible to a client only as a queryable field.

Datagram: one message on an ingestion channel, carrying either one record or a batch of them (§3.7, §3.11).

Effective query range: the half-open interval a query examines, [SINCE, UNTIL), with the bounds resolved as §3.19 defines.

Concrete identifier: the event type, log origin or metric name that a stored record actually carries — as distinct from the pattern a query or a Security Descriptor uses to match one. Access control resolves per concrete identifier (§3.28).

A collector listens on three AF_UNIX sockets. Two carry data inward, one carries it out.

The pathnames are configuration and this chapter does not fix them. A collector MUST serve each interface on a distinct socket; it MUST NOT multiplex two of them onto one.

3.3.1 Why ingestion is datagram #

Each submission is an independent message. A datagram either arrives whole or does not arrive, so there is no framing to get wrong, no length prefix to parse, no partial read to reassemble, and no connection state to keep for a producer that submits one line an hour. The record boundary is the datagram boundary.

The property that matters more is that a datagram socket cannot exert backpressure. When the receive queue is full the kernel discards the datagram and the sender proceeds. That is the behaviour §3.4 requires, and choosing a stream socket would make it unachievable: a full stream buffer blocks the writer, which is precisely the outcome this design forbids.

3.3.2 Why the query channel is a stream #

A query result is arbitrarily large, is delivered in several messages, and must not be silently truncated — the opposite requirements. It also needs a caller identity, and a peer token can be obtained from a connected stream socket. Both push the same way.

3.3.3 Separation #

The three channels are separated for the same two reasons.

The first is admission. Each listening socket has its own receive queue. Log volume is orders of magnitude above query volume on any normal system, and a burst of either must not delay the other. Three sockets means three populations of caller that cannot starve one another, whatever load any of them is under.

The second is access control. The service manager alone writes the log channel, while service processes write the metric channel under their own conveyed identities; the set that may read either is different again. Those want different Security Descriptors, and a descriptor is a property of a socket.

The separation is not for isolation. One collector serves all three, so a defect or a hang in any of them reaches the others regardless, and this chapter does not pretend otherwise.

3.3.4 Protecting the channels #

A collector MUST protect each socket with a Security Descriptor.

The log socket descriptor MUST deny FILE_WRITE_DATA to the Service logon group before allowing SYSTEM. The service manager runs as SYSTEM without the Service group; every service it launches, including a SYSTEM service, carries that group. The ordered deny therefore makes the service manager the log broker without making every SYSTEM service one (§3.6).

The metric socket descriptor is coarse admission only. Every metric datagram also carries a KACS sender token and every metric name is authorized for EVENTD_PUBLISH (§3.9). A collector SHOULD let every authenticated caller send by default, so that a publish grant to anyone can be used; an operator MAY narrow it.

The query socket descriptor MUST let every authenticated caller connect. What a caller may read is decided per identifier against the read policy (§3.28), so a socket that admitted fewer callers would only make a policy grant to anyone else impossible to use.

A collector MUST NOT rely on the socket's POSIX mode bits for this. On Peios an access decision is routed through the object's Security Descriptor, not through mode bits, so a mode set on a socket pathname does not restrict anything; and an inode created without a descriptor is denied to every caller, so a collector that binds a socket into a directory carrying no inheritable ACEs produces a socket nothing can reach. A collector MUST establish the descriptor on each socket before it begins accepting on it.

The single decision that shapes both ingestion interfaces is this: a producer is never slowed down, and never told that a record was lost.

3.4.1 The obligation #

A collector MUST NOT exert backpressure on a producer. A producer MUST NOT stall, block, retry or otherwise change its behaviour because a collector is slow, busy, or absent.

The consequence is accepted openly. When a collector cannot drain an ingestion socket as fast as producers fill it, the kernel discards datagrams. Neither party is notified. A collector MUST NOT report the loss to the producer, because there is no reply message on a datagram channel and adding one would reintroduce the coupling this rule exists to prevent.

3.4.2 Why loss is acceptable here #

A lost log line is an inconvenience. A lost metric sample is a visible gap in a chart. Neither is a failure of the system, and neither is worth the cost of the alternative — which is either blocking the producer or buffering without bound, and the second is only the first with a delay.

Events are the counter-example, and the reason the boundary between events and logs matters. An event may be a security audit record whose absence is itself the finding, so events do not travel on either interface in this chapter: they travel through KMES, where loss is detected, bounded and recorded. A program with data that must not be lost emits an event; a program with output for a human to read writes a log.

3.4.3 What a collector must not do about it #

A collector MUST NOT emit an event, write a log entry, or perform any other work proportional to the volume of malformed or unwanted input it receives.

Ingestion input is untrusted even though its admission is authenticated (§3.3). A collector that reacted to bad input — by logging it, by counting it in a way a client can observe, or by emitting a diagnostic event — would hand an admitted process an amplification primitive: a cheap malformed datagram producing an expensive durable record. Silence is the defence. In-memory diagnostic counters and rate-limited standard error output are permitted because they do not grow with durable input.

The rule binds only on responses to input. A collector MAY record its own internal conditions, and the mainline collector records several, as synthetic events and as health metrics (eventd TRMP §2.6, §5.7).

3.4.4 Ordering and duplication #

A collector MUST NOT assume that datagrams arrive in the order they were sent, and MUST NOT reorder or deduplicate the records inside one. A producer MUST NOT assume that submitting two datagrams in order causes them to be stored in that order; the timestamp field (§3.7, §3.11) is the only ordering a producer controls.

Records are not deduplicated. A producer that submits the same record twice has produced two records.

3.5.1 MessagePack #

Every structured value on all three interfaces — log records, metric samples, query requests, query responses — is encoded as MessagePack. Strings are UTF-8.

The choice is inherited rather than made here: KMES event payloads are a single MessagePack value, so a collector already carries a decoder and a query result can carry an event payload outward without re-encoding it.

A decoder MUST accept any valid MessagePack encoding of a value it is given. In particular a producer MAY use any length-prefix width that can represent the value, and a collector MUST NOT require the shortest.

3.5.2 Canonical MessagePack #

Where this chapter requires a canonical encoding, the value MUST be encoded as follows:

• Nil and booleans use the fixed singleton encodings.
• Integers use the shortest encoding that preserves signedness: non-negative values use positive fixint, uint8, uint16, uint32 or uint64; negative values use negative fixint, int8, int16, int32 or int64.
• Floats are encoded as float64. Finite values use their IEEE-754 binary64 representation; the infinities use the normal binary64 encodings; any NaN is encoded as the single quiet NaN bit pattern 0x7ff8000000000000.
• Strings, binary values, arrays and maps use the shortest length-prefix form capable of representing the length.
• Arrays encode each element recursively, in order.
• Maps encode keys and values recursively, with entries sorted by the canonical encoded key bytes; ties are broken by the canonical encoded value bytes.

Canonical encoding exists so that two values that are equal are also byte-identical, which is what makes them comparable and orderable without decoding. It is required in exactly two places: histogram sample storage, where it makes a sample map a stable value (§3.11), and array comparison in query ordering and grouping (§3.21).

It does not constrain what a producer sends. Ingestion accepts any valid encoding.

3.5.3 Timestamps #

A timestamp is wall-clock time in nanoseconds since the Unix epoch, UTC, as a signed 64-bit value.

The timestamp domain is 0 to 9223372036854775807 inclusive. A value outside it is invalid wherever it appears: as a producer-supplied timestamp (§3.8, §3.12), as a query time literal (§3.19), or as a value in a result record.

The domain has no negative half. A collector MUST reject a negative timestamp rather than storing a time before 1970, and a query whose time arithmetic lands below zero — SINCE 100000d ago, for example — MUST produce an error rather than clamping.

Wall-clock time is not monotonic. A collector MUST store the timestamp it is given or derives without correcting it, and MUST NOT assume that timestamps within one time series increase (§3.13). A clock step backwards produces records that are out of order with respect to their arrival, and every ordering rule in this chapter is defined to remain total and deterministic when that happens.

A collector MUST expose an AF_UNIX SOCK_DGRAM socket for log ingestion, protected by a Security Descriptor as §3.3 requires.

3.6.1 The datagram ceiling #

A collector declares a maximum accepted datagram size, the log datagram ceiling. The portable ceiling is 262144 bytes. Every collector MUST accept a whole datagram up to that size, and every portable producer MUST bound its encoded datagrams to that size without discovering collector configuration. A collector MAY accept more, but a producer MUST NOT depend on that when implementing this interface.

A collector MUST receive log datagrams into a buffer of at least its declared ceiling, and MUST discard a datagram the kernel reports as truncated rather than storing the prefix that fitted.

A producer MUST NOT send a log datagram larger than the ceiling. One that does is discarded whole, taking every record in it, and the producer is not told (§3.4).

A sending socket's own buffer can be smaller than the ceiling. A producer that may send a datagram longer than its socket's default send capacity SHOULD first raise SO_SNDBUF enough to accommodate it.

On Linux, a Unix datagram longer than the effective SO_SNDBUF returned by getsockopt minus 32 bytes is refused with EMSGSIZE. Linux doubles the value requested with setsockopt(SO_SNDBUF) for bookkeeping; the effective value must therefore be at least the datagram's length plus 32 bytes. An ordinary unprivileged request is capped at net.core.wmem_max before that doubling. The initial buffer comes from net.core.wmem_default; neither sysctl has a universal value.

For example, with both sysctls set to 212992 bytes, the initial buffer allows datagrams of at most 212960 bytes, below the portable ceiling. Requesting 212992 bytes gives an effective buffer of 425984 bytes and a datagram limit of 425952 bytes. A larger datagram needs a higher wmem_max for an ordinary unprivileged sender. A collector's configured ceiling does not itself enlarge the sender's buffer.

No capability exchange is needed. The 262144-byte floor is part of the wire contract rather than a tunable assumption shared out of band. Raising a collector's ceiling is a local extension; lowering it below the portable ceiling is non-conforming.

3.6.2 The receive queue is the buffer #

A collector MAY enlarge the socket receive queue, to at most four times the datagram ceiling. It MUST NOT buffer beyond it.

That queue is the only cushion between a producer and the collector's storage. While a collector is committing a batch it is not draining the socket, and datagrams arriving in that window occupy the queue; when the queue fills, they are discarded. This is the designed degradation (§3.4), not a failure to be tuned away — a larger buffer moves the threshold without changing what happens at it, and an unbounded one converts data loss into memory exhaustion.

3.6.3 The service-manager broker #

Every service produces output, including programs that know nothing about the collector, but service processes do not write this socket. The service manager holds each service's standard output and standard error at fork, determines the origin from the pipe it read, and forwards the resulting records (peinit TRM).

The log socket MUST implement the deny-Service, allow-SYSTEM descriptor in §3.3. A collector MUST accept log origins from this broker without a per-record KACS token or AccessCheck. A service can choose the bytes it writes to its pipe but cannot choose another service's origin.

Direct log submission is not part of this interface. A program that wants structured, authenticated observability data emits an event or a metric; it does not bypass the service manager to assert a log origin.

A log datagram carries either one record, encoded as a MessagePack map, or several, encoded as a MessagePack array of maps. A collector MUST accept both forms. A producer MAY use either at any time; there is no mode and no negotiation.

3.7.1 Fields #

A collector MUST ignore fields it does not recognise (§3.29).

3.7.2 origin #

origin is what the producer says it is. A collector MUST NOT verify it, because it has no way to: the channel is a datagram socket and carries no peer identity (§3.4). Two producers MAY use the same origin, and one producer MAY use several.

An origin is nonetheless the unit that read access is granted on (§3.28), and a collector matches it against patterns using dot-delimited prefix semantics: the pattern svc matches the origin svc and any origin beginning svc., and matches neither svc_daemon nor svcfoo. It also matches every origin whose part before a slash is svc, such as svc/HealthCheck, because matching considers only that part.

A producer therefore SHOULD choose an origin that names it stably and distinguishably, and SHOULD use dots for hierarchy, because an administrator writing an access rule has nothing else to write it against.

An origin MUST match:

origin    := component | component "/" producer
producer  := component | component "[" [0-9]+ "]"
component := [A-Za-z0-9_][A-Za-z0-9_.-]*

A collector MUST discard a record whose origin does not.

A single component is the identifier grammar of §3.19, widened only to admit a leading digit, and names a program. The optional second component names a producer within that program — a hook, a reload command, a health check, one submitted job — and the bracketed integer distinguishes several of the same kind. A forwarding service manager uses it for exactly that: jellyfin/ExecStartPre[0] is a pre-start hook's output, jellyfin/HealthCheck a health check's, jobs/<guid> a submitted job's. There is at most one slash, and the index may appear only after one.

Everything the grammar still excludes, it excludes on purpose. An origin is not merely a label: it is matched against patterns in which * is the wildcard, so an origin containing * could not be selected exactly and could match a rule its producer was never meant to satisfy; and it is the name an access rule is stored under, so an origin carrying a backslash, a quoting character or whitespace could land somewhere other than where the administrator who wrote the rule believes it is. Constraining the producer is the only point at which either can be prevented.

The slash is safe there only because it is never written into a stored rule. A collector MUST resolve the access rule for an origin from the part before the slash, so that a service's hooks, reloads, health checks and jobs answer to the rule written against the service — and a producer cannot reach a rule of its own by inventing a second component.

A collector MUST count discarded origins and MUST make that count observable to an operator. An unrecognised origin means a producer's vocabulary and a collector's have drifted apart, and every line that producer sends is being dropped; a silent discard makes the most complete kind of log loss the least visible one.

Quoted forms remain valid syntax everywhere an origin may be written (§3.24). A single-component origin beginning with a letter or _ never needs them; every other conforming origin does, because the query language's identifiers admit neither a slash, a bracket nor a leading digit (§3.19). A pattern may need them too, and a collector holding origins stored before this rule applied must still be able to return and select them.

3.7.3 is_error #

is_error is a boolean and deliberately not a severity level.

A forwarding producer can distinguish standard output from standard error and nothing more; inventing five levels out of two file descriptors would be a guess presented as data. A producer with real severity levels either writes them into the message text, where they are text and are searched as text, or emits events, which have types.

A system service SHOULD write informational messages to standard output and warnings and errors to standard error. This keeps informational startup and status messages out of is_error == true queries. Warnings stay in them: they are diagnostics on standard error, and the collector does not infer severity from text. Peios's own system daemons follow this rule. A daemon's best-effort kernel-log mirror, where it has one, is independent of the stream a line is written to.

3.7.4 timestamp #

A producer SHOULD supply the timestamp it captured when the line was produced, not when it submitted it. A producer that batches (§3.8) and omits the field attributes every line in the batch to the moment the collector happened to read it, which discards the timing information the batch was accumulated over.

3.7.5 job_id #

job_id correlates a line to a single execution rather than to a program. A forwarding producer sets it so that the output of one run of a service can be separated from the run before and the run after; a producer with no such notion omits it.

A collector MUST treat it as an opaque 16-byte value. Nothing in this chapter interprets it, and a producer MAY use it for any correlation of its own, provided the value is a GUID.

Every failure on this channel is silent (§3.4). What differs between failures is how much is lost: the whole datagram, one record, or only one field.

3.8.1 The three scopes of failure #

The datagram is discarded when it cannot be resolved into records at all:

• it is not valid MessagePack
• it decodes to something that is neither a map nor an array of maps
• the kernel reported it truncated (§3.6)

One record is discarded, and the others in the same datagram are still processed, when the record itself is unusable:

• a required field is absent
• a required field has the wrong type — origin an integer, say
• origin is the empty string
• the map contains a duplicate top-level key

One field is ignored, and the record is still stored, when an optional field is unusable:

• timestamp is not an integer, is negative, or is outside the timestamp domain (§3.5)
• job_id is not binary, or is binary of a length other than 16

A collector MUST implement all three scopes as stated. In particular it MUST NOT discard a record because an optional field was malformed: a producer with a broken clock or a mangled correlation key still has a log line worth keeping, and the field it got wrong is the field of least value in the record.

3.8.2 Duplicate keys #

A record map carrying the same top-level key twice MUST be discarded.

A collector MUST NOT resolve the duplicate by taking the first or the last. MessagePack decoders differ on which they keep, and the fields here are entirely producer-controlled, so a rule that depended on decoder behaviour would let a producer choose which of two origin values a given collector saw. Discarding is the only answer that is the same everywhere.

3.8.3 Batches #

A batch is validated per record. A malformed record in a batch MUST NOT cost the valid records beside it.

A producer SHOULD batch under sustained load. Batching amortises the syscall over many records, and the ceiling (§3.6) is per datagram, so a batch is also the only way to use the channel's capacity efficiently.

The encoded datagram, batched or not, MUST NOT exceed the ceiling. A producer that batches without bounding the encoded size will eventually build a datagram that is discarded whole — which is the one case where batching loses more than sending singly would have.

3.8.4 What a collector adds #

A collector supplies the boot ID (§3.2) and, when the record omitted timestamp, its own clock reading at receipt. It MUST NOT alter any other field, and MUST store message byte-for-byte as given.

A collector MUST NOT parse message. If the text happens to be JSON, or logfmt, or anything else structured, that is the producer's business: this interface carries lines, and a producer with structured data to record emits an event instead (§3.4).

A collector MUST expose an AF_UNIX SOCK_DGRAM socket for metric ingestion, protected by a Security Descriptor as §3.3 requires, separate from the log socket.

Before its first send, a producer MUST enable KACS_SO_PASS_TOKEN on its persistent sending socket. KACS then attaches the producer's effective token to every datagram as KACS_SCM_TOKEN. A collector MUST receive with ancillary space for that token and MUST discard the whole datagram when the token is absent or the data or control message was truncated.

For every valid record, the collector resolves the metric name's publication policy and runs KACS AccessCheck for EVENTD_PUBLISH against the conveyed token. A denied record is discarded silently. Records with other authorized names in the same datagram remain valid. An implementation MAY cache a verdict by stable token identity, token modification identity, metric name and policy generation.

The channel works exactly as the log channel does, for the reasons given there: the 262144-byte portable datagram ceiling, truncated datagrams discarded whole, a receive queue of at most four times the collector's declared ceiling, no backpressure and no notification (§3.6). A collector MUST accept at least the portable ceiling and a portable producer MUST bound its encoded datagrams to it. A producer that may send a datagram longer than its socket's default send capacity SHOULD first raise SO_SNDBUF, as on the log channel (§3.6).

3.9.1 A sink, not a collector of its own #

The collector is pushed to. It MUST NOT scrape an endpoint, read a kernel interface, or poll anything to obtain metrics; every sample it holds arrived on this socket because a producer sent it.

What gathers the measurements is a separate concern and a separate program. A collection agent that reads system counters and submits them is an ordinary producer here, with no privileged position and no interface of its own.

3.9.2 Batching #

Batching matters more here than it does for logs. It amortizes both the datagram and conveyed-token costs. A collection sweep produces many samples at once — every CPU core, every disk, every interface — and they share a moment, so a producer SHOULD submit a sweep as one batched datagram rather than as one datagram per sample.

The rules are the log rules: one map or an array of maps, per-record validation, and the encoded datagram bounded by the ceiling (§3.8, §3.12).

A metric is a quantity that varies and is worth watching over time: a utilisation, a queue depth, a running total, a distribution of latencies. Metrics are dense where events and logs are sparse — many measurements of the same thing rather than a record of a thing that happened — and the model reflects that.

A sample is one measurement, of one time series, at one moment. It carries:

• a name, identifying what is measured
• a label set, identifying which instance of it
• a type, fixing how the value is to be read
• a timestamp
• a value, whose shape depends on the type

Name and labels together identify the series (§3.13). The type is a property of the series, not of the sample.

3.10.1 Names #

A name MUST match the identifier grammar of §3.19:

[A-Za-z_][A-Za-z0-9_.-]*

A collector MUST discard a record whose name does not, for the same two reasons an origin is constrained (§3.7): names are matched against patterns in which * is the wildcard, and names are what read access is granted on.

Beyond that, naming is convention and a collector MUST NOT enforce any. The conventions in use are a dot-separated hierarchy from general to specific, the unit as the last component, and a cumulative name for a cumulative quantity:

system.cpu.usage
disk.read.bytes
request.duration.seconds
http.requests.total

3.10.2 Labels #

Labels are the dimensions of a measurement: which core, which device, which method. cpu.usage with core="0" and cpu.usage with core="1" are two series, not two samples of one.

Label keys and values MUST be non-empty UTF-8 strings. A key MUST match the identifier grammar above; a value MUST NOT contain = (0x3D) or , (0x2C), which are reserved as delimiters in the collector's canonical representation of a label set (§3.13). A key MUST NOT be repeated within one sample, and MUST NOT be any of the six fixed field names a metric result carries — timestamp, boot_id, name, type, value, overflow — because labels and fixed fields share one flat namespace in a result record (§3.22) and a collision would make the record ambiguous.

A record violating any of these is discarded (§3.12).

Label cardinality is the producer's responsibility. Each distinct combination of label values is a distinct series, so labels whose values are unbounded — request identifiers, user-supplied strings, timestamps — produce series without limit, and a collector is required neither to cap them nor to degrade gracefully when they arrive. A producer SHOULD use labels whose value sets are small and known. A dimension that is not bounded belongs in an event payload, where it costs one field, not in a label, where it costs a series.

A collector may bound its total metric storage through retention without limiting series creation. Peios eventd does this by default: it continues accepting new series but removes the oldest samples when the metric store's logical live size exceeds its configured ceiling. That protects the volume; it does not make high-cardinality labels safe, because an active set larger than the series cache can still thrash ingestion.

3.10.3 Types #

The type is fixed when the series is first seen and is immutable. A sample that resolves to an existing series but declares a different type is discarded (§3.12), permanently and without a response to the producer. A producer that changes the type of a metric it already emits has stopped emitting it. The mainline collector counts these conflicts, retains the latest metric name and expected and received types in memory, and may report them through its bounded operator diagnostics (§3.12).

3.10.3.1 Counter #

A value that only increases, and resets to zero when the producer restarts. Used for cumulative quantities: requests served, bytes transmitted, errors encountered.

A counter value MUST be a finite, non-negative binary64 value.

The raw value is rarely what a reader wants; the rate of change is (§3.25). A decrease is read as a restart rather than as a negative change, which is why the type must be declared: the same number sequence means something different for a gauge.

3.10.3.2 Gauge #

A value that may move in either direction. Used for current state: a utilisation, an amount in use, a depth, a temperature.

A gauge value MUST be a finite binary64 value, and MAY be negative.

3.10.3.3 Histogram #

A distribution of observations across buckets the producer chose. Used where the shape matters more than the mean — latencies above all.

A histogram value carries:

• boundaries: a non-empty array of bucket upper bounds, strictly increasing in the order given
• counts: one cumulative count per boundary, each being the number of observations less than or equal to that boundary; non-decreasing, and each no greater than the total
• total_count: the number of observations
• sum: the finite sum of the observations

Boundaries are part of the series identity (§3.13). A collector MUST NOT sort or reinterpret them; a producer that changes them has started a new series, and SHOULD therefore keep them fixed for the life of a metric.

The final count MAY be less than the total: observations above the highest boundary are the difference between them, and are not otherwise represented. A total of zero is a valid empty sample, in which case every count and the sum MUST be zero.

3.10.4 Values are floating point #

Numeric input MAY be a MessagePack integer or a MessagePack float; both are converted to binary64 with round-to-nearest, ties-to-even. Every value a collector stores and every numeric value a query returns is a finite binary64. The null value in an explicit percentile-overflow result is not a numeric metric value (§3.25).

Non-finite values are refused rather than stored: a record whose value converts to NaN or to either infinity is discarded (§3.12). There is no representation for a missing measurement — a producer with nothing to report sends nothing, and the gap is the answer (§3.13).

A metric datagram carries one record, encoded as a MessagePack map, or several, encoded as an array of maps. A collector MUST accept both (§3.9). Each map is one sample of one series.

3.11.1 Fields #

A collector MUST ignore fields it does not recognise (§3.29).

3.11.2 The histogram value #

For a histogram, value is a map:

Counts and total_count MUST be MessagePack unsigned integers, or non-negative signed integers. boundaries and sum MAY be integers or floats and are converted as §3.10 requires.

3.11.3 type is per record, not per series #

Every record declares its type, including the second and every subsequent sample of a series that already exists.

This is redundant on the wire and deliberately so. A producer holds no state about what a collector already knows, has no way to ask, and MUST NOT be required to establish a series before sampling it: the first sample of a series and the millionth are the same message. The redundancy is what makes a producer stateless, and the cost is one short string per sample.

The collector uses the declaration only on first sight. Afterwards it is a consistency check, and a record that fails it is discarded (§3.10).

3.11.4 Timestamps need not increase #

A collector MUST store a valid sample whose timestamp is older than samples it already holds for that series.

Producers batch, clocks step, and a collection sweep may be submitted out of order or retried. A collector that refused late samples would turn any of those into silent data loss, so it accepts them and defines every ordering it performs over timestamp rather than over arrival (§3.21, §3.25). Two samples of one series MAY share a timestamp; the collector orders them deterministically and a client MUST NOT depend on which comes first.

Failures on this channel are silent, exactly as on the log channel (§3.4, §3.8). The scopes are the same, with one difference that matters: a metric record has no ignorable field. Every field a metric record carries participates either in the series identity or in the measurement, so there is nothing whose loss leaves a usable record behind. A malformed timestamp costs the log line nothing and costs the sample everything.

3.12.1 The datagram is discarded #

• it is not valid MessagePack
• it decodes to something that is neither a map nor an array of maps
• no KACS sender token arrived
• the kernel reported its data or ancillary data truncated (§3.9)
• the collector cannot query the sender token or resolve publication policy

3.12.2 The record is discarded #

A collector MUST discard a record, leaving the rest of its batch untouched, for any of the following.

Structure

• a required field is absent, or has the wrong type
• the map contains a duplicate top-level key
• type is not exactly "counter", "gauge" or "histogram"

Name and labels (§3.10)

• name is empty or does not match the identifier grammar
• a label key or value is not a string, or is empty
• a label key does not match the identifier grammar
• a label value contains = or ,
• a label key is repeated within the record
• a label key is one of timestamp, boot_id, name, type, value, overflow

Timestamp

• timestamp is present and is not an integer, is negative, or is outside the timestamp domain (§3.5)

Counter and gauge values

• the value is not a number
• it converts to a non-finite binary64
• it is negative and the type is counter

Histogram values

• the value is not a map, or its map has a duplicate key
• a field is absent or has the wrong type
• boundaries is empty
• a boundary or sum converts to a non-finite binary64
• counts and boundaries differ in length
• the converted boundaries are not strictly increasing
• a count is negative, the counts are not non-decreasing, or a count exceeds total_count
• total_count is zero and any count or sum is non-zero

Series consistency

• the record resolves to an existing series whose type differs (§3.10)

Publication policy

• the conveyed KACS token is denied EVENTD_PUBLISH for name (§3.9)

3.12.3 Why the timestamp rule differs from logs #

On the log channel a malformed timestamp is ignored and the record kept; here it discards the record.

The asymmetry is not an inconsistency. A log line with the wrong time is still the line, and reading it is still worth doing. A sample is a (time, value) pair and nothing else: attaching the collector's receipt time to a measurement taken at an unknown moment does not recover the sample, it fabricates one, and it fabricates one that will be charted next to real ones. Discarding leaves a gap, which is honest (§3.4).

Absence is different from malformation. A record that simply omits timestamp is asserting "now", and the collector's clock is the right answer to that.

3.12.4 Silence, again #

A collector MUST NOT emit an event, write a durable log, or increment anything a query client can observe in response to any failure in this article — with no exception for the series-consistency failure, which is the one that most looks like it deserves one. Local in-memory diagnostic counters and rate-limited standard-error output are permitted by §3.4.

A producer that changes a metric's type is misconfigured, and the misconfiguration is permanent: every sample is discarded for as long as the series exists, and the producer receives no response. Peios eventd increments an in-memory total, retains the latest conflicting metric name and expected and received types, emits at most one coalesced standard-error warning per minute, and includes the total and latest conflict in its SIGQUIT diagnostic dump. None of those signals is part of the query interface or grows in proportion to rejected input.

Two samples belong to the same time series when they agree on:

• the name, compared exactly, byte for byte; and
• the label set, compared as an unordered set of key-value pairs, each compared exactly; and
• for histograms, the bucket boundaries, compared as an ordered sequence of binary64 values.

Nothing else participates. Type does not: a record whose type disagrees resolves to the series and is then discarded for disagreeing (§3.10). Boot ID does not: a series continues across a reboot, and a client that wants one boot's worth filters for it (§3.25). Time does not.

3.13.1 Order does not distinguish a label set #

{core: "0", host: "a"} and {host: "a", core: "0"} are the same series. A collector MUST compare label sets as sets.

To do so it needs a canonical form, and the form in use is the reason label values may not contain = or , (§3.10): pairs are sorted by key in unsigned UTF-8 byte order, each written key=value, and joined with commas. Because neither delimiter can occur inside a key or a value, no escaping is needed and no two distinct label sets can produce the same string.

The byte form itself is the collector's business and this chapter does not require it. What it requires is the property: a label set has exactly one identity, independent of the order the producer wrote it in. The delimiter reservation is stated normatively because it binds the producer, and a producer cannot see the encoding that motivates it.

3.13.2 Absent labels and empty labels #

A record with no labels field, a record with an empty labels map, and a record whose labels were all discarded are the same series: the one with no labels. There is no distinction between "unlabelled" and "labelled with nothing".

3.13.3 Boundaries are identity, not metadata #

Two histogram samples with different boundaries are different series even when name and labels agree.

This is the consequence that surprises producers, and it is unavoidable: cumulative counts against one set of bucket edges cannot be compared with counts against another, so calling them one series would mean computing percentiles across incommensurable distributions. A producer that re-tunes its buckets each collection cycle creates a series each cycle, each holding a single sample and each surviving until retention removes it.

A collector MUST NOT defend against this. It is a producer defect, it is indistinguishable at the interface from legitimately introducing a new metric, and every defence available — capping series, merging near-equal boundary sets, rejecting a second boundary set for a name — would break a correct producer to inconvenience an incorrect one.

3.13.4 Series are created, never announced #

A series comes into existence when its first sample arrives. There is no registration message, no schema, and no way to declare a series in advance or to retire one.

A collector MUST NOT require a series to be known before a sample of it is accepted, and MUST NOT limit how many series exist. A series with no remaining samples ceases to exist when retention removes the last of them; nothing else removes one.

3.13.5 Gaps are preserved #

A collector MUST NOT interpolate, backfill, or synthesise a sample that a producer did not send.

A missing sample is a real fact about the system — the producer was down, the datagram was dropped, the sweep was late — and it is a fact the metric is often being watched for. A series with a hole in it is returned with a hole in it, and a client that wants a value across the hole computes one itself.

A collector MUST expose an AF_UNIX SOCK_STREAM socket for queries, protected by a Security Descriptor as §3.3 requires.

One socket serves all three data types. The mode a query runs in — events, logs or metrics — is determined by parsing the query string (§3.18), never by the transport, so a client needs no connection setup, no mode selection, and no separate endpoint per data type.

3.14.1 One query per connection #

A connection carries exactly one query. A client that wants two concurrent queries opens two connections.

A collector MUST close the connection after the terminal message of a non-streaming query (§3.16), and MUST treat a client disconnect as cancellation of a streaming one.

3.14.2 Identity #

A collector MUST establish the client's identity from the connected socket, before executing anything, by obtaining the peer's token. This is possible here and not on the ingestion channels, and it is the whole reason the query channel is a stream (§3.3).

The token is captured once, at connection time, and is a snapshot. A client whose privileges change while a query runs — and in particular while a streaming query runs, which may be indefinitely — is evaluated throughout against the token it connected with.

If a collector cannot obtain the peer token, it MUST refuse the query. It MUST NOT execute a query for an unidentified caller, and MUST NOT fall back to any other means of identifying one.

3.14.3 No credentials cross this channel #

There is no message with which a client offers a credential and none with which a collector asks for one. Identity is established from the connection and from nothing else.

3.14.4 Concurrency #

A collector MUST bound the number of queries it will execute at once, and MUST reject a query beyond the bound with an error rather than queueing it behind the others.

The streaming bound is the lower of the two and is enforced separately, because a streaming query holds its resources for as long as its client stays connected while an ordinary one holds them for at most a timeout (§3.16).

Both bounds are global. Because every authenticated caller may connect (§3.3), a collector MUST also bound the queries one caller may have running at once, so that one caller cannot occupy every slot. The caller is the user SID of the token captured at connect. A collector MAY leave SYSTEM's queries uncounted. A query beyond the caller's bound is rejected with an error, like one beyond the global bounds.

Neither bound may affect ingestion: queries and ingestion are separate channels precisely so that exhausting one cannot exhaust the other (§3.3).

Every message in either direction is a length-prefixed MessagePack value:

length is little-endian, as PSPU §1.2 requires of every integer field in this document. It counts the payload only; the four bytes of the prefix are not included.

There is no magic value and no version field. The channel is a Unix socket at a configured path, so there is no possibility of reaching the wrong service by accident in the way a shared header guards against (PSPU §2.7), and versioning is handled as §3.29 describes.

3.15.1 The two size controls #

A collector declares two independent sizes. The query request ceiling is a hard inbound limit. The query response target is a soft outbound batching target. They are separate because an untrusted client must not be able to force an arbitrarily large allocation merely by declaring a request length, while a record the collector has already accepted must remain possible to return.

Inbound. A collector MUST refuse a request whose length exceeds the request ceiling. It MUST do so without reading the payload — the point of checking the prefix is to avoid allocating for a request that a malicious or broken client has declared too large — and it MUST send an error response (§3.16) before closing the connection. A collector MUST NOT close on an oversized request silently: a bare close is indistinguishable from a crash, and leaves a client unable to tell that shortening its query is the remedy.

Outbound. A collector SHOULD chunk result records so an ordinary response does not exceed the response target (§3.16). The target is not a ceiling: when one complete record would exceed it, the collector MUST send that record alone in one response rather than split, truncate, skip, or reject it.

3.15.2 Every stored record must remain returnable #

A collector MUST NOT store a record whose encoded result representation cannot fit in the protocol's u32 payload length. This structural framing bound is the only outbound ceiling.

A single result record is never split across messages (§3.16). A collector MUST therefore ensure that its accepted record encodings have a bounded result expansion that fits one frame. It MAY establish that statically from the protocol's record limits; it need not flatten or encode each record on the ingestion path merely to check response size. An implementation MAY encode a large response incrementally after writing its known length; it need not allocate another buffer as large as the record.

The ingestion ceilings (§3.6, §3.9) remain independent configuration. Raising one does not require raising the response target: it merely makes a single-record response above that target more likely. The structural u32 requirement is a property of the wire encodings, not a configurable coupling between those values.

3.15.3 Requests #

A request is a MessagePack map:

A collector MUST send an error response and close the connection if the payload is not valid MessagePack, is not a map, omits query, gives query a non-string value, or contains a duplicate top-level key. It MUST ignore unrecognised fields that are not duplicates (§3.29).

Unlike the ingestion channels, nothing here is silent. A query client is identified (§3.14), is one of a bounded number, and is asking a question, so telling it what went wrong is neither an amplification vector nor an information leak — with the one exception §3.28 sets out.

A response is a MessagePack map whose status field names its kind. There are four.

3.16.1 Result messages #

An "ok" message carries records, an array of flat maps (§3.22).

Records are chunked at record boundaries: a successful query sends one or more "ok" messages, ordinarily at or below the response target (§3.15). A collector MUST NOT split one record across two messages. If one complete record exceeds the target, that record occupies an otherwise empty "ok" message by itself. It MUST NOT be truncated, partially sent, skipped, or turned into a query error merely because it exceeds the target.

Each record is self-describing and records in one response MAY carry different sets of keys — event payload fields vary by event type, metric labels vary by series. A client MUST NOT assume a uniform schema across a result set, and MUST NOT infer that a key absent from one record is absent from the data.

A successful query with no matching records sends exactly one "ok" message with an empty records array, then its terminal message. A collector MUST NOT omit it: "no records" and "the query has not yet produced records" are different states and a client must be able to tell them apart.

3.16.2 The two terminal messages #

"end" terminates a non-streaming query. "watch" marks the point in a streaming query where the stored result set ends and live delivery begins (§3.27). A query sends exactly one of them, never both.

Until one has arrived, the query has not succeeded. A collector that fails partway through MUST send "error", and a client that receives "error" before either terminal message MUST discard every "ok" message it received for that query. Partial results are not results: they are an arbitrary prefix of an ordering that was never completed, and a client that kept them would silently under-report.

An "error" after "watch" is different. It terminates the stream, and the records already delivered remain valid — they were complete when they were sent, and the ordering they belonged to had already closed.

3.16.3 Errors #

An "error" message carries error, a human-readable string.

There is no error code and no machine-readable classification. This is a deliberate limit on the interface: an error here is a parse failure, a type mismatch, a timeout, a limit, or a refusal, and a client's response to all of them is the same — show it to whoever wrote the query. A client MUST NOT parse the string, and a collector MAY change the wording of any error at any time.

A collector MUST NOT include in an error message any value the client was not authorized to read (§3.28).

3.16.4 Timeouts #

A collector MUST bound the time a query may take to reach its terminal message.

The clock starts once the request has been decoded and the caller's token obtained, and it covers everything that follows: parsing, access checks, cross-type pre-computation, execution, merging, aggregation, pagination, projection and transmission. A collector MUST send "end", or for a streaming query "watch", before it expires.

The timeout bounds the initial result set only. Once a streaming query has sent "watch" its watch phase is not time-limited; what bounds it instead is the streaming concurrency limit (§3.14), the distinct-value limit (§3.27), and the client's own ability to keep up (§3.27).

On expiry a collector MUST cancel the query and send "error". Any "ok" messages already sent are discarded by the client under the rule above.

3.16.5 Memory #

A non-aggregating query in the default order (§3.21) needs to hold nothing while it runs: its records can be read in that order and sent as they are read. A collector SHOULD answer it so, and MUST NOT refuse it for the size of its result, however large. A client that wants every record of a large store asks for exactly this.

Other queries must hold something to answer: a sort on another field holds the records it orders, or with TAKE only the best SKIP + TAKE of them, and an aggregation holds its groups. A collector MAY bound the memory queries hold. When a query would exceed the bound, the collector MUST fail it with "error". It MUST NOT truncate the result, drop groups, or answer from part of the data instead.

Every value in a result record is encoded as follows.

A GUID is rendered as a string rather than as sixteen bytes because a result record is read by people as often as by programs, and a raw GUID in a terminal is unreadable. The canonical form is lowercase 8-4-4-4-12 hexadecimal within braces, as PCDS defines it. A query comparing against a GUID accepts either braced or unbraced input, and compares case-insensitively (§3.19); a result always uses the canonical form.

3.17.1 Maps do not appear as values #

An event payload is a MessagePack map, and result records are flat (§3.22). A map in a stored payload is therefore a container to be flattened, not a value to be emitted: its entries become top-level keys of the record, joined by dots, and the map itself never appears.

Arrays are different. An array is emitted as an array value at its flattened path, and a collector MUST NOT traverse into it. Maps nested inside an array are preserved as that array's contents, unflattened and unqueryable.

The asymmetry is deliberate. A map has keys, so its entries have names that can be addressed, granted access to and indexed. An array has positions, and a path like hops.3.address would mean something different in every record — so an array is carried across whole and treated as one value.

Being one value is also why == against an array field compares the whole array. The question usually being asked of one — does it hold this element — is HAS (§3.20).

Binary values in a payload stay binary. A collector MUST NOT render bin as a string, in either direction.

3.17.2 Missing and null are the same value #

A field absent from an event payload or from a metric's label set encodes as nil, exactly as an explicitly null one does, and the two are indistinguishable in a result record.

This is consistent throughout: they compare equal, they sort together, and they group together (§3.20, §3.21). A collector MUST NOT distinguish them anywhere in the query surface, and a client MUST NOT attempt to.

A query is one string. It names a mode, narrows to some data, and says what to do with it.

EVENTS kacs.* SINCE 1h ago WHERE emitter.process.guid == "550e8400-e29b-41d4-a716-446655440000" TAKE 100
LOGS FROM loregd ERROR ONLY CONTAINING "connection refused" SINCE 1d ago
METRIC cpu.usage[core="0"] SINCE 1h ago AVG_OVER 5m

3.18.1 Three modes #

The first token selects the mode, and a collector MUST reject a query whose first token is not one of them.

• EVENTS searches structured event records, primarily by event type (§3.23).
• LOGS searches log output, primarily by origin (§3.24).
• METRIC evaluates measurements, primarily by name and labels (§3.25).

Events and logs are record-oriented: collections you search, returning the records that matched. Metrics are value-oriented: measurements you evaluate, returning numbers computed from samples. The modes differ because the data differs, and forcing all three through one shape would serve none of them.

3.18.2 The primary selector #

Immediately after the mode comes an optional primary selector, specific to the mode: an event type pattern, FROM with one or more log origins, or a metric name with an optional label selector. It narrows the data before anything else runs.

A primary selector MUST NOT be repeated unless its mode defines a list form — LOGS FROM a, b is one selector naming two origins, not two selectors.

3.18.3 Clauses #

Everything after the primary selector is a clause, and clauses MAY appear in any order. EVENTS SINCE 1h ago TAKE 10 and EVENTS TAKE 10 SINCE 1h ago are the same query.

Order of appearance never affects meaning. Execution follows the fixed sequence below regardless of how the string was written, so a collector MUST NOT derive semantics from clause position.

These clauses work identically in all three modes:

3.18.4 Execution order #

Whatever order the clauses were written in, a collector MUST evaluate them in this sequence:

Two positions in that list are load-bearing.

Access control is third, before every filter, aggregate, sort and limit. It is part of the query's logical execution and not a filter applied to the output (§3.28).

SELECT is last. It shapes the output and nothing else; a field it omits is still available to every earlier phase (§3.22).

3.18.5 Repetition #

A clause MUST appear at most once, and a collector MUST reject a repeat as a parse error, with two exceptions:

• WHERE is repeatable. Multiple WHERE clauses are combined with AND, each treated as a parenthesised group: WHERE a == 1 OR b == 2 followed by WHERE c == 3 means (a == 1 OR b == 2) AND c == 3.
• SELECT is repeatable where it is valid at all, and is additive: SELECT event.time SELECT event.type names both fields.

Both exist so that a query can be built up in pieces — by a tool appending a filter, or by a person adding one to a query they already have — without rewriting what is already there.

3.18.6 Counts #

TAKE, SKIP and the N of TOP N BY are unsigned decimal integers that MUST fit in 64 bits. A negative, hexadecimal, floating-point or missing count is a parse error.

SKIP defaults to 0. TAKE omitted means no limit. TAKE 0 and TOP 0 BY are valid, and return no records after every earlier phase has run — which is not the same as not running the query, because a TOP 0 BY still counts and a TAKE 0 still enforces access control.

3.18.7 Case #

Keywords are matched case-insensitively, using ASCII case folding, in grammar positions where a keyword is expected. This document writes them in uppercase by convention only.

Identifiers are case-sensitive, except where the language defines a named alias for a value (§3.23).

A word spelled like a keyword MAY be used where the grammar expects an identifier or a value: LOGS FROM stream selects the origin stream, while LOGS STREAM enables streaming. A collector MUST resolve the ambiguity by grammar position and MUST NOT reserve keywords globally.

A query string is UTF-8. Whitespace separates tokens outside quoted strings and is otherwise insignificant.

3.19.1 Identifiers #

An unquoted identifier is ASCII and matches:

[A-Za-z_][A-Za-z0-9_.-]*

Identifiers name fields, payload paths, metric names, label keys, event type patterns, log origins and value aliases. ., _ and - are permitted inside one; /, :, whitespace, quotes, brackets, parentheses, commas and the comparison operators are not.

This grammar is the same one that constrains a metric name and a metric label key at ingestion (§3.10), which is what makes every stored metric identifier writable here without quoting. A log origin is the one exception: it is constrained to a near relative of this grammar (§3.7) that also admits a leading digit and a producer component such as /ExecStartPre[0], and an origin using either is written as a quoted string.

A value that cannot be written as an identifier MUST be written as a quoted string. Quoted forms are accepted anywhere an identifier is — they are never required for a conforming identifier, but a pattern may need one, and a collector holding identifiers stored under an earlier revision must still be able to select them.

3.19.2 Strings #

A string literal is double-quoted UTF-8. The escapes are \", \\, \n, \r, \t, and \uXXXX for a scalar value in U+0000 to U+FFFF written as four hexadecimal digits.

A collector MUST reject any other backslash escape as a parse error, and MUST reject \uXXXX naming a surrogate code point in U+D800 to U+DFFF. Surrogate pairs are not decoded: a character outside the basic multilingual plane is written directly as UTF-8, not as two escapes.

3.19.3 Binary #

A binary literal is a lowercase x, a double quote, an even number of hexadecimal digits, and a closing quote:

WHERE target_sid == x"010500000000000515000000"

Hexadecimal digits inside are case-insensitive. x"" is valid and is the empty byte string. Whitespace inside the payload, an odd digit count, and any non-hexadecimal character are parse errors.

A binary literal compares only against MessagePack bin values. A collector MUST NOT coerce one to a string or a string to one: x"6162" and "ab" are different values and never compare equal (§3.20).

3.19.4 Integers #

An integer literal is decimal or hexadecimal.

WHERE emitter.class == 2
WHERE access.granted == 0x1F01FF

A decimal integer MAY carry a leading -, in which case it MUST fit in signed 64 bits; without one it MUST fit in unsigned 64 bits. A hexadecimal integer is 0x followed by one or more digits, is always non-negative, and MUST fit in unsigned 64 bits. A leading + is not valid. An out-of-range literal is a parse error.

3.19.5 Floats #

A float literal is a finite decimal number with an optional leading - and either a fractional part or an exponent: 42.0, 0.001, 1e6, -1.25e-3. A token that looks like an integer, such as 42, is an integer literal and not a float.

Float literals are binary64 and MUST be finite. NaN, Infinity, -Infinity and any literal that overflows to infinity are parse errors. A leading + is not valid.

3.19.6 Booleans and null #

true and false are matched case-insensitively with ASCII folding.

NULL, likewise folded, is valid only in IS NULL and IS NOT NULL. A collector MUST reject field == NULL and field != NULL as parse errors rather than evaluating them.

3.19.7 Durations #

A duration is an unsigned decimal integer followed immediately by s, m, h or d — seconds, minutes, hours or days. A zero duration is a parse error.

3.19.8 Times #

Absolute literals are a fixed UTC subset. Components MUST be zero-padded exactly as shown, the date MUST be a valid Gregorian date, hours are 00–23, minutes and seconds 00–59. Leap seconds are not accepted. Timezone suffixes and fractional seconds are not part of this revision and MUST produce a parse error.

3.19.9 The evaluation time #

A collector MUST capture the evaluation time once, before execution begins, and MUST use that one reading for every ago, every hence, and for an omitted UNTIL, throughout the query — including throughout the watch phase of a streaming query.

A query that read the clock more than once could produce a range whose end preceded its start, or a window that grew while it was being scanned. One reading makes the effective query range a fixed interval for the life of the query.

SINCE is inclusive, UNTIL is exclusive: the effective query range is [SINCE, UNTIL). If SINCE is greater than or equal to UNTIL the query returns no records — which is a successful query with an empty result (§3.16), not an error. A time literal that evaluates outside the timestamp domain MUST produce an error (§3.5).

3.20.1 Operators #

IN and NOT_IN take a non-empty parenthesised, comma-separated list of literals. An empty list is a parse error.

WHERE origin IN ("loregd", "peinit")
WHERE emitter.class NOT_IN (kacs, lcs)

= is not a comparison operator and MUST produce a parse error, with one exception: inside a metric label selector, where = and == are both equality (§3.25).

3.20.2 HAS asks about an array #

HAS is true when the field holds an array one of whose elements equals the value, compared as == compares it, so a binary literal matches a binary element exactly and a string element folds case.

EVENTS WHERE subject.token.groups HAS x"010200000000000520000000200200"

A payload field MAY hold an array — a token's group SIDs are the canonical example — and == against an array is whole-value equality: an array equals only an array of the same length whose elements are equal in order. That is what an operand of a different type compares against, so == between an array field and a single element is a type mismatch and evaluates false, exactly as §3.20's "types do not coerce" says. HAS is the operator that asks the other question.

HAS applied to a field that is not an array evaluates false for that record, for the same reason an ordering operator applied to a non-numeric value does: a payload field's type varies from record to record. Elements are compared one level deep — an element that is itself an array is compared whole, not searched.

HAS is not a set operator and MUST NOT be confused with IN. IN asks whether a scalar field equals one of several literals; HAS asks whether one array field holds one literal. CONTAINS is substring matching on a string and has nothing to do with either.

A collector MAY evaluate HAS by scanning: array containment need not be indexable, and answering correctly without an index is required where answering quickly is not.

3.20.3 Strings fold case #

Every string comparison — ==, !=, STARTS_WITH, ENDS_WITH, CONTAINS, HAS, IN, NOT_IN — is case-insensitive, using ASCII-only folding: bytes A–Z compare equal to a–z, and every non-ASCII byte compares exactly.

This applies uniformly: to event header fields, to payload fields, to log messages, to metric label values, and to the pattern matching of primary selectors. Integers, floats, GUIDs, timestamps and binary values are unaffected.

3.20.4 Numbers compare mathematically #

Integers and floats compare by mathematical value, not by casting both to one storage type.

An integer equals a finite float only when the float represents exactly that value. Ordering between an integer and a float MUST be exact, including for integers outside the range binary64 can represent exactly. A collector MUST NOT resolve 9007199254740993 > 9007199254740992.0 by converting the left operand to a float, which would make it false.

3.20.4.1 NaN #

An event payload may carry a NaN (§3.5), though a literal never can (§3.19). Every NaN is one value: it equals every other NaN and nothing else, so x == 5 is false and x != 5 is true for a record whose x is a NaN.

An ordering operator with a NaN on either side evaluates false, as IEEE 754 has it. x < 5 and x >= 5 are both false for that record. Sorting and grouping still need a place for a NaN, and §3.21 gives it one.

3.20.5 Types do not coerce #

Values of different non-numeric types are never equal. The string "1" is not the integer 1, and != between them is true.

An ordering operator applied to a field whose runtime value is non-numeric evaluates false for that record — not an error, because a payload field's type varies from record to record and a query cannot know in advance.

An ordering operator applied to a known fixed field whose declared type cannot be ordered is different: a collector MUST reject the query during parsing or planning rather than executing a predicate that can never match. WHERE message > 5 is a mistake the collector can see, and returning zero records for it would be a wrong answer that looks like a right one.

Binary values compare by exact byte equality under ==, !=, HAS, IN and NOT_IN. Ordering is not defined for binary values, and a predicate applying an ordering operator to a binary literal MUST produce a parse error.

3.20.6 Absent fields #

A field absent from an event payload or from a metric's label set resolves to null (§3.17).

Every comparison against null evaluates false, except IS NULL, which is true, and IS NOT NULL, which is false. In particular WHERE field != "x" does not match records lacking the field: a record with no opinion is not a record with a different opinion.

3.20.7 Combining predicates #

Predicates within one WHERE combine with AND and OR. AND binds tighter than OR. Parentheses override.

Multiple WHERE clauses combine with AND, each parenthesised as a group (§3.18).

There is no NOT. Negation is written with the negative operators — !=, NOT_IN, IS NOT NULL — and a collector MUST reject NOT as a parse error rather than silently treating it as an identifier.

Two rules govern this article, and both exist for the same reason.

Ordering MUST be total and deterministic. For a fixed set of stored records, one query MUST produce one order. Without that, SKIP and TAKE are meaningless: a client paging through results would see records twice and never see others, and would have no way to tell.

Equality here MUST be the query language's, not the storage engine's. A collector that grouped by whatever its database considers equal would group differently depending on how it was built.

3.21.1 SORT #

SORT orders by one or more fields. Each defaults to ascending; ASC may be written, DESC reverses that field.

SORT timestamp DESC
SORT origin ASC, timestamp DESC

If the named fields do not uniquely order two records, a collector MUST append internal tiebreakers until the order is total. The tiebreakers are not query-language fields, are never emitted in a result record, and a client MUST NOT depend on their identity — only on their effect, which is that the order is stable.

When no SORT is present:

• Events and logs are ordered by timestamp descending, most recent first — the order a person reading a log wants.
• Metrics are ordered by timestamp ascending, the order a chart wants.

3.21.2 Value ordering #

SORT uses the query language's ordering, not the storage engine's. Missing fields and explicit nulls are equivalent. Ascending order sorts by type first, in this order:

1. Null
2. Boolean, false before true
3. Numeric, integers and floats compared mathematically (§3.20)
4. String and GUID, ASCII-folded
5. Binary, unsigned lexicographic
6. Array

DESC reverses the whole ordering, type order included.

A NaN is numeric and sorts after every other number, the infinities included. Every NaN is one value, so NaNs tie with each other and fall to the tiebreakers. Without a place for it the order would not be total: a NaN is neither less than, greater than nor equal to any number under IEEE 754, and a sort given such a comparison may produce any order at all, or fail.

Strings and GUIDs compare with the same ASCII folding as predicates. Two strings equal under folding are ordered by their original UTF-8 bytes, so that folding never costs totality. Binary values compare as unsigned bytes. Arrays compare by their canonical MessagePack encoding (§3.5).

Maps do not appear as result values (§3.17) and MUST NOT appear as sort keys.

3.21.3 Grouping #

COUNT BY, TOP N BY, GROUP and DISTINCT use query-language equality:

• missing and null are one group
• integers and floats that are numerically equal are one group
• every NaN is one group
• strings and GUIDs group under ASCII folding
• binary values group by exact bytes

3.21.4 The canonical representative #

When a group's members are equal under those rules but not byte-identical — "Loregd" and "loregd", or 1 and 1.0 — the value emitted for the group MUST be its canonical representative:

Choosing the smallest rather than the first makes the representative a property of the set, independent of the order records were read in — which matters because a collector may read them from several places at once and merge (eventd TRMP §6.4).

3.21.5 Ordering of aggregates #

COUNT BY and GROUP … COUNT results are ordered by count descending. Ties are broken by the group key under the value ordering above — for a GROUP of several fields, each key in turn in the order the fields are named — then by the representative's encoded bytes.

GROUP … SUM, AVG, MIN and MAX results MUST be ordered by their group keys ascending under the value ordering above, comparing each key in the order the fields are named. This order applies before SKIP and TAKE. An explicit SORT overrides it; groups tied under that sort retain their group-key order.

TOP N BY is exactly COUNT BY with TAKE N applied after that ordering.

DISTINCT results are ordered by the distinct value under the value ordering, unless an explicit SORT overrides it.

Every result record is a flat MessagePack map. There is no nesting in a result, in any mode.

Flatness is what makes one set of rules — for access control, for ordering, for grouping, for projection — apply uniformly to a header field, a payload field and a metric label alike. A nested result would need a path language, and a path language would need to be reproduced identically by every SD author and every client.

3.22.1 Event fields #

These names resolve to header fields, by the mapping of PGSS §6.4:

event.time, event.sequence, event.cpu, event.type, event.boot.guid, emitter.class, emitter.token.guid, emitter.true-token.guid, emitter.process.guid

Every other name resolves to a payload field.

3.22.1.1 Header names are reserved #

Header field names are reserved in the query language and in result maps. If a payload carries a value whose flattened path is a header field's name, the header wins.

The colliding payload value is stored unchanged, and is retrievable as part of the raw payload by whatever holds it, but it is not exposed through field resolution, SELECT, WHERE, aggregation, access control or result maps. If the payload carries a map at a header field's path, the map's entire subtree is suppressed in the same way.

Only the paths themselves are reserved. A payload field beside them, such as emitter.process.pid, resolves normally. An emitter MUST NOT write a value at a header path (PGSS §6.4).

3.22.1.2 Flattening #

Payload maps are flattened recursively, path segments joined with .: a payload {source: {name: "x"}} exposes the field source.name.

Each map key on a queryable path MUST be a MessagePack string matching:

[A-Za-z_][A-Za-z0-9_-]*

Note that . is not permitted in a segment, though it is permitted in an identifier generally (§3.19) — a key containing a dot could not be distinguished from a path through two maps.

A key that is not a string, contains ., or does not match the grammar is stored unchanged and is not queryable: it does not resolve, does not appear in a result map, and has no field identity for access control. An empty map produces no field at all.

Maps are containers; every non-map value, arrays included, is emitted at its flattened path (§3.17).

If two payload entries flatten to the same path, the first in MessagePack map order wins and later duplicates are suppressed.

3.22.2 Log fields #

timestamp, origin, is_error, message, boot_id, job_id

The set is closed. There are no payload fields and no flattening, and a collector MUST reject any other log field name as a parse error rather than resolving it to null. A log record has a fixed shape, so a name outside it is a mistake the collector can see — unlike an event payload field, which may legitimately be absent from a given record.

is_error is a boolean in the query language, and compares against true/false or against 1/0.

3.22.3 Metric fields #

timestamp, boot_id, name, type, value, overflow

Every other name resolves to a label. Ingestion refuses labels colliding with these six (§3.10), so the flat namespace is unambiguous by construction rather than by a precedence rule.

type is the series type as a string: "counter", "gauge" or "histogram".

overflow is an output-only boolean marker. It is present and true only when a histogram percentile cannot be located below the series' highest boundary (§3.25); otherwise it is absent. Predicates and label selectors therefore see it as null, like any other output not yet computed.

3.22.3.1 Types belong to the query mode #

Header aliases and GUID literal forms apply only where a name is a header in that query mode. A metric label named emitter.class remains a string label: "kacs" is not the event-header class number. Likewise, a metric label named job_id or an event payload field named boot_id does not acquire the log header's GUID type. Ordinary string comparison rules apply, including to bracket selectors and cross-type metric selectors. The actual GUID headers still accept braced and unbraced input (§3.17).

3.22.4 What a record contains #

Event records carry the header fields plus every non-suppressed flattened payload field, as top-level keys.

Log records carry the log fields.

Raw metric sample records carry timestamp, boot_id, name, type, value, and the series' labels as top-level keys. A percentile-overflow result additionally carries overflow.

Aggregated metric results carry name, type and value, plus labels when the result belongs to one label set. They carry boot_id only when the query restricted the samples to exactly one boot by a boot_id equality predicate, in which case the value is that boot ID; a result that could span boots omits it rather than picking one. An aggregate tainted by a percentile overflow additionally carries overflow (§3.25).

Aggregation results in event and log mode carry the group key fields and the aggregate output, with the fixed schemas of §3.23.

3.22.5 SELECT #

SELECT narrows a result record to the named fields. It is valid only for non-aggregating event and log queries.

A collector MUST reject SELECT combined with COUNT BY, TOP N BY, DISTINCT or GROUP, and MUST reject it in metric mode: all of those have fixed output schemas, and a clause that reshapes a fixed schema is a contradiction rather than a refinement.

SELECT is applied last, after every other phase (§3.23). It controls the shape of the output and nothing else: a field not selected is still available to WHERE, to SORT, and to grouping. Narrowing what is displayed MUST NOT narrow what is filtered on.

EVENTS [type_pattern] [clauses…]

3.23.1 The type pattern #

The primary selector is an optional event type pattern, placed immediately after EVENTS.

EVENTS kacs.audit.access.checked   -- exactly that type
EVENTS kacs.*                      -- every type beginning "kacs."
EVENTS *.rejected                  -- every type ending ".rejected"
EVENTS kacs.*.changed              -- kacs.audit.descriptor.changed, kacs.caap.policy.changed, …
EVENTS                             -- every type

* is the only metacharacter, and matches zero or more of any character, dots included. ?, [ and { have no special meaning and a collector MUST NOT treat them as any. Matching folds case, like every string comparison (§3.20).

A pattern with no * is exactly WHERE event.type == "…". A pattern whose only * is trailing is exactly WHERE event.type STARTS_WITH "…". Anything else is a glob.

3.23.2 Origin class aliases #

emitter.class, the origin class, accepts named aliases as well as its integer values:

EVENTS WHERE emitter.class == kacs SINCE 1h ago

These are the only aliased values in the language. A collector MUST accept both forms and MUST treat them as identical.

3.23.3 Aggregation #

Grouping equality, canonical representatives and tie ordering are defined in §3.21. Every aggregation below has a fixed output schema, and rejects SELECT (§3.22).

3.23.3.1 COUNT BY #

Counts records grouped by one field, ordered by count descending.

EVENTS SINCE 24h ago COUNT BY event.type

Output: {<field>: representative, count: <unsigned integer>}.

3.23.3.2 TOP N BY #

COUNT BY with a limit — the N most frequent values.

EVENTS SINCE 1h ago TOP 10 BY emitter.process.guid

Output: the COUNT BY schema.

3.23.3.3 DISTINCT #

The distinct values of one field.

EVENTS SINCE 24h ago DISTINCT event.type

Output: {<field>: representative}.

3.23.3.4 GROUP #

Groups by one or more fields, followed by an aggregation function: COUNT, or SUM, AVG, MIN, MAX with a field argument.

EVENTS SINCE 1h ago GROUP emitter.class COUNT
EVENTS SINCE 1h ago GROUP emitter.class, event.type COUNT
EVENTS SINCE 1h ago GROUP event.type AVG operation.duration

Output, for GROUP a, b:

Group-key fields carry canonical representatives (§3.21).

3.23.3.5 What is aggregated #

For SUM, AVG, MIN and MAX, records whose field is null or non-numeric are excluded from the aggregate — not treated as zero. COUNT counts every record regardless. If no record in a group contributes a numeric value, the group's aggregate is null and the group is still present, because COUNT of it is still meaningful.

3.23.3.6 Result types #

• COUNT returns an unsigned integer.
• SUM over integers returns an integer when the exact mathematical sum fits in signed or unsigned 64 bits. If it does not, or if any input was a float, it returns a float64. If that would be non-finite, the query MUST fail with an error rather than returning an infinity.
• AVG returns a float64 whenever at least one numeric value contributed.
• MIN and MAX return the winning value itself, under exact numeric comparison. When an integer and a float tie, the integer wins.

3.23.4 Ordering #

Without SORT, results are ordered by event.time descending, ties broken as §3.21 requires.

3.23.5 INDEX #

EVENTS INDEX target_sid

INDEX asks the collector to prioritise a field for query acceleration immediately, rather than waiting for it to be observed often enough to be prioritised automatically. It exists for incident response, where the field that suddenly matters has never been queried before.

INDEX is an administrative operation, not a query. It returns no records. A collector MUST check the caller's token against a Security Descriptor governing administration of the collector — one distinct from the read-path descriptors of §3.28 — and MUST refuse a caller that does not hold it. A collector without such a descriptor MUST refuse INDEX outright.

A collector MAY treat INDEX as advisory and MAY decline the request, shed the acceleration later, or do nothing at all. It is a hint about priority; the accelerations a collector maintains are its own business, and a conforming collector that maintains none accepts INDEX and has nothing to do.

There is no command to undo it, because there is nothing to undo: a collector reconsiders its own accelerations continuously and the hint decays with disuse.

LOGS [FROM origin[, origin…]] [ERROR ONLY] [CONTAINING "text"] [clauses…]

Log mode has three primary selectors rather than one, all optional and all combinable. Each is sugar for a WHERE predicate, and each exists because it is the thing a person actually types.

3.24.1 FROM #

Selects by origin. Several may be listed, comma-separated.

LOGS FROM loregd
LOGS FROM loregd, peinit
LOGS

FROM is exactly WHERE origin == "…" for one origin and WHERE origin IN ("…", "…") for several.

Origins are written as identifiers (§3.19) or as quoted strings. An origin that names a producer within a service, such as jellyfin/HealthCheck or jobs/<guid>, is not an identifier and is written quoted (§3.7):

LOGS FROM "jellyfin/ExecStartPre[0]"

3.24.2 ERROR ONLY #

Selects lines that came from standard error.

LOGS ERROR ONLY
LOGS FROM loregd SINCE 1h ago ERROR ONLY

It is exactly WHERE is_error == true, and like every clause it may appear anywhere after LOGS without changing the meaning (§3.18).

3.24.3 CONTAINING #

Selects lines whose message contains the given text — a substring match, folding case like every string comparison (§3.20).

LOGS CONTAINING "connection refused"
LOGS FROM loregd CONTAINING "failed to open"

It is exactly WHERE message CONTAINS "…".

CONTAINING is a log-specific keyword because searching text is the primary operation on log data, and the primary operation deserves the shortest spelling. It is a substring scan, not an indexed text search: a collector MUST NOT restrict what it matches, and combining it with SINCE is what keeps it affordable.

3.24.4 Projection and aggregation #

SELECT narrows non-aggregating results to named log fields, and is additive across clauses (§3.22).

COUNT BY, TOP N BY, DISTINCT and GROUP work exactly as in event mode (§3.23), with the same fixed output schemas, the same result types, and the same prohibition on combining them with SELECT.

LOGS SINCE 1h ago COUNT BY origin
LOGS SINCE 1h ago TOP 5 BY origin

3.24.5 Ordering #

Without SORT, results are ordered by timestamp descending, ties broken as §3.21 requires.

3.24.6 No payload fields #

Log mode has a closed field set (§3.22). A collector MUST reject an unknown log field name as a parse error, in a WHERE, a SORT, a SELECT or a grouping clause alike.

This differs from event mode, where an unknown name is a payload field that resolves to null. The difference is that a log record's shape is fixed and known: a name outside it cannot be a field that this record happens to lack, so treating it as null would answer a question the client did not ask.

METRIC name[label_selector] [transform] [aggregation] [clauses…]

Metric mode evaluates rather than searches. A collector MUST reject SELECT in metric mode: the result schemas are fixed (§3.22).

3.25.1 Selecting series #

The primary selector is a metric name, optionally followed by a label selector in brackets. The name supports * with the same glob semantics as an event type pattern (§3.23).

METRIC cpu.usage
METRIC cpu.*

The brackets — present, absent, or present and empty — decide how multiple matching series are handled, and this is the distinction that governs the rest of the mode.

No brackets — aggregate. Every matching series is combined into one result.

METRIC cpu.usage                    -- average across all cores
METRIC cpu.usage MAX                -- maximum across all cores

Empty brackets — break out. Each series is returned separately.

METRIC cpu.usage[]                  -- latest value per core
METRIC cpu.usage[] SINCE 1h ago     -- a time series per core

Filled brackets — select. Only series matching the label predicates.

METRIC cpu.usage[core="0"]
METRIC cpu.usage[core="0", host="srv1"]
METRIC disk.usage[device STARTS_WITH "sd"]

Label predicates are comma-separated and combined with AND. They use the operators of §3.20, and within a label selector = is accepted as equality alongside ==. Label keys are identifiers; values are identifiers or quoted strings. An absent label resolves to null, so [device IS NULL] selects the series that carry no device label.

3.25.2 Homogeneity #

After the name, the label selector, WHERE predicates and access filtering have been applied, the remaining series MUST be of one type. A collector MUST reject a selection spanning more than one type at execution time, with an error asking for a narrower name or an explicit WHERE type == ….

A selection resolving to zero series returns no records — a successful query with an empty result, not an error.

The rule exists because every function below is defined on one type. A selection mixing counters and gauges has no meaningful rate, and a selection mixing either with histograms has no meaningful value at all.

3.25.3 Function stages #

Function keywords execute in fixed stages regardless of where they were written:

1. Transform — RATE, DELTA, P50, P95 or P99. Operates within each series independently and produces scalars. At most one per query.
2. Terminal aggregation — either a scalar aggregation (AVG, MIN, MAX, SUM) or a window aggregation (AVG_OVER, MIN_OVER, MAX_OVER, SUM_OVER). At most one per query; specifying both a scalar and a window aggregation is a parse error.

The pipeline ordinarily operates on scalars throughout. Counter and gauge samples are already scalar; a histogram sample is not scalar until a percentile function has been applied. The sole exceptional result is a tagged percentile overflow, whose propagation is defined below. A collector MUST therefore reject, at execution time when the type is known, a query that resolves to a histogram series without a percentile function, or that applies RATE, DELTA, or any scalar or window aggregation directly to one.

Every numeric output is a finite binary64. If any computation would produce NaN or an infinity, the query MUST fail with an error rather than returning it. The null in a tagged percentile-overflow result is not a numeric output.

3.25.4 Transforms #

3.25.4.1 RATE and DELTA #

DELTA is the change between consecutive samples; RATE is that change per second. Both apply only to counter series, and a collector MUST reject them on a gauge or histogram at execution time.

Both use the same pair construction. Samples of one series are taken in ascending timestamp order, with a deterministic tiebreaker among samples sharing a timestamp. Each consecutive pair (s1, s2) whose s2 falls inside the effective query range, and where s2 is later than s1, produces one scalar at s2's timestamp. The immediately preceding sample before the first in-range one MUST be used as s1 for the first pair, when such a sample exists — without it the first point of every range would be missing, and a chart would show a notch at the start of every window.

The adjusted delta is s2 - s1 when the value rose, and s2 alone when it fell, because a fall means the counter restarted from zero. RATE is that adjusted delta divided by the elapsed seconds. A pair with non-positive elapsed time contributes nothing.

METRIC http.requests.total SINCE 1h ago RATE
METRIC http.requests.total SINCE 1h ago DELTA

3.25.4.2 P50, P95, P99 #

Percentiles of histogram series only; a collector MUST reject them on a counter or gauge at execution time. Each histogram sample yields one value.

Evaluation is nearest-rank over the sample's cumulative counts: for percentile q, compute rank = ceil(q × total_count), and take the first boundary whose cumulative count is at least rank.

A sample with total_count == 0 yields no record. A sample whose rank falls above the final cumulative count — meaning the percentile lies in the overflow region above the highest boundary — yields a record with value null and overflow true. The distribution proves that the percentile exceeds the highest boundary but does not record a finite value for it. overflow is absent from ordinary percentile results.

METRIC request.duration P95
METRIC request.duration[origin="loregd"] SINCE 1h ago P99

3.25.5 Scalar aggregations #

AVG, MIN, MAX and SUM reduce scalars to one value. They MUST NOT be applied to a histogram series directly.

What they aggregate over depends on the brackets:

• Bracketed, so one result per series: over time, within each series.
• Unbracketed without SINCE: over the latest transformed value of each matching series. The result timestamp is the greatest of the contributing timestamps. A series that cannot produce a value — a RATE with fewer than two samples, say — contributes nothing.
• Unbracketed with SINCE: valid only when the selector resolves to zero or one series. More than one MUST be rejected with an error asking for a window aggregation.

METRIC cpu.usage AVG
METRIC http.requests.total RATE SUM
METRIC cpu.usage[] SINCE 1d ago AVG
METRIC cpu.usage[core="0"] SINCE 1h ago MIN

The unbracketed default aggregation, when no SINCE and no explicit function is given, is AVG. No implicit scalar aggregation is added when a window aggregation is present.

If nothing contributes to an aggregation, the query returns no record for that output group. Otherwise the output timestamp is the greatest contributing timestamp — for RATE and DELTA, the later sample of the contributing pair.

If any contributing percentile result has overflow true, a scalar aggregation MUST conservatively return value null and overflow true. It MUST NOT ignore the result and compute from the remaining finite values.

3.25.5.1 Why unbracketed plus SINCE needs a window #

A collector MUST NOT synthesise a merged time series from samples that do not share timestamps.

Two series sampled at unrelated moments cannot be averaged point by point without inventing values between the points, and interpolation would make the collector responsible for a number nobody measured. A window aggregation supplies the common time grid explicitly, which is why it is required rather than assumed.

3.25.6 Window aggregations #

AVG_OVER, MIN_OVER, MAX_OVER and SUM_OVER take a duration and produce one value per window. They require SINCE; a collector MUST reject a window aggregation without one as a parse error.

Windows are fixed and aligned to Unix-epoch multiples of the duration — not to the query's start — so that the same window boundaries fall in the same places for every query. The output timestamp is the window start. Windows with nothing in them are omitted rather than emitted as null.

METRIC cpu.usage SINCE 1d ago AVG_OVER 1h
METRIC cpu.usage[] SINCE 1d ago AVG_OVER 5m
METRIC http.requests.total SINCE 1h ago RATE SUM_OVER 5m
METRIC request.duration P95 SINCE 1h ago AVG_OVER 5m

AVG and AVG_OVER are different keywords and a collector MUST NOT treat them as synonyms: AVG produces one value for the range, AVG_OVER one per window.

A collector MAY cache aligned, complete window results as an implementation optimisation. Such a cache MUST be transparent: freshness must be proved from the authoritative samples before reuse, partial windows must retain their ordinary semantics, and a missing, stale or disabled cache MUST produce exactly the same result as direct evaluation.

For raw and percentile-transformed values, a window contains the scalars whose timestamps fall inside it, and the function is applied to those. No interpolation is performed.

If a window contains a percentile result with overflow true, its output MUST also carry value null and overflow true. This propagation applies again when per-series window results are combined.

For RATE and DELTA with a window aggregation, each series first produces at most one scalar per window: the window DELTA is the sum of reset-adjusted deltas for pairs whose later sample is in the window, and the window RATE is that divided by the elapsed seconds those pairs covered. The preceding-sample rule applies to the first pair of each window. The terminal aggregation then combines the per-series window values — so RATE SUM_OVER 5m sums the series' five-minute rates, and RATE AVG_OVER 5m averages them. Where the selector resolves to exactly one series, all four window functions return that series' window value.

Bracketed window queries keep labels in the result rows. Unbracketed ones omit them, unless the selector resolved to exactly one series.

3.25.7 Without SINCE #

With no SINCE, the query returns the latest value.

METRIC cpu.usage[core="0"]
METRIC cpu.usage[]
METRIC cpu.usage

"Latest" is per series, by timestamp with the deterministic tiebreaker. For RATE and DELTA, it is the latest valid consecutive pair with positive elapsed time; a series with no such pair returns nothing.

3.25.8 Boot filtering #

Samples carry boot_id but series continue across boots (§3.13). A query MAY restrict to one boot:

METRIC cpu.usage[] WHERE boot_id == "550e8400-e29b-41d4-a716-446655440000"

A boot-filtered metric query MUST be evaluated from raw samples. A collector MUST NOT serve one from any pre-computed aggregate that is not itself partitioned by boot.

3.25.9 Results #

One record per raw sample; one per valid pair for RATE and DELTA, timestamped at the later sample; one per non-empty histogram sample after a percentile transform, including an explicit overflow record; one per window for window aggregations; one for a scalar aggregation.

A finite histogram result carries only the percentile in value. An overflow result carries value null and overflow true. The boundaries, counts, total and sum are not returned by the query language in this revision, in any mode.

{timestamp: 1714000000000000000, boot_id: "{550e8400-…}", name: "cpu.usage", type: "gauge", core: "0", value: 42.7}
{timestamp: 1714000300000000000, name: "cpu.usage", type: "gauge", core: "0", value: 39.8}

Without SORT, metric results are ordered by timestamp ascending (§3.21) — the opposite of events and logs, because a metric result is read as a series rather than as a list of occurrences.

A cross-type filter narrows one data type by a condition on another. It is the only correlation mechanism in the language; there is no join.

EVENTS kacs.* SINCE 1h ago WHERE METRIC cpu.usage[core="0"] > 80
LOGS FROM loregd SINCE 1h ago WHERE EVENT kacs.audit.access.checked EXISTS
METRIC cpu.usage[] SINCE 1h ago WHERE EVENT eventd.store.quarantined EXISTS
EVENTS kacs.* SINCE 1h ago WHERE LOG loregd CONTAINING "error" EXISTS

3.26.1 How it is evaluated #

A collector MUST evaluate the cross-type condition first, producing the set of time ranges over which it holds, and then apply those ranges as additional timestamp bounds on the primary source.

The condition is evaluated against the referenced data's own resolution — the metric's sample interval, or the density of matching events — and not once per record of the primary source. It is computed once for the query.

3.26.2 Metric conditions #

WHERE METRIC operates on raw scalar samples of counter and gauge series only. Transform, scalar aggregation and window aggregation keywords are not valid in one, and a condition resolving to a histogram series MUST be rejected.

The selector MUST resolve to zero or one series. Zero produces no true ranges. More than one MUST be rejected with an error asking for a bracketed or narrower selector.

Within the effective query range, a sample's value is treated as active over [sample.timestamp, next_sample.timestamp), clipped to the range, and the final sample stays active through the upper bound. A collector MUST include the latest sample before SINCE as the initial state when one exists; without it the condition would be false from the start of every range until the first sample inside it, which for a fifteen-second sampling interval is fifteen seconds of wrongly excluded records. If no earlier sample exists, the condition is false until the first in-range sample.

Samples sharing a timestamp are ordered deterministically; the earlier ones create zero-width intervals and the last at that timestamp is the active value.

This is interpolation of a kind, and it should be understood as such: it assumes the condition held continuously between two samples. A metric that crossed a threshold and crossed back between samples is invisible.

3.26.3 Existence conditions #

WHERE EVENT … EXISTS and WHERE LOG … EXISTS are true when at least one matching record lies near the primary record in time. The event type supports * globbing (§3.23); the log form names an origin and optionally a CONTAINING text.

"Near" is a centred half-open window of a configured width W. With lower = floor(W / 2) and upper = W - lower, the condition is true for a primary timestamp t when a matching record exists with:

timestamp >= t - lower
timestamp <  t + upper

Equivalently, a matching record at e contributes the true range [e - lower, e + upper). When W is odd the extra nanosecond falls on the upper side, so the width is exactly W and never W ± 1.

3.26.4 The lookback limit #

A collector MUST bound how far back a cross-type filter may scan.

If the effective query range exceeds the limit, a collector MUST reject the cross-type filter with an error saying the range is too large, and the error SHOULD suggest narrowing it with SINCE or UNTIL.

A query with a cross-type filter and no SINCE MUST be rejected. An unbounded cross-type scan is never permitted, in any mode, at any configured limit.

The reason is that a cross-type filter reads a second store in full before the first query begins. Its cost is set by the referenced data's density, which the client did not select and cannot see, so a query that looks cheap can scan a hundred times more than it returns.

3.26.5 Cost #

A cross-type filter is efficient when it is selective — narrow true ranges eliminating most of the primary source — and expensive when it is broadly true, which is the case where it also eliminates nothing. A condition that holds across the whole range costs the full scan of both stores and returns exactly what the query would have returned without it.

STREAM turns a query into a live tail. It is a flag, may appear anywhere in the string, and takes no argument.

Streaming is available for event and log queries only. A collector MUST reject STREAM in metric mode.

3.27.1 The shape of a streaming query #

1. The collector executes the query normally and sends the initial result set as "ok" messages.
2. It sends "watch" (§3.16). The query is established at this point and not before.
3. It stays open. As records are committed, it evaluates them against the query and sends those that match.
4. It continues until the client disconnects, an error terminates it, or the collector shuts down.

There is no "end" message for a streaming query, ever.

3.27.2 What may be streamed #

Raw record queries and DISTINCT queries. A collector MUST reject STREAM combined with COUNT BY, TOP N BY or GROUP as a parse error — those produce one answer about a set, and a set that is still growing has no answer yet.

A collector MUST reject STREAM combined with UNTIL. An upper time bound and an unbounded live tail are contradictory requests.

SINCE is permitted and applies to both phases, resolved against the evaluation time captured at query start (§3.19).

3.27.3 What still applies during the watch phase #

Access control, the primary selector, the SINCE bound and every WHERE predicate — cross-type conditions included — are evaluated against each new record.

SORT, TAKE and SKIP apply to the initial result set only. Streamed records are delivered in commit order and a collector MUST NOT reorder, limit or skip them: there is no total order over records that have not arrived, and applying TAKE to a stream would silently end it.

SELECT applies to streamed records as it does to initial ones.

3.27.4 DISTINCT streaming #

EVENTS kacs.* DISTINCT emitter.process.guid STREAM
LOGS DISTINCT origin STREAM

A DISTINCT stream emits a value the first time it is seen, and never again. The output schema is DISTINCT's fixed one (§3.23) in both phases.

The initial result set is the complete distinct set visible at query start, after access control and every filter. The collector then holds a seen set initialised from it. Each newly committed record that passes access control and the filters is reduced to its value for the field, and emitted only if that value is not already in the seen set under the grouping equality of §3.21; emitted values are then added.

A collector MUST bound the seen set.

If initialising the set or inserting a value would exceed the bound, the collector MUST terminate the query with an error. It MUST NOT evict: "not seen before" is the entire meaning of the output, and a set that forgets would re-emit values it had already reported, which is worse than stopping.

A collector MUST reject DISTINCT … STREAM combined with SORT, TAKE or SKIP, so that the seen set always corresponds to the complete initial visible set. SELECT is already invalid with DISTINCT (§3.22).

3.27.5 Cross-type conditions during the watch phase #

The pre-computed time ranges of §3.26 describe the past. A collector MUST NOT reuse them for streamed records.

For a metric condition, the selector has already been required to resolve to exactly one series (§3.26). For each committed batch, the collector finds that series' active sample at the batch's latest candidate timestamp under §3.26's interval rules and evaluates the condition against it. If no sample is active there, the condition is false. A false condition filters out the whole batch; a true one leaves the batch to be filtered by the remaining predicates as usual.

For an existence condition, the collector applies §3.26's centred window to each candidate record's own timestamp. These are evaluated per record, not per batch, because a matching record may be near some of a batch and not the rest.

3.27.6 Backpressure #

If a client cannot keep up, the collector MUST drop the query rather than buffer for it.

Backpressure is detected on the socket send buffer: when a result message cannot be sent because the buffer is full, the collector MUST terminate the query immediately and MUST NOT block on the send. It sends an error if the socket will still take one, and closes otherwise.

Streaming MUST NOT slow or block ingestion. A streaming client is the lowest-priority consumer of a collector's time, and a slow one is disconnected rather than accommodated — the same principle as §3.4, applied on the way out.

3.27.7 Latency #

Delivery latency is bounded below by the collector's commit interval for the store concerned, because a record is only streamable once it is committed. A client that needs lower latency than that is not served by this interface: the KMES ring buffer is the lower-latency path and is specified in PSPK.

A collector MUST enforce read access on every query, against the token captured when the client connected (§3.14).

How it does so is its own design, and the mechanism the mainline collector uses is described in the eventd TRMP. What this chapter fixes is the part a client can observe: which results it gets, and what it is told about the ones it does not.

3.28.1 The unit of access is the concrete identifier #

Access is resolved per concrete identifier — the event type, log origin or metric name a stored record actually carries (§3.2) — and not per query, per store, or per pattern the query happened to write.

A collector MUST resolve each identifier that a query's data could touch independently. A broad selector authorizes nothing by itself: EVENTS with no pattern, EVENTS kacs.*, LOGS with no FROM, and METRIC cpu.* are all resolved identifier by identifier, and a client permitted to read one matching identifier and not another sees only the first.

Identifiers are matched to rules by dot-delimited prefix, most specific first, falling back to a wildcard default: for kacs.audit.access.checked, a rule for kacs.audit.access.checked, then one for kacs.audit.access, then kacs.audit, then kacs, then the default.

A collector MUST fail closed. If no rule resolves — including because the default is missing — access is denied.

3.28.2 Filtering is silent #

Records and fields removed by access control are removed without comment. A collector MUST NOT indicate in a response that anything was withheld, and a client MUST NOT assume a result set is complete.

The consequences are precise and a client needs all of them:

• A record whose identifier the client may not read is absent, not redacted.
• A field the client may not read is absent from the record, and is indistinguishable from a field the record never carried (§3.17).
• COUNT, COUNT BY, TOP N BY, DISTINCT and every aggregation reflect only authorized records. A count is a count of what the client may see.
• A cross-type condition referencing data the client may not read evaluates as though no matching data exists (§3.26). It does not fail the query.
• TAKE and SKIP page over the authorized records only.

3.28.3 Access control runs before everything #

A collector MUST remove unauthorized records from the logical row set before predicates, transforms, grouping, aggregation, sorting, pagination and projection (§3.18).

This is not tidiness. Counting, ordering or paginating over records a client may not read leaks them through the count, through the ordering, and through the gaps in pagination — a client could establish how many records of a type it cannot read exist, and roughly when, without ever seeing one.

A collector MAY reach the result however it likes: pushing the authorization down into its storage engine, or reading candidates and discarding them before aggregating. What it MUST NOT do is produce a different answer from the one filtering-first produces.

3.28.4 Denied fields do not fail the query #

When a query references a field in a predicate, a grouping, a sort or an aggregation, and some matching identifier does not grant that field, the records under that identifier contribute nothing — exactly as if their identifier had been denied outright.

A collector MUST NOT reject the query.

Authorization for a field is resolved from the field as written, against each concrete identifier, and does not depend on whether any record of that identifier actually carries it. Payload fields vary between records of the same type, so a rule that turned on presence would be undecidable before the scan it was meant to authorize.

3.28.5 What is not a field #

Derived aggregate outputs — count, sum, avg, min, max — are not source fields, have no access identity of their own, and are visible whenever the client is authorized for the records and the source fields they were computed from.

Values internal to preserving query semantics — row identifiers, series identifiers, ordering tiebreakers, series type checks — are likewise not query-language fields (§3.21). A metric result's value is a source field, because it is a raw sample or a scalar derived from raw samples.

3.28.6 Errors say nothing #

A collector MUST NOT include a value the client is not authorized to read in any error message (§3.16), including in errors raised by internal consistency checks.

3.28.7 Streaming #

Access decisions made for the initial result set are reused during the watch phase, but a collector MUST resolve any new concrete identifier that appears in a streamed batch and check it before using the record or its distinct value — a new event type or a new log origin appearing mid-stream has never been authorized.

If a rule changes during a streaming query, a collector MUST re-check subsequent batches against the new rule.

The token does not change. It was captured at connection (§3.14), so a client whose group memberships change mid-stream continues to be evaluated against what it connected with, and a client whose access is revoked keeps receiving records until it disconnects.

3.28.8 What write authentication proves #

Log origin is broker-attested. The service manager chooses it from the pipe being read, and the log socket excludes service processes (§3.6). It proves which service context the manager associated with the line; it does not prove that the line's text is true.

Metric name is publisher-authorized. KACS attests the effective token that sent the datagram and the collector verifies EVENTD_PUBLISH for that name (§3.9). Labels and values remain producer statements. A publisher authorized for http.requests can report any value and any valid label set for that name.

Read authorization is independent. Permission to publish an identifier does not imply permission to query it, and permission to query it does not imply permission to publish it.

There is no version number on any of the three interfaces. No datagram carries one, no query message carries one, and there is no exchange in which either party could state or discover what the other speaks.

That is a deliberate consequence of the shapes chosen, and it is worth being explicit about, because it means every rule below is the only mechanism available.

Ingestion is one-way over a datagram socket: there is no reply in which a collector could announce a version and no state in which a producer could remember one. The query channel could carry a version — it is a stream, and it has a request message — and does not, because a version field is only useful if a party may then behave differently, and a client cannot usefully vary: it either asks a question the collector understands or does not.

What replaces negotiation is a set of rules under which both sides may change without either being told.

3.29.1 Unknown fields are ignored #

A collector MUST ignore fields it does not recognise in a log record (§3.7), in a metric record (§3.11), and in a query request (§3.15).

This is what allows a field to be added. A producer built against a later revision may send a field this collector has never heard of, and the record is still stored; a producer built against an earlier one omits a field that has since been added, and the record is still stored because everything added is optional.

A field added to any of these three maps MUST therefore be optional, and a collector MUST NOT require one to be present.

3.29.2 Unknown values are refused, not ignored #

The rule does not extend to values.

An unrecognised type in a metric record discards the record (§3.12); a first token that is not a mode fails the query (§3.18); an unrecognised keyword is a parse error. A collector MUST NOT guess at an unrecognised value, and MUST NOT skip a field it recognised but could not interpret.

The asymmetry is the point. An unknown field is something the sender knows about and this collector does not, and ignoring it loses only what was never understood. An unknown value in a known field is the sender saying something specific about this record, and proceeding without understanding it stores something other than what was sent.

3.29.3 Response statuses #

A client MUST treat a response whose status it does not recognise as an error terminating the query, and MUST discard the "ok" messages it has received for that query unless "end" or "watch" had already arrived (§3.16).

A status is the control flow of the response stream, so there is no ignoring one: a client that skipped an unknown status would be waiting for a terminal message that had already been sent, or treating an incomplete result as complete. Failing is the only safe reading.

A collector MUST NOT introduce a new status for a condition that the four existing ones can express.

3.29.4 What may change without notice #

• New optional fields in a log record, a metric record or a query request.
• New fields in result records. A client MUST tolerate a key it does not recognise, and MUST NOT reject a record for carrying one.
• New query keywords, clauses and functions. A client sending one the collector does not know receives a parse error, which is the correct answer.
• Wording of any error string (§3.16).
• New event types, log origins and metric names. These are data, not interface; nothing enumerates the valid set of any of them.

3.29.5 What may not change #

• The meaning of an existing field, in either direction. A field is added or it is left alone.
• The type of an existing field.
• The four response statuses, or the rule that exactly one terminal message ends a query.
• The framing of §3.15, which has no version field and therefore no way to change compatibly.
• A required field becoming optional, or an optional one becoming required.

3.29.6 Limits are not the interface #

The declared bounds — the datagram ceilings above their portable floor (§3.6, §3.9), the query request ceiling and response target (§3.15), the concurrency and timeout bounds (§3.14, §3.16), the existence window and lookback limit (§3.26) — are configuration, and an administrator may change any of them within its stated range.

A collector MUST behave identically at any value in its supported range. A producer or client MUST NOT infer a bound from having exceeded one, or from not having exceeded one, and MUST NOT depend on the mainline defaults quoted in this chapter.

The 262144-byte log and metric datagram floor is different: it is part of the interface. A collector may raise its declared ceiling but MUST NOT lower it below that floor, and a portable producer does not discover or depend on the raised value (§3.6).

A conforming implementation of any role MUST satisfy every requirement in this chapter. This section collects the obligations that are not tied to one message.

3.30.1 A collector #

Serve three separate channels. Two SOCK_DGRAM for ingestion, one SOCK_STREAM for queries, each on its own socket, each protected by a Security Descriptor established before it accepts anything (§3.3).

Never exert backpressure. No producer stalls because of a collector, under any load, in any failure state (§3.4).

Never react to input. No event, no log entry, no client-observable counter, in response to a malformed, unwanted or excessive submission (§3.4).

Authenticate ingestion. Admit only the service-manager broker on the log socket; require a complete KACS token on every metric datagram and authorize every metric name for EVENTD_PUBLISH (§3.3, §3.6, §3.9).

Validate at the stated scope. Datagram, record, or field — as §3.8 and §3.12 set out, and no more broadly. In particular a malformed record MUST NOT cost the valid records batched with it, and a malformed optional field MUST NOT cost a log record.

Store what you were given. A log message byte-for-byte, an event payload unmodified, a timestamp uncorrected, a histogram's boundaries in the order sent (§3.8, §3.10, §3.5).

Preserve gaps. No interpolation, no backfill, no synthesised sample (§3.13).

Identify every query client from the connection, before executing anything, and refuse the query if you cannot (§3.14).

Order totally and deterministically. Every result, for a fixed set of stored records, in one order — so that SKIP and TAKE mean something (§3.21).

Use query-language semantics, not your storage engine's, for every comparison, ordering, grouping and equality test the language defines (§3.20, §3.21).

Enforce access before you compute, per concrete identifier, failing closed, and silently (§3.28).

Bound everything a client can consume: concurrent queries, streaming queries, message size, query time, distinct-stream values, cross-type lookback (§3.14, §3.15, §3.16, §3.26, §3.27).

Behave identically across your configured ranges (§3.29).

3.30.2 A producer #

Send well-formed records and accept that malformed ones vanish without notice (§3.8, §3.12).

Stay within the 262144-byte portable datagram ceiling, batched or not. Do not add discovery or depend on a collector's larger local setting (§3.6).

Choose a stable, conforming metric name. It must match the identifier grammar, name the producer distinguishably and use dots for hierarchy, because publication and read rules are written against it and queries select on it (§3.10).

Convey metric identity. Enable KACS_SO_PASS_TOKEN before sending metrics and retain the sending socket so the captured-token and authorization caches remain effective (§3.9).

Timestamp at production, not at submission (§3.7).

Bound your label cardinality, and keep histogram boundaries fixed for the life of a metric (§3.10, §3.13).

Never assume delivery. No acknowledgement exists, none is coming, and a record that mattered should have been an event (§3.4).

Never change a metric's type. Doing so ends the series silently and permanently (§3.10).

3.30.3 A client #

Tolerate unknown keys in result records, and unknown statuses as errors (§3.29).

Discard partial results. An error before "end" or "watch" means every "ok" message for that query is void (§3.16).

Assume nothing about completeness. Results are silently filtered by access, counts count only what you may see, and an absent field is indistinguishable from a denied one (§3.28).

Interpret provenance narrowly. A log origin is broker-attested and a metric name is publisher-authorized; neither proves the message, value or labels are truthful (§3.28).

Do not parse error strings (§3.16).

Open one connection per query (§3.14).

3.30.4 What this chapter does not require of a collector #

A conforming collector need not accelerate anything, pre-compute anything, shard anything, or retain anything for any particular period. It need not honour INDEX beyond accepting it (§3.23). Its storage, indexing, retention and query planning are entirely its own, and every requirement above is stated about the answer rather than about how the answer is reached.

Every bound this chapter requires a collector to enforce, with the value and adjustable range of the mainline collector. The mainline values are informative: a conforming collector chooses its own, and a producer or client MUST NOT depend on any of them (§3.29).

The mainline configuration key names are those of eventd, whose configuration is catalogued in the eventd TRMP §A.

3.A.1 Ingestion #

3.A.2 Queries #

3.A.3 Cross-type filtering #

3.A.4 Fixed by this chapter #

These are not configuration and a collector MUST NOT vary them.

The response target is not a ceiling. A complete record larger than the target travels alone, so the configurable ingestion ceilings do not need to be coupled to it (§3.15).

An index of the language, and of where each construct is valid. The normative definitions are in §3.18 to §3.27; nothing here adds a rule.

3.B.1 Shape #

EVENTS [type_pattern]                             [clauses…]
LOGS   [FROM o[, o…]] [ERROR ONLY] [CONTAINING s] [clauses…]
METRIC name[label_selector] [transform] [aggregation] [clauses…]

3.B.2 Clause validity #

WHERE and SELECT are the only repeatable clauses (§3.18).

3.B.3 Combinations that are rejected #

"Parse" failures need no data. "Execution" failures depend on what the store holds, so the same query string may succeed on one system and fail on another.

3.B.4 Metric functions #

Transforms feed aggregations; a query may have at most one of each (§3.25).

3.B.5 Operators #

== != > >= < <= STARTS_WITH ENDS_WITH CONTAINS HAS IN NOT_IN IS NULL IS NOT NULL, combined with AND and OR (§3.20).

CONTAINS is substring on a string; HAS is membership in an array.

There is no NOT and no =.

3.B.6 Literals #

3.B.7 Fields #

3.B.8 Aliases #

emitter.class accepts userspace (0), kmes (1), kacs (2), lcs (3). These are the only aliased values in the language (§3.23).

3.B.9 Default ordering #

All ties are broken to a total order (§3.21).

The three interfaces here are not novel, and each has a well-known counterpart whose shape informed it. What follows compares the contracts — this appendix is about wire shapes and the obligations they place on either side. The eventd TRMP §1.4 compares the systems.

3.C.1 Log ingestion #

The closest relative is journald's native socket: a Unix datagram socket, world-writable, accepting a self-describing record from any local process, with no acknowledgement and no notification of loss. The agreements are substantive — datagram rather than stream, silent drop under pressure, and a forwarder bridging programs that only know standard output. The identity model differs: this interface admits only the service-manager broker, which assigns the origin from the pipe it read rather than accepting a process's self-assertion.

The differences are three. The record here is MessagePack rather than a line-oriented key-value text format, because the collector already carries a MessagePack decoder for event payloads and a second parser would be a second thing to get wrong. Severity is a boolean rather than a syslog priority, because a forwarder can distinguish two file descriptors and inventing eight levels from two would be a guess presented as data (§3.7). And a batch is a first-class datagram shape rather than a stream of records, which is what lets a forwarder amortise the syscall without giving up the datagram's all-or-nothing property.

Classic syslog over /dev/log is the older relative, and the departure from it is the same one journald made: a record with named fields rather than a formatted line that every consumer re-parses with a regular expression.

3.C.2 Metric ingestion #

The shape is StatsD's: push, datagram, fire-and-forget, no series registration, sender-named series. KACS sender identity and per-name publication authorization are deliberate additions. It is the opposite of Prometheus's, where the collector pulls from endpoints it has been configured to know about.

The choice follows from the loss model rather than from taste (§3.9). A pulling collector must reach every producer on a schedule, which makes it responsible for their availability; pushing keeps a slow or dead producer invisible except for the gap it leaves.

What is taken from the Prometheus data model rather than from StatsD is the identity of a series: a name plus a set of labels, with each distinct label combination a distinct series, and the cardinality warning that comes with it (§3.10). The histogram is Prometheus's cumulative-bucket form, including the property that the top bucket is an overflow whose contents are counted but not located.

Two things are deliberately absent. There is no text exposition format, because nothing scrapes. And there is no summary type — a producer that has already computed its own quantiles cannot submit them, because quantiles do not aggregate and a stored one could not be combined with another (§3.25).

3.C.3 The query interface #

The unusual choice here is having a query language at all.

journald exposes a cursor and a set of field matchers, and computation belongs to the client. The Windows Event Log exposes XPath over an XML representation. Prometheus exposes PromQL, a genuine language, but only for metrics. This interface puts one language over all three data types, with a shared clause vocabulary and per-type modes (§3.18).

The reason is access control. Filtering, grouping and aggregation must happen on the side that knows what the caller may see, because a count computed by a client is a count of what the client was given and a count computed by the collector can be a count of what the client is entitled to (§3.28). A cursor interface pushes the computation across the trust boundary and takes the enforcement point with it.

The framing — a length-prefixed MessagePack request, a sequence of chunked result messages, one terminal message — is unremarkable and deliberately so. What it does not have is more interesting: no version field (§3.29), no error codes (§3.16), no multiplexing (§3.14), and no cursor. A query is one connection, and paging is SKIP and TAKE over a total order (§3.21) rather than an opaque token the collector must keep state for.

3.C.4 Where these interfaces sit #

This chapter defines the two interfaces a Peios service manager offers: the control channel, by which a program manages services, and the notification channel, by which a supervised service reports on itself.

Both are Unix-domain sockets between userspace parties, and both have a publicly implementable side. A monitoring tool, an orchestration agent, a shell utility, or a privileged action broker implements the client side of the control channel. Every supervised service that reports readiness, sends keepalives, or preserves file descriptors across a restart implements the producer side of the notification channel.

4.1.1 The roles #

The manager is the process that supervises services. It listens on both channels. On Peios this is peinit, running as PID 1, but nothing here depends on that beyond the manager being a single process holding both sockets.

A client connects to the control channel to issue commands and read answers. A client is any process; it holds no special relationship with the manager beyond the one its token establishes.

A service is a process the manager started, and speaks the notification channel about itself. A service does not connect to the control channel in that capacity — a program that does both is acting in two roles.

A job the manager started on a submitter's behalf (§7) speaks the notification channel exactly as a service does, and is observed and stopped from the control channel by the job commands of §4.14. The channel by which it is submitted is §7's.

Requirements are stated against the role, not the program.

4.1.2 What this chapter covers #

• the two channels, their addressing, and how each is reached
• message framing and encoding on both
• how a client's identity is established, and how a command is authorised
• the command set, the response shapes, and the error vocabulary
• what a command does to a service in each of its states
• how a service's notification is authenticated, and what a service may say
• the file-descriptor store
• the rules under which either channel may be extended
• the conformance requirements for each role

4.1.3 What this chapter does not cover #

• How the manager supervises anything. Dependency resolution, restart policy, timers, cgroups, the boot sequence and shutdown are the manager's own design. This chapter defines what a client can ask for and what it is told, not how the answer comes about.
• How service definitions are expressed. On Peios they are registry keys, administered like any other registry data. That is the service manager's own design.
• What a service state means. The vocabulary is fixed here (§4.B) because it appears on the wire; what causes a service to be in one of those states is not.
• Kernel interfaces. Establishing a peer's identity and evaluating an access decision are kernel operations, specified in PSPK and in the kernel's own reference manual.
• Submitting a job. The channel by which a process asks the manager to run a program under supervision, the identity such a job runs as, and the commands its submitter issues about it are §7. This chapter covers only what a control-channel client sees of such a job.

Service. A named unit of execution the manager supervises. Service names are opaque to this chapter except for the character restriction in §4.8.

Job. One process execution. A service that has been restarted has had more than one job. A job submitted on the jobs channel (§7) has no service; it belongs to its submitter.

Operation. A requested state machine action on a service, with an identity and a lifecycle of its own. Lifecycle commands do not act directly; they create operations, and an operation is what a client observes and waits on.

Activation generation. A counter the manager increments each time a service begins starting. It distinguishes one incarnation of a service from the next.

Right. A named permission on a service or on the manager itself, represented as a bit in an access mask and evaluated against a Security Descriptor. §4.7.

Dependent-satisfying state. A service state in which the services that depend on the service may proceed. Which states these are is the manager's design; that a state is or is not one of them is observable through the state vocabulary.

Terminal state. For an operation, one of completed, failed, cancelled, merged or aborted. An operation in a terminal state does not change again.

Frame. One newline-terminated line on the control channel, carrying exactly one JSON object.

Datagram. One message on the notification channel, carrying zero or more KEY=VALUE lines and optionally file descriptors.

The two channels differ in almost every respect, and the differences are deliberate.

4.3.1 Why the notification channel is a datagram socket #

A service reporting on itself must not be able to block the manager, and must not block itself. A stream socket gives both parties a queue that fills, and a service writing into a full queue either blocks — hanging a service on the manager's scheduling — or gets an error it has to handle in the middle of doing something else.

A datagram socket has neither problem. A send either goes or is dropped, and the manager can drain at whatever rate it manages. The cost is that a notification can be lost, which is why nothing in §4.19 is a transaction: every field is either idempotent or a statement of current condition, and a service that needs a lost keepalive to have arrived sends another one.

4.3.2 Why the control channel is a stream socket #

A command has an answer, and a client waiting for one needs to know it did not arrive rather than assuming. It also needs framing: a request can be large, and a response certainly can.

4.3.3 Reaching either socket #

Both sockets are protected by the Security Descriptor on the socket's own inode, and a party that may not reach the socket is refused when it connects or sends, before any content is exchanged.

The manager MUST NOT rely on POSIX mode bits for this. On a Peios system access to a filesystem object is routed through its Security Descriptor, mode bits are not consulted, and a chmod on either socket has no effect whatever.

The manager MUST ensure that each socket, and each directory containing one, carries a Security Descriptor that admits the parties intended to use it. A socket created where nothing inheritable applies acquires no descriptor, and an object with no descriptor is denied to every caller — so a manager that leaves this to chance produces a socket nobody can reach, including principals its own default policy grants access to.

The manager MUST listen on a Unix SOCK_STREAM socket at a well-known path. On Peios that path is:

/run/services/peinit/control.sock

The socket MUST exist for as long as the manager is serving, and the manager MUST unlink it when it stops.

The manager MUST create the listening socket and every accepted connection with close-on-exec set, so that no connection descriptor is inherited by a process the manager starts.

4.4.1 A connection #

A client connects, issues one or more commands, and closes. The manager MUST NOT require a client to issue any command before another, and MUST NOT hold state across connections: a connection carries an identity (§4.6) and nothing else.

Requests on one connection MUST be answered in the order they were received. The manager MAY read no further frames from a connection while a response on it is outstanding.

4.4.2 Limits #

The manager MUST enforce three limits, and MUST make their values discoverable to an administrator through the same configuration surface that sets them. The values a Peios service manager uses by default are in §4.A.

Concurrent connections. A connection accepted while the manager is already at its limit MUST be closed at the socket level, without a response. There is no error code for this condition: the manager has declined to enter the protocol at all, and a client MUST treat an immediate close with no response as a refusal rather than as a protocol error.

The limit MUST be applied to each caller, counted by the user SID of the peer's token, as well as to the connections in all: a connection accepted while its caller already holds its limit is closed the same way. The manager MAY exempt the machine's own principal (SYSTEM) from the per-caller limit. A channel that admits every authenticated principal and bounds only the total lets any one of them fill it and shut everyone else out, administrators included.

Request size. A request frame whose content exceeds the limit MUST be answered with REQUEST_TOO_LARGE and the connection MUST then be closed. The limit applies to the frame's content and MUST NOT count the terminating newline, so a request of exactly the limit plus its newline is within bounds.

Idle timeout. A connection with no request outstanding MAY be closed once it has been idle for the configured period. The manager MUST NOT treat a connection as idle while a request on it is outstanding — in particular a connection blocked on a wait=true operation (§4.13) is not idle, however long the operation runs, and MUST be held open until the operation resolves. Such a connection is bounded by the operation's own timeout, not by the idle timeout.

A connection closed for idleness MUST be closed without a response.

4.5.1 Frames #

Every message in both directions is one frame: a single JSON object, serialised compactly, followed by one 0x0A byte. This applies to requests and to responses alike, and the manager MUST terminate every response with a newline.

Framing is byte-oriented and is performed before any JSON is parsed. A 0x0A byte ends the frame wherever it appears, so a raw newline inside what a sender intended as a JSON string does not produce one frame with an embedded newline — it produces two malformed ones. (A raw 0x0A inside a JSON string is not valid JSON in any case; the \n escape sequence is unaffected and is the way to carry a newline in a value.)

The manager MUST NOT emit pretty-printed JSON, and MUST NOT emit more than one object per frame.

4.5.2 Encoding #

Frames are UTF-8. The manager MUST reject a frame that is not well-formed UTF-8 with MALFORMED_REQUEST.

4.5.3 What is malformed #

The manager MUST answer with MALFORMED_REQUEST when a frame:

• is empty — a bare newline with no content;
• is not well-formed UTF-8;
• is not valid JSON;
• is valid JSON but not an object. An array, a string, a number, true, false and null are all malformed requests.

4.5.4 Closing after an error #

The manager MUST distinguish two classes of failure, because they say different things about the connection.

A frame-level failure means the manager cannot trust the stream's framing any more: it does not know where the next frame begins. MALFORMED_REQUEST for an empty frame and REQUEST_TOO_LARGE are both frame-level. The manager MUST send the error response, discard any buffered input, and close the connection.

A command-level failure means the frame was well-formed and the command in it could not be carried out: unparseable JSON content, an unknown command, missing arguments, a denied access check, an unknown service. The manager MUST send the error response and MUST keep the connection open.

A client MUST NOT assume a connection survives an error response, and MUST be prepared for either.

4.5.5 Timestamps #

Every timestamp field the manager emits MUST be a UTC RFC 3339 string with exactly nine fractional-second digits and the literal offset marker Z:

"2026-06-01T12:34:56.123456789Z"

The manager MUST NOT emit a numeric offset in place of Z, and MUST NOT vary the number of fractional digits.

These are wall-clock instants, presented for a reader. The manager MUST NOT derive elapsed-time decisions — timeouts, retries, ordering — from wall-clock differences, and a client MUST NOT assume that two timestamps in the same response were taken from a clock that did not move between them.

The manager MUST establish the identity of every client from the kernel. There is no credential exchange in this protocol, and a client MUST NOT be able to assert who it is.

4.6.1 Obtaining the identity #

On accepting a connection, the manager MUST obtain the peer's token through the kernel's peer-identity facility for connected local sockets (the Peios Kernel TRM §3.5). The token is captured by the kernel when the client connects; nothing in this protocol carries it, and a client has no way to supply one.

The token reflects the identity the client was acting under when it connected. A client that is impersonating another principal is therefore captured as the principal it is impersonating, not as its own service identity — which is the intended behaviour: access decisions reflect the identity a client is actually acting under.

4.6.2 When it is captured #

The manager MUST capture the identity once, when the connection is accepted, and MUST use that identity for every command on the connection.

A client MUST NOT expect a change of identity mid-connection to affect authorisation. A client that needs to act under a different identity MUST open a new connection.

4.6.3 Failure #

If the manager cannot obtain the peer's identity, it MUST close the connection without a response. There is no error code, because the manager has no basis on which to decide whether this caller may be told anything at all.

A client MUST treat an immediate close with no response as a refusal. This is the same observable outcome as exceeding the connection limit (§4.4), and a client cannot distinguish the two — deliberately, since distinguishing them would tell an unauthenticated caller about the manager's state.

4.6.4 The manager's identity #

The kernel's facility is symmetric: a client that has connected MAY obtain the manager's identity through the same facility and verify it before sending a command — that the peer is the principal it expects the manager to run as, rather than something else that bound the path. A client MUST NOT treat the control socket's path as proof of the manager's identity; the path is a rendezvous, and the token is what the kernel attests.

4.6.5 The identity is not a UID #

The manager MUST NOT use the peer's UID or GID as an authorisation input. Identity on a Peios system is a token, and the token is what the kernel attests.

Every command is authorised against a Security Descriptor, using the peer's token. There is no command the manager performs without a check, and no principal exempt from one.

4.7.1 Rights #

Commands acting on a service are checked against that service's descriptor:

Commands acting on a submitted job are checked against that job's own descriptor, with the rights and generic mapping §7.8 defines: JOB_QUERY (0x0001), JOB_STOP (0x0002), JOB_SIGNAL (0x0004).

Commands acting on the system are checked against the manager's own descriptor:

4.7.2 Generic mappings #

The manager MUST use these generic mappings when evaluating a descriptor, so that a descriptor written in generic terms means the same thing to every implementation.

For a service descriptor:

For the manager's descriptor:

GENERIC_READ maps to the one right on the manager's descriptor that changes nothing, as it maps to SERVICE_QUERY_STATUS on a service.

4.7.3 Per command #

boot is checked like any other command, although what it reports is not a secret: a descriptor decides who may ask, and §4.15 asks that the default one let every authenticated principal.

reset requires SERVICE_STOP because clearing a terminal state is the tail of stopping something rather than the head of starting it.

4.7.4 The sequence #

1. If the manager is shutting down, apply §4.15's restriction. The shutdown restriction is evaluated before the access check, so a caller who would have been denied is told the command is invalid for the current state. A client MUST NOT infer anything about its own rights from an INVALID_STATE received during shutdown.
2. Resolve the target. A command naming no service the manager knows of MUST be answered UNKNOWN_SERVICE. The manager MUST NOT synthesise a descriptor for a service that does not exist.
3. Evaluate the access check with the peer's token, the target's descriptor, the appropriate generic mapping, and the required right, naming the target in the check's audit context (PGSS §6.7).
4. On denial, answer ACCESS_DENIED. The attempt is recorded by the access check itself, under the target descriptor's SACL, with the caller, the target and the rights requested and granted; the manager MUST NOT write an access-denied event of its own (PGSS §6.7). A descriptor the manager supplies by default MUST carry a SACL that audits every refusal, so that a denial is not silent unless a descriptor someone wrote says so.
5. On grant, proceed.

A status, and each job view on this channel, also tells the caller which of these rights it holds on the target, as granted (§4.14). The manager finds them with this same check, asking for MAXIMUM_ALLOWED in step 3 instead of the command's right. That check is a question asked on the caller's behalf, not a command: a right it does not grant is left out of granted, and it is not a denial for step 4.

4.7.5 Filtering rather than denying #

list MUST return only the services the caller may query, and MUST omit the rest rather than denying the command. A caller with no query rights on anything receives an empty list and a successful response.

The response does not say whether services were omitted. What the filtering protects is each service's state, which is what SERVICE_QUERY_STATUS grants, and not whether the service exists: a command naming a service is answered UNKNOWN_SERVICE or by its access check according to whether it exists (step 2 above), and §4.10 does not hide existence either. A client MAY therefore learn from elsewhere which services are defined and show the ones list left out as ones whose state the caller may not see.

job-list MUST be filtered the same way, by JOB_QUERY on each job, and MUST NOT reveal that jobs were omitted.

4.7.6 Not revealing what a caller may not see #

A service's existence is not protected (above). Where a command names an operation the caller may not query, the manager MUST NOT let the answer distinguish "this does not exist" from "you may not see this".

For operation-status this means the authorisation check MUST be evaluated before the operation's existence is reported: a caller lacking SERVICE_QUERY_STATUS on an operation's target MUST receive ACCESS_DENIED whether or not the identifier names a real operation, and MUST NOT receive UNKNOWN_OPERATION for one that exists.

Where the caller's rights cannot be established because the target cannot be resolved, UNKNOWN_OPERATION is the correct answer.

For job-status and job-stop the rule is §7.10's, not this one: a job identifier is unguessable, so a caller that names one was told it, and the manager answers UNKNOWN_JOB for an identifier that names nothing and ACCESS_DENIED for one that names a job the caller may not touch.

A request is one JSON object.

{"command": "start", "service": "jellyfin", "wait": true}

4.8.1 Fields #

command MUST be present and MUST be a string naming a command the manager implements. A request whose command is absent, is not a string, or names no known command MUST be answered INVALID_COMMAND.

service MUST be present and a string for start, stop, restart, reload, reset and status. Its absence, or a non-string value, MUST be answered INVALID_ARGUMENTS.

wait MUST be a boolean when present. A non-boolean MUST be answered INVALID_ARGUMENTS. Its default is per command (§4.13).

type MUST be present and MUST be exactly one of the three values for shutdown. Anything else MUST be answered INVALID_ARGUMENTS.

operation_id MUST be present and a string for operation-status. A value that is not a well-formed identifier MUST be answered INVALID_ARGUMENTS.

job_id MUST be present and a string for job-status and job-stop, under the same rule. The four job-list filters are each optional; one that is present with a malformed value — a SID that does not parse, a state outside §7.B — MUST be answered INVALID_ARGUMENTS rather than matching nothing, and filters that are all present MUST all hold.

4.8.2 Fields that do not apply #

A field the command does not use MUST be ignored, not rejected. A service on a list, or a wait on a status, is accepted and has no effect.

This is what makes the request shape extensible: a client written against a later revision may send a field an earlier manager does not know, and the earlier manager ignores it. §4.21.

4.8.3 Service names #

A service name is 1 to 128 bytes drawn from [A-Za-z0-9._-]. The manager MUST NOT accept a name outside that set, and a client MUST NOT send one.

The restriction exists because service names are used as path components and as configuration key names by managers that store their definitions in a hierarchy. / and : are excluded specifically: the first because it is a separator wherever the name is used as a path component, and the second because it is conventionally reserved for a manager's own synthetic naming.

Every response carries a status field, which MUST be exactly "ok" or "error". What else it carries depends on which of four shapes it is.

4.9.1 The acknowledgement shape #

Returned by a lifecycle command that created, merged into, queued, cancelled, cleared or executed an operation.

{"status": "ok", "operation_id": "a1b2c3d4-…", "service": "jellyfin",
 "state": "active", "cause": "explicit_start", "warnings": []}

warnings here is an array of strings. The status response uses the same field name for an array of objects (§4.14); a client MUST distinguish them by which command it sent, not by inspecting the array.

4.9.2 The status shape #

Returned by status, and also by a lifecycle command that had no effect — see §4.12. §4.14 gives it in full, including granted, the rights the caller holds on the service, which it carries either way.

4.9.3 The system shape #

Returned by shutdown:

{"status": "ok"}

Nothing else. A shutdown has no operation to observe and no service to report on. reload-config and boot have shapes of their own (§4.15).