6.1 Parsing and Planning

4 min read

A query arrives as one string (PSPU §3.15). Turning it into an answer has four phases before any data is read: parse, plan, authorize, execute.

6.1.1 Parsing #

The string is parsed into a syntax tree. The parser:

  1. Identifies the mode from the first token — EVENTS, LOGS or METRIC.
  2. Extracts the primary selector: a type pattern, a FROM list, or a metric name with an optional label selector.
  3. Collects every clause, in whatever order they appear.
  4. Validates that the clauses suit the mode — CONTAINING only in log mode, RATE only in metric mode, SELECT only where a result schema is not fixed.

Parse errors are returned immediately, before anything is opened, read or authorized. A malformed query costs a decode and a parse.

6.1.2 What can only fail later #

Some failures need data. The parser cannot know whether a metric name resolves to a counter or a histogram, how many series a selector matches, or whether the effective range exceeds the cross-type lookback limit — those depend on the store, so they surface at planning or execution time (PSPU §3.B).

The practical consequence is that the same query string can parse everywhere and fail on one machine: a metric selector matching one series on a two-core box matches two on a four-core one.

6.1.3 Planning #

Planning resolves what the query will actually touch:

  • which concrete identifiers the data could carry — event types, log origins, metric names — because access control resolves per identifier and a broad selector authorizes nothing by itself (§7.4)
  • which series a metric selector matches, and whether they are type-homogeneous
  • which fields the query references, for both authorization and frequency accounting (§6.5)
  • which stores are involved, including any cross-type source

Identifier discovery reads compact catalogues: the union of every shard's event_types, the log store's log_origins, and the metric store's series rows (§3.1, §4.2, §5.2). The primary selector filters that set before access checks. Discovery therefore never runs a DISTINCT scan over a hot records table and does not depend on an adaptive event index being present.

Catalogues may conservatively contain a name whose final retained row was deleted. That safe superset can cause an extra access check but cannot expose a row or omit a concrete identifier that committed successfully. Low-priority cleanup is optional and is never query-critical.

6.1.4 Header literal types #

Literal conversion uses the namespace of the record being compared. emitter.class aliases apply only to event headers. GUID forms apply to the event GUID headers, log boot_id and job_id, and metric boot_id. The same spelling in an ordinary payload field or metric label remains a string; it does not borrow another namespace's header type (PSPU §3.22).

The rule holds for comparisons, membership and compound predicates, metric bracket selectors, and metric selectors inside historical or streaming cross-type conditions. A metric condition uses the metric namespace even when the query's primary source is events or logs.

6.1.5 When payloads are decoded #

Event payloads are stored as opaque MessagePack and are never decoded on the write path (§3.1). Decoding happens here, on the read path, and only where a query needs it: to evaluate a payload predicate, to build a flat result record, or to compute a payload expression index's key on insert.

At high result counts this dominates the query path. Constructing flat maps from thousands of events means decoding thousands of payloads and applying the flattening rules of PSPU §3.22 to each. Partial extraction is the lever: with a SELECT present, only the named paths need decoding, and without one a streaming decoder that emits flattened pairs avoids materialising the payload at all (§C).

6.1.6 Read connections #

Execution uses read-only SQLite connections, which in WAL mode do not block writer threads. The one exception is SQLite's own. A reader that catches the wal-index header while a writer is updating it takes the write lock for the moment it needs to read the header again. A writer therefore waits up to 25 milliseconds for the write lock rather than failing a commit at once, and fails only if the lock is held longer.

An event query opens one connection per shard database in the directory (§6.4). A log or metric query opens one. eventd supports concurrent queries up to its admission limit (§6.5), subject to the operating system actually having the descriptors and memory; where it cannot allocate what an admitted query needs, it fails that query rather than blocking a writer or exceeding the limit.

Edit this page