3.19 Lexical Rules and Literals

5 min read

A query string is UTF-8. Whitespace separates tokens outside quoted strings and is otherwise insignificant.

3.19.1 Identifiers #

An unquoted identifier is ASCII and matches:

[A-Za-z_][A-Za-z0-9_.-]*

Identifiers name fields, payload paths, metric names, label keys, event type patterns, log origins and value aliases. ., _ and - are permitted inside one; /, :, whitespace, quotes, brackets, parentheses, commas and the comparison operators are not.

This grammar is the same one that constrains a log origin, a metric name and a metric label key at ingestion (§3.7, §3.10), which is what makes every stored identifier writable here without quoting.

A value that cannot be written as an identifier MUST be written as a quoted string. Quoted forms are accepted anywhere an identifier is — they are never required for a conforming identifier, but a pattern may need one, and a collector holding identifiers stored under an earlier revision must still be able to select them.

3.19.2 Strings #

A string literal is double-quoted UTF-8. The escapes are \", \\, \n, \r, \t, and \uXXXX for a scalar value in U+0000 to U+FFFF written as four hexadecimal digits.

A collector MUST reject any other backslash escape as a parse error, and MUST reject \uXXXX naming a surrogate code point in U+D800 to U+DFFF. Surrogate pairs are not decoded: a character outside the basic multilingual plane is written directly as UTF-8, not as two escapes.

3.19.3 Binary #

A binary literal is a lowercase x, a double quote, an even number of hexadecimal digits, and a closing quote:

WHERE target_sid == x"010500000000000515000000"

Hexadecimal digits inside are case-insensitive. x"" is valid and is the empty byte string. Whitespace inside the payload, an odd digit count, and any non-hexadecimal character are parse errors.

A binary literal compares only against MessagePack bin values. A collector MUST NOT coerce one to a string or a string to one: x"6162" and "ab" are different values and never compare equal (§3.20).

3.19.4 Integers #

An integer literal is decimal or hexadecimal.

WHERE origin_class == 2
WHERE granted_access == 0x1F01FF

A decimal integer MAY carry a leading -, in which case it MUST fit in signed 64 bits; without one it MUST fit in unsigned 64 bits. A hexadecimal integer is 0x followed by one or more digits, is always non-negative, and MUST fit in unsigned 64 bits. A leading + is not valid. An out-of-range literal is a parse error.

3.19.5 Floats #

A float literal is a finite decimal number with an optional leading - and either a fractional part or an exponent: 42.0, 0.001, 1e6, -1.25e-3. A token that looks like an integer, such as 42, is an integer literal and not a float.

Float literals are binary64 and MUST be finite. NaN, Infinity, -Infinity and any literal that overflows to infinity are parse errors. A leading + is not valid.

3.19.6 Booleans and null #

true and false are matched case-insensitively with ASCII folding.

NULL, likewise folded, is valid only in IS NULL and IS NOT NULL. A collector MUST reject field == NULL and field != NULL as parse errors rather than evaluating them.

3.19.7 Durations #

A duration is an unsigned decimal integer followed immediately by s, m, h or d — seconds, minutes, hours or days. A zero duration is a parse error.

3.19.8 Times #

Absolute literals are a fixed UTC subset. Components MUST be zero-padded exactly as shown, the date MUST be a valid Gregorian date, hours are 00–23, minutes and seconds 00–59. Leap seconds are not accepted. Timezone suffixes and fractional seconds are not part of this revision and MUST produce a parse error.

3.19.9 The evaluation time #

A collector MUST capture the evaluation time once, before execution begins, and MUST use that one reading for every ago, every hence, and for an omitted UNTIL, throughout the query — including throughout the watch phase of a streaming query.

A query that read the clock more than once could produce a range whose end preceded its start, or a window that grew while it was being scanned. One reading makes the effective query range a fixed interval for the life of the query.

SINCE is inclusive, UNTIL is exclusive: the effective query range is [SINCE, UNTIL). If SINCE is greater than or equal to UNTIL the query returns no records — which is a successful query with an empty result (§3.16), not an error. A time literal that evaluates outside the timestamp domain MUST produce an error (§3.5).

Edit this page