skaidb wire protocol

This is the canonical specification every official driver implements. It describes the binary fast-path protocol spoken on the server's quic_port (default 7000, raw TCP today). A simpler HTTP/JSON gateway also exists on rest_port (default 7080) and is documented at the end.

The binary protocol is deliberately small: a length-prefixed frame, a four-step SCRAM-SHA-256 handshake, then request/response query frames. The reference implementation is the Rust driver (crates/skaidb-driver, published as the skaidb crate from https://github.com/porcupin26/skaidb-rust) and the Python driver (https://github.com/porcupin26/skaidb-python), which is verified against a live cluster. The published copy of this document is https://skaidb.org/docs/PROTOCOL.html. Every official driver runs the shared conformance vectors generated from the reference encoders — https://skaidb.org/conformance/vectors.json, harness contract at https://skaidb.org/conformance/README.md — against its public API, so a new driver has a byte-exact test suite to start from.

All multi-byte integers are little-endian (LE) except the frame length prefix, which is big-endian (BE). Watch that distinction — it is the single most common porting mistake.

---

1. Framing

Every message — handshake step or query — is one frame:

+-----------------+--------------------------+
| u32 length (BE) | payload (length bytes)   |
+-----------------+--------------------------+

• length is the byte count of payload, big-endian.
• Maximum accepted payload is 64 MiB (64 * 1024 * 1024).
• Read: read 4 bytes → length → read exactly length bytes.
• Write: write the BE length, then the payload, then flush.

Within payloads, all length fields are u32 little-endian unless stated.

The separate member-to-member protocol extends its index-count request (opcode 67) with an optional trailing byte 1 requesting an eligible-key set proof. Absent means the ordinary count. The proof reply uses the existing counter response prefix (u64 count) followed by 32 raw SHA-256 bytes over the ordered eligible primary keys, each preceded by its u64 little-endian length. Older decoders ignore these trailers. A coordinator requesting a proof never accepts an older peer's bare counter as membership agreement; it uses the normal consistency-level row merge. Client query frames are unchanged.

---

2. Handshake (SCRAM-SHA-256)

Exactly four frames, always — even when the server has auth disabled (it then accepts any proof). Run it immediately after connecting, before any query.

client → AuthStart     { username, client_nonce }
server → AuthChallenge { salt, iterations, server_nonce }
client → AuthFinish    { client_proof }
server → AuthOutcome   Ok{ server_signature } | Denied{ reason }

Each handshake payload starts with a 1-byte tag:

Message	Tag
AuthStart	10
AuthChallenge	11
AuthFinish	12
AuthOutcome	13

Encodings (a str/bytes field = u32 LE length followed by the bytes; strings are UTF-8):

AuthStart (client → server)

u8  = 10
str username
str client_nonce

client_nonce is any fresh, unique ASCII string (e.g. "c"+pid+"."+counter).

AuthChallenge (server → client)

u8  = 11
bytes salt              (u32 LE len + salt bytes)
u32 iterations (LE)
str server_nonce

AuthFinish (client → server)

u8  = 12
32 bytes client_proof    (raw, NOT length-prefixed — always exactly 32 bytes)

AuthOutcome (server → client)

u8  = 13
u8  ok_flag
  if ok_flag == 1:  32 bytes server_signature   (raw, exactly 32 bytes)
  if ok_flag == 0:  str reason

2.1 Computing the proof

First build the auth message (identical on both sides). salt_hex is the salt as lowercase hex; iterations is its decimal ASCII form; \0 is a single NUL byte:

auth_message = username + "\0" + client_nonce + "\0" + server_nonce
                        + "\0" + salt_hex      + "\0" + str(iterations)

Then, with HMAC = HMAC-SHA-256, H = SHA-256, PBKDF2 = PBKDF2-HMAC-SHA-256:

salted    = PBKDF2(password, salt, iterations, dkLen = 32)
clientKey = HMAC(salted, "Client Key")
storedKey = H(clientKey)
clientSig = HMAC(storedKey, auth_message)
clientProof = clientKey XOR clientSig           # 32 bytes, sent in AuthFinish

Note the HMAC key argument is first: HMAC(key, message).

2.2 Verifying the server (mutual auth — optional but recommended)

When the password is non-empty, verify the server_signature from a successful AuthOutcome:

serverKey      = HMAC(salted, "Server Key")
expectedServerSig = HMAC(serverKey, auth_message)

If expectedServerSig != server_signature, treat the connection as untrustworthy and close it. Skip this check when connecting anonymously (empty password).

2.3 Anonymous connections

If the server has auth disabled, connect with username "anonymous" and an empty password. The handshake still runs all four frames; the server returns Ok. Do not verify the server signature when the password is empty.

2.4 Choosing a mechanism

AuthStart may end with one more byte, the mechanism. A client that omits it gets SCRAM-SHA-256; a server that predates a mechanism refuses it (the connection closes).

u8  = 10
str username
str client_nonce
u8  mechanism        (optional: 0 = SCRAM-SHA-256, 1 = GSSAPI, 2 = EXTERNAL)

• 0 — SCRAM-SHA-256: the four frames above.
• 1 — GSSAPI (Kerberos): after AuthStart, both sides exchange AuthToken frames (u8 = 14, bytes token) until the context is established; the server then sends AuthOutcome. The username is only a claim — the Kerberos principal is the identity. See KERBEROS.md.
• 2 — EXTERNAL (TLS client certificate): the connection must be TLS and present a client certificate that chains to the server's auth.x509_ca_file (with auth.x509_enabled). No exchange follows AuthStart: the server answers AuthOutcome at once. The certificate's subject Common Name is the username; AuthStart.username must be empty or equal it, and client_nonce may be empty. A successful outcome carries 32 zero bytes as server_signature — do not verify it (TLS already authenticated the server).

---

3. Query request / response

After a successful handshake the connection is a simple request → response loop. One request frame, one response frame, repeat. The connection is persistent and may be reused for many queries (and should be — pool/reuse it).

3.1 Request

u8  = 1                       # OP_QUERY
u8  consistency               # 0 = ONE, 1 = QUORUM, 2 = ALL
u32 sql_len (LE)
sql bytes (UTF-8)

Consistency selects how many replicas must acknowledge (writes) or be consulted (reads) before the server answers. Drivers default to QUORUM (1).

Servers ≥ 0.17.0 also accept the prepared-statement opcodes 2–4 (§3.3); a driver that only ever sends OP_QUERY is fully compatible in both directions.

3.2 Response

First byte is a tag:

Tag	Meaning
0	Rows
1	Mutation
2	Ddl
3	Error
4	Prepared
5	RowsHeader
6	RowsChunk
7	RowsEnd
8	ResultSets

Rows (0) — a SELECT result set:

u8  = 0
u32 ncols (LE)
ncols × [ u32 len (LE) + column name (UTF-8) ]
u32 nrows (LE)
nrows × row, where each row is:
    u32 ncells (LE)
    ncells × [ u32 vlen (LE) + value bytes ]      # value bytes per §4

ncells always equals ncols. Each cell's value bytes is a self-describing Value.

RowsHeader (5) / RowsChunk (6) / RowsEnd (7) — the streamed form of a result set, sent only in reply to OP_QUERY_STREAM (§3.4):

u8  = 5                       # RowsHeader, once, before any row
u32 ncols (LE)
ncols × [ u32 len (LE) + column name (UTF-8) ]

u8  = 6                       # RowsChunk, zero or more times
u32 nrows (LE)
nrows × row                   # row layout exactly as in Rows (0)

u8  = 7                       # RowsEnd, once, terminates the stream

Each frame is an ordinary length-prefixed frame (§2) — a chunk is not a continuation of the header's frame. Read frames until RowsEnd.

ResultSets (8) — several result sets in one reply: a CALL whose body ran EMIT <select> statements. The emitted sets come first, in order, the call's final result last (an ordinary CALL without EMIT still answers with Rows (0)):

u8  = 8
u32 nsets (LE)
nsets × [ columns block + rows block exactly as in Rows (0), i.e.
          u32 ncols, ncols × name, u32 nrows, nrows × row ]

Drivers with a native multi-set API iterate from the first set (Python cursor.nextset(), Go Rows.NextResultSet, Java ResultSet.nextResultSet(), .NET SkaidbDataReader.NextResult()); the others expose every set under result_sets/resultSets with the LAST set as the result's own rows. REST /query answers the same way (result_sets plus the last set inline).

Mutation (1) — INSERT/UPDATE/DELETE:

u8  = 1
u64 affected (LE)

Ddl (2) — CREATE/DROP/etc. succeeded:

u8  = 2

Error (3) — the statement failed:

u8  = 3
str message      (u32 LE len + UTF-8)

Drivers should raise/return this message as a query error.

Prepared (4) — reply to OP_PREPARE (§3.3):

u8  = 4
u32 statement id (LE)
u16 parameter count (LE)

3.3 Prepared statements (server ≥ 0.17.0)

A statement containing ? placeholders can be parsed once and executed many times with different bindings — skipping the per-request SQL parse and giving a typed, injection-safe parameter path. Prepared ids are scoped to the connection that created them: they are invalid on any other connection, and gone when the connection closes. Only SELECT/INSERT/UPDATE/DELETE — and EXPLAIN of any of those (servers ≥ 0.86.0) — can be prepared; DDL and session statements are refused with Error.

OP_PREPARE — parse and cache; answered with Prepared:

u8  = 2                       # OP_PREPARE
u32 sql_len (LE)
sql bytes (UTF-8), may contain `?` placeholders

OP_EXECUTE — run a prepared statement; answered like a normal query:

u8  = 3                       # OP_EXECUTE
u8  consistency               # as in OP_QUERY
u32 statement id (LE)
u16 nparams (LE)
nparams × [ u32 len (LE) + value bytes ]   # §4 Value encoding, in `?` order

The binding count must equal the statement's parameter count exactly, else Error.

OP_CLOSE — free a prepared statement's slot (a server caps open statements per connection at 256); answered with Ddl on success:

u8  = 4                       # OP_CLOSE
u32 statement id (LE)

OP_EXECUTE_BATCH (servers ≥ 0.87.0) — run a prepared statement once per parameter row in a single round-trip (the executemany wire form); answered with Mutation carrying the total affected count:

u8  = 7                       # OP_EXECUTE_BATCH
u8  consistency               # as in OP_QUERY
u32 statement id (LE)
u32 nrows (LE)
nrows × [ u16 nparams (LE) + nparams × [ u32 len (LE) + value bytes ] ]

Each row observes the same semantics as a separate OP_EXECUTE (per-row autocommit, per-row arity check). Plain-INSERT batches without ON CONFLICT are executed as spliced multi-row statements in chunks (servers ≥ 0.196.4) — same results, one write-path traversal per chunk; other statements execute row by row. On the first failing row the reply is an Error naming the row index and how many rows applied before it — earlier rows stay applied, matching the loop the driver would otherwise run. The whole request must fit one frame (§2, 64 MiB).

OP_HELLO (servers ≥ 0.203.0) — best-effort client self-identification, sent once after authentication (and again after any reconnect, since the registry row dies with the connection):

u8  = 8                       # OP_HELLO
u32 name_len (LE)             # driver name, e.g. "python"
name bytes (UTF-8)
u32 version_len (LE)
version bytes (UTF-8)

Answered with Ddl. The values surface in the drivers table's client_name/client_version columns (server-side truncated to 64 characters). An old server answers with Error("… unknown opcode"), which the client MUST ignore — identity is telemetry, never load-bearing. Every official driver sends it automatically, reporting its own package version.

3.4 Streaming a result set

OP_QUERY_STREAM — run a statement and receive its rows in chunks instead of one large frame:

u8  = 5                       # OP_QUERY_STREAM
u8  consistency               # as in OP_QUERY
u32 sql_len (LE)
sql bytes (UTF-8)

The reply depends on what the statement produced:

• Row-producing — RowsHeader, then zero or more RowsChunk, then RowsEnd. Columns arrive in the header, before any row, so a driver can expose them immediately; this is why the column set has to be knowable up front (see the note on SELECT * in QUERY_SYNTAX.md).
• Anything else (INSERT, DDL, USE, …) — one ordinary Mutation or Ddl frame. There is no header and no RowsEnd, so a client must branch on the first frame's tag rather than assuming a stream.
• Failure — an Error frame. If it arrives before the header the statement simply failed. It can also arrive after the header, when a statement fails partway through (a node dying mid-scan, a scan budget tripping); the rows already delivered are valid and the stream ends there.

The connection is busy for the whole stream: no other request may be sent on it until RowsEnd or Error. A client that abandons a stream early must drain the remaining frames before reusing the connection, or close it.

Whether the server also streams internally depends on the statement. An eligible plain SELECT is executed page by page, so the node holds one page rather than the whole result; other shapes are materialised server-side and merely delivered in chunks. Either way the client's memory is bounded, which is the portable guarantee.

An old server answers unknown opcodes with Error("… unknown opcode") — drivers can feature-detect by preparing once and falling back to client-side interpolation (§5), and fall back from OP_EXECUTE_BATCH to a per-row OP_EXECUTE loop the same way.

---

4. Value encoding

Each result cell is one value, encoded losslessly. First byte is a type tag, followed by the type's payload:

Tag	Type	Payload
0	Null	(none)
1	Bool	u8 (0 or 1)
2	Int	i64 LE
3	Float	f64 LE (IEEE-754 bits, little-endian)
4	Decimal	i128 mantissa LE (16 bytes) + u32 scale LE
5	String	u32 len LE + UTF-8 bytes
6	Bytes	u32 len LE + raw bytes
7	Uuid	16 raw bytes (big-endian / RFC 4122 byte order)
8	Timestamp	i64 LE — Unix time in milliseconds
9	Array	u32 count LE + count values (recursive)
10	Document	u32 count LE + count × [ u32 keylen LE + key + value ]

Notes: - Decimal value = mantissa / 10^scale. Drivers may surface it as a string/decimal type to avoid precision loss. Any u32 scale is accepted, stored and returned unchanged. In text form (CAST(x AS TEXT), JSON results, skaidbsh) a scale up to 1000 renders positionally (0.005); a larger one renders exactly in scientific notation (1E-70000), so a driver that parses decimals from text must accept both spellings. - Counts and lengths are validated against the payload before anything is allocated: an Array/Document count larger than the bytes that follow (an element needs at least one byte, a field at least five) is a bad value encoding error, and no count sizes a buffer beyond the frame. - Timestamp is milliseconds since the Unix epoch (can be negative). - Document preserves insertion order of keys. - Map these to the most natural language type (Array→list, Document→map/dict, Uuid→UUID type or canonical string, Bytes→byte array).

---

5. Parameter binding (client-side fallback)

Against servers ≥ 0.17.0 prefer the server-side prepared statements of §3.3. For older servers (or drivers that have not implemented §3.3), offer the parameterized API (?/%s/$1 placeholders) by interpolating arguments into the SQL client-side, with correct SQL quoting:

• string: wrap in single quotes; escape each ' by doubling it → ''. (skaidb uses standard SQL '' escaping; backslashes are literal.)
• integer / float: numeric literal as-is (use a round-trip-safe float format; reject NaN/Inf).
• bool: TRUE / FALSE.
• null / None / nil: NULL.
• bytes: hex/blob literal if supported, else document as unsupported.

Always quote-escape strings — this is the SQL-injection boundary. Provide the placeholder style idiomatic to the language (see each driver's README).

---

6. Connection lifecycle & errors

1. TCP connect (set TCP_NODELAY for low latency).
2. Run the 4-frame handshake. On Denied, raise an auth error and close — unless the reason starts with too many connections: (below).
3. Loop: send request frame, read response frame.
4. On any framing/IO error, the connection is dead — discard it (don't reuse).
5. An Error response is a statement error, not a connection error; the connection stays usable for the next query.

Capacity refusals are connection failures, never auth failures. A node at server.max_connections refuses a new connection in one of two ways, and a driver must treat both as "this connection failed — retry, or use another node", not as rejected credentials (a pool that stops on an auth error must not stop here):

• the socket is closed without a handshake frame — at accept, or right after the server read AuthStart (the few slots above the ceiling are reserved for the configured superuser, and a client that names anyone else is refused before any challenge). The driver sees EOF or a reset where it expected AuthChallenge: an I/O error, per rule 4;
• AuthOutcome Denied whose reason starts with too many connections: — only where the server learns who the client is from the exchange itself (GSSAPI without a username in AuthStart). Map it to your connection-error class, keeping the reason as the message.

A cluster has multiple nodes (default internode/clients on each). Every node accepts reads and writes (leaderless), so a driver may connect to any node, or round-robin a list of hosts for availability. Token-aware routing is optional.

---

7. REST/JSON gateway (alternative, port 7080)

For environments where a binary client is inconvenient (browsers, quick scripts), the server also speaks HTTP/1.1:

• POST /query with the SQL as the body (raw text) or JSON {"sql": "..."}.
• Auth: HTTP Basic (Authorization: Basic base64(user:pass)) when the server requires auth.
• One request per connection (Connection: close); request bodies are capped at 64 MiB (HTTP 413 past it), and sockets carry 30 s read / 60 s write timeouts (servers ≥ 0.83.1).
• Response JSON:
• rows: {"columns": [...], "rows": [[...], ...]} (values as JSON) — servers ≥ 0.87.0 send row results with Transfer-Encoding: chunked (no size cap; any HTTP client reassembles transparently). JSON has no non-finite numbers: a float that is ±Infinity or NaN is the string "Infinity", "-Infinity" or "NaN"; null is SQL NULL only. (The binary protocol's float is the IEEE value itself.)
• mutation: {"affected": N}
• ddl: {"ok": true}
• error: {"error": "..."} (HTTP 400)
• GET /metrics returns Prometheus text.

The REST path loses binary type fidelity (UUID/Bytes/Timestamp arrive as their JSON forms) and opens a fresh connection per request, so the binary protocol is preferred for application drivers. Drivers may offer REST as a fallback mode.