skaidb benchmarks

The latest measured results for the standard comparison scenarios. Numbers are only comparable within a section: each section is one measurement round — all systems in it measured together, in the same environment, driven by the same client harness. Never compare figures across sections. Superseded results live in git history, never alongside current numbers.

Sections: the CRUD comparison suite, the skaidb-only fleet round, the workstation A/B and hardware scaling, search workloads vs Elasticsearch, vector search vs Qdrant, and the MQTT broker vs Mosquitto.

Numbers are for relative comparison on small nodes, not absolute peak throughput. Run-to-run spread on this shared host is ±10–15% when the host is busy; measured on a quiet host with warm runs it tightens to ±3% (five consecutive write 1c runs: 973/972/935/963/986). Treat differences inside the band as noise, and never compare figures from different sections or environments.

Environment

Host. One Proxmox host — Intel Core i7-8550U (4 cores / 8 threads, 1.8 GHz), 8 GB RAM. Every node is an unprivileged LXC container on this host, bridged on one VLAN. The host also carries unrelated background load; the methodology section below describes how interference is kept out of conclusions.

Nodes. Identical containers: 1 vCPU / 512 MB RAM / 4 GB disk, Debian 12. A 3-node configuration is three such containers.

Client. A dedicated VLAN-local client container for multi-node scenarios; colocated on the server container for the single-node C0 scenario (true loopback).

The systems compared (skaidb is the release build under test in each round):

PostgreSQL	MongoDB	MariaDB	Elasticsearch	Qdrant
17.10	7.0.37 / 8.0.26	11.4	8.19.18	1.19.1

Durability is matched across systems. In each config a write is acknowledged only after the same number of nodes have made it durable:

Config	Nodes	A write is acked after…	skaidb	MongoDB	PostgreSQL	MariaDB
C0	1	local WAL/journal fsync	RF=1, `ONE`	single-member rs	no standbys	binlog only
C1	2	both nodes	`QUORUM`	`w:2`	sync standby (`FIRST 1`)	semi-sync
C2	2	the primary only (async replica)	`ONE`	`w:1`	async (`''`)	semi-sync off
C3	3	all 3 nodes	`ALL`	`w:3`	`FIRST 2` sync standbys	— ¹
C4	3	any 2 of 3 (quorum)	`QUORUM`	`w:majority`	`ANY 1` standby	semi-sync ¹

¹ MariaDB semi-sync acknowledges after the first replica responds and has no "wait for N replicas" knob, so true all-3 durability isn't expressible; its C3 row is the same semi-sync mode as C4 (≈ 2-of-3), a single measurement marked *.

Workloads (throughput in ops/sec, higher is better):

write 1c — single connection inserting unique keys (durable-write latency floor)
write 16c — 16 connections inserting (concurrent write throughput)
read 16c — 16 connections, point read by primary key over a 1,000-row table
mixed 16c — 16 connections, 50/50 read/write

Every client connects and authenticates before the timed window opens.

C0 — 1 node, no replication

One node, no peers, client colocated (loopback). skaidb: replication_factor = 1, consistency ONE. PostgreSQL: synchronous_standby_names cleared, standbys stopped. MongoDB: reconfigured to a genuine single-member replica set. MariaDB: semi-sync off. Elasticsearch runs the same 1 vCPU / 512 MB spec with a 256 MB heap (its default 1 GB does not fit this node class — the number reflects a system at its memory floor).

Workload	skaidb	PostgreSQL	MongoDB 7	MariaDB	Elasticsearch
write 1c	1,877	1,692	703	319	79
write 16c	5,522	3,004	1,582	1,523	206
read 16c	13,660	4,143	1,673	3,209	973
mixed 16c	8,211	4,244	1,601	2,748	480

skaidb leads every C0 cell: 1.1× PostgreSQL on the single-connection durable-write floor, 1.8× on concurrent writes, 3.3× on reads, 1.9× on mixed.

C1 — 2 nodes, writes wait for both

Workload	skaidb	PostgreSQL	MongoDB 7	MongoDB 8	MariaDB
write 1c	863	698	268	197	316
write 16c	3,855	2,814	1,134	607	1,666
read 16c	11,464	4,441	1,730	2,297	3,092
mixed 16c	5,991	3,582	1,393	1,000	2,861

C2 — 2 nodes, writes wait for the primary only (async replica)

Workload	skaidb	PostgreSQL	MongoDB 7	MongoDB 8	MariaDB
write 1c	924	981	761	778	348
write 16c	3,530	3,779	1,567	1,782	1,756
read 16c	11,967	4,609	1,457	2,221	3,022
mixed 16c	6,175	4,008	1,541	1,739	3,138

C3 — 3 nodes, writes wait for all 3

Workload	skaidb	PostgreSQL	MongoDB 7	MongoDB 8	MariaDB*
write 1c	790	523	171	174	267*
write 16c	3,158	2,433	856	535	1,345*
read 16c	11,721	4,573	1,382	2,139	3,029*
mixed 16c	5,358	3,167	1,239	889	2,410*

C4 — 3 nodes, writes wait for 2 of 3 (quorum)

Workload	skaidb	PostgreSQL	MongoDB 7	MongoDB 8	MariaDB
write 1c	786	629	221	210	263
write 16c	3,231	2,539	874	1,057	1,262
read 16c	11,577	4,528	1,573	2,273	3,047
mixed 16c	5,260	3,318	1,163	1,578	2,350

Reading the standings

skaidb leads every replicated-config cell except C2 writes, where PostgreSQL's async replication edges it inside the noise band (981 vs 924 at 1c, 3,779 vs 3,530 at 16c). Largest margins are reads (2.5–2.6× over PostgreSQL at 3 nodes; 5–8× over MongoDB) and strict-durability writes (C3 write 1c: 1.5× PostgreSQL, 4.5× MongoDB).
skaidb's reads scale with members (every node coordinates against its full local copy); the other systems read from a single primary regardless of cluster size.
Durability level barely moves skaidb throughput (C1 ≈ C2 ≈ C3 ≈ C4 within noise at 16 connections): the fsync is group-committed and the peer round-trip pipelined, so stricter acks cost latency headroom, not throughput. MongoDB pays heavily for w:3; PostgreSQL is comparatively flat like skaidb.
These runs include skaidb's per-row consensus machinery for clustered UPDATEs (see CLUSTER_CONSISTENCY.md) — it costs nothing on these workloads because INSERT/SELECT take the plain paths by design.

skaidb-only fleet round

Its own measurement round, skaidb only: same host, same node specs, same harness and the same four workloads as the comparison suite, warm runs (the first run of each workload discarded). Per this document's own rule, do not read a number here against a competitor column above: those were measured in an earlier round, and the bench host has since gained a colocated production node.

Config	write 1c	write 16c	read 16c	mixed 16c
C0 — 1 node, `ONE`	1,858	5,520	16,645	8,505
C1 — 2 nodes, both	1,008	4,328	15,164	6,920
C2 — 2 nodes, primary only	992	4,158	15,597	7,057
C3 — 3 nodes, all 3	737	3,996	21,012	6,750
C4 — 3 nodes, quorum	710	4,006	20,148	6,589

Three-node reads exceed one-node reads: every member coordinates against its full local copy, so a read quorum spreads over more cores than a single node has.
write 1c is the least trustworthy line on a shared host: one connection deep, it measures durable-write latency and is the most exposed to a colocated neighbour.

What an A/B between two builds has to control

These rules came out of tracking the point-read path across releases on this fleet, and bench/README.md carries the recipes:

Hold the client fixed. In C0 the bench client is colocated on the 1-vCPU node, so a cheaper client buys the server CPU: a cross-round "gain" on reads turned out to be the client's, and with one client interleaved against two server binaries the server had moved the other way. Compare server binaries only under one client, same day, legs interleaved.
Alternate which binary goes first. The first binary measured after a fleet wipe reads low (about 10% at C0 write 1c on the fleet, about 3% on a quiet workstation); discarding the first run of each workload does not cover it. Alternate the order per leg and compare medians.
Whole-server throughput does not localize a per-statement cost; instruction counts do. bench/callgrind/server_point_read.sh runs the release server under valgrind for exact instructions per statement, which resolves the 1% effects timing benches cannot. On the point-read path the engine's leg is about 6.3k instructions per statement — the WHERE constraints, the primary-key column list and the projection set are read by reference, never cloned per statement, and a point lookup skips the partition-fold route built for scans (Cluster::is_pk_point_lookup) — and what remains there is the storage layer itself (memtable range probe, read-cache hash, negative-cache insert). About 90% of the CPU per point read lives above the engine — request handling, admission gates, per-statement metrics, protocol encode — which makes that the larger target whenever the per-operation cost moves.
Concurrency exposes what a single connection hides. A few microseconds added per statement show only at 16 connections on 1 vCPU, where every spare microsecond of server CPU is throughput; the single-connection lines stay put because they wait on the fsync.

Workstation A/B and how the numbers scale with hardware

Its own round, on a different class of machine from the fleet above: one 32-core workstation with an NVMe disk, server and client on the same host (loopback), data directories on disk (never tmpfs — a tmpfs data directory has no fsync and makes every write figure meaningless). Two consecutive builds, the current release and its predecessor, run interleaved with the order alternated per leg. These figures are skaidb-only and are not comparable to the fleet tables: the hardware is the reason, as the next section explains.

Workload	current	predecessor	p50 latency
standalone, write 1c	4,756–4,966	4,752–4,954	0.20 ms
standalone, write 16c	29,042–29,825	29,543–31,379	0.43 ms
standalone, read 16c	264,727–277,622	269,010–273,613	0.05 ms
standalone, mixed 16c	59,418–62,103	58,443–61,059	0.20 ms
3 nodes on the same host, RF 3, `QUORUM`, write 16c, TTL table	18,653–19,233	18,703–19,197	0.80 ms
same, plain table	19,173	18,796	0.78 ms

Every row is inside its run-to-run spread: the release is flat against its predecessor, including on a TTL table, where the current release maintains a content digest on every write that its predecessor did not.

What scales with what

The fleet rows and the workstation rows measure the same binary on machines that differ by 32× in cores and by ~2.7× in fsync latency. Their ratio is the scaling law, and it is different for every workload:

Workload	fleet (1 vCPU, C0)	workstation (32 cores)	ratio	bound by
write 1c	1,858 (0.54 ms/op)	4,900 (0.20 ms/op)	2.6×	the storage device's fsync latency
write 16c	5,520	29,100	5.3×	cores, until WAL group commit serializes
read 16c	16,645	269,000	16×	cores (reads share nothing on the hot path)
mixed 16c	8,505	61,100	7.2×	in between

A single connection writing is a storage benchmark. Every acked write is one WAL fsync, so one connection gets about 1 / fsync-latency operations per second whatever the CPU: 0.54 ms on the fleet's disk, 0.20 ms on NVMe. Faster CPUs do not move this row; faster disks do.
Concurrent writes scale with cores, sub-linearly. Sixteen connections share the WAL append and fsync; the group commit amortizes the fsync across them, so throughput grows with cores until the commit path serializes. 32× the cores bought 5.3×.
Point reads scale about linearly with cores. Each connection's reads are independent (block cache, no shared lock hold on the read path), so the read column follows core count: 16× on 32× the cores, with the remainder being the loopback socket round-trip that dominates a 0.05 ms operation.
Therefore the read-to-write ratio is a property of the hardware, not the engine: 3× at 16 connections on a 1-vCPU node, 9× on 32 cores. Do not read a high or low ratio on your own hardware as a code difference.
Replication costs latency headroom first, throughput second. At one connection the quorum round-trip adds itself to every write (fleet C0 → C4: 0.54 → 1.41 ms/op); at 16 connections the round-trip is pipelined behind the fsync and costs −27% on separate hosts. Three nodes sharing one machine — the workstation cluster above — cost −35%, because the replicas compete for the same cores and disk; that is the pessimistic bound, not what separate hosts see.

Expectations on other hardware

Writes at one connection: 1 / (fsync latency + replica RTT at QUORUM) operations per second. Measure your device's fsync latency (fio --fsync=1 --bs=4k) before expecting more.
Writes at many connections: the fleet's 5,500 per vCPU-class node scales with cores at roughly the square root of the core ratio on this data (5.3× for 32×); disks with fast fsync lift the ceiling.
Point reads: expect roughly linear with cores from the 1-vCPU figure of ~16,600, until the network round-trip, not CPU, is the floor (0.05 ms on loopback here).
Replication: budget one peer RTT per write at one connection and about a quarter of 16-connection throughput at RF 3 QUORUM on separate hosts.
Tables with a TTL, secondary indexes or full-text indexes add per-write maintenance; on these workloads (a plain primary-key table and a TTL table) the difference was inside the noise.

Search workloads vs Elasticsearch 9.4.4

Workstation round (32 cores / 91 GB, loopback): skaidb (release build, defaults, auth off) vs Elasticsearch 9.4.4 (Docker, single node, 1 shard / 0 replicas, security off, 4 GB heap). One 200k-document corpus generated deterministically (zipf vocabulary of 5,000 terms; text title/body, keyword category (50 values), numeric price/stock) and ingested into both through the same _bulk driver; ES explicitly mapped, skaidb auto-mapped by its ES gateway. Semantics verified before timing: hit counts and aggregation buckets agree exactly on every workload (ES caps total.value at 10k by default; skaidb reports exact totals). Queries: 16 threads × 10 s, identical bodies via POST /bench/_search; two interleaved legs (A→B then B→A) agreed within noise.

Workload	skaidb	Elasticsearch
`_bulk` ingest, 8 threads × 1k-doc batches (docs/s)	73,500	104,600
`match` (single term, mixed frequency), size 10 (q/s)	52,300	18,900
`match_phrase` (real bigrams), size 10 (q/s)	6,700	11,300
`bool`: high-freq `match` must + `term` filter (q/s)	41,700	17,100
`terms` agg + `avg` sub-agg over high-freq `match` (q/s)	12,300	34,800

What the rows reflect:

match leads on the REST gateway's persistent connections (p50 0.21 ms).
bool executes as a postings intersection: the gateway's auto-mapping declares a .keyword twin for every string field (mirroring ES dynamic mapping) and translates string term filters to exact matches on it, rather than materializing 10–25k candidate rows per query — 2.4× Elasticsearch on the identical corpus, with identical hit counts.
Keyword-grouped metric aggregations push down as a fold over fast-field columns (matching doc set → per-segment ord-indexed accumulators) with exact buckets. Elasticsearch leads this row — its shard request cache serves repeated size:0 aggregations from cache (on by default), while skaidb recomputes the ~24k-doc fold every query (p50 0.98 ms, proportional to match count).
Phrase and ingest are the two rows where a dedicated search engine leads, at the same order of magnitude.

Vector search vs Qdrant 1.19.1

One node each on identical containers (2 vCPU / 2 GB — larger than the C0–C4 nodes above, since an HNSW graph of this size does not fit 512 MB), 100,000 × 768-dimension vectors, cosine, HNSW m = 16 / ef_construct = 200 on both sides, and a payload index on the filter field on both sides (Qdrant a keyword index, skaidb a secondary index — without one, either system must read rows to evaluate a filter). Both driven over REST by the same client (bench/clients/vector_bench.py), 250 queries per point, k = 10, recall@10 against exact brute force. Filtered queries select one category holding 10% of the rows.

ef	skaidb recall	Qdrant recall	skaidb p50	Qdrant p50
32	0.8899	0.9008	3.68 ms	2.95 ms
64	0.9698	0.9769	4.25 ms	3.39 ms
128	0.9962	0.9976	5.34 ms	4.46 ms
256	0.9999	1.0000	7.30 ms	6.46 ms

Unfiltered, the two are within ~25% on latency at matched recall; the dot/l2 kernels vectorize.

Filtered (the same queries plus WHERE cat = …):

ef	skaidb recall	Qdrant recall	skaidb p50	Qdrant p50
32	0.9974	0.9552	11.56 ms	2.92 ms
64	1.0000	0.9951	15.65 ms	3.24 ms
128	1.0000	0.9999	22.36 ms	3.97 ms
256	1.0000	1.0000	33.06 ms	5.00 ms

skaidb returns more of the true neighbours here (the filter gates admission inside the traversal, and a filter the graph cannot satisfy is answered exactly), but pays 3–4× the latency at matched recall: walking until k rows PASS the filter visits several times more nodes than walking until k rows are found (a post-filtered over-fetch would be faster and inexact: about 0.85 recall on this corpus).

Index build, the same corpus, time until the index answers queries:

	skaidb	Qdrant
Exact (f32) index	742 s	506 s
Binary-quantized index	222 s	506 s

QUANTIZED BINARY builds fastest because a Hamming distance over twelve machine words costs far less than 768 float multiplies, and the exact rescore recovers the ranking: 0.9949 plain recall at ef = 32 (better than our own exact index at that ef) and 1.0000 filtered, at a flat 13–15 ms plain / 25–31 ms filtered. Building an index no longer blocks the node either — see VECTOR.md.

Read the standings the way this document says elsewhere: these are two systems doing different jobs. Qdrant is a specialist that stores vectors and payloads; skaidb is a durable, tunably-consistent database that also indexes vectors, so the same rows carry SQL, full text, geo and time series, and a vector query can join them.

MQTT broker vs Mosquitto 2.0.22

Workstation round (same host, loopback): skaidb's native MQTT broker (standalone, state_consistency = one, $SYS off) vs Mosquitto 2.0.22 (persistence false, max_inflight_messages 64, max_queued_messages 1000 — matching skaidb's outbox). One client drove both — cargo run -p skaidb-mqtt --example mqtt_bench, which speaks the protocol through this workspace's codec (see bench/README.md); 64-byte payloads, clean sessions, repeated rounds within ~5%.

Scenario	skaidb	Mosquitto	ratio
QoS 1 acked publish throughput, 64 in flight (msg/s)	200,000	130,000	1.54×
Fan-out: 50 subscribers, aggregate deliveries (del/s)	882,000	411,000	2.15×
QoS 0 delivered throughput, 1 pub → 1 sub (msg/s)	224,000	302,000	0.74×
Delivery latency p50 / p99 (µs)	19 / 27	20 / 31	—

Mosquitto wins QoS 0 to a single subscriber, and the flow-control philosophies are why. Offered ~240k msg/s by a paced publisher, Mosquitto delivers every message (it can accept ~480k/s and applies TCP backpressure when it cannot keep up); skaidb reads the publisher flat out and sheds ~6% at the per-session outbox under its QoS 0 drop policy. The comparable figure is what each broker delivers at a publish rate neither client-limits; comparing one broker's delivered rate with the other's throttled accepted rate under a flat-out publisher compares two different quantities and flatters whichever sheds. Fan-out and acked QoS 1 publishing, where the work is the broker's own routing and acknowledgement paths, remain skaidb's by a wide margin.

Broker RSS after the runs: ~31 MB for skaidb (a full database node) vs ~8 MB (a dedicated broker). Idle connections cost skaidb a registered descriptor rather than a thread pair (mqtt.idle_park_secs), and parking is free under load: on/off is within noise on every scenario above. Mosquitto-client interop (QoS 0/1/2, retained, MQTT 5, persistent sessions, state surviving a broker restart) is exercised separately — see MQTT.md.

Methodology

One system benchmarked at a time; suites run one after another within a single day.
Clients pre-connect before the timed window; connection setup is never measured.
Config transitions are verified, not assumed: MariaDB semi-sync via Rpl_semi_sync_master_clients (the master-side flag alone can lie), MongoDB topology via rs.status() member counts, PostgreSQL via pg_stat_replication.
The bench host is shared; interference shows up as temporally-clustered swings. Perf claims about code changes therefore require interleaved same-day A/B runs with alternating binaries, agreeing across legs separated in time — a single leg (even an interleaved one inside an interference window) is not evidence.
The client must be VLAN-local. Driving the suite from a developer machine measures the network, not the database: RTT from a routed workstation to the bench VLAN is ~16 ms against ~0.05 ms from a container on the bench host — at one connection that is the entire measurement (69 ops/s vs 760 for the identical C4 write). Every figure here comes from a client container on the bench host.
Discard the first run of each workload. A cold engine reads consistently low and recovers within one run: C4 write 1c measured 677 cold, then 735/736/741/728. Steady-state spread on that workload is ±3%, so a single cold run reads as a ~10% regression that does not exist.
Alternate the binaries within every leg of an A/B, and read medians. The first binary in an interleaved pair reads low even on a quiet workstation over loopback (~3% on write 1c, in whichever order the pair runs), so an A/B whose legs always start with the same build shows a regression that does not exist. Three legs with the order alternated, compared on medians, resolve 3% on the workstation; on the shared fleet the same rule resolves about 10%.
To reproduce: the client harness (run_suite.sh + per-system clients) lives on the bench client container; each system's config-switching procedure is described in its scenario section above.