release · v0.2.1-rc1 · 4 complete · 3 partial · 0 valid negative / 7 lanes

Antfly Circus — v0.2.1-rc1

Fresh Antfly-only standard honed evidence for v0.2.1-rc1 with reviewed competitor values and exact version provenance.

Reading contract

Each selected outcome binds its Antfly evidence to v0.2.1-rc1. Machines, dates, and protocols may differ.

2 of 2 lanes measured

VectorDBBench

Qualified vector search quality, throughput, serial latency, serving demand, and database volume across operating points.

vector-vdbbench-50k

complete

50K

Fresh three-lifecycle Antfly and same-machine peer evidence for the 1536d/50K vector comparison.

Methodology scope · protocol · interpretation

VectorDBBench methodology

Antfly subject and execution lane

Circus accepts an explicit subject manifest containing the asserted release tag, commit, and Linux/arm64 Antfly binary. It records the binary path, SHA-256, architecture, and observed --version output, while treating the supplied tag and commit as release metadata. The honed matrix mounts that binary read-only into Antfly’s Linux container. Every comparison engine runs in the same private Docker network; no native macOS database is mixed into this benchmark.

The public benchmarks/VectorDBBench checkout remains clean. Circus-owned parameter selection, compatibility fixes, query partitioning, and diagnostics live in benchmarks/VectorDBBench-harness and are loaded as a runtime overlay.

Workloads and query protocol

The publication matrix contains six engines on both upstream workloads:

Performance1536D50K: 50,000 vectors with 1,536 dimensions.
Performance768D1M: 1,000,000 vectors with 768 dimensions.

For each workload, the 1,000 upstream query/ground-truth pairs are ranked by a fixed SHA-256 seed and split into 200 calibration queries and 800 publication queries. Their exact overlap is zero; both partition fingerprints are retained.

Each engine/workload trial has one fresh service and one fixed index lifecycle:

Start lifecycle memory sampling before indexing.
Load and build the index once using fixed construction parameters.
Sweep only the query-time parameter on the 200-query calibration partition.
Freeze the lowest setting reaching recall 0.955. If none qualifies, retain the highest-recall setting and mark it below the qualification floor.
Without rebuilding, run the selected setting on the untouched 800-query partition at 1, 5, 10, 20, and 30 concurrent clients for 30 seconds each, followed by the upstream serial quality and latency pass.
Capture one diagnostic post-workload storage observation, gracefully stop and verify the service, capture final storage, retain the artifacts, and only then reclaim the engine’s data.

The 50K and 1M settings are selected independently because the larger graph may need a higher search setting. Publication results never influence selection. A row below the 0.950 publication floor remains visible and is not silently retuned or omitted. No process-restart measurement is published.

Tuned and fixed parameters

Engine	Fixed index/build parameters	Query-time selection
Antfly	AKNN, one shard, packed float32 queries	`search_effort` sweep
Milvus	HNSW `M=16`, `efConstruction=256`	`ef_search` sweep
Chroma	HNSW `M=16`, `efConstruction=256`	fixed `ef_search=256` qualification
Weaviate	HNSW `maxConnections=16`, `efConstruction=256`	`ef` sweep
pgvector	HNSW `m=16`, `ef_construction=128`	`ef_search` sweep
Elasticsearch	HNSW `m=16`, `ef_construction=100`	`num_candidates` sweep

The final report prints the selected value for every workload and engine. Weaviate and Elasticsearch are local containers; their upstream adapter class names (WeaviateCloud and ElasticCloud) do not describe the deployment.

Memory

The table reports both memory views. Demand working set is the maximum sampled cgroup-v2 anon + shmem from before indexing through readiness, calibration, and publication queries; it excludes reclaimable file cache and follows MEASURING_ANTFLY.md. Cache-inclusive peak is the maximum concurrently sampled sum of memory.current across the complete service stack and includes file cache and cgroup-charged kernel memory. Sampling runs every 200 ms and retains the timeline. Kernel memory.peak is also retained as an exact peak for a single cgroup, or a conservative sum of independent peaks for a multi-container stack.

Honed publication requires three fresh lifecycles per engine/workload. The headline table shows each arithmetic mean; detail artifacts retain every observation, the mean, and the individual-trial range. A trial is rejected if the canonical sampler reports a failure, misses any container in the service stack, starts after indexing, stops before publication finishes, or leaves a timeline gap above five seconds. Peak throughput, its concurrency, C=30 throughput and concurrent p99, recall, nDCG, serial latency, and memory are arithmetic means of the three compatible lifecycle observations. Raw curves and ranges remain retained. The selected setting and protocol identity must match across all three trials.

The complete Antfly standalone container is measured, so all Antfly processes and threads in that cgroup are included. Termite is not part of this workload because VectorDBBench supplies precomputed vectors.

Persistent database volume

Every publication engine receives one newly created, run-labeled Docker local volume at its complete persistence root. The digest-pinned Alpine probe captures whole-volume allocated and apparent bytes once at empty, ready, and post_workload, then at the verified post-stop final barrier. Host-path/no-network Antfly is rejected for comparative storage. Pre-stop background compaction remains diagnostic and does not gate the post-stop metric.

Final allocated bytes are the headline. The same three construction lifecycles define performance, memory, and storage aggregates. Every value, arithmetic mean, range, receipt, and barrier sample remains under raw/storage.

Internal Antfly scale diagnostic

make vdbbench-scale-fast and make vdbbench-scale-big run an Antfly-only diagnostic beside the cross-engine publication matrix. The fast profile uses 20K, 40K, 60K, 80K, and 100K vectors; the big profile uses 200K, 400K, 600K, 800K, and 1M vectors. make vdbbench-scale selects both. All points use 768 dimensions and the same 200-query calibration partition, disjoint 800-query held-out partition, serial pass, and 1/5/10/20/30-client 30-second capped sweep described above.

The diagnostic has exactly two fresh index lifecycles per point. Trial one runs from the smallest to largest selected corpus; trial two reverses that order. This controls simple order effects without claiming the three-lifecycle publication cohort. The aggregate and every row set publication_eligible: false, retain both raw values plus their arithmetic mean and range, and state the two-lifecycle caveat. Scale results must not be merged into the honed cross-engine cohort.

The source corpus is the pinned upstream Cohere 768-dimensional 1M-vector fixture. Circus verifies its immutable source files, computes exact cosine top-k ground truth against deterministic prefix sizes, and writes explicit custom-dataset manifests. A streaming iterator is bounded by the manifest’s declared prefix size; a smaller point cannot consume rows belonging to a larger shared Parquet file.

Each point/trial is an immutable publication-cache cell with workload-specific dataset identity. A cache miss receives one fresh retry, never an in-place resume. Only a complete positive lifecycle or a classified held-out capacity/rejection result can be promoted. The canonical diagnostic artifact retains every cell’s receipt/cache reference, query-partition digest, selected search effort, request-error evidence, memory timeline, final allocated volume, and insert_duration + optimize_duration build time.

Output and retention

The canonical artifact contains report.md, normalized result.json, provenance, a receipt, selected settings, the raw capped-concurrency JSON/CSV diagnostic, memory rows and timelines, logs, supervisor/status evidence, and upstream result JSON. Standard publication runs VectorDBBench’s increasing concurrency progression at 1, 5, 10, 20, and 30 clients. Normalized rows report peak QPS and its concurrency as well as qps_c30 and p99_c30_ms.

A lifecycle that completes the selected-setting, held-out capped sweep but has explicit nonzero request rejections is retained as a classified capacity terminal negative. It is reusable only with the exact concurrency schedule, query partition, selected setting, full memory timeline, quiet-host assertion, logs, and supervisor evidence. Setup/index/configuration failures, crashes, timeouts, partial lifecycles, and unknown failures are never reusable publication cells. Database volumes are deleted only after the verified final capture unless an operator explicitly opts into retaining them for debugging. The retained storage.json receipt and complete optional-trial evidence are results; the live volume is not.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: vector-vdbbench-50k
Domain: vector
Source: https://github.com/antflydb/VectorDBBench
Upstream: https://github.com/zilliztech/VectorDBBench
Known Antfly versions: v0.1.3, v0.2.*
Expected duration: 480 min
Enabled: yes

Benchmark parameters

{"concurrency_duration_seconds":30,"concurrency_schedule":[1,5,10,20,30],"query_count":800,"query_partition_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","search_parameter":"search_effort","search_value":0.425}

Trial variation

[{"engine":"Antfly","ranges":{"demand_working_set":{"maximum":5870,"mean":4820.9,"minimum":4071.6,"unit":"MB"},"final_database_volume":{"maximum":354.254848,"mean":353.247232,"minimum":352.587776,"unit":"MB"},"p99_at_30_concurrent":{"maximum":70,"mean":69.233,"minimum":68.2,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Chroma","ranges":{"demand_working_set":{"maximum":466.7,"mean":463.3,"minimum":460.9,"unit":"MB"},"final_database_volume":{"maximum":332.488704,"mean":332.485973,"minimum":332.480512,"unit":"MB"},"p99_at_30_concurrent":{"maximum":179,"mean":177.6,"minimum":175.3,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Elasticsearch","ranges":{"demand_working_set":{"maximum":1952.7,"mean":1941,"minimum":1934.8,"unit":"MB"},"final_database_volume":{"maximum":312.082432,"mean":312.067413,"minimum":312.049664,"unit":"MB"},"p99_at_30_concurrent":{"maximum":15.1,"mean":14.467,"minimum":14,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Milvus 2.6.22","ranges":{"demand_working_set":{"maximum":2766.6,"mean":2666.2,"minimum":2563.6,"unit":"MB"},"final_database_volume":{"maximum":2393.47712,"mean":2393.459371,"minimum":2393.432064,"unit":"MB"},"p99_at_30_concurrent":{"maximum":11.4,"mean":11.233,"minimum":11.1,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"pgvector","ranges":{"demand_working_set":{"maximum":2303.8,"mean":2303.7,"minimum":2303.7,"unit":"MB"},"final_database_volume":{"maximum":1523.970048,"mean":1523.967317,"minimum":1523.965952,"unit":"MB"},"p99_at_30_concurrent":{"maximum":33.1,"mean":32.033,"minimum":31.4,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Weaviate","ranges":{"demand_working_set":{"maximum":977.9,"mean":953.8,"minimum":936.6,"unit":"MB"},"final_database_volume":{"maximum":391.55712,"mean":391.514795,"minimum":391.491584,"unit":"MB"},"p99_at_30_concurrent":{"maximum":89.6,"mean":86.967,"minimum":85.3,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Milvus 3.0.0","ranges":{"demand_working_set":{"maximum":2949.6,"mean":2819,"minimum":2719.7,"unit":"MB"},"final_database_volume":{"maximum":2489.167872,"mean":2489.156949,"minimum":2489.143296,"unit":"MB"},"p99_at_30_concurrent":{"maximum":12,"mean":11.667,"minimum":11.4,"unit":"ms"}},"workload":"1536d-50K"}]

Peer versions

Chroma1.5.9
Elasticsearch9.5.2
Milvus 2.6.222.6.22
pgvector0.8.6 (PostgreSQL 16)
Weaviate1.39.2
Milvus 3.0.03.0.0

Evidence digests

Receipt: 73fc49859cec7b58f9e7593a995e01c91ff2bfa790188e28b05ec34a1a3adbf9
Aggregate: e9ec0062de305e6d42c0c70d33327426df2967c7049aac01b470d2596f08358f
Provenance: 6e46e7e398eb899e2e4db51cd23763420dadb899fcfa47b1710db9d3bcecd9e5
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Peer cohort evidence · 1536d-50K

Completed: 28 Aug 2026 · 14:56 UTC
Fresh lifecycles: 18
Peer variants: 6
Receipt: 782e8e470765fae9914945e67c9da3f3e82a1b38571ebe6e5940228a8248601c
Aggregate: 0eb49aba01e15e065ec68b1667beb2c41a580cde5dab2cb928b6f8c01c16daf4
Provenance: 700d49bf35039da97e9b5366d4a4e6ec4c417fdfdb5fd720b980349a538d68c3

Selected measurement

Completed: 27 Aug 2026 · 05:50 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 33m 09s
Platform: linux · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Peer cohort · 1536d-50K

Completed: 28 Aug 2026 · 14:56 UTC
Variants: 6 independently executed peers
Lifecycles: 18 fresh
Compatibility: Same machine · protocol, input, and environment identity-bound

Peak throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 1536d-50K · three fresh lifecycles · Higher is better

Serial query p95

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 1536d-50K · three fresh lifecycles · Lower is better

Demand working set

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 1536d-50K · three fresh lifecycles · Lower is better

Final database volume

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 1536d-50K · three fresh lifecycles · Lower is better

Performance, quality, and resource evidence

Antfly v0.2.1-rc1 · 279381f7

Engine	Workload	Recall	nDCG	Peak QPS queries/s	p95 ms	Demand working set MB	Cache-inclusive peak MB	Final database volume MB	Trials
Antfly	1536d-50K	0.9580	0.9630	1,557.1	2.600	4,820.9 MB	5,347.2 MB	353.2 MB	3
Chroma	1536d-50K	0.9870	0.9890	180.9	8.067	463.3 MB	812.9 MB	332.5 MB	3
Elasticsearch	1536d-50K	0.9560	0.9640	4,434.6	2.033	1,941.0 MB	4,385.1 MB	312.1 MB	3
Milvus 2.6.22	1536d-50K	0.9630	0.9700	5,293.1	1.600	2,666.2 MB	5,426.1 MB	2,393.5 MB	3
pgvector	1536d-50K	0.9690	0.9750	2,721.0	4.167	2,303.7 MB	3,533.5 MB	1,524.0 MB	3
Weaviate	1536d-50K	0.9550	0.9630	1,008.8	6.300	953.8 MB	1,511.3 MB	391.5 MB	3
Milvus 3.0.0	1536d-50K	0.9630	0.9700	5,019.7	1.700	2,819.0 MB	5,557.7 MB	2,489.2 MB	3

vector-vdbbench-1m

complete

1M

Fresh three-lifecycle Antfly and same-machine peer evidence for the 768d/1M vector comparison.

Methodology scope · protocol · interpretation

VectorDBBench methodology

Antfly subject and execution lane

Workloads and query protocol

The publication matrix contains six engines on both upstream workloads:

Performance1536D50K: 50,000 vectors with 1,536 dimensions.
Performance768D1M: 1,000,000 vectors with 768 dimensions.

Each engine/workload trial has one fresh service and one fixed index lifecycle:

Start lifecycle memory sampling before indexing.
Load and build the index once using fixed construction parameters.
Sweep only the query-time parameter on the 200-query calibration partition.
Freeze the lowest setting reaching recall 0.955. If none qualifies, retain the highest-recall setting and mark it below the qualification floor.
Without rebuilding, run the selected setting on the untouched 800-query partition at 1, 5, 10, 20, and 30 concurrent clients for 30 seconds each, followed by the upstream serial quality and latency pass.
Capture one diagnostic post-workload storage observation, gracefully stop and verify the service, capture final storage, retain the artifacts, and only then reclaim the engine’s data.

Tuned and fixed parameters

Engine	Fixed index/build parameters	Query-time selection
Antfly	AKNN, one shard, packed float32 queries	`search_effort` sweep
Milvus	HNSW `M=16`, `efConstruction=256`	`ef_search` sweep
Chroma	HNSW `M=16`, `efConstruction=256`	fixed `ef_search=256` qualification
Weaviate	HNSW `maxConnections=16`, `efConstruction=256`	`ef` sweep
pgvector	HNSW `m=16`, `ef_construction=128`	`ef_search` sweep
Elasticsearch	HNSW `m=16`, `ef_construction=100`	`num_candidates` sweep

Memory

Persistent database volume

Internal Antfly scale diagnostic

Output and retention

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: vector-vdbbench-1m
Domain: vector
Source: https://github.com/antflydb/VectorDBBench
Upstream: https://github.com/zilliztech/VectorDBBench
Known Antfly versions: v0.1.3, v0.2.*
Expected duration: 480 min
Enabled: yes

Benchmark parameters

{"concurrency_duration_seconds":30,"concurrency_schedule":[1,5,10,20,30],"query_count":800,"query_partition_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","search_parameter":"search_effort","search_value":0.45}

Trial variation

[{"engine":"Antfly","ranges":{"demand_working_set":{"maximum":16381.9,"mean":15415.5,"minimum":14760.8,"unit":"MB"},"final_database_volume":{"maximum":3823.136768,"mean":3811.874133,"minimum":3806.12608,"unit":"MB"},"p99_at_30_concurrent":{"maximum":401.2,"mean":399.5,"minimum":396.3,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Chroma","ranges":{"demand_working_set":{"maximum":3699.1,"mean":3693.6,"minimum":3688.8,"unit":"MB"},"final_database_volume":{"maximum":3460.194304,"mean":3459.731456,"minimum":3458.838528,"unit":"MB"},"p99_at_30_concurrent":{"maximum":136.1,"mean":135.567,"minimum":134.6,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Elasticsearch","ranges":{"demand_working_set":{"maximum":2135.8,"mean":2095.2,"minimum":2029.8,"unit":"MB"},"final_database_volume":{"maximum":3148.750848,"mean":3148.500992,"minimum":3148.361728,"unit":"MB"},"p99_at_30_concurrent":{"maximum":21.6,"mean":20.967,"minimum":19.7,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Milvus 2.6.22","ranges":{"demand_working_set":{"maximum":13999.4,"mean":13920.6,"minimum":13837.1,"unit":"MB"},"final_database_volume":{"maximum":25669.378048,"mean":25642.164224,"minimum":25626.836992,"unit":"MB"},"p99_at_30_concurrent":{"maximum":13.7,"mean":13.633,"minimum":13.5,"unit":"ms"}},"workload":"768d-1M"},{"engine":"pgvector","ranges":{"demand_working_set":{"maximum":2318.6,"mean":2317.3,"minimum":2316.6,"unit":"MB"},"final_database_volume":{"maximum":9329.364992,"mean":9329.318571,"minimum":9329.27488,"unit":"MB"},"p99_at_30_concurrent":{"maximum":76.6,"mean":72.867,"minimum":69.9,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Weaviate","ranges":{"demand_working_set":{"maximum":7370.7,"mean":7345.1,"minimum":7331,"unit":"MB"},"final_database_volume":{"maximum":3955.232768,"mean":3947.47904,"minimum":3942.985728,"unit":"MB"},"p99_at_30_concurrent":{"maximum":68.3,"mean":67.2,"minimum":65.7,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Milvus 3.0.0","ranges":{"demand_working_set":{"maximum":13551.8,"mean":13262.6,"minimum":13016.7,"unit":"MB"},"final_database_volume":{"maximum":35112.308736,"mean":33718.551893,"minimum":33003.88864,"unit":"MB"},"p99_at_30_concurrent":{"maximum":14.4,"mean":14.233,"minimum":14.1,"unit":"ms"}},"workload":"768d-1M"}]

Peer versions

Chroma1.5.9
Elasticsearch9.5.2
Milvus 2.6.222.6.22
pgvector0.8.6 (PostgreSQL 16)
Weaviate1.39.2
Milvus 3.0.03.0.0

Evidence digests

Receipt: 13ddaa83b17224513328c8ceded53d2a9b5386bf55e8208aa02c00d735def9f5
Aggregate: f0adf45a9a4affe8dd5f72c87a3d20c9d8466e0f5e2fd0ff0b798c51aa29dd66
Provenance: 6725fa7083c788eadda90264c209daf27ca0f451037a5107c68f7aa1055a751c
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Peer cohort evidence · 768d-1M

Completed: 28 Aug 2026 · 15:20 UTC
Fresh lifecycles: 18
Peer variants: 6
Receipt: 4366569aa9cfe5f7e796c5eadc30a6d2ae610cde650543eb3546ddcb856dfebf
Aggregate: 828e2d97f299926c3bb1f39866db983ef61895793243ec02a569fe955507a197
Provenance: 4af875e079c7c2538ed378027ac14be10294f30a7761e8fe0300f885ea3a1ea4

Selected measurement

Completed: 27 Aug 2026 · 08:43 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 2h 51m
Platform: linux · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Peer cohort · 768d-1M

Completed: 28 Aug 2026 · 15:20 UTC
Variants: 6 independently executed peers
Lifecycles: 18 fresh
Compatibility: Same machine · protocol, input, and environment identity-bound

Peak throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 768d-1M · three fresh lifecycles · Higher is better

Serial query p95

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 768d-1M · three fresh lifecycles · Lower is better

Demand working set

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 768d-1M · three fresh lifecycles · Lower is better

Final database volume

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · 768d-1M · three fresh lifecycles · Lower is better

Performance, quality, and resource evidence

Antfly v0.2.1-rc1 · 279381f7

Engine	Workload	Recall	nDCG	Peak QPS queries/s	p95 ms	Demand working set MB	Cache-inclusive peak MB	Final database volume MB	Trials
Antfly	768d-1M	0.9670	0.9700	133.7	9.833	15,415.5 MB	22,425.4 MB	3,811.9 MB	3
Chroma	768d-1M	0.9560	0.9620	238.7	6.600	3,693.6 MB	7,289.8 MB	3,459.7 MB	3
Elasticsearch	768d-1M	0.9590	0.9610	3,110.8	3.233	2,095.2 MB	11,426.8 MB	3,148.5 MB	3
Milvus 2.6.22	768d-1M	0.9610	0.9660	4,085.9	2.200	13,920.6 MB	21,605.9 MB	25,642.2 MB	3
pgvector	768d-1M	0.9660	0.9700	955.8	10.767	2,317.3 MB	11,974.9 MB	9,329.3 MB	3
Weaviate	768d-1M	0.9560	0.9620	1,300.8	5.700	7,345.1 MB	13,791.8 MB	3,947.5 MB	3
Milvus 3.0.0	768d-1M	0.9610	0.9670	3,898.5	2.367	13,262.6 MB	21,631.0 MB	33,718.6 MB	3

vector-filtered-ann

partial

Filtered ANN

Six Antfly attempts completed filtered search measurement but failed the required post-workload index-policy gate.

Methodology scope · protocol · interpretation

Filtered ANN Methodology

This benchmark is intentionally separate from the unfiltered VectorDBBench matrix. The honed publication workload measures equality-filtered, tenant-scoped vector search; the broader portable matrix and gauntlet retain numeric-filter diagnostics. All use exact filter-aware ground truth and the same database adapters and service topology as the regular vector benchmark.

Portable 50K matrix

The Circus-owned portable matrix loads each engine once with the vector, integer ID, and a deterministic categorical bucket, then runs six scenarios:

numeric tails retaining 10%, 2%, 1%, and 0.2% of the corpus;
a fixed categorical bucket retaining 1% / 500 scattered rows; and
a rotating categorical bucket, changed per query, also retaining 1%.

Rows use bucket_{id % 100}. The fixed case selects bucket_037; the rotating case selects (query_index * 37 + 11) % 100, which visits every bucket ten times over the 1,000-query test set. Each scenario has exact cosine top-100 ground truth. The 0.2% numeric point deliberately leaves exactly 100 eligible rows and is the minimum valid selectivity for top-100.

Each database performs three complete search sweeps, with scenario order deterministically shuffled per database and repetition. Reports use the median with min-max whiskers and show Recall@100, peak QPS, and p99 at the concurrency producing peak QPS. Performance execution fails closed while unrelated benchmark processes or containers own the shared host.

Honed profile

Dataset: Small OpenAI, 50,000 vectors, 1,536 dimensions, cosine distance
Tenant model: 100 equal buckets defined by bucket_{id % 100}, with 500 vectors per tenant
Filter: categorical equality on the tenant bucket selected for each query
Quality schedule: (source_query_index * 37 + 11) % 100, which visits every tenant and remains stable across the deterministic calibration/publication partition
Throughput schedule: every worker rotates across tenant buckets so the measurement is not a repeated hot-filter cache lookup
Top-k: 100
Engines: Antfly, Weaviate, Elasticsearch, Milvus, pgvector, and Chroma. This keeps the honed comparison focused on the source benchmark target, a mainstream hybrid-search engine, a vector-native engine, and the ubiquitous Postgres extension baseline, while retaining the popular embedded/developer-oriented vector store.
Lifecycle: three fresh indexes per engine. Each index is tuned only on the deterministic 200-query calibration partition, then measured on the disjoint 800-query holdout. The same three lifecycles define performance, memory, and storage aggregates.
Storage: each lifecycle uses one newly labeled Docker local volume at the engine’s complete persistence root. Capture whole-volume allocated/apparent bytes once at empty, ready, and post_workload, then at the verified post-stop final barrier before reclamation. Host paths are ineligible. Retain every value and report the arithmetic mean and range across all three lifecycles.
Search mode: serial latency/recall plus QPS at concurrency 1, 8, and 32 for 10 seconds each.

Antfly calibration-only topology qualification

Before the three publication lifecycles, Antfly alone qualifies shard counts 1, 2, and 4 on fresh indexes using only the same deterministic 200-query calibration partition. Each topology sweeps the preregistered search-effort ladder 0, 0.025, 0.05, …, 0.35; the lowest effort reaching recall 0.955 is selected. Holdout queries, publication concurrency, and publication metrics are forbidden inputs to this topology decision.

When all three topology candidates qualify, the selector minimizes calibration (p95, p99, shard count) in that order. If any candidate is quality-negative or operationally incomplete after one retry, publication uses the preregistered one-shard control rather than selecting from a partial successful set. The chosen shard count is frozen across all three Antfly publication lifecycles and included in their cache identity.

Gauntlet profile

Dataset: Medium OpenAI, 500,000 vectors, 1,536 dimensions, cosine distance
Filter: id >= 495,000, leaving 1% / 5,000 candidates
Engines: the same six-engine set. Qdrant is not part of either profile. Quickwit is a lexical/log-search engine in this suite and does not expose a comparable VectorDBBench filtered-ANN adapter.
Metrics: same as the honed profile, with the exhaustive concurrency 1, 5, 10, 20, 30, 40, 60, and 80 sweep at 30 seconds per point
Peak-memory sampling is enabled by the enclosing gauntlet benchmark profile

Correctness and fairness

The honed adapters implement categorical equality against the same labels field and load the same 100-bucket assignment. Its exact cosine top-100 ground truth uses the tenant selected for each source query. The query schedule is applied before the deterministic 200/800 partition so each retained vector is paired with the same tenant in calibration, publication, and independent reproduction.

The gauntlet’s numeric adapters implement inclusive >= semantics. VectorDBBench compares each result to neighbors computed within the surviving candidate set, so unfiltered or post-filtered top-k results lose recall rather than appearing artificially fast. Results report recall, nDCG, peak concurrent QPS, the concurrency and p99 latency at that peak, p95/p99 serial latency, insert time, and total load time. Index and query knobs are recorded in the command logs.

Filtered services use their own container names, data roots, result labels, logs, status rows, and host ports. Antfly defaults to port 28081 so it cannot attach to or stop the regular/full-text Antfly service on port 28080.

Canonical output retains every storage.json, barrier sample, declared volume, trial value, arithmetic mean, and range under raw/storage. Final allocated bytes are the headline; apparent bytes and engine-reported index sizes are diagnostics. Filtered peer storage appears in the comparative review only when the existing review-inclusion gate passes, while canonical/raw evidence is always retained.

The portable matrix still does not constitute a complete claim about filtered search. Planned follow-ups add vector-correlated categories, skewed hot/cold filter churn, conjunctions/exclusions, concurrent-ingest interference, matched-recall tuning, and 500K/1M datasets.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: vector-filtered-ann
Domain: vector/filtered-ann
Source: https://github.com/antflydb/VectorDBBench
Upstream: https://github.com/zilliztech/VectorDBBench
Known Antfly versions: v0.2.*
Expected duration: 270 min
Enabled: yes

Benchmark parameters

{"attempted_lifecycles":6,"completed_lifecycles":0,"diagnostic_policy":"Attempt-level diagnostics only. No aggregate, ranking, ratio, or comparative performance claim survives the required post-workload index-policy gate.","http_status":404,"workload":"tenant-scoped-1536d-50K-1pct"}

Peer versions

Peer versions were not recorded in the published result fields.

Evidence digests

Receipt: ade5670d389a388c06ec2392d6a2b7080896926c5484c17b8770e7ac592d0fe3
Aggregate: bb1bfb97152c4aa586b3685a8f080ccd81cdcb62411e713f3c33257ee3653066
Provenance: 047c7c5a2f715a3593a07e2288e5568d4672dbff346ba2ddb062c2ac0abb96e3
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Scope notes

Attempt values are diagnostic only; no aggregate, ranking, ratio, or comparative claim survives the lifecycle gate.

Selected partial evidence

Completed: 27 Aug 2026 · 10:38 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 49m 43s
Platform: linux · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Attempt values are diagnostic only; no aggregate, ranking, ratio, or comparative claim survives the lifecycle gate.

Filtered ANN terminal result

Antfly v0.2.1-rc1 · 279381f7

Stage	Attempts	Status	Observed
post-workload-index-policy	6	partial	The required retained default full-text index was absent after every otherwise completed filtered ANN workload attempt.

Unqualified attempt-level diagnostics

Antfly v0.2.1-rc1 · 279381f7

Trial · attempt	Recall	nDCG	Peak QPS	QPS @ C32	p99 @ C32 ms	Serial p99 ms	Demand MB	Final volume MB	Lifecycle gate
T1 · A1	1.0000	1.0000	2,680.3	1,542.0	50.2	2.1	7,196.9	353.3	HTTP 404 after measurement
T1 · A2	1.0000	1.0000	2,806.7	1,578.9	43.5	2.1	6,018.5	354.4	HTTP 404 after measurement
T2 · A1	1.0000	1.0000	2,791.2	1,586.9	43.2	2.4	8,419.2	353.8	HTTP 404 after measurement
T2 · A2	1.0000	1.0000	2,935.6	1,740.1	39.6	2.1	6,174.6	353.8	HTTP 404 after measurement
T3 · A1	1.0000	1.0000	2,794.1	1,662.2	40.2	2.2	6,187.6	355.5	HTTP 404 after measurement
T3 · A2	1.0000	1.0000	2,809.5	1,656.8	40.3	2.2	5,082.0	354.0	HTTP 404 after measurement

III

fulltext-search-benchmark-game

partial

Search Benchmark Game

Fresh Antfly results over 250,000 Wikipedia documents, four query classes, two result windows, and two concurrency levels.

Methodology scope · protocol · interpretation

Search Benchmark Game Methodology

What this lane measures

This is an end-to-end, self-hosted database comparison over the first 250,000 documents in the Search Benchmark Game Wikipedia corpus. Every timed adapter uses a persistent database connection and requests ranked document IDs without stored document bodies. Tantivy is deliberately absent: it is an embedded search library, while every measured row here is a database service.

The run uses the 902 upstream term, union, intersection, and phrase queries. The 60 mixed required/optional and negated Lucene-syntax queries remain excluded because Antfly cannot represent their matching and scoring semantics directly. If the remaining workload proves too Lucene-oriented for database comparison, that will motivate a separate benchmark rather than further changing this one.

Database matrix and capability policy

Antfly 0.2.0-rc.18: term, union, intersection, phrase
Elasticsearch 8.15.3: term, union, intersection, phrase
Weaviate 1.38.2: term, union, intersection; exact phrase unsupported
Quickwit 0.8.2: term, union, intersection, phrase
PostgreSQL 16 from the pgvector image: term, union, intersection, phrase; pgvector is not involved in lexical execution
Milvus 2.5.8: retained as unsupported because its BM25 raw-text input is a VARCHAR capped at 65,535 bytes, while 20 corpus documents exceed that limit and the maximum is 202,798 bytes
Chroma self-hosted: retained as unsupported because its text surface provides containment/regex filtering rather than ranked lexical top-k retrieval; the richer Chroma Search API is cloud-only

Missing capabilities are never emulated client-side. Public output uses —* for a product limitation and failed† for a setup or execution failure, with the reason preserved in JSON and rendered beside the table.

Execution protocol

The benchmark runs TOP_10 and TOP_100 at concurrency 1 and 8. Each cell has a five-second warmup followed by two measured rounds. Concurrency uses one persistent connection per worker; individual request latency and aggregate QPS are both recorded. Only one database is timed at once. Indexing is sequential, uses the unchanged corpus, and uses native bulk paths (COPY for PostgreSQL, bulk APIs elsewhere).

Antfly uses a prebuilt release binary; the benchmark never builds Zig. It runs one shard and sends fields: []. Elasticsearch uses one primary shard, zero replicas, a 1 GiB heap, and a one-segment force merge. Weaviate runs one node with vectorization disabled. Quickwit runs one node with a positional, fieldnorm-enabled text index and BM25 score ordering; only external IDs are stored. PostgreSQL uses a stored tsvector, GIN index, and ts_rank_cd ordering.

Where configurable, adapters approximate Antfly’s fixed Bleve analyzer with standard/word tokenization, lowercase normalization, and English stopwords. Quickwit’s built-in default tokenizer lowercases but does not expose a matching stop-word filter. Residual analyzer and scoring differences are reported through an untimed top-10 ID/count diagnostic; exact cross-engine ranking is not a pass condition.

Interpretation

The headline table shows TOP_10 concurrency-8 p95 latency by query class so partial capability is visible. Full artifacts retain p50/p95/p99, QPS, sample counts, status, and reasons for TOP_10/TOP_100 at both concurrency levels. This benchmark has no relevance judgments, so it supports performance and capability claims only—not claims that one engine ranks documents more correctly.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: fulltext-search-benchmark-game
Domain: full-text
Source: https://github.com/quickwit-oss/search-benchmark-game
Known Antfly versions: v0.1.3, v0.2.*
Enabled: yes

Benchmark parameters

{"concurrency":[1,8],"corpus_documents":250000,"engine_details":{},"query_classes":["term","union","intersection","phrase"],"result_windows":["TOP_10","TOP_100"]}

Peer versions

Elasticsearch8.15.3
Weaviate1.38.2
Quickwit0.8.2
PostgreSQL16
Milvus2.5.8

Evidence digests

Receipt: 497cd5bb71933a4870d77594d3a9077af7a3a873d7c1b03e20ab84c8f0a9fb66
Aggregate: 0d60283a261325d0d59a95e51fb47259572da07982813583531dc65b75607a17
Provenance: 121237470c0ad51498a070266c901994203dde752a1344072f5ce967c4f25514
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Selected partial evidence

Completed: 27 Aug 2026 · 10:42 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 2m 54s
Platform: darwin · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Top 10 · concurrency 1

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · Apple M4 Max · 250,000 documents · 902-query workload · QPS, higher is better

Top 10 · concurrency 8

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · Apple M4 Max · 250,000 documents · 902-query workload · QPS, higher is better

Top 100 · concurrency 1

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · Apple M4 Max · 250,000 documents · 902-query workload · QPS, higher is better

Top 100 · concurrency 8

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · Apple M4 Max · 250,000 documents · 902-query workload · QPS, higher is better

Search behavior by query class and result window

Antfly v0.2.1-rc1 · 279381f7

Engine	Query class	Result window	Concurrency	Queries	QPS	p95 ms	Status
Antfly	term	TOP_10	1	1	5,972.4	0.121	measured
Antfly	union	TOP_10	1	301	1,369.6	2.739	measured
Antfly	intersection	TOP_10	1	300	2,174.5	1.482	measured
Antfly	phrase	TOP_10	1	300	3,940.1	0.465	measured
Antfly	term	TOP_10	8	1	6,111.5	0.124	measured
Antfly	union	TOP_10	8	301	8,039.7	3.478	measured
Antfly	intersection	TOP_10	8	300	12,925.1	1.655	measured
Antfly	phrase	TOP_10	8	300	21,324.5	0.720	measured
Antfly	term	TOP_100	1	1	6,149.1	0.123	measured
Antfly	union	TOP_100	1	301	1,164.7	3.057	measured
Antfly	intersection	TOP_100	1	300	2,049.7	1.675	measured
Antfly	phrase	TOP_100	1	300	3,618.4	0.507	measured
Antfly	term	TOP_100	8	1	4,417.0	0.221	measured
Antfly	union	TOP_100	8	301	7,020.0	3.605	measured
Antfly	intersection	TOP_100	8	300	12,444.2	1.736	measured
Antfly	phrase	TOP_100	8	300	20,502.3	0.722	measured
weaviate-1.38.2	term	TOP_10	1	1	586.6	1.829	measured
weaviate-1.38.2	union	TOP_10	1	301	1,151.5	1.177	measured
weaviate-1.38.2	intersection	TOP_10	1	300	1,103.3	1.196	measured
weaviate-1.38.2	phrase	TOP_10	1	300	Unsupported	Unsupported	unsupported
quickwit-0.8.2	term	TOP_10	1	1	318.3	3.104	measured
quickwit-0.8.2	union	TOP_10	1	301	326.7	3.357	measured
quickwit-0.8.2	intersection	TOP_10	1	300	381.9	3.282	measured
quickwit-0.8.2	phrase	TOP_10	1	300	586.3	3.037	measured
postgresql-16	term	TOP_10	1	1	3,472.5	0.240	measured
postgresql-16	union	TOP_10	1	301	13.8	215.429	measured
postgresql-16	intersection	TOP_10	1	300	511.4	6.565	measured
postgresql-16	phrase	TOP_10	1	300	534.6	5.612	measured
milvus-2.5.8	term	TOP_10	1	1	Unsupported	Unsupported	unsupported
milvus-2.5.8	union	TOP_10	1	301	Unsupported	Unsupported	unsupported
milvus-2.5.8	intersection	TOP_10	1	300	Unsupported	Unsupported	unsupported
milvus-2.5.8	phrase	TOP_10	1	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	term	TOP_10	1	1	Unsupported	Unsupported	unsupported
chroma-self-hosted	union	TOP_10	1	301	Unsupported	Unsupported	unsupported
chroma-self-hosted	intersection	TOP_10	1	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	phrase	TOP_10	1	300	Unsupported	Unsupported	unsupported
weaviate-1.38.2	term	TOP_10	8	1	905.6	0.909	measured
weaviate-1.38.2	union	TOP_10	8	301	6,874.6	1.537	measured
weaviate-1.38.2	intersection	TOP_10	8	300	6,955.8	1.690	measured
weaviate-1.38.2	phrase	TOP_10	8	300	Unsupported	Unsupported	unsupported
quickwit-0.8.2	term	TOP_10	8	1	296	3.286	measured
quickwit-0.8.2	union	TOP_10	8	301	1,762	5.956	measured
quickwit-0.8.2	intersection	TOP_10	8	300	2,019	5.318	measured
quickwit-0.8.2	phrase	TOP_10	8	300	2,460.8	5.227	measured
postgresql-16	term	TOP_10	8	1	3,481	0.239	measured
postgresql-16	union	TOP_10	8	301	76.2	311.210	measured
postgresql-16	intersection	TOP_10	8	300	2,268.7	10.292	measured
postgresql-16	phrase	TOP_10	8	300	2,411.9	8.667	measured
milvus-2.5.8	term	TOP_10	8	1	Unsupported	Unsupported	unsupported
milvus-2.5.8	union	TOP_10	8	301	Unsupported	Unsupported	unsupported
milvus-2.5.8	intersection	TOP_10	8	300	Unsupported	Unsupported	unsupported
milvus-2.5.8	phrase	TOP_10	8	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	term	TOP_10	8	1	Unsupported	Unsupported	unsupported
chroma-self-hosted	union	TOP_10	8	301	Unsupported	Unsupported	unsupported
chroma-self-hosted	intersection	TOP_10	8	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	phrase	TOP_10	8	300	Unsupported	Unsupported	unsupported
weaviate-1.38.2	term	TOP_100	1	1	837	0.963	measured
weaviate-1.38.2	union	TOP_100	1	301	621.5	2.115	measured
weaviate-1.38.2	intersection	TOP_100	1	300	797.2	1.984	measured
weaviate-1.38.2	phrase	TOP_100	1	300	Unsupported	Unsupported	unsupported
quickwit-0.8.2	term	TOP_100	1	1	49.2	20.797	measured
quickwit-0.8.2	union	TOP_100	1	301	53.4	21.169	measured
quickwit-0.8.2	intersection	TOP_100	1	300	111.5	19.710	measured
quickwit-0.8.2	phrase	TOP_100	1	300	327.4	14.740	measured
postgresql-16	term	TOP_100	1	1	3,519.6	0.236	measured
postgresql-16	union	TOP_100	1	301	13.6	220.023	measured
postgresql-16	intersection	TOP_100	1	300	482.5	6.983	measured
postgresql-16	phrase	TOP_100	1	300	529.3	5.977	measured
milvus-2.5.8	term	TOP_100	1	1	Unsupported	Unsupported	unsupported
milvus-2.5.8	union	TOP_100	1	301	Unsupported	Unsupported	unsupported
milvus-2.5.8	intersection	TOP_100	1	300	Unsupported	Unsupported	unsupported
milvus-2.5.8	phrase	TOP_100	1	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	term	TOP_100	1	1	Unsupported	Unsupported	unsupported
chroma-self-hosted	union	TOP_100	1	301	Unsupported	Unsupported	unsupported
chroma-self-hosted	intersection	TOP_100	1	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	phrase	TOP_100	1	300	Unsupported	Unsupported	unsupported
weaviate-1.38.2	term	TOP_100	8	1	2,046.7	0.448	measured
weaviate-1.38.2	union	TOP_100	8	301	3,145.7	4.015	measured
weaviate-1.38.2	intersection	TOP_100	8	300	4,033.2	3.467	measured
weaviate-1.38.2	phrase	TOP_100	8	300	Unsupported	Unsupported	unsupported
quickwit-0.8.2	term	TOP_100	8	1	50.7	19.463	measured
quickwit-0.8.2	union	TOP_100	8	301	373	24.984	measured
quickwit-0.8.2	intersection	TOP_100	8	300	749.4	21.959	measured
quickwit-0.8.2	phrase	TOP_100	8	300	1,783.1	17.652	measured
postgresql-16	term	TOP_100	8	1	3,285	0.265	measured
postgresql-16	union	TOP_100	8	301	73.9	322.166	measured
postgresql-16	intersection	TOP_100	8	300	2,309.4	10.066	measured
postgresql-16	phrase	TOP_100	8	300	2,328.8	8.822	measured
milvus-2.5.8	term	TOP_100	8	1	Unsupported	Unsupported	unsupported
milvus-2.5.8	union	TOP_100	8	301	Unsupported	Unsupported	unsupported
milvus-2.5.8	intersection	TOP_100	8	300	Unsupported	Unsupported	unsupported
milvus-2.5.8	phrase	TOP_100	8	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	term	TOP_100	8	1	Unsupported	Unsupported	unsupported
chroma-self-hosted	union	TOP_100	8	301	Unsupported	Unsupported	unsupported
chroma-self-hosted	intersection	TOP_100	8	300	Unsupported	Unsupported	unsupported
chroma-self-hosted	phrase	TOP_100	8	300	Unsupported	Unsupported	unsupported

inference-termite-circus

complete

Inference

Seven requested Antfly inference workloads, each with three fresh lifecycles and category-specific quality gates.

Methodology scope · protocol · interpretation

Antfly Inference Methodology

Subject contract

Inference is delegated to ../termite-circus. Each campaign receives exactly one explicit ANTFLY_BIN from the antfly-circus subject manifest. The harness records the executable’s version, architecture, and SHA-256; it does not fetch, install, update, or build Antfly. Comparing Antfly releases means comparing separate campaign bundles.

The rc.24 acceptance matrix covers generation, dense/sparse/multimodal embedding, reranking, chunking, GLiNER2 extraction, OCR/readers, and transcription. Rewriters are reported as known-incompatible rather than timed.

Profiles

smoke runs only the supplied Antfly binary, one lifecycle and one measured request, against one certified representative per working category. It is a compatibility gate, not publication evidence.
full adds category-appropriate local reference runtimes and certified model alternates. Each provider/model cell uses three fresh lifecycles, one warmup per lifecycle, and the tracked request count from matrix.json. Provider order alternates to reduce order bias.

Reference runtimes are local only: llama.cpp, Ollama, and vLLM-Metal for generation; ONNX Runtime and Sentence Transformers for embedding/reranking; GLiNER2 for extraction; Transformers for Florence-2; and faster-whisper plus whisper.cpp for transcription.

Provider-native formats are permitted when a runtime cannot load an identical artifact. Every result retains the logical model, exact artifact/revision, variant or quantization, and checksum. Such rows are not described as byte-identical comparisons.

Validity and metrics

Every cell must pass a deterministic category-specific quality gate before its latency or throughput is valid. The gates check semantic ordering, sparse output, cross-modal output, relevant-document ordering, valid chunk coverage, expected extraction fields, OCR/transcription word recovery, or a nonempty constrained completion. These fixtures detect broken inference; they are not a general model-quality leaderboard.

Warm request latency is reported as p50 and p95 within measured requests. Throughput uses successful measured requests only. Cold model-load/readiness time and raw provider output remain in the campaign artifacts rather than being mixed into warm latency.

Peak inference memory is the maximum of two concurrently sampled views:

rooted process-tree RSS, including provider helpers and model servers;
positive growth in system anonymous + compressor-occupied + wired memory from the pre-launch baseline.

Samples are retained every 200 ms. System growth is publishable only when the operator sets QUIET_HOST_ACK=1 on a quiet host; otherwise the memory value is diagnostic. Full reports retain each lifecycle and report the arithmetic mean and range rather than selecting the best trial.

Artifacts

The full antfly-circus command writes directly into $BASTILLE_OUT/data/inference-termite-circus/:

report.md: compact per-category tables;
result.json: canonical normalized rows consumed by antfly-circus;
provenance.json: subject, host, model, provider, and input fingerprints;
receipt.json: completeness, failures, and quality-gate status;
raw/: provider logs and memory timelines.

A failed campaign still retains the bundle and failed rows. Publication reads result.json and the receipt; Markdown is derived output and is never parsed as the source of truth.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: inference-termite-circus
Domain: inference
Source: https://github.com/antflydb/termite-circus
Known Antfly versions: v0.2.*
Enabled: yes

Benchmark parameters

{"completed_workloads":7,"failed_workloads":0,"generation_correction":{"host_budget_mb":8192,"model_policy":"text-only pull with projector disabled","reason":"The first run reused a stale projector-bearing cache despite a text-only pull declaration and omitted the host-tier budget required by the E4B model."},"memory_metric":null,"memory_unit":null,"model_artifacts":[{"artifact_sha256":"676c35070db6dbe52f93e9c864ee0fba4eddea94b9c875d9cb10daff453fbaee","benchmark_id":"generation-gemma4-e4b-qat","model":"google/gemma-4-E4B-it-qat-q4_0-gguf","variant":"QAT q4_0"},{"artifact_sha256":"449c8a2f7115155ad21291a887c8c7a136b0642f598a910018a37978c167d934","benchmark_id":"dense-bge-small","model":"BAAI/bge-small-en-v1.5","variant":"f32"},{"artifact_sha256":"a00e24a8e937f7f25434403396a3ac7ba76ffb4917b7403f629007168dbe611c","benchmark_id":"sparse-splade-tiny","model":"sparse-encoder-testing/splade-bert-tiny-nq-onnx","variant":"onnx"},{"artifact_sha256":"e27744c7a1716a21a6652de9b17d01b8773e782230faeefaae7947f7b7c7961a","benchmark_id":"rerank-mxbai-base","model":"mixedbread-ai/mxbai-rerank-base-v1","variant":"f32"},{"artifact_sha256":null,"benchmark_id":"chunk-fixed-bert","model":"fixed_bert","variant":"built-in"},{"artifact_sha256":"0eb40d545ed9a3849d76e49104ae6f3e1b4702bb96394ab1afff6b641dd9477b","benchmark_id":"extract-gliner2-base","model":"antflydb/gliner2-base-v1","variant":"f32"},{"artifact_sha256":"868e1ad35ae82615228c7d3503c9ed1e7310804b72b9a85ddc604af546ca5958","benchmark_id":"transcribe-whisper-tiny","model":"openai/whisper-tiny","variant":"f32"}],"planned_workloads":7,"source_memory_unit":null,"trials_per_workload":3,"workloads":[]}

Trial variation

[{"model":"ggml-org/gemma-4-E4B-it-GGUF","peak_memory":{"maximum":7726.65625,"mean":7434.364583333333,"minimum":7111,"unit":"MB"},"provider":"llama.cpp","variant":"Q4_0"},{"model":"ggml-org/gemma-4-E4B-it-GGUF","peak_memory":{"maximum":11632.296875,"mean":10929.015625,"minimum":10415.78125,"unit":"MB"},"provider":"ollama","variant":"Q4_0"},{"model":"ggml-org/gemma-4-E4B-it-GGUF","peak_memory":{"maximum":27998.640625,"mean":25524.130208333332,"minimum":20975.46875,"unit":"MB"},"provider":"vllm-metal","variant":"Q4_0"},{"model":"google/gemma-4-E4B-it-qat-q4_0-gguf","peak_memory":{"maximum":8404.0625,"mean":7605.09375,"minimum":7201.640625,"unit":"MB"},"provider":"llama.cpp","variant":"QAT q4_0"},{"model":"google/gemma-4-E4B-it-qat-q4_0-gguf","peak_memory":{"maximum":7804.75,"mean":7387.28125,"minimum":6806.921875,"unit":"MB"},"provider":"ollama","variant":"QAT q4_0"},{"model":"google/gemma-4-E4B-it-qat-q4_0-gguf","peak_memory":{"maximum":28953.59375,"mean":28351.203125,"minimum":28043.796875,"unit":"MB"},"provider":"vllm-metal","variant":"QAT q4_0"},{"model":"unsloth/Qwen3-1.7B-GGUF","peak_memory":{"maximum":5665.421875,"mean":5665.348958333333,"minimum":5665.25,"unit":"MB"},"provider":"llama.cpp","variant":"Q4_K_M"},{"model":"unsloth/Qwen3-1.7B-GGUF","peak_memory":{"maximum":5348.796875,"mean":5162.703125,"minimum":5059.1875,"unit":"MB"},"provider":"ollama","variant":"Q4_K_M"},{"model":"unsloth/Qwen3-1.7B-GGUF","peak_memory":{"maximum":28489.640625,"mean":25832.145833333332,"minimum":24216.65625,"unit":"MB"},"provider":"vllm-metal","variant":"Q4_K_M"},{"model":"BAAI/bge-small-en-v1.5","peak_memory":{"maximum":1027.65625,"mean":1024.2864583333333,"minimum":1022.125,"unit":"MB"},"provider":"onnxruntime","variant":"f32"},{"model":"BAAI/bge-small-en-v1.5","peak_memory":{"maximum":1734.4375,"mean":1296.7708333333333,"minimum":531.328125,"unit":"MB"},"provider":"sentence-transformers","variant":"f32"},{"model":"BAAI/bge-large-en-v1.5","peak_memory":{"maximum":3448.859375,"mean":3448.4322916666665,"minimum":3448.21875,"unit":"MB"},"provider":"onnxruntime","variant":"f32"},{"model":"BAAI/bge-large-en-v1.5","peak_memory":{"maximum":3779.78125,"mean":2297.2447916666665,"minimum":534.21875,"unit":"MB"},"provider":"sentence-transformers","variant":"f32"},{"model":"mixedbread-ai/mxbai-embed-large-v1","peak_memory":{"maximum":3447.1875,"mean":3446.8854166666665,"minimum":3446.671875,"unit":"MB"},"provider":"onnxruntime","variant":"f32"},{"model":"mixedbread-ai/mxbai-embed-large-v1","peak_memory":{"maximum":3188.9375,"mean":2633.1770833333335,"minimum":1609.125,"unit":"MB"},"provider":"sentence-transformers","variant":"f32"},{"model":"sparse-encoder-testing/splade-bert-tiny-nq-onnx","peak_memory":{"maximum":630.59375,"mean":628.4947916666666,"minimum":626.75,"unit":"MB"},"provider":"onnxruntime","variant":"onnx"},{"model":"sparse-encoder-testing/splade-bert-tiny-nq-onnx","peak_memory":{"maximum":1570.609375,"mean":1275.5416666666667,"minimum":707,"unit":"MB"},"provider":"sentence-transformers","variant":"onnx"},{"model":"mixedbread-ai/mxbai-rerank-base-v1","peak_memory":{"maximum":4774.015625,"mean":4766.505208333333,"minimum":4760.4375,"unit":"MB"},"provider":"onnxruntime","variant":"f32"},{"model":"mixedbread-ai/mxbai-rerank-base-v1","peak_memory":{"maximum":2340.1875,"mean":1827.171875,"minimum":1177.53125,"unit":"MB"},"provider":"sentence-transformers","variant":"f32"},{"model":"cross-encoder/ms-marco-MiniLM-L6-v2","peak_memory":{"maximum":962.453125,"mean":960.0885416666666,"minimum":958.375,"unit":"MB"},"provider":"onnxruntime","variant":"f32"},{"model":"cross-encoder/ms-marco-MiniLM-L6-v2","peak_memory":{"maximum":1560.453125,"mean":886.6770833333334,"minimum":521.984375,"unit":"MB"},"provider":"sentence-transformers","variant":"f32"},{"model":"fixed_bert","peak_memory":{"maximum":205.90625,"mean":205.58854166666666,"minimum":205.328125,"unit":"MB"},"provider":"tokenizers","variant":"built-in"},{"model":"fixed_bpe","peak_memory":{"maximum":206.234375,"mean":205.75,"minimum":205.5,"unit":"MB"},"provider":"tokenizers","variant":"built-in"},{"model":"mirth/chonky_mmbert_small_multilingual_1","peak_memory":{"maximum":635.234375,"mean":623.4427083333334,"minimum":600.0625,"unit":"MB"},"provider":"tokenizers","variant":"f32"},{"model":"antflydb/gliner2-base-v1","peak_memory":{"maximum":1779.546875,"mean":1401.1354166666667,"minimum":1211.796875,"unit":"MB"},"provider":"gliner2","variant":"f32"},{"model":"openai/whisper-tiny","peak_memory":{"maximum":498.125,"mean":394.1822916666667,"minimum":188.6875,"unit":"MB"},"provider":"whisper.cpp","variant":"f32"}]

Peer versions

llama.cppb9870 (2d973636e)
Ollama0.31.1
vLLM-Metal0.3.0.dev20260811042359 (vLLM 0.27.0+cpu)
ONNX Runtime1.28.0
Sentence Transformers5.6.1
Transformers tokenizer4.45.2
GLiNER21.3.2
Transformers4.45.2
faster-whisper1.2.1
whisper.cpp1.9.2

Evidence digests

Receipt: 54158183d66e79744b43b85ecb2bc35bc13bb6b3f82a8d40e1f1090541612b33
Aggregate: b73ae83efdd559551c83d6d44b60d0b946aead59ba7059071d58f2a7da0a3bc1
Provenance: 30df78e7b1169f356e6303a2c442357aae9c6161fa7090993806dd9d890116cf
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Selected measurement

Completed: 27 Aug 2026 · 10:44 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 57.3s
Platform: darwin · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Measurement group

complete

Generation

Three fresh Antfly lifecycles.

Details execution · comparison · provenance

generation-gemma4-e4b-qat

Completed: 27 Aug 2026 · 15:09 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 21.8s
Platform: darwin · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Receipt: 54158183d66e79744b43b85ecb2bc35bc13bb6b3f82a8d40e1f1090541612b33
Aggregate: b73ae83efdd559551c83d6d44b60d0b946aead59ba7059071d58f2a7da0a3bc1
Provenance: 30df78e7b1169f356e6303a2c442357aae9c6161fa7090993806dd9d890116cf
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Generation result

Antfly v0.2.1-rc1 · 279381f7

Model	Variant	Provider	p95 ms	Throughput ops/s	Quality	Peak memory MiB
google/gemma-4-E4B-it-qat-q4_0-gguf	QAT q4_0	antfly	916.5	1.10	1.000 pass	5,716.7
ggml-org/gemma-4-E4B-it-GGUF	Q4_0	llama.cpp	1,809.9	0.56	1 pass	7,434.4
ggml-org/gemma-4-E4B-it-GGUF	Q4_0	ollama	995.2	1.02	1 pass	10,929.0
ggml-org/gemma-4-E4B-it-GGUF	Q4_0	vllm-metal	801.8	1.26	1 pass	25,524.1
google/gemma-4-E4B-it-qat-q4_0-gguf	QAT q4_0	llama.cpp	1,870.7	0.56	1 pass	7,605.1
google/gemma-4-E4B-it-qat-q4_0-gguf	QAT q4_0	ollama	913.6	1.13	1 pass	7,387.3
google/gemma-4-E4B-it-qat-q4_0-gguf	QAT q4_0	vllm-metal	1,082.2	0.93	1 pass	28,351.2
unsloth/Qwen3-1.7B-GGUF	Q4_K_M	llama.cpp	897.7	1.14	1 pass	5,665.3
unsloth/Qwen3-1.7B-GGUF	Q4_K_M	ollama	412.1	2.55	1 pass	5,162.7
unsloth/Qwen3-1.7B-GGUF	Q4_K_M	vllm-metal	318.5	3.16	1 pass	25,832.1

Throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · google/gemma-4-E4B-it-qat-q4_0-gguf · three fresh lifecycles · Higher is better

Peak memory

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · google/gemma-4-E4B-it-qat-q4_0-gguf · three fresh lifecycles · Lower is better

Measurement group

complete

Dense Embedding

Three fresh Antfly lifecycles.

Dense Embedding result

Antfly v0.2.1-rc1 · 279381f7

Model	Variant	Provider	p95 ms	Throughput ops/s	Quality	Peak memory MiB
BAAI/bge-small-en-v1.5	f32	antfly	126.8	8.28	0.251 cosine	693.3
BAAI/bge-small-en-v1.5	f32	onnxruntime	13.4	80.00	0.207 cosine	1,024.3
BAAI/bge-small-en-v1.5	f32	sentence-transformers	8.8	141.59	0.207 cosine	1,296.8
BAAI/bge-large-en-v1.5	f32	onnxruntime	58.5	18.56	0.192 cosine	3,448.4
BAAI/bge-large-en-v1.5	f32	sentence-transformers	20.9	64.92	0.192 cosine	2,297.2
mixedbread-ai/mxbai-embed-large-v1	f32	onnxruntime	58.6	18.69	0.271 cosine	3,446.9
mixedbread-ai/mxbai-embed-large-v1	f32	sentence-transformers	20.2	65.97	0.271 cosine	2,633.2

Throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · BAAI/bge-small-en-v1.5 · three fresh lifecycles · Higher is better

Peak memory

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · BAAI/bge-small-en-v1.5 · three fresh lifecycles · Lower is better

Measurement group

complete

Sparse Embedding

Three fresh Antfly lifecycles.

Sparse Embedding result

Antfly v0.2.1-rc1 · 279381f7

Model	Variant	Provider	p95 ms	Throughput ops/s	Quality	Peak memory MiB
sparse-encoder-testing/splade-bert-tiny-nq-onnx	onnx	antfly	12.3	96.50	283.000 entries	535.3
sparse-encoder-testing/splade-bert-tiny-nq-onnx	onnx	onnxruntime	4.4	253.38	334 entries	628.5
sparse-encoder-testing/splade-bert-tiny-nq-onnx	onnx	sentence-transformers	10.5	130.63	334 entries	1,275.5

Throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · sparse-encoder-testing/splade-bert-tiny-nq-onnx · three fresh lifecycles · Higher is better

Peak memory

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · sparse-encoder-testing/splade-bert-tiny-nq-onnx · three fresh lifecycles · Lower is better

Measurement group

complete

Reranking

Three fresh Antfly lifecycles.

Reranking result

Antfly v0.2.1-rc1 · 279381f7

Model	Variant	Provider	p95 ms	Throughput ops/s	Quality	Peak memory MiB
mixedbread-ai/mxbai-rerank-base-v1	f32	antfly	118.5	9.42	1.000 pass	1,668.9
mixedbread-ai/mxbai-rerank-base-v1	f32	onnxruntime	52.5	19.39	1 pass	4,766.5
mixedbread-ai/mxbai-rerank-base-v1	f32	sentence-transformers	16.7	91.24	1 pass	1,827.2
cross-encoder/ms-marco-MiniLM-L6-v2	f32	onnxruntime	10.5	102.11	1 pass	960.1
cross-encoder/ms-marco-MiniLM-L6-v2	f32	sentence-transformers	6.8	199.59	1 pass	886.7

Throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · mixedbread-ai/mxbai-rerank-base-v1 · three fresh lifecycles · Higher is better

Peak memory

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · mixedbread-ai/mxbai-rerank-base-v1 · three fresh lifecycles · Lower is better

Measurement group

complete

Chunking

Three fresh Antfly lifecycles.

Chunking result

Antfly v0.2.1-rc1 · 279381f7

Model	Variant	Provider	p95 ms	Throughput ops/s	Quality	Peak memory MiB
fixed_bert	built-in	antfly	5.9	189.46	3.000 chunks	369.4
fixed_bert	built-in	tokenizers	0.1	21,789.55	3 chunks	205.6
fixed_bpe	built-in	tokenizers	0.1	23,741.61	3 chunks	205.8
mirth/chonky_mmbert_small_multilingual_1	f32	tokenizers	0.1	28,908.90	3 chunks	623.4

Throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · fixed_bert · three fresh lifecycles · Higher is better

Peak memory

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · fixed_bert · three fresh lifecycles · Lower is better

Measurement group

complete

Extraction

Three fresh Antfly lifecycles.

Extraction result

Antfly v0.2.1-rc1 · 279381f7

Model	Variant	Provider	p95 ms	Throughput ops/s	Quality	Peak memory MiB
antflydb/gliner2-base-v1	f32	antfly	41.5	25.70	1.000 recall	992.6
antflydb/gliner2-base-v1	f32	gliner2	29.0	35.48	1 recall	1,401.1

Throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · antflydb/gliner2-base-v1 · three fresh lifecycles · Higher is better

Peak memory

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · antflydb/gliner2-base-v1 · three fresh lifecycles · Lower is better

Measurement group

complete

Transcription

Three fresh Antfly lifecycles.

Transcription result

Antfly v0.2.1-rc1 · 279381f7

Model	Variant	Provider	p95 ms	Throughput ops/s	Quality	Peak memory MiB
openai/whisper-tiny	f32	antfly	1,589.1	0.63	1.000 word recall	940.0
openai/whisper-tiny	f32	faster-whisper	—	—	0.600 word recall	—
openai/whisper-tiny	f32	whisper.cpp	124.0	8.61	0.800 word recall	394.2

Throughput

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · openai/whisper-tiny · three fresh lifecycles · Higher is better

Peak memory

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · darwin-arm64 · openai/whisper-tiny · three fresh lifecycles · Lower is better

graph-benchmark

complete

Graph

Graph query performance, Pokec-small cross-engine diagnostics, official LSQB capability, and bounded SNB Interactive qualification.

Methodology scope · protocol · interpretation

Graph Database Benchmark v2 Methodology

Scope and provenance

This benchmark has two lanes and does not calculate a composite winner. LSQB-derived is an Antfly-owned anchored microbenchmark over LSQB data. The Pokec suite is a clean-room implementation informed by the public categories and workload ratios described by Memgraph’s Benchgraph material. It reuses the public Pokec import files, not Benchgraph implementation code. Neither lane is an audited LDBC result.

The scorecard keeps these measurements separate:

Lane	Scorecard metrics	Direction
LSQB-derived F2	warm serial p50, warm serial p99	latency lower
Pokec W1	completed QPS, peak process RSS	QPS higher; RSS lower

Full statistics, errors, run ranges, and raw samples remain mandatory even when the scorecard is used.

Engines and equal client work

The comparison is Antfly, Neo4j 5.26.29, and Memgraph 3.12.0. Kuzu is excluded because it is not exercised through a comparable network service. Every timed throughput cell uses the same Go executable, deterministic schedule, quantile implementation, response-consumption rule, worker count, and wall-clock. Antfly is accessed over persistent HTTP/1.1; both peers use the same official Neo4j Go Bolt driver. Transport is part of observed client latency and is reported, not hidden.

Only equivalent User(id) and benchmark-namespace ID indexes are created. There is no engine-specific query tuning. Each run starts from a fresh database and performs complete correctness checks before timing. Mutating cells use a reserved BenchUser namespace so source reads cannot change; that namespace is restored to its 1,024-node ring baseline after every mutating cell. Timed requests are never retried. Errors are retained and exclude a run from the publication gate.

LSQB-derived lane

The pinned source, data loading, parameter selection, BLAKE3 result oracle, and N1/P2/F2 definitions are specified in graph-lsqb-derived.md. The public LSQB-derived metric is warmed serial latency for this exact endpoint-filtered lookup, returning matching Post IDs:

(Forum)-[:CONTAINER_OF]->(Post)-[:HAS_TAG]->(Tag {id: $tag_id})

Unfiltered P2 C12 throughput remains a retained diagnostic. It is not part of the graph scorecard or public filtered-lookup claim. Default, medium, and full use LSQB SF0.003, SF0.1, and SF1 respectively.

Pokec data and parameter oracle

Profiles use the public Memgraph Pokec import files:

Profile	Dataset	Users	Directed `FRIEND` edges	Anchors
quick	Pokec-small	10,000	121,716	12
medium	Pokec-medium	100,000	1,768,515	48
full	Pokec-large	1,632,803	30,622,564	128

The harness records the downloaded source SHA-256 and refuses unexpected node or edge counts. Anchors are selected deterministically across out-degree deciles with seed 0xB37C4. Every selected anchor must return its complete result below 900 rows through four hops; this stays below rc18’s 1,000 pattern match ceiling rather than comparing truncated results.

Expansion semantics are one endpoint result per directed walk. Antfly uses a chain of fixed one-hop pattern steps. Cypher uses one fixed MATCH clause per hop, avoiding vendor differences in variable-length path uniqueness. The offline SQLite oracle stores the exact result count and canonical BLAKE3 digest including multiplicity for every parameter. Aggregates and shortest path have exact scalar oracles. All 101 quick-profile checks per engine/run must pass before timing. Writes are executed and verified in the isolated benchmark namespace.

Pokec operations

ID	Operation
R1	exact indexed node lookup
WN	insert a new benchmark node
U1	increment one benchmark-node property
WE	insert one directed benchmark edge
AG1	age buckets and counts
AG2	age count/min/max/sum/average
E1–E4	exact fixed 1-, 2-, 3-, or 4-hop directed expansion
E2F	exact 2-hop expansion with endpoint `age >= 18`
N2	concatenated exact 1-hop and exact 2-hop neighborhood
SP	directed unweighted shortest path, maximum depth 8

Every operation has an unwarmed serial observation and a prewarmed serial distribution. Isolated throughput uses 12 clients. Each non-mutating operation also receives a 70% operation / 30% write cell, split evenly between node and edge insert. The deterministic realistic workloads are:

W1: 20% analytical, 40% point read, 10% update, 30% write;
W2: 70% point read, 30% write;
W3: 50% point read, 50% write;
W4: 30% point read, 70% write.

The quick profile uses one-second isolated/mixed cells and two-second W1–W4 cells. Medium uses 5/10 seconds; full uses 15/30 seconds. Warmup is untimed. Every successful timed request is retained in JSONL. Reports contain completed QPS, error counts, p50/p95/p99, per-operation statistics, and peak PID RSS.

Persistent database volume

The honed LSQB SF0.1 F2 lane creates a new, run-labeled Docker local volume for every engine and repetition. Only the complete writable database root is measured (/data for Antfly and Neo4j, /var/lib/memgraph for Memgraph); read-only dataset/tool volumes and client scratch are excluded. A digest-pinned Alpine probe captures whole-volume allocated and apparent bytes once at empty, service-ready-before-ingestion, and post-workload, then at the verified post-stop final barrier. Final allocated bytes are the headline.

All three storage values, their arithmetic mean and range, apparent-byte diagnostics, barrier samples, volume manifests, and lifecycle receipts are retained. A provisional stop timeout remains visible but cannot support a comparative ratio.

Runs, ordering, and reporting

Each profile requires three clean process runs. Engine order rotates by run and uses a different rotation between lanes. Public serial point estimates are the medians of the three run-level p50, p95, and p99 values. Every result row also retains the three run-level percentile records, their min–max ranges, pooled sample percentiles as diagnostics, and all raw samples. Load/index readiness is diagnostic and is never included in query metrics.

summary.json is the machine-readable source of truth. The Markdown report contains the full cell table; scorecard.svg is only the compact two-panel view. There is deliberately no geometric mean or composite rank.

The honed LSQB SF0.1 F2 lane caches only the complete three-engine cohort. Promotion requires Antfly, Neo4j, and Memgraph to each retain three fresh database constructions and process runs, 15,000 timed samples, all three run-level percentile records and ranges, three complete storage receipts and their allocated-byte mean/range, 1,000/1,000 correctness, a valid canonical receipt, and the quiet-host record. The cache identity binds the Antfly subject, both peer image digests, the Linux client and storage-probe image, dataset manifest, harness sources, and protocol. A hit materializes the entire cohort; per-engine cache mixing is not permitted.

Internal fresh-service F2 readiness and convergence

The honed and Circus profiles retain a fresh-service diagnostic that measures how quickly a preloaded graph service becomes useful and settles into its warm filtered-two-hop behavior. It is graph-specific: readiness is followed immediately by the same exact LSQB SF0.1 F2 request used by the warm latency lane. It is not a general process-startup or system-wide Antfly claim.

This cohort is deliberately internal. It is attached to the canonical graph run report and structured result, but the public benchmark-page renderer does not render its table. It is never eligible for the honed publication cache: each honed or gauntlet graph invocation collects it afresh with make graph-f2-warmup-run. The retained diagnostic contract requires:

the same Antfly binary, pinned Neo4j and Memgraph images, official LSQB SF0.1 dataset, and Linux/arm64 Docker environment as the primary graph lane;
a recorded physical-host state, including whether the honed quiet-host assertion was active, without treating a diagnostic run as publication data;
one database construction followed by five fresh service lifecycles per engine, with balanced engine ordering;
restart-to-ready measured from immediately before service launch through the first successful engine-native probe: Antfly GET /readyz, Neo4j cypher-shell RETURN 1 plus the pinned-version check, and Memgraph autocommit SHOW VERSION over Bolt;
the first F2 latency and first-100 p50/p95 from a fixed 100-anchor panel selected by SHA-256 rank independently of result cardinality;
convergence at the first point where three consecutive adjacent 100-query window p50 changes are each within 5%, with a 10,000-query ceiling;
steady-state p50/p95 from five deterministic sweeps over all 1,000 anchors after convergence;
every timed result checked against the independent count+BLAKE3 oracle, with all 1,000 anchors covered in every engine/lifecycle; and
median-of-five run metrics with complete run ranges, raw timestamped samples, lifecycle logs, host evidence, exact schedules, and a canonical receipt.

These seven values are an indivisible internal reporting family: restart-to-ready, first-query, first-100 p50, first-100 p95, queries-to-stable-p50, steady p50, and steady p95. The retained report must show all seven for all three engines; it must not headline convergence while omitting the steady-state latency guardrail or Memgraph.

Qualification and publication

The default command is a small, complete optimization loop with a target wall time below 15 minutes. Medium and full are explicit diagnostic commands. A mixed native-macOS/container run is qualification-only because its process RSS views are not comparable across engines.

The release-facing honed F2 lane is a separate, environment-qualified publication protocol. Antfly, Neo4j, and Memgraph all run sequentially inside the same Docker Desktop Linux/arm64 VM on an explicitly quiescent Apple Silicon host. The published claim must name that environment and is limited to warm serial filtered two-hop latency; it is not a bare-metal Linux result or a claim about general graph-database performance. Publication requires:

the exact v0.2.0 Linux/arm64 binary and expected SHA-256;
the pinned Neo4j 5.26.29, Memgraph 3.12.0, and Linux client image digests;
official LSQB SF0.1 with 432,235 nodes and 1,827,523 relationships;
three fresh database constructions and process runs per engine, each on a newly created labeled database volume;
five warmed serial sweeps per run over 1,000 deterministic anchors;
1,000/1,000 count+BLAKE3 correctness in every engine/run and zero timed errors;
median run-level p50/p95/p99 with all run ranges, pooled diagnostics, raw samples, lifecycle logs, and the physical-host record retained; and
a canonical receipt for the indivisible three-engine cohort.

The broader graph-v2/Pokec publication gate remains native-Linux-only and is not satisfied by this bounded F2 artifact.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: graph-benchmark
Domain: graph benchmark
Source: https://github.com/ldbc/lsqb
Known Antfly versions: v0.1.3, v0.2.*
Expected duration: 100 min
Enabled: yes

Benchmark parameters

{"peers_not_run":["neo4j","memgraph"],"query":"F2","runs":3,"samples":15000}

Trial variation

[{"engine":"memgraph","ranges":{"final_database_volume":{"maximum":311.906304,"mean":311.906304,"minimum":311.906304,"unit":"MB"},"query_p95":{"maximum":0.14466905,"mean":0.1440441,"minimum":0.14354615,"unit":"ms"}}},{"engine":"neo4j","ranges":{"final_database_volume":{"maximum":635.670528,"mean":635.663701,"minimum":635.650048,"unit":"MB"},"query_p95":{"maximum":0.6908785500000003,"mean":0.6795417000000001,"minimum":0.6769125000000001,"unit":"ms"}}}]

Peer versions

Memgraph3.12.0
Neo4j5.26.29

Evidence digests

Receipt: d2715c7123ad44103cba8b3857c75b37a80ea79f48b6e3f907a3ecdb91695962
Aggregate: a8d15198bbb63dcd28d2f04f9d6f28307cf80404b8309cef9bcfbf9b1f8fed66
Provenance: 1597eaf97f60522712affe834995ab669f784367aa8022b3f591d1c3d8bed9b2
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Selected measurement

Completed: 27 Aug 2026 · 11:04 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 6m 15s
Platform: linux · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Warm serial p95

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · LSQB-derived SF0.1 F2 · 15,000 samples / three runs · Lower is better

Demand working set

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · LSQB-derived SF0.1 F2 · 15,000 samples / three runs · Lower is better

Final database volume

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · linux-arm64 · LSQB-derived SF0.1 F2 · 15,000 samples / three runs · Lower is better

Latency, resources, and served samples

Antfly v0.2.1-rc1 · 279381f7

Engine	Version	Query	p50 ms	p95 ms	p99 ms	Demand working set MB	Final database volume MB	Served samples
antfly	antfly dev (zig runtime)	F2	0.237	0.412	0.625	2,750.3 MB	176.9 MB	15,000
memgraph	3.12.0	F2	0.113	0.144	0.175	Not recorded MB	311.9 MB	15,000
neo4j	5.26.29	F2	0.376	0.680	1.076	Not recorded MB	635.7 MB	15,000

graph-diagnostics

partial

Pokec-small

Same-host Pokec-small coverage across Antfly, Neo4j, and Memgraph over 29 operation and workload cells.

Methodology scope · protocol · interpretation

Graph Database Benchmark v2 Methodology

Scope and provenance

The scorecard keeps these measurements separate:

Lane	Scorecard metrics	Direction
LSQB-derived F2	warm serial p50, warm serial p99	latency lower
Pokec W1	completed QPS, peak process RSS	QPS higher; RSS lower

Full statistics, errors, run ranges, and raw samples remain mandatory even when the scorecard is used.

Engines and equal client work

LSQB-derived lane

(Forum)-[:CONTAINER_OF]->(Post)-[:HAS_TAG]->(Tag {id: $tag_id})

Pokec data and parameter oracle

Profiles use the public Memgraph Pokec import files:

Profile	Dataset	Users	Directed `FRIEND` edges	Anchors
quick	Pokec-small	10,000	121,716	12
medium	Pokec-medium	100,000	1,768,515	48
full	Pokec-large	1,632,803	30,622,564	128

Pokec operations

Cell identifier	Full operation name
`R1`	exact indexed user lookup returning the stored user document
`WN`	insert a new benchmark user node
`U1`	increment one benchmark-user property
`WE`	add one directed benchmark friendship edge
`AG1`	whole-table age buckets and counts
`AG2`	whole-table age count/min/max/sum/average
`E1–E4`	exact fixed 1-, 2-, 3-, or 4-hop directed expansion
`E2F`	exact 2-hop expansion with endpoint `age >= 18`
`N2`	concatenated exact 1-hop and exact 2-hop neighborhood
`SP`	directed unweighted shortest path, maximum depth 8

Four-hop expansion (E4) returns endpoints reached by exactly four directed hops; one-plus-two-hop neighborhood (N2) concatenates the exact one-hop and exact two-hop neighborhoods (including walk multiplicity). They measure different work and remain separate rows.

Mixed analytical/read/update/write workload (W1): 20% analytical (AG1/AG2), 40% exact indexed user lookup, 10% property update, and 30% write split evenly between node and edge mutation;
Read-heavy mixed workload (W2): 70% exact indexed user lookup and 30% write;
Balanced read/write workload (W3): 50% exact indexed user lookup and 50% write;
Write-heavy mixed workload (W4): 30% exact indexed user lookup and 70% write.

How to interpret representative cells

Within one cell, higher completed QPS and lower p50/p95/p99 indicate better serving performance. QPS values across different cells do not compare equal amounts of work.

Exact indexed user lookup (R1) fetches one known user and returns the stored document. Its isolated cell measures focused point-read behavior with 12 clients and thousands of served samples per run.

Directed unweighted shortest path (SP) invokes each engine’s shortest-path primitive with a maximum depth of eight. The deterministic quick-profile parameters select targets one or two hops away, so this cell demonstrates short bounded pathfinding rather than long-path or high-diameter behavior.

Mixed analytical/read/update/write workload (W1) supplies the broadest database-behavior signal. Twelve clients concurrently issue 20% whole-table age analytics, 40% exact user lookups, 10% property increments, and 30% node or edge writes. Slow analytics or mutation maintenance can therefore delay otherwise-fast reads; the cell intentionally captures this cross-operation interference. Low completion counts make its tail percentiles directional, so reports retain counts and every raw response alongside QPS and latency.

Current Antfly rejects transforms against reserved _edges paths. Its WE compatibility request replaces the complete deterministic benchmark edge set through a whole-document upsert, while Neo4j and Memgraph create one edge. Diagnostic presentations disclose that difference and make no publication or apples-to-apples write-throughput claim.

Persistent database volume

Runs, ordering, and reporting

Internal fresh-service F2 readiness and convergence

the same Antfly binary, pinned Neo4j and Memgraph images, official LSQB SF0.1 dataset, and Linux/arm64 Docker environment as the primary graph lane;
a recorded physical-host state, including whether the honed quiet-host assertion was active, without treating a diagnostic run as publication data;
one database construction followed by five fresh service lifecycles per engine, with balanced engine ordering;
restart-to-ready measured from immediately before service launch through the first successful engine-native probe: Antfly GET /readyz, Neo4j cypher-shell RETURN 1 plus the pinned-version check, and Memgraph autocommit SHOW VERSION over Bolt;
the first F2 latency and first-100 p50/p95 from a fixed 100-anchor panel selected by SHA-256 rank independently of result cardinality;
convergence at the first point where three consecutive adjacent 100-query window p50 changes are each within 5%, with a 10,000-query ceiling;
steady-state p50/p95 from five deterministic sweeps over all 1,000 anchors after convergence;
every timed result checked against the independent count+BLAKE3 oracle, with all 1,000 anchors covered in every engine/lifecycle; and
median-of-five run metrics with complete run ranges, raw timestamped samples, lifecycle logs, host evidence, exact schedules, and a canonical receipt.

Bounded internal Maxi diagnostic source

graph-diagnostics canonicalizes medium Pokec and LSQB-derived SF0.1 instrumentation without changing the honed F2 publication protocol. One balanced three-engine collection produces two independent sub-lanes:

lsqb-derived-anchored-n1-p2-f2-v1 retains separate warm-serial N1, P2, and F2 median-of-three run percentiles, ranges, result-count buckets, served sample counts, and every count+BLAKE3 parameter oracle; and
pokec-graph-operations-v1 retains R1, E1–E4, E2F, N2, SP, AG1, AG2, WN, U1, WE, W1–W4, exact mutation-state checks, and raw schedules on the Pokec-medium corpus.

The default runner passes zero throughput seconds, so it never starts the P2 concurrency workload. It also never starts the five-lifecycle fresh-service cohort. graph-p2-c12-sf01-run and graph-f2-warmup-run retain those protocols as explicit opt-in diagnostics. SF1 and SF10 likewise require explicit scale targets and never enter the default gauntlet.

The finalizer rejects timed errors, missing runs, failed count/hash/state oracles, and any result-budget contact. The canonical result uses graph-diagnostics-v2 with classification internal_diagnostic. PID RSS remains under diagnostic_process_rss; the source declares comparable_resource_metrics: false, emits no memory or storage chart, and defines no composite winner. This source enters the gauntlet and Maxi incubation views only, never the honed allowlist or public graph section.

Qualification and publication

the exact v0.2.0 Linux/arm64 binary and expected SHA-256;
the pinned Neo4j 5.26.29, Memgraph 3.12.0, and Linux client image digests;
official LSQB SF0.1 with 432,235 nodes and 1,827,523 relationships;
three fresh database constructions and process runs per engine, each on a newly created labeled database volume;
five warmed serial sweeps per run over 1,000 deterministic anchors;
1,000/1,000 count+BLAKE3 correctness in every engine/run and zero timed errors;
median run-level p50/p95/p99 with all run ranges, pooled diagnostics, raw samples, lifecycle logs, and the physical-host record retained; and
a canonical receipt for the indivisible three-engine cohort.

The broader graph-v2/Pokec publication gate remains native-Linux-only and is not satisfied by this bounded F2 artifact.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: graph-diagnostics
Domain: graph database diagnostics
Source: https://github.com/ldbc/lsqb
Known Antfly versions: v0.1.3, v0.2.*
Enabled: yes

Benchmark parameters

{"correctness":{"antfly-1":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"antfly-2":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"antfly-3":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"memgraph-1":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"memgraph-2":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"memgraph-3":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"neo4j-1":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"neo4j-2":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1},"neo4j-3":{"failed":0,"oracle":"complete result multiset/count or exact aggregate/path value","passed":101,"rate":1}},"dataset":{"dataset":"pokec-small","edges":121716,"nodes":10000,"source_sha256":"3e64bffd118c18a23342233f30fc710b753f4ea173a2d166351a28b44976c308","source_url":"https://s3.eu-west-1.amazonaws.com/deps.memgraph.io/dataset/pokec/benchmark/pokec_small_import.cypher"},"total_cells":29,"valid_cells":26}

Peer versions

Memgraph3.12.0
Neo4j5.26.29

Evidence digests

Receipt: e12cd03f8c21ff64a19c859ba33d9e2ad9da8f1f4a2ee409707279c49345142c
Aggregate: 80cb55462826e6608afe02ea5b88aace553a9def86c952589f3efe5733200bbf
Provenance: c63d91082134fb1cccf8b2b67e70b47ac2d3c0700ded2cc362a155c7251be8eb
Harness commit: 099cb05da1d0503fcd47ee5f454b0bde617b5529

Scope notes

Three errorful cells publish no QPS or latency claim.
SP is a fixed one-hop timed microbenchmark, not a general shortest-path scale result.

Selected partial evidence

Completed: 27 Aug 2026 · 10:53 UTC
Antfly: v0.2.1-rc1 · 26 August 2026 · 20:47 UTC
Runtime: 7m 27s
Platform: darwin · arm64
Machine: Apple M4 Max
Build: ReleaseFast

Three errorful cells publish no QPS or latency claim.

SP is a fixed one-hop timed microbenchmark, not a general shortest-path scale result.

Pokec-small throughput by task and engine

Antfly v0.2.1-rc1 (279381f7, ReleaseFast/full) · Apple M4 Max physical host · Antfly native Darwin/arm64 · Neo4j Linux/arm64 Docker · Memgraph Linux/arm64 Docker · Pokec-small · three runs · errorful cells excluded · Each task resets red→yellow→green from lowest to highest QPS; compare within the same task/row only

Pokec-small graph operations

Antfly v0.2.1-rc1 · 279381f7

Task	Antfly	Neo4j	Memgraph
Serial read sweep (unwarmed)	QPS 39.1 · p50 0.859 / p95 141.503 / p99 145.730 ms	QPS 57.3 · p50 21.1 / p95 34.9 / p99 35.8 ms	QPS 926.2 · p50 0.3 / p95 4.3 / p99 5.6 ms
Serial read sweep (prewarmed)	QPS 37.8 · p50 0.332 / p95 132.421 / p99 148.394 ms	QPS 602.6 · p50 1.5 / p95 3.5 / p99 4.3 ms	QPS 784.1 · p50 0.3 / p95 5.4 / p99 6.1 ms
Isolated · exact lookup (R1)	QPS 30,441.8 · p50 0.348 / p95 0.748 / p99 0.997 ms	QPS 11,921.6 · p50 0.9 / p95 1.8 / p99 2.6 ms	QPS 21,058.2 · p50 0.5 / p95 0.9 / p99 1.1 ms
Isolated · age buckets (AG1)	QPS 19.7 · p50 603.145 / p95 609.649 / p99 610.026 ms	QPS 4,850.6 · p50 2.2 / p95 4.3 / p99 5.8 ms	QPS 6,215.7 · p50 1.7 / p95 3.2 / p99 4.0 ms
Isolated · age stats (AG2)	QPS 19.9 · p50 601.144 / p95 619.238 / p99 624.736 ms	QPS 4,513.2 · p50 2.3 / p95 4.5 / p99 5.8 ms	QPS 2,788.3 · p50 4.0 / p95 6.0 / p99 10.6 ms
Isolated · 1-hop expansion (E1)	QPS 34,884.5 · p50 0.279 / p95 0.704 / p99 0.953 ms	QPS 7,150.1 · p50 1.3 / p95 2.8 / p99 4.1 ms	QPS 20,916.5 · p50 0.5 / p95 0.9 / p99 1.1 ms
Isolated · 2-hop expansion (E2)	QPS 28,203.4 · p50 0.370 / p95 0.810 / p99 1.093 ms	QPS 7,833.6 · p50 1.4 / p95 2.7 / p99 3.5 ms	QPS 19,690.2 · p50 0.6 / p95 1.0 / p99 1.4 ms
Isolated · 2-hop age filter (E2F)	QPS 10,130.1 · p50 1.104 / p95 2.081 / p99 2.677 ms	QPS 8,185.3 · p50 1.3 / p95 2.6 / p99 3.6 ms	QPS 19,477.9 · p50 0.6 / p95 1.0 / p99 1.3 ms
Isolated · 3-hop expansion (E3)	QPS 18,801.0 · p50 0.584 / p95 1.150 / p99 1.492 ms	QPS 6,546.9 · p50 1.7 / p95 2.9 / p99 3.6 ms	QPS 12,030.8 · p50 1.0 / p95 1.6 / p99 1.9 ms
Isolated · 4-hop expansion (E4)	QPS 6,438.0 · p50 1.763 / p95 2.996 / p99 3.663 ms	QPS 2,051.5 · p50 5.7 / p95 7.4 / p99 8.6 ms	QPS 3,264.6 · p50 3.6 / p95 5.1 / p99 6.0 ms
Isolated · 1+2-hop neighborhood (N2)	QPS 22,954.2 · p50 0.500 / p95 0.957 / p99 1.209 ms	QPS 15,161.2 · p50 0.7 / p95 1.3 / p99 1.8 ms	QPS 19,724.2 · p50 0.6 / p95 1.0 / p99 1.4 ms
Isolated · shortest path ≤8 (SP)	QPS 39,075.6 · p50 0.246 / p95 0.642 / p99 0.864 ms	QPS 17,243.1 · p50 0.6 / p95 1.2 / p99 1.9 ms	QPS 1,499.9 · p50 7.3 / p95 11.5 / p99 16.7 ms
Isolated · node insert (WN)	QPS 452.3 · p50 18.117 / p95 71.196 / p99 111.896 ms	QPS 8,100.6 · p50 1.5 / p95 2.1 / p99 2.8 ms	QPS 20,049.2 · p50 0.6 / p95 0.9 / p99 1.3 ms
Isolated · property update (U1)	QPS 792.7 · p50 7.522 / p95 55.642 / p99 116.061 ms	QPS 10,989.2 · p50 1.1 / p95 1.5 / p99 1.8 ms	QPS 20,835.1 · p50 0.5 / p95 0.9 / p99 1.2 ms
Isolated · edge mutation (WE)	QPS 339.2 · p50 24.427 / p95 106.099 / p99 139.225 ms	QPS 8,225.6 · p50 1.4 / p95 1.9 / p99 3.0 ms	QPS 20,120.2 · p50 0.6 / p95 0.9 / p99 1.2 ms
30% writes · exact lookup (R1)	QPS 933.2 · p50 8.679 / p95 32.117 / p99 73.693 ms	QPS 13,404.8 · p50 0.6 / p95 2.1 / p99 2.6 ms	QPS 21,463.4 · p50 0.5 / p95 0.9 / p99 1.1 ms
30% writes · age buckets (AG1)	failed · 35 timed errors	QPS 5,745.1 · p50 1.9 / p95 3.6 / p99 4.7 ms	QPS 4,931.2 · p50 2.7 / p95 4.3 / p99 6.2 ms
30% writes · age stats (AG2)	failed · 24 timed errors	QPS 5,237.7 · p50 2.1 / p95 4.1 / p99 5.4 ms	QPS 2,938.7 · p50 4.6 / p95 7.3 / p99 11.2 ms
30% writes · 1-hop expansion (E1)	QPS 1,454.9 · p50 6.360 / p95 20.111 / p99 60.440 ms	QPS 13,496.8 · p50 0.6 / p95 2.2 / p99 2.6 ms	QPS 20,882.2 · p50 0.5 / p95 0.9 / p99 1.2 ms
30% writes · 2-hop expansion (E2)	QPS 1,644.6 · p50 6.161 / p95 15.765 / p99 49.338 ms	QPS 12,998.9 · p50 0.7 / p95 2.1 / p99 2.6 ms	QPS 20,142.4 · p50 0.6 / p95 1.0 / p99 1.2 ms
30% writes · 2-hop age filter (E2F)	QPS 1,368.6 · p50 7.250 / p95 17.688 / p99 54.709 ms	QPS 13,234.2 · p50 0.6 / p95 2.1 / p99 2.6 ms	QPS 19,609.8 · p50 0.6 / p95 1.0 / p99 1.3 ms
30% writes · 3-hop expansion (E3)	QPS 1,579.6 · p50 6.264 / p95 16.576 / p99 53.650 ms	QPS 10,632.7 · p50 1.0 / p95 2.1 / p99 2.6 ms	QPS 16,321.1 · p50 0.7 / p95 1.3 / p99 1.7 ms
30% writes · 4-hop expansion (E4)	QPS 1,343.3 · p50 7.510 / p95 19.075 / p99 57.687 ms	QPS 2,808.9 · p50 4.6 / p95 8.2 / p99 9.4 ms	QPS 4,467.0 · p50 2.9 / p95 5.6 / p99 6.5 ms
30% writes · 1+2-hop neighborhood (N2)	QPS 1,501.9 · p50 6.298 / p95 18.525 / p99 72.388 ms	QPS 12,722.7 · p50 0.7 / p95 2.1 / p99 2.6 ms	QPS 20,435.4 · p50 0.5 / p95 1.0 / p99 1.3 ms
30% writes · shortest path ≤8 (SP)	QPS 1,601.3 · p50 5.911 / p95 17.285 / p99 50.800 ms	QPS 13,353.4 · p50 0.6 / p95 2.1 / p99 2.6 ms	QPS 2,111.2 · p50 6.5 / p95 9.6 / p99 16.4 ms

Pokec-small read/write mixes

Antfly v0.2.1-rc1 · 279381f7

Task	Antfly	Neo4j	Memgraph
Mixed workload · 20/40/10/30 (W1)	failed · 25 timed errors	QPS 9,512.5 · p50 1.1 / p95 2.5 / p99 3.9 ms	QPS 11,174.5 · p50 0.4 / p95 4.3 / p99 7.1 ms
Read-heavy mix · 70/30 (W2)	QPS 1,608.7 · p50 6.109 / p95 16.782 / p99 74.644 ms	QPS 13,572.6 · p50 0.6 / p95 2.1 / p99 2.6 ms	QPS 21,203.4 · p50 0.5 / p95 0.9 / p99 1.2 ms
Balanced mix · 50/50 (W3)	QPS 983.8 · p50 7.547 / p95 33.297 / p99 100.749 ms	QPS 11,806.7 · p50 0.8 / p95 2.1 / p99 2.5 ms	QPS 21,006.3 · p50 0.5 / p95 0.9 / p99 1.2 ms
Write-heavy mix · 30/70 (W4)	QPS 688.5 · p50 10.072 / p95 52.615 / p99 109.983 ms	QPS 10,423.6 · p50 1.2 / p95 2.0 / p99 2.3 ms	QPS 20,258.6 · p50 0.6 / p95 0.9 / p99 1.2 ms

VIII

1 of 1 lane measured

End-to-end

Agent-facing product activation and first retrieval from clean interactive Pi sessions.

quickstart-first-retrieval

partial

Quickstart

11 database-version rows retain every recording. Median time summarizes observed retrievals; validation reports which sessions completed cleanly.

Database	Version	Median time	Validation	Sessions
Antfly	v0.1.3	228.554 s	passed	Watch sessions 1
Antfly	v0.2.0	228.262 s	passed	Watch sessions 1 2
Antfly	v0.2.1-rc1	111.846 s	passed	Watch sessions 1 2 3
Chroma	v1.5.9	79.072 s	passed	Watch sessions 1
Elasticsearch	v9.5.2	445.708 s	passed	Watch sessions 1
Memgraph	v3.12.0	67.161 s	passed	Watch sessions 1
Milvus Lite	v3.0.1	198.412 s	passed	Watch sessions 1
Neo4j	v2026.07.1	60.369 s	passed	Watch sessions 1
Quickwit	v0.9.0	61.978 s	passed	Watch sessions 1
Weaviate	v1.39.2	228.033 s	passed	Watch sessions 1
Antfly	v0.2.1	32.093 s	partial · 2/3 valid	Watch sessions 1 2 3

Methodology Timing · grouping · validation

Timing boundary

Time to first retrieval runs from PTY capture start through the first successful qualifying search result observed in the Pi session. Acquisition, installation, setup, retries, and agent reasoning all contribute to elapsed time.

Grouping and median

Each row groups retained sessions by database and exact observed version. The median includes every observed qualifying retrieval, including a retrieval followed by failed post-session verification. Validation remains separate. Models and task boundaries can differ between cohorts; these rows are not controlled product-speed comparisons.

Protocol history

Historical Antfly recordings use protocol 3 and its qualifying retrieval command. New natural recordings use protocol 4 passive Pi result correlation, without an agent-visible marker. Container-dependent sessions substitute rootless Podman for Docker and retain that runtime choice in provenance. Numbered session links preserve the full trajectory and protocol evidence behind every row.

Container boundary

Container-dependent sessions use an operator-approved rootless Podman host-user API. Bubblewrap limits the agent shell but does not constrain bind mounts requested through that API. The runner snapshots preexisting Podman resources, restricts verification to containers created by the attempt, removes attempt-owned resources afterward, scans published transcripts for credentials, and records the socket scope in provenance.

Validation states

Passed: Every included session produced a qualifying retrieval and passed its retained verifier.
Partial: At least one included session passed while another lacked complete valid evidence.
Failed: Included sessions completed without a qualifying verified retrieval.
Unverified: Retained evidence lacks a terminal verifier result.

Three fresh sequential v0.2.1 sessions used OpenRouter GLM-5.3 Flash with high thinking, host CPUs, and an explicit hello/first stopping objective. All three produced the expected retrieval; two passed post-session verification. Trial 2 shut down Antfly afterward, causing ConnectionRefused. Median includes all three observed retrievals, not just verified sessions. Release and prompt differ from earlier cohorts; this is not an isolated product speed comparison. Model installation, semantic search and RAG were not exercised.