release · v0.2.1 · 7 complete · 1 partial · 0 valid negative / 8 lanes

Antfly Circus — v0.2.1

wow such release. many ant. wow

Reading contract

Each selected outcome binds its Antfly evidence to v0.2.1. Machines, dates, and protocols may differ.

3 of 3 lanes measured

VectorDBBench

Qualified vector search quality, throughput, serial latency, serving demand, and database volume across operating points.

vector-vdbbench-50k

complete

50K

Three independent v0.2.1 lifecycles on 50K 1536-dimensional vectors; historical peer references.

Methodology scope · protocol · interpretation

Subject and execution

Official v0.2.1, commit e5261e9e2937d03943b518974bf8063351ad1449, unchanged Linux ARM64 release executable. Runtime: rootless Podman 6.1.1 on Apple M4 Max, 10 VM CPUs and 24 GiB VM memory. The build is identified as the official release; compiler optimization flags are not independently asserted.

Only Antfly was measured in this run. Retained peer rows have their original versions and evidence dates; they are historical references, not a matched runtime cohort or evidence of a release-to-release speedup.

Vector protocol

Three independently built indexes per workload. Search parameters are selected only on the deterministic calibration partition; publication queries use the disjoint holdout partition. Memory demand is concurrently sampled cgroup v2 anon + shmem peak. Final database volume is post-stop allocated bytes across the complete declared Docker volume.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: vector-vdbbench-50k
Domain: vector
Source: https://github.com/antflydb/VectorDBBench
Upstream: https://github.com/zilliztech/VectorDBBench
Known Antfly versions: v0.1.3, v0.2.*
Expected duration: 480 min
Enabled: yes

Benchmark parameters

{"concurrency_duration_seconds":30,"concurrency_schedule":[1,5,10,20,30],"execution_environment":{"container_runtime":"rootless Podman 6.1.1","processor":"Apple M4 Max","vm_cpus":10,"vm_memory_gib":24},"lifecycle_search_values":[0.425,0.425,0.425],"query_count":800,"query_partition_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","search_parameter":"search_effort"}

Trial variation

[{"engine":"Antfly","ranges":{"demand_working_set":{"maximum":8908,"mean":6992.3,"minimum":5551.7,"unit":"MB"},"final_database_volume":{"maximum":354.934784,"mean":354.820096,"minimum":354.639872,"unit":"MB"},"p99_at_30_concurrent":{"maximum":83.7,"mean":83.6,"minimum":83.4,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Chroma","ranges":{"demand_working_set":{"maximum":466.7,"mean":463.3,"minimum":460.9,"unit":"MB"},"final_database_volume":{"maximum":332.488704,"mean":332.485973,"minimum":332.480512,"unit":"MB"},"p99_at_30_concurrent":{"maximum":179,"mean":177.6,"minimum":175.3,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Elasticsearch","ranges":{"demand_working_set":{"maximum":1952.7,"mean":1941,"minimum":1934.8,"unit":"MB"},"final_database_volume":{"maximum":312.082432,"mean":312.067413,"minimum":312.049664,"unit":"MB"},"p99_at_30_concurrent":{"maximum":15.1,"mean":14.467,"minimum":14,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Milvus 2.6.22","ranges":{"demand_working_set":{"maximum":2766.6,"mean":2666.2,"minimum":2563.6,"unit":"MB"},"final_database_volume":{"maximum":2393.47712,"mean":2393.459371,"minimum":2393.432064,"unit":"MB"},"p99_at_30_concurrent":{"maximum":11.4,"mean":11.233,"minimum":11.1,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"pgvector","ranges":{"demand_working_set":{"maximum":2303.8,"mean":2303.7,"minimum":2303.7,"unit":"MB"},"final_database_volume":{"maximum":1523.970048,"mean":1523.967317,"minimum":1523.965952,"unit":"MB"},"p99_at_30_concurrent":{"maximum":33.1,"mean":32.033,"minimum":31.4,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Weaviate","ranges":{"demand_working_set":{"maximum":977.9,"mean":953.8,"minimum":936.6,"unit":"MB"},"final_database_volume":{"maximum":391.55712,"mean":391.514795,"minimum":391.491584,"unit":"MB"},"p99_at_30_concurrent":{"maximum":89.6,"mean":86.967,"minimum":85.3,"unit":"ms"}},"workload":"1536d-50K"},{"engine":"Milvus 3.0.0","ranges":{"demand_working_set":{"maximum":2949.6,"mean":2819,"minimum":2719.7,"unit":"MB"},"final_database_volume":{"maximum":2489.167872,"mean":2489.156949,"minimum":2489.143296,"unit":"MB"},"p99_at_30_concurrent":{"maximum":12,"mean":11.667,"minimum":11.4,"unit":"ms"}},"workload":"1536d-50K"}]

Peer versions

Chroma1.5.9
Elasticsearch9.5.2
Milvus 2.6.222.6.22
pgvector0.8.6 (PostgreSQL 16)
Weaviate1.39.2
Milvus 3.0.03.0.0

Evidence digests

Receipt: 72199dfaf7a6a4372eb68a6f075191e346970157eb1f3142144a7a8eb8731054
Aggregate: ef513ae591baae5707c38b0d6bac56e6e83e91546f3836857e7c64bd48fc1a1f
Provenance: ef513ae591baae5707c38b0d6bac56e6e83e91546f3836857e7c64bd48fc1a1f
Harness commit

Peer cohort evidence · 1536d-50K

Completed: 28 Aug 2026 · 14:56 UTC
Fresh lifecycles: 18
Peer variants: 6
Receipt: 782e8e470765fae9914945e67c9da3f3e82a1b38571ebe6e5940228a8248601c
Aggregate: 0eb49aba01e15e065ec68b1667beb2c41a580cde5dab2cb928b6f8c01c16daf4
Provenance: 700d49bf35039da97e9b5366d4a4e6ec4c417fdfdb5fd720b980349a538d68c3

Selected measurement

Completed: 9 Sep 2026 · 05:53 UTC
Antfly: v0.2.1 · 9 September 2026 · 00:29 UTC
Runtime: 2h 48m
Platform: linux · arm64
Machine: Apple M4 Max
Build: official-release

Peer cohort · 1536d-50K

Completed: 28 Aug 2026 · 14:56 UTC
Variants: 6 independently executed peers
Lifecycles: 18 fresh
Compatibility: Historical reference · not a matched runtime

Peak throughput

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · 1536d-50K · three fresh lifecycles · Higher is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Serial query p95

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · 1536d-50K · three fresh lifecycles · Lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Demand working set

Final database volume

Performance, quality, and resource evidence

Antfly v0.2.1 · e5261e9e

Engine	Workload	Recall	nDCG	Peak QPS queries/s	p95 ms	Demand working set MB	Cache-inclusive peak MB	Final database volume MB	Trials
Antfly	1536d-50K	0.9580	0.9630	984.7	3.533	6,992.3 MB	7,746.0 MB	354.8 MB	3
Chroma	1536d-50K	0.9870	0.9890	180.9	8.067	463.3 MB	812.9 MB	332.5 MB	3
Elasticsearch	1536d-50K	0.9560	0.9640	4,434.6	2.033	1,941.0 MB	4,385.1 MB	312.1 MB	3
Milvus 2.6.22	1536d-50K	0.9630	0.9700	5,293.1	1.600	2,666.2 MB	5,426.1 MB	2,393.5 MB	3
pgvector	1536d-50K	0.9690	0.9750	2,721.0	4.167	2,303.7 MB	3,533.5 MB	1,524.0 MB	3
Weaviate	1536d-50K	0.9550	0.9630	1,008.8	6.300	953.8 MB	1,511.3 MB	391.5 MB	3
Milvus 3.0.0	1536d-50K	0.9630	0.9700	5,019.7	1.700	2,819.0 MB	5,557.7 MB	2,489.2 MB	3

vector-vdbbench-1m

complete

1M

Three independent v0.2.1 lifecycles on 1M 768-dimensional vectors; historical peer references. 1M storage is provisional.

Methodology scope · protocol · interpretation

Subject and execution

Vector protocol

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: vector-vdbbench-1m
Domain: vector
Source: https://github.com/antflydb/VectorDBBench
Upstream: https://github.com/zilliztech/VectorDBBench
Known Antfly versions: v0.1.3, v0.2.*
Expected duration: 480 min
Enabled: yes

Benchmark parameters

{"concurrency_duration_seconds":30,"concurrency_schedule":[1,5,10,20,30],"execution_environment":{"container_runtime":"rootless Podman 6.1.1","processor":"Apple M4 Max","vm_cpus":10,"vm_memory_gib":24},"lifecycle_search_values":[0.45,0.45,0.425],"query_count":800,"query_partition_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","search_parameter":"search_effort","storage_status":{"reasons":[{"detail":"service did not exit cleanly: {\"container\": \"antfly_bench_vdbb_antfly_t1_768d-1M\", \"exit_code\": 1, \"forced\": false, \"oom_killed\": false, \"stop_command_error\": null}","kind":"abnormal_stop","last_observation_retained":true,"limit_seconds":60,"stage":"final","trial":"1"},{"detail":"service did not exit cleanly: {\"container\": \"antfly_bench_vdbb_antfly_t2_768d-1M\", \"exit_code\": 1, \"forced\": false, \"oom_killed\": false, \"stop_command_error\": null}","kind":"abnormal_stop","last_observation_retained":true,"limit_seconds":60,"stage":"final","trial":"2"},{"detail":"service did not exit cleanly: {\"container\": \"antfly_bench_vdbb_antfly_t3_768d-1M\", \"exit_code\": 1, \"forced\": false, \"oom_killed\": false, \"stop_command_error\": null}","kind":"abnormal_stop","last_observation_retained":true,"limit_seconds":60,"stage":"final","trial":"3"}],"status":"provisional"}}

Trial variation

[{"engine":"Antfly","ranges":{"demand_working_set":{"maximum":16455.4,"mean":15732.8,"minimum":15317.9,"unit":"MB"},"final_database_volume":{"maximum":3832.250368,"mean":3831.590912,"minimum":3831.0912,"unit":"MB"},"p99_at_30_concurrent":{"maximum":595.5,"mean":580.567,"minimum":565.6,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Chroma","ranges":{"demand_working_set":{"maximum":3699.1,"mean":3693.6,"minimum":3688.8,"unit":"MB"},"final_database_volume":{"maximum":3460.194304,"mean":3459.731456,"minimum":3458.838528,"unit":"MB"},"p99_at_30_concurrent":{"maximum":136.1,"mean":135.567,"minimum":134.6,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Elasticsearch","ranges":{"demand_working_set":{"maximum":2135.8,"mean":2095.2,"minimum":2029.8,"unit":"MB"},"final_database_volume":{"maximum":3148.750848,"mean":3148.500992,"minimum":3148.361728,"unit":"MB"},"p99_at_30_concurrent":{"maximum":21.6,"mean":20.967,"minimum":19.7,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Milvus 2.6.22","ranges":{"demand_working_set":{"maximum":13999.4,"mean":13920.6,"minimum":13837.1,"unit":"MB"},"final_database_volume":{"maximum":25669.378048,"mean":25642.164224,"minimum":25626.836992,"unit":"MB"},"p99_at_30_concurrent":{"maximum":13.7,"mean":13.633,"minimum":13.5,"unit":"ms"}},"workload":"768d-1M"},{"engine":"pgvector","ranges":{"demand_working_set":{"maximum":2318.6,"mean":2317.3,"minimum":2316.6,"unit":"MB"},"final_database_volume":{"maximum":9329.364992,"mean":9329.318571,"minimum":9329.27488,"unit":"MB"},"p99_at_30_concurrent":{"maximum":76.6,"mean":72.867,"minimum":69.9,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Weaviate","ranges":{"demand_working_set":{"maximum":7370.7,"mean":7345.1,"minimum":7331,"unit":"MB"},"final_database_volume":{"maximum":3955.232768,"mean":3947.47904,"minimum":3942.985728,"unit":"MB"},"p99_at_30_concurrent":{"maximum":68.3,"mean":67.2,"minimum":65.7,"unit":"ms"}},"workload":"768d-1M"},{"engine":"Milvus 3.0.0","ranges":{"demand_working_set":{"maximum":13551.8,"mean":13262.6,"minimum":13016.7,"unit":"MB"},"final_database_volume":{"maximum":35112.308736,"mean":33718.551893,"minimum":33003.88864,"unit":"MB"},"p99_at_30_concurrent":{"maximum":14.4,"mean":14.233,"minimum":14.1,"unit":"ms"}},"workload":"768d-1M"}]

Peer versions

Chroma1.5.9
Elasticsearch9.5.2
Milvus 2.6.222.6.22
pgvector0.8.6 (PostgreSQL 16)
Weaviate1.39.2
Milvus 3.0.03.0.0

Evidence digests

Receipt: b9f7307708da80c2bc2453be8aca9cc310815f49e5da9c3ad2a3a907917c2ed4
Aggregate: e12152c49b18305cf4b16d08967d8eaf0082b419519b4ed31844373451ebea91
Provenance: e12152c49b18305cf4b16d08967d8eaf0082b419519b4ed31844373451ebea91
Harness commit

Peer cohort evidence · 768d-1M

Completed: 28 Aug 2026 · 15:20 UTC
Fresh lifecycles: 18
Peer variants: 6
Receipt: 4366569aa9cfe5f7e796c5eadc30a6d2ae610cde650543eb3546ddcb856dfebf
Aggregate: 828e2d97f299926c3bb1f39866db983ef61895793243ec02a569fe955507a197
Provenance: 4af875e079c7c2538ed378027ac14be10294f30a7761e8fe0300f885ea3a1ea4

Selected measurement

Completed: 9 Sep 2026 · 05:53 UTC
Antfly: v0.2.1 · 9 September 2026 · 00:29 UTC
Runtime: 2h 48m
Platform: linux · arm64
Machine: Apple M4 Max
Build: official-release

Peer cohort · 768d-1M

Completed: 28 Aug 2026 · 15:20 UTC
Variants: 6 independently executed peers
Lifecycles: 18 fresh
Compatibility: Historical reference · not a matched runtime

Peak throughput

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · 768d-1M · three fresh lifecycles · Higher is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Serial query p95

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · 768d-1M · three fresh lifecycles · Lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Demand working set

Final database volume

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · 768d-1M · three fresh lifecycles · Antfly storage provisional; measured value retained in the table · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Performance, quality, and resource evidence

Antfly v0.2.1 · e5261e9e

Engine	Workload	Recall	nDCG	Peak QPS queries/s	p95 ms	Demand working set MB	Cache-inclusive peak MB	Final database volume MB	Trials	Clean stops / total
Antfly	768d-1M	0.9630	0.9670	86.5	14.900	15,732.8 MB	18,866.7 MB	3,831.6 MB (provisional)	3	0 / 3
Chroma	768d-1M	0.9560	0.9620	238.7	6.600	3,693.6 MB	7,289.8 MB	3,459.7 MB	3	3 / 3
Elasticsearch	768d-1M	0.9590	0.9610	3,110.8	3.233	2,095.2 MB	11,426.8 MB	3,148.5 MB	3	3 / 3
Milvus 2.6.22	768d-1M	0.9610	0.9660	4,085.9	2.200	13,920.6 MB	21,605.9 MB	25,642.2 MB	3	3 / 3
pgvector	768d-1M	0.9660	0.9700	955.8	10.767	2,317.3 MB	11,974.9 MB	9,329.3 MB	3	3 / 3
Weaviate	768d-1M	0.9560	0.9620	1,300.8	5.700	7,345.1 MB	13,791.8 MB	3,947.5 MB	3	3 / 3
Milvus 3.0.0	768d-1M	0.9610	0.9670	3,898.5	2.367	13,262.6 MB	21,631.0 MB	33,718.6 MB	3	3 / 3

iii

vector-vdbbench-scale

complete

Scale

Official v0.2.1 completed all twenty query lifecycles across ten Cohere 768D corpus sizes. Storage is provisional from 80K upward.

Methodology scope · protocol · interpretation

{ “diagnostic_trials”: 2, “profiles”: [ { “profile”: “fast”, “sizes”: [ 20000, 40000, 60000, 80000, 100000 ] }, { “profile”: “big”, “sizes”: [ 200000, 400000, 600000, 800000, 1000000 ] } ], “protocol”: { “calibration_queries”: 200, “concurrency_schedule”: [ 1, 5, 10, 20, 30 ], “dimensions”: 768, “memory_yardstick”: “cgroup-v2-anon-plus-shmem-full-lifecycle”, “metric”: “cosine”, “publication_queries”: 800, “query_partition_seed”: “antfly-circus-vdbbench-v1”, “storage_metric”: “final_allocated_bytes”, “window_seconds”: 30 }, “schema_version”: 1, “scope”: “Internal scale diagnostic: performance, memory, storage, and build ranges contain two fresh lifecycles rather than the three-lifecycle honed/public cohort.”, “statistics”: “Arithmetic mean, range, and both fresh lifecycle observations at each size.”, “storage”: “Post-stop allocated bytes remain measured when shutdown is abnormal; per-trial qualification is retained separately.” }

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: vector-vdbbench-scale
Domain: vector/scale
Source: https://github.com/antflydb/VectorDBBench
Upstream: https://github.com/zilliztech/VectorDBBench
Known Antfly versions: v0.2.*
Enabled: yes

Benchmark parameters

{"calibration_queries":200,"concurrency_schedule":[1,5,10,20,30],"dimensions":768,"memory_yardstick":"cgroup-v2-anon-plus-shmem-full-lifecycle","metric":"cosine","publication_queries":800,"query_partition_seed":"antfly-circus-vdbbench-v1","storage_metric":"final_allocated_bytes","window_seconds":30}

Where the time went

Two fresh lifecycles per size, ascending then descending; all held-out recall observations exceed 0.95, with zero query errors. Mean serial p95 rises from 2.50 ms at 20K to 14.65 ms at 1M; mean peak throughput falls from 2,048 to 84.4 QPS. Demand peaks average 2,058 MB and 16,480 MB respectively. All fourteen stop attempts at 80K–1M exited 1 without OOM or forced termination. Their post-stop volume captures succeeded but remain provisional, shown in the tables and excluded from storage curves. This is an Antfly-only two-lifecycle diagnostic, not a three-lifecycle competitor ranking.

Trial variation

Each size retains ordered trials 1 and 2, held-out qualification and selected effort. When captured, concurrency observations report QPS and p95/p99 in milliseconds at every client count; original publication metric latency arrays remain in seconds. These are two-lifecycle diagnostics, not the three-lifecycle honed cohort.

[{"metrics":{"demand_mb":{"max":2090,"mean":2058.3,"min":2026.6,"values":[2090,2026.6]},"final_allocated_bytes":{"max":78864384,"mean":78833664,"min":78802944,"values":[78802944,78864384]},"index_build_seconds":{"max":6.8161000000000005,"mean":6.62205,"min":6.428,"values":[6.8161000000000005,6.428]},"lifecycle_seconds":{"max":633.0043892860413,"mean":632.8403525352478,"min":632.6763157844543,"values":[632.6763157844543,633.0043892860413]},"peak_clients":{"max":10,"mean":10,"min":10,"values":[10,10]},"peak_qps":{"max":2113.1056,"mean":2048.27115,"min":1983.4367,"values":[1983.4367,2113.1056]},"serial_p95_ms":{"max":2.6,"mean":2.5,"min":2.4,"values":[2.6,2.4]}},"profile":"fast","qualification":{"ndcg_min":0.9645,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9586,"request_errors":0,"trials":[{"ndcg":0.9723,"recall":0.9681,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.5},"trial":1},{"ndcg":0.9645,"recall":0.9586,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.475},"trial":2}]},"storage_qualification":{"qualified_trials":2,"status":"qualified","total_trials":2,"trials":[{"exit_code":0,"forced":false,"oom_killed":false,"status":"qualified","stop_command_error":null,"trial":1},{"exit_code":0,"forced":false,"oom_killed":false,"status":"qualified","stop_command_error":null,"trial":2}]},"vectors":20000},{"metrics":{"demand_mb":{"max":3686.1,"mean":2675.2,"min":1664.3,"values":[3686.1,1664.3]},"final_allocated_bytes":{"max":155537408,"mean":155244544,"min":154951680,"values":[154951680,155537408]},"index_build_seconds":{"max":17.738300000000002,"mean":15.310550000000001,"min":12.8828,"values":[17.738300000000002,12.8828]},"lifecycle_seconds":{"max":663.1834547519684,"mean":663.0952868461609,"min":663.0071189403534,"values":[663.1834547519684,663.0071189403534]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":1179.6115,"mean":1178.3641499999999,"min":1177.1168,"values":[1177.1168,1179.6115]},"serial_p95_ms":{"max":3,"mean":3,"min":3,"values":[3,3]}},"profile":"fast","qualification":{"ndcg_min":0.9666,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9612,"request_errors":0,"trials":[{"ndcg":0.9668,"recall":0.9612,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.45},"trial":1},{"ndcg":0.9666,"recall":0.9614,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.45},"trial":2}]},"storage_qualification":{"qualified_trials":2,"status":"qualified","total_trials":2,"trials":[{"exit_code":0,"forced":false,"oom_killed":false,"status":"qualified","stop_command_error":null,"trial":1},{"exit_code":0,"forced":false,"oom_killed":false,"status":"qualified","stop_command_error":null,"trial":2}]},"vectors":40000},{"metrics":{"demand_mb":{"max":3118.8,"mean":2913.7,"min":2708.6,"values":[3118.8,2708.6]},"final_allocated_bytes":{"max":233750528,"mean":233414656,"min":233078784,"values":[233750528,233078784]},"index_build_seconds":{"max":21.959,"mean":21.8901,"min":21.8212,"values":[21.8212,21.959]},"lifecycle_seconds":{"max":663.2622854709625,"mean":663.0437227487564,"min":662.8251600265503,"values":[662.8251600265503,663.2622854709625]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":852.3412,"mean":849.1302000000001,"min":845.9192,"values":[845.9192,852.3412]},"serial_p95_ms":{"max":3.6,"mean":3.6,"min":3.6,"values":[3.6,3.6]}},"profile":"fast","qualification":{"ndcg_min":0.9714,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.967,"request_errors":0,"trials":[{"ndcg":0.9731,"recall":0.9692,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.45},"trial":1},{"ndcg":0.9714,"recall":0.967,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.45},"trial":2}]},"storage_qualification":{"qualified_trials":2,"status":"qualified","total_trials":2,"trials":[{"exit_code":0,"forced":false,"oom_killed":false,"status":"qualified","stop_command_error":null,"trial":1},{"exit_code":0,"forced":false,"oom_killed":false,"status":"qualified","stop_command_error":null,"trial":2}]},"vectors":60000},{"metrics":{"demand_mb":{"max":3400.2,"mean":3279.95,"min":3159.7,"values":[3159.7,3400.2]},"final_allocated_bytes":{"max":311717888,"mean":311324672,"min":310931456,"values":[310931456,311717888]},"index_build_seconds":{"max":30.7219,"mean":30.15365,"min":29.5854,"values":[29.5854,30.7219]},"lifecycle_seconds":{"max":693.2378511428833,"mean":693.1864391565323,"min":693.1350271701813,"values":[693.1350271701813,693.2378511428833]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":632.5033,"mean":627.5659499999999,"min":622.6286,"values":[622.6286,632.5033]},"serial_p95_ms":{"max":4.3,"mean":4.3,"min":4.3,"values":[4.3,4.3]}},"profile":"fast","qualification":{"ndcg_min":0.9745,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9706,"request_errors":0,"trials":[{"ndcg":0.9755,"recall":0.9717,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.45},"trial":1},{"ndcg":0.9745,"recall":0.9706,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.45},"trial":2}]},"storage_qualification":{"qualified_trials":0,"status":"provisional","total_trials":2,"trials":[{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":1},{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":2}]},"vectors":80000},{"metrics":{"demand_mb":{"max":4221,"mean":4061.25,"min":3901.5,"values":[3901.5,4221]},"final_allocated_bytes":{"max":388235264,"mean":387825664,"min":387416064,"values":[388235264,387416064]},"index_build_seconds":{"max":39.116499999999995,"mean":38.66575,"min":38.215,"values":[39.116499999999995,38.215]},"lifecycle_seconds":{"max":693.1322038173676,"mean":693.0248359441757,"min":692.9174680709839,"values":[693.1322038173676,692.9174680709839]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":532.0495,"mean":520.60535,"min":509.1612,"values":[509.1612,532.0495]},"serial_p95_ms":{"max":4.8,"mean":4.55,"min":4.3,"values":[4.8,4.3]}},"profile":"fast","qualification":{"ndcg_min":0.9632,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9583,"request_errors":0,"trials":[{"ndcg":0.974,"recall":0.9701,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.45},"trial":1},{"ndcg":0.9632,"recall":0.9583,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":2}]},"storage_qualification":{"qualified_trials":0,"status":"provisional","total_trials":2,"trials":[{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":1},{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":2}]},"vectors":100000},{"metrics":{"demand_mb":{"max":7984,"mean":7862.45,"min":7740.9,"values":[7740.9,7984]},"final_allocated_bytes":{"max":770670592,"mean":770097152,"min":769523712,"values":[770670592,769523712]},"index_build_seconds":{"max":92.46740000000001,"mean":92.26135000000001,"min":92.0553,"values":[92.0553,92.46740000000001]},"lifecycle_seconds":{"max":753.3726377487183,"mean":753.3653289079666,"min":753.358020067215,"values":[753.358020067215,753.3726377487183]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":272.8647,"mean":271.4648,"min":270.0649,"values":[272.8647,270.0649]},"serial_p95_ms":{"max":6.7,"mean":6.65,"min":6.6,"values":[6.6,6.7]}},"profile":"big","qualification":{"ndcg_min":0.9726,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9685,"request_errors":0,"trials":[{"ndcg":0.9726,"recall":0.9689,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":1},{"ndcg":0.9726,"recall":0.9685,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":2}]},"storage_qualification":{"qualified_trials":0,"status":"provisional","total_trials":2,"trials":[{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":1},{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":2}]},"vectors":200000},{"metrics":{"demand_mb":{"max":9741.4,"mean":9695.9,"min":9650.4,"values":[9650.4,9741.4]},"final_allocated_bytes":{"max":1540198400,"mean":1539745792,"min":1539293184,"values":[1540198400,1539293184]},"index_build_seconds":{"max":302.7898,"mean":293.1481,"min":283.5064,"values":[302.7898,283.5064]},"lifecycle_seconds":{"max":994.8732340335846,"mean":979.6541351079941,"min":964.4350361824036,"values":[994.8732340335846,964.4350361824036]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":148.6918,"mean":147.76265,"min":146.8335,"values":[148.6918,146.8335]},"serial_p95_ms":{"max":9.3,"mean":9.3,"min":9.3,"values":[9.3,9.3]}},"profile":"big","qualification":{"ndcg_min":0.9614,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9571,"request_errors":0,"trials":[{"ndcg":0.9641,"recall":0.9598,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.4},"trial":1},{"ndcg":0.9614,"recall":0.9571,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.4},"trial":2}]},"storage_qualification":{"qualified_trials":0,"status":"provisional","total_trials":2,"trials":[{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":1},{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":2}]},"vectors":400000},{"metrics":{"demand_mb":{"max":11374.6,"mean":11266.95,"min":11159.3,"values":[11374.6,11159.3]},"final_allocated_bytes":{"max":2319671296,"mean":2318622720,"min":2317574144,"values":[2319671296,2317574144]},"index_build_seconds":{"max":661.1655999999999,"mean":628.5563999999999,"min":595.9472,"values":[661.1655999999999,595.9472]},"lifecycle_seconds":{"max":1356.4823632240295,"mean":1326.3594111204147,"min":1296.2364590168,"values":[1356.4823632240295,1296.2364590168]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":122.2716,"mean":121.8983,"min":121.525,"values":[122.2716,121.525]},"serial_p95_ms":{"max":12.7,"mean":12.7,"min":12.7,"values":[12.7,12.7]}},"profile":"big","qualification":{"ndcg_min":0.971,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9678,"request_errors":0,"trials":[{"ndcg":0.971,"recall":0.9678,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":1},{"ndcg":0.9733,"recall":0.9701,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":2}]},"storage_qualification":{"qualified_trials":0,"status":"provisional","total_trials":2,"trials":[{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":1},{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":2}]},"vectors":600000},{"metrics":{"demand_mb":{"max":13988.5,"mean":13759.75,"min":13531,"values":[13988.5,13531]},"final_allocated_bytes":{"max":3075039232,"mean":3075026944,"min":3075014656,"values":[3075014656,3075039232]},"index_build_seconds":{"max":1222.4703,"mean":1209.6261,"min":1196.7819,"values":[1222.4703,1196.7819]},"lifecycle_seconds":{"max":1960.324383020401,"mean":1929.8124438524246,"min":1899.3005046844482,"values":[1960.324383020401,1899.3005046844482]},"peak_clients":{"max":5,"mean":5,"min":5,"values":[5,5]},"peak_qps":{"max":121.9874,"mean":120.2367,"min":118.486,"values":[118.486,121.9874]},"serial_p95_ms":{"max":12.9,"mean":12.9,"min":12.9,"values":[12.9,12.9]}},"profile":"big","qualification":{"ndcg_min":0.9675,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9639,"request_errors":0,"trials":[{"ndcg":0.9675,"recall":0.9639,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":1},{"ndcg":0.9676,"recall":0.9639,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":2}]},"storage_qualification":{"qualified_trials":0,"status":"provisional","total_trials":2,"trials":[{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":1},{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":2}]},"vectors":800000},{"metrics":{"demand_mb":{"max":16489.3,"mean":16479.6,"min":16469.9,"values":[16489.3,16469.9]},"final_allocated_bytes":{"max":3832627200,"mean":3831857152,"min":3831087104,"values":[3832627200,3831087104]},"index_build_seconds":{"max":2042.1998,"mean":2021.1676000000002,"min":2000.1354000000001,"values":[2000.1354000000001,2042.1998]},"lifecycle_seconds":{"max":2774.631982564926,"mean":2759.362026333809,"min":2744.0920701026917,"values":[2744.0920701026917,2774.631982564926]},"peak_clients":{"max":30,"mean":30,"min":30,"values":[30,30]},"peak_qps":{"max":84.7592,"mean":84.39615,"min":84.0331,"values":[84.7592,84.0331]},"serial_p95_ms":{"max":16,"mean":14.65,"min":13.3,"values":[13.3,16]}},"profile":"big","qualification":{"ndcg_min":0.9591,"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall_min":0.9547,"request_errors":0,"trials":[{"ndcg":0.9601,"recall":0.956,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":1},{"ndcg":0.9591,"recall":0.9547,"request_errors":0,"selected_effort":{"parameter":"search_effort","value":0.425},"trial":2}]},"storage_qualification":{"qualified_trials":0,"status":"provisional","total_trials":2,"trials":[{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":1},{"exit_code":1,"forced":false,"oom_killed":false,"status":"provisional","stop_command_error":null,"trial":2}]},"vectors":1000000}]

Peer versions

Peer versions were not recorded in the published result fields.

Evidence digests

Receipt: 8926aafda670aa07cf588ebb090dd21a24e8abc9e54f5df6758aa71256b5772a
Aggregate: 193d82a31b6565de8a6ffb41cfe31765206651604cc0d2e41f10d7f219c0b022
Provenance: 193d82a31b6565de8a6ffb41cfe31765206651604cc0d2e41f10d7f219c0b022
Harness commit: 3003c4db794a671ed204917d98e4dfcb85531ff1

Selected measurement

Completed: 9 Sep 2026 · 23:25 UTC
Antfly: v0.2.1 · 9 September 2026 · 00:29 UTC
Runtime: 6h 9m
Platform: linux · arm64
Machine: Apple M4 Max
Build: official-release

Execution span of all twenty retained lifecycles; publication assembly excluded.

Measurement group

Fast scale

Runtime: 1h 51m 30s across 10 fresh lifecycles; cache-only publication assembly excluded.

Two-lifecycle mean, range, and raw trial values

Antfly v0.2.1 · e5261e9e

Vectors	Query status	Clean stops / total	Serial p95 ms	Peak QPS queries/s	Peak clients	Demand anon + shmem MB	Final volume MB	Build s	Recall min	Selected effort
20,000	ok	2 / 2	2.50 [2.40–2.60]; trials 2.60, 2.40	2,048.27 [1,983.44–2,113.11]; trials 1,983.44, 2,113.11	10, 10	2,058.30 [2,026.60–2,090.00]; trials 2,090.00, 2,026.60	78.83 [78.80–78.86]; trials 78.80, 78.86	6.62 [6.43–6.82]; trials 6.82, 6.43	0.9586	0.5, 0.475
40,000	ok	2 / 2	3.00 [3.00–3.00]; trials 3.00, 3.00	1,178.36 [1,177.12–1,179.61]; trials 1,177.12, 1,179.61	5, 5	2,675.20 [1,664.30–3,686.10]; trials 3,686.10, 1,664.30	155.24 [154.95–155.54]; trials 154.95, 155.54	15.31 [12.88–17.74]; trials 17.74, 12.88	0.9612	0.45, 0.45
60,000	ok	2 / 2	3.60 [3.60–3.60]; trials 3.60, 3.60	849.13 [845.92–852.34]; trials 845.92, 852.34	5, 5	2,913.70 [2,708.60–3,118.80]; trials 3,118.80, 2,708.60	233.41 [233.08–233.75]; trials 233.75, 233.08	21.89 [21.82–21.96]; trials 21.82, 21.96	0.9670	0.45, 0.45
80,000	ok	0 / 2	4.30 [4.30–4.30]; trials 4.30, 4.30	627.57 [622.63–632.50]; trials 622.63, 632.50	5, 5	3,279.95 [3,159.70–3,400.20]; trials 3,159.70, 3,400.20	311.32 [310.93–311.72]; trials 310.93, 311.72 (provisional)	30.15 [29.59–30.72]; trials 29.59, 30.72	0.9706	0.45, 0.45
100,000	ok	0 / 2	4.55 [4.30–4.80]; trials 4.80, 4.30	520.61 [509.16–532.05]; trials 509.16, 532.05	5, 5	4,061.25 [3,901.50–4,221.00]; trials 3,901.50, 4,221.00	387.83 [387.42–388.24]; trials 388.24, 387.42 (provisional)	38.67 [38.22–39.12]; trials 39.12, 38.22	0.9583	0.45, 0.425

Serial query p95 (1 client)

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 20,000–100,000 vectors · 768d · serial after client sweep · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Lower is better

Peak throughput (best client count)

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 20,000–100,000 vectors · 768d · best measured client count · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Higher is better

Lifecycle demand (anon + shmem)

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 20,000–100,000 vectors · 768d · cgroup anon + shmem peak from pre-index through 1/5/10/20/30-client sweep · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Lower is better

Final database volume

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 20,000–100,000 vectors · 768d · post-stop allocated bytes · Provisional captures omitted; measured values retained in the table · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Lower is better

Index build time

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 20,000–100,000 vectors · 768d · insert + optimize · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Lower is better

Measurement group

Big scale

Runtime: 4h 18m 17s across 10 fresh lifecycles; cache-only publication assembly excluded.

Two-lifecycle mean, range, and raw trial values

Antfly v0.2.1 · e5261e9e

Vectors	Query status	Clean stops / total	Serial p95 ms	Peak QPS queries/s	Peak clients	Demand anon + shmem MB	Final volume MB	Build s	Recall min	Selected effort
200,000	ok	0 / 2	6.65 [6.60–6.70]; trials 6.60, 6.70	271.46 [270.06–272.86]; trials 272.86, 270.06	5, 5	7,862.45 [7,740.90–7,984.00]; trials 7,740.90, 7,984.00	770.10 [769.52–770.67]; trials 770.67, 769.52 (provisional)	92.26 [92.06–92.47]; trials 92.06, 92.47	0.9685	0.425, 0.425
400,000	ok	0 / 2	9.30 [9.30–9.30]; trials 9.30, 9.30	147.76 [146.83–148.69]; trials 148.69, 146.83	5, 5	9,695.90 [9,650.40–9,741.40]; trials 9,650.40, 9,741.40	1,539.75 [1,539.29–1,540.20]; trials 1,540.20, 1,539.29 (provisional)	293.15 [283.51–302.79]; trials 302.79, 283.51	0.9571	0.4, 0.4
600,000	ok	0 / 2	12.70 [12.70–12.70]; trials 12.70, 12.70	121.90 [121.53–122.27]; trials 122.27, 121.53	5, 5	11,266.95 [11,159.30–11,374.60]; trials 11,374.60, 11,159.30	2,318.62 [2,317.57–2,319.67]; trials 2,319.67, 2,317.57 (provisional)	628.56 [595.95–661.17]; trials 661.17, 595.95	0.9678	0.425, 0.425
800,000	ok	0 / 2	12.90 [12.90–12.90]; trials 12.90, 12.90	120.24 [118.49–121.99]; trials 118.49, 121.99	5, 5	13,759.75 [13,531.00–13,988.50]; trials 13,988.50, 13,531.00	3,075.03 [3,075.01–3,075.04]; trials 3,075.01, 3,075.04 (provisional)	1,209.63 [1,196.78–1,222.47]; trials 1,222.47, 1,196.78	0.9639	0.425, 0.425
1,000,000	ok	0 / 2	14.65 [13.30–16.00]; trials 13.30, 16.00	84.40 [84.03–84.76]; trials 84.76, 84.03	30, 30	16,479.60 [16,469.90–16,489.30]; trials 16,489.30, 16,469.90	3,831.86 [3,831.09–3,832.63]; trials 3,832.63, 3,831.09 (provisional)	2,021.17 [2,000.14–2,042.20]; trials 2,000.14, 2,042.20	0.9547	0.425, 0.425

Serial query p95 (1 client)

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 200,000–1,000,000 vectors · 768d · serial after client sweep · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Lower is better

Peak throughput (best client count)

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 200,000–1,000,000 vectors · 768d · best measured client count · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Higher is better

Lifecycle demand (anon + shmem)

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 200,000–1,000,000 vectors · 768d · cgroup anon + shmem peak from pre-index through 1/5/10/20/30-client sweep · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Lower is better

Index build time

Antfly v0.2.1 (e5261e9e2937d03943b518974bf8063351ad1449) · official-release · Platform: linux-arm64 · Processor: Apple M4 Max · 200,000–1,000,000 vectors · 768d · insert + optimize · seed antfly-circus-vdbbench-v1 · 200 calibration / 800 held-out · 1/5/10/20/30 clients × 30 s · Lower is better

vector-filtered-ann

complete

Filtered ANN

Three independent v0.2.1 lifecycles on a 1% tenant filter over 50K vectors.

Methodology scope · protocol · interpretation

Subject and execution

Filtered protocol

50K 1536-dimensional vectors, a 1% tenant filter (500 candidates), top 100, 200 calibration queries and 800 disjoint holdout queries. One shard; calibration selects search effort on the calibration partition only.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: vector-filtered-ann
Domain: vector/filtered-ann
Source: https://github.com/antflydb/VectorDBBench
Upstream: https://github.com/zilliztech/VectorDBBench
Known Antfly versions: v0.2.*
Expected duration: 270 min
Enabled: yes

Benchmark parameters

{"execution_environment":{"container_runtime":"rootless Podman 6.1.1","processor":"Apple M4 Max","vm_cpus":10,"vm_memory_gib":24},"num_shards":1,"quality_floor_passed":true,"selected_settings":[{"num_shards":1,"parameter":"search_effort","trial":1,"value":0.35},{"num_shards":1,"parameter":"search_effort","trial":2,"value":0.35},{"num_shards":1,"parameter":"search_effort","trial":3,"value":0.35}],"workload":"tenant-scoped-1536d-50K-1pct"}

Trial variation

[{"below_qualification_floor":false,"cache_inclusive_peak_mb":{"max":1746.4,"mean":1652.0333,"min":1586.7,"trials":[1746.4,1586.7,1623]},"cgroup_current_peak_mb":{"max":1744.7,"mean":1647.0667,"min":1575.3,"trials":[1744.7,1575.3,1621.2]},"demand_peak_mb":{"max":977.9,"mean":962.6,"min":942.7,"trials":[977.9,967.2,942.7]},"display_name":"Weaviate","engine":"weaviate","engine_version":"1.39.2","images":["cr.weaviate.io/semitechnologies/weaviate:1.39.2@sha256:5c62e5cbce4c48fc770abcee71099d9de32aad3b44d12299611f8135ee412362"],"insert_s":{"max":57.3,"mean":56.9,"min":56.4,"trials":[56.4,57.3,57]},"load_s":{"max":57.3,"mean":56.9333,"min":56.5,"trials":[56.5,57.3,57]},"ndcg":{"max":1,"mean":1,"min":1,"trials":[1,1,1]},"peak_p99_ms":{"max":11.937,"mean":11.7119,"min":11.4516,"trials":[11.451609040009306,11.747034159616303,11.936994999414313]},"peak_qps":{"max":1154.1408,"mean":1143.9375,"min":1129.6205,"trials":[1154.1408,1148.0512,1129.6205]},"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall":{"max":1,"mean":1,"min":1,"trials":[1,1,1]},"request_errors":0,"selected_settings":[{"parameter":"ef","trial":1,"value":64},{"parameter":"ef","trial":2,"value":64},{"parameter":"ef","trial":3,"value":64}],"serial_p95_ms":{"max":5.9,"mean":5.8667,"min":5.8,"trials":[5.8999999999999995,5.8999999999999995,5.8]},"serial_p99_ms":{"max":6.9,"mean":6.7333,"min":6.6,"trials":[6.7,6.6,6.8999999999999995]},"storage":{"aggregation_protocol_fingerprint":"ea275ea5ccbda1ec0de40401b1294b82af6ff50c8ab742670ad542995d1ba957","apparent_mean_bytes":392665790,"apparent_values_bytes":[392614606,392813351,392569414],"engine":"weaviate","max_bytes":392814592,"mean_bytes":392669867,"metric":"final_allocated_bytes","min_bytes":392572928,"reasons":[],"receipt_protocol_fingerprints":["197ff7a2f59206ab08da22e17085c8769e6df7314f808a85e13d476d7bbb4ac9","a7a4704672c308b64e8264b7617da187447b0e252ee95badeb70752aaf427f94","1d70071cef9e57b4494d01e1a15f9f270bfdaa102f41251cdbf8767bfe706469"],"relative_range":0.0006154381084840056,"status":"qualified","trial_count":3,"trial_ids":["1","2","3"],"values_bytes":[392622080,392814592,392572928],"workload":"tenant-scoped-1536d-50K-1pct"},"trial_count":3},{"below_qualification_floor":false,"cache_inclusive_peak_mb":{"max":4023.2,"mean":3866.8667,"min":3749.8,"trials":[4023.2,3749.8,3827.6]},"cgroup_current_peak_mb":{"max":3951.9,"mean":3796.7667,"min":3684.8,"trials":[3951.9,3684.8,3753.6]},"demand_peak_mb":{"max":1951.6,"mean":1912.8333,"min":1858,"trials":[1951.6,1858,1928.9]},"display_name":"Elasticsearch","engine":"elasticsearch","engine_version":"9.5.2","images":["docker.elastic.co/elasticsearch/elasticsearch:9.5.2@sha256:9c1e1afc2bda921b35025e21c72ec6e392266995aa35ad6a47887363592718be"],"insert_s":{"max":26.1,"mean":26.0333,"min":26,"trials":[26,26,26.1]},"load_s":{"max":26.2,"mean":26.1333,"min":26.1,"trials":[26.1,26.1,26.2]},"ndcg":{"max":1,"mean":1,"min":1,"trials":[1,1,1]},"peak_p99_ms":{"max":14.358,"mean":13.5081,"min":12.8423,"trials":[13.3240393799656,12.842340830211468,14.357986519808037]},"peak_qps":{"max":5893.4067,"mean":5810.7257,"min":5670.9428,"trials":[5867.8277,5893.4067,5670.9428]},"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall":{"max":1,"mean":1,"min":1,"trials":[1,1,1]},"request_errors":0,"selected_settings":[{"parameter":"num_candidates","trial":1,"value":128},{"parameter":"num_candidates","trial":2,"value":128},{"parameter":"num_candidates","trial":3,"value":128}],"serial_p95_ms":{"max":1.5,"mean":1.5,"min":1.5,"trials":[1.5,1.5,1.5]},"serial_p99_ms":{"max":1.8,"mean":1.8,"min":1.8,"trials":[1.8,1.8,1.8]},"storage":{"aggregation_protocol_fingerprint":"e732be26c67ae8214d7b4d520ce0600f718aa964f033df134d7eb16ae1edbb24","apparent_mean_bytes":312139626,"apparent_values_bytes":[312156451,312116394,312146033],"engine":"elasticsearch","max_bytes":312373248,"mean_bytes":312359595,"metric":"final_allocated_bytes","min_bytes":312340480,"reasons":[],"receipt_protocol_fingerprints":["2a2599c1202fa1cda60d720c6cc8e8082643a012b317595599335a394e5e765e","b2f306b562646df562499643123c2dcfa6f0b97ae2cbba25a8128d18f6f30782","58b18aa47d8b9d2c810518873441111a7aa9eb9f226aae32c0fe82fd30c30a86"],"relative_range":0.00010490473338899112,"status":"qualified","trial_count":3,"trial_ids":["1","2","3"],"values_bytes":[312373248,312340480,312365056],"workload":"tenant-scoped-1536d-50K-1pct"},"trial_count":3},{"below_qualification_floor":false,"cache_inclusive_peak_mb":{"max":5760,"mean":5485.7333,"min":5052.2,"trials":[5760,5645,5052.2]},"cgroup_current_peak_mb":{"max":5608.1,"mean":5284.5667,"min":4846.1,"trials":[5608.1,5399.5,4846.1]},"demand_peak_mb":{"max":3038.8,"mean":2980.7667,"min":2942.6,"trials":[2960.9,3038.8,2942.6]},"display_name":"Milvus","engine":"milvus","engine_version":"v2.6.22","images":["milvusdb/milvus:v2.6.22@sha256:a8ac051e59eb084d41bd317ec51aac28553d664e91c43a806ccd6a1538abc1df"],"insert_s":{"max":4,"mean":3.7333,"min":3.6,"trials":[3.6,3.6,4]},"load_s":{"max":51.2,"mean":44.2667,"min":40.5,"trials":[51.2,40.5,41.1]},"ndcg":{"max":1,"mean":1,"min":1,"trials":[1,1,1]},"peak_p99_ms":{"max":9.6052,"mean":9.163,"min":8.8996,"trials":[8.89960063977469,8.984309360421321,9.605236999559574]},"peak_qps":{"max":6925.8747,"mean":6789.2853,"min":6564.5179,"trials":[6925.8747,6877.4633,6564.5179]},"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall":{"max":1,"mean":1,"min":1,"trials":[1,1,1]},"request_errors":0,"selected_settings":[{"parameter":"ef_search","trial":1,"value":100},{"parameter":"ef_search","trial":2,"value":100},{"parameter":"ef_search","trial":3,"value":100}],"serial_p95_ms":{"max":1.3,"mean":1.2667,"min":1.2,"trials":[1.2,1.3,1.3]},"serial_p99_ms":{"max":1.5,"mean":1.5,"min":1.5,"trials":[1.5,1.5,1.5]},"storage":{"aggregation_protocol_fingerprint":"111f529febc3e0471dff12b6f3cca736bfdbebc83effe035b42e196666c87a65","apparent_mean_bytes":2393476179,"apparent_values_bytes":[2393471634,2393481213,2393475690],"engine":"milvus","max_bytes":2394193920,"mean_bytes":2394193920,"metric":"final_allocated_bytes","min_bytes":2394193920,"reasons":[],"receipt_protocol_fingerprints":["008d4446ef82b56a8fe5ae2e57039d1f0f565bbd5635570bac1d4f6aab57bc77","ecebc9003fbc200dda6984ef8c80f21525d37dba8d6683c3f5aeb319ebe553c3","1fe4ef93612ee35620b31a1465bf4a079fddd695d8ee17d86869e475e1d37682"],"relative_range":0,"status":"qualified","trial_count":3,"trial_ids":["1","2","3"],"values_bytes":[2394193920,2394193920,2394193920],"workload":"tenant-scoped-1536d-50K-1pct"},"trial_count":3},{"below_qualification_floor":false,"cache_inclusive_peak_mb":{"max":3570.5,"mean":3545.6,"min":3532.3,"trials":[3570.5,3532.3,3534]},"cgroup_current_peak_mb":{"max":3533,"mean":3270.7667,"min":3126.3,"trials":[3153,3126.3,3533]},"demand_peak_mb":{"max":2303.8,"mean":2303.6667,"min":2303.6,"trials":[2303.6,2303.8,2303.6]},"display_name":"pgvector","engine":"pgvector","engine_version":"0.8.6-pg16","images":["pgvector/pgvector:0.8.6-pg16@sha256:ccc6e83d6e35e931dc7c5def2022729d5a6c370318d099181995567ff1fb4d6b"],"insert_s":{"max":4.9,"mean":4.8,"min":4.7,"trials":[4.9,4.8,4.7]},"load_s":{"max":20.4,"mean":20.3,"min":20.2,"trials":[20.4,20.2,20.3]},"ndcg":{"max":0.9662,"mean":0.9661,"min":0.9661,"trials":[0.9661,0.9661,0.9662]},"peak_p99_ms":{"max":228.1445,"mean":224.4917,"min":221.6439,"trials":[223.68666072001358,221.64391372021782,228.14453434006282]},"peak_qps":{"max":170.4544,"mean":167.6261,"min":165.3733,"trials":[167.0507,170.4544,165.3733]},"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall":{"max":0.9574,"mean":0.9573,"min":0.9573,"trials":[0.9573,0.9573,0.9574]},"request_errors":0,"selected_settings":[{"parameter":"ef_search","trial":1,"value":64},{"parameter":"ef_search","trial":2,"value":64},{"parameter":"ef_search","trial":3,"value":64}],"serial_p95_ms":{"max":53.2,"mean":52.9333,"min":52.4,"trials":[52.400000000000006,53.199999999999996,53.199999999999996]},"serial_p99_ms":{"max":56.4,"mean":56.0667,"min":55.8,"trials":[56,55.800000000000004,56.4]},"storage":{"aggregation_protocol_fingerprint":"5bd5eec398e2a4331074e1c5b70954eb4b35b184ec21ad674304add90fee2c42","apparent_mean_bytes":1523896040,"apparent_values_bytes":[1523896040,1523896040,1523896040],"engine":"pgvector","max_bytes":1523974144,"mean_bytes":1523972779,"metric":"final_allocated_bytes","min_bytes":1523970048,"reasons":[],"receipt_protocol_fingerprints":["b215304569a34e3c9a5029b2499e743c3c360d6cf4048b65869b9447c0a98098","79a5600ae276ddc97e36adeb9a74a3951b164581b47050a0e370e2f2b3792856","332ca12a96fb884d59a84fb2c5a80aafb8450c8a55fb071bab593c75088901f6"],"relative_range":0.000002687712049281888,"status":"qualified","trial_count":3,"trial_ids":["1","2","3"],"values_bytes":[1523970048,1523974144,1523974144],"workload":"tenant-scoped-1536d-50K-1pct"},"trial_count":3},{"below_qualification_floor":false,"cache_inclusive_peak_mb":{"max":882.2,"mean":862.0333,"min":850.7,"trials":[882.2,850.7,853.2]},"cgroup_current_peak_mb":{"max":881.1,"mean":860.4333,"min":848,"trials":[881.1,848,852.2]},"demand_peak_mb":{"max":502.9,"mean":499.9,"min":496.5,"trials":[500.3,496.5,502.9]},"display_name":"Chroma","engine":"chroma","engine_version":"1.5.9","images":["chromadb/chroma:1.5.9@sha256:1e0b73a187a28757c572acba508c46f48c9e8b0acaf5c20e6d95cdedce1acdf6"],"insert_s":{"max":45,"mean":44.8333,"min":44.5,"trials":[45,45,44.5]},"load_s":{"max":45.2,"mean":45,"min":44.6,"trials":[45.2,45.2,44.6]},"ndcg":{"max":0.9999,"mean":0.9999,"min":0.9999,"trials":[0.9999,0.9999,0.9999]},"peak_p99_ms":{"max":581.5998,"mean":573.4335,"min":562.5068,"trials":[581.5997719297643,562.5068286099986,576.1939451607034]},"peak_qps":{"max":111.6785,"mean":111.5999,"min":111.5605,"trials":[111.5606,111.5605,111.6785]},"query_indices_sha256":"7be5c5f24dfa31b66c5dfc3280c8f1bc5c1c8db3a35046a1a1ab9d50a0d232f7","recall":{"max":0.9999,"mean":0.9999,"min":0.9999,"trials":[0.9999,0.9999,0.9999]},"request_errors":0,"selected_settings":[{"parameter":"ef_search","trial":1,"value":256},{"parameter":"ef_search","trial":2,"value":256},{"parameter":"ef_search","trial":3,"value":256}],"serial_p95_ms":{"max":78.9,"mean":78.5333,"min":78.2,"trials":[78.2,78.5,78.89999999999999]},"serial_p99_ms":{"max":80.8,"mean":79.9667,"min":79.3,"trials":[79.8,79.3,80.8]},"storage":{"aggregation_protocol_fingerprint":"e7b2b75a3aebddfdf2974e664eb58fe9b6557eac089395de53a54c6cb663802e","apparent_mean_bytes":336122824,"apparent_values_bytes":[336812844,336689896,334865732],"engine":"chroma","max_bytes":336818176,"mean_bytes":336131413,"metric":"final_allocated_bytes","min_bytes":334880768,"reasons":[],"receipt_protocol_fingerprints":["5a5b6124abe744abc0730121250f99efd3a41a5ffe5ec59ff8fe9c7c80b54a7a","63979eedbce3ea1b3b7818eedb15452a384d7cc4a3ad348a869e8719382c9234","1e0ccdc987a18862bfda60391f36e086c8d99191e76f02263f33b5971dbfa28e"],"relative_range":0.0057638409358625456,"status":"qualified","trial_count":3,"trial_ids":["1","2","3"],"values_bytes":[336818176,336695296,334880768],"workload":"tenant-scoped-1536d-50K-1pct"},"trial_count":3}]

Peer versions

1.39.2
9.5.2
v2.6.22
0.8.6-pg16
1.5.9

Evidence digests

Receipt: 4e7304550ecc9bc00d8901bffba9af3620288081b26c86fd068edaccb1877b38
Aggregate: 9f1b3189bed258d7a7dc355215ac388d6b1f011cf6ab6a064bd7448b4bcb3617
Provenance: 9f1b3189bed258d7a7dc355215ac388d6b1f011cf6ab6a064bd7448b4bcb3617
Harness commit

Peer cohort evidence · 1536d-50K-tenant-scoped-1pct

Completed: 31 Aug 2026 · 16:11 UTC
Fresh lifecycles: 15
Peer variants: 5
Receipt: 069d2e8eb1e613e4c513af08320564af81dd4a41d8991adddcd7064db043f85b
Aggregate: 04d4f8a9071bf1fd4abf048b985f02d332a6eefa68560aaea086f67568f43c96
Provenance: 04d4f8a9071bf1fd4abf048b985f02d332a6eefa68560aaea086f67568f43c96

Selected measurement

Completed: 9 Sep 2026 · 06:19 UTC
Antfly: v0.2.1 · 9 September 2026 · 00:29 UTC
Runtime: 25m 46s
Platform: linux · arm64
Machine: Apple M4 Max
Build: official-release

Peer cohort · 1536d-50K-tenant-scoped-1pct

Completed: 31 Aug 2026 · 16:11 UTC
Variants: 5 independently executed peers
Lifecycles: 15 fresh
Compatibility: Historical reference · not a matched runtime

Peak filtered throughput

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · tenant-scoped 1536d/50K · 3 fresh lifecycles · Higher is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Peak-concurrency p99

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · tenant-scoped 1536d/50K · 3 fresh lifecycles · Lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Demand working set

Tenant-scoped filtered ANN

Antfly v0.2.1 · e5261e9e

Engine	Recall@100	nDCG	Peak QPS queries/s	Peak p99 ms	Serial p99 ms	Demand MB	Final volume MB	Trials
Antfly	1.0000	1.0000	2,879.8	47.57	2.57	7,645.4	356.5	3
Weaviate	1.0000	1.0000	1,143.9	11.71	6.73	962.6	392.7	3
Elasticsearch	1.0000	1.0000	5,810.7	13.51	1.80	1,912.8	312.4	3
Milvus	1.0000	1.0000	6,789.3	9.16	1.50	2,980.8	2,394.2	3
pgvector	0.9573	0.9661	167.6	224.49	56.07	2,303.7	1,524.0	3
Chroma	0.9999	0.9999	111.6	573.43	79.97	499.9	336.1	3

III

fulltext-miracl-scale

complete

MIRACL full-text scale

MIRACL English lexical scaling across a 32.9M-passage source corpus, Antfly-only with three trials per size at 250K/1M/2M/3M.

Methodology scope · protocol · interpretation

Full-text MIRACL scale methodology

Scope

fulltext-miracl-scale measures full-text operational cost with retrieval-quality guardrails as a judged English passage field grows. Antfly, Elasticsearch, Quickwit, and PostgreSQL built-in full-text search index identical nested fixtures and answer the same MIRACL English development queries. The lane reports complete service memory demand, final allocated database-volume bytes, index duration, query p95/QPS, and judged ranking metrics.

This lane supplies million-scale lexical operational evidence. Search Benchmark Game owns query-class throughput, BEIR owns native-hybrid uplift, and STaRK owns structured retrieval. No service-count, round-trip consolidation, or official MIRACL leaderboard claim follows from this lane.

This is an explicit diagnostic lane, not a member of the honed profile.

For an Antfly-first release refresh, set MIRACL_SCALE_ENGINES=antfly. Explicit engine subsets retain all four common passage sizes, three fresh lifecycles per size, the complete judged query panel, and the same memory and storage checks. Reports identify selected engines and unrun peers separately from lifecycle success: a successful Antfly-only run is not a complete cross-engine cohort. Existing peer evidence is not rerun or relabeled by this selection.

Source and license

The corpus is the English portion of MIRACL v1.0: 32,893,221 passages from 5,758,285 English Wikipedia articles. The public development panel contains 799 queries and 8,350 human positive and negative judgments. Corpus repository revision d921ec7e349ce0d28daf30b2da9da5ee698bef0d and judgment repository revision 5be20db9509754dadad47689368639fcec739c00 are immutable cache identities. Every corpus shard is pinned by its Hugging Face LFS SHA-256; topics and qrels are pinned by observed SHA-256.

The MIRACL dataset cards declare Apache-2.0. The corpus is derived from English Wikipedia, so applicable Wikipedia attribution and share-alike obligations are retained rather than treating the text as newly permissive content.

Citation:

Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. “MIRACL: A Multilingual Retrieval Dataset Covering 18 Diverse Languages.” Transactions of the Association for Computational Linguistics 11, 2023. DOI 10.1162/tacl_a_00595.

Deterministic nested fixtures

Each source passage becomes one canonical UTF-8 JSONL record with MIRACL docid as id and title + U+000A + text as the searchable text. Source strings are not normalized, truncated, or analyzer-normalized.

Every document appearing in a development qrel is placed first, ordered by UTF-8 document ID. Natural distractors are ordered by SHA-256 of the UTF-8 document ID. The implementation scans every pinned corpus shard, retains low SHA-256 prefix buckets, externally sorts only the retained candidate set under LC_ALL=C, and writes exact nested prefixes of 250,000, 1,000,000, 2,000,000, 3,000,000, and 5,000,000 passages. Every judged document must occur in every prefix. Selection does not inspect relevance grade.

Queries are exact MIRACL development topic text in UTF-8 query-ID order. Qrels are canonical four-column TREC rows in query-ID/document-ID order. Preparation rejects duplicate judgments, missing positive queries, missing judged documents, malformed rows, source count drift, and digest drift. preparation-receipt.json binds every source asset, full-corpus canonical digest, prefix identity, query order, qrels, grade histogram, judged-document counts, and selection algorithm.

Engine and query protocol

All engines receive one text field with product-default English lexical analysis. Antfly retains bounded ingest retries, a 900-second request timeout required by observed 5M-pass compaction stalls, and its searchable-readiness gate. Elasticsearch uses one primary shard, no replicas, a 1 GiB heap, refresh, and force merge. Quickwit uses its default tokenizer with positions and field norms; startup ingest retries tolerate the temporary no-shard gate, and publication waits up to 900 seconds for the exact indexed count. PostgreSQL 16 uses a generated English tsvector, a GIN index, and ts_rank_cd, with otherwise default server settings.

The runner streams the declared fixture and requires the indexed count to match. The first ten ordered queries are warmups. The measured pass includes all 799 queries once at concurrency one with top_k=100. Per-query latency spans request dispatch through complete response decoding; QPS is query count divided by measured wall time. Raw evidence contains only query ID, one-based ranked IDs, latency, and normalized error status.

Quality metrics

MIRACL English dev judgments are binary. The lane macro-averages:

nDCG@10;
MRR@10;
Recall@100;
Judged@100, the fraction of returned unique IDs having either a positive or negative human judgment.

MIRACL uses pooled judgments: an unjudged passage is not proven irrelevant. Judged@100 exposes assessment coverage, while nDCG, MRR, and Recall act as consistent cross-engine guardrails. Verification recomputes every metric from retained ranked IDs and rejects request errors, unknown IDs, duplicates, more than 100 hits, query-order drift, and summary disagreement.

Resource lifecycle and failures

The canonical publication cohort is four engines by four common prefix sizes (250K, 1M, 2M, and 3M) by three fresh trials. The retained 5M fixture is a diagnostic boundary: exploratory evidence showed PostgreSQL ranking would consume roughly another half day and Antfly entered sustained full-text merge backpressure, while Elasticsearch and Quickwit completed. Odd canonical trials run Antfly, Elasticsearch, Quickwit, PostgreSQL; even trials reverse the order. Cgroup v2 sampling begins before indexing and continues through all queries at 300 ms. Every engine is measured as one complete service container, and PostgreSQL shmem is included. Headline demand is peak concurrent anon + shmem; memory.current and kernel peaks remain diagnostics.

Fresh labeled Docker volumes capture empty, ready, post_workload, and stopped final barriers. post_workload follows a fixed 30-second settlement. The shared storage finalize command owns the 60-second service-stop policy and captures final bytes even after an abnormal or forced stop. Such observations remain provisional evidence in the failed cell; they do not enter the successful-cell cache or positive aggregate. Final allocated and apparent bytes use arithmetic mean plus every value/range; demand, index duration, p95, QPS, and judged metrics use median plus every value/range.

Successful cells require the shared schema-v3 container-memory contract and a finalized, internally consistent retained timeline. The memory validator, sampler, engine adapters, preparation code, and cell-receipt writer are included in the cache source identity. Cache objects retain every file listed with its checksum in the cell receipt, including lifecycle phase and runner logs.

Every planned cell writes normalized status and a list of file checksums. All cells are attempted once. Aggregate status is ok for complete success, failed for a repeated uniform terminal gate across a complete engine/size group, and partial for every other fully accounted mix. Unaccounted cells invalidate finalization. Partial accounting is generated from the report’s validated canonical cohort and included in its summary and aggregate; publication selects the finalized result.json rather than a raw summary copy.

The smaller CRAG full-text footprint lane remains available through the internal gauntlet profile, not the active honed publication profile. Its frozen evidence and defaults are not relabeled as MIRACL scaling evidence.

Circus packages presentation-neutral aggregate, source receipt, summary, methodology, input manifest, and preparation receipt through evidence_handoff.py; the site repository owns evidence selection and presentation.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: fulltext-miracl-scale
Domain: full-text/retrieval
Source: https://huggingface.co/datasets/miracl
Known Antfly versions: v0.2.1
Enabled: yes

Benchmark parameters

{"common_complete_through_passages":3000000,"corpus_passages":32893221,"corpus_sizes":[250000,1000000,2000000,3000000],"dataset":"MIRACL v1.0 English development split","evidence_class":"3-trial descriptive scaling evidence","execution_environment":{"container_runtime":"rootless Podman 6.1.1","processor":"Apple M4 Max","vm_cpus":10,"vm_memory_gib":24},"fresh_lifecycles_per_point":3,"judgments":8350,"memory_metric":"cgroup v2 anon + shmem","omissions":[],"queries":799,"storage_metric":"final allocated database-volume bytes"}

Trial variation

[{"engine":"antfly","passages":250000,"statistics":{"cache_inclusive_peak_bytes":{"max":662773760,"median":643526656,"min":635961344,"values":[643526656,635961344,662773760]},"demand_peak_bytes":{"max":367497216,"median":360222720,"min":335945728,"values":[335945728,367497216,360222720]},"final_allocated_bytes":{"max":165314560,"mean":160458069,"min":157028352,"values":[165314560,157028352,159031296]},"final_apparent_bytes":{"max":165030878,"mean":160178727,"min":156763639,"values":[165030878,156763639,158741663]},"index_seconds":{"max":49.768688917160034,"median":49.60022999998182,"min":49.302907791920006,"values":[49.60022999998182,49.768688917160034,49.302907791920006]},"judged_at_100":{"max":0.09754862596887764,"median":0.09724825049954099,"min":0.09701045325298278,"values":[0.09724825049954099,0.09701045325298278,0.09754862596887764]},"kernel_peak_sum_bytes":{"max":697794560,"median":662773760,"min":652513280,"values":[652513280,697794560,662773760]},"mrr_at_10":{"max":0.561415559131454,"median":0.5597433299560959,"min":0.5557288872996007,"values":[0.5557288872996007,0.561415559131454,0.5597433299560959]},"ndcg_at_10":{"max":0.5742171280594587,"median":0.5720035349502223,"min":0.571478915951799,"values":[0.571478915951799,0.5720035349502223,0.5742171280594587]},"p95_ms":{"max":2.1347706904634824,"median":2.056112978607416,"min":2.0411999197676773,"values":[2.1347706904634824,2.0411999197676773,2.056112978607416]},"qps":{"max":914.3338863821377,"median":905.5369070160918,"min":871.0463479490764,"values":[871.0463479490764,914.3338863821377,905.5369070160918]},"recall_at_100":{"max":0.9461876497858975,"median":0.941420236695581,"min":0.9379606001007753,"values":[0.941420236695581,0.9379606001007753,0.9461876497858975]},"request_errors":{"max":0,"median":0,"min":0,"values":[0,0,0]}}},{"engine":"antfly","passages":1000000,"statistics":{"cache_inclusive_peak_bytes":{"max":2677829632,"median":2385211392,"min":2316537856,"values":[2677829632,2316537856,2385211392]},"demand_peak_bytes":{"max":1998782464,"median":1554051072,"min":1549901824,"values":[1998782464,1554051072,1549901824]},"final_allocated_bytes":{"max":601178112,"mean":597860352,"min":594464768,"values":[594464768,597938176,601178112]},"final_apparent_bytes":{"max":600757379,"mean":597454873,"min":594075450,"values":[594075450,597531791,600757379]},"index_seconds":{"max":313.23897816613317,"median":310.7671229999978,"min":303.05622674990445,"values":[310.7671229999978,313.23897816613317,303.05622674990445]},"judged_at_100":{"max":0.09390969734044519,"median":0.09373447831666547,"min":0.09359680622655282,"values":[0.09373447831666547,0.09359680622655282,0.09390969734044519]},"kernel_peak_sum_bytes":{"max":2712698880,"median":2499264512,"min":2435997696,"values":[2712698880,2435997696,2499264512]},"mrr_at_10":{"max":0.5361538232314202,"median":0.5347517730496454,"min":0.5317673878061863,"values":[0.5317673878061863,0.5347517730496454,0.5361538232314202]},"ndcg_at_10":{"max":0.5307170736865128,"median":0.5250093716515419,"min":0.5246507972307869,"values":[0.5246507972307869,0.5250093716515419,0.5307170736865128]},"p95_ms":{"max":5.691096489317709,"median":5.549583327956498,"min":5.4536085575819,"values":[5.549583327956498,5.691096489317709,5.4536085575819]},"qps":{"max":437.7340742196044,"median":435.75771050991904,"min":431.34391028314,"values":[435.75771050991904,431.34391028314,437.7340742196044]},"recall_at_100":{"max":0.9242979138786398,"median":0.9211098559346369,"min":0.9118778072157295,"values":[0.9211098559346369,0.9118778072157295,0.9242979138786398]},"request_errors":{"max":0,"median":0,"min":0,"values":[0,0,0]}}},{"engine":"antfly","passages":2000000,"statistics":{"cache_inclusive_peak_bytes":{"max":5736464384,"median":4884746240,"min":4874244096,"values":[4874244096,5736464384,4884746240]},"demand_peak_bytes":{"max":4415750144,"median":3606478848,"min":3264712704,"values":[3606478848,4415750144,3264712704]},"final_allocated_bytes":{"max":1232748544,"mean":1176558251,"min":1143627776,"values":[1232748544,1153298432,1143627776]},"final_apparent_bytes":{"max":1232444850,"mean":1176267727,"min":1143366458,"values":[1232444850,1152991874,1143366458]},"index_seconds":{"max":831.3961841668934,"median":796.4509052911308,"min":793.8975587501191,"values":[793.8975587501191,831.3961841668934,796.4509052911308]},"judged_at_100":{"max":0.09120910942599818,"median":0.08410022331836364,"min":0.08252325210434612,"values":[0.08410022331836364,0.09120910942599818,0.08252325210434612]},"kernel_peak_sum_bytes":{"max":5756252160,"median":4986253312,"min":4965072896,"values":[4986253312,5756252160,4965072896]},"mrr_at_10":{"max":0.5111548165365437,"median":0.48877366549456663,"min":0.4734370343882234,"values":[0.48877366549456663,0.5111548165365437,0.4734370343882234]},"ndcg_at_10":{"max":0.4928254390744837,"median":0.45439202899143494,"min":0.4410892122173445,"values":[0.45439202899143494,0.4928254390744837,0.4410892122173445]},"p95_ms":{"max":10.222457675263284,"median":9.933420829474915,"min":9.785341680981213,"values":[10.222457675263284,9.933420829474915,9.785341680981213]},"qps":{"max":268.3976692864554,"median":264.89940337972695,"min":264.05064489159383,"values":[264.05064489159383,268.3976692864554,264.89940337972695]},"recall_at_100":{"max":0.8927798456584439,"median":0.8147511680365247,"min":0.8032830540339927,"values":[0.8147511680365247,0.8927798456584439,0.8032830540339927]},"request_errors":{"max":0,"median":0,"min":0,"values":[0,0,0]}}},{"engine":"antfly","passages":3000000,"statistics":{"cache_inclusive_peak_bytes":{"max":10431827968,"median":9545912320,"min":9250742272,"values":[10431827968,9545912320,9250742272]},"demand_peak_bytes":{"max":7675752448,"median":6884290560,"min":6631145472,"values":[7675752448,6884290560,6631145472]},"final_allocated_bytes":{"max":1716150272,"mean":1706610688,"min":1701502976,"values":[1702178816,1716150272,1701502976]},"final_apparent_bytes":{"max":1715565681,"mean":1706016010,"min":1700898652,"values":[1701583696,1715565681,1700898652]},"index_seconds":{"max":1438.0724941249937,"median":1411.5304007499944,"min":1400.1734712920152,"values":[1411.5304007499944,1400.1734712920152,1438.0724941249937]},"judged_at_100":{"max":0.09017052565707134,"median":0.08997027534418023,"min":0.07545212765957447,"values":[0.07545212765957447,0.08997027534418023,0.09017052565707134]},"kernel_peak_sum_bytes":{"max":10534768640,"median":9556123648,"min":9281851392,"values":[10534768640,9556123648,9281851392]},"mrr_at_10":{"max":0.4994526888769692,"median":0.4969028746250273,"min":0.41213719530365334,"values":[0.41213719530365334,0.4969028746250273,0.4994526888769692]},"ndcg_at_10":{"max":0.47599767081134936,"median":0.47111881767231883,"min":0.3818040945864337,"values":[0.3818040945864337,0.47111881767231883,0.47599767081134936]},"p95_ms":{"max":14.670753595419217,"median":14.365978911519022,"min":14.152849861420679,"values":[14.365978911519022,14.152849861420679,14.670753595419217]},"qps":{"max":193.96626738319878,"median":193.37089362886644,"min":192.88069080848746,"values":[193.37089362886644,193.96626738319878,192.88069080848746]},"recall_at_100":{"max":0.8841118587989676,"median":0.8791368900880792,"min":0.7443128766658178,"values":[0.7443128766658178,0.8791368900880792,0.8841118587989676]},"request_errors":{"max":0,"median":0,"min":0,"values":[0,0,0]}}}]

Peer versions

Peer versions were not recorded in the published result fields.

Evidence digests

Receipt: f8310fa780bb3b3d814fc31a851877460700e94dd9132ef548cf379a63638d10
Aggregate: 741d87f03a5e28f3c4ff8809a74f0fb2d25c83b547d5107ea9981b2127a8f38f
Provenance: a66e05d6e85be4b0fb05a6723703c3bb1a083b53e0bfc39753e1c058f6144011
Harness commit

Peer cohort evidence · MIRACL passages-250000, MIRACL passages-1000000, MIRACL passages-2000000, MIRACL passages-3000000

Completed: 2 Sep 2026 · 18:00 UTC
Fresh lifecycles: 12
Peer variants: 3
Receipt: 0e9849866c690064980eef699805cb5123c4b19ea3979f02fd20995036488c95
Aggregate: 4069aa330155db2450d71df331764658fb9e7bdacfd3c4c8962e5c0fb105e162
Provenance: b89529140dde5c5556f0e7c32839ec62b59266fcde0e20a7d664c52636aef0ee

Selected measurement

Completed: 9 Sep 2026 · 08:39 UTC
Antfly: v0.2.1 · 9 September 2026 · 00:29 UTC
Runtime: 2h 18m
Platform: linux · arm64
Machine: Apple M4 Max
Build: official-release

Peer cohort · MIRACL passages-250000, MIRACL passages-1000000, MIRACL passages-2000000, MIRACL passages-3000000

Completed: 2 Sep 2026 · 18:00 UTC
Variants: 3 independently executed peers
Lifecycles: 12 fresh
Compatibility: Historical reference · not a matched runtime

Query p95

Antfly v0.2.1 (e5261e9e, official-release) · Linux/arm64 containers on Apple M4 Max · MIRACL English dev · 3 fresh lifecycles planned per point · Log scale · lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Demand working set

Antfly v0.2.1 (e5261e9e, official-release) · Linux/arm64 containers on Apple M4 Max · MIRACL English dev · 3 fresh lifecycles planned per point · Lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB · Historical peers, not a matched run

Final database volume

Working set and complete post-stop volume

Antfly v0.2.1 · e5261e9e

Engine	Passages	Demand MB	Cache + memory MB	Final volume MB	Valid / total
Antfly	250,000	360.2 MB [335.9–367.5]	643.5 MB [636.0–662.8]	160.5 MB [157.0–165.3]	3 / 3
Elasticsearch	250,000	1,712.7 MB	2,305.5 MB	123.3 MB	1 / 1
PostgreSQL	250,000	207.6 MB	901.3 MB	665.9 MB	1 / 1
Quickwit	250,000	547.4 MB	767.7 MB	202.7 MB	1 / 1
Antfly	1,000,000	1,554.1 MB [1,549.9–1,998.8]	2,385.2 MB [2,316.5–2,677.8]	597.9 MB [594.5–601.2]	3 / 3
Elasticsearch	1,000,000	1,801.2 MB	2,886.6 MB	477.4 MB	1 / 1
PostgreSQL	1,000,000	293.1 MB	2,758.3 MB	2,322.6 MB	1 / 1
Quickwit	1,000,000	1,102.4 MB	1,725.0 MB	390.6 MB	1 / 1
Antfly	2,000,000	3,606.5 MB [3,264.7–4,415.8]	4,884.7 MB [4,874.2–5,736.5]	1,176.6 MB [1,143.6–1,232.7]	3 / 3
Elasticsearch	2,000,000	1,881.6 MB	5,001.3 MB	947.0 MB	1 / 1
PostgreSQL	2,000,000	327.0 MB	3,936.5 MB	3,495.2 MB	1 / 1
Quickwit	2,000,000	1,827.8 MB	2,936.2 MB	773.4 MB	1 / 1
Antfly	3,000,000	6,884.3 MB [6,631.1–7,675.8]	9,545.9 MB [9,250.7–10,431.8]	1,706.6 MB [1,701.5–1,716.2]	3 / 3
Elasticsearch	3,000,000	1,880.9 MB	4,903.8 MB	1,416.2 MB	1 / 1
PostgreSQL	3,000,000	321.3 MB	5,114.4 MB	4,645.2 MB	1 / 1
Quickwit	3,000,000	2,210.9 MB	3,818.9 MB	1,029.1 MB	1 / 1

Query and judged retrieval evidence

Antfly v0.2.1 · e5261e9e

Engine	Passages	QPS	p95 ms	nDCG@10	Recall@100	Judged@100
Antfly	250,000	905.537 [871.046–914.334]	2.06 [2.04–2.13]	0.572 [0.571–0.574]	0.941 [0.938–0.946]	0.097 [0.097–0.098]
Elasticsearch	250,000	363.665	3.44	0.552	0.930	0.094
PostgreSQL	250,000	2.719	1,081.78	0.239	0.791	0.083
Quickwit	250,000	47.024	22.80	0.555	0.939	0.095
Antfly	1,000,000	435.758 [431.344–437.734]	5.55 [5.45–5.69]	0.525 [0.525–0.531]	0.921 [0.912–0.924]	0.094 [0.094–0.094]
Elasticsearch	1,000,000	302.605	4.36	0.493	0.892	0.090
PostgreSQL	1,000,000	0.350	12,385.26	0.143	0.630	0.061
Quickwit	1,000,000	43.014	25.66	0.498	0.903	0.091
Antfly	2,000,000	264.899 [264.051–268.398]	9.93 [9.79–10.22]	0.454 [0.441–0.493]	0.815 [0.803–0.893]	0.084 [0.083–0.091]
Elasticsearch	2,000,000	258.106	5.62	0.453	0.866	0.087
PostgreSQL	2,000,000	0.091	37,382.05	0.097	0.508	0.048
Quickwit	2,000,000	40.320	27.82	0.458	0.877	0.088
Antfly	3,000,000	193.371 [192.881–193.966]	14.37 [14.15–14.67]	0.471 [0.382–0.476]	0.879 [0.744–0.884]	0.090 [0.075–0.090]
Elasticsearch	3,000,000	195.861	7.15	0.425	0.849	0.085
PostgreSQL	3,000,000	0.050	61,861.82	0.076	0.432	0.041
Quickwit	3,000,000	38.805	29.29	0.427	0.856	0.086

deployment-artifacts

complete

Deployment artifacts

Checksum-verified official Linux ARM64 v0.2.1 release artifacts.

Methodology scope · protocol · interpretation

Deployment artifacts

Official v0.2.1 Linux/arm64 archive and executable, commit e5261e9e2937d03943b518974bf8063351ad1449. The archive checksum matches release metadata. Peer download bytes are the compressed layer sums for the pinned Linux/arm64 image manifests collected in this release run. No service was started and no timing or runtime-memory comparison is implied.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: deployment-artifacts
Domain: deployment/artifacts
Source: https://github.com/antflydb/antfly/releases
Known Antfly versions: v0.1.3, v0.2.*
Expected duration: 2 min
Enabled: yes

Benchmark parameters

{"artifacts":[{"archive_name":"antfly_0.2.1_Linux_arm64.tar.gz","archive_sha256":"7ed1cad8ef60f5c6d19254a6890642122f539f94bfb7299f2262e9b0ee472a81","bundle_components":[],"engine":"antfly","executable_sha256":"652ff445479450e753203f6941fc0d5411adfd20d444703de7e76609c909a162","linkage":"static","prerequisites":[]}],"platform":"linux-arm64"}

Peer versions

Weaviate1.39.2
pgvector0.8.6
Chroma1.5.9
Milvus2.6.22
Elasticsearch9.5.2

Evidence digests

Receipt: b6f981c0d6bfdfc6d1ca15d015b42d14284c021b5894bbb86bced2ec3f5ed7ca
Aggregate: 61852a4346ca693a5932e43ad8e9887abe2c8cd9da6c80dd39fa44f3a87cd9b4
Provenance: 61852a4346ca693a5932e43ad8e9887abe2c8cd9da6c80dd39fa44f3a87cd9b4
Harness commit

Selected measurement

Completed: 9 Sep 2026 · 02:47 UTC
Antfly: v0.2.1 · 9 September 2026 · 00:29 UTC
Runtime: 20.0s
Platform: linux · arm64
Machine: Apple M4 Max
Build: official-release

Linux/arm64 deployable artifacts

Antfly v0.2.1 · e5261e9e

Engine	Deployable form	Artifact origin	Download MB	Executable MB	Status
antfly	Static executable	publisher release archive	47.4	83.6	ok
weaviate	Weaviate server container image	OCI registry manifest	88.2	—	ok
pgvector	PostgreSQL + pgvector container image	OCI registry manifest	154.1	—	ok
chroma	Chroma server container image	OCI registry manifest	165.2	—	ok
milvus	Standalone container image	OCI registry manifest	363.6	—	ok
elasticsearch	Container image with JVM	OCI registry manifest	758.8	—	ok

document-ohr-bench

partial

OHR-Bench · controlled extraction

Controlled OHR extraction retrieval over all 8,498 questions in each paired input: Antfly v0.2.1 scored 44.33% mean word-LCS versus 70.44% for matched ground-truth text; 73 of 8,561 retained pages had OCR failures.

Methodology scope · protocol · interpretation

Controlled extraction measurement

This lane compares two document-text inputs under one controlled retrieval protocol: text extracted by the official Antfly v0.2.1 ReleaseFast Darwin ARM64 binary and matched OHR ground-truth text. Both use the same fixed 1,261-document candidate manifest, canonical logical paths, tokenizer/model sources, 1,024-token SimpleNodeParser/SentenceSplitter chunks without overlap, and deterministic UUID5 node IDs. Scores are percentage mean word-level evidence LCS at top-2, averaged across all 8,498 questions and both independent BM25 and exact full-precision BGE-M3 lanes. Every question stays in the denominator; unavailable evidence scores zero. There is no RRF or evidence-type reweighting.

Extraction quality

Antfly loaded all 1,261 PDFs and retained page arrays for all 8,561 pages. Seventy-three page OCR outcomes failed across 35 PDFs (0.85% of pages); these are extraction failures, not PDF or page-load failures, and a failed OCR page can retain native text. Two corpus-only PDFs produced no retrievable chunks. The native structural gate therefore remains failed, while the paired controlled scores remain valid descriptive measurements over the full question denominator.

Interpretation

The controlled Antfly and matched-ground-truth rows are directly comparable only to each other. This is not an exact reproduction of the published OHR ranking: the fixed corpus excludes one published ground-truth-only distractor, canonical paths and UUID5 IDs remove filesystem and UUID4 variation, and 42 strict filename-mismatch QA rows remain zero rather than receiving alias repair. Published rows retain their upstream labels and values as a visibly separate reference group. Native Antfly analyzer and BGE-small retrieval metrics are separate diagnostics, not substitutes for the controlled scores. Concurrent workloads make retained timing non-comparable.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: document-ohr-bench
Domain: document/raw-pdf-retrieval
Source: https://github.com/opendatalab/OHR-Bench
Known Antfly versions: v0.2.*
Enabled: yes

Benchmark parameters

{"controlled_measurement":{"candidate_documents":1261,"chunks":27592,"failures_kept_in_denominator":true,"lane_scores_per_input":16996,"metric":"mean word-level evidence LCS at top-2","questions":8498,"retained_pages":8561},"extraction_quality":{"loaded_pdfs":1261,"ocr_attempt_failure_rate":0.04694533762057878,"ocr_attempted_pages":1555,"ocr_failed_documents":35,"ocr_failed_pages":73,"ocr_failed_rate_all_pages":0.00852704123350076,"pages_without_ocr_failure":8488,"retained_pages":8561,"status":"failed","zero_chunk_documents":2},"native_retrieval_diagnostic":{"classification":"separate Antfly analyzer and BGE-small diagnostic; not a controlled score","lanes":{"dense":{"empty_queries":0,"mean_latency_seconds":0.036739,"questions":8498,"top1_mean_lcs":0.262424,"top1_page_recall":0.454931,"top1_questions":8498,"top2_mean_lcs":0.341372,"top2_page_recall":0.566016,"top2_questions":8498,"top5_mean_lcs":0.413611,"top5_page_recall":0.667804,"top5_questions":8498},"hybrid":{"empty_queries":0,"mean_latency_seconds":0.001878,"questions":8498,"top1_mean_lcs":0.302222,"top1_page_recall":0.50153,"top1_questions":8498,"top2_mean_lcs":0.383135,"top2_page_recall":0.607908,"top2_questions":8498,"top5_mean_lcs":0.454335,"top5_page_recall":0.701459,"top5_questions":8498},"lexical":{"empty_queries":948,"mean_latency_seconds":0.001371,"questions":8498,"top1_mean_lcs":0.317588,"top1_page_recall":0.501059,"top1_questions":8498,"top2_mean_lcs":0.386779,"top2_page_recall":0.583549,"top2_questions":8498,"top5_mean_lcs":0.446812,"top5_page_recall":0.659096,"top5_questions":8498}},"missing_provenance":0,"quality_status":"passed","query_errors":0,"structural_status":"failed"},"paired_validation":{"official-antfly":{"lane_scores":16996,"questions":8498,"recomputed_mean":0.44326071858099025,"unique_questions":8498,"verified_domain_artifacts":49},"official-gt":{"lane_scores":16996,"questions":8498,"recomputed_mean":0.7044283388810646,"unique_questions":8498,"verified_domain_artifacts":49},"paired_controls":{"candidate_documents":1261,"identical_candidate_names_and_logical_paths":true,"identical_protocol_environment_model_sources_tokenizers":true,"identical_qa_order":true,"native_declared_failure_documents":35}},"protocol_controls":{"canonical_logical_paths":"data/retrieval_base/evaluation","chunk_overlap":0,"chunk_size":1024,"chunker":"llama_index.node_parser.SimpleNodeParser/SentenceSplitter","deterministic_node_ids":"UUID5(NAMESPACE_URL, logical_file_path + NUL + zero_based_page_idx + NUL + chunk_ordinal); document ordinal=-1","embedding_model":"BAAI/bge-m3","embedding_revision":"5617a9f61b028005a4858fdac845db406aefb181","fixed_corpus_difference":"one published GT-only distractor excluded","published_ranking_claim":false,"retrieval_lanes":["bm25","dense_bge_m3"],"strict_filename_mismatch_qa_rows":42,"top_k":2},"timing":{"comparable":false,"concurrent_workloads":true,"native_setup_wall_seconds":11737.516}}

Retrieval systems

Antfly extraction · BM25 controlled top-2 word-LCS 45.19% mean 8,498 queries
Antfly extraction · BGE-M3 dense controlled top-2 word-LCS 43.46% mean 8,498 queries
Matched ground truth · BM25 controlled top-2 word-LCS 71.34% mean 8,498 queries
Matched ground truth · BGE-M3 dense controlled top-2 word-LCS 69.54% mean 8,498 queries

Published comparison sources

Published OHR evaluation table Upstream reference values; not rerun in the controlled pair Different corpus and identity controls; no ranking claim Source

Peer versions

Peer versions were not recorded in the published result fields.

Evidence digests

Receipt: b749fd8d782fdd7b4adddd679f880ba465b41b1c203a48088f8f252043fd89b2
Aggregate: a47315641ace8ffd8d415fcfed93e11a6c9797300f4461c6bd577a25a804c435
Provenance: f4d505e6547ba01e8d779919b3c356bcbb6b0ae58186c30c3841e4b29d9cecfe
Harness commit: f7c4024229d4f597430f2389950961638a8116ca

Selected partial evidence

Completed: 9 Sep 2026 · 20:10 UTC
Antfly: v0.2.1
Runtime: —
Platform: darwin · arm64
Machine: Apple M5 Pro
Build: ReleaseFast

Overall duration is omitted because controlled and diagnostic workloads overlapped; completion marks the evidence seal.

Controlled extraction scores · published references kept separate

Antfly v0.2.1 · e5261e9e

Evidence group	Document source	TXT	TAB	FOR	CHA	RO	MUL	ALL %	OCR-failed pages
Controlled paired run	Antfly v0.2.1 extraction	58.79	45.24	36.90	23.44	6.89	40.88	44.33	73 / 8,561 (0.85%)
Controlled paired run	Matched OHR ground truth	81.75	70.10	75.91	68.24	10.05	62.68	70.44	—
Published upstream table	Ground Truth	81.6	69.8	75.2	70.3	9.8	—	70.4	—
Published upstream table	MinerU 0.9.3	68.1	48.6	51.3	16.5	5.9	—	50.5	—
Published upstream table	Marker 1.2.3	75.5	58.2	55.5	20.0	5.9	—	57.0	—
Published upstream table	Azure Document Intelligence	78.0	59.4	55.2	45.2	5.8	—	60.6	—
Published upstream table	GOT	62.5	41.1	49.0	17.4	3.7	—	45.8	—
Published upstream table	Nougat	59.5	32.8	44.3	11.3	4.4	—	41.2	—
Published upstream table	Qwen2.5-VL-72B	75.1	60.0	60.0	38.2	5.3	—	59.6	—
Published upstream table	InternVL2.5-78B	68.6	57.9	55.6	45.1	2.7	—	56.2	—
Published upstream table	olmOCR-7B-0225-preview	72.5	58.4	55.4	24.8	5.0	—	56.6	—
Published upstream table	MonkeyOCR	74.6	56.5	55.5	16.5	5.7	—	55.9	—
Published upstream table	Nanonets-OCR-s	71.8	59.8	57.4	43.7	4.4	—	58.3	—

Extraction quality and corpus retention

Antfly v0.2.1 · e5261e9e

Outcome	Valid / total	Observed boundary	Failure rate %
PDF loading	1,261 / 1,261	0 load failures	0.00%
Page retention	8,561 / 8,561	0 dropped pages	0.00%
Pages without OCR failure	8,488 / 8,561	73 failures across 35 PDFs	0.85%
Retrievable chunks	1,259 / 1,261	2 zero-chunk PDFs	0.16%

graph-benchmark

complete

Graph

Graph query performance, Pokec-small cross-engine diagnostics, official LSQB capability, and bounded SNB Interactive qualification.

Methodology scope · protocol · interpretation

Subject and execution

Antfly was measured first. Neo4j 5.26.29 and Memgraph 3.12.0 were measured separately on September 9 on the same host/runtime, with three fresh lifecycles each. Each row’s latency, memory, and storage come from its own measured lifecycles; no historical memory was grafted onto another run. These are separate cohorts, not an interleaved nine-lifecycle run or evidence of a release-to-release speedup.

F2 protocol

LSQB-derived SF0.1 endpoint-filtered two-hop lookup: Forum → Post → Tag, returning matching Post IDs. This is an anchored microbenchmark, not an audited LDBC result. Warm serial latency over three independent lifecycles per engine, 5,000 samples per run (15,000 total), each checked against a count-and-BLAKE3 result oracle. Demand memory is the mean of the three lifecycle peaks of concurrent cgroup anon + shmem, excluding reclaimable file cache. The peer sampler covers offline import or ingestion through the final query without replacing the measured container.

Run evidence parameters · variation · provenance

Benchmark metadata

Benchmark ID: graph-benchmark
Domain: graph benchmark
Source: https://github.com/ldbc/lsqb
Known Antfly versions: v0.1.3, v0.2.*
Expected duration: 100 min
Enabled: yes

Benchmark parameters

{"execution_environment":{"container_runtime":"rootless Podman 6.1.1","processor":"Apple M4 Max","vm_cpus":10,"vm_memory_gib":24},"peers_not_run":["neo4j","memgraph"],"query":"F2","runs":3,"samples":15000}

Trial variation

[{"display_name":"Memgraph","engine":"memgraph","engine_version":"3.12.0","images":["docker.io/memgraph/memgraph@sha256:c162cb9a6f76dac6080119da919d9b36e69c8b982af7bf34dd7df2b3723bd69c"],"memory":{"cache_inclusive_max_bytes":746938368,"cache_inclusive_mean_bytes":619343872,"cache_inclusive_min_bytes":555184128,"container_runtimes":["docker","docker","docker"],"contract_sha256":"bd8c80526cbebb19ffebfe9d467edddd75450f3aa45a95e589f82d039349df93","demand_max_bytes":403644416,"demand_mean_bytes":403206144,"demand_min_bytes":402731008,"metric":"swap_backed_demand_excluding_reclaimable_file_cache","runtime_comparison_identity":{"architecture":"arm64","os_type":"linux"},"status":"qualified","trial_count":3,"trials":[{"cache_inclusive_peak_bytes":746938368,"container_runtime":{"architecture":"arm64","cli":"docker","name":"localhost.localdomain","os_type":"linux","storage_driver":"overlay","version":"6.1.1"},"demand_peak_bytes":402731008,"failed_samples":0,"samples":26,"summary":"raw/memory/memgraph-trial-1.json","timeline":"raw/memory/memgraph-trial-1.jsonl","trial":"1"},{"cache_inclusive_peak_bytes":555184128,"container_runtime":{"architecture":"arm64","cli":"docker","name":"localhost.localdomain","os_type":"linux","storage_driver":"overlay","version":"6.1.1"},"demand_peak_bytes":403243008,"failed_samples":0,"samples":25,"summary":"raw/memory/memgraph-trial-2.json","timeline":"raw/memory/memgraph-trial-2.jsonl","trial":"2"},{"cache_inclusive_peak_bytes":555909120,"container_runtime":{"architecture":"arm64","cli":"docker","name":"localhost.localdomain","os_type":"linux","storage_driver":"overlay","version":"6.1.1"},"demand_peak_bytes":403644416,"failed_samples":0,"samples":26,"summary":"raw/memory/memgraph-trial-3.json","timeline":"raw/memory/memgraph-trial-3.jsonl","trial":"3"}],"yardstick":"linux_cgroup_v2_anon_plus_shmem.v1"},"p50_ms":0.117584,"p95_ms":0.15692840000000005,"p99_ms":0.2031539100000006,"pooled_percentiles_ms":{"p50":0.117875,"p95":0.15754409999999996,"p99":0.20508441},"query_id":"F2","run_percentiles":[{"p50_ms":0.11847949999999999,"p95_ms":0.15596315,"p99_ms":0.2031539100000006,"run_id":"1","samples":5000},{"p50_ms":0.117417,"p95_ms":0.15692840000000005,"p99_ms":0.20272442000000035,"run_id":"2","samples":5000},{"p50_ms":0.117584,"p95_ms":0.15859025000000002,"p99_ms":0.205125,"run_id":"3","samples":5000}],"run_range_ms":{"p50":[0.117417,0.11847949999999999],"p95":[0.15596315,0.15859025000000002],"p99":[0.20272442000000035,0.205125]},"runs":3,"samples":15000,"storage":{"aggregation_protocol_fingerprint":"c0fe0faaafcd91e703c18f41c784fa81e78473977b120e7e172a46e0c4cdc333","apparent_mean_bytes":311431795,"apparent_values_bytes":[311431796,311431794,311431796],"container_runtimes":["docker","docker","docker"],"engine":"memgraph","max_bytes":311590912,"mean_bytes":311590912,"metric":"final_allocated_bytes","min_bytes":311590912,"reasons":[],"receipt_protocol_fingerprints":["fb2fde54a31166b89d43be3d4a75df17c8a66068a945006d0f4749b2661b277b","a4e555079315cee1ecdb144cdfc54426011f8beabaaa01a37c544d1902ca0b35","11de75c4ca52def97aace75ddd5d0ae39914ef7babb6c558cca81fda47e2b8b7"],"relative_range":0,"status":"qualified","trial_count":3,"trial_ids":["1","2","3"],"values_bytes":[311590912,311590912,311590912],"workload":"lsqb-sf0.1-filtered-two-hop"},"trial_count":3},{"display_name":"Neo4j","engine":"neo4j","engine_version":"5.26.29","images":["docker.io/library/neo4j@sha256:d9dd3dc7d1c78fa959191ff02dbdcbefadceaf83eee23428fb92a58cac8ad3fe"],"memory":{"cache_inclusive_max_bytes":5262098432,"cache_inclusive_mean_bytes":5259158869.333333,"cache_inclusive_min_bytes":5256122368,"container_runtimes":["docker","docker","docker"],"contract_sha256":"bd8c80526cbebb19ffebfe9d467edddd75450f3aa45a95e589f82d039349df93","demand_max_bytes":4924825600,"demand_mean_bytes":4921341269.333333,"demand_min_bytes":4917903360,"metric":"swap_backed_demand_excluding_reclaimable_file_cache","runtime_comparison_identity":{"architecture":"arm64","os_type":"linux"},"status":"qualified","trial_count":3,"trials":[{"cache_inclusive_peak_bytes":5256122368,"container_runtime":{"architecture":"arm64","cli":"docker","name":"localhost.localdomain","os_type":"linux","storage_driver":"overlay","version":"6.1.1"},"demand_peak_bytes":4917903360,"failed_samples":0,"samples":62,"summary":"raw/memory/neo4j-trial-1.json","timeline":"raw/memory/neo4j-trial-1.jsonl","trial":"1"},{"cache_inclusive_peak_bytes":5262098432,"container_runtime":{"architecture":"arm64","cli":"docker","name":"localhost.localdomain","os_type":"linux","storage_driver":"overlay","version":"6.1.1"},"demand_peak_bytes":4924825600,"failed_samples":0,"samples":63,"summary":"raw/memory/neo4j-trial-2.json","timeline":"raw/memory/neo4j-trial-2.jsonl","trial":"2"},{"cache_inclusive_peak_bytes":5259255808,"container_runtime":{"architecture":"arm64","cli":"docker","name":"localhost.localdomain","os_type":"linux","storage_driver":"overlay","version":"6.1.1"},"demand_peak_bytes":4921294848,"failed_samples":0,"samples":62,"summary":"raw/memory/neo4j-trial-3.json","timeline":"raw/memory/neo4j-trial-3.jsonl","trial":"3"}],"yardstick":"linux_cgroup_v2_anon_plus_shmem.v1"},"p50_ms":0.467106,"p95_ms":0.8681099000000001,"p99_ms":1.2966702500000018,"pooled_percentiles_ms":{"p50":0.463106,"p95":0.8641161499999996,"p99":1.29838224},"query_id":"F2","run_percentiles":[{"p50_ms":0.4534815,"p95_ms":0.8555141000000002,"p99_ms":1.2799499100000007,"run_id":"1","samples":5000},{"p50_ms":0.468856,"p95_ms":0.8681099000000001,"p99_ms":1.2990160000000004,"run_id":"2","samples":5000},{"p50_ms":0.467106,"p95_ms":0.8705584500000001,"p99_ms":1.2966702500000018,"run_id":"3","samples":5000}],"run_range_ms":{"p50":[0.4534815,0.468856],"p95":[0.8555141000000002,0.8705584500000001],"p99":[1.2799499100000007,1.2990160000000004]},"runs":3,"samples":15000,"storage":{"aggregation_protocol_fingerprint":"486e7bc359a920f5b0f16cdf1f6265604cb0f0c53bf13c79140d7375f9945c32","apparent_mean_bytes":635527933,"apparent_values_bytes":[635511549,635536125,635536125],"container_runtimes":["docker","docker","docker"],"engine":"neo4j","max_bytes":635539456,"mean_bytes":635531264,"metric":"final_allocated_bytes","min_bytes":635514880,"reasons":[],"receipt_protocol_fingerprints":["5aaa77183006377e3e625f680b1966a186d30129c506b1b30b0e22b41df8befb","09bc3a2ec55e059e3e76c6f90e86f4e25bdf725349aa4f7ed4c8bb7abfba803b","7bf7c42418e4f8a4cff815e6c76c195a967bbb910869fbda8464c50919bcd5ac"],"relative_range":0.00003867000947415232,"status":"qualified","trial_count":3,"trial_ids":["1","2","3"],"values_bytes":[635514880,635539456,635539456],"workload":"lsqb-sf0.1-filtered-two-hop"},"trial_count":3}]

Peer versions

3.12.0
5.26.29

Evidence digests

Receipt: 26f4e807f3fbb16355c24395cd42908c9ae35c49471f1030738ec9504da7feca
Aggregate: b443a62cfe49f98515c1b429eaa147c349e4946a9bf25d0a4706bb934362b9f4
Provenance: b443a62cfe49f98515c1b429eaa147c349e4946a9bf25d0a4706bb934362b9f4
Harness commit: 3003c4db794a671ed204917d98e4dfcb85531ff1

Peer cohort evidence · lsqb-sf0.1-filtered-two-hop-v1

Completed: 9 Sep 2026 · 16:59 UTC
Fresh lifecycles: 6
Peer variants: 2
Receipt: d3626afd681cfa06259e38a633f30507f62df0f93a147dcd435eee98d73b4c79
Aggregate: 7e49842de11f49a48c31a6b1349a47ce5a38999eb5aa0b3cd5cd3429275bdcb6
Provenance: 7e49842de11f49a48c31a6b1349a47ce5a38999eb5aa0b3cd5cd3429275bdcb6

Selected measurement

Completed: 9 Sep 2026 · 03:03 UTC
Antfly: v0.2.1 · 9 September 2026 · 00:29 UTC
Runtime: 8m 15s
Platform: linux · arm64
Machine: Apple M4 Max
Build: official-release

Peer cohort · lsqb-sf0.1-filtered-two-hop-v1

Completed: 9 Sep 2026 · 16:59 UTC
Variants: 2 independently executed peers
Lifecycles: 6 fresh
Compatibility: Same machine and runtime · separately measured cohort

Warm serial p95

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · LSQB-derived SF0.1 F2 · 15,000 samples / three runs · Lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB

Demand working set

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · LSQB-derived SF0.1 F2 · 15,000 samples / three runs · Lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB

Final database volume

Antfly v0.2.1 (e5261e9e, official-release) · linux-arm64 · LSQB-derived SF0.1 F2 · 15,000 samples / three runs · Lower is better · Apple M4 Max · rootless Podman 6.1.1 · 10 VM CPUs / 24 GiB

Latency, resources, and served samples

Antfly v0.2.1 · e5261e9e

Engine	Version	Query	p50 ms	p95 ms	p99 ms	Demand working set MB	Final database volume MB	Served samples
antfly	antfly 0.2.1 (zig runtime)	F2	0.360	0.530	0.742	1,982.0 MB	200.7 MB	15,000
Memgraph	3.12.0	F2	0.118	0.157	0.203	403.2 MB	311.6 MB	15,000
Neo4j	5.26.29	F2	0.467	0.868	1.297	4,921.3 MB	635.5 MB	15,000

VIII

1 of 1 lane measured

End-to-end

Agent-facing product activation and first retrieval from clean interactive Pi sessions.

quickstart-first-retrieval

partial

Quickstart

11 database-version rows retain every recording. Median time summarizes observed retrievals; validation reports which sessions completed cleanly.

Database	Version	Median time	Validation	Sessions
Antfly	v0.1.3	228.554 s	passed	Watch sessions 1
Antfly	v0.2.0	228.262 s	passed	Watch sessions 1 2
Antfly	v0.2.1-rc1	111.846 s	passed	Watch sessions 1 2 3
Chroma	v1.5.9	79.072 s	passed	Watch sessions 1
Elasticsearch	v9.5.2	445.708 s	passed	Watch sessions 1
Memgraph	v3.12.0	67.161 s	passed	Watch sessions 1
Milvus Lite	v3.0.1	198.412 s	passed	Watch sessions 1
Neo4j	v2026.07.1	60.369 s	passed	Watch sessions 1
Quickwit	v0.9.0	61.978 s	passed	Watch sessions 1
Weaviate	v1.39.2	228.033 s	passed	Watch sessions 1
Antfly	v0.2.1	32.093 s	partial · 2/3 valid	Watch sessions 1 2 3

Methodology Timing · grouping · validation

Timing boundary

Time to first retrieval runs from PTY capture start through the first successful qualifying search result observed in the Pi session. Acquisition, installation, setup, retries, and agent reasoning all contribute to elapsed time.

Grouping and median

Each row groups retained sessions by database and exact observed version. The median includes every observed qualifying retrieval, including a retrieval followed by failed post-session verification. Validation remains separate. Models and task boundaries can differ between cohorts; these rows are not controlled product-speed comparisons.

Protocol history

Historical Antfly recordings use protocol 3 and its qualifying retrieval command. New natural recordings use protocol 4 passive Pi result correlation, without an agent-visible marker. Container-dependent sessions substitute rootless Podman for Docker and retain that runtime choice in provenance. Numbered session links preserve the full trajectory and protocol evidence behind every row.

Container boundary

Container-dependent sessions use an operator-approved rootless Podman host-user API. Bubblewrap limits the agent shell but does not constrain bind mounts requested through that API. The runner snapshots preexisting Podman resources, restricts verification to containers created by the attempt, removes attempt-owned resources afterward, scans published transcripts for credentials, and records the socket scope in provenance.

Validation states

Passed: Every included session produced a qualifying retrieval and passed its retained verifier.
Partial: At least one included session passed while another lacked complete valid evidence.
Failed: Included sessions completed without a qualifying verified retrieval.
Unverified: Retained evidence lacks a terminal verifier result.

Three fresh sequential v0.2.1 sessions used OpenRouter GLM-5.3 Flash with high thinking, host CPUs, and an explicit hello/first stopping objective. All three produced the expected retrieval; two passed post-session verification. Trial 2 shut down Antfly afterward, causing ConnectionRefused. Median includes all three observed retrievals, not just verified sessions. Release and prompt differ from earlier cohorts; this is not an isolated product speed comparison. Model installation, semantic search and RAG were not exercised.