Reproducing ENGRA-KB-v2 benchmarks on your machine
Download the JSON suite, run the same embedder and query set, and compare structured atoms vs chunk RAG before you migrate production memory.
What is in the suite
ENGRA-KB-v2 includes: • 550 synthetic enterprise documents • 400 labeled queries with relevance judgments • Precomputed atom graph and chunk baselines • Results for BM25, chunk-512 RAG, and structured atoms
Download artifacts from /products/memory/benchmarks. Methodology and metric definitions live in /docs/memory/benchmarks.
Metrics we report
Recall@10 — Did the right document appear in the top ten results? MRR — How highly ranked was the first relevant hit? Latency — End-to-end retrieval time at reference hardware (documented per suite).
On ENGRA-KB-v2, structured atoms reach 75.3% Recall@10 vs 54.8% for fixed-size chunk RAG. Read the analysis in our atoms vs RAG post.
Fair comparison checklist
• Use the same embedder (default: Xenova/all-MiniLM-L6-v2) across arms. • Do not tune chunk size only on the atom arm. • Keep query text identical—no prompt rewriting per pipeline. • Report both Recall@10 and MRR; recall alone hides ranking regressions.
After local reproduction
Import a slice of your production corpus into a staging library. Model Library/Scope/Topic for your top use cases, then A/B atom retrieval against your existing stack.
Wire agents via Memory MCP or Gateway routes once staging metrics beat your baseline.