Status, up front
This is a working prototype. The pipeline runs end to end and answers with citations; evaluation, testing and retrieval tuning are still in progress.
The problem
Security policies are long, formal and full of exact identifiers. Pure semantic search misses a literal term like SDM-01; pure keyword search misses “what risks does this cover?”. Compliance answers also have to show their source.
Design decisions
| Decision | Why |
|---|---|
| Hybrid search (dense + BM25) | BM25 catches exact policy terms; dense search catches conceptual matches. |
| RRF fusion, not a weighted sum | It needs no tuning and behaves well across query types. |
| Cross-encoder rerank | Reads the query and chunk as a pair and filters the bi-encoder’s false positives. |
| Structure-preserving ingest | PyMuPDF4LLM keeps headings, so chunks are section-aware and citations name real sections. |
| Honest health check | /health returns loading during the roughly 18-second model warm-up and only reports ok once the models are ready. |
What’s next
An evaluation set of questions with known answers, measured retrieval quality (recall@k, MRR), and tuning of chunk size, top_k and reranker thresholds against it.