Versioned Retrieval Evaluation
Madeenan Search Quality Baseline
A transparent view of what the retrieval system finds, where it misses, and how quickly it responds across a 279-query evaluation set.
Method
The evaluation contains exact references, conceptual searches, stories, duas, and cross-source questions. Each query has expected sources and may include results that should not appear.
The baseline records retrieval output against a fixed corpus fingerprint and evaluation-set hash. It is a regression tool, not a claim that every religious question has one correct answer.
Baseline Metrics
Recall at 5
50.65%Share of expected relevant sources found in the first five results.
Recall at 10
53.88%Share of expected relevant sources found in the first ten results.
Mean Reciprocal Rank
50.20%How early the first expected source appears.
NDCG at 10
51.14%Ranking quality with higher value for relevant results near the top.
Forbidden Hit Rate
2.15%Rate at which a result explicitly marked forbidden appeared.
Corpus Snapshot
- quran
- 6,236
- hadith
- 34,455
- tafsir
- 31,180
- dua
- 267
- migration revision
- 0016
- embedding model
- zembed-1
Latency Snapshot
Median retrieval latency was 276 ms. The 95th percentile was 4,673.6 ms. These measurements came from the recorded local evaluation environment and should not be treated as a production service-level promise.
Limitations
- The query set encodes expected retrieval behavior, not religious authority.
- Recall depends on the selected references and relevance judgments.
- Latency varies with infrastructure, cache state, and provider conditions.
- The public JSON excludes local host details, raw diagnostics, and credentials.