Problem
It’s easy to call an off-the-shelf search library and get relevance ranking for free — and just as easy to never actually understand why a query matched what it matched. IR Lab exists to close that gap: build the actual mechanics of information retrieval — document and query models, analyzers, inverted indexes, Boolean query evaluation — from first principles, so the abstractions a production search stack hides are ones I’ve implemented, not just used.
Constraints
A self-directed learning project run alongside coursework and other engagements, so the design goal was clarity and correctness over performance or feature completeness — this is a lab, not a product, and it’s explicitly structured to stay that way.
Approach & architecture
flowchart LR
A[Document ingestion] --> B[Analyzers<br/>tokenize / normalize]
B --> C[Indexing abstractions<br/>inverted index]
D[Query representation] --> E[Boolean query evaluation]
C --> E
E --> F[Evaluation / results]The system deliberately separates five concerns — ingestion, analysis, indexing, querying, and evaluation — behind explicit interfaces, so each one can be swapped or extended (a different analyzer, a different index structure) without touching the rest. Query representation and document representation are modeled independently, which is what makes Boolean retrieval implementable cleanly on top: the evaluator only ever talks to the abstractions, never to raw text.
What I’d change now
The retrieval model stops at Boolean querying — no ranking, no relevance scoring yet. The natural next step, and the reason this project stays “in-progress” rather than archived, is layering a ranked retrieval model (TF-IDF or a vector-similarity layer) on top of the same indexing abstractions, to see how much of the existing separation of concerns survives the added complexity.
Impact
The value here isn’t a benchmark number — it’s the architecture itself: five independently testable stages instead of one monolithic search function, which is what let later projects (CrawlViz’s relevance cascade, in particular) reuse the same mental model of separating representation from evaluation.