Retrieval tiers
| tier | LLM calls | for |
|---|---|---|
search / build_context | zero | the default path: deterministic BM25 + vector + entity fusion |
build_context(organizer=...) | an organizer pass | "list every X you know about me" |
answer | a planner loop | hard multi-hop questions worth the tokens |
The cheap tier is genuinely free
build_context returns a prompt-ready block with citations and makes no model call at all. Most systems hand you a list of records and leave the assembly to you — and the token budgeting, and the deduplication.
Free also means deterministic. Same store, same query, same result, every time. There is no sampling step to blame when a retrieval surprises you.
The middle tier is Python-only on purpose
build_context(organizer=...) runs an LLM-backed organizer — value-board, enumeration-sweep — over the retrieved set. It is the right tool for enumerations, where the answer is "all of them" and ranking is not enough.
/v1/context never accepts an organizer, so the zero-LLM guarantee on that route cannot be flipped by a request parameter. If you are running the service for other teams, that is a property you can state without qualification.
The expensive tier
answer runs a planner loop: it issues its own queries against the store, inspects sessions, counts evidence and walks entity timelines until it can answer, then a reader writes the answer.
Every tool the planner uses browses the memory store. SodaMem ships no web-search tool — there is nothing in sodamem/tools/ that can reach the open internet.
Cost is reported split by operation, because ingest is output-token heavy and answer is input-heavy, and a single total hides the only comparison worth making. See Metrics and cost.
Choosing
Start at build_context. Move up only when you can name the question it fails on:
- it returned the right facts but you needed them exhaustively enumerated → organizer
- the question needs two hops the retriever cannot make in one shot →
answer - neither: stay where you are, the tier costs nothing