microRAG — guided walkthrough

RAG (Retrieval-Augmented Generation) is how every modern enterprise AI assistant — internal Q&A, customer support, "ChatGPT on our docs" — connects a frozen language model to fresh, organization-specific knowledge. The pattern is the same everywhere: chunk documents, index them two ways, retrieve the top hits, paste them into the prompt, call the LLM. Five steps. The attacks in Lab 15 all target one of those five.

Most production RAG systems wrap LangChain or LlamaIndex around a vendor-hosted vector store (Pinecone, Weaviate, OpenSearch) plus a vendor LLM. Each layer adds capability and obscures structure. microRAG strips that obscurity away: one Python file, two retrievers (TF-IDF + hashed-feature embeddings), one hybrid scorer, one prompt assembler, one LLM call. Read top to bottom and you have read everything LangChain's RetrievalQA does — and you can point at exactly which step each attack in Lab 15 lands on.

This lab leans on a few retrieval terms — RAG, TF-IDF, embedding, BM25. Every term underlined like this is hoverable (or click / tab-and-Enter): a plain-language explainer opens right below the line you're reading and pushes the rest of the page down — nothing gets covered.

Where to find it

• microRAG source — rag/micro_rag.py. ~200 lines, stdlib + one optional embedding lib. Read top to bottom. ↓ download lab-14-code.zip
• knowledge base — microdata/. Four markdown files: password policy, network help, onboarding, architecture.
• production server for §4 and Lab 15 — ../lab-15/server/baseline_rag.py. Same five steps, HTTP transport, five planted vulnerabilities.
• protocol references — the original RAG paper (Lewis 2020) · all-MiniLM-L6-v2 · BM25.
• next lab · the attacks — Lab 15 · Attacking RAG. Five canonical attacks against a server that's structurally microRAG.

1 · Anatomy of RAG

A RAG system runs two pipelines that meet at a shared store. The ingestion path runs ahead of time: read the documents, slice them, index them. The query path runs when a question arrives and reads: score the question against the index, assemble a prompt, call the model. The asymmetry matters for security — a query can only surface what ingestion already stored, so in Lab 15 the attacker's goal is to get the ingestion path to store something it shouldn't.

The asymmetry is the security story: a query can only surface what ingestion already wrote. That's why most Lab 15 attacks target the ingestion path — poison what gets written, and every later query reads it back.

The pipeline runs left-to-right at query time. Steps 1 and 2 also run once at ingest time (when documents are first loaded into the corpus). Lab 15 attacks every box and arrow in this diagram — knowing the diagram cold is the prerequisite for reading the attacks.

2 · The microRAG code · five steps

Open rag/micro_rag.py. ↓ download lab-14-code.zip The same five steps, about 200 lines. Each subsection below is one step in file order, and each step's output is the next step's input: raw text becomes chunks, chunks become two indexes, a query becomes a ranked list, the list becomes a prompt, the prompt becomes an answer.

2.1 Step 1 · chunk

Sliding window over the raw text. 300 characters per chunk, 60-character overlap. Real systems chunk by tokens (so the LLM's context budget is exact); we chunk by characters because it's smaller and the structural lesson is the same.

def chunk(text: str, size: int = 300, overlap: int = 60) -> list[str]:
    out, i = [], 0
    while i < len(text):
        out.append(text[i : i + size].strip())
        i += size - overlap
    return [c for c in out if c]

The overlap matters for retrieval quality — a sentence that lands awkwardly on a chunk boundary survives because the next chunk repeats its last 60 chars. It also matters for attacks: Lab 15 §7.2 (slow-drip poisoning) places malicious instructions in the overlap band specifically because most monitoring previews show only the start of each chunk.

Three failure modes to feel: (1) size too small → critical context spans many chunks, retrieval misses; (2) size too large → the LLM's context budget bursts when top-K is concatenated; (3) overlap = 0 → sentences split across chunk boundaries die in both indexes.

Knobs that matter

2.2 Step 2 · two indexes (TF-IDF + embedding)

A pile of chunks can't be searched directly: both the chunks and the query first have to become something numeric and comparable. microRAG does this twice, because the two methods notice different things — TF-IDF matches on the exact words that appear, while an embedding matches on meaning.

Start with TF-IDF — term frequency × inverse document frequency, the workhorse of text search since long before neural anything. Its whole idea fits in one sentence: a word points to a chunk in proportion to how often it appears here and how rarely it appears everywhere. Those are the two halves it multiplies together:

• TF — term frequency. How many times does the word occur in this chunk? "password" showing up four times is strong evidence this chunk is about passwords. In the code that's Counter(tokenize(c["body"])) — a per-chunk word count.
• IDF — inverse document frequency. How many chunks contain the word? A word that's in every chunk (the, your, megacorpai) can't tell you which chunk to pick, so its weight is crushed toward zero. A word in just one chunk (postgresql) is almost a fingerprint, so its weight is amplified. That's the idf dictionary — computed once, over the whole corpus, from the document-frequency counter df.

Multiply TF by IDF for each word, and a chunk becomes a list of weighted words — common filler terms weigh almost nothing, rare distinctive terms weigh a lot. To score a chunk against a query, you add up the weights of just the words the two have in common — so the chunk carrying the query's rarest words floats to the top. That one move, down-weight the common, up-weight the rare, is the entire reason TF-IDF can spot "password" or "database" against a wall of "the / a / your".

Worked example · IDF for the MegaCorpAI corpus

With N = 13 chunks, the formula is idf(t) = log((N + 1) / (df_t + 1)) + 1.0. A term that appears in exactly one chunk (df = 1) gets log(14/2) + 1 ≈ 2.95. A term that appears in every chunk (df = 13) gets log(14/14) + 1 = 1.00. The ratio matters: a rare term contributes ≈3× more weight per occurrence than a corpus-wide one.

"megacorpai" appears in every doc → low IDF → not informative. "postgresql" appears in one chunk → high IDF → exactly the term that would rescue an architecture query. Lab 15's KB-leakage attack rides on this: the rarest, highest-IDF tokens in the corpus are also the highest-signal secrets.

Here is the TF-IDF index in code:

def build_tfidf(chunks):
    df = Counter()
    for c in chunks:
        df.update(set(tokenize(c["body"])))
    N = len(chunks)
    idf = {t: math.log((N + 1) / (df_t + 1)) + 1.0 for t, df_t in df.items()}
    for c in chunks:
        tf = Counter(tokenize(c["body"]))
        c["tfidf"] = {t: cnt * idf.get(t, 0.0) for t, cnt in tf.items()}
    return idf

Embeddings · what the hash is faking

The second index, the embedding, is blunter: hash each token into one of 128 buckets, count, and normalize. Both indexes share one interface — text in, comparable numbers out, cosine for similarity — which is precisely why a real model like all-MiniLM-L6-v2 can drop in later without the retrieval code noticing the swap. The hash itself is eight lines:

def embed(text):
    # hashed-feature embedding · 128 dims · no GPU / no model download
    vec = [0.0] * 128
    for tok in tokenize(text):
        h = int(hashlib.md5(tok.encode()).hexdigest(), 16)
        vec[h % 128] += 1.0
    n = math.sqrt(sum(v * v for v in vec)) or 1.0
    return [v / n for v in vec]

A real embedding model maps each token to a position in semantic space — "database" and "DB" land near each other, "VPN" and "tunnel" land near each other. Cosine similarity between two text vectors then measures topical proximity, not literal word overlap. That is the same learned-embedding idea you met in Lab 02, where microGPT's wte matrix mapped each token id to a trained vector — with two differences here: a real retrieval model learns its vectors (microRAG's hash does not), and it embeds a whole chunk at once rather than one token at a time. The hashed-feature embedding can't do that: hashing reduces "database" to bucket 47, "DB" to bucket 19, and they cosine-out to zero. What hashed-features do capture is exact-token overlap, which is approximately what TF-IDF already gives you. So in microRAG, the embedding leg is doing a poor man's TF-IDF — useful as a fallback, structurally identical to a real embedding from the calling code's perspective.

database and DB nearly coincide; database and VPN sit far apart (dashed). That geometry is what "matches on meaning" means — and it's the one thing TF-IDF and the hashed fallback, which only see exact tokens, cannot give you.

Three properties production teams pick embeddings for: (1) dimension — 384 (MiniLM) is fast; 1536 (OpenAI text-embedding-3-small) is more discriminating; 3072 (text-embedding-3-large) is overkill for most. (2) Language coverage — multilingual vs English-only. (3) Domain fit — generic web vs code (jina-code, nomic-embed-code) vs biomedical (BioBERT). Swapping embeddings is one line of code; getting the right one for your corpus is a multi-week evaluation.

2.3 Step 3 · hybrid retrieval

The TF-IDF index and the embedding index have each scored every chunk now, and they disagree about which chunk wins. Their scores also sit on different scales — different numeric ranges. A TF-IDF score sums the tf × idf weight of every query word a chunk contains, so here it runs about 0–12 (a few query words, each worth up to the ~2.95 IDF cap from the worked example, with no hard ceiling). An embedding score is a cosine, so it always falls between roughly 0 and 1. Add the two raw scores together and the far-larger TF-IDF numbers drown out the embedding score completely — the ranking collapses to keyword-only, whatever the embedding judged.

So Step 3 puts them on equal footing before combining them. It normalizes each index's scores into a common [0, 1] range (min–max: the round's best chunk becomes 1, its worst 0), then blends the two with a single dial, w_kw, that sets how much to trust keywords versus meaning. The top-K chunks that survive the blend are what the answer gets built from.

def retrieve(query, chunks, idf, top_k=4, w_kw=0.5):
    q_emb = embed(query)
    kw = [tfidf_score(query, c["tfidf"], idf) for c in chunks]
    em = [cosine(q_emb, c["emb"]) for c in chunks]

    def normalize(xs):                       # min-max → [0, 1]
        lo, hi = min(xs), max(xs)
        return [0.0]*len(xs) if hi == lo else [(x - lo) / (hi - lo) for x in xs]

    kw_n, em_n = normalize(kw), normalize(em)
    scored = [
        {**c, "kw_score": kw_n[i], "em_score": em_n[i],
              "hybrid_score": w_kw * kw_n[i] + (1 - w_kw) * em_n[i]}
        for i, c in enumerate(chunks)
    ]
    scored.sort(key=lambda x: x["hybrid_score"], reverse=True)
    return scored[:top_k]

Three knobs: top-K, the keyword/embedding weight w_kw, and the score-normalization scheme. All three are attack surfaces — Lab 15 §4 (embedding collision) specifically exploits a high w_kw + min-max normalization, since a single multi-topic document can dominate the BM25 component for many unrelated queries.

Walk three queries: "how do I reset my password?" lands chunk 0 of password_policy.md on both legs; "what database hostnames exist?" lands architecture.md on keyword but a near-irrelevant chunk on embedding (a known weakness of the hashed-feature fallback); "employee onboarding new hire" lands clean and unanimous. The same widget is the diagnostic tool you'd reach for in production when a user reports a wrong answer.

What the weights mean in practice

• w_kw = 0.0 · embedding-only. Best for paraphrase-heavy queries ("what's the deal with…"). Worst for exact-token queries (hostnames, error codes, version strings) — the embedding can't see literals.
• w_kw = 0.5 · balanced (microRAG default). What most production systems start with. Easy to reason about; rarely the absolute best but rarely catastrophic either.
• w_kw = 1.0 · keyword-only. Behaves like classic IR — TF-IDF / BM25. Strong on technical jargon and exact strings, weak on synonyms ("sign in" vs "log in").
• Dynamic weighting. Some systems set w_kw as a function of query length (long natural-language queries lean on embeddings; short keyword queries lean on TF-IDF) or trigger token category (a query containing UPPERCASE_TOKEN-shaped strings dials up keyword weight).

2.4 Step 4 · augment

Augmentation glues the top chunks into a prompt. It looks trivial, but it hides RAG's one genuinely load-bearing design choice — the one every Lab 15 attack ultimately exploits.

SYSTEM_PROMPT = """You answer questions using ONLY the retrieved context below.
If the context does not contain the answer, say so plainly. Cite the source
filename at the end of every fact you take from the context."""


def augment(query, top):
    context = "\n\n".join(
        f"[source: {c['doc_path']} · chunk {c['chunk_idx']}]\n{c['body']}"
        for c in top
    )
    user = f"# Retrieved context\n\n{context}\n\n# User question\n\n{query}"
    return [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user",   "content": user},
    ]

This is the entire "RAG" insight: paste the retrieved chunks into the user prompt. The LLM has no special channel for "retrieved data" — chunks arrive as ordinary user tokens. This is the same trust-boundary problem Lab 04 §2.5 named and Lab 05 attacked. Lab 15 §5 (retrieval hijacking) drops instructions into uploaded documents specifically because those instructions reach the LLM as part of the "user" role and bypass any input filter.

2.5 Step 5 · generate

def chat(messages):
    import httpx
    url = os.environ["LLM_BASE_URL"].rstrip("/") + "/chat/completions"
    # The RC GenAI endpoint streams Server-Sent Events even for one-shot
    # calls, so ask for a stream and stitch the content deltas together.
    r = httpx.post(url,
        headers={"Authorization": f"Bearer {os.environ['LLM_API_KEY']}"},
        json={"model": os.environ["LLM_MODEL"], "messages": messages,
              "temperature": 0.2, "stream": True},
        timeout=120)
    r.raise_for_status()
    out = []
    for line in r.text.splitlines():
        if not line.startswith("data: "):
            continue
        payload = line[len("data: "):].strip()
        if payload == "[DONE]":
            break
        try:
            delta = json.loads(payload)["choices"][0]["delta"]
        except (json.JSONDecodeError, KeyError, IndexError):
            continue
        if delta.get("content"):
            out.append(delta["content"])
    return "".join(out).strip()

OpenAI-compatible round-trip — the same client microagent and microMCP use. The RC GenAI endpoint streams its reply as Server-Sent Events (even for a one-shot call), so chat() asks for a stream and stitches the delta.content pieces back together. Set the three LLM_* env vars (Rivanna's userguide) and you're done.

2.6 Scaling Step 2 · the vector database

microRAG's retrieve() brute-forces a cosine against every vector in a Python list. That's fine for 13 chunks and hopeless for 13 million — a single query would have to compare against all of them. The data structure production reaches for instead is a vector database (Pinecone, Weaviate, Qdrant, pgvector). Two things define it:

• What it stores. Not documents — records. Each record is an id, the chunk's embedding vector, and a payload of metadata (source file, chunk index, the raw text, an access label). The vector is what you search by; the payload is what you get back and paste into the prompt.
• What it's optimized for. One operation: given a query vector, return the k nearest stored vectors — fast, over millions of records. It does this with an approximate nearest-neighbor (ANN) index (HNSW, IVF-PQ) that walks toward the closest vectors without scanning every one, trading a sliver of accuracy for enormous speed. And because every record carries that payload metadata, the search can be filtered before it even runs — WHERE payload.label = 'public' — so a user never retrieves a chunk they aren't allowed to see.

The widget below is that search, drawn in two dimensions. Real embeddings live in hundreds of dimensions, but the picture is the same: your query is a point, and retrieval hands back its nearest neighbors.

Drag toward a cluster and its chunks win; drag to empty space and the "nearest" neighbors come back anyway — a vector database never says "no match," it always returns the closest k. That is the same edge case Step 3's exercise flagged, now visible in the geometry.

3 · Run it ~15 min

You'll run microRAG on Rivanna: set up a session, pull the code, then run it two ways — first retrieval alone (no LLM, no keys — instant feedback on which chunks a query pulls), then the full pipeline with a model turning those chunks into an answer.

3.1 Get a session and the code

microRAG needs no GPU — it only calls the remote model over the network — so a plain CPU session is enough. Open Open OnDemand → Interactive Apps → Code Server and launch it on the Interactive partition (1 core, 6 GB, no GPU, Work Directory /scratch/$USER). When it shows Running, click Connect to Code Server and open a terminal (Terminal → New Terminal). Pull the code straight from the lab site and unzip it:

$ cd /scratch/$USER && mkdir -p lab-14 && cd lab-14
$ wget https://researcher111.github.io/ML-Security-Public/labs/lab-14/lab-14-code.zip
$ unzip lab-14-code.zip
$ ls
microdata/  rag/  README.md

3.2 Retrieval only · no LLM needed

$ python rag/micro_rag.py retrieve "how do I reset my password?"
# Loading microdata …
# 13 chunks indexed, embedding dim=128

[0.846 kw=1.00 em=0.69] password_policy.md · chunk 0
  # Password Reset Policy  To reset your password at MegaCorpAI:  1. Visit the login page at https://login.megacorpai.local…
[0.500 kw=0.00 em=1.00] password_policy.md · chunk 2
  T Operations.
[0.140 kw=0.00 em=0.28] password_policy.md · chunk 1
  et a new passphrase with at least sixteen characters, including one symbol and one number…

The score columns are kw (keyword/TF-IDF), em (embedding cosine), hybrid (weighted blend). Chunk 0 wins on keywords because it contains "password" and "reset" verbatim. Chunk 2 wins on embedding because its hashed bucket happens to overlap with the query's. The hybrid blend favors chunk 0 — exactly what a user would want.

3.3 Full pipeline · against Rivanna's Kimi K2.5

$ export LLM_BASE_URL=https://open-webui.rc.virginia.edu/api
$ export LLM_API_KEY=...      # open-webui.rc.virginia.edu → Settings → Account → API keys
$ export LLM_MODEL="Kimi K2.5"
$ python rag/micro_rag.py ask "how do I reset my password?"
To reset your password at MegaCorpAI:
1. Visit https://login.megacorpai.local and click "Need help signing in".
2. Authenticate with Okta Verify (preferred) or SMS fallback.
3. Set a new passphrase with at least sixteen characters, including one symbol and one number.

Source: password_policy.md
If you're locked out, contact the helpdesk at extension 4357 [source: password_policy.md].

That's it — the entire RAG round trip in one CLI invocation. The same five steps a production system runs, no abstractions in the way.

4 · Scaling up · the production server ~30 min

Open ../lab-15/server/baseline_rag.py. ↓ download lab-15-code.zip It's ~250 lines — structurally a microRAG with an HTTP front door and three things scaled up:

• HTTP instead of CLI. FastAPI exposes /query, /ingest, /reset, /health. Multiple clients, persistent index, attacker-controlled ingestion.
• A file-read tool the LLM can call. Modern RAG often combines retrieval with on-demand tool calls (read fresh data, fetch a URL, query a DB). Lab 15 §5 (retrieval hijacking) abuses this.
• Weak input/output guardrails. Substring blocklist on the input; regex redaction on the output. Both are pattern matchers; both bypassable. Lab 15 §6 walks the encoded / substituted bypasses.

And four planted vulnerabilities — clearly marked with # Vuln N · … comments — matching the canonical RAG-attack taxonomy:

As you read each one, ask yourself the question the previous "scaling up" labs primed you with: what does the new piece trust, and what happens if that trust is misplaced? Lab 15 calls each vulnerability out explicitly with a Vulnerability · … callout.

5 · What's next · before the attacks

Lab 15 · Attacking RAG takes the same baseline_rag.py you just read and runs five canonical attacks against it. Before you click through, take a few minutes with §4. Four of the additions there each open a different kind of trouble:

1. The seed corpus ingested without curation in seed_from_disk().
2. The /ingest endpoint that accepts any text with any claimed source.
3. The hybrid retrieval weight w_kw = 0.6 with no per-source diversity cap.
4. The file-read tool with a literal-path blocklist.

For each one, sketch an attack on your own. Even naming the seam and the kind of input that would exercise it is most of the work. Lab 15 walks the attacks; the further you can get pencil-and-paper first, the more useful that lab will be.

6 · Wrapping up · what a production RAG actually has ~15 min

microRAG is structurally complete — the same five steps are the skeleton underneath essentially every commercial RAG system you're likely to meet. What's missing from microRAG is the cost-and-correctness machinery production systems add around those five steps. Read this list as a map of the next several thousand lines of code you'd write if microRAG were going to support a thousand-user company tomorrow.

The full vendor stack at a glance

A typical 2026 production RAG chains those pieces into a single request path — and every stage is a scaled-up version of one of microRAG's five steps:

Total latency: ~800 ms. Total cost: a few cents per query. Total LOC: thousands. Total moving pieces: the eight cards above plus whatever your particular domain demands.

7 · Is the answer grounded? · NLI verification ~15 min

Retrieval can pull the perfect chunks and the LLM can still lie. Step 4's system prompt asks the model to use only the retrieved context, but nothing enforces it — the model can blend in a plausible fact from its training data, or quietly contradict the source. The assignment's grounding gate checks the retrieval side ("did we find anything relevant?"). This section checks the answer side: does each sentence the model wrote actually follow from a retrieved chunk? The production tool for that job is an NLI model.

What NLI does

Natural Language Inference takes two short texts — a premise and a hypothesis — and returns one of three labels: the premise entails the hypothesis (if the premise holds, the hypothesis must too), contradicts it, or is neutral (the premise neither proves nor disproves it). It's a small, fast, decades-old NLP task that modern encoder models do very well.

Wiring it into RAG is one idea: premise = a retrieved chunk, hypothesis = one sentence of the answer. A sentence is grounded if at least one retrieved chunk entails it. If every chunk comes back neutral, the model made the sentence up — it's in the answer but supported by nothing you retrieved. If a chunk contradicts it, worse: the model misread its own source. Either way, flag it — drop the sentence, refuse the answer, or attach a "low-confidence" warning.

Grounded ≠ true, though: if a poisoned chunk (Lab 15 §3, ingestion poisoning) entails the sentence, NLI happily marks it grounded. NLI checks the answer against the corpus — it can't check the corpus.

The models, and the cost

The workhorses are small encoders fine-tuned on NLI data — DeBERTa-v3 or RoBERTa trained on MNLI, or purpose-built ones like Vectara's hallucination_evaluation_model. They're a few hundred million parameters, run in tens of milliseconds on CPU, and cost a rounding error next to the generation call. The alternative — LLM-as-judge, asking a big model "is this sentence supported by that context?" — is more flexible but adds a second full LLM call per answer, which is slower and far pricier at scale. That's why the §6 pipeline uses a dedicated NLI step rather than a second LLM.

Assignment · harden the retriever ~90 min · autograded

microRAG's retriever always returns something — the least-bad guess even for a question the corpus can't answer — and it lets one multi-chunk document dominate the whole top-K. Both are real weaknesses (the §2.3 edge-case exercise and the Lab 15 §4 attack). You'll fix them by implementing one function, rank(query), in a template that already contains a working microRAG retriever over a small bundled corpus.

Grounding gate — decide whether retrieval found anything, before you answer. A RAG retriever always hands back its top-K, ranked best-first — even when the corpus has no answer at all. Worse, as §2.3 showed, min–max normalization makes that top chunk's hybrid score look like a confident 1.0 no matter how weak the real match is. Paste those irrelevant chunks into the prompt and the LLM will usually produce a fluent, wrong answer out of the noise. A grounding gate is a relevance floor you check before calling the LLM: if the strongest genuine match — here, the best un-normalized keyword score — is too weak, the system abstains and says "I don't have that" instead of guessing. Production RAG calls this an answerability or retrieval-confidence check; it's the cheapest defense there is against hallucinating on out-of-scope questions.

Diversity cap — don't let one document own the whole answer. Retrieval ranks chunks by score alone, so a single document with several high-scoring chunks can fill every slot in the top-K — and then the LLM only ever sees that one source's view. That's a quality bug and a security one: in Lab 15 §4 (embedding collision) an attacker uploads one document engineered to win retrieval for many unrelated queries, sweeping the context out from under every legitimate source. A diversity cap fixes both by letting at most a couple of chunks from any single document into the top-K, forcing the results to span sources.

In short, one knob decides whether to answer, the other decides which sources get to shape the answer. Here they are side by side:

The hook

base_retrieve(query) is given; it returns every chunk already sorted best-first. Each item:

# base_retrieve(query) -> list[dict], sorted by "hybrid" descending
{ "id": "database.md#2", "doc": "database.md",
  "raw_kw": 0.52,   # RAW (un-normalized) TF-IDF score — what the grounding gate needs
  "raw_em": 0.31,   # raw embedding cosine
  "hybrid": 0.90 }  # normalized blend used for the sort

def rank(query):        # <-- the only thing you write
    scored = base_retrieve(query)
    # 1. grounding gate:  return None if max(raw_kw) < MIN_KW
    # 2. diversity cap:   keep <= MAX_PER_SOURCE per doc, then take TOP_K
    ...                 # return a list of chunk ids, or None to REFUSE

Why raw_kw and not hybrid? Min–max normalization makes the top chunk's hybrid score ≈ 1.0 even when the query matches nothing — so the gate has to read the un-normalized keyword score (exactly the trap §2.3 described).

I/O protocol · one query per line

$ echo "how do I reset my password?" | python3 assignment.py
OK: password.md#0, database.md#2, onboarding.md#1, network.md#0
$ echo "what is the office address in Tokyo?" | python3 assignment.py
REFUSE

Files & the autograder

↓ assignment.py ↓ test_assignment.py ↓ autograder.zip

Edit only the body of rank(). Grade yourself locally — the same checks Gradescope runs:

$ python3 test_assignment.py
[PASS] 10/10  benign: 'reset my password okta' -> top is password.md
…
[PASS] 25/25  grounding gate: nonsense query REFUSES (returns None)
[PASS] 25/25  diversity cap: 'database' <=2 per source, >=2 sources
        100/100

Rubric · 100 points, all automatic

Submit your edited assignment.py to Gradescope — nothing else. The grader imports your rank() and runs the checks above.

FAQ

Why not just use LangChain?

Same answer as microagent: LangChain runs the same five steps. It also runs a hundred other things — output parsers, callback systems, chains of chains. Use LangChain in production; use microRAG to understand what LangChain is doing under the hood, and to see exactly where Lab 15's attacks land.

Will the hashed-feature embedding catch any semantic similarity?

Surprisingly, yes — token overlap proxies for semantic similarity when the corpus is small and the queries share vocabulary with the documents. The retrieval traces in §3 work cleanly on a 4-document corpus. Quality drops sharply at scale; that's why production uses real embeddings. The lab uses the fallback so it runs without a model download.

What's the relationship to Lab 06 (microMCP)?

Both are ways to give an LLM external knowledge it didn't have at training time. MCP gives it tool-calling capability — fetch this URL, query this DB, read this file. RAG pre-fetches relevant text and pastes it into the prompt. Many production assistants use both (RAG for "what we already know," MCP for "what we can find out right now"). The two attack surfaces overlap — prompt injection, trust boundaries — but the failure modes differ enough to be worth covering separately.

What's the relationship to Lab 03 (nanochat)?

Lab 03 built the LLM. Lab 14 builds the retrieval system that connects an LLM to external knowledge. Together they cover both halves of "how a modern AI assistant answers a question." Production assistants in 2026 almost always pair a hosted LLM with a custom RAG over the organization's documents.