Fine-tune vs. retrieval · one model, three systems

Yu-Gi-Oh SLM Arena

Yu-Gi-Oh is a trading card game with thousands of cards and famously fiddly rules — cards interact in ways even seasoned players argue about, which makes it a genuinely hard test for a small AI model. So I built the same Gemma 2 2B three ways and put them against the same questions: A the model straight out of the box, B after I fine-tuned it on Yu-Gi-Oh questions and answers, and C that fine-tune paired with a retriever that looks up the real card text first. Every answer is scored 0–10 by a blind judge that is handed the correct answer, so the scores are checkable — and the point is simple: fine-tuning teaches the model how to answer; retrieval is what gives it the facts.

3 systemsGemma 2 2B one model 2,683 QA pairs60 held-out1 blind judge
Section 1 · One question, three systems, judged live in front of you.
Live arena · Gemma 2 2B ×3
connecting…

Ask one Yu-Gi-Oh question and the same model answers it three ways — base, fine-tuned, and fine-tuned + retrieval. Pick a question by category (those carry a written answer key, so the judge grades against a reference) or type your own (graded from the judge's own knowledge). Then a blind judge scores all three 0–10.

same question · 3 systems · judged 0–10
A · Baseclosed book
Ask a question to see the untouched Gemma 2 2B answer.
B · Fine-tuneclosed book
Our QLoRA fine-tune, no retrieval.
C · +retrievalRAG · winner
Fine-tune with the retriever supplying passages.
Frozen held-out eval · 60 questions

Not the live arena — the offline evaluation. Each system answered the same 60 decontaminated held-out questions and a reference-grounded blind judge scored every answer 0–10. Shown on two sets: the original (biased toward card-facts and rulings) and a category-balanced unbiased 60. C wins both; only C survives the balanced mix.

A base · B fine-tune (closed book)C · +retrieval

Measured mean judge score /10 per system on both held-out sets — the biased original 60 (left) and the balanced unbiased 60 (right). C wins on both; only C holds up when the mix is balanced.

The lesson in both: the same question exposes the same gap every time — the two closed-book systems guess from memory, and only the one that looked up the card actually answers. Fine-tuning lifts B over the raw base (+1.27 on the biased set), but the leap is B→C: +2.80 on the biased 60 and +5.63 on the balanced 60. When the question mix is fair, both closed-book systems collapse toward 2/10 while C stays above 8/10 — because C is the only one that reads the real card text before it speaks.
Section 2 · Why C wins — retrieval supplies the facts fine-tuning can't hold.
Hybrid retriever · 42,412 chunks

Before C answers, a retriever pulls the five most relevant snippets from the corpus and pastes them into the prompt. Two searches run together — dense (by meaning, MiniLM-L6) and BM25 (by exact words) — and are merged with reciprocal-rank fusion. The merged (hybrid) list is what feeds C.

DenseBM25Hybrid · feeds C

Measured recall@k on the 60 held-out questions for three retrievers. Hybrid (RRF) — the one that feeds C — reaches 0.93 at k=5, the top-5 actually used.

Blind judge · gemini-3.1-flash-lite

Every answer is scored by a judge that never sees which system wrote it and grades one answer at a time. It is handed the question's gold answer and source evidence, so scores are checkable against a known-good reference — four dimensions that add up to /10.

  • Reference-grounded — graded against the gold answer + evidence, not vibes.
  • Blind & pointwise — never told which system wrote the answer, and scores one at a time.
  • Groundedness scored separately — an answer can be honest but thin, or fluent and fabricated.
  • A correct refusal beats a confident wrong answer — bluffing is punished, not rewarded.
  • Paired testing — same 60 questions per system, compared with a bootstrap 95% CI (t + Wilcoxon).

The 10-point rubric, split into its four dimensions.

The lesson in both: retrieval supplies the facts fine-tuning can't hold in 20.8M adapter weights. The hybrid retriever finds the right card text 93% of the time, and a strict judge that scores groundedness separately is what lets us see the difference: groundedness climbs 0.18 → 0.87 → 1.65 and correctness only jumps (2.35 → 3.85) once real passages are in the prompt. Fine-tuning taught the model the shape of a good answer; retrieval is what put the facts in front of it.
Section 3 · What built it, and what it cost.
Corpus · 27.5 MB across sources

The fine-tune's supervised set is 2,683 grounded QA pairs, distilled by a teacher model from a curated Yu-Gi-Oh corpus and filtered through a blind-judge gauntlet. Two free sources: Yugipedia editorial prose (CC BY-SA 4.0) and YGOPRODeck card facts (used only as labelled fair-use context).

Measured share of the 27.5 MB corpus by source category — tips and rulings dominate, lore and mechanics trail.

2,683grounded QA pairs kept
Corpus
27.5 MB
QA generated
2,796
Held-out
60 Q+gold
Cost to build · $3.26 of $25

Two bills built this: Modal for the GPU fine-tune and the Gemini API for generating the QA data and running the judge. The base is Google's (imported free); the retriever, serving and evaluation all ran on a local RTX 3060, so they cost nothing.

Measured invoiced spend by bill — Modal GPU $0.26 and Gemini API $3.00; the local RTX 3060 cost $0. Total $3.26.

$3.26total invoiced · of a $25 budget
Gemini API
$3.00 92%
Modal GPU
$0.26 8%
Local GPU
$0 0%
The lesson in both: a grounded system is mostly careful data plumbing, not a big GPU bill. The whole build cost $3.26 — 92% of it the API that wrote and judged the 2,796 QA pairs (2,683 kept), and just $0.26 for the QLoRA fine-tune itself; the retriever, serving and evaluation were free on a home GPU. The expensive, valuable part was not the training run — it was assembling a clean 27.5 MB corpus and the passages C reads at answer time. Fine-tuning teaches the answer shape; retrieval supplies the facts.