docs: May 2026 tool-calling update + agentic retrieval landscape

Two new corpus docs plus targeted GOTCHAS corrections based on
Ollama/llama.cpp/vLLM issue tracker review, late-2025/early-2026
research papers, and convergent empirical evidence from two production
Gemma 4 agents.

New:
- CORPUS_tool_calling_2026-05.md: current state of tool calling.
  Properly characterizes the think flag (parser-side bug, not
  model-side; recipe by loop shape), surfaces ollama/ollama#15539,
  documents the "doesn't fire tools without explicit ask" research
  (Probe & Prefill, BiasBusters, When2Call), XGrammar-2.
- CORPUS_agentic_retrieval.md: post-RAG landscape. Hybrid retrieval
  + cross-encoder rerank as the settled base, CRAG-shaped flows,
  LightRAG/LazyGraphRAG, mem0 vs Letta MemFS, deep-research agents.
  Production-vs-experimental sorting for self-hosted small-model use.

Updates:
- GOTCHAS.md: the think:false rule, Vulkan unused-token loop, and
  Ollama 0.20 streaming bug all marked SUPERSEDED/FIXED/PARTIALLY
  FIXED in-place with pointers to the May update.
- README.md: indexed the two new docs.
- .gitignore: added private/ for bot-specific notes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

.gitignore

@@ -13,3 +13,6 @@ scripts/bakeoff/runs*/*/work/
 # Editor / OS
 .DS_Store
 *.swp
+
+# Private — bot-specific findings, recommendations tied to non-public projects
+private/

CORPUS_agentic_retrieval.md

@@ -0,0 +1,308 @@
+# Agentic Information Retrieval Beyond RAG, May 2026
+
+> A view of where retrieval and memory architectures are converging for
+> agentic applications, with explicit production-vs-experimental sorting
+> for self-hosted small-model deployments (Gemma 4 26B–31B, Qwen3 8–14B).
+>
+> Last updated: 2026-05-25. Cuts off long-context-as-RAG-replacement,
+> graph retrieval, agent memory, and deep-research-agent patterns.
+
+## Headlines
+
+1. **The 2023-era named patterns (Self-RAG, CRAG, FLARE, MemGPT) survived
+   as design templates but their original codebases are mostly stale.**
+   The 2026 production stack reimplements them as graph nodes inside
+   LangGraph or LlamaIndex Workflows. Use the patterns; don't try to use
+   the original repos.
+2. **Long context did not replace RAG.** Multi-needle retrieval at 1M
+   tokens regressed in some frontier models in 2026; even where it works,
+   the cost/latency math means RAG-then-stuff stays the winning shape.
+   The "just dump the corpus" approach is now a niche choice for
+   single-document deep reasoning, not corpus retrieval.
+3. **GraphRAG got cost-viable.** Microsoft's
+   [LazyGraphRAG](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/)
+   matches or beats full GraphRAG quality at indexing cost equal to vector
+   RAG and query cost 0.1% of full GraphRAG. GA targeting Q1–Q2 2026 in
+   the MSR repo. For SMB corpora today, [LightRAG](https://github.com/hkuds/lightrag)
+   is the production option.
+4. **Memory architectures split into two camps**: managed CRUD layer
+   (mem0, Zep) vs. agent-edited state (Letta with Context Repositories
+   "MemFS", A-MEM). For small open models, lean toward managed — Gemma 4
+   will sometimes drop tool calls and you'll lose memory silently if the
+   model owns it.
+5. **The settled retrieval base layer is hybrid BM25 + dense + RRF
+   fusion + cross-encoder rerank.** "Dense embeddings alone" has been
+   retired in serious systems. The lexical signal is load-bearing for
+   error codes, identifiers, and technical corpora that embeddings
+   underperform on.
+
+---
+
+## 1. Agentic RAG — patterns, not libraries
+
+The original papers all date from 2023. In 2026 they exist as patterns
+absorbed into agent frameworks.
+
+| Pattern | Origin | What survived | What to use |
+|---------|--------|---------------|-------------|
+| **Self-RAG** | Asai et al., ICLR 2024 | Reflection-token idea (model decides per-step whether to retrieve, critique chunks) | Re-implement as graph nodes; the trained 7B/13B models aren't being updated |
+| **CRAG (Corrective RAG)** | Yan et al., 2024 | retrieve → evaluate → correct → generate | LangGraph cookbook implementation is now the canonical artifact |
+| **FLARE** | Jiang et al., 2023 | Uncertainty-triggered retrieval (low-confidence span ⇒ re-retrieve) | Pattern survives, original repo stale |
+| **PRISM / RELOOP** | arxiv 2510.14278, 2510.20505 (late 2025) | New, multi-hop with precision-focused control flow | Experimental — track if multi-hop precision is your bottleneck |
+
+**Frameworks ranked by fit for branching agentic RAG flows:**
+
+- **LangGraph** (graph-native, best for CRAG/Self-RAG-shaped branching)
+- **LlamaIndex Workflows** (event-driven, similar story)
+- **Haystack** (more structured pipelines, weaker on dynamic branching)
+
+GitHub star order, January 2026:
+LangChain 125K · Dify 114K · RAGFlow 70K · LlamaIndex 46.5K.
+([source](https://florinelchis.medium.com/top-10-rag-frameworks-on-github-by-stars-january-2026-e6edff1e0d91))
+
+**Convergence:** yes, on the pattern — agent decides when to retrieve,
+evaluates retrieved chunks, optionally re-queries. **No convergence** on
+which framework owns it.
+
+---
+
+## 2. GraphRAG: the cost story
+
+Microsoft's original GraphRAG had a ~$33K indexing bill on large corpora,
+making it a non-starter outside well-funded enterprises. 2026 changed
+that.
+
+| Project | Maturity | License | When to use |
+|---------|----------|---------|-------------|
+| [Microsoft GraphRAG](https://github.com/microsoft/graphrag) | Production but expensive | MIT | Global-question answering over stable corpora when budget is unconstrained |
+| [LazyGraphRAG](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/) | Beta, GA Q1–Q2 2026 | MIT | The new default — same quality, 0.1% query cost |
+| [LightRAG](https://github.com/hkuds/lightrag) (EMNLP 2025) | Production | MIT | SMB corpora today, Docker-deployable, PG+pgvector+AGE one-DB path |
+| [nano-graphrag](https://github.com/gusye1234/nano-graphrag), [fast-graphrag](https://github.com/circlemind-ai/fast-graphrag) | Experimental | MIT | Hackable baselines, sub-1M-doc cases |
+
+**2026 consensus on graph vs. vector** (per
+[GraphRAG-Bench, ICLR'26](https://github.com/GraphRAG-Bench/GraphRAG-Benchmark)):
+graph wins when (a) coherent domain corpus, (b) questions require
+synthesizing across many documents, (c) indexing pass is affordable.
+Vector wins on point-fact lookup, noisy/heterogeneous corpora, cost,
+latency.
+
+**Recommendation:** for a self-hosted small-model setup with a coherent
+domain corpus, LightRAG is the production try-this. Skip full Microsoft
+GraphRAG (cost). Watch for LazyGraphRAG GA and swap when shipped.
+
+---
+
+## 3. Long context did not replace RAG
+
+The 2026 reality killed the "just stuff the corpus" dream.
+
+- **Claude Opus 4.7** regressed on multi-needle 1M-context retrieval —
+  32.2% on MRCR v2 8-needle at 1M vs. 78.3% for Opus 4.6.
+  ([source](https://blog.wentuo.ai/en/claude-opus-4-7-long-context-regression-en.html))
+- Multi-needle leaderboard at 1M tokens: Gemini 3 leads at 89%,
+  GPT-5.5 at 74%, Opus 4.7 at 56%.
+  ([source](https://www.digitalapplied.com/blog/gpt-5-5-vs-claude-opus-4-7-frontier-comparison))
+- **RAGAS 2.0 (April 2026) study** found faithfulness dropped up to **40%**
+  when context windows were overloaded with irrelevant filler.
+  ([source](https://open-techstack.com/blog/rag-vs-long-context-2026/))
+
+**Effective context for multi-needle workloads is 200–400K** for current
+frontier models even when the marketed window is 1M+. Above that, attention
+dilutes, "lost in the middle" kicks in, latency explodes, cost is linear.
+
+**Gemma 4 specifically: 128K context.** Plenty for the retrieve-then-stuff
+pattern, but cap effective working context at ~64K to stay above the
+lost-in-the-middle floor for 8–31B models. Not a corpus replacement.
+
+**The winning shape:** RAG retrieves the top 50–200K most relevant tokens
+→ long-context model reasons carefully. "Just dump everything" is the
+niche choice for single long-document analysis (legal contract, research
+paper).
+
+---
+
+## 4. Memory architectures
+
+Real movement in 2025–2026. Five-way taxonomy
+([SurePrompts](https://sureprompts.com/blog/agent-memory-architectures-compared-2026)):
+provider-managed, self-managing (Letta), CRUD memory layer (mem0), vector
+RAG, custom in-app schema.
+
+| Project | Camp | Maturity | License | Best for |
+|---------|------|----------|---------|----------|
+| [Letta](https://github.com/letta-ai/letta) (formerly MemGPT) | Agent-edited state. **Feb 2026: Context Repositories ("MemFS")** — memory projected into git-backed files operated on via bash/computer-use tools | Production | Apache 2.0 | Long-horizon coherence as the product, strong models only |
+| [mem0](https://mem0.ai/research) | CRUD memory layer, vector + optional graph, three-level hierarchy | Production | Apache 2.0 | "Remember the user across sessions" consumer-app cases |
+| Zep | Graph-based temporal memory, claims 75.14% on LoCoMo (contested vs mem0) | Production | Commercial + community OSS | Temporal recall + graph relationships |
+| [A-MEM](https://github.com/agiresearch/a-mem) (arxiv 2502.12110) | Zettelkasten-inspired — adding new memories triggers updates to existing ones, evolving graph | Experimental | Research | If you want to play with self-organizing memory graphs |
+| Hindsight, Memvid, Supermemory, MemPalace | Various | Experimental | Various | Watch, don't commit |
+
+**For Gemma-4-sized models specifically:** lean toward mem0-style managed
+memory. Letta's self-edit pattern needs a model that reliably calls the
+memory-save tool every time it should — Gemma 4 won't. The benchmarking
+fight between mem0 and Zep on LoCoMo is contested; pick on integration
+fit, not on the leaderboard.
+
+---
+
+## 5. Retrieval-augmented planning and multi-hop
+
+The 2023-era patterns (ReAct, IRCoT, Self-Ask, DRAGIN) remain the
+conceptual foundation. Known weaknesses: error propagation, noisy evidence
+sets, fixed step budgets.
+
+**Practical pattern that converged in 2026:**
+```
+ReAct loop + cross-encoder reranker on every retrieval
+         + explicit "do I have enough?" verification step before answering
+```
+
+The verification step is the real change from 2023. Agents are expected
+to check their evidence rather than just synthesize.
+
+**Recent academic:**
+- [PRISM (arxiv 2510.14278)](https://arxiv.org/pdf/2510.14278) and
+  [RELOOP (arxiv 2510.20505)](https://arxiv.org/pdf/2510.20505) — late
+  2025 frameworks attempting to fix the precision-recall imbalance of
+  IRCoT-style approaches. Experimental.
+- [Four-axis design framework survey (arxiv 2601.00536)](https://arxiv.org/html/2601.00536v1)
+  — useful 2026 organization of the design space.
+
+---
+
+## 6. Deep-research agents
+
+Heavy activity. OpenAI Deep Research (early 2025) set the template;
+open-source caught up fast.
+
+| Project | Maturity | License | Notes |
+|---------|----------|---------|-------|
+| OpenAI / Google Deep Research | Closed | — | Reference target. OpenAI ~67% GAIA validation |
+| [smolagents/open_deep_research](https://github.com/huggingface/smolagents/tree/main/examples/open_deep_research) | Production code, experimental quality with small models | Apache 2.0 | 55% GAIA validation with GPT-4o. CodeAgent uses ~30% fewer steps than ToolCallingAgent |
+| [gpt-researcher](https://github.com/assafelovic/gpt-researcher) | Production | Apache 2.0 | Most mature open option. Planner + search + reader + report writer with citations |
+| [LangChain open_deep_research](https://github.com/langchain-ai/open_deep_research) | Production | MIT | Tavily search default, MCP support, model-agnostic |
+| Perplexity Sonar API | Production, closed | — | 37% citation hallucination rate (vs ChatGPT Search 67%, Grok 3 94%) per Columbia Journalism Review audit. Best closed-source citation grounding |
+
+**For Gemma 4 specifically:** 7B–13B class models work in deep-research
+frameworks but quality drops noticeably vs. GPT-4o-class. **The 31B
+variant is probably the sweet spot for self-hosted deep research;
+smaller variants struggle with the planning step.**
+
+---
+
+## 7. Hybrid / late-interaction retrievers — alive and well
+
+"Dense + reranker won" is **not** the 2026 consensus.
+
+| Approach | Maturity | License | When |
+|----------|----------|---------|------|
+| **ColBERT v2** late-interaction, per-token embeddings | Production | Apache 2.0 | Retrieval-heavy reference architecture |
+| [**ColPali**](https://arxiv.org/html/2407.01449v5) — late-interaction over document images via VLMs, OCR-free | Beta → production transition | Apache 2.0 | Visually rich documents (tables, charts, forms). Storage cost is the gotcha (multi-vector per page). Zilliz/Milvus and Weaviate ship integrations |
+| **SPLADE** — learned sparse, lexical + semantic | Production | Apache 2.0 | Hybrid pipelines. Lexical signal load-bearing for technical corpora |
+| [**BGE-M3**](https://bge-model.com/bge/bge_m3.html) — dense + sparse + multi-vector in one model | Production | MIT | Multilingual default |
+| [**Qwen3-Embedding**](https://github.com/QwenLM/Qwen3-Embedding) (0.6B/4B/8B) — 8B topped MTEB multilingual at 70.58 mid-2025 | Production | Apache 2.0 | Strong open default, especially multilingual |
+
+**Settled production pattern**
+([source](https://tianpan.co/blog/2026-04-12-hybrid-search-production-bm25-dense-embeddings)):
+
+```
+BM25 + dense ANN (parallel) → RRF fusion → top-100 candidates
+                                        → cross-encoder rerank → top-5 → LLM
+```
+
+BM25 stubbornly wins on literal matches (error codes, product SKUs,
+identifiers) — embeddings underperform on those. "Embeddings alone" has
+been mostly retired in serious systems.
+
+---
+
+## 8. Reranking and verification
+
+Cross-encoder reranking is table stakes. Choice of model:
+
+| Reranker | License | Notes |
+|----------|---------|-------|
+| [BGE-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3) | MIT | Multilingual, self-hostable, price/performance default |
+| Cohere Rerank v3.5 | Closed/paid | 4096-token context, JSON / semi-structured. "Just works" choice |
+| [Jina Reranker v3](https://jina.ai/models/jina-reranker-v3/) | Apache 2.0 | 0.6B listwise (query + all candidates in one window). 81.33% Hit@1 at 188ms |
+| Qwen3 Reranker (0.6B/4B/8B) | Apache 2.0 | Competitive with BGE/Jina |
+| Voyage rerank-2.5, mixedbread mxbai, nemotron reranker | Various | Also in the mix. Nemotron at 83% Hit@1 / 243ms currently the accuracy leader per AIMultiple benchmark |
+
+**Verification patterns (genuinely new in 2026):**
+- Explicit "is this passage relevant?" LLM call after rerank
+- Citation-supported answers — every claim must point to a retrieved
+  passage
+- Groundedness scores via [Ragas](https://github.com/explodinggradients/ragas)
+
+**Ragas production thresholds**: faithfulness ≥ 0.9, answer relevancy
+≥ 0.85, context precision ≥ 0.8.
+
+---
+
+## 9. Production-vs-experimental quick reference
+
+| Thing | Maturity | License | When to use |
+|-------|----------|---------|-------------|
+| Hybrid BM25 + dense + RRF + cross-encoder rerank | **Production**, settled | varies | Default starting point for any new RAG |
+| BGE-reranker-v2-m3 / Jina v3 / Qwen3-Reranker | **Production** | MIT/Apache | The reranker step above |
+| LangGraph / LlamaIndex Workflows | **Production** | MIT | Implementing CRAG / Self-RAG–shaped flows |
+| gpt-researcher / smolagents open_deep_research | **Production** (open) | Apache/MIT | Deep-research agents |
+| LightRAG | **Production** for SMB corpora | MIT | Graph-aware retrieval without MSR cost |
+| LazyGraphRAG | **Beta** (Q1–Q2 2026 GA) | MIT | Replacing full GraphRAG when shipped |
+| mem0 | **Production** | Apache 2.0 | "Remember the user" memory |
+| Letta + Context Repositories (MemFS) | **Beta–Production** | Apache 2.0 | Long-horizon agent state, strong models only |
+| ColPali | **Beta–production** | Apache 2.0 | Visually rich documents, accept storage cost |
+| Long context as RAG-replacement | **Niche** | n/a | Single long document, not corpus |
+| Self-RAG original models, FLARE original repo | **Paper-only** | research | Read the paper, re-implement the pattern |
+| A-MEM | **Experimental** | research | Self-organizing memory graph experiments |
+| PRISM / RELOOP | **Experimental** | research | Multi-hop precision experiments |
+
+**Genuine convergence:**
+- Hybrid retrieval + cross-encoder rerank is the universal base.
+- CRAG-shaped self-correction is the dominant agentic RAG pattern.
+- Long context complements RAG; doesn't replace it.
+- Citations / grounded answers are expected.
+
+**Still no consensus on:**
+- Which agent framework wins (LangGraph vs LlamaIndex Workflows vs
+  smolagents vs roll-your-own).
+- Memory: managed CRUD layer vs agent-edited state.
+- When graph beats vector for a given corpus (domain-dependent).
+- Whether tool-call agents or code-writing agents are the better
+  substrate (smolagents data favors code; production deployments split).
+- Memory benchmarking — LoCoMo numbers actively contested.
+
+---
+
+## 10. Picking a stack for a Gemma 4 agent
+
+For a self-hosted Gemma 4 26B/31B agent that already does basic
+tool-calling + embedding RAG and wants to step up:
+
+1. **Base retrieval**: hybrid BM25 + dense (Qwen3-Embedding-4B or
+   BGE-M3) with RRF fusion, then BGE-reranker-v2-m3 cross-encoder.
+   Non-experimental but the foundation everything else assumes.
+2. **Agentic layer**: CRAG-shaped flow in LangGraph — evaluate retrieved
+   chunks, fall back to web search if score is low, verify before
+   answering. Skip Self-RAG-the-models (won't work with Gemma); use
+   the pattern.
+3. **Memory**: mem0 for user-facing recall. Skip Letta unless the agent
+   specifically needs long-horizon autonomous behavior — Gemma 4 will
+   sometimes drop tool calls and you'll lose memory silently if the
+   model owns it.
+4. **Web search**: gpt-researcher or smolagents/open_deep_research,
+   with Gemma 4 31B doing the planning/synthesis. Expect 30–40% lower
+   quality than GPT-4o-class; budget for it.
+5. **Graph experiment**: LightRAG on whichever surface has a coherent
+   domain corpus. Skip full MSR GraphRAG (cost). Watch for LazyGraphRAG
+   GA and swap.
+6. **Long context**: don't lean on it. 128K Gemma 4 context is for
+   "retrieve a lot then reason," not "skip retrieval." Cap effective
+   working context at ~64K.
+7. **Worth a one-off experiment**: ColPali on any visual-document
+   corpus (genuinely different OCR-free path), A-MEM for memory if
+   mem0 feels too static, PRISM-style multi-hop if you have a benchmark
+   question your CRAG flow can't handle.
+
+**Don't spend time on**: original 2023 implementations (Self-RAG models,
+FLARE repo, MemGPT pre-Letta). The ideas survived; the codebases did
+not.

CORPUS_tool_calling_2026-05.md

@@ -0,0 +1,429 @@
+# Gemma 4 Tool-Calling: State of the Art, May 2026
+
+> Six-week update on top of the April 2026 corpus. **Supersedes parts of
+> `CORPUS_tool_calling_format.md` and `GOTCHAS.md`** — read this first if
+> you're reaching for those documents to debug a tool-calling problem.
+>
+> Last updated: 2026-05-25. Based on Ollama + llama.cpp + vLLM issue tracker
+> review, two academic papers from late 2025, and convergent empirical
+> evidence from two independent production agents running `gemma4:26b`.
+
+## TL;DR
+
+1. **The April-2026 rule "`think: false` silently kills `gemma4:26b` in
+   multi-turn tool-call loops" is wrong as written.** The bug is parser-side
+   in Ollama, not model-side. It triggers on the **combination** `system
+   prompt + think:false + tools` — and affects `gemma4:e4b` too, not just
+   26B MoE. ([ollama/ollama#15539](https://github.com/ollama/ollama/issues/15539))
+2. **Ollama has shipped material tool-calling fixes since 0.20.4.** Current
+   stable is **v0.24.0** (2026-05-14). v0.20.6 explicitly improved Gemma 4
+   tool calling and parallel-streaming tool calls; v0.22.1 updated the Gemma 4
+   renderer for thinking and tool calling. If you're below 0.22.1 you're
+   running on patched-but-incomplete state.
+3. **The dominant production failure mode is not "model can't call tools" —
+   it's "model decides not to call a tool when it should."** This is a
+   research-characterized problem now, and the best fix in the literature
+   is **NOT prompt engineering** — it's a lightweight probe on the model's
+   own hidden states (see § Probe & Prefill).
+4. **`think` is not a single boolean rule.** Setting it correctly requires
+   reasoning about `num_predict` budget, tool-argument length, and prompt
+   complexity together. The recipe is at the end.
+5. **No community fine-tune of Gemma 4 fixes tool-calling reliability** as
+   of late May 2026. If you need a robust local agentic small model today,
+   the field still recommends Qwen3-Coder. Gemma 4 is workable but requires
+   the harness work documented below.
+
+---
+
+## 1. Ollama: what's been fixed, what's still broken
+
+Release cadence April–May 2026: 0.20.4 → 0.20.5 → 0.20.6 → 0.21.0 → 0.21.3 →
+0.22.0 → 0.22.1 → 0.23.x → 0.24.0. (0.30.0 is a pre-release reorganizing
+llama.cpp consumption and adding MLX on Apple Silicon.)
+
+### Fixed
+
+- **v0.20.6**: "Gemma 4 tool calling ability is improved and updated to use
+  Google's latest post-launch fixes" + "improved parallel tool calling for
+  streaming responses." First real fix for the streaming-drops-tool-calls
+  bug noted in the April corpus.
+  ([release notes](https://github.com/ollama/ollama/releases/tag/v0.20.6))
+- **v0.22.1**: "Updated the Gemma 4 renderer for thinking and tool calling
+  improvements."
+  ([release notes](https://github.com/ollama/ollama/releases/tag/v0.22.1))
+- **PR #15467 (closed via merge)**: Added native tool-call parsers for
+  `gemma4`, `qwen3`, `qwen3-coder`, `cogito`, `deepseek3`, `functiongemma`,
+  `lfm2`, `ministral`, `olmo3`, `qwen3vl`. Gemma 4 is now in the
+  "registered parser" list.
+
+### Still open (verify before assuming a bug is gone)
+
+| Issue | Status | What's broken | Workaround |
+|-------|--------|---------------|------------|
+| [#15539](https://github.com/ollama/ollama/issues/15539) | open, assigned | `system + think:false + tools` produces raw JSON in `content` with trailing `<channel\|>` token and empty `tool_calls`. Affects 26B AND e4b. | Set `think: True`, or remove the system prompt, or move to llama.cpp from source |
+| [#15719](https://github.com/ollama/ollama/issues/15719) | open | `gemma4:26b` infinite tool-call loop on 0.20.6/0.20.7 **only when called via LiteLLM proxy** — direct Ollama works, 0.20.5 works | Skip the proxy, or pin Ollama version |
+| [#15497](https://github.com/ollama/ollama/issues/15497) | open | OpenAI-compat streaming returns `Function.Index: 0` for every tool call when model has no registered parser | n/a now for Gemma 4 (parser registered); affects custom models |
+| [#15315](https://github.com/ollama/ollama/issues/15315) | reopened | Raw `call:tool{...}` text leaks out on e4b when arguments contain JS-style quotes/backticks | Strip / replace special chars in tool args before display |
+
+### Re-reading the April baseline against this
+
+The April corpus said the `think: false` failure was a 26B-MoE-specific
+defect — the model emits near-immediate EOS at decision turns. **Issue
+#15539's diagnosis suggests this was actually the Ollama parser failing
+to extract a tool call from output that, on the wire, contained one.** The
+model emitted the call; the framework returned `tool_calls=[]` and an
+empty `content` with a stray `<channel|>` token.
+
+If you have a `think: false` 26B harness that's failing, the right next
+diagnostic is to **remove the system prompt and re-test** — if it fires
+reliably, you've reproduced #15539. The fix is either upgrade Ollama past
+0.22.1 and re-verify, or move to llama.cpp built from source with the PRs
+in § 2 applied.
+
+---
+
+## 2. llama.cpp: what's been fixed
+
+### Resolved
+
+- **The `<unused49>` / channel-token loop across CUDA, ROCm, Vulkan, SYCL**
+  was a llama.cpp eval bug, not quantization. Fixed in b8691+. Old quants
+  work fine on patched builds.
+  ([unsloth/gemma-4-26B-A4B-it-GGUF#2](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/discussions/2))
+- **Special tokens leaking into tool-call argument values**
+  (e.g., `[<|"|>light<|"|>]`) — closed via fix in early April.
+  ([ggml-org/llama.cpp#21316](https://github.com/ggml-org/llama.cpp/issues/21316))
+- **Canonical "build from source for Gemma 4 tool calling" PR pair**:
+  [#21326](https://github.com/ggml-org/llama.cpp/pull/21326) (template)
+  + [#21343](https://github.com/ggml-org/llama.cpp/pull/21343) (tokenizer).
+  This combination is cited by multiple writeups as the working baseline.
+
+### Still open or unclear
+
+- **[#22080](https://github.com/ggml-org/llama.cpp/issues/22080)**:
+  `gemma-4-26B-A4B` "generation stops abruptly mid-output" on multi-GPU
+  llama-server with `--reasoning-budget` set. No root cause. May be the
+  same family as Ollama #15539 surfacing in a different runtime.
+- **31B + f16/f32 vision projector** still produces `<unused49>` floods on
+  current builds per one report — the eval-bug fix didn't cover the
+  multimodal projector path.
+- **Inverted thinking-mode guard** in Google's released chat template
+  (`models/templates/google-gemma-4-31B-it.jinja`): when
+  `enable_thinking=false`, the upstream template emits a closed empty
+  thought block; when true, it fails to open `<|channel>thought\n`.
+  Local-fork fix only as of writing.
+  ([CompleteTech writeup](https://complete.tech/blog/llamacpp-gemma4-thinking-prompt-local-fix/))
+
+`--jinja` is still required. The chat template is the contract.
+
+---
+
+## 3. vLLM: usable, with sharp edges
+
+vLLM ships an official Gemma 4 recipe and a `--tool-call-parser gemma4`
+flag. One open caveat:
+
+- **[#39392](https://github.com/vllm-project/vllm/issues/39392)**:
+  `<pad>` token floods under concurrent requests (2/5 fail at concurrency
+  5; 0/5 sequential). Reporter suspects shared mutable parser state —
+  not thread-safe. Workaround is a global serialization lock around the
+  tool-call path.
+
+vLLM also still suffers the heterogeneous-attention-head Triton fallback
+(LOW-severity entry in GOTCHAS.md) — throughput is well below what
+similarly-sized models get on the same hardware.
+
+---
+
+## 4. The `think` flag, properly characterized
+
+The April corpus had two contradictory entries: "always `think: false`
+for single-turn JSON" and "never `think: false` for multi-turn tool loops
+because it kills 26B." Both pointed at the same symptom (empty content,
+no usable response) with different prescriptions. Field evidence from two
+independent agents running `gemma4:26b` makes the actual interaction
+clearer.
+
+### The two failure paths
+
+**Path A — `think: True` + `num_predict` too small.**
+The hidden reasoning trace consumes the entire generation budget, leaving
+`message.content=""` and `done_reason="length"`. Observed against
+`num_predict=1024` with a complex system prompt + tools schema; thinking
+trace alone was reaching the cap.
+
+**Path B — `think: False` + (long tool args OR multi-turn agentic loop
+OR system prompt present).** Model emits a tool call on the wire that the
+parser fails to extract, or emits near-immediate EOS at a decision turn.
+Surface symptom: `tool_calls=[]`, empty content, sometimes a stray
+`<channel|>` token. Per Ollama #15539 this is parser-side; per the April
+bakeoff it also shows up in deep multi-turn loops where each decision
+turn is short.
+
+### The recipe
+
+| Loop shape | `think` | `num_predict` | Why |
+|------------|---------|---------------|-----|
+| Single-turn structured JSON | `False` | ≥ 2048 | No tool-call parsing involved; thinking adds latency without benefit; small budgets are safe |
+| Short tool args, ≤ 5-step loop | `False` | ≥ 1024 | Field-verified to fire reliably at temp ≤ 0.5 with explicit Trigger→Action prompt structure |
+| Long tool args (image-gen prompts, file payloads, code blocks) | `True` | ≥ 2048 | Field-verified: `think: False` produces empty `tool_calls` on the same context where `think: True` produces a proper call |
+| Deep multi-turn agentic (≥ 10 steps) | `True` | ≥ 2048 | Per April bakeoff: `think: False` produces silent EOS at decision turns |
+
+**Safety floor:** if you can afford the latency, `think: True` +
+`num_predict: 2048` works for *all* shapes above. The reason to deviate
+is one of:
+- Latency budget < ~3s per turn (thinking adds 1–2s)
+- Predictability is critical (no hidden token generation)
+- Schema-constrained output that excludes thinking tokens
+
+**Build a probe.** Both agents that diagnosed this in production wrote a
+probe script that called the same Ollama endpoint with each flag setting
+against a known-good message state and asserted `tool_calls != []`. If
+you're flipping `think`, write the probe first — same-context comparisons
+are how the parser-side failure surfaces.
+
+### Reproducing the parser-side diagnosis
+
+To verify whether your harness is hitting Ollama #15539 vs a different
+failure path:
+
+```python
+# Same model state, four conditions
+combos = [
+    {"system": None,  "think": True},
+    {"system": None,  "think": False},
+    {"system": "...", "think": True},
+    {"system": "...", "think": False},
+]
+for c in combos:
+    response = ollama.chat(model="gemma4:26b", messages=msgs(c), tools=TOOLS,
+                           think=c["think"])
+    print(c, "→ tool_calls:", len(response["message"]["tool_calls"]))
+```
+
+If only `system + think:False` produces zero tool calls, you've reproduced
+the parser bug, not the model defect.
+
+---
+
+## 5. The "doesn't fire tools without explicit ask" pattern
+
+This is the dominant production failure mode for small/mid agentic models
+as of May 2026 — the model is capable of using tools, calls them when
+explicitly told ("look this up", "search for"), and defaults to
+answering-from-training otherwise. Now characterized in the literature:
+
+### Quantified
+
+- **["To Call or Not to Call"](https://arxiv.org/html/2605.00737v1)**
+  measures under/over-calling across model families. **Gemma3-27B over-calls
+  (462 vs 297 optimal)**; Mistral3.1-24B severely under-calls. Gemma 4 not
+  directly benched but the family signal is toward over-calling under
+  reasoning-mode conditions. The "doesn't fire" observation in production
+  likely reflects `think: false` runs where the model goes straight to
+  answer without the reasoning step that would surface the tool-call
+  decision.
+- **[BiasBusters](https://arxiv.org/pdf/2510.00307)** quantifies
+  tool-selection bias driven by description metadata and position. Tools
+  with **edited** descriptions get **10× more usage** than identical tools
+  with original descriptions (GPT-4.1, Qwen2.5-7B). The wording of your
+  tool descriptions is itself tuning firing rate; this is not a marginal
+  effect.
+
+### What actually moves the needle
+
+**Probe & Prefill** ([arxiv 2605.09252](https://arxiv.org/html/2605.09252v1))
+— the most concrete fix in the recent literature, and the one most likely
+to apply to a Gemma-4-sized agent.
+
+Core finding: **tool-call decisions are linearly decodable from the
+model's pre-generation hidden states with AUROC 0.89–0.96** —
+substantially outperforming the model's own verbalized reasoning about
+whether to call a tool. The technique:
+
+1. Collect ~900 examples labeled "should call tool" vs "should not."
+2. Train a logistic regression on the final-layer hidden state at the
+   pre-generation position. Trains in seconds on CPU. < 1 ms inference
+   overhead.
+3. At inference, if the probe says "should call," prefill a short
+   steering sentence ("I'll use a tool for this. ") to bias decoding
+   toward the tool path.
+
+Demonstrated **48% reduction in unnecessary tool calls with 1.7% accuracy
+loss** on Qwen3 (1.7/4/14/32B) and Llama (3.1-8B, 3.3-70B). Not tested
+on Gemma 4 — but the technique is model-agnostic and the hidden-state
+signal has held across every architecture they tried.
+
+**This is the only technique in the May 2026 literature that beats
+careful prompt engineering on this specific problem.** Worth a serious
+experiment for any non-trivial Gemma 4 agent.
+
+### Prompt-engineering techniques that help (but won't fully solve)
+
+- **Imperative tool descriptions with activation triggers** beat passive
+  descriptions. "Call this when the user asks about current events"
+  fires more reliably than "Search the web for current information."
+  (BiasBusters' 10× finding is the upper bound; expect smaller effects
+  in practice.)
+- **Trigger → Action contracts in the system prompt** ("If the user
+  attempts a translation → call `score_vocab`") outperform softer
+  guidance. Field-verified to take a Gemma 4 26B agent from ~0%
+  tool-firing at default temperature to ~80% at temp 0.4.
+- **Lower temperature** (0.3–0.5) increases firing rate on agentic
+  loops. Default Ollama temperature (~0.8) biases toward
+  conversational mimicry of recent chat history.
+- **Reactive-mode nudge loop**: if the model emits text without calling
+  a tool on a turn where tools are available, append a system message
+  ("Was a tool needed? If yes, call it.") and re-prompt. One agent in
+  the field uses this in its autonomous-research loop but not its
+  reactive chat loop; the reactive loop has the firing-rate problem
+  the autonomous loop doesn't.
+
+### Benchmarking your fix
+
+[NVIDIA's When2Call](https://github.com/NVIDIA/When2Call) (NAACL 2025) is
+the relevant eval: 1000 MCQs scoring "answer directly / call tool / ask
+clarifying / decline." Their finding: **RPO consistently > SFT** for
+training models to make this decision well. Useful as the harness even
+if you're not training, because it gives you a single number to track
+across prompt / parameter changes.
+
+---
+
+## 6. Two-stage tool routing — still alive and well
+
+In-model selection has **not** caught up to two-stage routing for the
+"should-call" decision as of May 2026. The router pattern has shifted
+from "another LLM" to "small classifier":
+
+- **[ToolCallVerifier](https://huggingface.co/llm-semantic-router/toolcall-verifier)**:
+  ModernBERT-based 0.1B token classifier. Recall 0.92, precision 0.95,
+  F1 0.935. Designed for prompt-injection defense but the same pattern
+  works as a "should-call" gate.
+- **vLLM Semantic Router (v0.1 Iris, Jan 2026)**: production framework
+  for taxonomy-guided routing + confidence-cascade.
+- **Probe & Prefill** (above) is the limit case: the "router" is a
+  logistic regression on the *main model's own* hidden states. No
+  separate model to host.
+
+**Practical recommendation:** if you're under 5 tools and can't afford
+to train a probe, Trigger → Action prompting + temp 0.4 + reactive nudge
+loop will get you to ~70–80% firing rate. Beyond that, the probe technique
+is the lowest-cost concrete improvement.
+
+---
+
+## 7. Structured output: XGrammar-2 changes the math
+
+**[XGrammar-2](https://blog.mlc.ai/2026/05/04/xgrammar-2-fast-customizable-structured-generation)**
+(MLC blog 2026-05-04, [arxiv 2601.04426](https://arxiv.org/abs/2601.04426))
+is the headline structured-output development of the year:
+
+- New "Structural Tag" composable JSON protocol uniformly expressing
+  OpenAI Harmony, tool calling, reasoning channels, and custom output
+  structures
+- Cross-grammar caching with ~50% structure reuse on 50-tool schemas
+- Repetition state compression: 534ms → 5.37ms (100×)
+- Batching + speculative-decoding support
+- **Default structured-generation backend for vLLM, SGLang, TensorRT-LLM,
+  MLC-LLM** as of March 2026
+
+The per-token overhead is now low enough for production. Smaller models
+showed "substantial gains in output accuracy" on BFCL-V3.
+
+**Important distinction:** structured output forces the **format** once
+you've decided to call a tool. It does **not** solve "decide whether to
+call a tool." For that, see § 5.
+
+Ollama's native `format` schema field (not the broken `format: "json"`)
+remains the right Ollama-native option; community reports it's more
+reliable than free-form JSON prompting but still leaks for some model
+families. No Gemma-4-specific reliability data yet.
+
+---
+
+## 8. New Gemma 4 variants — slim pickings
+
+No community fine-tune released to date materially improves tool-calling
+reliability over base Gemma 4 26B-it. Notable releases since April:
+
+| Release | What | Tool-calling relevance |
+|---------|------|------------------------|
+| [`google/gemma-4-26B-A4B-it-assistant`](https://huggingface.co/google/gemma-4-26B-A4B-it-assistant) | 0.4B drafter for speculative decoding | Generation speed only — not a tool-calling tune |
+| `nvidia/Gemma-4-26B-A4B-NVFP4` | NVFP4 quant for Blackwell | Hardware target, not a tune |
+| [`Jackrong/Gemopus-4-26B-A4B-it-GGUF`](https://huggingface.co/Jackrong/Gemopus-4-26B-A4B-it-GGUF) | SFT for style consistency / Markdown structure | **Explicitly disclaims tool calling**: "known compatibility issues, not unique to this model" |
+| HF blog / TRL fine-tuning recipe | Fine-tune `gemma-4-E2B-it` for tool calling on H100 | Recipe, not a released model |
+| Unsloth GGUFs | All sizes, plus fine-tuning guide | No agentic tune |
+
+If you need a robust local agentic small model **today**, the field
+still recommends **Qwen3-Coder** family. Gemma 4 is workable for agentic
+use with the harness work above, but it's not the path of least
+resistance for that workload.
+
+---
+
+## 9. What this means for an existing harness
+
+A concrete checklist if you have a Gemma 4 26B agent in production and
+are debugging tool-call reliability:
+
+1. **Check your Ollama version.** Below 0.22.1, upgrade. Below 0.20.6,
+   you're missing the first round of Gemma 4 tool-call fixes entirely.
+2. **Write the four-condition probe** from § 4 against a known-good
+   message state. If only `system + think:False` zeros out
+   `tool_calls`, you have #15539, not a model defect.
+3. **Set the recipe** from § 4 by loop shape. The safe combo for almost
+   all cases is `think: True` + `num_predict: 2048`.
+4. **Audit your tool descriptions** for activation verbs. "Call this
+   when…" beats "Search for…" by a non-marginal margin per BiasBusters.
+5. **Lower temperature to 0.4** if you're at default. This alone took
+   one production agent from ~0% to ~80% firing on triggers.
+6. **Add a reactive-mode nudge loop**: if `tool_calls=[]` on a turn
+   where the user asked something factual, re-prompt with a system
+   message asking whether a tool was needed.
+7. **If still under-firing**, prototype Probe & Prefill on ~900 labeled
+   turns from your own log. This is the lowest-effort high-impact
+   technique in the current literature.
+8. **Consider Qwen3-Coder for the tool-heavy path** if you're not
+   committed to Gemma 4 for other reasons (vision, multilingual,
+   licensing).
+
+## Contradictions with prior corpus, flagged
+
+- `GOTCHAS.md` HIGH: "`think: false` Kills Gemma 4 26B in Multi-Turn
+  Tool-Calling Loops" — **partially superseded**. The failure is real
+  but is parser-side in Ollama (#15539), not model-side. Same fix
+  (use `think: True`) but the root cause framing was wrong, and the
+  failure also affects e4b, not just 26B.
+- `SYNTHESIS.md` "Mandatory Ollama Settings": the single-turn
+  "always `think: false`" rule is still correct *for single-turn JSON
+  pipelines*. The multi-turn agent guidance needs the recipe from § 4
+  applied.
+- `GOTCHAS.md` MEDIUM: "Tool Calling Broken in Ollama v0.20.0 Streaming"
+  — partially fixed (v0.20.6 + v0.22.1). Re-test on current version
+  before assuming streaming tool calls are still broken.
+- `GOTCHAS.md` LOW: "`<unused>` Token Infinite Loop (Vulkan backends)"
+  — fixed in llama.cpp b8691+.
+
+## Sources
+
+Primary:
+- [ollama/ollama#15539](https://github.com/ollama/ollama/issues/15539)
+- [ollama/ollama releases](https://github.com/ollama/ollama/releases)
+- [ggml-org/llama.cpp#21316](https://github.com/ggml-org/llama.cpp/issues/21316),
+  [#21321](https://github.com/ggml-org/llama.cpp/issues/21321),
+  [#22080](https://github.com/ggml-org/llama.cpp/issues/22080)
+- [vllm-project/vllm#39392](https://github.com/vllm-project/vllm/issues/39392)
+
+Research:
+- ["LLM Agents Already Know When to Call Tools" — Probe & Prefill](https://arxiv.org/html/2605.09252v1)
+- ["To Call or Not to Call"](https://arxiv.org/html/2605.00737v1)
+- [BiasBusters](https://arxiv.org/pdf/2510.00307)
+- [NVIDIA When2Call](https://github.com/NVIDIA/When2Call)
+- [XGrammar-2 paper](https://arxiv.org/abs/2601.04426),
+  [MLC blog](https://blog.mlc.ai/2026/05/04/xgrammar-2-fast-customizable-structured-generation)
+
+Community:
+- [Tim Gregg: llama.cpp Gemma 4 thinking-prompt fix](https://complete.tech/blog/llamacpp-gemma4-thinking-prompt-local-fix/)
+- [unsloth/gemma-4-26B-A4B-it-GGUF discussion #2 (unused49 root cause)](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/discussions/2)
+- [google/gemma-4-26B-A4B-it discussion #10 (thinking-mode behavior)](https://huggingface.co/google/gemma-4-26B-A4B-it/discussions/10)

GOTCHAS.md

@@ -2,6 +2,13 @@
 
 > Derived from Seth's production implementations (Simon, AI_Visualizer)
 > and community reports. These are hard-won lessons.
+>
+> **2026-05-25 corrections:** several entries below were corrected by
+> field evidence from two independent agents and by Ollama/llama.cpp
+> issue-tracker review. See `CORPUS_tool_calling_2026-05.md` for the
+> consolidated current state. The "`think: false` kills 26B" rule and
+> the `<unused>` Vulkan-loop entry both need re-reading against that
+> update — they are marked **SUPERSEDED** in place below.
 
 ## CRITICAL: Thinking Mode Eats Context (single-turn pipelines only)
 
@@ -75,10 +82,32 @@ Ollama defaults `num_predict` to 128 tokens. Almost any useful Gemma 4 output ex
 
 **Fix:** Always set `num_predict` explicitly. Minimum recommended: 512. For JSON output: 2048+.
 
-## HIGH: `think: false` Kills Gemma 4 26B in Multi-Turn Tool-Calling Loops
+## HIGH: `think: false` Kills Gemma 4 26B in Multi-Turn Tool-Calling Loops [SUPERSEDED 2026-05-25]
 
 **Severity: HIGH — silent agent-loop failure. Setting is what the old guidance said to do.**
 
+> **SUPERSEDED 2026-05-25:** the symptom is real but the diagnosis was
+> wrong. Per [ollama/ollama#15539](https://github.com/ollama/ollama/issues/15539),
+> the failure is an **Ollama parser bug** triggered by the combination
+> `system prompt + think:false + tools` — the model emits a valid tool
+> call on the wire, the framework returns `tool_calls=[]` with an empty
+> `content` and a stray `<channel|>` token. **The bug affects `gemma4:e4b`
+> too, not just 26B MoE.** Field evidence from a production agent at
+> `num_predict=1024` confirms `think: false` works fine for short tool
+> arguments and shallow loops — the original bakeoff hit the deep-loop
+> case where the parser fails.
+>
+> **Replacement rule:** see `CORPUS_tool_calling_2026-05.md` § 4 for the
+> proper recipe by loop shape. Safe default for all cases:
+> `think: True` + `num_predict ≥ 2048`. Diagnostic for whether you're
+> hitting this bug: run a four-condition probe (system × think) at the
+> same model state — if only `system + think:False` zeros tool_calls,
+> you've reproduced #15539.
+>
+> The April-2026 content below is preserved for the historical record but
+> should not be applied without first reading the May update.
+
+
 Reproduced on 2026-04-18 against `gemma4:26b` via Ollama 0.20.4 on a 3090 Ti
 (steel141). Contradicts the older "always think:false" guidance (see § "Thinking
 Mode Eats Context" below — now scoped to single-turn pipelines only).
@@ -178,9 +207,9 @@ Community-reported: Flash Attention causes Gemma 4 31B Dense to hang indefinitel
 
 **Fix:** Use 26B for long prompts, or disable Flash Attention if running 31B on affected hardware.
 
-## MEDIUM: Tool Calling Broken in Ollama v0.20.0 Streaming
+## MEDIUM: Tool Calling Broken in Ollama v0.20.0 Streaming [PARTIALLY FIXED]
 
-**Severity: MEDIUM — version-specific**
+**Severity: MEDIUM — version-specific. PARTIALLY FIXED in v0.20.6 + v0.22.1.**
 
 As of early April 2026, Gemma 4 tool calling has issues in Ollama v0.20.0: the tool call parser fails and streaming drops tool calls entirely. Community reports include format mismatches and continuous loops in llama.cpp / LM Studio.
 
@@ -188,6 +217,17 @@ As of early April 2026, Gemma 4 tool calling has issues in Ollama v0.20.0: the t
 
 **Fix:** Use non-streaming for tool calls (Simon does this). Test tool calling thoroughly when upgrading Ollama versions. Seth's implementations work reliably with non-streaming tool calls.
 
+> **2026-05-25 update:** Ollama [v0.20.6](https://github.com/ollama/ollama/releases/tag/v0.20.6)
+> explicitly improved Gemma 4 tool-calling and added "improved parallel
+> tool calling for streaming responses." [v0.22.1](https://github.com/ollama/ollama/releases/tag/v0.22.1)
+> updated the Gemma 4 renderer for thinking + tool-calling improvements.
+> Current stable is **v0.24.0** (2026-05-14). Re-test streaming on current
+> versions before assuming the original bug persists. **Open bugs**
+> remain — see `CORPUS_tool_calling_2026-05.md` § 1 for the current list,
+> especially [#15539](https://github.com/ollama/ollama/issues/15539)
+> (`system + think:false + tools` returns empty tool_calls) which is the
+> most likely failure mode you'll hit on a recent Ollama.
+
 ## MEDIUM: VRAM-Hungry for Context
 
 **Severity: MEDIUM — affects hardware planning**
@@ -222,14 +262,24 @@ Heterogeneous attention head dimensions in Gemma 4 force vLLM to fall back to a
 
 **Fix:** Use Ollama instead of vLLM for now, or wait for the fix.
 
-## LOW: `<unused>` Token Infinite Loop (Vulkan backends)
+## LOW: `<unused>` Token Infinite Loop (Vulkan backends) [FIXED]
 
-**Severity: LOW — Vulkan-specific**
+**Severity: LOW — Vulkan-specific. FIXED in llama.cpp b8691+.**
 
 Gemma 4 can generate `<unused>` or `<unused24>` tokens in an infinite loop on Vulkan backends in llama.cpp.
 
 **Source:** [ggml-org/llama.cpp#21516](https://github.com/ggml-org/llama.cpp/issues/21516)
 
+> **2026-05-25:** Per the
+> [unsloth/gemma-4-26B-A4B-it-GGUF discussion thread](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF/discussions/2),
+> the `<unused49>` / channel-token flood across CUDA / ROCm / Vulkan / SYCL
+> was a llama.cpp eval bug, **not** a quantization problem. Fixed in
+> b8691+. Old quants work fine on patched builds.
+>
+> Caveat: 31B + f16/f32 vision projector still produces `<unused49>`
+> floods on current builds per one report — the eval-bug fix didn't
+> cover the multimodal projector path.
+
 ## MEDIUM: `google/gemma_pytorch` Abandoned for Gemma 4
 
 **Severity: MEDIUM — wastes time on a dead-end path**

README.md

@@ -14,6 +14,8 @@ Research corpus and implementation guidance for Google Gemma 4, based on product
 | `CORPUS_capabilities.md` | Modalities (vision, audio, video, tools), what it can/can't do | When scoping what Gemma 4 can handle |
 | `CORPUS_benchmarks.md` | Full benchmark table vs Gemma 3, arena scores, agentic scores | When comparing Gemma 4 to alternatives |
 | `CORPUS_tool_calling_format.md` | Native token format + JSON API format for function calling | When implementing tool calling |
+| `CORPUS_tool_calling_2026-05.md` | **May 2026 update — read first for tool-call debugging.** Ollama parser fixes (#15539), `think` flag properly characterized, the "doesn't fire tools without explicit ask" research (Probe & Prefill, BiasBusters, When2Call), XGrammar-2. Supersedes parts of `GOTCHAS.md` (think:false rule, Vulkan loop, streaming bug all updated in-place with pointers here) | When debugging tool-call reliability, deciding `think` value, or planning a router/probe layer |
+| `CORPUS_agentic_retrieval.md` | Post-RAG landscape — hybrid retrieval + cross-encoder rerank as the settled base, CRAG-shaped flows in LangGraph/LlamaIndex, GraphRAG (LightRAG/LazyGraphRAG), memory architectures (mem0 vs Letta MemFS), deep-research agents (gpt-researcher, smolagents), production-vs-experimental sorting | When stepping up from basic embedding RAG to agentic retrieval, picking a memory layer, or scoping a deep-research agent |
 | `CORPUS_cli_coding_agent.md` | Positioning Gemma 4 for CLI coding agent use (openclaw / open code / pi / hermes / aider style). Honest take on what Google did and didn't measure, head-to-head with `qwen3-coder:30b`, homelab setup pointer | When scoping a CLI coding agent or deciding Gemma 4 vs Qwen3-Coder |
 | `docs/openwebui-setup.md` | How to configure Gemma 4 inside OpenWebUI — per-setting reference, two ready-to-bake Workspace Model profiles (chat + extract), and a symptom→cause troubleshooting table mapped back to GOTCHAS.md. Assumes Ollama + OpenWebUI are already running. | When setting up or debugging a Gemma 4 model in OpenWebUI, or handing the front-end config to someone else |
 | `docs/reference/bakeoff-2026-04-18.md` | CLI-coding-agent bakeoff on 3090 Ti. **Rounds 1/2 misidentified the cause; Round 3 (the correct one): `think: false` silent-stops gemma4:26b at certain multi-turn states on 32K context.** 31B and Qwen3-Coder robust to the flag. Harness at `scripts/bakeoff/` | When deciding which model to back a CLI agent with, writing a custom agent payload, or debugging a silent tool-call halt |