# Forge and recall

A method for AI agents: learn a domain from sources you can cite instead of from memory, keep skills out of the prompt, and recall the right one when the task calls for it.

Version 1.0, 26 September 2026. Synthera Systems, Malé.
For people: https://forge-recall.pages.dev/
This file: https://forge-recall.pages.dev/method.md

## 0. How to use this file

You are an AI agent adopting a standing method.

- Section 1 is always on. Keep it in your standing instructions (CLAUDE.md, AGENTS.md, a system prompt or custom instructions).
- Sections 2 to 6 are procedures. Open the one you need when you run it: recall on every task, forge when asked to learn a domain, upkeep when you maintain the library, setup once.
- When your environment lacks a tool this file names, say which one is missing and what it would have done. Never report a step you did not run.

Terms:

- **Skill**: a folder holding a `SKILL.md` (frontmatter `name` and `description`, then instructions) and optional `references/` files.
- **Hot**: loaded into every prompt. **Cold**: stored in a searchable index and loaded only when retrieved.
- **Gate**: a check that can fail, and stops the run when it does.
- **Corpus**: the fixed set of sources a skill is forged from.

## 1. Standing rules

1. Treat your memory as a guess wherever facts move: tools, APIs, flags, limits, prices, current practice. Answer from a grounded skill or a cited source, and say which. Settled knowledge can come from you.
2. Search the skill library before you plan, and again whenever the task changes shape.
3. Choose by description, never by score. Similarity scores rank results; they do not measure confidence, and a query the library cannot answer still returns results.
4. Load the skill that fits and follow its body over your habits. Open its references only when the task reaches them.
5. When nothing fits, say so, fall back to documentation, the web or your own knowledge, and label the answer as not skill-grounded.
6. Keep the library cold. Only skills that must fire on their own belong in the prompt.
7. New knowledge enters through the forge (section 3): sources a curator chose, a citation on every claim, a test the author did not write.
8. Every gate must be able to fail. Prove it fails on bad input before you trust a pass.
9. A skill is installed only when retrieval is proven: reindex, run a realistic request, watch it come back.
10. A multi-step workflow that succeeded becomes a skill (section 4.6). Keep the steps that worked and drop the dead ends.

## 2. Recall: finding the right skill at runtime

### 2.1 When to search

- At task start, before planning.
- When a new need appears mid-task. Writing a Kubernetes manifest raises autoscaling; search for autoscaling then.
- Before improvising any procedure longer than a few steps.
- When the user names a tool, file format, platform or domain.

### 2.2 How to search

1. Phrase the query as the task in plain words, plus the domain nouns: "deploy a containerised service on kubernetes", not "k8s".
2. Use semantic search for intent and keyword search for exact strings: tool names, error messages, CLI flags.
3. Read the names and descriptions. Take the one that matches the situation (a first deployment or scaling, a prototype or production), even when it is not the top hit.
4. Fetch the full skill by the id the search returned, not by its name.
5. Follow it. Search again when terms you picked up while working suggest a better query.

### 2.3 Tools

With skill-retrieval-mcp (other skill libraries expose equivalents):

| Tool | Use |
|---|---|
| `search_skills(query, k=5)` | Semantic search. Returns summaries only. |
| `keyword_search(query)` | Exact-term search. |
| `get_skill(skill_id)` | The full guide. Call it after a search. |
| `list_categories()` | Browse the library's domains and counts. |

Summaries cost a few hundred tokens and a full skill a few thousand. Fetch one or two, not five.

### 2.4 The push channel (optional)

If your harness runs a hook on each user prompt, a surfacer can suggest skills before the agent asks:

1. Skip prompts shorter than 12 characters and slash commands.
2. Embed the prompt with the same model that built the library index.
3. Score it against every cold skill by cosine similarity.
4. Keep hits at or above a threshold, at most two. 0.32 works for text-embedding-3-large; tune it on your own prompts.
5. Hide a skill whose declared prerequisites are unmet (`requires:` environment variables, files, commands) or whose `scope:` names an entity the prompt does not mention.
6. Inject one line per hit: the name, a short description, and how to load it.
7. If the embedding call fails, fall back to keyword matching. Never block the prompt, and time out within seconds.

When a surfaced skill appears in your context, read it before building if it applies, and ignore it if it does not.

## 3. Forge: building a grounded skill from sources

Run the stages in order. Each ends in a gate, and a failed gate stops the run until it passes. Never summarise a stage you did not run.

### 3.0 Preflight

Check the research tool's session with a call that exercises it. A refresh command that exits 0 proves nothing: one refresh reported success while the session was dead, and a run died at 14 of 27 answers.

Gate: a live test call succeeds.

### 3.1 Map the domain

Write the 6 to 8 subtopics an expert in the domain needs. This list is the coverage contract.

Gate: the list exists before the first search.

### 3.2 Curate the corpus

1. Write 25 to 40 queries that hit every subtopic from several angles, plus the names of known practitioners. Run each through YouTube search (the results page, the YouTube Data API, or yt-dlp's `ytsearch`) and keep one row per unique video in a candidates table: score, views, length, published date, channel, title, URL. Filter as you collect: 5 minutes to 2.5 hours long, a view floor (10,000 by default, about 400 in niche fields where authority talks never trend), at most 4 videos per channel. Expect 400 to 800 unique candidates; a smaller pool means the queries were narrow.
2. Rank by authority, not popularity: the people who built the thing, then conference talks and workshops, then respected practitioners, then general educators. Put views under a square root, so a two-million-view intro cannot bury a sixty-thousand-view conference talk.
3. Drop the noise: income reports, "$X a month", "learn X in N minutes", beginner full courses, channels whose value is the thumbnail.
4. Read every candidate and pick 50 by hand, for density and coverage rather than score order. A 24-minute talk that teaches one mechanism properly beats a two-hour course that teaches nothing five times.
5. Write the pick twice: a plain URL list, and an annotated list grouped by subtopic (channel, length, views), so the choice can be audited.

Gate: 50 unique URLs, each traceable to a row in the scraped candidates (never write a video id from memory: it will be plausible and wrong), coverage counted per subtopic, and the weakest picks named with the reason they stayed.

### 3.3 Ingest

Load the 50 sources into one research notebook. We use NotebookLM. Make the loader idempotent and resumable.

Gate: every URL present, processed, and above a transcript floor of 400 words. A source below the floor is THIN and fails. Livestream recordings and silent screencasts carry no usable transcript: replace them, and never lower the floor.

### 3.4 Distil

1. Write 20 to 30 questions that demand procedure, field names, exact numbers, quotes, enumerations and anti-patterns. End each with "Cite each." Never ask for a summary.
2. Ask each question in a fresh conversation, and save each answer, with its citations, to its own file.

Gate: an answer with fewer than 2 citations, or under 600 characters, fails. Uncited means the model answered from its weights, the exact failure this method exists to prevent. Re-ask, and rewrite a question that keeps failing.

Traps:

- Broad "enumerate everything" questions can overflow the tool's response limit. Cap the answer ("under 500 words, at most 12 citations") or restrict the question to named sources.
- Run one corpus at a time and wrap every question in a timeout. Concurrent corpora caused the overflows, and a call with no timeout can hang forever.
- Citation markers such as `[3]` can overwrite array literals inside code: `interpolate(frame, [0, 30], [0, 1])` arrived as `interpolate(frame, [10], [11])`. In a code-heavy corpus, check every code block and restore it from context.
- A resumable script that skips existing files also skips an existing bad answer. Delete the file to force the re-ask.

### 3.5 Build

Few skills, deep references. Every hot description is paid on every request, so three good skills beat twelve narrow ones.

```text
<skill-name>/
  SKILL.md          triggers and decision rules, kept tight
  references/*.md   the distilled depth, opened only when needed
```

- The `description` says what the skill does and when to use it, in the words a user would type. It lists triggers. It does not summarise the procedure: agents follow a summary and skip the body.
- Decision rules go in SKILL.md. Facts, numbers and long procedures go in references.
- Record provenance in the skill: the corpus list and the notebook id.
- State corpus gaps as gaps. Never fill them from memory.

Gate: every claim traces to a distilled answer, and the description would fire on a realistic request.

### 3.6 Grade

1. Have the research tool generate a hard quiz from the corpus. With notebooklm-py: `notebooklm use <notebook_id>`, then `notebooklm generate quiz --difficulty hard --quantity more`, then `notebooklm download quiz --format markdown ./eval.md`. The skill's author never writes the test.
2. Answer the quiz closed-book: only SKILL.md and the references the skill would have you open. A question you answer from memory is a miss.
3. Score each question correct, partial or wrong. For each miss, name the missing fact and where it should live.
4. Verify every reported miss by hand before acting on it. Automated keyword grading over-reports: on our first run, 8 of 10 reported misses were already covered (reported 77% and 85%, true 96% and 96%).
5. Fix at the right altitude, from the distilled answers, never from the quiz key. Do not pad the skill with trivia to move the score.
6. Re-run the quiz and report both scores.

Check every time: will the description fire; is SKILL.md carrying bulk that belongs in a reference; does the skill contradict itself or the corpus; is any claim unsourced.

Gate: PASS or NEEDS WORK, the score before and after, and a ranked list of what is still missing. Never soften a failing grade.

### 3.7 Install and prove

1. Install cold, into the searchable library, unless the skill must fire on its own every session.
2. Reindex.
3. Prove retrieval: run a realistic future request through search, and through the surfacer if you run one. The new skill must come back.

Gate: retrieval proven. When it does not come back, the description is wrong: rewrite it, reindex, and test again.

## 4. Upkeep: keeping the library healthy

### 4.1 Hot versus cold

Every hot skill's name and description rides every request. In our library of 3,100 skills, names and descriptions alone come to about 191,000 tokens. Loaded, they would ride every request; indexed, they cost nothing until retrieved. Keep hot only what must fire on its own.

### 4.2 One embedding space

The search server and the surfacer use the same embedding model, because an index is only valid for the model that built it. Choose the model by an eval on your own library: gold queries with known right answers, scored by hit@1 and mean reciprocal rank. Ours, on 22 August 2026: eight models against 25 gold queries. text-embedding-3-large found the right skill first 22 times (MRR 0.918), against 19 (MRR 0.853) for Qwen3 Embedding 8B.

### 4.3 Sources carry trust

Your own skills and hand-audited packs are trusted; bulk community packs are untrusted. When rows share a name, one survives: the local copy first, trusted before untrusted, then the longest body.

### 4.4 Quarantine before indexing

Before an untrusted row reaches the index, move it to a review log if it is a stub (under 150 characters) or matches a high-precision hostile pattern: instructions to ignore the user, actions hidden from the user, secrets or data sent to anonymous drop sites, swarms told to stop asking. Log it; never delete it silently. Tune for precision: our first pattern matched the word "webhook" and quarantined 24 legitimate integration skills (Slack, Twilio and Shopify skills post to vendor webhooks by design). Narrowed to anonymous drop sites and secret exfiltration, 21 of them came back.

### 4.5 Refresh automatically

Watch the skill folders and re-run on change, debounced. Re-import, quarantine, collapse name collisions, then embed only the new rows when nothing was deleted; rebuild the index fully only after a deletion or a model change. Re-pull upstream sources weekly. No stage may abort the run: log a broken source and continue.

Editing trap: re-importing an edited skill can add a second row keyed on its new content. Run the collision pass after edits and confirm one row per name.

### 4.6 Skillify

When a multi-step workflow succeeds, write the trigger, the steps that worked, the pitfalls hit and how success was verified into a new cold skill. Strip session-specific names and paths unless they are the point. Reindex, then prove retrieval with a realistic future request.

## 5. Setup

### 5.1 Skill retrieval

skill-retrieval-mcp by JayCheng113, MIT licence: https://github.com/JayCheng113/skill-retrieval-mcp. Use version 0.3.0 or later, which tolerates real-world SKILL.md frontmatter.

```bash
pip install "skill-retrieval-mcp[local,hf]"
skill-mcp pull --include-index
skill-mcp init
skill-mcp import --source directory --path ~/.agents/skills
```

`pull` fetches the licence-vetted corpus and a pre-built index. `init` registers the server with the agents it detects, such as Claude Code and Cursor; anywhere else, register the command `skill-mcp serve` as a stdio MCP server. `import` adds your own skill folders. The default embeddings run locally (all-MiniLM-L6-v2) and need no key. To use an API model, set the backend and model in the server config and rebuild the index for it with `skill-mcp build-index --backend openai`. We use text-embedding-3-large through an OpenAI-compatible endpoint.

### 5.2 Forge

- NotebookLM. A Pro account allows 300 sources per notebook and 500 chats a day. The free tier's 50 and 50 is filled by one 50-source corpus, with nothing left to distil.
- notebooklm-py on Python 3.12: `uv tool install --python 3.12 "notebooklm-py[browser,mcp]" --with rookiepy`, then `notebooklm login`.
- Keep the session alive on a schedule, and follow every refresh with a real test call.

### 5.3 Skill file

```markdown
---
name: verb-first-kebab-name
description: What it does and when to use it, in the words a user would type.
category: dev          # optional
requires:              # optional: the surfacer hides the skill while any is unmet
  env: [SOME_API_KEY]
  files: [~/.config/some-tool/token]
  cmd: [ffmpeg]
scope: [acme]          # optional: surfaced only when the prompt names one of these
---
# Title

Decision rules and the procedure. Point to references/ for depth.
```

## 6. Definition of done

A recall:

- [ ] Searched with the task in plain words before planning.
- [ ] Chose from descriptions, fetched one or two skills, and followed them.
- [ ] Searched again when the task changed.
- [ ] Said so when nothing fit, and labelled the fallback.

A forged skill:

- [ ] Subtopic map written before searching.
- [ ] 50 sources, each traced to a scraped row, with coverage counted per subtopic.
- [ ] Every source ingested above the transcript floor.
- [ ] Every distilled answer carries at least 2 citations.
- [ ] SKILL.md tight, references deep, provenance recorded, gaps stated.
- [ ] Graded closed-book on a quiz the corpus wrote, every miss checked by hand, both scores reported.
- [ ] Installed cold, reindexed, and retrieval proven with a realistic request.

## Credits

- skill-retrieval-mcp by JayCheng113, MIT licence.
- NotebookLM by Google. notebooklm-py.
- Method, gates and measurements: the Synthera fleet, August to September 2026.