Embedding model cache defaults to an ephemeral path in containers — silent re-downloads, opaque ONNX failure #7

Summary

When RTFM runs inside a container, the embedding model cache lands in a directory that does not survive a container recreate. The model (~85 MB) is silently re-downloaded on every recreate, and if a recreate happens between the download and the embed jobs, every embed job fails with a low-level ONNX error that doesn't say what's actually wrong.

Current behaviour

rtfm/core/embeddings.py:92:

os.environ.get("FASTEMBED_CACHE_PATH") or (Path.home() / ".cache" / "fastembed")

The fallback is sensible on a workstation. In a container, $HOME is on the ephemeral writable layer, so the cache is wiped by docker compose up -d / docker run after a rebuild — while the index itself (bind-mounted .rtfm/) persists. The two halves of the same state have different lifetimes.

When the model is absent at embed time, the failure surfaces as:

NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from
/tmp/fastembed_cache/models--qdrant--paraphrase-multilingual-MiniLM-L12-v2-onnx-Q/snapshots/.../

Nothing in that message points to the cache directory as the cause, and rtfm failed buckets it as other.

Impact

On a long-running indexing host (agent container, CI, K8s), every image rebuild costs a re-download, and any embed job that races a recreate fails. In our case the whole embed queue was failing while ingestion looked healthy — search silently degraded to lexical-only, which is easy to miss because search still returns results.

Suggestions (any one helps)

1. Prefer the project's own .rtfm/ directory when one exists — e.g. .rtfm/fastembed. The model then shares the lifetime of the index it serves, which is what users bind-mount anyway. Falls back to ~/.cache/fastembed when there's no project dir.
2. Document FASTEMBED_CACHE_PATH in the install / deployment notes, with an explicit "if you run RTFM in a container, point this at a persistent path" line.
3. Translate the ONNX error: catch NO_SUCHFILE on model load and raise something like "embedding model not found in cache <path> — it may have been cleared; set FASTEMBED_CACHE_PATH to a persistent directory".

Environment

• rtfm-ai 0.26.6, fastembed 0.8.0, Python 3.11, node:22-bookworm-slim container
• .rtfm/ bind-mounted from the host; $HOME inside the container's writable layer
• Workaround applied on our side: FASTEMBED_CACHE_PATH=<project>/.rtfm/fastembed — embeddings resumed immediately (22 314 chunks embedded on the first pass).

Note on an older version

We reached this while upgrading a container stuck on 0.24.1. Two things worth knowing, both already fixed in 0.26.6 — recording them only in case they're useful for release notes:

• 0.24.1 ignored honor_gitignore, so a corpus of gitignored PDFs indexed as zero files, with no warning.
• 0.24.1 requested qdrant/paraphrase-multilingual-MiniLM-L12-v2-onnx-Q, a name current fastembed rejects (ValueError: Model ... is not supported). 0.26.6 uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 and works.