The search tool

I implemented voracle, a Rust MCP / CLI that searches Obsidian notes with local embedding and reranking. It retrieves development notes and imports LLM conversations.

  • MCP server: stdio JSON-RPC 2.0, exposing search and read tools
  • CLI commands: -ingest (index building), -list / -rg (semantic search), -distil (external LLM conversation ingestion)
  • Multiple vault roots: specified via --root flags or VORACLE_PATH environment variable (colon-separated)
  • Embedding cache: differential ingest via .embeddings.jsonl. Unchanged notes are restored from cache
  • Model placement: ONNX models stored in ~/.local/share/voracle/models/
  # Build index (first run embeds all chunks, subsequent runs only diff)
voracle -ingest

# Semantic search
voracle -list "kubernetes networking"

# Start MCP server
voracle --server

# Ingest LLM conversation (from clipboard)
voracle -distil c

Background

Its predecessor, valut-oracle, used ONNX Runtime with pplx-embed-v1-0.6b and ColBERT mxbai-edge-colbert. MCP uses stdio JSON-RPC 2.0.

It handles vault search alongside the Rust MCP tools pathfinder and shelpa (now filesystem).

Articles, design notes, and domain files had outgrown my memory. I wanted to find chunks from vague queries and bring the existing distil conversation import into the same tool.

Multiple vault root support

I replaced vault_path: PathBuf with roots: Vec<PathBuf>. --root supplies multiple roots for recursive scanning by VaultResolver.

  voracle --root /path/to/vault1 --root /path/to/vault2 --server

Also configurable via the VORACLE_PATH environment variable, expanded as colon-separated paths. Resolution priority: --root flags > VORACLE_PATH > error.

  export VORACLE_PATH=/Volumes/VALUT/obsidian/docs:/Volumes/VALUT/obsidian/articles

Embedding cache

Embedding every chunk is costly with 500+ notes, so .embeddings.jsonl caches unchanged notes.

Each root has a JSONL cache with one row per note: note_id, content_hash (DefaultHasher), mtime_secs, and chunk vectors. The dot prefix hides it from Obsidian UI.

Matching mtime uses the cache. Otherwise, a content-hash check determines whether to re-embed. A touch or copy that only changes the timestamp can reuse the vectors.

voracle -ingest terminal output
voracle -ingest — embedding notes one by one

The -f flag forces a full re-embed, ignoring the cache. A cached_hits field was added to IngestResult to report how many notes were restored from cache.

Distil command integration

I integrated agent-gateway’s distil shell script for Claude CLI, Codex CLI, and Gemini CLI conversations. Its old POST /v1/obsidian/ingest handler had already been removed.

Three input sources are supported: inline as the second argument, piped from stdin, or from the clipboard via pbpaste if neither is provided. Model aliases map c/claude → claude-opus-4-6, x/codex → codex-5.3, g/gemini → gemini-3.1-pro.

  voracle -distil c              # from clipboard
voracle -distil claude "text"  # inline
cat file.md | voracle -distil x  # stdin

Output goes to _notes/llm-completions/{model_dir}/{correlation_id}.md in the first vault root. Frontmatter includes date, status: raw, correlation_id, and model. If the index is already built, a semantic search on the first 100 words is used to populate related note IDs in the related field.

MCP tool cleanup

Removing MCP ingest

MCP initially listed search, read, and ingest. Ingest became a CLI prerequisite; handle_tool_call no longer dispatched it, but its definition remained.

I removed ingest from tools.rs and tool_definitions() in server.rs, leaving search and read.

Results include a callable read_tool_call. Pages of about 1000 characters limit context use.

The final CLI interface looks like this:

  usage: voracle [--root <path>]... <command> [args]

options:
  --root <path>                   vault root directory (repeatable)
                                  falls back to VORACLE_PATH (colon-separated)

index:
  -ingest [-f]                    build index (required before search)

search (loads .index.bin):
  -list  <query> [-k N]           semantic file list, default top 3
  -rg    <query> [-k N]           grep-like semantic search with context
  -read  <note_id>                print note content to stdout

tools:
  -embed <text>                   print embedding vector (TSV)
  -rerank <query> <file>...       rerank files by ColBERT MaxSim score
  -status                         show index status
  -distil <model> [content]       ingest LLM conversation into vault
                                  model: c(laude), x(codex), g(emini)
                                  reads from: arg, stdin, or clipboard

server:
  --server                        stdio MCP server (JSON-RPC 2.0)

-list and -rg both load .index.bin for semantic search, but -list outputs file paths and scores while -rg displays matched chunks with surrounding context. -read prints full note content to stdout as a CLI-only command, separate from the MCP read tool. -embed and -rerank are low-level inference tools for pipeline debugging and integration with other tools.

Renaming to voracle

I shortened valut-oracle to voracle, including package, binary, and environment variable names.

  • Cargo.toml: package name changed to voracle
  • src/cli/mod.rs: environment variable changed from VALUT_ORACLE_PATH to VORACLE_PATH, all usage messages updated
  • src/config.rs: model directory changed to ~/.local/share/voracle/models/
  • src/main.rs: tracing filter changed to voracle=info
  • src/mcp/server.rs: ServerInfo.name changed to voracle
  • All documentation (README.md, 5 files in docs/) unified to the new name

Models live in ~/.local/share/voracle/models/, honoring XDG_DATA_HOME. pathfinder can embed about 32 MB with include_bytes!, but these 700 MB models stay external. git filter-repo removed them from repository history.