Module</th>	Lines</th>	Island Role</th></tr></thead>
`extract.rs</code></td>`	8</td>	The bottle opener</td></tr>
`chunk.rs</code></td>`	58</td>	The coconut cracker (+ 72 lines of tests)</td></tr>
`embed.rs</code></td>`	30</td>	The Pearl Forge</td></tr>
`db.rs</code></td>`	150</td>	The Smart Beach</td></tr>
`rag.rs</code></td>`	128</td>	The parrot whisperer (+ 47 lines of tests)</td></tr>
`lib.rs</code></td>`	15</td>	The base camp</td></tr>
`bin/sync.rs</code></td>`	142</td>	The gathering outpost</td></tr>
`bin/chat.rs</code></td>`	84</td>	The parrot's perch</td></tr>
Total</strong></td>	~734</strong></td>	(~615 without tests)</strong></td></tr> </tbody></table> About 615 lines of actual code. A fully functional, snake-free, local-first RAG system built entirely on Crab Island.</p> Future Expeditions</a></h2> This island has more to explore. Some trails for the adventurous:</p> Better coconut cracking:</strong></p> Sentence-aware splitting (crack on `.</code> boundaries, group to size)</li>` `Heading-aware splitting (if your scrolls have markdown structure)</li>` text-splitter</a> for semantic chunking with tiktoken</li> </ul> Smarter diving:</strong></p> Hybrid search: combine pearl diving (vectors) with word matching (FTS5) on the same beach</li> Re-ranking: use a second model to re-score the top results</li> Metadata filtering: dive only in certain sections of the beach</li> </ul> Island infrastructure:</strong></p> Turso</a> cloud sync: replicate your beach to the cloud (backup!)</li> Swap parrots: point rig-core at OpenAI or Anthropic — one line change</li> Build an MCP server</a>: let AI agents from other islands query your beach</li> </ul> Creature comforts:</strong></p> rustyline</a> for readline support (history, tab completion at the perch)</li> ratatui</a> for a full TUI — the parrot deserves a palace</li> Streaming responses (rig-core supports it — watch the parrot think in real time)</li> </ul> Farewell from Crab Island</a></h2> We started this expedition wondering: can you build a RAG system without snakes? The answer is a resounding yes — and the island is more pleasant than expected.</p> We got type-safe pipelines where the island's guardian catches wiring mistakes before we waste time. We got anyhow</code> keeping our error handling lean and context-rich. We got a single-file Smart Beach with both relational queries and vector search. And we got it all in ~615 lines of code, running entirely on our own island.</p> The Python continent across the sea has more established trade routes. You won't find a LangChain bazaar here with 500 integrations. But what you will</em> find on Crab Island are well-crafted tools — rig-core, libsql, kreuzberg — that compose beautifully and let you build exactly what you need.</p> The crabs are happy. The parrot is wise. And there's not a snake in sight.</p> The full expedition code lives in the mini-rag</code> directory. Clone the island, pull your parrot models (ollama pull nomic-embed-text && ollama pull deepseek-r1</code>), sync some treasures, and start asking questions.</p> Happy treasure hunting!</p> 🤿 Part 5: The Pearl Divers — Implementing Vector Search 2026-03-12T00:00:00+00:00 The Crab Island RAG Expedition — Part 5 of 6</em></p> The beach is full of buried pearls. Hundreds of them — 758 from the Comprehensive Rust scroll alone, each one a shimmering 768-faceted gem encoding a coconut piece's meaning.</p> Now comes the critical question: when someone asks "What is Rust's ownership model?", how do the crabs find the right</em> pearls?</p> They can't dig up every single one and compare. That would take forever. Instead, they need a diving technique — a way to take the question, forge a question-pearl from it, and then dive straight to the pearls that glow most similarly.</p> About 30 lines. And once we write them, the expedition is nearly complete.</p> The Diving Technique</a></h2> The dive has three steps:</p> Forge a question-pearl</strong> — embed the user's question using the same Forge (nomic-embed-text)</li> Dive the beach</strong> — ask libsql's vector_top_k</code> for the closest pearls</li> Surface with results</strong> — return the matching text chunks with source metadata</li> </ol> Here's our search function:</p> //</span> src/db.rs (continued)</span></span> </span> #</span>[</span>derive</span>(</span>Debug</span>)</span>]</span></span> pub</span> struct</span> SearchHit</span> {</span></span> pub</span> score</span>:</span> f64</span>,</span></span> pub</span> content</span>:</span> String</span>,</span></span> pub</span> doc_title</span>:</span> String</span>,</span></span> pub</span> doc_path</span>:</span> String</span>,</span></span> }</span></span> </span> pub</span> async</span> fn</span> vector_search</span>(</span></span> conn</span>:</span> &</span>Connection</span>,</span></span> query_embedding</span>:</span> &</span>[</span>f32</span>]</span>,</span></span> top_k</span>:</span> usize</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>Vec</span><</span>SearchHit</span>></span>></span> {</span></span> let</span> embedding_json</span> =</span> serde_json</span>::</span>to_string</span>(</span>query_embedding</span>)</span>?</span>;</span></span> let</span> sql</span> =</span> format!</span>(</span></span> "</span>SELECT c.content, d.title, d.source_path \</span></span> FROM vector_top_k('idx_chunks_embedding', vector32(?1), </span>{</span>top_k</span>}</span>) AS v \</span></span> JOIN chunks AS c ON c.rowid = v.id \</span></span> JOIN documents AS d ON d.id = c.document_id</span>"</span></span> )</span>;</span></span> let</span> mut</span> rows</span> =</span> conn</span>.</span>query</span>(</span>&</span>sql</span>,</span> params!</span>[</span>embedding_json</span>]</span>)</span>.</span>await</span>?</span>;</span></span> </span> let</span> mut</span> results</span> =</span> Vec</span>::</span>new</span>(</span>)</span>;</span></span> let</span> mut</span> rank</span> =</span> 0_</span>u32</span>;</span></span> let</span> total</span> =</span> u32</span>::</span>try_from</span>(</span>top_k</span>)</span>.</span>unwrap_or</span>(</span>u32</span>::</span>MAX</span>)</span>;</span></span> while</span> let</span> Some</span>(</span>row</span>)</span> =</span> rows</span>.</span>next</span>(</span>)</span>.</span>await</span>?</span> {</span></span> let</span> content</span> =</span> row</span>.</span>get</span>::</span><</span>String</span>></span>(</span>0</span>)</span>?</span>;</span></span> let</span> doc_title</span> =</span> row</span>.</span>get</span>::</span><</span>String</span>></span>(</span>1</span>)</span>?</span>;</span></span> let</span> doc_path</span> =</span> row</span>.</span>get</span>::</span><</span>String</span>></span>(</span>2</span>)</span>?</span>;</span></span> let</span> score</span> =</span> 1</span>.</span>0</span> -</span> f64</span>::</span>from</span>(</span>rank</span>)</span> /</span> f64</span>::</span>from</span>(</span>total</span>)</span>;</span></span> results</span>.</span>push</span>(</span>SearchHit</span> {</span></span> score</span>,</span></span> content</span>,</span></span> doc_title</span>,</span></span> doc_path</span>,</span></span> }</span>)</span>;</span></span> rank</span> +=</span> 1</span>;</span></span> }</span></span> </span> Ok</span>(</span>results</span>)</span></span> }</span></span></code></pre> A few things to unpack.</p> The JOIN: Pearls with Provenance</a></h2> Notice the JOIN documents</code> — we fetch not just the chunk content, but also which document it came from (title</code>, source_path</code>). This is critical for source attribution</strong>: when the parrot answers a question, the explorer can see where</em> the answer came from.</p> The SearchHit</code> struct carries everything we need: the matching text, a relevance score, and the document metadata.</p> Reef #1: The Beach Doesn't Tell Distance</a></h2> This one puzzled us for a while. libsql's vector_top_k()</code> returns results ordered by similarity — the closest pearls first. Great! But it does not tell you how close</em> they are</strong>. There's no distance column. No similarity score.</p> If you try SELECT v.distance FROM vector_top_k(...)</code>:</p> SqliteFailure: no such column: distance</span></span></code></pre> The beach knows the order, but keeps the measurements to itself.</p> Our workaround: rank-based scoring</strong>.</p> let</span> score</span> =</span> 1</span>.</span>0</span> -</span> f64</span>::</span>from</span>(</span>rank</span>)</span> /</span> f64</span>::</span>from</span>(</span>total</span>)</span>;</span></span></code></pre> With top_k = 5</code>: first pearl scores 1.0</code>, second 0.8</code>, third 0.6</code>, fourth 0.4</code>, fifth 0.2</code>. The actual numbers are cosmetic — our results are already in the right order. The crabs find the right pearls regardless.</p> Expedition tip</strong>: If you need true distance scores, consider usearch</a> — an embeddable single-header HNSW library with Rust bindings. Like libsql, it's in-process (no server), but it returns actual cosine distances and supports custom metrics. For a full-blown search server, qdrant</a> or lancedb</a> are solid options. We chose our Smart Beach (libsql) for its simplicity — one file handles both storage and search — and rank-based scoring is an acceptable trade-off.</p> </blockquote> Reef warning — no rejection of irrelevant pearls</strong>: Without actual distance scores, we can't tell whether a result is genuinely close or just the least far away</em>. Ask "What's a good pasta recipe?" and the crabs will still surface 5 pearls — they'll just be the least irrelevant Rust chunks. A production system would want a similarity threshold to reject results that are too far from the question. With libsql's vector_top_k</code>, that's not possible today. With usearch, qdrant, or lancedb, you'd filter on score > 0.7</code> (or whatever threshold fits your data). Something to keep in mind if your treasure hoard might receive off-topic questions.</p> </blockquote> Reef #2: top_k</code> Can't Be Parameterized</a></h2> Notice {top_k}</code> is interpolated into the SQL string, not passed as ?2</code>. That's because vector_top_k</code>'s second argument must be a literal integer in the SQL — it won't accept a parameter. Since top_k</code> is always a usize</code> from our own code (never user input), this is safe. But it's another reef to be aware of.</p> Wiring It Into the RAG Query</a></h2> With vector_search</code> in hand, our RAG module can stay refreshingly simple — no traits, no generics, just a direct pipeline:</p> //</span> src/rag.rs</span></span> use</span> libsql</span>::</span>Connection</span>;</span></span> use</span> rig</span>::</span>client</span>::</span>CompletionClient</span>;</span></span> use</span> rig</span>::</span>completion</span>::</span>Prompt</span>;</span></span> use</span> rig</span>::</span>providers</span>::</span>ollama</span>;</span></span> </span> use</span> crate</span>::</span>db</span>;</span></span> use</span> crate</span>::</span>embed</span>;</span></span> use</span> crate</span>::</span>DEEPSEEK_R1</span>;</span></span> </span> const</span> TOP_K</span>:</span> usize</span> =</span> 5</span>;</span></span> </span> pub</span> async</span> fn</span> query</span>(</span></span> client</span>:</span> &</span>ollama</span>::</span>Client</span>,</span></span> embedding_model</span>:</span> &</span>ollama</span>::</span>EmbeddingModel</span>,</span></span> conn</span>:</span> &</span>Connection</span>,</span></span> question</span>:</span> &</span>str</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>RagResponse</span>></span> {</span></span> //</span> 1. Forge a question-pearl</span></span> let</span> query_embedding</span> =</span> embed</span>::</span>embed_text</span>(</span>embedding_model</span>,</span> question</span>)</span>.</span>await</span>?</span>;</span></span> </span> //</span> 2. Dive for the top 5 pearls</span></span> let</span> hits</span> =</span> db</span>::</span>vector_search</span>(</span>conn</span>,</span> &</span>query_embedding</span>,</span> TOP_K</span>)</span>.</span>await</span>?</span>;</span></span> </span> //</span> 3. Build context and collect sources in one pass</span></span> let</span> mut</span> context</span> =</span> String</span>::</span>new</span>(</span>)</span>;</span></span> let</span> mut</span> sources</span> =</span> Vec</span>::</span>with_capacity</span>(</span>hits</span>.</span>len</span>(</span>)</span>)</span>;</span></span> for</span> (</span>i</span>,</span> hit</span>)</span> in</span> hits</span>.</span>into_iter</span>(</span>)</span>.</span>enumerate</span>(</span>)</span> {</span></span> if</span> !</span>context</span>.</span>is_empty</span>(</span>)</span> {</span></span> context</span>.</span>push_str</span>(</span>"</span>\</span>n</span>\</span>n</span>"</span>)</span>;</span></span> }</span></span> let</span> _</span> =</span> write!</span>(</span>context</span>,</span> "</span>[</span>{</span>}</span>] </span>{</span>}</span>"</span>,</span> i</span> +</span> 1</span>,</span> &</span>hit</span>.</span>content</span>)</span>;</span></span> sources</span>.</span>push</span>(</span>Source</span> {</span></span> excerpt</span>:</span> truncate_to_excerpt</span>(</span>&</span>hit</span>.</span>content</span>,</span> EXCERPT_LEN</span>)</span>,</span></span> title</span>:</span> hit</span>.</span>doc_title</span>,</span></span> path</span>:</span> hit</span>.</span>doc_path</span>,</span></span> }</span>)</span>;</span></span> }</span></span> </span> //</span> 4. Ask the parrot with explicit context</span></span> let</span> agent</span> =</span> client</span></span> .</span>agent</span>(</span>DEEPSEEK_R1</span>)</span></span> .</span>preamble</span>(</span></span> "</span>You are Kwaak 🦜, the wise parrot of Crab Island. \</span></span> You speak with parrot flair — sprinkle in the occasional \</span></span> squawk!, BRAWWK!, or ruffles feathers, and use \</span></span> emojis like 🦀🥥🏝️. But under the plumage, you are \</span></span> precise and knowledgeable. \</span></span> Use the provided context to answer accurately. If the \</span></span> context doesn't contain enough information, squawk \</span></span> honestly — never make up answers. Keep answers concise.</span>"</span>,</span></span> )</span></span> .</span>build</span>(</span>)</span>;</span></span> </span> let</span> prompt</span> =</span> format!</span>(</span>"</span>Context:</span>\</span>n</span>{</span>context</span>}</span>\</span>n</span>\</span>n</span>Question: </span>{</span>question</span>}</span>"</span>)</span>;</span></span> let</span> response</span> =</span> agent</span>.</span>prompt</span>(</span>prompt</span>)</span>.</span>await</span>?</span>;</span></span> </span> Ok</span>(</span>RagResponse</span> {</span></span> answer</span>:</span> strip_think_tags</span>(</span>&</span>response</span>)</span>,</span></span> sources</span>,</span></span> }</span>)</span></span> }</span></span></code></pre> No VectorStoreIndex</code> trait, no dynamic_context</code>, no serde::Deserialize</code> gymnastics. Just: embed the question → search the beach → build a prompt → ask the parrot. Four steps, all explicit, all debuggable.</p> Going further — rig's dynamic_context</code></strong>: rig-core has a VectorStoreIndex</code> trait and a dynamic_context</code> builder method that automates the embed-search-inject pipeline. You implement the trait for your database, and rig handles the orchestration. It's elegant when you have multiple vector stores or want rig to manage the context window. But it requires implementing two async trait methods with serde::Deserialize</code> bounds, and for our PoC the direct approach is simpler, more transparent, and gives us full control over source attribution. The trait earns its keep in larger systems — start direct, add the abstraction when you need it.</p> </blockquote> Alternative Diving Techniques</a></h2> Our vector_search</code> is purpose-built for the Smart Beach. Other options:</p> usearch</a></strong>: An embeddable HNSW index with Rust bindings. In-process like libsql, but returns actual distances and supports custom metrics. A good middle ground between our simple approach and a full search server.</li> rig's built-in integrations</strong>: rig-core has optional support for MongoDB, LanceDB, Neo4j, and more. Check the rig docs</a> — someone may have already built a diving crew for your preferred beach.</li> In-memory diving</strong>: For small treasure hoards, skip the beach entirely. Compute cosine similarity over a Vec<Vec<f32>></code> in memory. Fast for development, won't scale.</li> Hybrid search</strong>: Combine pearl diving (vector search) with word matching (FTS5 in SQLite). Use vector search for meaning and FTS for exact keywords. libsql supports both on the same beach.</li> </ul> Beach Report</a></h2> In ~30 lines of search code plus ~40 lines of RAG wiring, our diving crew can:</p> Take any question and forge a question-pearl</li> Navigate the f64/f32 reef at the boundary</li> Dive the Smart Beach using vector_top_k</code> with source metadata</li> Surface with ranked results including document provenance</li> Build explicit context for the parrot — no magic, full control</li> </ul> In the final post, we bring the whole expedition together. Two binaries, one database, and a conversation with our Rust books.</p> 💎 Part 4: The Pearl Forge — Embeddings and Vector Storage 2026-03-11T00:00:00+00:00 The Crab Island RAG Expedition — Part 4 of 6</em></p> Deep in the heart of Crab Island, past the coconut groves and through the bamboo thicket, lies the Pearl Forge.</p> This is where the real magic happens. The crabs bring their coconut pieces here, and the Forge transforms each one into a pearl</strong> — a shimmering 768-faceted gem that encodes the meaning</em> of the text. Two pieces about similar topics produce pearls that glow in similar ways. Two pieces about unrelated topics produce pearls that look completely different.</p> And the beach? It's no ordinary beach. It's a smart beach</strong> — powered by libsql — that remembers where every pearl is buried and can instantly find the ones most similar to any new pearl you show it.</p> SQLite learned to understand meaning. Let's see how.</p> How the Pearl Forge Works</a></h2> An embedding is a list of numbers (a vector) that captures the meaning of text. "How do I handle errors in Rust?" and "What's the best way to manage failures?" produce vectors that are close together in 768-dimensional space, even though they share almost no words.</p> The Forge is powered by nomic-embed-text</code> running locally via Ollama. Here's how we operate it:</p> //</span> src/embed.rs</span></span> use</span> rig</span>::</span>client</span>::</span>EmbeddingsClient</span>;</span></span> use</span> rig</span>::</span>embeddings</span>::</span>EmbeddingModel</span> as</span> _</span>;</span></span> use</span> rig</span>::</span>providers</span>::</span>ollama</span>;</span></span> </span> const</span> NOMIC_EMBED_TEXT</span>:</span> &</span>str</span> =</span> "</span>nomic-embed-text</span>"</span>;</span></span> pub</span> const</span> EMBEDDING_DIMS</span>:</span> usize</span> =</span> 768</span>;</span></span> </span> #</span>[</span>must_use</span>]</span></span> pub</span> fn</span> embedding_model</span>(</span>client</span>:</span> &</span>ollama</span>::</span>Client</span>)</span> -></span> ollama</span>::</span>EmbeddingModel</span> {</span></span> client</span>.</span>embedding_model_with_ndims</span>(</span>NOMIC_EMBED_TEXT</span>,</span> EMBEDDING_DIMS</span>)</span></span> }</span></span> </span> pub</span> async</span> fn</span> embed_text</span>(</span></span> model</span>:</span> &</span>ollama</span>::</span>EmbeddingModel</span>,</span></span> text</span>:</span> &</span>str</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>Vec</span><</span>f32</span>></span>></span> {</span></span> let</span> embedding</span> =</span> model</span>.</span>embed_text</span>(</span>text</span>)</span>.</span>await</span>?</span>;</span></span> Ok</span>(</span>to_f32</span>(</span>&</span>embedding</span>.</span>vec</span>)</span>)</span></span> }</span></span> </span> pub</span> async</span> fn</span> embed_texts</span>(</span></span> model</span>:</span> &</span>ollama</span>::</span>EmbeddingModel</span>,</span></span> texts</span>:</span> impl</span> IntoIterator</span><</span>Item</span> =</span> String</span>></span> +</span> Send</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>Vec</span><</span>Vec</span><</span>f32</span>></span>></span>></span> {</span></span> let</span> embeddings</span> =</span> model</span>.</span>embed_texts</span>(</span>texts</span>)</span>.</span>await</span>?</span>;</span></span> Ok</span>(</span>embeddings</span>.</span>iter</span>(</span>)</span>.</span>map</span>(</span>\|</span>e</span>\|</span> to_f32</span>(</span>&</span>e</span>.</span>vec</span>)</span>)</span>.</span>collect</span>(</span>)</span>)</span></span> }</span></span> </span> //</span> rig returns Vec<f64>, libsql F32_BLOB needs f32</span></span> #</span>[</span>allow</span>(</span>clippy</span>::</span>cast_possible_truncation</span>)</span>]</span></span> fn</span> to_f32</span>(</span>vec</span>:</span> &</span>[</span>f64</span>]</span>)</span> -></span> Vec</span><</span>f32</span>></span> {</span></span> vec</span>.</span>iter</span>(</span>)</span>.</span>map</span>(</span>\|</span>&</span>v</span>\|</span> v</span> as</span> f32</span>)</span>.</span>collect</span>(</span>)</span></span> }</span></span></code></pre> Two functions: embed_text</code> for single queries (the chat side), embed_texts</code> for batch ingestion (the sync side — we'll use this in Part 6 to speed things up). Both share a to_f32</code> helper that handles the f64→f32 conversion.</p> Wait. Did you notice that suspicious cast in to_f32</code>?</p> The Reef of Mismatched Gems</a></h2> Every island has a hidden reef, and this is ours — the f64/f32 mismatch</strong>.</p> rig-core</strong> produces pearls with 64-bit precision (Vec<f64></code>)</li> libsql</strong> stores pearls with 32-bit precision (F32_BLOB</code>)</li> </ul> So every time the Forge creates a pearl, we have to file it down from f64 to f32. And every time we search, same thing. The precision loss is negligible for similarity search (pearl facets are typically between -1.0 and 1.0), but Clippy — the island's quality inspector — flags it immediately.</p> We handle it with #[allow(clippy::cast_possible_truncation)]</code>. This is one of those cases where the cast is intentional, the inspector is noted, and life goes on.</p> Expedition tip</strong>: You could store pearls in full f64 precision, but libsql's vector functions are optimized for f32. The storage savings are real too: 768 dimensions × 4 bytes = 3KB per pearl vs. 6KB. With thousands of pearls buried on the beach, that adds up to a lighter island.</p> </blockquote> Other Pearl Forge settings:</strong></p> Model</th> Facets (dims)</th> Notes</th></tr></thead> nomic-embed-text</code></td> 768</td> Our pick — good balance of quality and size</td></tr> mxbai-embed-large</code></td> 1024</td> More facets, higher fidelity</td></tr> all-minilm</code></td> 384</td> Fewer facets, faster forging</td></tr> snowflake-arctic-embed</code></td> 1024</td> Great for larger pieces of text</td></tr> </tbody></table> Switching is easy — just change NOMIC_EMBED_TEXT</code> and EMBEDDING_DIMS</code>. The rest of the expedition doesn't care which Forge setting you use.</p> The Smart Beach</a></h2> Now let's design our beach. Two tables, one index:</p> //</span> src/db.rs</span></span> use</span> std</span>::</span>path</span>::</span>Path</span>;</span></span> </span> use</span> anyhow</span>::</span>Context</span>;</span></span> use</span> libsql</span>::</span>{</span>Builder</span>,</span> Connection</span>,</span> params</span>}</span>;</span></span> </span> pub</span> async</span> fn</span> init_db</span>(</span>path</span>:</span> &</span>Path</span>)</span> -></span> anyhow</span>::</span>Result</span><</span>Connection</span>></span> {</span></span> let</span> db</span> =</span> Builder</span>::</span>new_local</span>(</span>path</span>)</span>.</span>build</span>(</span>)</span>.</span>await</span>?</span>;</span></span> let</span> conn</span> =</span> db</span>.</span>connect</span>(</span>)</span>?</span>;</span></span> </span> conn</span>.</span>execute_batch</span>(</span></span> "</span>CREATE TABLE IF NOT EXISTS documents (</span></span> id INTEGER PRIMARY KEY AUTOINCREMENT,</span></span> title TEXT NOT NULL,</span></span> source_path TEXT NOT NULL,</span></span> mtime INTEGER NOT NULL DEFAULT 0,</span></span> created_at TEXT NOT NULL DEFAULT (datetime('now'))</span></span> );</span></span> </span> CREATE TABLE IF NOT EXISTS chunks (</span></span> id INTEGER PRIMARY KEY AUTOINCREMENT,</span></span> document_id INTEGER NOT NULL REFERENCES documents(id),</span></span> chunk_index INTEGER NOT NULL,</span></span> content TEXT NOT NULL,</span></span> embedding F32_BLOB(768)</span></span> );</span></span> </span> CREATE INDEX IF NOT EXISTS idx_chunks_embedding</span></span> ON chunks(libsql_vector_idx(embedding, 'metric=cosine'));</span>"</span>,</span></span> )</span></span> .</span>await</span>?</span>;</span></span> </span> Ok</span>(</span>conn</span>)</span></span> }</span></span></code></pre> Let's explore the beach:</p> documents</code> table</strong>: The registry — which treasures have we processed? The mtime</code> column is clever: it records when</em> the source file was last modified, so we can skip treasures we've already processed.</p> chunks</code> table</strong>: The pearl grid — each pearl has its content (the original coconut piece text) and its embedding (F32_BLOB(768)</code> — 768 facets of 32-bit shimmer).</p> libsql_vector_idx(embedding, 'metric=cosine')</code></strong>: This is the beach's magic — a cosine similarity index. When a crab dives looking for pearls, this index tells it which direction to swim. Cosine similarity measures the angle</em> between two pearl facet patterns — the standard metric for text embeddings.</p> The whole beach lives in a single file. No server, no infrastructure, no cloud. Just a .db</code> file on your island.</p> Burying Pearls</a></h2> pub</span> async</span> fn</span> insert_document</span>(</span></span> conn</span>:</span> &</span>Connection</span>,</span></span> title</span>:</span> &</span>str</span>,</span></span> source_path</span>:</span> &</span>str</span>,</span></span> mtime</span>:</span> i64</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>i64</span>></span> {</span></span> conn</span>.</span>execute</span>(</span></span> "</span>INSERT INTO documents (title, source_path, mtime) VALUES (?1, ?2, ?3)</span>"</span>,</span></span> params!</span>[</span>title</span>,</span> source_path</span>,</span> mtime</span>]</span>,</span></span> )</span></span> .</span>await</span>?</span>;</span></span> </span> let</span> mut</span> rows</span> =</span> conn</span>.</span>query</span>(</span>"</span>SELECT last_insert_rowid()</span>"</span>,</span> params!</span>[</span>]</span>)</span>.</span>await</span>?</span>;</span></span> let</span> row</span> =</span> rows</span></span> .</span>next</span>(</span>)</span></span> .</span>await</span>?</span></span> .</span>context</span>(</span>"</span>failed to get last insert rowid</span>"</span>)</span>?</span>;</span></span> let</span> id</span> =</span> row</span>.</span>get</span>::</span><</span>i64</span>></span>(</span>0</span>)</span>?</span>;</span></span> </span> Ok</span>(</span>id</span>)</span></span> }</span></span></code></pre> Notice .context("failed to get last insert rowid")?</code> — anyhow's way of turning an Option::None</code> into a descriptive error. Much cleaner than constructing a libsql::Error</code> by hand.</p> And burying each pearl — notice how the embedding gets JSON-serialized and passed through vector32()</code>:</p> pub</span> async</span> fn</span> insert_chunk</span>(</span></span> conn</span>:</span> &</span>Connection</span>,</span></span> document_id</span>:</span> i64</span>,</span></span> chunk_index</span>:</span> usize</span>,</span></span> content</span>:</span> &</span>str</span>,</span></span> embedding</span>:</span> &</span>[</span>f32</span>]</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>(</span>)</span>></span> {</span></span> let</span> embedding_json</span> =</span> serde_json</span>::</span>to_string</span>(</span>embedding</span>)</span>?</span>;</span></span> conn</span>.</span>execute</span>(</span></span> "</span>INSERT INTO chunks (document_id, chunk_index, content, embedding) \</span></span> VALUES (?1, ?2, ?3, vector32(?4))</span>"</span>,</span></span> params!</span>[</span></span> document_id</span>,</span></span> i64</span>::</span>try_from</span>(</span>chunk_index</span>)</span>.</span>expect</span>(</span>"</span>chunk index fits i64</span>"</span>)</span>,</span></span> content</span>,</span></span> embedding_json</span></span> ]</span>,</span></span> )</span></span> .</span>await</span>?</span>;</span></span> </span> Ok</span>(</span>(</span>)</span>)</span></span> }</span></span></code></pre> vector32(?4)</code> is libsql's pearl press — it takes a JSON array of floats ([0.123, -0.456, ...]</code>) and compresses it into a binary vector blob. Serialize with serde, pass as text, let the beach handle the rest.</p> The Clever Crabs: Idempotent Ingestion</a></h2> No crab wants to re-forge 758 pearls for a coconut they already processed yesterday. We use file modification time (mtime</code>) for cheap change detection:</p> pub</span> async</span> fn</span> find_document_by_path</span>(</span></span> conn</span>:</span> &</span>Connection</span>,</span></span> source_path</span>:</span> &</span>str</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>Option</span><</span>(</span>i64</span>,</span> i64</span>)</span>></span>></span> {</span></span> let</span> mut</span> rows</span> =</span> conn</span></span> .</span>query</span>(</span></span> "</span>SELECT id, mtime FROM documents WHERE source_path = ?1</span>"</span>,</span></span> params!</span>[</span>source_path</span>]</span>,</span></span> )</span></span> .</span>await</span>?</span>;</span></span> </span> match</span> rows</span>.</span>next</span>(</span>)</span>.</span>await</span>?</span> {</span></span> Some</span>(</span>row</span>)</span> =></span> {</span></span> let</span> id</span> =</span> row</span>.</span>get</span>::</span><</span>i64</span>></span>(</span>0</span>)</span>?</span>;</span></span> let</span> mtime</span> =</span> row</span>.</span>get</span>::</span><</span>i64</span>></span>(</span>1</span>)</span>?</span>;</span></span> Ok</span>(</span>Some</span>(</span>(</span>id</span>,</span> mtime</span>)</span>)</span>)</span></span> }</span></span> None</span> =></span> Ok</span>(</span>None</span>)</span>,</span></span> }</span></span> }</span></span> </span> pub</span> async</span> fn</span> delete_document</span>(</span></span> conn</span>:</span> &</span>Connection</span>,</span></span> doc_id</span>:</span> i64</span>,</span></span> )</span> -></span> anyhow</span>::</span>Result</span><</span>(</span>)</span>></span> {</span></span> conn</span>.</span>execute</span>(</span>"</span>DELETE FROM chunks WHERE document_id = ?1</span>"</span>,</span> params!</span>[</span>doc_id</span>]</span>)</span></span> .</span>await</span>?</span>;</span></span> conn</span>.</span>execute</span>(</span>"</span>DELETE FROM documents WHERE id = ?1</span>"</span>,</span> params!</span>[</span>doc_id</span>]</span>)</span></span> .</span>await</span>?</span>;</span></span> Ok</span>(</span>(</span>)</span>)</span></span> }</span></span></code></pre> The crab's decision tree:</p> "Have I seen this bottle before?" → Check find_document_by_path</code></li> Same mtime? → "Already processed, moving on!" (skip)</li> Different mtime? → "Bottle was refilled!" → dig up old pearls, forge new ones</li> Never seen it? → "Fresh treasure!" → forge and bury</li> </ol> Expedition tip</strong>: For a permanent settlement, you might use content hashing (SHA256) instead of mtime — it's more reliable across file systems and backups. But for our expedition, mtime is simple, fast, and does the job.</p> </blockquote> Expedition tip — schema migrations</strong>: Our CREATE TABLE IF NOT EXISTS</code> works fine for a fresh expedition. But a real settlement would need proper schema migration — adding columns, renaming tables, evolving the beach layout over time. Tools like refinery</a> or sqlx-migrate</a> handle this elegantly. For our adventure, we just delete the database and re-sync when the schema changes.</p> </blockquote> Note — the SmallVec</code> temptation</strong>: If you're a Rust performance enthusiast, you might think: "768 elements, always the same size — perfect for SmallVec<[f32; 768]></code> to avoid heap allocation!" Resist! A SmallVec</code> with N=768</code> puts 3KB on the stack (768 × 4 bytes</code>). That's unusually large for a stack allocation, and in practice SmallVec</code> will spill to the heap anyway once you pass it around. SmallVec</code> shines for tiny</em> collections (1–8 elements) where avoiding a heap allocation actually matters. For embeddings, a plain Vec<f32></code> is simpler and equally fast. Don't fight the allocator when it's already doing a great job.</p> </blockquote> Beach Report</a></h2> The Pearl Forge and Smart Beach are operational:</p> Embedding generation</strong>: text → 768-faceted pearl, forged locally via Ollama</li> Database schema</strong>: two tables with a cosine similarity index</li> CRUD operations</strong>: register treasures, bury pearls, dig up old ones</li> Idempotent ingestion</strong>: clever crabs skip what they've already processed</li> </ul> The pearls are buried. Now we need to teach the crabs how to find</em> the right ones. In the next post, we implement vector search — the diving technique that lets crabs find pearls by meaning.</p> 🥥 Part 3: Cracking Coconuts — Chunking Text 2026-03-10T00:00:00+00:00 The Crab Island RAG Expedition — Part 3 of 6</em></p> The crabs have gathered on the beach, claws clicking in anticipation. Before them lies the extracted text — 348,000 characters from the Comprehensive Rust scroll, 85,000 from Rust Design Patterns. That's a lot</em> of text.</p> Now, a crab can't carry an entire coconut. It needs to crack it into pieces — small enough to handle, but large enough to still be nutritious. Too small, and you get dust. Too large, and nobody can carry it. And here's the trick: you want the pieces to overlap</em> slightly, so that any juicy bit sitting right on the crack line appears in both pieces.</p> This is chunking. And it's quietly the most important part of the entire expedition.</p> Why Cracking Matters So Much</a></h2> Pop quiz: what's the single biggest factor in RAG answer quality?</p> Not the parrot's vocabulary. Not the embedding model. Not the database.</p> It's how you crack your coconuts.</p> Feed the parrot an entire 10-page chapter and it'll give you a vague, meandering answer. Feed it the exact right paragraph</em> and it'll nail it. Chunking is the art of breaking text into pieces that are small enough to be precise, but large enough to carry meaning.</p> The Sliding Window: A Crab's Best Friend</a></h2> There are many cracking strategies:</p> Strategy</th> Idea</th> Complexity</th></tr></thead> Fixed-size</td> Crack every N characters</td> Simple</td></tr> Sentence-based</td> Crack on sentence boundaries</td> Medium</td></tr> Semantic</td> Crack on topic changes</td> Complex (needs a model!)</td></tr> Recursive</td> Try big cracks, fall back to smaller</td> Medium</td></tr> </tbody></table> We're going with fixed-size with overlap</strong> — the simplest technique that actually works well. The overlap is the secret sauce: it ensures that any important idea sitting right on a crack line appears in both pieces.</p> Picture a crab walking along the coconut with a measuring shell:</p> Text: [=======piece 1========]</span></span> [=======piece 2========]</span></span> [=======piece 3========]</span></span> ←---→</span></span> overlap</span></span></code></pre> Each piece is 500 characters. Each new piece starts 450 characters after the previous one (500 - 50 overlap). A sentence that gets cracked at position 490? It still appears fully in the next piece.</p> The Code</a></h2> //</span> src/chunk.rs</span></span> #</span>[</span>derive</span>(</span>Debug</span>)</span>]</span></span> pub</span> struct</span> Chunk</span> {</span></span> pub</span> index</span>:</span> usize</span>,</span></span> pub</span> content</span>:</span> String</span>,</span></span> }</span></span> </span> #</span>[</span>must_use</span>]</span></span> pub</span> fn</span> chunk_text</span>(</span>text</span>:</span> &</span>str</span>,</span> chunk_size</span>:</span> usize</span>,</span> overlap</span>:</span> usize</span>)</span> -></span> Vec</span><</span>Chunk</span>></span> {</span></span> let</span> chars</span>:</span> Vec</span><</span>char</span>></span> =</span> text</span>.</span>chars</span>(</span>)</span>.</span>collect</span>(</span>)</span>;</span></span> let</span> overlap</span> =</span> overlap</span>.</span>min</span>(</span>chunk_size</span> /</span> 2</span>)</span>;</span></span> let</span> step</span> =</span> chunk_size</span> -</span> overlap</span>;</span></span> </span> if</span> chars</span>.</span>len</span>(</span>)</span> <=</span> chunk_size</span> {</span></span> let</span> trimmed</span> =</span> text</span>.</span>trim</span>(</span>)</span>;</span></span> if</span> trimmed</span>.</span>is_empty</span>(</span>)</span> {</span></span> return</span> Vec</span>::</span>new</span>(</span>)</span>;</span></span> }</span></span> return</span> vec!</span>[</span>Chunk</span> {</span></span> index</span>:</span> 0</span>,</span></span> content</span>:</span> trimmed</span>.</span>to_owned</span>(</span>)</span>,</span></span> }</span>]</span>;</span></span> }</span></span> </span> let</span> mut</span> chunks</span>:</span> Vec</span><</span>_</span>></span> =</span> chars</span></span> .</span>windows</span>(</span>chunk_size</span>)</span></span> .</span>step_by</span>(</span>step</span>)</span></span> .</span>enumerate</span>(</span>)</span></span> .</span>filter_map</span>(</span>\|</span>(</span>i</span>,</span> window</span>)</span>\|</span> {</span></span> let</span> content</span>:</span> String</span> =</span> window</span>.</span>iter</span>(</span>)</span>.</span>collect</span>(</span>)</span>;</span></span> let</span> trimmed</span> =</span> content</span>.</span>trim</span>(</span>)</span>;</span></span> (</span>!</span>trimmed</span>.</span>is_empty</span>(</span>)</span>)</span>.</span>then</span>(</span>\|</span>\|</span> Chunk</span> {</span></span> index</span>:</span> i</span>,</span></span> content</span>:</span> trimmed</span>.</span>to_owned</span>(</span>)</span>,</span></span> }</span>)</span></span> }</span>)</span></span> .</span>collect</span>(</span>)</span>;</span></span> </span> //</span> Handle the last chunk if it wasn't covered by windows</span></span> let</span> last_start</span> =</span> chunks</span>.</span>len</span>(</span>)</span> </span> step</span>;</span></span> if</span> last_start</span> <</span> chars</span>.</span>len</span>(</span>)</span> {</span></span> let</span> content</span>:</span> String</span> =</span> chars</span>.</span>get</span>(</span>last_start</span>..</span>)</span>.</span>unwrap_or_default</span>(</span>)</span>.</span>iter</span>(</span>)</span>.</span>collect</span>(</span>)</span>;</span></span> let</span> trimmed</span> =</span> content</span>.</span>trim</span>(</span>)</span>;</span></span> if</span> !</span>trimmed</span>.</span>is_empty</span>(</span>)</span> {</span></span> chunks</span>.</span>push</span>(</span>Chunk</span> {</span></span> index</span>:</span> chunks</span>.</span>len</span>(</span>)</span>,</span></span> content</span>:</span> trimmed</span>.</span>to_owned</span>(</span>)</span>,</span></span> }</span>)</span>;</span></span> }</span></span> }</span></span> </span> //</span> Re-index after filtering</span></span> for</span> (</span>i</span>,</span> chunk</span>)</span> in</span> chunks</span>.</span>iter_mut</span>(</span>)</span>.</span>enumerate</span>(</span>)</span> {</span></span> chunk</span>.</span>index </span>=</span> i</span>;</span></span> }</span></span> </span> chunks</span></span> }</span></span></code></pre> Let's examine the crab's technique:</p> Why clamp the overlap?</strong> If someone passes an overlap larger than the chunk size, the subtraction chunk_size - overlap</code> would underflow and panic. Even an overlap close to the chunk size makes no sense — you'd be re-reading almost the entire piece each time. Clamping to chunk_size / 2</code> is a sensible guard: overlapping more than half a piece means you're doing more repeating than progressing.</p> Why windows()</code> + step_by()</code>?</strong> Rust's slice method windows(n)</code> gives us a sliding window of size n</code>, advancing one element at a time. Combined with step_by(chunk_size - overlap)</code>, we get exactly the overlapping pieces we want — no manual index arithmetic needed. The standard library does the heavy lifting.</p> Why chars()</code> instead of bytes?</strong> Because the island speaks many languages. If we split on bytes, we might slice a multi-byte character in half. Imagine cracking a coconut marked "Größe" (German for "size") right through the "ö" — we'd get invalid UTF-8. The crabs are more careful than that.</p> Why handle the last chunk separately?</strong> windows()</code> only produces full-size windows. If the text doesn't divide evenly, the trailing piece (shorter than chunk_size</code>) needs special handling.</p> Why skip empty pieces?</strong> Some coconuts have hollow sections — long stretches of whitespace from page breaks or formatting artifacts. No point carrying an empty shell. The crabs just toss those aside.</p> Why #[must_use]</code>?</strong> If a crab cracks a coconut and nobody picks up the pieces, something has gone wrong. The island's guardian (the compiler) will raise an eyebrow.</p> The Crabs Get to Work</a></h2> Let's watch them crack our Rust books:</p> //</span> Comprehensive Rust: ~348K characters</span></span> let</span> text</span> =</span> extract</span>::</span>extract_text</span>(</span></span> Path</span>::</span>new</span>(</span>"</span>comprehensive-rust.pdf</span>"</span>)</span>,</span></span> )</span>.</span>await</span>?</span>;</span></span> let</span> chunks</span> =</span> chunk</span>::</span>chunk_text</span>(</span>&</span>text</span>,</span> 500</span>,</span> 50</span>)</span>;</span></span> //</span> => 758 pieces!</span></span> </span> //</span> Rust Design Patterns: ~85K characters</span></span> let</span> text</span> =</span> extract</span>::</span>extract_text</span>(</span></span> Path</span>::</span>new</span>(</span>"</span>rust-design-patterns.pdf</span>"</span>)</span>,</span></span> )</span>.</span>await</span>?</span>;</span></span> let</span> chunks</span> =</span> chunk</span>::</span>chunk_text</span>(</span>&</span>text</span>,</span> 500</span>,</span> 50</span>)</span>;</span></span> //</span> => 187 pieces</span></span></code></pre> 758 coconut pieces from a ~350-page course. Each piece is a paragraph-sized morsel of Rust knowledge — small enough to carry, meaningful enough to be useful.</p> Choosing the Right Crack Size</a></h2> The magic numbers — 500 chars, 50 overlap — aren't carved in sacred coral. Here's how to think about them:</p> Too small (e.g., 100 chars)</strong>:</p> Precise — you'll find exactly the right sentence</li> But context-free — "It's unsafe" means nothing without knowing what "it" refers to</li> More pieces = more work for the crabs, more pearls to bury</li> </ul> Too large (e.g., 2000 chars)</strong>:</p> Plenty of context in each piece</li> But diluted relevance — a piece about 5 different topics is vaguely relevant to all and strongly relevant to none</li> Embedding models have limits too (nomic-embed-text handles ~8192 tokens)</li> </ul> The overlap (50 chars)</strong>:</p> Too small: ideas at boundaries get lost in the crack</li> Too large: you're carrying near-duplicate pieces, wasting energy</li> 10% of chunk_size is a solid starting point</li> </ul> Expedition tip</strong>: These numbers are worth tuning for your specific treasure. Dense academic papers might want smaller pieces. Conversational text might want larger ones. Start with 500/50, test your answer quality, adjust.</p> </blockquote> Testing the Crab's Work</a></h2> Here's something the crabs insist on — and you won't see in most treasure-hunting tutorials: actual tests</strong>.</p> #</span>[</span>cfg</span>(</span>test</span>)</span>]</span></span> #</span>[</span>allow</span>(</span>clippy</span>::</span>indexing_slicing</span>)</span>]</span></span> mod</span> tests</span> {</span></span> use</span> super</span>::</span></span>;</span></span> </span> #</span>[</span>test</span>]</span></span> fn</span> empty_text_produces_no_chunks</span>(</span>)</span> {</span></span> let</span> chunks</span> =</span> chunk_text</span>(</span>"</span>"</span>,</span> 500</span>,</span> 50</span>)</span>;</span></span> assert!</span>(</span>chunks</span>.</span>is_empty</span>(</span>)</span>)</span>;</span></span> }</span></span> </span> #</span>[</span>test</span>]</span></span> fn</span> short_text_produces_single_chunk</span>(</span>)</span> {</span></span> let</span> chunks</span> =</span> chunk_text</span>(</span>"</span>hello world</span>"</span>,</span> 500</span>,</span> 50</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>.</span>len</span>(</span>)</span>,</span> 1</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>0</span>]</span>.</span>content</span>,</span> "</span>hello world</span>"</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>0</span>]</span>.</span>index</span>,</span> 0</span>)</span>;</span></span> }</span></span> </span> #</span>[</span>test</span>]</span></span> fn</span> text_is_split_with_overlap</span>(</span>)</span> {</span></span> let</span> text</span> =</span> "</span>a</span>"</span>.</span>repeat</span>(</span>1000</span>)</span>;</span></span> let</span> chunks</span> =</span> chunk_text</span>(</span>&</span>text</span>,</span> 500</span>,</span> 50</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>.</span>len</span>(</span>)</span>,</span> 3</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>0</span>]</span>.</span>content</span>.</span>len</span>(</span>)</span>,</span> 500</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>1</span>]</span>.</span>content</span>.</span>len</span>(</span>)</span>,</span> 500</span>)</span>;</span></span> //</span> Last chunk: 1000 - 900 = 100 chars</span></span> assert_eq!</span>(</span>chunks</span>[</span>2</span>]</span>.</span>content</span>.</span>len</span>(</span>)</span>,</span> 100</span>)</span>;</span></span> }</span></span> </span> #</span>[</span>test</span>]</span></span> fn</span> chunks_have_sequential_indices</span>(</span>)</span> {</span></span> let</span> text</span> =</span> "</span>x</span>"</span>.</span>repeat</span>(</span>2000</span>)</span>;</span></span> let</span> chunks</span> =</span> chunk_text</span>(</span>&</span>text</span>,</span> 500</span>,</span> 50</span>)</span>;</span></span> for</span> (</span>i</span>,</span> chunk</span>)</span> in</span> chunks</span>.</span>iter</span>(</span>)</span>.</span>enumerate</span>(</span>)</span> {</span></span> assert_eq!</span>(</span>chunk</span>.</span>index</span>,</span> i</span>)</span>;</span></span> }</span></span> }</span></span> </span> #</span>[</span>test</span>]</span></span> fn</span> overlap_clamped_to_half_chunk_size</span>(</span>)</span> {</span></span> let</span> text</span> =</span> "</span>a</span>"</span>.</span>repeat</span>(</span>1000</span>)</span>;</span></span> //</span> overlap 999 > chunk_size/2 (250), gets clamped to 250</span></span> let</span> chunks</span> =</span> chunk_text</span>(</span>&</span>text</span>,</span> 500</span>,</span> 999</span>)</span>;</span></span> let</span> expected</span> =</span> chunk_text</span>(</span>&</span>text</span>,</span> 500</span>,</span> 250</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>.</span>len</span>(</span>)</span>,</span> expected</span>.</span>len</span>(</span>)</span>)</span>;</span></span> for</span> (</span>a</span>,</span> b</span>)</span> in</span> chunks</span>.</span>iter</span>(</span>)</span>.</span>zip</span>(</span>expected</span>.</span>iter</span>(</span>)</span>)</span> {</span></span> assert_eq!</span>(</span>a</span>.</span>content</span>,</span> b</span>.</span>content</span>)</span>;</span></span> }</span></span> }</span></span> </span> #</span>[</span>test</span>]</span></span> fn</span> emoji_not_split_across_chunks</span>(</span>)</span> {</span></span> //</span> 🦀 is a single char (4 bytes), place it at the chunk boundary</span></span> let</span> text</span> =</span> format!</span>(</span>"</span>{</span>}</span>🦀</span>{</span>}</span>"</span>,</span> "</span>a</span>"</span>.</span>repeat</span>(</span>4</span>)</span>,</span> "</span>b</span>"</span>.</span>repeat</span>(</span>5</span>)</span>)</span>;</span></span> //</span> 10 chars total, chunk_size=5, overlap=1, step=4</span></span> let</span> chunks</span> =</span> chunk_text</span>(</span>&</span>text</span>,</span> 5</span>,</span> 1</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>.</span>len</span>(</span>)</span>,</span> 3</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>0</span>]</span>.</span>content</span>,</span> "</span>aaaa🦀</span>"</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>1</span>]</span>.</span>content</span>,</span> "</span>🦀bbbb</span>"</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>2</span>]</span>.</span>content</span>,</span> "</span>bb</span>"</span>)</span>;</span></span> }</span></span> </span> #</span>[</span>test</span>]</span></span> fn</span> whitespace_only_chunks_are_skipped</span>(</span>)</span> {</span></span> let</span> text</span> =</span> format!</span>(</span>"</span>{</span>}</span>{</span>}</span>content</span>"</span>,</span> "</span>a</span>"</span>.</span>repeat</span>(</span>500</span>)</span>,</span> "</span> "</span>.</span>repeat</span>(</span>500</span>)</span>)</span>;</span></span> let</span> chunks</span> =</span> chunk_text</span>(</span>&</span>text</span>,</span> 500</span>,</span> 0</span>)</span>;</span></span> //</span> First chunk: 500 'a's, second chunk: 500 spaces (skipped), third chunk: "content"</span></span> assert_eq!</span>(</span>chunks</span>.</span>len</span>(</span>)</span>,</span> 2</span>)</span>;</span></span> assert_eq!</span>(</span>chunks</span>[</span>1</span>]</span>.</span>content</span>,</span> "</span>content</span>"</span>)</span>;</span></span> }</span></span> }</span></span></code></pre> Empty coconuts, tiny coconuts, the overlap math, overlap clamping, emoji safety, sequential numbering, hollow sections — all verified. The emoji test is a nice illustration: the 🦀 crab (4 bytes, but 1 char) sits right at the boundary and appears in both adjacent chunks thanks to the overlap — never sliced in half. Pure functions are a joy to test: no mocks, no async, no setup. Just crack → check</code>.</p> cargo</span> nextest</span> run</span></span> #</span> 7 tests run: 7 passed</span></span></code></pre> The crabs are meticulous workers.</p> Fancier Cracking Tools</a></h2> Our sliding-window approach is the machete of chunking — simple, reliable, gets the job done. When you're ready for finer tools:</p> Sentence-aware cracking</strong>: Split on .</code> boundaries, then group sentences up to your size limit. Better coherence per piece.</li> text-splitter</a></strong>: A Rust crate for semantic-aware text splitting with tiktoken integration. The power tool version.</li> Recursive cracking</strong>: Try paragraphs first, then sentences, then characters. What LangChain's RecursiveCharacterTextSplitter</code> does on the Python continent.</li> Heading-aware cracking</strong>: If your text has structure (markdown headers), split on those boundaries. Each piece gets a natural topic.</li> </ul> Note — kreuzberg can crack coconuts too</strong>: kreuzberg has a built-in chunking</code> feature that splits text during extraction. So why did we write our own? Two reasons. First, rolling your own chunker teaches you what's happening — it's ~60 lines of pure Rust with no magic. Second, separating extraction from chunking gives you flexibility: you can experiment with chunk sizes, test with unit tests, or swap in a smarter strategy later without touching extraction. In production, you might prefer kreuzberg's built-in chunking for convenience.</p> </blockquote> But our 50-line cracker will carry you surprisingly far. The Comprehensive Rust book chunks up nicely, and our RAG system returns relevant answers. Upgrade the tool when the simple one stops working, not before.</p> Beach Report</a></h2> The crabs have cracked all the coconuts:</p> ~60 lines</strong> of pure, synchronous Rust</li> Zero dependencies</strong> — just String</code> and Vec</code></li> Unicode-safe</strong> — handles multi-byte characters without cracking through the middle</li> Tested</strong> — 7 tests covering every edge case</li> Configurable</strong> — piece size and overlap are parameters, not constants</li> </ul> The beach is covered in neatly numbered coconut pieces. But here's the thing about Crab Island — it's not an ordinary island. The coconuts here don't contain milk. Crack one open, and inside you'll find a pearl — a shimmering, dense little sphere that somehow captures the essence</em> of the text it held. The crabs have known this for generations: every piece of knowledge, once properly cracked, reveals a pearl.</p> Now it's time for the truly magical step — forging those pearls. Time to visit the pearl forge.</p> 🍾 Part 2: Opening the Bottles — Text Extraction 2026-03-09T00:00:00+00:00 The Crab Island RAG Expedition — Part 2 of 6</em></p> Day two on Crab Island. The morning mist lifts and reveals the shoreline, littered with treasures from across the seven seas — sealed bottles with letters inside, scrolls wrapped in oilcloth, maps etched on strange flat stones. Some look ancient and weathered. Others are suspiciously well-formatted.</p> These are our documents. PDFs, EPUBs, DOCX files — each one a different container holding knowledge we want to search through. Before the crabs can do anything useful with this treasure, we need to open</em> it and get the text out.</p> Here's the thing about these containers: they're all different. A bottle (PDF) isn't text at all — it's a set of page layout instructions. "Draw glyph 'H' at coordinates (72.4, 540.2). Draw glyph 'e' at (78.1, 540.2)." Turning that back into the word "Hello" is a feat of reverse engineering. A scroll (EPUB) is really a bundle of HTML pages wrapped in a zip file. A carved stone (DOCX) is XML wearing a trench coat.</p> Fortunately, we don't have to crack each one by hand. We have kreuzberg — the island's universal bottle opener.</p> kreuzberg: One Tool for Every Container</a></h2> kreuzberg</a> handles 75+ document formats through a single async function call. PDFs, EPUBs, DOCX, HTML, images (via OCR), plain text — hand it any container from the shore and it extracts what's inside.</p> Here's our entire extraction module:</p> //</span> src/extract.rs</span></span> use</span> std</span>::</span>path</span>::</span>Path</span>;</span></span> </span> use</span> kreuzberg</span>::</span>ExtractionConfig</span>;</span></span> </span> pub</span> async</span> fn</span> extract_text</span>(</span>path</span>:</span> &</span>Path</span>)</span> -></span> anyhow</span>::</span>Result</span><</span>String</span>></span> {</span></span> let</span> result</span> =</span> kreuzberg</span>::</span>extract_file</span>(</span>path</span>,</span> None</span>,</span> &</span>ExtractionConfig</span>::</span>default</span>(</span>)</span>)</span>.</span>await</span>?</span>;</span></span> Ok</span>(</span>result</span>.</span>content</span>)</span></span> }</span></span></code></pre> Four lines of actual code. The crabs hand kreuzberg a container, it hands back the text inside. No fuss, no format-specific logic.</p> extract_file</code> takes a path, an optional MIME type hint (we pass None</code> — "you figure it out"), and a config (we use defaults). The ?</code> works seamlessly because we use anyhow::Result</code>, which automatically converts any error implementing std::error::Error</code>.</p> Cracking Open the Rust Books</a></h2> Let's try it on some real treasures. Imagine we've found two legendary scrolls on the beach: Comprehensive Rust</a> (Google's Rust course) and Rust Design Patterns</a>:</p> let</span> text</span> =</span> extract</span>::</span>extract_text</span>(</span></span> Path</span>::</span>new</span>(</span>"</span>comprehensive-rust.pdf</span>"</span>)</span>,</span></span> )</span>.</span>await</span>?</span>;</span></span> tracing</span>::</span>info!</span>(</span>"</span>Extracted </span>{</span>}</span> characters</span>"</span>,</span> text</span>.</span>len</span>(</span>)</span>)</span>;</span></span> //</span> => Extracted 348572 characters</span></span></code></pre> That's 348,000 characters of Rust wisdom, liberated from its PDF prison. And the design patterns scroll:</p> let</span> text</span> =</span> extract</span>::</span>extract_text</span>(</span></span> Path</span>::</span>new</span>(</span>"</span>rust-design-patterns.pdf</span>"</span>)</span>,</span></span> )</span>.</span>await</span>?</span>;</span></span> tracing</span>::</span>info!</span>(</span>"</span>Extracted </span>{</span>}</span> characters</span>"</span>,</span> text</span>.</span>len</span>(</span>)</span>)</span>;</span></span> //</span> => Extracted 85431 characters</span></span></code></pre> Same function, different containers, same result: pure text. If tomorrow a crab washes ashore with an EPUB or a DOCX, the same code handles it. One bottle opener to rule them all.</p> The Hidden Reef: pdfium</a></h2> Every tropical island has a hidden reef, and here's ours. kreuzberg uses pdfium</a> under the hood for PDF extraction — the same C library Chrome uses to render PDFs. It needs to be available at runtime.</p> If you try to open a PDF bottle and get:</p> kreuzberg error: pdfium library not found</span></span></code></pre> ...you've hit the reef. You need to download libpdfium.dylib</code> (macOS) or libpdfium.so</code> (Linux) from the pdfium-binaries</a> releases and place it where the crabs can find it:</p> #</span> macOS (Apple Silicon)</span></span> mkdir</span> -</span>p</span> ~/lib</span> &&</span> cp</span> libpdfium.dylib</span> ~/lib/</span></span></code></pre> Expedition tip</strong>: The kreuzberg crate has a pdf</code> feature that must be explicitly enabled — the default features don't include PDF support! Make sure your manifest says kreuzberg = { version = "4.5.1", features = ["pdf"] }</code>. There's also a bundled-pdfium</code> feature, but despite its promising name, it doesn't actually bundle pdfium. It just enables the pdf</code> feature. We all ran aground on that reef at least once.</p> </blockquote> Why Plain Text?</a></h2> A keen-eyed explorer might ask: "We're throwing away structure! Headlines, bold text, tables — all gone. Isn't that wasteful?"</p> Fair point! But consider:</p> The crabs work with meaning, not formatting.</strong> Embedding models convert text</em> to vectors. They don't know what bold means.</li> Coconut cracking is simpler.</strong> Our chunker (next post) uses a sliding window — it doesn't need to know about headings or paragraphs.</li> It's model-agnostic.</strong> Whether our parrot speaks nomic-embed-text, OpenAI's ada, or something else entirely, they all eat plain text.</li> </ol> Could we do better? Absolutely — preserve markdown structure, split on heading boundaries, keep tables intact. But that's optimization for a future expedition. The crabs on this island believe in starting simple. The Rust Design Patterns scroll itself would approve — start with a working solution, refine later.</p> Going further — OCR: reading faded carvings</strong>: Some bottles on the beach contain images instead of text — scanned documents, photos of whiteboards, hand-drawn diagrams. kreuzberg can read those too via OCR (Optical Character Recognition), powered by Tesseract</a> or PaddleOCR</a>. Pass an image file to the same extract_file</code> function and it extracts whatever text it can find. You'll need to enable the ocr</code> or paddle-ocr</code> feature in Cargo.toml</code> and install the OCR engine on your system (brew install tesseract</code> on macOS for Tesseract). Results vary with image quality — a crisp scan yields great text, a blurry photo yields crab scratches. For the expedition we stick to digital documents, but if your treasure trove includes scanned PDFs or images, kreuzberg has you covered without any code changes.</p> </blockquote> Other Bottle Openers</a></h2> kreuzberg is our pick, but the island market has alternatives:</p> pdf-extract</a></strong>: Pure Rust, PDF-only. Lighter to carry, no pdfium reef to navigate. Perfect if you only find PDF bottles on your beach.</li> lopdf</a></strong>: Low-level PDF manipulation. For the explorer who wants to understand every byte of the container.</li> docx-rs</a></strong>: DOCX-specific. When your shore is covered in carved stones and nothing else.</li> </ul> We chose kreuzberg because we expect all kinds of treasures to wash up — PDFs today, EPUBs and research papers tomorrow. One tool, one function, every format.</p> Beach Report</a></h2> Our extraction outpost is ready:</p> 4 lines of code</strong> (not counting imports)</li> Format-agnostic</strong> — PDFs, EPUBs, DOCX, HTML, and 70+ more</li> Async</strong> — won't block the crabs while they wait for a large PDF to process</li> Error-handled</strong> — kreuzberg errors flow naturally through anyhow</code> via ?</code></li> </ul> The bottles are open and the messages are spread out on the beach. But these aren't ordinary messages — they're treasure maps, and they all point to the same thing: a grove of coconut palms deeper on the island. The text inside those coconuts holds the real knowledge. But a crab can't carry a whole coconut — it needs to crack them into manageable pieces first.</p> 🏕️ Part 1: Landfall — Setting Up Camp on Crab Island 2026-03-08T00:00:00+00:00 The Crab Island RAG Expedition — Part 1 of 6</em></p> Welcome, adventurer, to Crab Island.</p> This is a lush tropical paradise — dense jungle canopy, white sand beaches, crystal-clear lagoons. The locals are crabs. Resourceful, industrious, type-safe crabs. And here's the best part: there are absolutely no snakes on this island</strong>. Not a single one. The crabs made sure of that ages ago.</p> You've arrived here because you've heard rumors of ancient knowledge hidden across the island — treasure maps carved on coconut shells, messages sealed in bottles, scrolls tucked inside coral caves. You want to find that knowledge and ask questions</em> about it. But the island is vast, and there's no way to read everything yourself.</p> Fortunately, the crabs have a system. They can crack open any container, break the contents into pearl-sized pieces, bury those pearls in special spots on the beach, and later dive to find exactly the right ones when you have a question. They even have a wise old parrot who can synthesize answers from whatever pearls the crabs bring back.</p> This, dear adventurer, is a RAG system. And we're going to build it in Rust.</p> The Expedition Plan</a></h2> RAG (Retrieval-Augmented Generation) is just a treasure hunt with six steps:</p> Extract</strong> — Open the bottles, unroll the scrolls, decipher the maps (get text from PDFs, EPUBs, etc.)</li> Chunk</strong> — Crack the coconuts into bite-sized pieces (split text into small passages)</li> Embed</strong> — Translate each piece into the island's musical language (convert text to numerical vectors)</li> Store</strong> — Bury each pearl in a labeled spot on the beach (save vectors in a database)</li> Retrieve</strong> — Send crabs diving for the pearls that match your question (vector similarity search)</li> Generate</strong> — Bring the pearls to the wise parrot, who weaves them into an answer (LLM generation)</li> </ol> PDF/EPUB/HTML → extract → chunks → embed → store (vector DB)</span></span> ↓</span></span> query → embed → similarity search → LLM → answer</span></span></code></pre>Why This Island? (Why Rust?)</a></h2> The Python continent across the sea is wonderful — bustling ports, established trade routes, a bazaar with every tool imaginable (LangChain, LlamaIndex, ChromaDB). If you need something built fast with maximum community support, sail there. No judgment. The Python ecosystem for AI is genuinely incredible.</p> But Crab Island offers something different:</p> Type-safe pipelines</strong>: Wire things wrong and the island's guardian (the compiler) stops you before</em> you waste an afternoon embedding the wrong data.</li> Error handling that doesn't lie</strong>: No surprise NoneType has no attribute 'content'</code> erupting from a swamp at 2am. Every failure path is mapped.</li> Single-artifact deployment</strong>: cargo build --release</code> produces one executable. Carry it anywhere. No virtualenv, no requirements.txt</code>.</li> Async by default</strong>: Tokio keeps our I/O humming while crabs work in parallel across the beach.</li> No GIL</strong>: When we need the crabs to truly work simultaneously, nothing holds them back.</li> </ul> The Expedition Gear</a></h2> Four crates form our toolkit — each one a trusty piece of island equipment:</p> Crate</th> Role</th> Island Metaphor</th></tr></thead> rig-core</strong></td> LLM abstraction</td> The wise parrot's translation manual</td></tr> libsql</strong></td> Database + vectors</td> The beach where we bury and find pearls</td></tr> kreuzberg</strong></td> Document extraction</td> The bottle opener / scroll unfurler</td></tr> Ollama</strong></td> Local LLM runtime</td> The parrot itself, living on your island</td></tr> </tbody></table> Note that Ollama is external software (the LLM server), not a Rust crate — rig-core provides the Rust interface to it.</p> Other gear you could pack:</strong></p> Instead of rig-core</strong>, you could use llm-chain</a> or call Ollama's REST API by hand. rig-core wins because it provides clean abstractions for both embeddings and completions, with built-in Ollama support.</li> Instead of libsql</strong>, you could bring a qdrant</a> or lancedb</a> server. We chose libsql because it's just a single file on the beach — no infrastructure, no server process, and it handles both relational data and</em> vector search.</li> Instead of kreuzberg</strong>, you could use pdf-extract</a> for maps (PDFs) only. kreuzberg handles 75+ container types through a single function — bottles, scrolls, coconut carvings, you name it.</li> Instead of Ollama</strong>, you could summon a cloud parrot (OpenAI, Anthropic). We keep ours local so the entire expedition stays on the island.</li> </ul> Setting Up Base Camp</a></h2> Our camp has a library (shared knowledge) and two outposts — one for gathering treasures, one for asking questions:</p> #</span> Cargo.toml</span></span> [</span>package</span>]</span></span> name</span> =</span> "</span>mini-rag</span>"</span></span> version</span> =</span> "</span>0.1.0</span>"</span></span> edition</span> =</span> "</span>2024</span>"</span></span> rust-version</span> =</span> "</span>1.91.1</span>"</span></span> </span> [</span>lib</span>]</span></span> name</span> =</span> "</span>mini_rag</span>"</span></span> path</span> =</span> "</span>src/lib.rs</span>"</span></span> </span> [[</span>bin</span>]]</span></span> name</span> =</span> "</span>rag-sync</span>"</span></span> path</span> =</span> "</span>src/bin/sync.rs</span>"</span></span> </span> [[</span>bin</span>]]</span></span> name</span> =</span> "</span>rag-chat</span>"</span></span> path</span> =</span> "</span>src/bin/chat.rs</span>"</span></span> </span> [</span>dependencies</span>]</span></span> anyhow</span> =</span> "</span>1</span>"</span></span> kreuzberg</span> =</span> {</span> version</span> =</span> "</span>4.5.1</span>"</span>,</span> features</span> =</span> [</span>"</span>pdf</span>"</span>]</span> }</span></span> kreuzberg-pdfium-render</span> =</span> "</span>4.5.1</span>"</span> #</span> pin to match kreuzberg 4.5.x (upstream version mismatch)</span></span> libsql</span> =</span> "</span>0.9.30</span>"</span></span> owo-colors</span> =</span> "</span>4</span>"</span></span> rig-core</span> =</span> "</span>0.33.0</span>"</span></span> serde_json</span> =</span> "</span>1.0.149</span>"</span></span> tokio</span> =</span> {</span> version</span> =</span> "</span>1</span>"</span>,</span> features</span> =</span> [</span>"</span>macros</span>"</span>,</span> "</span>rt-multi-thread</span>"</span>]</span> }</span></span> tracing</span> =</span> "</span>0.1.44</span>"</span></span> tracing-subscriber</span> =</span> "</span>0.3.23</span>"</span></span></code></pre> Note</strong>: This project requires Rust 1.91.1+ (due to dependencies MSRV). Check with rustc --version</code>.</p> </blockquote> The library holds all shared modules, plus a few helpers that both binaries need:</p> //</span> src/lib.rs</span></span> use</span> anyhow</span>::</span>Context</span>;</span></span> use</span> rig</span>::</span>providers</span>::</span>ollama</span>;</span></span> </span> pub</span> mod</span> chunk</span>;</span></span> pub</span> mod</span> db</span>;</span></span> pub</span> mod</span> embed</span>;</span></span> pub</span> mod</span> extract</span>;</span></span> pub</span> mod</span> rag</span>;</span></span> </span> pub</span> const</span> DB_PATH</span>:</span> &</span>str</span> =</span> "</span>mini_rag.db</span>"</span>;</span></span> pub</span> const</span> DEEPSEEK_R1</span>:</span> &</span>str</span> =</span> "</span>deepseek-r1</span>"</span>;</span></span> </span> pub</span> fn</span> ollama_client</span>(</span>)</span> -></span> anyhow</span>::</span>Result</span><</span>ollama</span>::</span>Client</span>></span> {</span></span> ollama</span>::</span>Client</span>::</span>new</span>(</span>rig</span>::</span>client</span>::</span>Nothing</span>)</span></span> .</span>context</span>(</span>"</span>failed to create Ollama client</span>"</span>)</span></span> }</span></span></code></pre> Five modules for the core logic, a shared database path, the model name, and a helper to create the Ollama client. Both binaries import from here instead of duplicating setup code.</p> The Island's Danger Log</a></h2> Before we venture into the jungle, we need a way to record what goes wrong. On the Python continent, people sometimes use bare except:</code> blocks and hope for the best. On Crab Island, every ?</code> operator automatically captures and propagates errors with full context.</p> We use anyhow</code></a> — a pragmatic error handling crate that lets us write anyhow::Result<T></code> everywhere and attach human-readable context with .context("what went wrong")</code>:</p> //</span> No error.rs needed — anyhow handles it all</span></span> pub</span> fn</span> ollama_client</span>(</span>)</span> -></span> anyhow</span>::</span>Result</span><</span>ollama</span>::</span>Client</span>></span> {</span></span> ollama</span>::</span>Client</span>::</span>new</span>(</span>rig</span>::</span>client</span>::</span>Nothing</span>)</span></span> .</span>context</span>(</span>"</span>failed to create Ollama client</span>"</span>)</span></span> }</span></span></code></pre> The ?</code> operator works with any error type that implements std::error::Error</code> — io::Error</code>, libsql::Error</code>, kreuzberg::KreuzbergError</code> — they all flow through anyhow</code> automatically. No boilerplate, no manual From</code> impls. The danger log stays short and sweet.</p> Going further — typed error enums</strong>: For a library where callers need to match</code> on specific error variants (retry on network error, abort on auth error), consider thiserror</code></a> or derive_more</code></a> to define a custom error enum. For our expedition — a PoC where we only display errors, never match on them — anyhow</code> is the right tool.</p> </blockquote> Training the Parrot</a></h2> Before the expedition can begin, our parrot needs two skills:</p> #</span> Understanding language (converting text to meaning-vectors)</span></span> ollama</span> pull</span> nomic-embed-text</span></span> </span> #</span> Speaking wisdom (generating answers from context)</span></span> ollama</span> pull</span> deepseek-r1</span></span></code></pre> nomic-embed-text</code> gives the crabs their sense of direction — it converts text into 768-dimensional coordinates so they know where to bury and find pearls. Alternatives like mxbai-embed-large</code> (1024 dims) or all-minilm</code> (384 dims) work too — just different maps of the same beach.</p> deepseek-r1</code> is our parrot — a reasoning model that "thinks out loud" in <think></code> tags before speaking. We'll need to clean up its muttering later (spoiler for Part 6!), but the wisdom it delivers is excellent.</p> The Expedition Ahead</a></h2> Here's our route across the island:</p> Part 2</strong>: Open the bottles and unroll the scrolls — text extraction (it's 3 lines, seriously)</li> Part 3</strong>: Crack coconuts into pieces — the art of chunking</li> Part 4</strong>: Turn pieces into pearls and bury them on the beach — embeddings and vector storage</li> Part 5</strong>: Train the crabs to dive for the right pearls — vector search</li> Part 6</strong>: Ask the parrot — wire it all into sync and chat binaries</li> </ul> By Part 6, we'll have a fully working, snake-free RAG system. Grab your machete and let's head into the jungle.</p>

🦜 Part 6: Asking the Parrot — The Full RAG Pipeline

2026-03-13T00:00:00+00:00

The Crab Island RAG Expedition — Part 6 of 6

The sun is high over Crab Island. The expedition has covered a lot of ground — we've opened bottles, cracked coconuts, forged pearls, buried them on the Smart Beach, and trained the crabs to dive.
Now it's time for the grand finale: we connect everything together and actually talk to our documents. Two outposts, one beach, and a very wise parrot.
Picture the scene: you walk up to the parrot's perch overlooking the lagoon, and ask a question. The parrot squawks at the crabs. They dive, surface with a clawful of relevant pearls, and lay them at the parrot's feet. The parrot studies them, ruffles its feathers, and speaks wisdom grounded in your actual documents.
That's RAG. That's what we're building. Let's wire it up.
The Two Outposts</a></h2>
Our island has two outposts sharing the same Smart Beach:
┌─────────────┐ ┌──────────────────────────┐ ┌─────────────┐ │ rag-sync │────→│ mini_rag.db │←────│ rag-chat │ │ (gathering) │ │ treasures + pearls + │ │ (the perch) │ │ │ │ vector embeddings │ │ │ └─────────────┘ └──────────────────────────┘ └─────────────┘ │ │ ▼ ▼ open → crack → forge → bury question → dive → parrot → answer</code></pre> rag-sync</code> sends the crabs out to gather and process treasures. rag-chat</code> is the parrot's perch where you ask questions. They share the beach database but never need to run at the same time.
Shared Camp Gear</a></h2> Before we look at the outposts, both binaries share some equipment from the library: // src/lib.rs pub mod chunk; pub mod db; pub mod embed; pub mod extract; pub mod rag; use anyhow::Context; use rig::providers::ollama; pub const DB_PATH: &str = "mini_rag.db"; pub const DEEPSEEK_R1: &str = "deepseek-r1"; pub fn ollama_client() -> anyhow::Result<ollama::Client> { ollama::Client::new(rig::client::Nothing) .context("failed to create Ollama client") }</code></pre> The DB_PATH</code> and DEEPSEEK_R1</code> constants live here so both binaries use the same values. The ollama_client()</code> helper avoids duplicating the client creation logic — DRY, even on a tropical island. The Gathering Outpost</a></h2> This is the expedition's workhorse. Hand it file paths or directories, and the crabs open every container, crack the contents, forge pearls, and bury them — skipping anything they've already processed. // src/bin/sync.rs #![allow(clippy::print_stderr)] use std::path::{Path, PathBuf}; use std::time::SystemTime; use anyhow::Context; use rig::providers::ollama; use mini_rag::chunk; use mini_rag::{DB_PATH, db, embed, extract}; const CHUNK_SIZE: usize = 500; const CHUNK_OVERLAP: usize = 50;</code></pre> First, a way to read a bottle's wax seal date (file modification time): fn file_mtime(path: &Path) -> anyhow::Result<i64> { let metadata = std::fs::metadata(path)?; let mtime = metadata .modified()? .duration_since(SystemTime::UNIX_EPOCH) .context("system time before UNIX epoch")?; #[allow(clippy::cast_possible_wrap)] Ok(mtime.as_secs() as i64) }</code></pre> Notice how anyhow</code>'s .context()</code> replaces what would otherwise be a manual map_err</code> with an io::Error</code> — cleaner and more descriptive. Portability note: metadata.modified()</code> works on all major platforms (Linux, macOS, Windows) but can return Err</code> on exotic targets. The ?</code> handles that gracefully. A subtler issue: mtime can be unreliable on network filesystems (NFS clock skew) or after copying files across machines. For a production system, content hashing (e.g. SHA-256) would be more robust — but for a local expedition, mtime is simple and fast enough. </blockquote> Then, scouting the jungle for treasures — no extra dependency, just std::fs::read_dir</code>: fn collect_files(args: &[String]) -> anyhow::Result<Vec<PathBuf>> { let mut files = Vec::new(); for arg in args { let path = PathBuf::from(arg); if path.is_dir() { walk_dir(&path, &mut files)?; } else if path.is_file() { files.push(path); } else { tracing::warn!( "Skipping {}: not a file or directory", path.display(), ); } } Ok(files) } fn walk_dir(dir: &Path, files: &mut Vec<PathBuf>) -> anyhow::Result<()> { for entry in std::fs::read_dir(dir)? { let entry = entry?; let path = entry.path(); if path.is_dir() { walk_dir(&path, files)?; } else if path.is_file() { files.push(path); } } Ok(()) }</code></pre> Expedition tip: For a permanent settlement, consider walkdir</a> or ignore</a> (the crate powering ripgrep). They handle symlinks, permission errors, and .gitignore</code> rules gracefully. Our manual scouting works fine for the expedition. </blockquote> Now the heart of the operation — processing a single treasure: fn title_from_path(path: &Path) -> String { path.file_stem() .map_or_else( || "Untitled".to_owned(), |s| s.to_string_lossy().into_owned(), ) } async fn ingest_file( conn: &libsql::Connection, embedding_model: &ollama::EmbeddingModel, path: &Path, ) -> anyhow::Result<()> { let source_path = path.to_string_lossy(); let mtime = file_mtime(path)?; // The clever crab check: have we seen this before? if let Some((existing_id, existing_mtime)) = db::find_document_by_path(conn, &source_path).await? { if existing_mtime == mtime { tracing::info!("Skipping {} (unchanged)", path.display()); return Ok(()); } tracing::info!( "Re-ingesting {} (mtime changed)", path.display(), ); db::delete_document(conn, existing_id).await?; } tracing::info!("Ingesting {}...", path.display()); // The full pipeline: open → crack → forge → bury let content = extract::extract_text(path).await?; tracing::info!( "Extracted {} characters from {}", content.len(), path.display(), ); let title = title_from_path(path); let doc_id = db::insert_document( conn, &title, &source_path, mtime, ).await?; let chunks = chunk::chunk_text(&content, CHUNK_SIZE, CHUNK_OVERLAP); tracing::info!( "Created {} chunks for {}", chunks.len(), path.display(), ); for (i, c) in chunks.iter().enumerate() { let embedding = embed::embed_text( embedding_model, &c.content, ).await?; db::insert_chunk( conn, doc_id, i, &c.content, &embedding, ).await?; if i % 10 == 0 { tracing::info!( "Embedded chunk {}/{}", i + 1, chunks.len(), ); } } tracing::info!("Done ingesting {}", path.display()); Ok(()) }</code></pre> And the main function — the expedition leader: #[tokio::main] async fn main() -> anyhow::Result<()> { tracing_subscriber::fmt::init(); let args: Vec<String> = std::env::args().skip(1).collect(); if args.is_empty() { eprintln!("Usage: rag-sync <file-or-dir>..."); std::process::exit(1); } let files = collect_files(&args)?; if files.is_empty() { tracing::warn!("No files found to ingest"); return Ok(()); } tracing::info!("Found {} file(s) to process", files.len()); let conn = db::init_db(Path::new(DB_PATH)).await?; let client = mini_rag::ollama_client()?; let embedding_model = embed::embedding_model(&client); for file in &files { if let Err(e) = ingest_file( &conn, &embedding_model, file, ).await { tracing::error!( "Failed to ingest {}: {e}", file.display(), ); } } Ok(()) }</code></pre> Note mini_rag::ollama_client()?</code> — the shared helper from lib.rs</code>. Both binaries create the Ollama client the same way. Let's send the crabs to work on our Rust treasures: $ cargo run --bin rag-sync -- comprehensive-rust.pdf rust-design-patterns.pdf INFO Ingesting comprehensive-rust.pdf... INFO Extracted 348572 characters from comprehensive-rust.pdf INFO Created 758 chunks for comprehensive-rust.pdf INFO Embedded chunk 1/758 INFO Embedded chunk 11/758 ... INFO Done ingesting comprehensive-rust.pdf INFO Ingesting rust-design-patterns.pdf... INFO Extracted 85431 characters from rust-design-patterns.pdf INFO Created 187 chunks for rust-design-patterns.pdf ... INFO Done ingesting rust-design-patterns.pdf</code></pre> Run it again — the crabs are too smart to repeat work: $ cargo run --bin rag-sync -- comprehensive-rust.pdf rust-design-patterns.pdf INFO Skipping comprehensive-rust.pdf (unchanged) INFO Skipping rust-design-patterns.pdf (unchanged)</code></pre> Idempotent crabs. The best kind. Speeding Up the Crabs</a></h2> The simple loop works, but it's slow — 758 chunks means 758 individual HTTP requests to Ollama and 758 individual database commits. On a real beach, that takes about 2 minutes. Three quick wins can dramatically cut that time. Win #1: Batch embeddings. Ollama's embedding API accepts multiple texts in a single request. Instead of 758 HTTP round-trips, we send batches of 200 — just 4 requests: // src/embed.rs — add a batch variant pub async fn embed_texts( model: &ollama::EmbeddingModel, texts: impl IntoIterator<Item = String> + Send, ) -> anyhow::Result<Vec<Vec<f32>>> { let embeddings = model.embed_texts(texts).await?; Ok(embeddings.iter().map(|e| to_f32(&e.vec)).collect()) }</code></pre> Under the hood, rig-core's embed_texts</code> sends all texts in a single "input": [...]</code> JSON payload. The GPU processes them as a batch — much faster than one at a time. Win #2: Database transaction. Without an explicit transaction, each INSERT</code> auto-commits — that's 758 fsyncs to disk. Wrapping them in a single transaction means one fsync at the end: let tx = conn.transaction().await?; // ... all inserts go through &tx ... tx.commit().await?;</code></pre> Putting it all together — batch embedding with transactional inserts: const EMBED_BATCH_SIZE: usize = 200; // In ingest_file: let tx = conn.transaction().await?; let mut chunk_idx = 0; for batch in chunks.chunks(EMBED_BATCH_SIZE) { let texts: Vec<String> = batch.iter().map(|c| c.content.clone()).collect(); let embeddings = embed::embed_texts(embedding_model, texts).await?; for (chunk, embedding) in batch.iter().zip(embeddings) { db::insert_chunk(&tx, doc_id, chunk_idx, &chunk.content, &embedding).await?; chunk_idx += 1; } tracing::info!("Embedded {}/{} chunks", chunk_idx, chunks.len()); } tx.commit().await?;</code></pre> The biggest win is batch embedding — it eliminates per-chunk HTTP overhead. The transaction groups all disk writes into a single flush at commit. Why not rayon? The bottleneck is I/O (HTTP to Ollama + DB writes), not CPU. Rayon shines for CPU-bound parallelism — chunking, parsing, hashing. Here, tokio</code> is the right tool because we're dealing with async I/O. </blockquote> Why not concurrent files? Ollama serializes GPU inference — sending embedding requests from two files simultaneously just queues them. Concurrent files would only help overlap extraction of file B with embedding of file A, which is marginal. </blockquote> Going further — pipelining with tokio::spawn</summary> For larger datasets, you could overlap embedding batch N+1 with inserting batch N's results. The idea: spawn DB inserts on a background task while the next embedding call is in flight. let mut chunk_idx = 0; let mut pending_insert: Option<tokio::task::JoinHandle<anyhow::Result<()>>> = None; for batch in chunks.chunks(EMBED_BATCH_SIZE) { let texts: Vec<String> = batch.iter().map(|c| c.content.clone()).collect(); let embeddings = embed::embed_texts(embedding_model, texts).await?; if let Some(handle) = pending_insert.take() { handle.await??; } let tx_clone = tx.clone(); let batch_data: Vec<_> = batch.iter().zip(embeddings) .map(|(c, emb)| { chunk_idx += 1; (chunk_idx - 1, c.content.clone(), emb) }) .collect(); pending_insert = Some(tokio::spawn(async move { for (idx, content, emb) in &batch_data { db::insert_chunk(&tx_clone, doc_id, *idx, content, emb).await?; } Ok(()) })); } if let Some(handle) = pending_insert.take() { handle.await??; }</code></pre> In practice, DB inserts are fast relative to embedding, so the gain is small. The simpler sequential version above is what we actually ship — add the pipeline when profiling tells you to. </details> The Parrot's Brain</a></h2> We already built the query()</code> function in Part 5 — embed the question, dive the beach, build context, ask the parrot. Here we add the finishing touches: structured responses with source attribution, and pretty-printing for the terminal. The response includes the answer and where it came from: // src/rag.rs #[derive(Debug)] pub struct Source { pub title: String, pub path: String, pub excerpt: String, } #[derive(Debug)] pub struct RagResponse { pub answer: String, pub sources: Vec<Source>, }</code></pre> The sources get pretty-printed with colored output via owo-colors</code>: impl fmt::Display for RagResponse { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { writeln!(f, "{}", self.answer)?; if !self.sources.is_empty() { writeln!(f, "\n{}", "─── Sources ───".dimmed())?; for (i, source) in self.sources.iter().enumerate() { let idx = format!("[{}]", i + 1); let location = format!( "{} ({})", source.title.green(), source.path.dimmed() ); writeln!(f, " {} {location}", idx.bold())?; writeln!(f, " {}", format!("\"{}\"", source.excerpt).dimmed())?; } } Ok(()) } }</code></pre> A RAG answer without sources is like a parrot that speaks but won't say where it learned. Now the explorer can verify the answer against the original treasure. The Parrot's Muttering</a></h2> Our parrot (DeepSeek-R1) is a reasoning model. Before it speaks wisdom, it thinks out loud: <think> The user asks about ownership in Rust. Let me look at the context... Based on the chunks provided, I can see... </think> Ownership in Rust ensures memory safety without a garbage collector.</code></pre> Those <think></code> blocks are fascinating for understanding how the parrot reasons, but we don't want them in the final answer. Our feather-grooming function handles this: /// Strip `<think>...</think>` tags (used by reasoning models like Deepseek-r1). pub fn strip_think_tags(text: &str) -> String { let mut result = String::new(); let mut remaining = text; while let Some((before, after_open)) = remaining.split_once("<think>") { result.push_str(before); match after_open.split_once("</think>") { Some((_, after_close)) => remaining = after_close, // Unclosed tag — the parrot trailed off mid-thought None => return result.trim().to_owned(), } } result.push_str(remaining); result.trim().to_owned() }</code></pre> This function is pub</code> — if you later add a summarize</code> module that also uses a reasoning model, you import it instead of copy-pasting. Duplicated logic is the reef that scrapes you when you fix a bug in one copy but forget the other. Expedition tip: If you switch to a non-reasoning parrot (like llama3.2</code> or mistral</code>), there won't be any <think></code> tags. The function handles that gracefully — no tags, no stripping, text passes through unchanged. </blockquote> Reef Warning: UTF-8 and Text Truncation</a></h2> The truncate_to_excerpt</code> function for source previews needs care. If you naively write &text[..150]</code>, you'll panic on multi-byte characters — "café"[..3]</code> slices into the middle of é</code> and Rust rightfully refuses. Use char_indices()</code> to find a safe boundary: fn truncate_to_excerpt(text: &str, max_len: usize) -> String { let trimmed = text.trim().replace('\n', " "); if trimmed.len() <= max_len { return trimmed; } let boundary = trimmed .char_indices() .take_while(|&(i, _)| i < max_len) .last() .map_or(0, |(i, c)| i + c.len_utf8()); let boundary = trimmed[..boundary].rfind(' ').unwrap_or(boundary); format!("{}...", &trimmed[..boundary]) }</code></pre> The rfind(' ')</code> prefers breaking at a word boundary — nicer excerpts for the explorer. The Parrot's Perch</a></h2> Now the interactive chat — you ask, the crabs dive, the parrot answers: // src/bin/chat.rs #![allow(clippy::print_stdout, clippy::print_stderr)] use std::io::{BufRead, Write}; use std::path::Path; use std::sync::mpsc; use std::time::Duration; use owo_colors::OwoColorize; use mini_rag::{DB_PATH, db, embed, rag};</code></pre> RAG queries take a few seconds — the crabs need to dive and the parrot needs to reason. A blank terminal during that time feels broken. A braille spinner gives the explorer something to watch: const SPINNER_FRAMES: &[char] = &['⠋', '⠙', '⠹', '⠸', '⠼', '⠴', '⠦', '⠧', '⠇', '⠏']; fn start_spinner(message: &str) -> mpsc::Sender<()> { let (tx, rx) = mpsc::channel(); let msg = message.to_owned(); std::thread::spawn(move || { let mut i = 0; #[allow(clippy::indexing_slicing)] loop { print!( "\r\x1b[2K{} {msg}", SPINNER_FRAMES[i % SPINNER_FRAMES.len()].yellow() ); std::io::stdout().flush().ok(); if rx.recv_timeout(Duration::from_millis(80)).is_ok() { break; } i += 1; } print!("\r\x1b[2K"); std::io::stdout().flush().ok(); }); tx }</code></pre> The spinner runs on a separate OS thread (not a tokio task) so it keeps animating even when the async runtime is busy waiting for the LLM response. Expedition tip: Our hand-rolled spinner is ~20 lines. For a more polished expedition, consider indicatif</a> (progress bars, spinners, multi-bar support) or spinners</a> (80+ spinner styles, minimal API). We kept ours dependency-free since it's the only place we need it. </blockquote> And the main loop: #[tokio::main] async fn main() -> anyhow::Result<()> { tracing_subscriber::fmt::init(); let conn = db::init_db(Path::new(DB_PATH)).await?; let client = mini_rag::ollama_client()?; let embedding_model = embed::embedding_model(&client); let stdin = std::io::stdin(); let mut stdout = std::io::stdout(); println!( "{} — ask questions about your ingested documents", "rag-chat".bold().cyan() ); println!("Type {} or {} to leave.\n", "exit".dimmed(), "quit".dimmed()); loop { print!("{} ", ">".bold().green()); stdout.flush()?; let mut line = String::new(); let bytes_read = stdin.lock().read_line(&mut line)?; if bytes_read == 0 { break; // EOF — the explorer has left the island } let question = line.trim(); if question.is_empty() || question == "exit" || question == "quit" { break; } let spinner = start_spinner("Searching and thinking..."); let result = rag::query( &client, &embedding_model, &conn, question, ).await; spinner.send(()).ok(); std::thread::sleep(Duration::from_millis(10)); match result { Ok(response) => println!("\n{response}"), Err(e) => eprintln!("\n{} {e}\n", "Error:".red().bold()), } } Ok(()) }</code></pre> Notice how println!("\n{response}")</code> calls the Display</code> impl we wrote on RagResponse</code> — the answer prints first, followed by the colored source list. The owo-colors</code> crate handles terminal colors with zero allocation — .bold()</code>, .green()</code>, .dimmed()</code> are all zero-cost method chains. Going further — better CLI: For a permanent settlement, consider rustyline</a> for readline support (history, tab completion at the perch), ratatui</a> for a full TUI, or clap</a> for proper argument parsing (--db-path</code>, --chunk-size</code>). Our simple stdin loop works fine for the expedition. </blockquote> Talking to the Rust Books</a></h2> $ cargo run --bin rag-chat rag-chat — ask questions about your ingested documents Type exit or quit to leave. > What is the newtype pattern in Rust? ⠹ Searching and thinking... Ahoy there, human! 🦜 Let's talk about the **newtype pattern**! *squawk!* The newtype pattern wraps an existing type in a single-field tuple struct: pub struct UserId(u64); This gives it a distinct type identity — you can't accidentally mix a UserId with a raw u64. BRAWWK! Why use it? 🦀 - Enforces invariants: no accidental type mixing - Hides implementation: users don't see the wrapped type - Orphan rule workaround: implement foreign traits on foreign types Rusties love this pattern for building robust systems. 🏝️ *ruffles feathers* ─── Sources ─── [1] comprehensive-rust (comprehensive-rust.pdf) "The newtype pattern wraps an existing type in a single-field tuple..." [2] rust-design-patterns (rust-design-patterns.pdf) "Newtype is a pattern that provides strong type distinctions..." > exit</code></pre> The parrot answers with flair — and the sources let the explorer verify against the original scrolls. Note that the parrot's response contains markdown formatting (headers, code blocks, bold). In a production system, you'd render this nicely with a crate like termimad</a> or termcolor-markdown</a>. Our humble println!</code> does the job for now. We're chatting with the Comprehensive Rust book and Rust Design Patterns — locally, on our island, with no cloud parrots, with answers grounded in the actual treasure contents, and with source attribution so the explorer can verify. The crabs did their job beautifully. The Expedition Manifest</a></h2> Let's count what we packed: Module</th> Lines</th> Island Role</th></tr></thead> extract.rs</code></td> 8</td> The bottle opener</td></tr> chunk.rs</code></td> 58</td> The coconut cracker (+ 72 lines of tests)</td></tr> embed.rs</code></td> 30</td> The Pearl Forge</td></tr> db.rs</code></td> 150</td> The Smart Beach</td></tr> rag.rs</code></td> 128</td> The parrot whisperer (+ 47 lines of tests)</td></tr> lib.rs</code></td> 15</td> The base camp</td></tr> bin/sync.rs</code></td> 142</td> The gathering outpost</td></tr> bin/chat.rs</code></td> 84</td> The parrot's perch</td></tr> Total</td> ~734</td> (~615 without tests)</td></tr> </tbody></table> About 615 lines of actual code. A fully functional, snake-free, local-first RAG system built entirely on Crab Island. Future Expeditions</a></h2> This island has more to explore. Some trails for the adventurous: Better coconut cracking: Sentence-aware splitting (crack on .</code> boundaries, group to size)</li> Heading-aware splitting (if your scrolls have markdown structure)</li> text-splitter</a> for semantic chunking with tiktoken</li> </ul> Smarter diving: Hybrid search: combine pearl diving (vectors) with word matching (FTS5) on the same beach</li> Re-ranking: use a second model to re-score the top results</li> Metadata filtering: dive only in certain sections of the beach</li> </ul> Island infrastructure: Turso</a> cloud sync: replicate your beach to the cloud (backup!)</li> Swap parrots: point rig-core at OpenAI or Anthropic — one line change</li> Build an MCP server</a>: let AI agents from other islands query your beach</li> </ul> Creature comforts: rustyline</a> for readline support (history, tab completion at the perch)</li> ratatui</a> for a full TUI — the parrot deserves a palace</li> Streaming responses (rig-core supports it — watch the parrot think in real time)</li> </ul> Farewell from Crab Island</a></h2> We started this expedition wondering: can you build a RAG system without snakes? The answer is a resounding yes — and the island is more pleasant than expected. We got type-safe pipelines where the island's guardian catches wiring mistakes before we waste time. We got anyhow</code> keeping our error handling lean and context-rich. We got a single-file Smart Beach with both relational queries and vector search. And we got it all in ~615 lines of code, running entirely on our own island. The Python continent across the sea has more established trade routes. You won't find a LangChain bazaar here with 500 integrations. But what you will find on Crab Island are well-crafted tools — rig-core, libsql, kreuzberg — that compose beautifully and let you build exactly what you need. The crabs are happy. The parrot is wise. And there's not a snake in sight. The full expedition code lives in the mini-rag</code> directory. Clone the island, pull your parrot models (ollama pull nomic-embed-text && ollama pull deepseek-r1</code>), sync some treasures, and start asking questions. Happy treasure hunting! 🤿 Part 5: The Pearl Divers — Implementing Vector Search 2026-03-12T00:00:00+00:00 The Crab Island RAG Expedition — Part 5 of 6 The beach is full of buried pearls. Hundreds of them — 758 from the Comprehensive Rust scroll alone, each one a shimmering 768-faceted gem encoding a coconut piece's meaning. Now comes the critical question: when someone asks "What is Rust's ownership model?", how do the crabs find the right pearls? They can't dig up every single one and compare. That would take forever. Instead, they need a diving technique — a way to take the question, forge a question-pearl from it, and then dive straight to the pearls that glow most similarly. About 30 lines. And once we write them, the expedition is nearly complete. The Diving Technique</a></h2> The dive has three steps: Forge a question-pearl — embed the user's question using the same Forge (nomic-embed-text)</li> Dive the beach — ask libsql's vector_top_k</code> for the closest pearls</li> Surface with results — return the matching text chunks with source metadata</li> </ol> Here's our search function: // src/db.rs (continued) #[derive(Debug)] pub struct SearchHit { pub score: f64, pub content: String, pub doc_title: String, pub doc_path: String, } pub async fn vector_search( conn: &Connection, query_embedding: &[f32], top_k: usize, ) -> anyhow::Result<Vec<SearchHit>> { let embedding_json = serde_json::to_string(query_embedding)?; let sql = format!( "SELECT c.content, d.title, d.source_path \ FROM vector_top_k('idx_chunks_embedding', vector32(?1), {top_k}) AS v \ JOIN chunks AS c ON c.rowid = v.id \ JOIN documents AS d ON d.id = c.document_id" ); let mut rows = conn.query(&sql, params![embedding_json]).await?; let mut results = Vec::new(); let mut rank = 0_u32; let total = u32::try_from(top_k).unwrap_or(u32::MAX); while let Some(row) = rows.next().await? { let content = row.get::<String>(0)?; let doc_title = row.get::<String>(1)?; let doc_path = row.get::<String>(2)?; let score = 1.0 - f64::from(rank) / f64::from(total); results.push(SearchHit { score, content, doc_title, doc_path, }); rank += 1; } Ok(results) }</code></pre> A few things to unpack. The JOIN: Pearls with Provenance</a></h2> Notice the JOIN documents</code> — we fetch not just the chunk content, but also which document it came from (title</code>, source_path</code>). This is critical for source attribution: when the parrot answers a question, the explorer can see where the answer came from. The SearchHit</code> struct carries everything we need: the matching text, a relevance score, and the document metadata. Reef #1: The Beach Doesn't Tell Distance</a></h2> This one puzzled us for a while. libsql's vector_top_k()</code> returns results ordered by similarity — the closest pearls first. Great! But it does not tell you how close they are. There's no distance column. No similarity score. If you try SELECT v.distance FROM vector_top_k(...)</code>: SqliteFailure: no such column: distance</code></pre> The beach knows the order, but keeps the measurements to itself. Our workaround: rank-based scoring. let score = 1.0 - f64::from(rank) / f64::from(total);</code></pre> With top_k = 5</code>: first pearl scores 1.0</code>, second 0.8</code>, third 0.6</code>, fourth 0.4</code>, fifth 0.2</code>. The actual numbers are cosmetic — our results are already in the right order. The crabs find the right pearls regardless. Expedition tip: If you need true distance scores, consider usearch</a> — an embeddable single-header HNSW library with Rust bindings. Like libsql, it's in-process (no server), but it returns actual cosine distances and supports custom metrics. For a full-blown search server, qdrant</a> or lancedb</a> are solid options. We chose our Smart Beach (libsql) for its simplicity — one file handles both storage and search — and rank-based scoring is an acceptable trade-off. </blockquote> Reef warning — no rejection of irrelevant pearls: Without actual distance scores, we can't tell whether a result is genuinely close or just the least far away. Ask "What's a good pasta recipe?" and the crabs will still surface 5 pearls — they'll just be the least irrelevant Rust chunks. A production system would want a similarity threshold to reject results that are too far from the question. With libsql's vector_top_k</code>, that's not possible today. With usearch, qdrant, or lancedb, you'd filter on score > 0.7</code> (or whatever threshold fits your data). Something to keep in mind if your treasure hoard might receive off-topic questions. </blockquote> Reef #2: top_k</code> Can't Be Parameterized</a></h2> Notice {top_k}</code> is interpolated into the SQL string, not passed as ?2</code>. That's because vector_top_k</code>'s second argument must be a literal integer in the SQL — it won't accept a parameter. Since top_k</code> is always a usize</code> from our own code (never user input), this is safe. But it's another reef to be aware of. Wiring It Into the RAG Query</a></h2> With vector_search</code> in hand, our RAG module can stay refreshingly simple — no traits, no generics, just a direct pipeline: // src/rag.rs use libsql::Connection; use rig::client::CompletionClient; use rig::completion::Prompt; use rig::providers::ollama; use crate::db; use crate::embed; use crate::DEEPSEEK_R1; const TOP_K: usize = 5; pub async fn query( client: &ollama::Client, embedding_model: &ollama::EmbeddingModel, conn: &Connection, question: &str, ) -> anyhow::Result<RagResponse> { // 1. Forge a question-pearl let query_embedding = embed::embed_text(embedding_model, question).await?; // 2. Dive for the top 5 pearls let hits = db::vector_search(conn, &query_embedding, TOP_K).await?; // 3. Build context and collect sources in one pass let mut context = String::new(); let mut sources = Vec::with_capacity(hits.len()); for (i, hit) in hits.into_iter().enumerate() { if !context.is_empty() { context.push_str("\n\n"); } let _ = write!(context, "[{}] {}", i + 1, &hit.content); sources.push(Source { excerpt: truncate_to_excerpt(&hit.content, EXCERPT_LEN), title: hit.doc_title, path: hit.doc_path, }); } // 4. Ask the parrot with explicit context let agent = client .agent(DEEPSEEK_R1) .preamble( "You are Kwaak 🦜, the wise parrot of Crab Island. \ You speak with parrot flair — sprinkle in the occasional \ *squawk!*, *BRAWWK!*, or *ruffles feathers*, and use \ emojis like 🦀🥥🏝️. But under the plumage, you are \ precise and knowledgeable. \ Use the provided context to answer accurately. If the \ context doesn't contain enough information, squawk \ honestly — never make up answers. Keep answers concise.", ) .build(); let prompt = format!("Context:\n{context}\n\nQuestion: {question}"); let response = agent.prompt(prompt).await?; Ok(RagResponse { answer: strip_think_tags(&response), sources, }) }</code></pre> No VectorStoreIndex</code> trait, no dynamic_context</code>, no serde::Deserialize</code> gymnastics. Just: embed the question → search the beach → build a prompt → ask the parrot. Four steps, all explicit, all debuggable. Going further — rig's dynamic_context</code>: rig-core has a VectorStoreIndex</code> trait and a dynamic_context</code> builder method that automates the embed-search-inject pipeline. You implement the trait for your database, and rig handles the orchestration. It's elegant when you have multiple vector stores or want rig to manage the context window. But it requires implementing two async trait methods with serde::Deserialize</code> bounds, and for our PoC the direct approach is simpler, more transparent, and gives us full control over source attribution. The trait earns its keep in larger systems — start direct, add the abstraction when you need it. </blockquote> Alternative Diving Techniques</a></h2> Our vector_search</code> is purpose-built for the Smart Beach. Other options: usearch</a>: An embeddable HNSW index with Rust bindings. In-process like libsql, but returns actual distances and supports custom metrics. A good middle ground between our simple approach and a full search server.</li> rig's built-in integrations: rig-core has optional support for MongoDB, LanceDB, Neo4j, and more. Check the rig docs</a> — someone may have already built a diving crew for your preferred beach.</li> In-memory diving: For small treasure hoards, skip the beach entirely. Compute cosine similarity over a Vec<Vec<f32>></code> in memory. Fast for development, won't scale.</li> Hybrid search: Combine pearl diving (vector search) with word matching (FTS5 in SQLite). Use vector search for meaning and FTS for exact keywords. libsql supports both on the same beach.</li> </ul> Beach Report</a></h2> In ~30 lines of search code plus ~40 lines of RAG wiring, our diving crew can: Take any question and forge a question-pearl</li> Navigate the f64/f32 reef at the boundary</li> Dive the Smart Beach using vector_top_k</code> with source metadata</li> Surface with ranked results including document provenance</li> Build explicit context for the parrot — no magic, full control</li> </ul> In the final post, we bring the whole expedition together. Two binaries, one database, and a conversation with our Rust books. 💎 Part 4: The Pearl Forge — Embeddings and Vector Storage 2026-03-11T00:00:00+00:00 The Crab Island RAG Expedition — Part 4 of 6 Deep in the heart of Crab Island, past the coconut groves and through the bamboo thicket, lies the Pearl Forge. This is where the real magic happens. The crabs bring their coconut pieces here, and the Forge transforms each one into a pearl — a shimmering 768-faceted gem that encodes the meaning of the text. Two pieces about similar topics produce pearls that glow in similar ways. Two pieces about unrelated topics produce pearls that look completely different. And the beach? It's no ordinary beach. It's a smart beach — powered by libsql — that remembers where every pearl is buried and can instantly find the ones most similar to any new pearl you show it. SQLite learned to understand meaning. Let's see how. How the Pearl Forge Works</a></h2> An embedding is a list of numbers (a vector) that captures the meaning of text. "How do I handle errors in Rust?" and "What's the best way to manage failures?" produce vectors that are close together in 768-dimensional space, even though they share almost no words. The Forge is powered by nomic-embed-text</code> running locally via Ollama. Here's how we operate it: // src/embed.rs use rig::client::EmbeddingsClient; use rig::embeddings::EmbeddingModel as _; use rig::providers::ollama; const NOMIC_EMBED_TEXT: &str = "nomic-embed-text"; pub const EMBEDDING_DIMS: usize = 768; #[must_use] pub fn embedding_model(client: &ollama::Client) -> ollama::EmbeddingModel { client.embedding_model_with_ndims(NOMIC_EMBED_TEXT, EMBEDDING_DIMS) } pub async fn embed_text( model: &ollama::EmbeddingModel, text: &str, ) -> anyhow::Result<Vec<f32>> { let embedding = model.embed_text(text).await?; Ok(to_f32(&embedding.vec)) } pub async fn embed_texts( model: &ollama::EmbeddingModel, texts: impl IntoIterator<Item = String> + Send, ) -> anyhow::Result<Vec<Vec<f32>>> { let embeddings = model.embed_texts(texts).await?; Ok(embeddings.iter().map(|e| to_f32(&e.vec)).collect()) } // rig returns Vec<f64>, libsql F32_BLOB needs f32 #[allow(clippy::cast_possible_truncation)] fn to_f32(vec: &[f64]) -> Vec<f32> { vec.iter().map(|&v| v as f32).collect() }</code></pre> Two functions: embed_text</code> for single queries (the chat side), embed_texts</code> for batch ingestion (the sync side — we'll use this in Part 6 to speed things up). Both share a to_f32</code> helper that handles the f64→f32 conversion. Wait. Did you notice that suspicious cast in to_f32</code>? The Reef of Mismatched Gems</a></h2> Every island has a hidden reef, and this is ours — the f64/f32 mismatch. rig-core produces pearls with 64-bit precision (Vec<f64></code>)</li> libsql stores pearls with 32-bit precision (F32_BLOB</code>)</li> </ul> So every time the Forge creates a pearl, we have to file it down from f64 to f32. And every time we search, same thing. The precision loss is negligible for similarity search (pearl facets are typically between -1.0 and 1.0), but Clippy — the island's quality inspector — flags it immediately. We handle it with #[allow(clippy::cast_possible_truncation)]</code>. This is one of those cases where the cast is intentional, the inspector is noted, and life goes on. Expedition tip: You could store pearls in full f64 precision, but libsql's vector functions are optimized for f32. The storage savings are real too: 768 dimensions × 4 bytes = 3KB per pearl vs. 6KB. With thousands of pearls buried on the beach, that adds up to a lighter island. </blockquote> Other Pearl Forge settings: Model</th> Facets (dims)</th> Notes</th></tr></thead> nomic-embed-text</code></td> 768</td> Our pick — good balance of quality and size</td></tr> mxbai-embed-large</code></td> 1024</td> More facets, higher fidelity</td></tr> all-minilm</code></td> 384</td> Fewer facets, faster forging</td></tr> snowflake-arctic-embed</code></td> 1024</td> Great for larger pieces of text</td></tr> </tbody></table> Switching is easy — just change NOMIC_EMBED_TEXT</code> and EMBEDDING_DIMS</code>. The rest of the expedition doesn't care which Forge setting you use. The Smart Beach</a></h2> Now let's design our beach. Two tables, one index: // src/db.rs use std::path::Path; use anyhow::Context; use libsql::{Builder, Connection, params}; pub async fn init_db(path: &Path) -> anyhow::Result<Connection> { let db = Builder::new_local(path).build().await?; let conn = db.connect()?; conn.execute_batch( "CREATE TABLE IF NOT EXISTS documents ( id INTEGER PRIMARY KEY AUTOINCREMENT, title TEXT NOT NULL, source_path TEXT NOT NULL, mtime INTEGER NOT NULL DEFAULT 0, created_at TEXT NOT NULL DEFAULT (datetime('now')) ); CREATE TABLE IF NOT EXISTS chunks ( id INTEGER PRIMARY KEY AUTOINCREMENT, document_id INTEGER NOT NULL REFERENCES documents(id), chunk_index INTEGER NOT NULL, content TEXT NOT NULL, embedding F32_BLOB(768) ); CREATE INDEX IF NOT EXISTS idx_chunks_embedding ON chunks(libsql_vector_idx(embedding, 'metric=cosine'));", ) .await?; Ok(conn) }</code></pre> Let's explore the beach: documents</code> table: The registry — which treasures have we processed? The mtime</code> column is clever: it records when the source file was last modified, so we can skip treasures we've already processed. chunks</code> table: The pearl grid — each pearl has its content (the original coconut piece text) and its embedding (F32_BLOB(768)</code> — 768 facets of 32-bit shimmer). libsql_vector_idx(embedding, 'metric=cosine')</code>: This is the beach's magic — a cosine similarity index. When a crab dives looking for pearls, this index tells it which direction to swim. Cosine similarity measures the angle between two pearl facet patterns — the standard metric for text embeddings. The whole beach lives in a single file. No server, no infrastructure, no cloud. Just a .db</code> file on your island. Burying Pearls</a></h2> pub async fn insert_document( conn: &Connection, title: &str, source_path: &str, mtime: i64, ) -> anyhow::Result<i64> { conn.execute( "INSERT INTO documents (title, source_path, mtime) VALUES (?1, ?2, ?3)", params![title, source_path, mtime], ) .await?; let mut rows = conn.query("SELECT last_insert_rowid()", params![]).await?; let row = rows .next() .await? .context("failed to get last insert rowid")?; let id = row.get::<i64>(0)?; Ok(id) }</code></pre> Notice .context("failed to get last insert rowid")?</code> — anyhow's way of turning an Option::None</code> into a descriptive error. Much cleaner than constructing a libsql::Error</code> by hand. And burying each pearl — notice how the embedding gets JSON-serialized and passed through vector32()</code>: pub async fn insert_chunk( conn: &Connection, document_id: i64, chunk_index: usize, content: &str, embedding: &[f32], ) -> anyhow::Result<()> { let embedding_json = serde_json::to_string(embedding)?; conn.execute( "INSERT INTO chunks (document_id, chunk_index, content, embedding) \ VALUES (?1, ?2, ?3, vector32(?4))", params![ document_id, i64::try_from(chunk_index).expect("chunk index fits i64"), content, embedding_json ], ) .await?; Ok(()) }</code></pre> vector32(?4)</code> is libsql's pearl press — it takes a JSON array of floats ([0.123, -0.456, ...]</code>) and compresses it into a binary vector blob. Serialize with serde, pass as text, let the beach handle the rest. The Clever Crabs: Idempotent Ingestion</a></h2> No crab wants to re-forge 758 pearls for a coconut they already processed yesterday. We use file modification time (mtime</code>) for cheap change detection: pub async fn find_document_by_path( conn: &Connection, source_path: &str, ) -> anyhow::Result<Option<(i64, i64)>> { let mut rows = conn .query( "SELECT id, mtime FROM documents WHERE source_path = ?1", params![source_path], ) .await?; match rows.next().await? { Some(row) => { let id = row.get::<i64>(0)?; let mtime = row.get::<i64>(1)?; Ok(Some((id, mtime))) } None => Ok(None), } } pub async fn delete_document( conn: &Connection, doc_id: i64, ) -> anyhow::Result<()> { conn.execute("DELETE FROM chunks WHERE document_id = ?1", params![doc_id]) .await?; conn.execute("DELETE FROM documents WHERE id = ?1", params![doc_id]) .await?; Ok(()) }</code></pre> The crab's decision tree: "Have I seen this bottle before?" → Check find_document_by_path</code></li> Same mtime? → "Already processed, moving on!" (skip)</li> Different mtime? → "Bottle was refilled!" → dig up old pearls, forge new ones</li> Never seen it? → "Fresh treasure!" → forge and bury</li> </ol> Expedition tip: For a permanent settlement, you might use content hashing (SHA256) instead of mtime — it's more reliable across file systems and backups. But for our expedition, mtime is simple, fast, and does the job. </blockquote> Expedition tip — schema migrations: Our CREATE TABLE IF NOT EXISTS</code> works fine for a fresh expedition. But a real settlement would need proper schema migration — adding columns, renaming tables, evolving the beach layout over time. Tools like refinery</a> or sqlx-migrate</a> handle this elegantly. For our adventure, we just delete the database and re-sync when the schema changes. </blockquote> Note — the SmallVec</code> temptation: If you're a Rust performance enthusiast, you might think: "768 elements, always the same size — perfect for SmallVec<[f32; 768]></code> to avoid heap allocation!" Resist! A SmallVec</code> with N=768</code> puts 3KB on the stack (768 × 4 bytes</code>). That's unusually large for a stack allocation, and in practice SmallVec</code> will spill to the heap anyway once you pass it around. SmallVec</code> shines for tiny collections (1–8 elements) where avoiding a heap allocation actually matters. For embeddings, a plain Vec<f32></code> is simpler and equally fast. Don't fight the allocator when it's already doing a great job. </blockquote> Beach Report</a></h2> The Pearl Forge and Smart Beach are operational: Embedding generation: text → 768-faceted pearl, forged locally via Ollama</li> Database schema: two tables with a cosine similarity index</li> CRUD operations: register treasures, bury pearls, dig up old ones</li> Idempotent ingestion: clever crabs skip what they've already processed</li> </ul> The pearls are buried. Now we need to teach the crabs how to find the right ones. In the next post, we implement vector search — the diving technique that lets crabs find pearls by meaning. 🥥 Part 3: Cracking Coconuts — Chunking Text 2026-03-10T00:00:00+00:00 The Crab Island RAG Expedition — Part 3 of 6 The crabs have gathered on the beach, claws clicking in anticipation. Before them lies the extracted text — 348,000 characters from the Comprehensive Rust scroll, 85,000 from Rust Design Patterns. That's a lot of text. Now, a crab can't carry an entire coconut. It needs to crack it into pieces — small enough to handle, but large enough to still be nutritious. Too small, and you get dust. Too large, and nobody can carry it. And here's the trick: you want the pieces to overlap slightly, so that any juicy bit sitting right on the crack line appears in both pieces. This is chunking. And it's quietly the most important part of the entire expedition. Why Cracking Matters So Much</a></h2> Pop quiz: what's the single biggest factor in RAG answer quality? Not the parrot's vocabulary. Not the embedding model. Not the database. It's how you crack your coconuts. Feed the parrot an entire 10-page chapter and it'll give you a vague, meandering answer. Feed it the exact right paragraph and it'll nail it. Chunking is the art of breaking text into pieces that are small enough to be precise, but large enough to carry meaning. The Sliding Window: A Crab's Best Friend</a></h2> There are many cracking strategies: Strategy</th> Idea</th> Complexity</th></tr></thead> Fixed-size</td> Crack every N characters</td> Simple</td></tr> Sentence-based</td> Crack on sentence boundaries</td> Medium</td></tr> Semantic</td> Crack on topic changes</td> Complex (needs a model!)</td></tr> Recursive</td> Try big cracks, fall back to smaller</td> Medium</td></tr> </tbody></table> We're going with fixed-size with overlap — the simplest technique that actually works well. The overlap is the secret sauce: it ensures that any important idea sitting right on a crack line appears in both pieces. Picture a crab walking along the coconut with a measuring shell: Text: [=======piece 1========] [=======piece 2========] [=======piece 3========] ←---→ overlap</code></pre> Each piece is 500 characters. Each new piece starts 450 characters after the previous one (500 - 50 overlap). A sentence that gets cracked at position 490? It still appears fully in the next piece. The Code</a></h2> // src/chunk.rs #[derive(Debug)] pub struct Chunk { pub index: usize, pub content: String, } #[must_use] pub fn chunk_text(text: &str, chunk_size: usize, overlap: usize) -> Vec<Chunk> { let chars: Vec<char> = text.chars().collect(); let overlap = overlap.min(chunk_size / 2); let step = chunk_size - overlap; if chars.len() <= chunk_size { let trimmed = text.trim(); if trimmed.is_empty() { return Vec::new(); } return vec![Chunk { index: 0, content: trimmed.to_owned(), }]; } let mut chunks: Vec<_> = chars .windows(chunk_size) .step_by(step) .enumerate() .filter_map(|(i, window)| { let content: String = window.iter().collect(); let trimmed = content.trim(); (!trimmed.is_empty()).then(|| Chunk { index: i, content: trimmed.to_owned(), }) }) .collect(); // Handle the last chunk if it wasn't covered by windows let last_start = chunks.len() * step; if last_start < chars.len() { let content: String = chars.get(last_start..).unwrap_or_default().iter().collect(); let trimmed = content.trim(); if !trimmed.is_empty() { chunks.push(Chunk { index: chunks.len(), content: trimmed.to_owned(), }); } } // Re-index after filtering for (i, chunk) in chunks.iter_mut().enumerate() { chunk.index = i; } chunks }</code></pre> Let's examine the crab's technique: Why clamp the overlap? If someone passes an overlap larger than the chunk size, the subtraction chunk_size - overlap</code> would underflow and panic. Even an overlap close to the chunk size makes no sense — you'd be re-reading almost the entire piece each time. Clamping to chunk_size / 2</code> is a sensible guard: overlapping more than half a piece means you're doing more repeating than progressing. Why windows()</code> + step_by()</code>? Rust's slice method windows(n)</code> gives us a sliding window of size n</code>, advancing one element at a time. Combined with step_by(chunk_size - overlap)</code>, we get exactly the overlapping pieces we want — no manual index arithmetic needed. The standard library does the heavy lifting. Why chars()</code> instead of bytes? Because the island speaks many languages. If we split on bytes, we might slice a multi-byte character in half. Imagine cracking a coconut marked "Größe" (German for "size") right through the "ö" — we'd get invalid UTF-8. The crabs are more careful than that. Why handle the last chunk separately? windows()</code> only produces full-size windows. If the text doesn't divide evenly, the trailing piece (shorter than chunk_size</code>) needs special handling. Why skip empty pieces? Some coconuts have hollow sections — long stretches of whitespace from page breaks or formatting artifacts. No point carrying an empty shell. The crabs just toss those aside. Why #[must_use]</code>? If a crab cracks a coconut and nobody picks up the pieces, something has gone wrong. The island's guardian (the compiler) will raise an eyebrow. The Crabs Get to Work</a></h2> Let's watch them crack our Rust books: // Comprehensive Rust: ~348K characters let text = extract::extract_text( Path::new("comprehensive-rust.pdf"), ).await?; let chunks = chunk::chunk_text(&text, 500, 50); // => 758 pieces! // Rust Design Patterns: ~85K characters let text = extract::extract_text( Path::new("rust-design-patterns.pdf"), ).await?; let chunks = chunk::chunk_text(&text, 500, 50); // => 187 pieces</code></pre> 758 coconut pieces from a ~350-page course. Each piece is a paragraph-sized morsel of Rust knowledge — small enough to carry, meaningful enough to be useful. Choosing the Right Crack Size</a></h2> The magic numbers — 500 chars, 50 overlap — aren't carved in sacred coral. Here's how to think about them: Too small (e.g., 100 chars): Precise — you'll find exactly the right sentence</li> But context-free — "It's unsafe" means nothing without knowing what "it" refers to</li> More pieces = more work for the crabs, more pearls to bury</li> </ul> Too large (e.g., 2000 chars): Plenty of context in each piece</li> But diluted relevance — a piece about 5 different topics is vaguely relevant to all and strongly relevant to none</li> Embedding models have limits too (nomic-embed-text handles ~8192 tokens)</li> </ul> The overlap (50 chars): Too small: ideas at boundaries get lost in the crack</li> Too large: you're carrying near-duplicate pieces, wasting energy</li> 10% of chunk_size is a solid starting point</li> </ul> Expedition tip: These numbers are worth tuning for your specific treasure. Dense academic papers might want smaller pieces. Conversational text might want larger ones. Start with 500/50, test your answer quality, adjust. </blockquote> Testing the Crab's Work</a></h2> Here's something the crabs insist on — and you won't see in most treasure-hunting tutorials: actual tests. #[cfg(test)] #[allow(clippy::indexing_slicing)] mod tests { use super::*; #[test] fn empty_text_produces_no_chunks() { let chunks = chunk_text("", 500, 50); assert!(chunks.is_empty()); } #[test] fn short_text_produces_single_chunk() { let chunks = chunk_text("hello world", 500, 50); assert_eq!(chunks.len(), 1); assert_eq!(chunks[0].content, "hello world"); assert_eq!(chunks[0].index, 0); } #[test] fn text_is_split_with_overlap() { let text = "a".repeat(1000); let chunks = chunk_text(&text, 500, 50); assert_eq!(chunks.len(), 3); assert_eq!(chunks[0].content.len(), 500); assert_eq!(chunks[1].content.len(), 500); // Last chunk: 1000 - 900 = 100 chars assert_eq!(chunks[2].content.len(), 100); } #[test] fn chunks_have_sequential_indices() { let text = "x".repeat(2000); let chunks = chunk_text(&text, 500, 50); for (i, chunk) in chunks.iter().enumerate() { assert_eq!(chunk.index, i); } } #[test] fn overlap_clamped_to_half_chunk_size() { let text = "a".repeat(1000); // overlap 999 > chunk_size/2 (250), gets clamped to 250 let chunks = chunk_text(&text, 500, 999); let expected = chunk_text(&text, 500, 250); assert_eq!(chunks.len(), expected.len()); for (a, b) in chunks.iter().zip(expected.iter()) { assert_eq!(a.content, b.content); } } #[test] fn emoji_not_split_across_chunks() { // 🦀 is a single char (4 bytes), place it at the chunk boundary let text = format!("{}🦀{}", "a".repeat(4), "b".repeat(5)); // 10 chars total, chunk_size=5, overlap=1, step=4 let chunks = chunk_text(&text, 5, 1); assert_eq!(chunks.len(), 3); assert_eq!(chunks[0].content, "aaaa🦀"); assert_eq!(chunks[1].content, "🦀bbbb"); assert_eq!(chunks[2].content, "bb"); } #[test] fn whitespace_only_chunks_are_skipped() { let text = format!("{}{}content", "a".repeat(500), " ".repeat(500)); let chunks = chunk_text(&text, 500, 0); // First chunk: 500 'a's, second chunk: 500 spaces (skipped), third chunk: "content" assert_eq!(chunks.len(), 2); assert_eq!(chunks[1].content, "content"); } }</code></pre> Empty coconuts, tiny coconuts, the overlap math, overlap clamping, emoji safety, sequential numbering, hollow sections — all verified. The emoji test is a nice illustration: the 🦀 crab (4 bytes, but 1 char) sits right at the boundary and appears in both adjacent chunks thanks to the overlap — never sliced in half. Pure functions are a joy to test: no mocks, no async, no setup. Just crack → check</code>. cargo nextest run # 7 tests run: 7 passed</code></pre> The crabs are meticulous workers. Fancier Cracking Tools</a></h2> Our sliding-window approach is the machete of chunking — simple, reliable, gets the job done. When you're ready for finer tools: Sentence-aware cracking: Split on .</code> boundaries, then group sentences up to your size limit. Better coherence per piece.</li> text-splitter</a>: A Rust crate for semantic-aware text splitting with tiktoken integration. The power tool version.</li> Recursive cracking: Try paragraphs first, then sentences, then characters. What LangChain's RecursiveCharacterTextSplitter</code> does on the Python continent.</li> Heading-aware cracking: If your text has structure (markdown headers), split on those boundaries. Each piece gets a natural topic.</li> </ul> Note — kreuzberg can crack coconuts too: kreuzberg has a built-in chunking</code> feature that splits text during extraction. So why did we write our own? Two reasons. First, rolling your own chunker teaches you what's happening — it's ~60 lines of pure Rust with no magic. Second, separating extraction from chunking gives you flexibility: you can experiment with chunk sizes, test with unit tests, or swap in a smarter strategy later without touching extraction. In production, you might prefer kreuzberg's built-in chunking for convenience. </blockquote> But our 50-line cracker will carry you surprisingly far. The Comprehensive Rust book chunks up nicely, and our RAG system returns relevant answers. Upgrade the tool when the simple one stops working, not before. Beach Report</a></h2> The crabs have cracked all the coconuts: ~60 lines of pure, synchronous Rust</li> Zero dependencies — just String</code> and Vec</code></li> Unicode-safe — handles multi-byte characters without cracking through the middle</li> Tested — 7 tests covering every edge case</li> Configurable — piece size and overlap are parameters, not constants</li> </ul> The beach is covered in neatly numbered coconut pieces. But here's the thing about Crab Island — it's not an ordinary island. The coconuts here don't contain milk. Crack one open, and inside you'll find a pearl — a shimmering, dense little sphere that somehow captures the essence of the text it held. The crabs have known this for generations: every piece of knowledge, once properly cracked, reveals a pearl. Now it's time for the truly magical step — forging those pearls. Time to visit the pearl forge. 🍾 Part 2: Opening the Bottles — Text Extraction 2026-03-09T00:00:00+00:00 The Crab Island RAG Expedition — Part 2 of 6 Day two on Crab Island. The morning mist lifts and reveals the shoreline, littered with treasures from across the seven seas — sealed bottles with letters inside, scrolls wrapped in oilcloth, maps etched on strange flat stones. Some look ancient and weathered. Others are suspiciously well-formatted. These are our documents. PDFs, EPUBs, DOCX files — each one a different container holding knowledge we want to search through. Before the crabs can do anything useful with this treasure, we need to open it and get the text out. Here's the thing about these containers: they're all different. A bottle (PDF) isn't text at all — it's a set of page layout instructions. "Draw glyph 'H' at coordinates (72.4, 540.2). Draw glyph 'e' at (78.1, 540.2)." Turning that back into the word "Hello" is a feat of reverse engineering. A scroll (EPUB) is really a bundle of HTML pages wrapped in a zip file. A carved stone (DOCX) is XML wearing a trench coat. Fortunately, we don't have to crack each one by hand. We have kreuzberg — the island's universal bottle opener. kreuzberg: One Tool for Every Container</a></h2> kreuzberg</a> handles 75+ document formats through a single async function call. PDFs, EPUBs, DOCX, HTML, images (via OCR), plain text — hand it any container from the shore and it extracts what's inside. Here's our entire extraction module: // src/extract.rs use std::path::Path; use kreuzberg::ExtractionConfig; pub async fn extract_text(path: &Path) -> anyhow::Result<String> { let result = kreuzberg::extract_file(path, None, &ExtractionConfig::default()).await?; Ok(result.content) }</code></pre> Four lines of actual code. The crabs hand kreuzberg a container, it hands back the text inside. No fuss, no format-specific logic. extract_file</code> takes a path, an optional MIME type hint (we pass None</code> — "you figure it out"), and a config (we use defaults). The ?</code> works seamlessly because we use anyhow::Result</code>, which automatically converts any error implementing std::error::Error</code>. Cracking Open the Rust Books</a></h2> Let's try it on some real treasures. Imagine we've found two legendary scrolls on the beach: Comprehensive Rust</a> (Google's Rust course) and Rust Design Patterns</a>: let text = extract::extract_text( Path::new("comprehensive-rust.pdf"), ).await?; tracing::info!("Extracted {} characters", text.len()); // => Extracted 348572 characters</code></pre> That's 348,000 characters of Rust wisdom, liberated from its PDF prison. And the design patterns scroll: let text = extract::extract_text( Path::new("rust-design-patterns.pdf"), ).await?; tracing::info!("Extracted {} characters", text.len()); // => Extracted 85431 characters</code></pre> Same function, different containers, same result: pure text. If tomorrow a crab washes ashore with an EPUB or a DOCX, the same code handles it. One bottle opener to rule them all. The Hidden Reef: pdfium</a></h2> Every tropical island has a hidden reef, and here's ours. kreuzberg uses pdfium</a> under the hood for PDF extraction — the same C library Chrome uses to render PDFs. It needs to be available at runtime. If you try to open a PDF bottle and get: kreuzberg error: pdfium library not found</code></pre> ...you've hit the reef. You need to download libpdfium.dylib</code> (macOS) or libpdfium.so</code> (Linux) from the pdfium-binaries</a> releases and place it where the crabs can find it: # macOS (Apple Silicon) mkdir -p ~/lib && cp libpdfium.dylib ~/lib/</code></pre> Expedition tip: The kreuzberg crate has a pdf</code> feature that must be explicitly enabled — the default features don't include PDF support! Make sure your manifest says kreuzberg = { version = "4.5.1", features = ["pdf"] }</code>. There's also a bundled-pdfium</code> feature, but despite its promising name, it doesn't actually bundle pdfium. It just enables the pdf</code> feature. We all ran aground on that reef at least once. </blockquote> Why Plain Text?</a></h2> A keen-eyed explorer might ask: "We're throwing away structure! Headlines, bold text, tables — all gone. Isn't that wasteful?" Fair point! But consider: The crabs work with meaning, not formatting. Embedding models convert text to vectors. They don't know what bold means.</li> Coconut cracking is simpler. Our chunker (next post) uses a sliding window — it doesn't need to know about headings or paragraphs.</li> It's model-agnostic. Whether our parrot speaks nomic-embed-text, OpenAI's ada, or something else entirely, they all eat plain text.</li> </ol> Could we do better? Absolutely — preserve markdown structure, split on heading boundaries, keep tables intact. But that's optimization for a future expedition. The crabs on this island believe in starting simple. The Rust Design Patterns scroll itself would approve — start with a working solution, refine later. Going further — OCR: reading faded carvings: Some bottles on the beach contain images instead of text — scanned documents, photos of whiteboards, hand-drawn diagrams. kreuzberg can read those too via OCR (Optical Character Recognition), powered by Tesseract</a> or PaddleOCR</a>. Pass an image file to the same extract_file</code> function and it extracts whatever text it can find. You'll need to enable the ocr</code> or paddle-ocr</code> feature in Cargo.toml</code> and install the OCR engine on your system (brew install tesseract</code> on macOS for Tesseract). Results vary with image quality — a crisp scan yields great text, a blurry photo yields crab scratches. For the expedition we stick to digital documents, but if your treasure trove includes scanned PDFs or images, kreuzberg has you covered without any code changes. </blockquote> Other Bottle Openers</a></h2> kreuzberg is our pick, but the island market has alternatives: pdf-extract</a>: Pure Rust, PDF-only. Lighter to carry, no pdfium reef to navigate. Perfect if you only find PDF bottles on your beach.</li> lopdf</a>: Low-level PDF manipulation. For the explorer who wants to understand every byte of the container.</li> docx-rs</a>: DOCX-specific. When your shore is covered in carved stones and nothing else.</li> </ul> We chose kreuzberg because we expect all kinds of treasures to wash up — PDFs today, EPUBs and research papers tomorrow. One tool, one function, every format. Beach Report</a></h2> Our extraction outpost is ready: 4 lines of code (not counting imports)</li> Format-agnostic — PDFs, EPUBs, DOCX, HTML, and 70+ more</li> Async — won't block the crabs while they wait for a large PDF to process</li> Error-handled — kreuzberg errors flow naturally through anyhow</code> via ?</code></li> </ul> The bottles are open and the messages are spread out on the beach. But these aren't ordinary messages — they're treasure maps, and they all point to the same thing: a grove of coconut palms deeper on the island. The text inside those coconuts holds the real knowledge. But a crab can't carry a whole coconut — it needs to crack them into manageable pieces first. 🏕️ Part 1: Landfall — Setting Up Camp on Crab Island 2026-03-08T00:00:00+00:00 The Crab Island RAG Expedition — Part 1 of 6 Welcome, adventurer, to Crab Island. This is a lush tropical paradise — dense jungle canopy, white sand beaches, crystal-clear lagoons. The locals are crabs. Resourceful, industrious, type-safe crabs. And here's the best part: there are absolutely no snakes on this island. Not a single one. The crabs made sure of that ages ago. You've arrived here because you've heard rumors of ancient knowledge hidden across the island — treasure maps carved on coconut shells, messages sealed in bottles, scrolls tucked inside coral caves. You want to find that knowledge and ask questions about it. But the island is vast, and there's no way to read everything yourself. Fortunately, the crabs have a system. They can crack open any container, break the contents into pearl-sized pieces, bury those pearls in special spots on the beach, and later dive to find exactly the right ones when you have a question. They even have a wise old parrot who can synthesize answers from whatever pearls the crabs bring back. This, dear adventurer, is a RAG system. And we're going to build it in Rust. The Expedition Plan</a></h2> RAG (Retrieval-Augmented Generation) is just a treasure hunt with six steps: Extract — Open the bottles, unroll the scrolls, decipher the maps (get text from PDFs, EPUBs, etc.)</li> Chunk — Crack the coconuts into bite-sized pieces (split text into small passages)</li> Embed — Translate each piece into the island's musical language (convert text to numerical vectors)</li> Store — Bury each pearl in a labeled spot on the beach (save vectors in a database)</li> Retrieve — Send crabs diving for the pearls that match your question (vector similarity search)</li> Generate — Bring the pearls to the wise parrot, who weaves them into an answer (LLM generation)</li> </ol> PDF/EPUB/HTML → extract → chunks → embed → store (vector DB) ↓ query → embed → similarity search → LLM → answer</code></pre>Why This Island? (Why Rust?)</a></h2> The Python continent across the sea is wonderful — bustling ports, established trade routes, a bazaar with every tool imaginable (LangChain, LlamaIndex, ChromaDB). If you need something built fast with maximum community support, sail there. No judgment. The Python ecosystem for AI is genuinely incredible. But Crab Island offers something different: Type-safe pipelines: Wire things wrong and the island's guardian (the compiler) stops you before you waste an afternoon embedding the wrong data.</li> Error handling that doesn't lie: No surprise NoneType has no attribute 'content'</code> erupting from a swamp at 2am. Every failure path is mapped.</li> Single-artifact deployment: cargo build --release</code> produces one executable. Carry it anywhere. No virtualenv, no requirements.txt</code>.</li> Async by default: Tokio keeps our I/O humming while crabs work in parallel across the beach.</li> No GIL: When we need the crabs to truly work simultaneously, nothing holds them back.</li> </ul> The Expedition Gear</a></h2> Four crates form our toolkit — each one a trusty piece of island equipment: Crate</th> Role</th> Island Metaphor</th></tr></thead> rig-core</td> LLM abstraction</td> The wise parrot's translation manual</td></tr> libsql</td> Database + vectors</td> The beach where we bury and find pearls</td></tr> kreuzberg</td> Document extraction</td> The bottle opener / scroll unfurler</td></tr> Ollama</td> Local LLM runtime</td> The parrot itself, living on your island</td></tr> </tbody></table> Note that Ollama is external software (the LLM server), not a Rust crate — rig-core provides the Rust interface to it. Other gear you could pack: Instead of rig-core, you could use llm-chain</a> or call Ollama's REST API by hand. rig-core wins because it provides clean abstractions for both embeddings and completions, with built-in Ollama support.</li> Instead of libsql, you could bring a qdrant</a> or lancedb</a> server. We chose libsql because it's just a single file on the beach — no infrastructure, no server process, and it handles both relational data and vector search.</li> Instead of kreuzberg, you could use pdf-extract</a> for maps (PDFs) only. kreuzberg handles 75+ container types through a single function — bottles, scrolls, coconut carvings, you name it.</li> Instead of Ollama, you could summon a cloud parrot (OpenAI, Anthropic). We keep ours local so the entire expedition stays on the island.</li> </ul> Setting Up Base Camp</a></h2> Our camp has a library (shared knowledge) and two outposts — one for gathering treasures, one for asking questions: # Cargo.toml [package] name = "mini-rag" version = "0.1.0" edition = "2024" rust-version = "1.91.1" [lib] name = "mini_rag" path = "src/lib.rs" [[bin]] name = "rag-sync" path = "src/bin/sync.rs" [[bin]] name = "rag-chat" path = "src/bin/chat.rs" [dependencies] anyhow = "1" kreuzberg = { version = "4.5.1", features = ["pdf"] } kreuzberg-pdfium-render = "4.5.1" # pin to match kreuzberg 4.5.x (upstream version mismatch) libsql = "0.9.30" owo-colors = "4" rig-core = "0.33.0" serde_json = "1.0.149" tokio = { version = "1", features = ["macros", "rt-multi-thread"] } tracing = "0.1.44" tracing-subscriber = "0.3.23"</code></pre> Note: This project requires Rust 1.91.1+ (due to dependencies MSRV). Check with rustc --version</code>. </blockquote> The library holds all shared modules, plus a few helpers that both binaries need: // src/lib.rs use anyhow::Context; use rig::providers::ollama; pub mod chunk; pub mod db; pub mod embed; pub mod extract; pub mod rag; pub const DB_PATH: &str = "mini_rag.db"; pub const DEEPSEEK_R1: &str = "deepseek-r1"; pub fn ollama_client() -> anyhow::Result<ollama::Client> { ollama::Client::new(rig::client::Nothing) .context("failed to create Ollama client") }</code></pre> Five modules for the core logic, a shared database path, the model name, and a helper to create the Ollama client. Both binaries import from here instead of duplicating setup code. The Island's Danger Log</a></h2> Before we venture into the jungle, we need a way to record what goes wrong. On the Python continent, people sometimes use bare except:</code> blocks and hope for the best. On Crab Island, every ?</code> operator automatically captures and propagates errors with full context. We use anyhow</code></a> — a pragmatic error handling crate that lets us write anyhow::Result<T></code> everywhere and attach human-readable context with .context("what went wrong")</code>: // No error.rs needed — anyhow handles it all pub fn ollama_client() -> anyhow::Result<ollama::Client> { ollama::Client::new(rig::client::Nothing) .context("failed to create Ollama client") }</code></pre> The ?</code> operator works with any error type that implements std::error::Error</code> — io::Error</code>, libsql::Error</code>, kreuzberg::KreuzbergError</code> — they all flow through anyhow</code> automatically. No boilerplate, no manual From</code> impls. The danger log stays short and sweet. Going further — typed error enums: For a library where callers need to match</code> on specific error variants (retry on network error, abort on auth error), consider thiserror</code></a> or derive_more</code></a> to define a custom error enum. For our expedition — a PoC where we only display errors, never match on them — anyhow</code> is the right tool. </blockquote> Training the Parrot</a></h2> Before the expedition can begin, our parrot needs two skills: # Understanding language (converting text to meaning-vectors) ollama pull nomic-embed-text # Speaking wisdom (generating answers from context) ollama pull deepseek-r1</code></pre> nomic-embed-text</code> gives the crabs their sense of direction — it converts text into 768-dimensional coordinates so they know where to bury and find pearls. Alternatives like mxbai-embed-large</code> (1024 dims) or all-minilm</code> (384 dims) work too — just different maps of the same beach. deepseek-r1</code> is our parrot — a reasoning model that "thinks out loud" in <think></code> tags before speaking. We'll need to clean up its muttering later (spoiler for Part 6!), but the wisdom it delivers is excellent. The Expedition Ahead</a></h2> Here's our route across the island: Part 2: Open the bottles and unroll the scrolls — text extraction (it's 3 lines, seriously)</li> Part 3: Crack coconuts into pieces — the art of chunking</li> Part 4: Turn pieces into pearls and bury them on the beach — embeddings and vector storage</li> Part 5: Train the crabs to dive for the right pearls — vector search</li> Part 6: Ask the parrot — wire it all into sync and chat binaries</li> </ul> By Part 6, we'll have a fully working, snake-free RAG system. Grab your machete and let's head into the jungle.

Building a RAG System in Rust

🦜 Part 6: Asking the Parrot — The Full RAG Pipeline

🤿 Part 5: The Pearl Divers — Implementing Vector Search

The Diving Technique</a></h2> The dive has three steps:</p> Forge a question-pearl</strong> — embed the user's question using the same Forge (nomic-embed-text)</li> Dive the beach</strong> — ask libsql's vector_top_k</code> for the closest pearls</li>

💎 Part 4: The Pearl Forge — Embeddings and Vector Storage

🥥 Part 3: Cracking Coconuts — Chunking Text

🍾 Part 2: Opening the Bottles — Text Extraction

🏕️ Part 1: Landfall — Setting Up Camp on Crab Island