Building Production RAG

A client opens Briefcase and types "what did we decide about the homepage hero" into search. The decision exists: comment 12 on an approved deliverable says "going with option B, the full-bleed image." Your search is an ilike on '%homepage hero%', so the client gets zero rows and emails the account manager, who scrolls. Lesson 1.6 gave you the pipeline in theory and told you most products should not build it. This one should, and this appendix builds it end to end on the Part 4 capstone: semantic search and cited answers across deliverables and comments, gated by an eval, capped by a budget.

First, re-run the 1.6 decision honestly

Before a single migration: does Briefcase pass 1.6's gates? Not cleanly on corpus size. A young agency's comments might fit in a 200k window whole. What forces retrieval is the other two gates: end users fire queries at runtime and expect interactive latency, and the corpus is flat, thousands of comment fragments with no structure an agent could walk per question. Stuffing every org's history into a prompt on every search is a cost model, not a feature. So: RAG, chosen for runtime queries over a flat corpus with citations required, not because the acronym was available. Building on your own capstone instead? Re-run this paragraph against your corpus first.

The 1.6 pipeline, with real names on every box

You saw this figure in lesson 1.6 as theory. Here is the same figure with Briefcase's implementation mapped onto it, stage by stage:

• Chunking: lib/rag/chunk.ts. Deliverable titles and comment bodies, split to a few hundred tokens, each chunk prefixed with a deterministic metadata header (project, deliverable, author, status).
• Embedding: Voyage AI's voyage-4 over HTTPS, from a background job, never from a request handler.
• Storage and search: a search_chunks table in the same Postgres as everything else, a pgvector HNSW index for meaning plus a tsvector GIN index for keywords, fused with reciprocal rank fusion in one SQL function.
• Reranking: Voyage rerank-2.5-lite, wired but behind a flag, adopted only if the eval says it earns its latency.
• Generation: claude-sonnet-5 with the retrieved chunks passed as search_result content blocks, so citations come back as structured pointers to real deliverable and comment ids, not prose the model promised to write.
• The eval loop around the whole figure: 3.7's harness, retooled with golden queries and expected sources. The merge gate for every dial above.

One position up front, and it should sound familiar from 4.8: your vector database is the Postgres you already run. A dedicated vector store is a second place your data lives, a second failure mode, and a second wall to build around tenant data, in exchange for scale properties an agency workspace does not have. pgvector (0.8.4 as of July 2026) gives you exact and approximate nearest-neighbor search inside the database that already enforces your RLS wall and runs your pgTAP suite. The test that attacks deliverables can attack your vectors.

Second position: HNSW over IVFFlat. IVFFlat builds faster and uses less memory, but has a worse speed-recall tradeoff and wants data in the table before the index exists. HNSW costs more at build time, wins at query time, and works on an empty table in a migration, which is where you are.

The schema: vectors live behind the same wall

One migration, following every 4.5 convention: org_id on the tenant table, cascading FKs, RLS enabled in the same file that creates the table.

create extension if not exists vector;

-- One row per chunk. Sources are deliverables (title) and comments (body).
create table search_chunks (
id uuid primary key default gen_random_uuid(),
org_id uuid not null references orgs (id) on delete cascade,
project_id uuid not null references projects (id) on delete cascade,
source_table text not null
  check (source_table in ('deliverables', 'comments')),
source_id uuid not null,
chunk_index integer not null default 0,
content text not null,
fts tsvector generated always as (to_tsvector('english', content)) stored,
embedding vector(1024),          -- voyage-4 default dimension
embedding_model text,            -- series compatibility is not forever
created_at timestamptz not null default now(),
unique (source_table, source_id, chunk_index)
);

create index on search_chunks using gin (fts);
create index on search_chunks using hnsw (embedding vector_cosine_ops);
create index on search_chunks (org_id, project_id);
create index on search_chunks (source_table, source_id);

alter table search_chunks enable row level security;

-- Read: identical visibility to the source rows. Nothing more.
create policy search_chunks_select on search_chunks
for select to authenticated
using (private.can_see_project(project_id));
-- No insert/update/delete policies: only the service-role worker writes,
-- the same enforcement-by-absence as 4.5's approvals table.

-- Spend ledger + per-org ceiling (cents per calendar month).
create table ai_usage (
id uuid primary key default gen_random_uuid(),
org_id uuid not null references orgs (id) on delete cascade,
kind text not null check (kind in ('embed', 'rerank', 'generate')),
prompt_version text,
tokens integer not null,
estimated_cents numeric(10, 4) not null,
created_at timestamptz not null default now()
);
create index on ai_usage (org_id, created_at);
alter table ai_usage enable row level security;
create policy ai_usage_select on ai_usage
for select to authenticated
using (private.is_org_admin(org_id));

alter table orgs add column ai_budget_cents integer not null default 500;

create function private.org_ai_spend_cents(target_org uuid)
returns numeric
language sql
security definer
set search_path = ''
stable
as $$
select coalesce(sum(u.estimated_cents), 0)
from public.ai_usage u
where u.org_id = target_org
  and u.created_at >= date_trunc('month', now());
$$;

Three details carry weight. The select policy reuses private.can_see_project, 4.5's one correct definition of visibility, so a chunk of a comment is exactly as visible as the comment. The write policies do not exist, so under RLS only the service-role ingest path touches a vector. And embedding_model is a column, not an assumption: Voyage's 4-series embeddings are compatible with each other but not with other series, and the day you change models you must know which rows are stale.

Ingest: embed on write, through 4.8's jobs table

Embedding is a network call to a vendor that will occasionally have a bad two seconds. That sentence ends the debate: this is 4.8's definition of a job, not request work. Producers enqueue, claim_next_job claims, handlers retry with backoff and dead-letter at the cap. The only new decision is who enqueues, and the answer is the database itself, so no write path can ever forget:

-- Enqueue an embed job whenever source text changes. security definer,
-- because the jobs table denies clients under RLS (4.8) and the trigger
-- fires as whoever wrote the comment.
create function private.enqueue_embed_job()
returns trigger
language plpgsql
security definer
set search_path = ''
as $$
declare
src_text text;
begin
if tg_table_name = 'deliverables' then
  src_text := new.title;
else
  src_text := new.body;
end if;
insert into public.jobs (kind, payload, idempotency_key)
values (
  'embed_chunks',
  jsonb_build_object('source_table', tg_table_name, 'source_id', new.id),
  'embed:' || tg_table_name || ':' || new.id || ':' || md5(src_text)
)
on conflict (idempotency_key) do nothing;
return new;
end;
$$;

create trigger deliverables_embed
after insert or update of title on deliverables
for each row execute function private.enqueue_embed_job();
-- insert-only is deliberate: comments are immutable under 4.5's RLS
-- (no update policy), so there is no edit path to re-embed
create trigger comments_embed
after insert on comments
for each row execute function private.enqueue_embed_job();

-- Also register 'embed_chunks' as a jobs.kind value here, in whichever
-- form your TECH-SPEC encoded the first two kinds (enum value or check
-- constraint). New migration, never an edit to a committed one.

The idempotency key hashes the content, which does two jobs at once: a retried request enqueues nothing new (4.8's guarantee), and a save that did not change the text enqueues nothing at all, your first cost control. The handler is a plain 4.8 handler:

import { createServiceClient } from '@/lib/supabase/service'
import { chunkSource } from '@/lib/rag/chunk'
import { recordUsage, overBudget } from '@/lib/rag/budget'

const EMBED_MODEL = 'voyage-4' // 1024 dims, matches vector(1024)

export async function embedChunks(
payload: { source_table: 'deliverables' | 'comments'; source_id: string },
) {
const db = createServiceClient()
const source = await loadSource(db, payload) // row + org, project, title
if (await overBudget(source.org_id)) {
  throw new Error('org at AI budget ceiling') // retries after month reset
}

// Deterministic metadata header per chunk: 1.6's contextual-retrieval
// idea, bought with string concatenation instead of a model call.
const chunks = chunkSource(source)
// e.g. header: [Comment by Dana on deliverable "Homepage mockup",
//   project Website Redesign, status approved]

const res = await fetch('https://api.voyageai.com/v1/embeddings', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    Authorization: 'Bearer ' + process.env.VOYAGE_API_KEY,
  },
  body: JSON.stringify({
    input: chunks.map((c) => c.content),
    model: EMBED_MODEL,
    input_type: 'document', // documents at write time, 'query' at read time
  }),
})
if (!res.ok) throw new Error('voyage ' + res.status) // -> retry, backoff
const body = await res.json() // { data: [{ embedding, index }], usage }

// Replace, never append: stale chunks of an edited source must die here.
await db.from('search_chunks').delete().match(payload)
await db.from('search_chunks').insert(
  chunks.map((c, i) => ({
    org_id: source.org_id,
    project_id: source.project_id,
    ...payload,
    chunk_index: i,
    content: c.content,
    embedding: body.data[i].embedding,
    embedding_model: EMBED_MODEL,
  })),
)
await recordUsage(source.org_id, 'embed', body.usage.total_tokens)
}

Model choice, defended: as of July 2026 Voyage's current text models are the 4 series (voyage-4-large at $0.12 per million tokens, voyage-4 at $0.06, voyage-4-lite at $0.02, all 32k context, 1024 dimensions default). Take the middle: an agency's comment stream will not embed enough tokens for the lite tier's savings to matter, and the large tier is for corpora where retrieval quality is the product. Anthropic still ships no first-party embedding model; its docs point at Voyage, so the multi-vendor seam is permanent and belongs behind one swappable module.

Hybrid retrieval as one RPC

Lesson 1.6's argument for hybrid was exact identifiers: embeddings miss "TS-999". Briefcase's version is file names and invoice codes in comments ("logo_v3.fig", "BRF-2041"). The implementation is the Supabase hybrid-search pattern nearly verbatim, keyword and semantic CTEs fused with reciprocal rank fusion:

create function hybrid_search_chunks(
query_text text,
query_embedding vector(1024),
match_count int,
full_text_weight float = 1,
semantic_weight float = 1,
rrf_k int = 50
)
returns setof search_chunks
language sql
stable
as $$
with full_text as (
select id,
  row_number() over (
    order by ts_rank_cd(fts, websearch_to_tsquery(query_text)) desc
  ) as rank_ix
from search_chunks
where fts @@ websearch_to_tsquery(query_text)
limit least(match_count, 30) * 2
),
semantic as (
select id,
  row_number() over (order by embedding <=> query_embedding) as rank_ix
from search_chunks
where embedding is not null
limit least(match_count, 30) * 2
)
select search_chunks.*
from full_text
full outer join semantic on full_text.id = semantic.id
join search_chunks
on coalesce(full_text.id, semantic.id) = search_chunks.id
order by
coalesce(1.0 / (rrf_k + full_text.rank_ix), 0.0) * full_text_weight +
coalesce(1.0 / (rrf_k + semantic.rank_ix), 0.0) * semantic_weight
desc
limit least(match_count, 30)
$$;

The most important property of this function is what it does not say: no where org_id = ... anywhere, yet it is tenant-safe, because it runs as security invoker (the default), so every read of search_chunks inside it passes through the RLS policy from 0005. The caller matters just as much:

Reranking, behind a flag

Wire the reranker, do not adopt it by default. Voyage's rerank-2.5-lite (32k context, $0.02 per million tokens as of July 2026) is one HTTPS call: POST https://api.voyageai.com/v1/rerank with query, documents, model, and top_k, returning relevance-scored indexes into your candidates. Retrieval over-fetches 30, the reranker keeps 8. Lesson 1.6 reported Anthropic's benchmark case on a huge flat corpus; Briefcase's corpus is small and metadata-boosted by the chunk headers, so hybrid alone may already saturate your golden set. The position: at this corpus shape, hybrid is the default and the reranker is a measured upgrade. Flip RAG_RERANK=1, run the eval, keep it only if recall moves. A stage that costs latency on every query needs a number attached, not a vibe.

Generation: citations that point home

The last stage turns chunks into an answer that can prove itself. The Claude API supports exactly this shape: pass each retrieved chunk as a search_result content block with citations enabled, and the response interleaves text blocks with citation objects naming which block of which source supported which sentence.

import Anthropic from '@anthropic-ai/sdk'
import { ANSWER_PROMPT } from '@/lib/rag/prompts'
import { recordUsage } from '@/lib/rag/budget'

const client = new Anthropic()

// Rows from hybrid_search_chunks, already reranked and trimmed
type Chunk = {
source_table: 'deliverables' | 'comments'
source_id: string
content: string // first line is the metadata header from chunk.ts
}

export async function answerQuestion(
query: string, chunks: Chunk[], orgId: string,
) {
const results = chunks.map((c) => ({
  type: 'search_result' as const,
  // source is any identifier; ours round-trips to a real row
  source: 'briefcase://' + c.source_table + '/' + c.source_id,
  title: c.content.split('
')[0], // the chunk.ts metadata header
  content: [{ type: 'text' as const, text: c.content }],
  citations: { enabled: true },
}))

const res = await client.messages.create({
  model: 'claude-sonnet-5',
  max_tokens: 1024,
  system: ANSWER_PROMPT.text,
  messages: [{ role: 'user', content: [...results,
    { type: 'text' as const, text: query }] }],
})

await recordUsage(orgId, 'generate',
  res.usage.input_tokens + res.usage.output_tokens, ANSWER_PROMPT.id)

// Response text blocks carry citations: search_result_index points into
// our results array, so each citation resolves to a deliverable or
// comment id, and the UI renders a link that opens the real thing.
return res.content
}

Why the API feature instead of "please cite your sources" in the prompt: parsed citations are guaranteed to point at provided sources (a prompt-based citation can be hallucinated whole), cited_text does not count toward output tokens, and Anthropic's evals found the feature cites more relevant quotes than prompting. One real constraint: citations are incompatible with structured outputs, so this endpoint returns cited prose, not JSON. For Briefcase that is the product, and the digest job reuses the same function with a canned query per project.

This is where 1.6's silent-failure warning becomes a feature: the system prompt instructs the model to say "I could not find this in the workspace" when the sources do not answer. An uncited answer is a bug. The eval enforces that.

Prompts are code, spend has a ceiling

Two production disciplines finish the build, both small.

Prompt versioning, in-repo. The answer prompt lives at lib/rag/prompts.ts as an exported constant with an id: answer.v3 plus the text. It changes by adding answer.v4 in a PR, never by editing v3 in place, because v3's id is stamped into ai_usage.prompt_version and every eval report. When quality moves, you can name the prompt that moved it. A prompt living in a dashboard is a schema change made by clicking, the sin 4.5 banned.

Per-org cost ceilings, before launch. Every paid call writes ai_usage with estimated cents (as of July 2026: voyage-4 at 6 cents, rerank-2.5-lite at 2 cents per million tokens; claude-sonnet-5 at $3 in, $15 out per million). The check compares private.org_ai_spend_cents(org) against orgs.ai_budget_cents before spending, with a per-stage failure mode: search degrades to keyword-only (the FTS half of the RPC is free), embed jobs park and retry after the month resets, generation returns an honest "budget reached" state. Ceilings go in before launch, not after the first surprising invoice, because the thing that discovers a runaway loop should be your dashboard, not your card statement. A $5 default ceiling on a $49/mo customer is generous; an unlimited one is a business model with a hole in it.

The eval is the gate

Every dial above (chunk size, header format, weights, rrf_k, rerank, prompt version) is tunable, and 1.6 named what tuning without measurement is: five dials, blindfolded. The 3.7 harness transfers almost unchanged, golden queries with expected sources instead of golden repos with expected changelogs:

export interface GoldenQuery {
name: string
query: string           // what a real user would type
asUser: string          // fixture identity: staff or granted client
mustRetrieve: string[]  // source ids that must appear in top k
forbid?: string[]       // ids that must NEVER appear (other org's rows)
mustCite?: string[]     // ids the generated answer must cite
}

// Ten queries against the seeded fixture org, including:
// paraphrase ("hero image decision" vs the comment's actual wording),
// exact identifier ("BRF-2041"), client-visibility (granted project only),
// cross-tenant probe (forbid: agency B ids), and a no-answer query
// (mustCite empty: the model must decline, not improvise).

The runner embeds each query, calls the RPC as the fixture user, scores recall against mustRetrieve, then runs generation and checks cited source ids against mustCite. All code-based grading, per 3.7's hierarchy; no judge needed until you score answer prose. Threshold from a baseline of honest runs, as 3.7 taught, with one promotion: forbid is a hard per-case assertion. A tenancy miss fails the suite at any aggregate score, because "94 percent and leaking" is not a passing grade. This suite green is the definition of done for the lab below, rerun in CI on any change to lib/rag/, the prompts file, or the migrations.

Where it breaks

Stale embeddings. The trigger closes the classic hole (an edit path that forgets to re-embed), but the quiet version survives: embed jobs that fail repeatedly dead-letter, and a dead-lettered job is a comment that exists in the app but not in the index. Search does not error, it just cannot see the newest content, which users report as "search is bad" and never more precisely. Countermeasures: alert on dead-lettered embed_chunks jobs, and run a coverage query (sources with no chunks, chunks with null embeddings) in the eval so drift turns red instead of invisible.

Tenant leakage through the vector table. Vectors feel like infrastructure, so they get built outside the wall: a search route on the service client, an RPC marked security definer "for performance", a new chunks table with RLS forgotten. Each reduces isolation to hoping every query remembers a filter. The countermeasures are all 4.5's: RLS on in the creating migration, security invoker retrieval, the schema-wide tests.rls_enabled('public') assertion, and a pgTAP negative test where agency B searches and gets zero of agency A's chunks.

Cost blowups. The shapes to expect: an agent "improvement" that re-embeds the whole corpus on deploy (the content-hash key is the fuse), a rerank on every keystroke of a typeahead (rerank on submit only), and a digest fanning out generation calls across every project of every org nightly. The ceiling turns each from an invoice into a parked job and a red dashboard number.

Eval-less quality drift. The failure 3.7 predicts: someone tightens the answer prompt for one complaint, the demo query improves, and paraphrase recall quietly drops for everyone else. Without golden queries there is no evidence either way, and RAG degrades the way it always does, silently, at some percentage. If you build one thing from this appendix, build the eval.

Knowledge check

Sources

• pgvector README (v0.8.4, vector types, HNSW vs IVFFlat, distance operators including cosine, HNSW index syntax, iterative index scans for filtered queries): https://github.com/pgvector/pgvector (fetched July 2026)
• Supabase pgvector extension (enabling the extension, vector columns, filtered approximate-index caveat): https://supabase.com/docs/guides/database/extensions/pgvector (fetched July 2026)
• Supabase Hybrid search (tsvector generated column, GIN plus HNSW indexes, the hybrid_search function with reciprocal rank fusion, weights, rrf_k default 50): https://supabase.com/docs/guides/ai/hybrid-search (fetched July 2026)
• Embeddings, Claude Platform docs (no first-party embedding model; Voyage AI integration, embed endpoint and input_type): https://docs.claude.com/en/docs/build-with-claude/embeddings (fetched July 2026)
• Voyage AI Text embeddings (voyage-4 series names, 32k context, 1024-dimension default, input_type query/document, response shape): https://docs.voyageai.com/docs/embeddings (fetched July 2026)
• Voyage AI Rerankers (rerank-2.5 and rerank-2.5-lite, 32k context, rerank endpoint fields and relevance scores): https://docs.voyageai.com/docs/reranker (fetched July 2026)
• Voyage AI Pricing (embedding and reranker per-token prices, free-tier token allowances, as of July 2026): https://docs.voyageai.com/docs/pricing (fetched July 2026)
• Search results, Claude Platform docs (search_result content blocks, citations enabled, search_result_location fields, claude-sonnet-5 support): https://docs.claude.com/en/docs/build-with-claude/search-results (fetched July 2026)
• Citations, Claude Platform docs (guaranteed-valid citation pointers, cited_text token accounting, incompatibility with structured outputs): https://docs.claude.com/en/docs/build-with-claude/citations (fetched July 2026)
• The search_chunks schema, triggers, RPC, and eval shape are this course's reference artifacts, authored against the fetched pgvector and Supabase syntax and consistent with the Briefcase schema from 4.5 and the jobs table from 4.8.