RAG on Postgres — you don't need a vector database

August 1, 2026

A pattern I keep seeing: someone decides to add an AI feature — a chatbot over their own documents, a “search our knowledge base by asking a question” box — and the very first tutorial tells them to sign up for a vector database. Pinecone, Weaviate Cloud, whatever’s trending. Another dashboard, another monthly bill, another service to run and secure.

Hold on. For most teams, that’s a bill you don’t need to pay. The database you almost certainly already run — Postgres — does the same job. Let me show you, with real numbers, on my own writing.

What a “vector database” actually does

The feature everyone wants is called RAG — retrieval-augmented generation. Plain version: before the AI answers, you retrieve the handful of your own documents relevant to the question and hand them to the model, so it answers from your material instead of guessing.

To find the “relevant” documents, you turn each chunk of text into a list of numbers — an embedding — that captures its meaning. Similar meaning, similar numbers. When a question comes in, you turn it into numbers the same way and find the stored chunks whose numbers are closest. That “find the closest” step is the whole reason people think they need special infrastructure.

Here’s the part the tutorials gloss over: that’s just an index. “Find the rows closest to this one” is something databases have always done. It isn’t magic, and it isn’t a new category of thing you have to go buy.

A vector search is just another index.

Postgres already does this

Postgres has an extension called pgvector. Turn it on, add a column that holds the numbers, and you get exactly the nearest-match search a dedicated vector database sells you — inside the database you already back up, already secure, and already know how to run.

I wanted to prove it to myself, so I built a tiny demo and pointed it at my own blog — the 50 posts on this site. The whole thing is one small dependency and about a hundred lines of code, and it’s open source: github.com/smallestbusiness/rag-pgvector-demo.

I ran it on a spare machine with no cloud and no API keys — the models run locally. It read all 50 posts, split them into 282 chunks, and embedded them in 33 seconds. Then I asked it questions in plain English.

Ask it “How do I stop an AI feature from running up an unlimited bill?” and it pulls the right passages out of my own posts and answers from them:

Get two numbers for the feature — its model spend last month and how many times it ran. Move steady workloads off the meter onto flat-rate boxes. Cap the parts that can run away, with hard ceilings and per-user quotas. Put an automated billing kill switch on it. Stop the feature first, then fight the charge.

That’s my own advice, retrieved from my own writing and assembled on the fly — with the source post and a similarity score printed next to each result, so I can see exactly where the answer came from. No Pinecone. No new bill.

One honest note, because it matters: ask it something the posts don’t actually cover and the local model will usually just say “I don’t know” rather than invent an answer. That’s the retrieval doing its job — the reply is only as good as what it found, and it isn’t papering over the gap. For anything customer-facing, that’s a feature, not a bug.

The whole thing, in code

There genuinely isn’t much to it. You set the table up once, then query it on every question. Here’s the setup — turn the extension on, add the embedding column, build the index:

-- Enable pgvector. It ships with Postgres; this is the only "install" step.
CREATE EXTENSION IF NOT EXISTS vector;

-- One ordinary table. The embedding is just another column.
CREATE TABLE docs (
    id        bigserial PRIMARY KEY,
    post      text,            -- which post this chunk came from
    chunk     text,            -- the passage itself
    embedding vector(1536)     -- its meaning as numbers (1536 per chunk)
);

-- The entire "special infrastructure": an index for nearest-match search.
-- HNSW + cosine distance is the standard choice for text embeddings.
CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops);

-- Retrieval is one SELECT: the 5 chunks closest in meaning to query $1.
-- <=> is pgvector's cosine-distance operator — smaller means more similar.
SELECT post, chunk
FROM docs
ORDER BY embedding <=> $1
LIMIT 5;

On the query side you embed the incoming question the same way the chunks were embedded, then hand that vector to the same SELECT:

import os
import psycopg
from pgvector.psycopg import register_vector
from openai import OpenAI

ai = OpenAI()

def retrieve(question: str, k: int = 5):
    # 1. Turn the question into numbers, same model that embedded the chunks.
    resp = ai.embeddings.create(
        model="text-embedding-3-small",   # 1536 dims — matches vector(1536)
        input=question,
    )
    qvec = resp.data[0].embedding

    # 2. Let Postgres find the k nearest chunks. This is the whole "vector DB".
    with psycopg.connect(os.environ["DATABASE_URL"]) as conn:
        register_vector(conn)             # lets psycopg send a list as a vector
        return conn.execute(
            "SELECT post, chunk FROM docs ORDER BY embedding <=> %s LIMIT %s",
            (qvec, k),
        ).fetchall()

The rows that come back are the passages you drop into the prompt as context — that’s the entire retrieval layer. No client library for a separate service, no second thing to run and secure. (The demo above swaps a local embedding model into that first call, which is why it needs no API key at all.)

The cost part — because that’s what this site is about

A managed vector database is another meter. It scales with your data and your traffic, and it’s one more line on the bill that grows exactly when you start succeeding. pgvector rides the Postgres you’re already paying for — zero new services. If that Postgres sits on a flat-rate rented box, the marginal cost of your “vector database” is essentially nothing.

You don't need another database — you already have one.

And to be straight with you, the way I try to always be: there is a point where a dedicated vector store earns its keep. Tens of millions of vectors, or heavy filtering at large scale — the specialised tools are genuinely better there, and worth paying for. But almost nobody starts there. Most teams reaching for one have a few thousand documents. At that size, “we stood up a whole managed database” is the expensive mistake, not the clever one. Start on Postgres. Move only when a real limit — not a tutorial — forces you to.

Sometimes you don’t even need vectors

There’s a cheaper step below this one, and it’s worth knowing. Embeddings earn their keep when you need semantic search — matching meaning, not the exact words. But a lot of “chat with my documents” features are really just search-plus-a-summary over a fixed set of documents. If those documents already have a plain full-text search index, you can skip embeddings entirely: search, grab the top few passages, hand them to the model.

I built a second small demo to show it — an offline question-answerer over a local copy of Wikipedia. It reuses the search index that already ships inside the Wikipedia file, so it answers questions — correctly pulling up the right articles for “why is Canberra the capital of Australia” — with zero embeddings, no vector database, and no internet at all. Embedding the whole of Wikipedia would have meant 100+ GB and days of compute; the index that was already sitting there cost nothing.

The point isn’t “never embed.” It’s the same as the rest of this post: reach for the cheapest retrieval that actually answers your question, and add machinery only when a real limit forces you to.

The two demos, same question

To make it concrete, I asked both of my local demos the same thing — “What is prompt injection, and how do you defend against it?” — on the same machine, with the same model, no cloud. The only difference is where they look and how they retrieve.

Demo A, from my own writing (it pulled up my prompt-injection and OWASP-LLM posts):

Prompt injection is a security issue where an attacker writes input that competes with and overrides the system instructions — it works because the model has no structural line between “system” and “user” text. Defend by raising the bar (input filters and classifiers), shrinking the blast radius (keep secrets and keys out of the model’s reach), and designing so that a successful hit is an annoyance, not a catastrophe.

Demo B, from Wikipedia (it used the “Prompt injection” article):

Prompt injection is a cybersecurity exploit targeting large language models, where attackers craft inputs to cause unintended behaviour or bypass safeguards. Defend by training the model to separate developer instructions from user input, validating inputs to filter malicious prompts, and patching as new vulnerabilities appear.

Both run entirely on the same local box, and both cite where the answer came from. One answers in my voice from my own posts; the other answers from the encyclopedia — and needed no embeddings at all. Same question, two honest ways to ground an answer, neither of which required a managed vector database.

The harder half comes after

Getting retrieval working is the easy 80%. The moment that box is live on your site taking questions from the public, the real work starts: it can be talked into saying things you never intended, it calls a paid model on every request with no ceiling, and someone scripting it can turn it into free compute on your bill. Retrieval is a solved problem. Governing the thing you built around it — spend caps, input filtering, a kill switch — is where the real risk and the real cost live. That’s its own post, and increasingly it’s most of what I do.

But the vector database? For your first version, and probably your tenth, you already own it.

If you’ve got an AI feature in the works — or one already live and quietly running up a bill — that’s exactly the kind of thing I look at. Every message comes straight to me; I read and reply to each one myself, usually within a day. Send me your setup or a recent bill and I’ll tell you what to cap, what to cut, and what to leave alone — free, within a business day.