Inspecting the vector index with SQL¶

The RAG vector index reference documents the per-node markers and h-validate. This page is for when that isn't enough — when you're an operator staring at an empty mode: vector result and want to know why from the database itself. It explains how to read the index state in Postgres, and how the query changes with the memory's embedding source.

This is a debugging and operations aid, not part of the public API. Names below are the physical Postgres columns (spec 033), which can change — so the API-level field embeddingSource you set via updateMemory is the embedding_source column here, and this page uses the snake_case column form throughout. Treat the queries as a diagnostic starting point, not a stable interface.

The two tables¶

Index state lives in two tables:

nodes — one row per node. Carries the per-node embedding markers (embedding_pending_at, embedding_failed_at, embedding_error, embedding_attempts) plus the source material the index is built from (abstract, content).
node_embeddings — the stored vectors. Zero, one, or many rows per node depending on the source. This is where you confirm a vector actually landed.

A node is indexed when it has a node_embeddings row of the wanted kind and all four markers on its nodes row are null.

`node_embeddings.kind` per source¶

The memory's embedding_source decides what the index is built from, and that determines the shape of node_embeddings:

`embedding_source`	`node_embeddings.kind`	Rows per node	Vector of
`abstract` (default)	`'abstract'`	one	`nodes.abstract`
`contentChunks`	`'chunk'`	N — one per chunk	one content chunk
`both`	both kinds coexist	1 abstract + N chunk	both

Chunk rows carry locator columns that abstract rows leave null: chunk_index, char_start, char_end, and chunk_text. That's how you tell a chunk row from an abstract row without reading kind.

Per-node markers and what they mean¶

On the nodes row (see the reference for the API-level field names):

Column	Meaning
`embedding_pending_at`	The node is queued or in-flight for (re-)embedding. Non-null → the worker hasn't finished it.
`embedding_failed_at`	Stamped on a failed attempt. Stays set on terminal failure so the node stays diagnosable.
`embedding_error`	The last failure reason (e.g. `encrypted-no-plaintext (#206)`). Set alongside `embedding_failed_at`.
`embedding_attempts`	Per-job attempt counter; caps transient retries before terminal failure.

A node is done when it has a node_embeddings row of the wanted kind and every one of these is null.

What "can't be indexed" looks like¶

Not every node produces a vector, and the ones that can't are silently absent — no marker, no failure, no row in node_embeddings:

On embedding_source = abstract: nodes with no abstract (abstract IS NULL or blank) have nothing to embed.
On embedding_source = contentChunks: nodes with no content (content IS NULL or blank) have nothing to chunk.

These are not failures — they carry no embedding_failed_at. They simply never enter the pipeline. If mode: vector looks emptier than you expect, this is the first thing to rule out: the nodes may have no source material, not a broken index. (Terminal failures, by contrast, keep their embedding_failed_at marker and are surfaced by h-validate.)

Aggregate queries¶

Run these against the Postgres database, substituting the memory's URN. They summarize the whole index in one row so you can see at a glance whether a backfill is still draining or the index is complete.

`contentChunks`¶

SELECT
  count(*)                                                   AS live_nodes,
  count(*) FILTER (WHERE n.content IS NOT NULL
                     AND btrim(n.content) <> '')             AS have_content,
  count(*) FILTER (WHERE n.embedding_pending_at IS NOT NULL) AS still_pending,
  count(*) FILTER (WHERE n.embedding_failed_at  IS NOT NULL) AS failed,
  count(DISTINCT e.node_id)                                  AS nodes_with_chunk_vectors,
  count(e.id)                                                AS total_chunk_vectors
FROM nodes n
JOIN memories m ON m.id = n.memory_id
LEFT JOIN node_embeddings e ON e.node_id = n.id AND e.kind = 'chunk'
WHERE m.urn = '<memory-urn>'
  AND n.deleted_at IS NULL;

`abstract`¶

The abstract variant swaps the join kind and the source predicate:

SELECT
  count(*)                                                   AS live_nodes,
  count(*) FILTER (WHERE n.abstract IS NOT NULL
                     AND btrim(n.abstract) <> '')            AS have_abstract,
  count(*) FILTER (WHERE n.embedding_pending_at IS NOT NULL) AS still_pending,
  count(*) FILTER (WHERE n.embedding_failed_at  IS NOT NULL) AS failed,
  count(DISTINCT e.node_id)                                  AS nodes_with_abstract_vectors,
  count(e.id)                                                AS total_abstract_vectors
FROM nodes n
JOIN memories m ON m.id = n.memory_id
LEFT JOIN node_embeddings e ON e.node_id = n.id AND e.kind = 'abstract'
WHERE m.urn = '<memory-urn>'
  AND n.deleted_at IS NULL;

For a memory on both, run each query — the two kind filters are independent, so each tells you the state of one half of the index.

How to read the result¶

still_pending > 0 — the backfill (or a recent write) is draining. Vectors are landing; check again shortly.
failed > 0 — some nodes terminally failed. Look at embedding_error on those rows (drop the aggregation and select embedding_error per node) — on encrypted memories this is usually encrypted-no-plaintext (#206).
nodes_with_*_vectors ≈ have_content / have_abstract — the index is complete: (nearly) every node with source material has a vector.
have_* far below live_nodes — most nodes have no source material for this source. That's the "can't be indexed" case above, not a failure. If you expected them indexed, the memory may be on the wrong embedding_source.

RAG vector index — the feature reference: the markers, h-validate, search modes, and the async pipeline these tables back.
The RAG tech stack — why the index is pgvector in the same Postgres database, so these tables sit alongside everything else.
Data model — the generated Node, Memory, and NodeEmbedding entity reference.

Inspecting the vector index with SQL¶

The two tables¶

node_embeddings.kind per source¶