Context Items

Overview

Context items are reusable text snippets that can be attached to chats as system prompts. They support semantic search powered by pgvector embeddings, allowing users to find relevant context by meaning rather than keywords.

How context works

When a chat is created with a context_text value, the backend inserts it as a system message at position 0:

POST /api/sessions/{session_id}/chats
{
  "model_name": "GPT-4",
  "title": "Medical Q&A",
  "context_text": "You are a medical assistant specializing in cardiology."
}

This results in the following message history:

Position	Role	Content
0	system	You are a medical assistant specializing in cardiology.
1	user	What causes atrial fibrillation?
2	assistant	Atrial fibrillation is caused by…

Every subsequent LLM call includes the system message, giving the model persistent instructions.

Context item management

Creating a context item

POST /api/context-items
{
  "title": "Cardiology Expert",
  "category": "medical",
  "tags": ["cardiology", "expert"],
  "content": "You are a cardiologist with 20 years of experience..."
}

When created, the backend embeds the text with whatever embedder this workspace resolves to (see below) and stores the vector in context_items.embedding. If the workspace has no embedder, or the one it has does not answer, the item is created without an embedding — it still works as a system prompt, it just will not be found by semantic search.

Listing context items

GET /api/context-items?offset=0&limit=50

Returns items owned by the current user plus shared items (where owner = 'all').

Searching context items

GET /api/context-items/search?q=heart+disease&limit=10

This endpoint:

1. Embeds the query text with the workspace’s system embedder
2. Performs a cosine similarity search using pgvector
3. Returns results ranked by similarity score

SELECT *, 1 - (embedding <=> $1::vector) AS similarity
FROM context_items
WHERE (owner = 'all' OR owner = $2) AND embedding IS NOT NULL
ORDER BY embedding <=> $1::vector
LIMIT $3

The embedding column is an unsized vector, so switching to a model with a different number of dimensions needs no migration. It does need a re-vectorise, though — see the caution below.

Tip

Semantic search finds conceptually related items even when exact keywords differ. A search for “heart problems” will match context items about “cardiovascular disease” or “cardiac conditions” because their embeddings are close in vector space.

Which embedder a workspace uses

Not Ollama’s, unless you chose Ollama’s. This page used to say embeddings came from a nomic-embed-text you had to pull yourself, which was true when the only embedder was a local Ollama and stopped being true when embedding moved to provider:model against any configured provider.

One place answers the question — EmbedderService — and it answers it twice, because a workspace has two embedder settings that are resolved separately:

Setting	What it embeds
The system embedder	Context items, prompt search, the docs index
The Vault embedder	The research corpus (see the Vault)

Each is stored as provider:model — novita:baai/bge-m3, ollama:nomic-embed-text, and so on — so the provider is part of the choice, not an assumption.

A workspace with no keys does not fall back to a default

It falls back to what the platform lends, and only if it is eligible for that — which means having no credentials of its own at all. A workspace with its own API keys pays for its own embedding, and if it has not chosen an embedder it simply does not embed. The setting screen says which of the two you are looking at, and whether it is lent.

Change the embedder, then re-vectorise — in that order, without a gap

Two things go wrong, and they are not equally forgiving.

The mild one: the platform sends the task prefixes (search_query: / search_document:) that two-tower models expect, so rows embedded before that compare worse until they are rewritten.

The sharp one: this search does not filter by model. Unlike the Vault — which keeps rows from different embedders apart and simply re-embeds what it needs — the query above compares the incoming vector against every row that has one. If the new model has a different number of dimensions from the old, that is a database error, not a worse result: prompt search stops working until the rewrite finishes.

Either way the fix is the same: Re-vectorize now, in Settings.

Query suggestions

A separate table query_items stores pre-populated query suggestions with embeddings. The suggestions endpoint finds semantically similar queries:

GET /api/suggestions?q=explain+transformers&limit=5

Response:

[
  { "id": "...", "query": "How do transformer models work?", "similarity": 0.92 },
  { "id": "...", "query": "Explain attention mechanisms", "similarity": 0.87 }
]

Ownership model

Context items have an owner field:

• 'all' — Shared with all users (read-only for non-owners)
• User UUID string — Private to that user

RLS policies enforce this at the database level:

• Anyone can SELECT items where owner = 'all' or owner matches their user ID
• Users can only INSERT, UPDATE, and DELETE items they own

Frontend: ContextPicker

The ContextPicker.svelte component provides a search interface for context items when creating a new chat. It calls the search endpoint as the user types and displays results ranked by similarity. Selecting an item populates the context_text field in the Add Chat dialog.

pgvector indexing

The context_items table uses an IVFFlat index for fast approximate nearest neighbor search:

CREATE INDEX idx_context_items_embedding ON public.context_items
    USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);

Caution

IVFFlat indexes require a minimum number of rows to be effective (roughly lists * 10). With fewer than ~1000 context items, the index provides marginal benefit over a sequential scan. For small datasets, pgvector will still work correctly but may use a sequential scan instead.