Architecture Overview

System diagram

graph TB
    subgraph Client["Browser"]
        FE["SvelteKit Frontend"]
    end

    subgraph Server["Server"]
        BE["Rust Axum Backend"]
        OL["Ollama"]
    end

    subgraph Database["Supabase"]
        AUTH["Supabase Auth<br/>(Google OAuth)"]
        PG["PostgreSQL<br/>+ pgvector"]
    end

    subgraph Cloud["Cloud Providers"]
        OPENAI["OpenAI API"]
        MISTRAL["Mistral API"]
        GEMINI["Gemini API"]
        HF["HuggingFace API"]
    end

    FE -->|"REST API<br/>(HTTP)"| BE
    FE <-->|"WebSocket<br/>(streaming)"| BE
    FE -->|"OAuth flow"| AUTH
    AUTH -->|"JWT tokens"| FE

    BE -->|"SQLx queries"| PG
    BE -->|"Chat + Embeddings"| OL
    BE -->|"Chat completions"| OPENAI
    BE -->|"Chat completions"| MISTRAL
    BE -->|"Chat completions"| GEMINI
    BE -->|"Inference"| HF

    AUTH -->|"User management"| PG

Component overview

Frontend (SvelteKit)

The frontend is a SvelteKit application that handles:

• Authentication via Supabase Auth (Google OAuth), using @supabase/ssr for cookie-based sessions
• Session and chat management through REST API calls to the backend
• Real-time streaming via a WebSocket connection per active session
• UI rendering with shadcn-svelte components (tabs, dialogs, inputs)

Key modules: lib/api.ts (REST client), lib/ws.ts (WebSocket client), lib/stores/ (Svelte stores for state).

Backend (Rust Axum)

The backend is a Rust web server built on Axum that provides:

• REST API for CRUD operations on sessions, chats, context items, and models
• WebSocket endpoint for streaming chat completions token by token
• Plugin system (LlmProvider trait) for integrating multiple LLM providers
• JWT authentication middleware that validates Supabase-issued tokens
• Embedding service that calls Ollama to generate vectors for context items

Key modules: routes/ (HTTP handlers), plugins/ (LLM providers), auth/ (JWT middleware), services/ (embeddings).

Database (Supabase PostgreSQL)

A single PostgreSQL database with:

• Core tables: profiles, sessions, chats, messages, session_prompts
• Context tables: context_items (with vector(768) column), query_items
• Row Level Security on all tables, scoped to auth.uid()
• pgvector extension for cosine similarity search on embeddings

LLM Providers

The plugin registry (PluginRegistry) is a HashMap<String, Arc<dyn LlmProvider>> populated at startup:

• OpenAI — SSE streaming via /v1/chat/completions
• Mistral — SSE streaming via /v1/chat/completions
• Gemini — SSE streaming via Google’s generateContent API
• HuggingFace — Inference API (non-streaming)
• Ollama — NDJSON streaming via /api/chat, models auto-discovered at startup

Data flow

Chat query (streaming)

1. User types a message in the frontend
2. Frontend sends a query or query_all message over the WebSocket
3. Backend validates JWT, verifies chat ownership, inserts user message into messages
4. Backend loads full conversation history and sends it to the LLM provider
5. Provider streams tokens back; backend forwards each as a token WebSocket message
6. When the stream completes, backend inserts the assistant message and sends done
7. Frontend accumulates tokens in a Svelte store and renders them progressively

Chat query (REST, non-streaming)

1. Frontend calls POST /api/chats/{chat_id}/query with the user message
2. Backend inserts user message, calls provider.send() (blocking), inserts assistant message
3. Returns the full ChatWithMessages object

Authentication flow

1. User clicks “Sign in with Google” on the login page
2. Supabase Auth redirects to Google OAuth consent screen
3. Google redirects back to Supabase with an auth code
4. Supabase exchanges the code for tokens, creates a user in auth.users
5. A database trigger creates a profiles row for the new user
6. Supabase returns a JWT to the frontend, stored in cookies via @supabase/ssr
7. All backend API calls include the JWT in the Authorization: Bearer header