Multi-Chat Comparison

Overview

Chatty the Lab’s core feature is the ability to run the same prompt against multiple LLM models simultaneously and compare their responses. This is organized around sessions, chats, and the master chat concept.

Sessions and chats

A session is a workspace that contains one or more chats. Each chat is bound to a specific model (e.g., “GPT-4”, “Mistral Large”, “llama3”) and maintains its own independent conversation history.

graph LR
    S["Session: 'Compare summarizers'"]
    S --> C1["Chat 0: GPT-4"]
    S --> C2["Chat 1: Mistral Large"]
    S --> C3["Chat 2: llama3"]

Users can:

• Add new chats to a session at any time via the Add Chat dialog
• Remove chats by deleting them from the tab bar
• Switch between chats using the tab interface

Master chat

Each session has a master chat, identified by the master_chat_index field. The master chat serves as the primary conversation — when the user sends a prompt through the master chat, it is tracked in the session_prompts table for session-level history.

The master chat index can be changed via the API:

PATCH /api/sessions/{id}/master
Content-Type: application/json

{ "master_chat_index": 1 }

Tip

The master chat is typically the first chat in the session (index 0), but you can reassign it to any chat position. This is useful when you want a different model to be the “reference” for comparison.

Multi-link mode (query_all)

The query_all WebSocket message sends the same prompt to multiple chats simultaneously:

{
  "type": "query_all",
  "content": "Explain quantum computing in simple terms",
  "chat_ids": [
    "uuid-chat-1",
    "uuid-chat-2",
    "uuid-chat-3"
  ]
}

On the backend, this spawns a separate tokio::spawn task for each chat, so all models stream their responses concurrently. The frontend receives interleaved token messages tagged with chat_id, allowing it to update each chat panel independently.

Side-by-side view

The frontend renders chats as tabs within the ChatArea component. The active tab shows the full conversation, while the tab headers display the model name.

The typical workflow for model comparison:

1. Create a new session
2. Add multiple chats, each with a different model
3. Type a prompt and use query_all to send it to all chats
4. Switch between tabs to compare responses
5. Continue the conversation — each chat maintains independent history

Adding context to chats

When creating a chat, you can attach a context item as a system prompt:

{
  "model_name": "GPT-4",
  "title": "GPT-4 with medical context",
  "context_text": "You are a medical assistant. Answer questions about symptoms and treatments."
}

This allows comparing how different models behave with the same system prompt, or how the same model behaves with different contexts.

See Context Items for more on the context system.

Sync endpoint

The sync endpoint (POST /api/chats/{chat_id}/sync) lets you rewrite conversation history from a specific point:

{
  "msg_idx": 2,
  "content": "Rephrase: explain it for a 5-year-old"
}

This truncates all messages after position msg_idx, inserts a new user message, and gets a fresh response. It is useful for branching conversations or correcting earlier prompts.