Migrating to Completions v2

Deprecation Notice: The Chat and Message API endpoints are being deprecated. Please migrate as soon as possible.

Choosing your migration target: If you don’t need routing, model_family, or grounded completions, migrate directly to the Responses API — Gloo’s recommended, OpenAI-compatible surface, with guardrails and tradition included. If you do need those features, this guide’s path to Completions V2 is the right choice. The request/response mapping below adapts to either; see the Completions → Responses field mapping.

Overview

The Chat and Message API provided a convenient stateful approach to building conversational AI, with automatic session management and history tracking. As we evolve the Gloo platform, we’re transitioning to the Completions v2 API powered by a layered, production-ready AI architecture. This is a more flexible architecture that gives you complete control while delivering values-aligned AI with intelligent model routing. This guide will help you migrate your existing Chat API integrations to Completions v2, maintaining all the conversational capabilities you need while gaining access to powerful new features.

Why Migrate to Completions v2?

Values-Aligned Infrastructure

Access AI infrastructure built for human flourishing with values alignment and safety at every layer

Intelligent Routing

Auto-routing, model family selection, or direct model choice—optimize for quality, cost, and intent

Full Control

Manage conversation state, history, and context exactly how your application needs it

Theological Alignment

Built-in support for tradition-aware responses (evangelical, catholic, mainline)

Quick Start: Minimal Migration Path

If you want to get started quickly, here’s the fastest path to migrate:

Update your endpoint

Change from /ai/v1/message to /ai/v2/chat/completions

Set up conversation storage

Create a simple database table for storing chat history

CREATE TABLE messages (
  id UUID PRIMARY KEY,
  chat_id UUID NOT NULL,
  role VARCHAR(50) NOT NULL,
  content TEXT NOT NULL,
  created_at TIMESTAMP DEFAULT NOW()
);

Build conversation history

Fetch messages from your database and format them for Completions v2

const messages = await db.query(
  'SELECT role, content FROM messages WHERE chat_id = $1 ORDER BY created_at',
  [chatId]
);

Call Completions v2 with auto-routing

Let Gloo automatically select the best model

const response = await fetch(
  'https://platform.ai.gloo.com/ai/v2/chat/completions',
  {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${token}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      messages: messages,
      auto_routing: true,
      tradition: 'evangelical' // Optional: maintain theological alignment
    })
  }
);

Understanding the Changes

Architecture Comparison

Before: Chat API
After: Completions v2

Gloo managed:

Chat session creation
Message storage
History retrieval
Context management

Key Differences

Feature	Chat API	Completions v2
State Management	Server-managed by Gloo	Client-managed (your database)
Message History	Automatic tracking	You build and send full context
Chat Sessions	Auto-generated `chat_id`	You generate session IDs
API Calls	Incremental (one message at a time)	Full conversation context each time
Model Selection	Fixed model	Auto-routing, family selection, or direct choice
Tradition Support	Not available	Built-in (`evangelical`, `catholic`, `mainline`)
Tool Use	Limited	Full support
Streaming	Available	Enhanced streaming support

Step-by-Step Migration Guide

Step 1: Design Your Conversation Storage

The first step is creating your own storage for conversations. Here’s a recommended schema that mirrors the Chat API’s capabilities:

-- Chat sessions table
CREATE TABLE chats (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    user_id VARCHAR(255) NOT NULL,
    title VARCHAR(500),
    tradition VARCHAR(50), -- 'evangelical', 'catholic', 'mainline'
    metadata JSONB,
    created_at TIMESTAMP DEFAULT NOW(),
    updated_at TIMESTAMP DEFAULT NOW(),
    
    INDEX idx_user_id (user_id),
    INDEX idx_updated_at (updated_at)
);

-- Messages table
CREATE TABLE messages (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    chat_id UUID NOT NULL REFERENCES chats(id) ON DELETE CASCADE,
    role VARCHAR(50) NOT NULL, -- 'user', 'assistant', 'system'
    content TEXT NOT NULL,
    model VARCHAR(100), -- Store which model generated this response
    tokens_used INTEGER, -- Track token usage per message
    metadata JSONB,
    created_at TIMESTAMP DEFAULT NOW(),
    
    INDEX idx_chat_id_created (chat_id, created_at)
);

-- Optional: Store sources/citations
CREATE TABLE message_sources (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    message_id UUID NOT NULL REFERENCES messages(id) ON DELETE CASCADE,
    source_title VARCHAR(500),
    source_url TEXT,
    publisher VARCHAR(255),
    relevance_score DECIMAL(3,2),
    created_at TIMESTAMP DEFAULT NOW()
);

// Chats collection
{
  _id: ObjectId,
  user_id: String,
  title: String,
  tradition: String, // 'evangelical', 'catholic', 'mainline'
  metadata: {
    tags: [String],
    publishers: [String],
    // Custom fields as needed
  },
  created_at: Date,
  updated_at: Date
}

// Messages collection
{
  _id: ObjectId,
  chat_id: ObjectId,
  role: String, // 'user', 'assistant', 'system'
  content: String,
  model: String, // Model that generated this response
  tokens_used: Number,
  sources: [
    {
      title: String,
      url: String,
      publisher: String,
      relevance_score: Number
    }
  ],
  metadata: Object,
  created_at: Date
}

// Recommended indexes
db.messages.createIndex({ chat_id: 1, created_at: 1 });
db.chats.createIndex({ user_id: 1, updated_at: -1 });

Pro Tip: Include fields for tradition, model, and tokens_used to track theological alignment, model usage patterns, and costs over time.

Step 2: Build Conversation Management

Create functions to manage chat sessions and messages:

import { v4 as uuidv4 } from 'uuid';

interface Message {
  id: string;
  chat_id: string;
  role: 'user' | 'assistant' | 'system';
  content: string;
  model?: string;
  tokens_used?: number;
  created_at: Date;
}

interface Chat {
  id: string;
  user_id: string;
  title?: string;
  tradition?: 'evangelical' | 'catholic' | 'mainline';
  created_at: Date;
  updated_at: Date;
}

class ConversationManager {
  private db: any; // Your database client
  
  constructor(db: any) {
    this.db = db;
  }
  
  /**
   * Create a new chat session
   */
  async createChat(
    userId: string, 
    tradition?: Chat['tradition'],
    title?: string
  ): Promise<Chat> {
    const chat: Chat = {
      id: uuidv4(),
      user_id: userId,
      title: title || 'New Conversation',
      tradition,
      created_at: new Date(),
      updated_at: new Date()
    };
    
    await this.db.chats.insert(chat);
    return chat;
  }
  
  /**
   * Add a message to the conversation
   */
  async addMessage(
    chatId: string,
    role: Message['role'],
    content: string,
    metadata?: {
      model?: string;
      tokens_used?: number;
    }
  ): Promise<Message> {
    const message: Message = {
      id: uuidv4(),
      chat_id: chatId,
      role,
      content,
      model: metadata?.model,
      tokens_used: metadata?.tokens_used,
      created_at: new Date()
    };
    
    await this.db.messages.insert(message);
    
    // Update chat's updated_at timestamp
    await this.db.chats.update(
      { id: chatId },
      { updated_at: new Date() }
    );
    
    return message;
  }
  
  /**
   * Get conversation history formatted for Completions v2
   */
  async getConversationHistory(
    chatId: string,
    options?: {
      maxMessages?: number;
      maxTokens?: number;
    }
  ): Promise<Array<{ role: string; content: string }>> {
    const messages = await this.db.messages
      .find({ chat_id: chatId })
      .sort({ created_at: 'asc' })
      .toArray();
    
    // Apply any filtering/truncation based on options
    let filtered = messages;
    
    if (options?.maxMessages) {
      filtered = messages.slice(-options.maxMessages);
    }
    
    if (options?.maxTokens) {
      filtered = this.truncateByTokens(filtered, options.maxTokens);
    }
    
    // Format for Completions v2
    return filtered.map(msg => ({
      role: msg.role,
      content: msg.content
    }));
  }
  
  /**
   * Truncate messages to fit within token limit
   */
  private truncateByTokens(
    messages: Message[],
    maxTokens: number
  ): Message[] {
    let totalTokens = 0;
    const result: Message[] = [];
    
    // Keep system message if present
    if (messages[0]?.role === 'system') {
      result.push(messages[0]);
      totalTokens += this.estimateTokens(messages[0].content);
      messages = messages.slice(1);
    }
    
    // Add messages from most recent, working backwards
    for (let i = messages.length - 1; i >= 0; i--) {
      const tokens = this.estimateTokens(messages[i].content);
      if (totalTokens + tokens > maxTokens) break;
      
      result.unshift(messages[i]);
      totalTokens += tokens;
    }
    
    return result;
  }
  
  /**
   * Rough token estimation (4 chars ≈ 1 token)
   */
  private estimateTokens(text: string): number {
    return Math.ceil(text.length / 4);
  }
  
  /**
   * Get all chats for a user
   */
  async getUserChats(userId: string): Promise<Chat[]> {
    return await this.db.chats
      .find({ user_id: userId })
      .sort({ updated_at: 'desc' })
      .toArray();
  }
  
  /**
   * Delete a conversation
   */
  async deleteChat(chatId: string): Promise<void> {
    await this.db.messages.deleteMany({ chat_id: chatId });
    await this.db.chats.delete({ id: chatId });
  }
}

import uuid
from datetime import datetime
from typing import Optional, List, Dict, Any, Literal

class ConversationManager:
    def __init__(self, db):
        self.db = db
    
    def create_chat(
        self,
        user_id: str,
        tradition: Optional[Literal['evangelical', 'catholic', 'mainline']] = None,
        title: Optional[str] = None
    ) -> Dict[str, Any]:
        """Create a new chat session."""
        chat = {
            'id': str(uuid.uuid4()),
            'user_id': user_id,
            'title': title or 'New Conversation',
            'tradition': tradition,
            'created_at': datetime.now(),
            'updated_at': datetime.now()
        }
        
        self.db.chats.insert_one(chat)
        return chat
    
    def add_message(
        self,
        chat_id: str,
        role: Literal['user', 'assistant', 'system'],
        content: str,
        model: Optional[str] = None,
        tokens_used: Optional[int] = None
    ) -> Dict[str, Any]:
        """Add a message to the conversation."""
        message = {
            'id': str(uuid.uuid4()),
            'chat_id': chat_id,
            'role': role,
            'content': content,
            'model': model,
            'tokens_used': tokens_used,
            'created_at': datetime.now()
        }
        
        self.db.messages.insert_one(message)
        
        # Update chat's updated_at timestamp
        self.db.chats.update_one(
            {'id': chat_id},
            {'$set': {'updated_at': datetime.now()}}
        )
        
        return message
    
    def get_conversation_history(
        self,
        chat_id: str,
        max_messages: Optional[int] = None,
        max_tokens: Optional[int] = None
    ) -> List[Dict[str, str]]:
        """Get conversation history formatted for Completions v2."""
        messages = list(
            self.db.messages
            .find({'chat_id': chat_id})
            .sort('created_at', 1)
        )
        
        # Apply filtering
        if max_messages:
            messages = messages[-max_messages:]
        
        if max_tokens:
            messages = self._truncate_by_tokens(messages, max_tokens)
        
        # Format for Completions v2
        return [
            {'role': msg['role'], 'content': msg['content']}
            for msg in messages
        ]
    
    def _truncate_by_tokens(
        self,
        messages: List[Dict],
        max_tokens: int
    ) -> List[Dict]:
        """Truncate messages to fit within token limit."""
        total_tokens = 0
        result = []
        
        # Keep system message if present
        if messages and messages[0].get('role') == 'system':
            result.append(messages[0])
            total_tokens += self._estimate_tokens(messages[0]['content'])
            messages = messages[1:]
        
        # Add messages from most recent
        for msg in reversed(messages):
            tokens = self._estimate_tokens(msg['content'])
            if total_tokens + tokens > max_tokens:
                break
            
            result.insert(0 if not result else 1, msg)
            total_tokens += tokens
        
        return result
    
    @staticmethod
    def _estimate_tokens(text: str) -> int:
        """Rough token estimation (4 chars ≈ 1 token)."""
        return len(text) // 4 + 1
    
    def get_user_chats(self, user_id: str) -> List[Dict[str, Any]]:
        """Get all chats for a user."""
        return list(
            self.db.chats
            .find({'user_id': user_id})
            .sort('updated_at', -1)
        )
    
    def delete_chat(self, chat_id: str) -> None:
        """Delete a conversation."""
        self.db.messages.delete_many({'chat_id': chat_id})
        self.db.chats.delete_one({'id': chat_id})

Step 3: Integrate with Completions v2

Now integrate your conversation manager with the Completions v2 API:

interface CompletionsV2Config {
  apiKey: string;
  baseUrl?: string;
}

interface CompletionOptions {
  tradition?: 'evangelical' | 'catholic' | 'mainline';
  autoRouting?: boolean;
  model?: string;
  modelFamily?: 'openai' | 'anthropic' | 'google' | 'open source';
  stream?: boolean;
  temperature?: number;
  maxTokens?: number;
}

class GlooCompletionsV2Client {
  private apiKey: string;
  private baseUrl: string;
  
  constructor(config: CompletionsV2Config) {
    this.apiKey = config.apiKey;
    this.baseUrl = config.baseUrl || 'https://platform.ai.gloo.com';
  }
  
  async createCompletion(
    messages: Array<{ role: string; content: string }>,
    options: CompletionOptions = {}
  ) {
    const {
      tradition,
      autoRouting = true,
      model,
      modelFamily,
      stream = false,
      temperature,
      maxTokens
    } = options;
    
    // Build request body based on routing strategy
    const body: any = {
      messages,
      stream
    };
    
    // Routing configuration
    if (autoRouting) {
      body.auto_routing = true;
    } else if (model) {
      body.model = model;
      body.auto_routing = false;
    } else if (modelFamily) {
      body.model_family = modelFamily;
      body.auto_routing = false;
    }
    
    // Optional parameters
    if (tradition) body.tradition = tradition;
    if (temperature !== undefined) body.temperature = temperature;
    if (maxTokens) body.max_tokens = maxTokens;
    
    const response = await fetch(
      `${this.baseUrl}/ai/v2/chat/completions`,
      {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${this.apiKey}`,
          'Content-Type': 'application/json'
        },
        body: JSON.stringify(body)
      }
    );
    
    if (!response.ok) {
      const error = await response.json();
      throw new Error(`Completions API error: ${error.message || response.statusText}`);
    }
    
    const data = await response.json();
    return data;
  }
}

// Complete chat service combining conversation management and Completions v2
class ChatService {
  private conversationManager: ConversationManager;
  private completionsClient: GlooCompletionsV2Client;
  
  constructor(
    conversationManager: ConversationManager,
    completionsClient: GlooCompletionsV2Client
  ) {
    this.conversationManager = conversationManager;
    this.completionsClient = completionsClient;
  }
  
  /**
   * Create a new conversation
   */
  async createConversation(
    userId: string,
    tradition?: 'evangelical' | 'catholic' | 'mainline',
    title?: string
  ) {
    return await this.conversationManager.createChat(userId, tradition, title);
  }
  
  /**
   * Send a message and get AI response
   */
  async sendMessage(
    chatId: string,
    userMessage: string,
    options?: CompletionOptions
  ): Promise<{
    response: string;
    model: string;
    tokensUsed: number;
  }> {
    // 1. Store user message
    await this.conversationManager.addMessage(chatId, 'user', userMessage);
    
    // 2. Get conversation history
    const history = await this.conversationManager.getConversationHistory(
      chatId,
      {
        maxMessages: 50, // Keep last 50 messages
        maxTokens: 8000  // Adjust based on your model's context window
      }
    );
    
    // 3. Get chat details for tradition
    const chat = await this.conversationManager.db.chats.findOne({ id: chatId });
    const completionOptions = {
      ...options,
      tradition: options?.tradition || chat?.tradition
    };
    
    // 4. Call Completions v2
    const completion = await this.completionsClient.createCompletion(
      history,
      completionOptions
    );
    
    const assistantMessage = completion.choices[0].message.content;
    const tokensUsed = completion.usage.total_tokens;
    const model = completion.model;
    
    // 5. Store assistant response
    await this.conversationManager.addMessage(
      chatId,
      'assistant',
      assistantMessage,
      {
        model,
        tokens_used: tokensUsed
      }
    );
    
    return {
      response: assistantMessage,
      model,
      tokensUsed
    };
  }
  
  /**
   * Get conversation history
   */
  async getHistory(chatId: string) {
    return await this.conversationManager.getConversationHistory(chatId);
  }
  
  /**
   * List user's conversations
   */
  async listConversations(userId: string) {
    return await this.conversationManager.getUserChats(userId);
  }
  
  /**
   * Delete a conversation
   */
  async deleteConversation(chatId: string) {
    await this.conversationManager.deleteChat(chatId);
  }
}

// Usage Example
const conversationManager = new ConversationManager(db);
const completionsClient = new GlooCompletionsV2Client({
  apiKey: process.env.GLOO_API_KEY!
});
const chatService = new ChatService(conversationManager, completionsClient);

// Create a new conversation
const chat = await chatService.createConversation(
  'user-123',
  'evangelical',
  'Finding Purpose'
);

// Send messages
const response1 = await chatService.sendMessage(
  chat.id,
  'How can I find meaning and purpose when facing life\'s greatest challenges?',
  { autoRouting: true }
);

console.log('AI Response:', response1.response);
console.log('Model Used:', response1.model);
console.log('Tokens:', response1.tokensUsed);

// Continue the conversation
const response2 = await chatService.sendMessage(
  chat.id,
  'Can you give me practical steps I can take today?'
);

console.log('Follow-up Response:', response2.response);

import httpx
from typing import Optional, Dict, List, Any, Literal

class GlooCompletionsV2Client:
    def __init__(self, api_key: str, base_url: str = 'https://platform.ai.gloo.com'):
        self.api_key = api_key
        self.base_url = base_url
    
    async def create_completion(
        self,
        messages: List[Dict[str, str]],
        tradition: Optional[Literal['evangelical', 'catholic', 'mainline']] = None,
        auto_routing: bool = True,
        model: Optional[str] = None,
        model_family: Optional[str] = None,
        stream: bool = False,
        temperature: Optional[float] = None,
        max_tokens: Optional[int] = None
    ) -> Dict[str, Any]:
        """Call the Completions v2 API."""
        
        # Build request body
        body = {
            'messages': messages,
            'stream': stream
        }
        
        # Routing configuration
        if auto_routing:
            body['auto_routing'] = True
        elif model:
            body['model'] = model
            body['auto_routing'] = False
        elif model_family:
            body['model_family'] = model_family
            body['auto_routing'] = False
        
        # Optional parameters
        if tradition:
            body['tradition'] = tradition
        if temperature is not None:
            body['temperature'] = temperature
        if max_tokens:
            body['max_tokens'] = max_tokens
        
        async with httpx.AsyncClient() as client:
            response = await client.post(
                f'{self.base_url}/ai/v2/chat/completions',
                headers={
                    'Authorization': f'Bearer {self.api_key}',
                    'Content-Type': 'application/json'
                },
                json=body,
                timeout=60.0
            )
            response.raise_for_status()
            return response.json()


class ChatService:
    def __init__(
        self,
        conversation_manager: ConversationManager,
        completions_client: GlooCompletionsV2Client
    ):
        self.conversation_manager = conversation_manager
        self.completions_client = completions_client
    
    async def create_conversation(
        self,
        user_id: str,
        tradition: Optional[str] = None,
        title: Optional[str] = None
    ) -> Dict[str, Any]:
        """Create a new conversation."""
        return self.conversation_manager.create_chat(user_id, tradition, title)
    
    async def send_message(
        self,
        chat_id: str,
        user_message: str,
        **options
    ) -> Dict[str, Any]:
        """Send a message and get AI response."""
        
        # 1. Store user message
        self.conversation_manager.add_message(chat_id, 'user', user_message)
        
        # 2. Get conversation history
        history = self.conversation_manager.get_conversation_history(
            chat_id,
            max_messages=50,
            max_tokens=8000
        )
        
        # 3. Get chat details for tradition
        chat = self.conversation_manager.db.chats.find_one({'id': chat_id})
        tradition = options.get('tradition') or chat.get('tradition')
        
        # 4. Call Completions v2
        completion = await self.completions_client.create_completion(
            messages=history,
            tradition=tradition,
            **options
        )
        
        assistant_message = completion['choices'][0]['message']['content']
        tokens_used = completion['usage']['total_tokens']
        model = completion['model']
        
        # 5. Store assistant response
        self.conversation_manager.add_message(
            chat_id,
            'assistant',
            assistant_message,
            model=model,
            tokens_used=tokens_used
        )
        
        return {
            'response': assistant_message,
            'model': model,
            'tokens_used': tokens_used
        }
    
    def get_history(self, chat_id: str) -> List[Dict[str, str]]:
        """Get conversation history."""
        return self.conversation_manager.get_conversation_history(chat_id)
    
    def list_conversations(self, user_id: str) -> List[Dict[str, Any]]:
        """List user's conversations."""
        return self.conversation_manager.get_user_chats(user_id)
    
    def delete_conversation(self, chat_id: str) -> None:
        """Delete a conversation."""
        self.conversation_manager.delete_chat(chat_id)


# Usage Example
async def main():
    conversation_manager = ConversationManager(db)
    completions_client = GlooCompletionsV2Client(
        api_key=os.getenv('GLOO_API_KEY')
    )
    chat_service = ChatService(conversation_manager, completions_client)
    
    # Create a new conversation
    chat = await chat_service.create_conversation(
        'user-123',
        tradition='evangelical',
        title='Finding Purpose'
    )
    
    # Send messages
    response1 = await chat_service.send_message(
        chat['id'],
        'How can I find meaning and purpose when facing life\'s greatest challenges?',
        auto_routing=True
    )
    
    print(f"AI Response: {response1['response']}")
    print(f"Model Used: {response1['model']}")
    print(f"Tokens: {response1['tokens_used']}")
    
    # Continue the conversation
    response2 = await chat_service.send_message(
        chat['id'],
        'Can you give me practical steps I can take today?'
    )
    
    print(f"Follow-up Response: {response2['response']}")

Step 4: Migrate Existing Chat Data

Before the Chat API is deprecated, export your existing conversations:

async function migrateExistingChats(userId: string) {
  console.log(`Starting migration for user: ${userId}`);
  
  // 1. Get all chats from old Chat API
  const chatsResponse = await fetch(
    `https://platform.ai.gloo.com/ai/v1/chat?user_id=${userId}`,
    {
      headers: { 'Authorization': `Bearer ${oldApiToken}` }
    }
  );
  const oldChats = await chatsResponse.json();
  
  console.log(`Found ${oldChats.length} chats to migrate`);
  
  for (const oldChat of oldChats) {
    try {
      // 2. Get messages for this chat
      const messagesResponse = await fetch(
        `https://platform.ai.gloo.com/ai/v1/chat/${oldChat.id}/messages`,
        {
          headers: { 'Authorization': `Bearer ${oldApiToken}` }
        }
      );
      const oldMessages = await messagesResponse.json();
      
      // 3. Create new chat in your system
      const newChat = await conversationManager.createChat(
        oldChat.user_id,
        undefined, // Set tradition if you have this info
        oldChat.title || 'Migrated Conversation'
      );
      
      // 4. Migrate all messages
      for (const oldMessage of oldMessages.messages) {
        await conversationManager.addMessage(
          newChat.id,
          oldMessage.role,
          oldMessage.content || oldMessage.message
        );
      }
      
      console.log(`✓ Migrated chat ${oldChat.id} → ${newChat.id}`);
      
    } catch (error) {
      console.error(`✗ Failed to migrate chat ${oldChat.id}:`, error);
    }
  }
  
  console.log('Migration complete!');
}

// Run migration
await migrateExistingChats('user-123');

async def migrate_existing_chats(user_id: str):
    """Migrate chats from Chat API to new system."""
    print(f"Starting migration for user: {user_id}")
    
    async with httpx.AsyncClient() as client:
        # 1. Get all chats from old Chat API
        chats_response = await client.get(
            f'https://platform.ai.gloo.com/ai/v1/chat?user_id={user_id}',
            headers={'Authorization': f'Bearer {old_api_token}'}
        )
        old_chats = chats_response.json()
        
        print(f"Found {len(old_chats)} chats to migrate")
        
        for old_chat in old_chats:
            try:
                # 2. Get messages for this chat
                messages_response = await client.get(
                    f'https://platform.ai.gloo.com/ai/v1/chat/{old_chat["id"]}/messages',
                    headers={'Authorization': f'Bearer {old_api_token}'}
                )
                old_messages = messages_response.json()
                
                # 3. Create new chat in your system
                new_chat = conversation_manager.create_chat(
                    old_chat['user_id'],
                    tradition=None,  # Set if you have this info
                    title=old_chat.get('title', 'Migrated Conversation')
                )
                
                # 4. Migrate all messages
                for old_message in old_messages.get('messages', []):
                    conversation_manager.add_message(
                        new_chat['id'],
                        old_message['role'],
                        old_message.get('content') or old_message.get('message')
                    )
                
                print(f"✓ Migrated chat {old_chat['id']} → {new_chat['id']}")
                
            except Exception as error:
                print(f"✗ Failed to migrate chat {old_chat['id']}: {error}")
        
        print('Migration complete!')

# Run migration
await migrate_existing_chats('user-123')

Important: Run your migration script as soon as you can to ensure all data is safely transferred. Test thoroughly with a subset of users first.

Leveraging Completions v2 Features

1. Intelligent Model Routing

Take advantage of Gloo’s routing capabilities:

Auto-Routing
Model Family Selection
Direct Model Selection (Recommended)

Let Gloo automatically select the optimal model:

const response = await chatService.sendMessage(
  chatId,
  'How does God\'s love transform suffering?',
  {
    autoRouting: true,
    tradition: 'evangelical'
  }
);

Best for: Quick prototyping or when you prefer hands-off model management

Choose a provider family and let Gloo select the best model:

const response = await chatService.sendMessage(
  chatId,
  'Draft a sermon outline on hope',
  {
    modelFamily: 'anthropic',
    tradition: 'mainline'
  }
);

Best for: When you have provider preferences or are testing across families

Specify the exact model for your use case:

const response = await chatService.sendMessage(
  chatId,
  'Translate this passage to Spanish',
  {
    model: 'gloo-openai-gpt-5.5-pro',
    autoRouting: false
  }
);

Best for: Benchmarking, specialized tasks, or strict reproducibility requirements

2. Tradition-Aware Responses

Maintain theological alignment throughout conversations:

// Set tradition at chat creation
const chat = await chatService.createConversation(
  userId,
  'catholic', // evangelical, catholic, or mainline
  'Understanding the Trinity'
);

// The tradition is automatically applied to all subsequent messages
const response = await chatService.sendMessage(
  chat.id,
  'How do we understand the Holy Spirit?'
);
// Response will be aligned with Catholic theological perspective

3. Streaming Responses

Implement real-time streaming for better UX:

async function* streamChatResponse(
  chatId: string,
  userMessage: string
) {
  // Store user message
  await conversationManager.addMessage(chatId, 'user', userMessage);
  
  // Get history
  const history = await conversationManager.getConversationHistory(chatId);
  
  // Stream from Completions v2
  const response = await fetch(
    'https://platform.ai.gloo.com/ai/v2/chat/completions',
    {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({
        messages: history,
        auto_routing: true,
        stream: true
      })
    }
  );
  
  const reader = response.body?.getReader();
  const decoder = new TextDecoder();
  let fullResponse = '';
  
  while (true) {
    const { done, value } = await reader!.read();
    if (done) break;
    
    const chunk = decoder.decode(value);
    const lines = chunk.split('\n').filter(line => line.trim());
    
    for (const line of lines) {
      if (line.startsWith('data: ')) {
        const data = line.slice(6);
        if (data === '[DONE]') {
          // Store complete response
          await conversationManager.addMessage(
            chatId,
            'assistant',
            fullResponse
          );
          return;
        }
        
        const parsed = JSON.parse(data);
        const content = parsed.choices[0]?.delta?.content;
        if (content) {
          fullResponse += content;
          yield content;
        }
      }
    }
  }
}

// Usage with Express
app.post('/api/chat/:chatId/stream', async (req, res) => {
  res.setHeader('Content-Type', 'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  
  for await (const chunk of streamChatResponse(req.params.chatId, req.body.message)) {
    res.write(`data: ${JSON.stringify({ chunk })}\n\n`);
  }
  
  res.write('data: [DONE]\n\n');
  res.end();
});

Advanced Patterns

Pattern 1: Conversation Branching

Create alternative conversation paths:

async function createBranch(
  originalChatId: string,
  fromMessageId: string,
  newBranchTitle: string
): Promise<Chat> {
  // Get original chat
  const originalChat = await db.chats.findOne({ id: originalChatId });
  
  // Create new chat
  const branchChat = await conversationManager.createChat(
    originalChat.user_id,
    originalChat.tradition,
    `${newBranchTitle} (Branch)`
  );
  
  // Copy messages up to branch point
  const messages = await db.messages
    .find({ chat_id: originalChatId })
    .sort({ created_at: 'asc' })
    .toArray();
  
  for (const msg of messages) {
    if (msg.id === fromMessageId) break;
    await conversationManager.addMessage(branchChat.id, msg.role, msg.content);
  }
  
  return branchChat;
}

Pattern 2: Multi-Source Integration

Leverage multiple content publishers:

interface PublisherFilter {
  publishers?: string[];
  sources_limit?: number;
}

async function sendMessageWithSources(
  chatId: string,
  userMessage: string,
  filters?: PublisherFilter
): Promise<any> {
  await conversationManager.addMessage(chatId, 'user', userMessage);
  
  const history = await conversationManager.getConversationHistory(chatId);
  
  // Add system message with publisher context if needed
  if (filters?.publishers) {
    history.unshift({
      role: 'system',
      content: `Focus on information from these publishers: ${filters.publishers.join(', ')}`
    });
  }
  
  const completion = await completionsClient.createCompletion(history, {
    autoRouting: true
  });
  
  // Store response with source metadata
  const response = completion.choices[0].message.content;
  await conversationManager.addMessage(chatId, 'assistant', response);
  
  return {
    response,
    sources: completion.sources || []
  };
}

Pattern 3: Conversation Summarization

Auto-summarize long conversations:

async function summarizeConversation(chatId: string): Promise<string> {
  const messages = await db.messages
    .find({ chat_id: chatId })
    .sort({ created_at: 'asc' })
    .toArray();
  
  if (messages.length < 10) {
    return ''; // Don't summarize short conversations
  }
  
  // Create summary prompt
  const conversationText = messages
    .map(m => `${m.role}: ${m.content}`)
    .join('\n\n');
  
  const completion = await completionsClient.createCompletion(
    [
      {
        role: 'system',
        content: 'You are a helpful assistant that creates concise 2-3 sentence summaries of conversations.'
      },
      {
        role: 'user',
        content: `Please summarize this conversation:\n\n${conversationText}`
      }
    ],
    {
      autoRouting: true,
      maxTokens: 150
    }
  );
  
  const summary = completion.choices[0].message.content;
  
  // Update chat title with summary
  await db.chats.update(
    { id: chatId },
    {
      title: summary.slice(0, 100),
      metadata: { full_summary: summary }
    }
  );
  
  return summary;
}

Testing Your Migration

Unit Tests

import { describe, it, expect, beforeEach } from 'vitest';

describe('Chat Service Migration', () => {
  let chatService: ChatService;
  let testUserId: string;
  
  beforeEach(() => {
    const conversationManager = new ConversationManager(testDb);
    const completionsClient = new GlooCompletionsV2Client({
      apiKey: process.env.TEST_GLOO_API_KEY!
    });
    chatService = new ChatService(conversationManager, completionsClient);
    testUserId = 'test-user-123';
  });
  
  it('should create a new conversation with tradition', async () => {
    const chat = await chatService.createConversation(
      testUserId,
      'evangelical',
      'Test Chat'
    );
    
    expect(chat.id).toBeDefined();
    expect(chat.user_id).toBe(testUserId);
    expect(chat.tradition).toBe('evangelical');
  });
  
  it('should send and receive messages', async () => {
    const chat = await chatService.createConversation(testUserId);
    const result = await chatService.sendMessage(
      chat.id,
      'What brings meaning to life?',
      { autoRouting: true }
    );
    
    expect(result.response).toBeDefined();
    expect(result.model).toBeDefined();
    expect(result.tokensUsed).toBeGreaterThan(0);
  });
  
  it('should maintain conversation context', async () => {
    const chat = await chatService.createConversation(testUserId);
    
    await chatService.sendMessage(chat.id, 'My name is John');
    await chatService.sendMessage(chat.id, 'What is my name?');
    
    const history = await chatService.getHistory(chat.id);
    expect(history.length).toBe(4); // 2 user + 2 assistant messages
  });
  
  it('should respect tradition parameter', async () => {
    const chat = await chatService.createConversation(
      testUserId,
      'catholic'
    );
    
    const result = await chatService.sendMessage(
      chat.id,
      'Tell me about the Eucharist'
    );
    
    // Response should reflect Catholic perspective
    expect(result.response).toBeDefined();
  });
});

Integration Tests

describe('End-to-End Conversation Flow', () => {
  it('should handle complete conversation lifecycle', async () => {
    // Create conversation
    const chat = await chatService.createConversation(
      'user-123',
      'evangelical',
      'Purpose and Meaning'
    );
    
    // Send multiple messages
    const response1 = await chatService.sendMessage(
      chat.id,
      'How can I find purpose in difficult times?',
      { autoRouting: true }
    );
    expect(response1.response).toContain('purpose' || 'meaning');
    
    const response2 = await chatService.sendMessage(
      chat.id,
      'Can you give practical steps?'
    );
    expect(response2.response).toBeDefined();
    
    // Verify history
    const history = await chatService.getHistory(chat.id);
    expect(history.length).toBe(4);
    
    // Test context retention
    const response3 = await chatService.sendMessage(
      chat.id,
      'Tell me more about the first step'
    );
    expect(response3.response).toBeDefined();
    
    // Delete conversation
    await chatService.deleteConversation(chat.id);
    const chats = await chatService.listConversations('user-123');
    expect(chats.find(c => c.id === chat.id)).toBeUndefined();
  });
});

Troubleshooting

Context Window Exceeded

Problem: API returns error about token limitSolution: Implement proper context truncation in your getConversationHistory method:

const history = await conversationManager.getConversationHistory(
  chatId,
  {
    maxMessages: 50,
    maxTokens: 8000 // Adjust based on model's context window
  }
);

For models with smaller context windows, consider conversation summarization to maintain essential context while reducing token count.

Messages Out of Order

Problem: Messages appearing in wrong chronological orderSolution: Ensure your database queries always sort by created_at ascending:

SELECT * FROM messages 
WHERE chat_id = ? 
ORDER BY created_at ASC  -- Critical!

Also use transactions when writing multiple messages to prevent race conditions.

Tradition Not Being Applied

Problem: Responses don’t reflect the specified theological traditionSolution: Verify tradition is being passed to Completions v2:

// Check that tradition is in the request
console.log('Tradition:', completionOptions.tradition);

// Ensure it's a valid value
const validTraditions = ['evangelical', 'catholic', 'mainline'];
if (tradition && !validTraditions.includes(tradition)) {
  throw new Error(`Invalid tradition: ${tradition}`);
}

High Latency

Problem: Responses taking too longSolutions:

Use Streaming: Provides immediate feedback to users
{ stream: true }

Optimize Context: Only send necessary messages

{ maxMessages: 20 } // Fewer messages = faster

Cache Recent Conversations: Use Redis for hot data

const cached = await redis.get(`chat:${chatId}:messages`);
if (cached) return JSON.parse(cached);

Choose Appropriate Models: Use model family selection

{ modelFamily: 'openai' } // Generally faster than anthropic

Concurrent Request Issues

Problem: Race conditions with simultaneous messagesSolution: Use database transactions and row-level locking:

await db.transaction(async (trx) => {
  // Lock the chat row
  await trx.raw('SELECT * FROM chats WHERE id = ? FOR UPDATE', [chatId]);
  
  // Store message
  await trx('messages').insert(userMessage);
  
  // Get history and call API
  const history = await trx('messages')
    .where({ chat_id: chatId })
    .orderBy('created_at');
  
  const completion = await completionsClient.createCompletion(history);
  
  // Store response
  await trx('messages').insert(assistantMessage);
});

Migration Checklist

Benefits of Migrating

Values-Aligned AI

Built-in values alignment and safety at every layer

Intelligent Routing

Auto-routing optimizes every query for quality, cost, and intent automatically

Full Control

Complete control over storage, retention, context management, and conversation workflows

Theological Alignment

Native support for tradition-aware responses (evangelical, catholic, mainline)

Advanced Features

Build sophisticated UX with branching, editing, summarization, and more

Cost Optimization

Control what context you send and optimize token usage based on your needs

Better Performance

Optimize with caching, indexing, and custom logic for your specific use case

Enhanced Privacy

Keep sensitive conversations in your own infrastructure with full control

Support and Resources

Migration Support

Email our team for migration assistance

Completions v2 Docs

Complete API reference for Completions v2

Chat API Reference

Reference for the deprecated Chat API

Tool Use Guide

Enhance your completions with tool calling

Next Steps

Review Architecture

Understand the architectural changes

Follow Migration Steps

Work through the detailed migration guide

Test Your Integration

Set up testing before going to production

Questions about migration? We’re here to help! Reach out to hello@gloo.us or use the support link in the top navigation.

​Overview

​Why Migrate to Completions v2?

Values-Aligned Infrastructure

Intelligent Routing

Full Control

Theological Alignment

​Quick Start: Minimal Migration Path

​Understanding the Changes

​Architecture Comparison

​Key Differences

​Step-by-Step Migration Guide

​Step 1: Design Your Conversation Storage

​Step 2: Build Conversation Management

​Step 3: Integrate with Completions v2

​Step 4: Migrate Existing Chat Data

​Leveraging Completions v2 Features

​1. Intelligent Model Routing

​2. Tradition-Aware Responses

​3. Streaming Responses

​Advanced Patterns

​Pattern 1: Conversation Branching

​Pattern 2: Multi-Source Integration

​Pattern 3: Conversation Summarization

​Testing Your Migration

​Unit Tests

​Integration Tests

​Troubleshooting

​Migration Checklist

​Benefits of Migrating

Values-Aligned AI

Intelligent Routing

Full Control

Theological Alignment

Advanced Features

Cost Optimization

Better Performance

Enhanced Privacy

​Support and Resources

Migration Support

Completions v2 Docs

Chat API Reference

Tool Use Guide

​Next Steps

Review Architecture

Follow Migration Steps

Test Your Integration

Overview

Why Migrate to Completions v2?

Quick Start: Minimal Migration Path

Understanding the Changes

Architecture Comparison

Key Differences

Step-by-Step Migration Guide

Step 1: Design Your Conversation Storage

Step 2: Build Conversation Management

Step 3: Integrate with Completions v2

Step 4: Migrate Existing Chat Data

Leveraging Completions v2 Features

1. Intelligent Model Routing

2. Tradition-Aware Responses

3. Streaming Responses

Advanced Patterns

Pattern 1: Conversation Branching

Pattern 2: Multi-Source Integration

Pattern 3: Conversation Summarization

Testing Your Migration

Unit Tests

Integration Tests

Troubleshooting

Migration Checklist

Benefits of Migrating

Support and Resources

Next Steps