Learn how to implement streaming responses when using Metorial with Vercel's AI SDK for real-time user experiences.

Why Stream?

Streaming responses provide a better user experience by showing results as they're generated rather than waiting for the complete response. This is especially important for longer responses or when tool calls are involved.

Basic Streaming Setup

Use the streamText function from the AI SDK:

import { streamText } from 'ai';
import { openai } from '@ai-sdk/openai';

await metorial.withProviderSession(
  metorialAiSdk,
  { serverDeployments: ['your-deployment-id'] },
  async session => {
    const result = streamText({
      model: openai('gpt-4'),
      messages: [
        { role: 'user', content: 'Explain how our CRM integration works' }
      ],
      tools: session.tools
    });
    
    // Stream the text
    for await (const chunk of result.textStream) {
      process.stdout.write(chunk);
    }
  }
);

Handling Tool Calls in Streams

The AI SDK automatically handles tool calls even during streaming:

const result = streamText({
  model: openai('gpt-4'),
  messages: [
    { role: 'user', content: 'Find my next meeting and summarize it' }
  ],
  tools: session.tools,
  maxToolRoundtrips: 5
});

// Listen for tool calls
result.onToolCall((toolCall) => {
  console.log('Tool called:', toolCall.name);
});

// Stream the final text
for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

Full Delta Stream

For complete control over the stream, including tool calls:

const result = streamText({
  model: openai('gpt-4'),
  messages,
  tools: session.tools
});

for await (const delta of result.fullStream) {
  switch (delta.type) {
    case 'text-delta':
      process.stdout.write(delta.textDelta);
      break;
    case 'tool-call':
      console.log('Calling tool:', delta.toolName);
      break;
    case 'tool-result':
      console.log('Tool result received');
      break;
  }
}

Server-Side Streaming (Next.js)

Here's an example for Next.js API routes:

import { streamText } from 'ai';
import { metorialAiSdk } from '@metorial/ai-sdk';
import { Metorial } from 'metorial';

export async function POST(req: Request) {
  const { messages } = await req.json();
  
  const metorial = new Metorial({
    apiKey: process.env.METORIAL_API_KEY!
  });
  
  return metorial.withProviderSession(
    metorialAiSdk,
    { serverDeployments: [process.env.DEPLOYMENT_ID!] },
    async session => {
      const result = streamText({
        model: openai('gpt-4'),
        messages,
        tools: session.tools
      });
      
      return result.toDataStreamResponse();
    }
  );
}

Best Practices

• Always use streaming for user-facing applications
• Monitor tool calls to show loading states to users
• Handle stream errors with proper error boundaries
• Consider implementing retry logic for failed streams
• Use toDataStreamResponse() for easy Next.js integration