OpenAI Integration

Monitor OpenAI API calls including GPT-4, GPT-3.5, and embeddings

The OpenAI integration automatically instruments OpenAI SDK calls to capture performance data and errors.

Installation

Node.js

The integration is enabled by default:

import * as Sentry from '@sentry/node';

Sentry.init({
  dsn: 'your-dsn',
  // openAIIntegration is included by default
});

Browser

Use manual instrumentation:

import * as Sentry from '@sentry/browser';
import { instrumentOpenAiClient } from '@sentry/browser';
import OpenAI from 'openai';

Sentry.init({
  dsn: 'your-dsn',
});

const openai = instrumentOpenAiClient(new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
  dangerouslyAllowBrowser: true,
}));

Basic Usage

Just use the OpenAI SDK normally:

import OpenAI from 'openai';

const openai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

// Automatically instrumented
const completion = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is the capital of France?' },
  ],
});

Configuration

Default Behavior

By default, prompts and responses are not captured:

Sentry.init({
  dsn: 'your-dsn',
  sendDefaultPii: false, // Default: no prompts/responses
});

Capture Prompts and Responses

Global Setting

Enable for all AI integrations:

Sentry.init({
  dsn: 'your-dsn',
  sendDefaultPii: true, // Captures all inputs/outputs
});

OpenAI Only

Enable only for OpenAI:

Sentry.init({
  dsn: 'your-dsn',
  sendDefaultPii: false,
  integrations: [
    Sentry.openAIIntegration({
      recordInputs: true,
      recordOutputs: true,
    }),
  ],
});

Inputs Only

Capture prompts but not responses:

Sentry.init({
  dsn: 'your-dsn',
  integrations: [
    Sentry.openAIIntegration({
      recordInputs: true,
      recordOutputs: false,
    }),
  ],
});

Integration Options

recordInputs · boolean · default: "sendDefaultPii"

Capture prompt messages

recordOutputs · boolean · default: "sendDefaultPii"

Capture completion responses

Captured Data

Always Captured

These attributes are always included:

{
  'gen_ai.operation.name': 'chat',
  'gen_ai.request.model': 'gpt-4',
  'gen_ai.system': 'openai',
  'gen_ai.usage.input_tokens': 25,
  'gen_ai.usage.output_tokens': 100,
  'gen_ai.response.finish_reasons': ['stop'],
  'gen_ai.request.temperature': 1.0,
  'gen_ai.request.max_tokens': 256,
}

When recordInputs: true

Prompt messages are captured:

{
  'gen_ai.prompt.0.role': 'system',
  'gen_ai.prompt.0.content': 'You are a helpful assistant.',
  'gen_ai.prompt.1.role': 'user',
  'gen_ai.prompt.1.content': 'What is the capital of France?',
}

When recordOutputs: true

Completion responses are captured:

{
  'gen_ai.completion.0.role': 'assistant',
  'gen_ai.completion.0.content': 'The capital of France is Paris.',
  'gen_ai.completion.0.finish_reason': 'stop',
}

Supported Operations

The integration instruments all OpenAI operations:

Chat Completions

const completion = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Hello!' }],
});
// Span: gen_ai.chat.completions

Streaming Completions

const stream = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'Hello!' }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
// Span includes full streaming response

Embeddings

const embedding = await openai.embeddings.create({
  model: 'text-embedding-ada-002',
  input: 'Your text here',
});
// Span: gen_ai.embeddings.create

Function Calling

const completion = await openai.chat.completions.create({
  model: 'gpt-4',
  messages: [{ role: 'user', content: 'What is the weather?' }],
  functions: [
    {
      name: 'get_weather',
      description: 'Get current weather',
      parameters: {
        type: 'object',
        properties: {
          location: { type: 'string' },
        },
      },
    },
  ],
});
// Tool calls captured in span attributes

Practical Examples

Customer Support Bot

import * as Sentry from '@sentry/node';
import OpenAI from 'openai';

const openai = new OpenAI();

async function answerSupportQuery(userId, query) {
  return await Sentry.startSpan(
    {
      name: 'Answer Support Query',
      op: 'ai.support',
      attributes: {
        'user.id': userId,
        'support.category': 'general',
      },
    },
    async () => {
      const completion = await openai.chat.completions.create({
        model: 'gpt-4',
        messages: [
          {
            role: 'system',
            content: 'You are a helpful customer support agent.',
          },
          { role: 'user', content: query },
        ],
      });
      
      return completion.choices[0].message.content;
    }
  );
}

Content Generation

async function generateBlogPost(topic, keywords) {
  return await Sentry.startSpan(
    {
      name: 'Generate Blog Post',
      op: 'ai.content_generation',
      attributes: {
        'content.topic': topic,
        'content.keywords': keywords.join(', '),
      },
    },
    async () => {
      const completion = await openai.chat.completions.create({
        model: 'gpt-4',
        messages: [
          {
            role: 'system',
            content: 'You are a professional content writer.',
          },
          {
            role: 'user',
            content: `Write a blog post about ${topic} including these keywords: ${keywords.join(', ')}`,
          },
        ],
        temperature: 0.8,
        max_tokens: 2000,
      });
      
      return completion.choices[0].message.content;
    }
  );
}

Semantic Search with Embeddings

async function searchDocuments(query, documents) {
  return await Sentry.startSpan(
    { name: 'Semantic Search', op: 'ai.search' },
    async () => {
      // Get query embedding
      const queryEmbedding = await openai.embeddings.create({
        model: 'text-embedding-ada-002',
        input: query,
      });
      
      // Compare with document embeddings
      const results = findSimilarDocuments(
        queryEmbedding.data[0].embedding,
        documents
      );
      
      return results;
    }
  );
}

Error Handling

async function generateWithRetry(prompt, maxRetries = 3) {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const completion = await openai.chat.completions.create({
        model: 'gpt-4',
        messages: [{ role: 'user', content: prompt }],
      });
      
      return completion.choices[0].message.content;
    } catch (error) {
      Sentry.captureException(error, {
        contexts: {
          openai: {
            attempt,
            max_retries: maxRetries,
            model: 'gpt-4',
          },
        },
      });
      
      if (attempt === maxRetries) {
        throw error;
      }
      
      // Exponential backoff
      await new Promise(resolve => setTimeout(resolve, 2 ** attempt * 1000));
    }
  }
}

Performance Monitoring

View OpenAI performance in Sentry:

• Response Times: Track API latency
• Token Usage: Monitor prompt and completion tokens
• Error Rates: Identify rate limits and failures
• Model Comparison: Compare performance across models

Source Code

The OpenAI integration is implemented in:

packages/node/src/integrations/tracing/openai/index.ts:11

Privacy Best Practices

Be careful about capturing sensitive user data in prompts and responses.

Filter Sensitive Content

Sentry.init({
  dsn: 'your-dsn',
  sendDefaultPii: true,
  
  beforeSendSpan(span) {
    // Remove PII from prompts
    const promptContent = span.attributes?.['gen_ai.prompt.1.content'];
    if (promptContent && typeof promptContent === 'string') {
      span.attributes['gen_ai.prompt.1.content'] = promptContent
        .replace(/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/g, '[EMAIL]')
        .replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[SSN]');
    }
    
    return span;
  },
});

Conditional Capture

const isSensitiveQuery = query.toLowerCase().includes('password');

const integration = isSensitiveQuery
  ? Sentry.openAIIntegration({ recordInputs: false, recordOutputs: false })
  : Sentry.openAIIntegration({ recordInputs: true, recordOutputs: true });

Troubleshooting

Spans Not Appearing

Ensure tracing is enabled:

Sentry.init({
  dsn: 'your-dsn',
  tracesSampleRate: 1.0, // Capture 100% of traces
});

Missing Token Data

Token usage is always captured from OpenAI’s response:

// Check the span attributes in Sentry for:
// - gen_ai.usage.input_tokens
// - gen_ai.usage.output_tokens

Related

• AI Monitoring Overview
• LangChain Integration
• Anthropic Integration