Code review bot with GitHub

Build an AI-powered code review bot that reads changes from your git repository, analyzes them for security vulnerabilities, code smells, and style issues, then posts general and line-specific review comments to GitHub — requesting changes when it finds problems and approving clean pull requests.

Prerequisites

Before building the code review bot, ensure you have:

1. Metorial setup:

   • Active Metorial account at platform.metorial.com
   • Project created in your organization
   • Metorial API key (generate in Dashboard → Developer → API Keys)

2. GitHub repository:

   • Local clone of your GitHub repository
   • Repository with write access on GitHub
   • At least one pull request for testing
   • Admin access to authorize OAuth

3. AI provider:

   • Anthropic API key (Claude Sonnet 4 or newer recommended for code analysis)

4. Development environment:

   • Node.js 18+ (TypeScript) or Python 3.9+ installed
   • Basic knowledge of async/await patterns

Architecture overview

The code review bot workflow:

1. Input: User provides local repository path, branch names, and PR number
2. Fetch Code Changes: Bot reads git diff locally from your repository
3. AI Analysis: Claude analyzes the diff for:
   • Security vulnerabilities (SQL injection, XSS, exposed secrets)
   • Code smells (duplication, long functions, complexity)
   • Style violations (naming, formatting, consistency)
   • Best practices (error handling, documentation, testing)
4. Post Review: Bot posts the review to GitHub via the GitHub provider:
   • Overall summary comment
   • Line-specific feedback on issues
   • Review decision (approve or request changes)

Tools used: Local git (read code changes) + AI Model (code analysis) + GitHub provider (post reviews)

Step 1: Configure GitHub provider

Configure the GitHub provider from Metorial's catalog to enable your bot to interact with GitHub.

Step 2: Set up OAuth authentication

Your code review bot needs permission to access your GitHub repositories.

npm install metorial @metorial/anthropic @anthropic-ai/sdk

import { Metorial } from 'metorial';

const metorial = new Metorial({
  apiKey: "YOUR-METORIAL-API-KEY"
});

async function setupGitHubOAuth() {
  const githubSetup = await metorial.providers.setupSessions.create({
    providerId: 'YOUR-GITHUB-PROVIDER-ID',
    providerAuthMethodId: 'oauth'
  });

  console.log('Authorize GitHub here:', githubSetup.url);

  // Wait for authorization
  const completed = await metorial.waitForSetupSession([githubSetup]);
  console.log('✓ GitHub authorized!');

  // Save the auth config ID for future use
  return completed.authConfig.id;
}

setupGitHubOAuth();

import asyncio
from metorial import Metorial

async def setup_github_oauth():
    metorial = Metorial(api_key="YOUR-METORIAL-API-KEY")

    github_setup = await metorial.providers.setup_sessions.create(
        provider_id="YOUR-GITHUB-PROVIDER-ID",
        provider_auth_method_id="oauth"
    )

    print(f"Authorize GitHub here: {github_setup.url}")

    # Wait for authorization
    completed = await metorial.wait_for_setup_session(github_setup)
    print("✓ GitHub authorized!")

    # Save the auth config ID for future use
    return completed.auth_config.id

asyncio.run(setup_github_oauth())

Step 3: Build the code review bot

Create the main bot that analyzes pull requests and posts review comments.

import { Metorial } from 'metorial';
import { metorialAnthropic } from '@metorial/anthropic';
import Anthropic from '@anthropic-ai/sdk';
import { execSync } from 'child_process';
import * as path from 'path';
import * as fs from 'fs';

const metorial = new Metorial({
  apiKey: "YOUR-METORIAL-API-KEY"
});

const anthropic = new Anthropic({
  apiKey: "YOUR_ANTHROPIC_API_KEY"
});

// GitHub integration credentials
const GITHUB_DEPLOYMENT_ID = "GITHUB_DEPLOYMENT_ID";
const GITHUB_AUTH_CONFIG_ID = "GITHUB_AUTH_CONFIG_ID";

async function reviewPullRequest(
  repoPath: string,
  owner: string,
  repo: string,
  branchName: string,
  prNumber: number,
  baseBranch: string = 'main'
) {
  console.log(`Starting review of PR #${prNumber} (${branchName})`);

  // Validate repository path
  if (!fs.existsSync(repoPath)) {
    throw new Error(`Repository path does not exist: ${repoPath}`);
  }

  const gitDir = path.join(repoPath, '.git');
  if (!fs.existsSync(gitDir)) {
    throw new Error(`Not a git repository: ${repoPath}`);
  }

  // Get diff from local repository
  console.log(`Fetching diff: ${baseBranch}...${branchName}`);
  const gitDiffCmd = `cd "${repoPath}" && git diff ${baseBranch}...${branchName}`;

  let diffContent: string;
  try {
    diffContent = execSync(gitDiffCmd, { encoding: 'utf-8' });
  } catch (error) {
    throw new Error(`Git diff failed: ${error}`);
  }

  if (!diffContent.trim()) {
    console.log('No changes found in diff');
    return;
  }

  console.log(`Analyzing ${diffContent.length} characters of diff`);

  // Create Metorial session with GitHub integration
  await metorial.withProviderSession(
    metorialAnthropic,
    {
      providers: [
        {
          providerDeploymentId: GITHUB_DEPLOYMENT_ID,
          providerAuthConfigId: GITHUB_AUTH_CONFIG_ID
        }
      ]
    },
    async session => {
      // Prepare review prompt with diff content
      const messages: Anthropic.MessageParam[] = [
        {
          role: 'user',
          content: `You are an expert code reviewer. You MUST post a review to GitHub after analyzing the code.

Repository: ${owner}/${repo}
Branch: ${branchName}
PR Number: #${prNumber}

DIFF CONTENT:
\`\`\`diff
${diffContent}
\`\`\`

CRITICAL: You MUST call the create_pull_request_review tool. Do NOT just provide analysis - you must POST the review to GitHub.

INSTRUCTIONS:
1. Analyze the diff for:
   - Security vulnerabilities (SQL injection, XSS, exposed secrets, unsafe operations)
   - Code quality issues (duplication, complexity, error handling)
   - Style problems (naming, formatting, consistency)
   - Best practices (documentation, testing, edge cases)

2. IMMEDIATELY call create_pull_request_review tool with:
   - owner: "${owner}"
   - repo: "${repo}"
   - pull_number: ${prNumber}
   - body: Your summary of findings
   - event: "APPROVE" (if no issues) or "REQUEST_CHANGES" (if issues found)
   - comments: Array of line-specific comments if you found issues

You MUST use the tool to post the review. START NOW.`
        }
      ];

      // Send initial request to Claude
      let response = await anthropic.messages.create({
        model: 'claude-sonnet-4-5',
        max_tokens: 8192,
        tools: session.tools,
        messages
      });

      // Handle tool calls in agentic loop
      while (response.stop_reason === 'tool_use') {
        const toolUseBlocks = response.content.filter(
          (block): block is Anthropic.ToolUseBlock => block.type === 'tool_use'
        );

        console.log(`Executing ${toolUseBlocks.length} tool(s)...`);

        // Execute tools via Metorial
        const toolResults = await session.callTools(toolUseBlocks);

        // Add assistant response and tool results to conversation
        messages.push({ role: 'assistant', content: response.content });
        messages.push(toolResults);

        // Continue conversation
        response = await anthropic.messages.create({
          model: 'claude-sonnet-4-5',
          max_tokens: 8192,
          messages,
          tools: session.tools
        });
      }

      // Get final summary
      const finalText = response.content
        .filter((block): block is Anthropic.TextBlock => block.type === 'text')
        .map(block => block.text)
        .join('\n');

      console.log(`Review completed: ${finalText}`);
    }
  );
}

What this code does:

1. Reads git diff locally from your repository between the base and feature branches
2. Creates a provider session with GitHub provider for posting reviews
3. Sends the diff to Claude with analysis instructions
4. AI analyzes the code and identifies issues
5. AI posts review to GitHub using create_pull_request_review tool with findings
6. Handles multi-step workflow through agentic loop until review is complete
7. Handles errors gracefully: If tool calls fail, the AI receives error messages and can retry or adjust its approach

Step 4: Test with a security issue

Let's test the bot with a PR containing a security vulnerability.

Scenario: Create a test PR with SQL injection vulnerability.

Test PR content (example):

function getUserData(userId) {
  const query = `SELECT * FROM users WHERE id = ${userId}`;
  return database.query(query);
}

Run the bot:

reviewPullRequest(
  '/path/to/your/local/repo',
  'your-username',
  'your-repo',
  'feature-branch',
  123
);

Expected behavior:

1. Bot reads git diff from local repository for PR #123
2. AI detects SQL injection vulnerability in the query string
3. Bot posts review with:
   • General comment: "Found 1 security vulnerability that needs immediate attention."
   • Line-specific comment on the SQL query line: "🚨 SQL injection vulnerability detected. User input is directly interpolated into the query. Use parameterized queries instead: SELECT * FROM users WHERE id = ? with bound parameters."
4. Bot submits review with REQUEST_CHANGES status

Step 5: Test with clean code

Test the bot with a clean PR to verify the approval workflow.

Scenario: PR with well-written code.

Test PR content (example):

export function isValidEmail(email: string): boolean {
  const emailRegex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;
  return emailRegex.test(email);
}

export function sanitizeInput(input: string): string {
  return input
    .replace(/&/g, '&')
    .replace(/</g, '&lt;')
    .replace(/>/g, '&gt;')
    .replace(/"/g, '&quot;')
    .replace(/'/g, '&#x27;');
}

Run the bot:

reviewPullRequest(
  '/path/to/your/local/repo',
  'your-username',
  'your-repo',
  'feature-branch',
  124
);

Expected behavior:

1. Bot reads git diff from local repository for PR #124
2. AI finds:
   • ✓ Proper documentation with JSDoc comments
   • ✓ Clear function names following conventions
   • ✓ Security-conscious implementation (XSS prevention)
   • ✓ No code smells or style violations
3. Bot posts review comment: "Code looks excellent! Clean implementation with proper documentation, security considerations, and clear naming. The XSS sanitization is thorough and the email validation regex is appropriate."
4. Bot submits review with APPROVE status

Troubleshooting

Common issues and solutions when building your code review bot:

Advanced customization

Enhance your code review bot with these customizations:

Example: Adding custom rules

Update the AI prompt with your standards:

const customPrompt = `You are an expert code reviewer following these standards:

Company Coding Standards:
- All functions must have JSDoc comments
- Use async/await instead of .then() promises
- Maximum function length: 50 lines
- All API responses must include error handling
- Never use "any" type in TypeScript

Review pull request #${prNumber}...`;

Production considerations

Before deploying to production:

1. Webhook Integration: Set up GitHub webhooks to trigger reviews automatically when PRs are opened or updated. You'll need the admin:repo_hook OAuth scope and a webhook endpoint that receives GitHub events. See GitHub's Webhook documentation for implementation details.
2. Rate Limiting: Implement rate limiting to avoid hitting GitHub API limits (5000 requests/hour for authenticated apps)
3. Concurrency: Queue reviews to handle multiple PRs simultaneously without overwhelming the AI API
4. Error Handling: Add try/catch blocks and retry logic for API failures
5. Review History: Store review results in a database for analytics and team insights
6. Configurable Rules: Allow teams to customize review criteria per repository via config files
7. Cost Management: Monitor AI API usage and token costs, especially for large PRs with many files
8. Privacy: Ensure sensitive code doesn't get logged or sent to unauthorized services