What You’ll Build
Build an AI-powered code review bot that:- Reads code changes locally from your git repository
- Analyzes code for security vulnerabilities, code smells, and style issues
- Posts detailed review comments to GitHub (both general feedback and line-specific suggestions)
- Automatically requests changes for issues or approves clean PRs
What you’ll learn:
- Deploying the GitHub MCP server
- Setting up OAuth for GitHub repositories
- Creating an AI agent that reviews code
- Posting GitHub review comments programmatically
- Create a Metorial account
- Create API keys
- GitHub account with a repository and at least one PR
- Anthropic API key (Claude Sonnet 4 or newer recommended)
Prerequisites
Before building the code review bot, ensure you have:-
Metorial setup:
- Active Metorial account at app.metorial.com
- Project created in your organization
- Metorial API key (generate in Dashboard → Home → Connect to Metorial)
-
GitHub repository:
- Local clone of your GitHub repository
- Repository with write access on GitHub
- At least one pull request for testing
- Admin access to authorize OAuth
-
AI provider:
- Anthropic API key (Claude Sonnet 4 or newer recommended for code analysis)
-
Development environment:
- Node.js 18+ (TypeScript) or Python 3.9+ installed
- Basic knowledge of async/await patterns
Architecture Overview
The code review bot workflow:- Input: User provides local repository path, branch names, and PR number
- Fetch Code Changes: Bot reads git diff locally from your repository
- AI Analysis: Claude analyzes the diff for:
- Security vulnerabilities (SQL injection, XSS, exposed secrets)
- Code smells (duplication, long functions, complexity)
- Style violations (naming, formatting, consistency)
- Best practices (error handling, documentation, testing)
- Post Review: Bot posts the review to GitHub via the GitHub MCP server:
- Overall summary comment
- Line-specific feedback on issues
- Review decision (approve or request changes)
Step 1: Deploy GitHub MCP Server
Deploy the GitHub MCP server from Metorial’s catalog to enable your bot to interact with GitHub.Deploy GitHub Server
Click the GitHub server, then click Deploy Server → Server Deployment.Give your deployment a descriptive name like “Code Review Bot GitHub”.
Save your GitHub deployment ID—you’ll need it for OAuth setup (Step 2) and in your bot code (Step 3).
Step 2: Set Up OAuth Authentication
Your code review bot needs permission to access your GitHub repositories.Authorize in Browser
- Open the printed OAuth URL in your browser
- Sign in to GitHub if needed
- Review and approve the permissions (the bot needs
reposcope to read PRs and post comments) - You’ll be redirected to your callback URL (or see a confirmation page)
Required OAuth Scopes:The GitHub MCP server requires the
repo scope for posting reviews, which provides:- Write access to post review comments and line-specific feedback
- Permission to approve PRs or request changes on your repositories
admin:repo_hook. The basic bot functionality shown in this tutorial only requires repo.The required scopes are automatically requested when you authorize via the OAuth URL.Step 3: Build the Code Review Bot
Create the main bot that analyzes pull requests and posts review comments.- Reads git diff locally from your repository between the base and feature branches
- Creates a provider session with GitHub MCP server for posting reviews
- Sends the diff to Claude with analysis instructions
- AI analyzes the code and identifies issues
- AI posts review to GitHub using
create_pull_request_reviewtool with findings - Handles multi-step workflow through agentic loop until review is complete
- Handles errors gracefully: If tool calls fail, the AI receives error messages and can retry or adjust its approach
This uses Claude’s agentic capabilities—the AI decides which tools to call and when. You don’t need to write explicit logic for fetching files, analyzing code, or posting comments.
Step 4: Test with a Security Issue
Let’s test the bot with a PR containing a security vulnerability. Scenario: Create a test PR with SQL injection vulnerability. Test PR content (example):- Bot reads git diff from local repository for PR #123
- AI detects SQL injection vulnerability in the query string
- Bot posts review with:
- General comment: “Found 1 security vulnerability that needs immediate attention.”
- Line-specific comment on the SQL query line: ”🚨 SQL injection vulnerability detected. User input is directly interpolated into the query. Use parameterized queries instead:
SELECT * FROM users WHERE id = ?with bound parameters.”
- Bot submits review with REQUEST_CHANGES status
The bot workflow:
- Reads git diff from your local repository
- Sends diff content to AI for analysis
- AI identifies the security issue in the code
- AI calls
create_pull_request_reviewtool to post review to GitHub withREQUEST_CHANGESstatus and detailed comments
Step 5: Test with Clean Code
Test the bot with a clean PR to verify the approval workflow. Scenario: PR with well-written code. Test PR content (example):- Bot reads git diff from local repository for PR #124
- AI finds:
- ✓ Proper documentation with JSDoc comments
- ✓ Clear function names following conventions
- ✓ Security-conscious implementation (XSS prevention)
- ✓ No code smells or style violations
- Bot posts review comment: “Code looks excellent! Clean implementation with proper documentation, security considerations, and clear naming. The XSS sanitization is thorough and the email validation regex is appropriate.”
- Bot submits review with APPROVE status
Troubleshooting
Common issues and solutions when building your code review bot:Bot doesn't post reviews or returns empty responses
Bot doesn't post reviews or returns empty responses
Possible causes:
- OAuth session expired or invalid
- PR doesn’t exist or has no changed files
- AI model didn’t call the review tools
- Verify your OAuth session is active: re-run the OAuth setup if needed
- Check the PR exists and has commits:
gh pr view <pr-number> - Increase
max_tokensin the AI request (try 8192 instead of 4096) - Review Metorial dashboard logs to see which tools were called
- Ensure your prompt explicitly instructs the AI to post reviews (see code examples)
Error: Tool not found or tool call failed
Error: Tool not found or tool call failed
Possible causes:
- Incorrect tool name in code
- GitHub MCP server deployment not active
- OAuth permissions insufficient
- Verify you’re using the correct tool name:
create_pull_request_review - Check your GitHub server deployment is running in Metorial dashboard
- Confirm OAuth session is associated with the correct deployment ID
- Verify the
reposcope was granted during OAuth authorization
Reviews are generic or miss obvious issues
Reviews are generic or miss obvious issues
Possible causes:
- AI prompt lacks specific guidelines
- Code context is truncated
- Model capabilities insufficient
- Add specific coding standards to your prompt (see “Custom Review Rules” in Advanced Customization)
- Increase
max_tokensto allow longer analysis (8192-16384 for large PRs) - Use Claude Sonnet 4 or newer for better code understanding
- Provide example issues in the prompt to guide the AI’s analysis style
OAuth authorization fails or shows permission errors
OAuth authorization fails or shows permission errors
Bot times out on large pull requests
Bot times out on large pull requests
Possible causes:
- Too many changed files
- AI context window exceeded
- API rate limits
- Filter files by extension or directory (modify the prompt to focus on specific file types)
- Implement batching: review files in chunks rather than all at once
- Set a maximum file size limit (skip files >1000 lines)
- Use streaming responses to handle longer processing times
- For PRs with >20 files, consider the performance optimizations in Production Considerations
Rate limiting: GitHub API requests failing
Rate limiting: GitHub API requests failing
GitHub API limits: 5,000 requests/hour for authenticated appsSolutions:
- Implement exponential backoff when rate limit errors occur
- Cache PR data when running multiple reviews on the same PR
- Use conditional requests with ETags to avoid fetching unchanged data
- For production deployment, consider GitHub Enterprise with higher limits
- Track your usage in the Metorial dashboard to identify bottlenecks
If you encounter errors not covered here, check the Metorial dashboard logs (Monitoring section) to see detailed tool execution traces and error messages. You can also inspect the actual API requests being made.
Advanced Customization
Enhance your code review bot with these customizations:Custom Review Rules
Add company-specific coding standards to the AI prompt (e.g., “all public functions must have JSDoc comments”, “use async/await instead of promises”).
Language-Specific Analysis
Customize prompts for different languages:
- Python: PEP 8 compliance, type hints
- TypeScript: strict mode, interface usage
- JavaScript: ESLint rules, modern syntax
Review Severity Levels
Categorize issues as CRITICAL, WARNING, or SUGGESTION and adjust review status accordingly. Only block PRs for critical security issues.
Automatic Fixes
Generate suggested code fixes for common issues. The AI can propose corrections in review comments (e.g., reformatted code, added error handling).
Production Considerations
Before deploying to production:- Webhook Integration: Set up GitHub webhooks to trigger reviews automatically when PRs are opened or updated. You’ll need the
admin:repo_hookOAuth scope and a webhook endpoint that receives GitHub events. See GitHub’s Webhook documentation for implementation details. - Rate Limiting: Implement rate limiting to avoid hitting GitHub API limits (5000 requests/hour for authenticated apps)
- Concurrency: Queue reviews to handle multiple PRs simultaneously without overwhelming the AI API
- Error Handling: Add try/catch blocks and retry logic for API failures
- Review History: Store review results in a database for analytics and team insights
- Configurable Rules: Allow teams to customize review criteria per repository via config files
- Cost Management: Monitor AI API usage and token costs, especially for large PRs with many files
- Privacy: Ensure sensitive code doesn’t get logged or sent to unauthorized services
Performance Tip:For large PRs (>20 files), consider:
- Reviewing only changed lines instead of full files
- Batching file reviews to reduce token usage
- Implementing a maximum file size limit
- Allowing users to request specific file reviews
What’s Next?
Congratulations! You’ve built an AI-powered code review bot that analyzes pull requests for security issues, code quality, and best practices.Learn More
SDK Documentation
Explore advanced SDK features and patterns.
OAuth Guide
Learn more about managing GitHub authorization.
Monitoring Guide
Monitor bot performance and review activity.
Related Sample Projects
Slack Standup Bot
Build a bot that collects and summarizes team standup updates in Slack.
Interview Coordinator Bot
Create an AI coordinator that schedules interviews and sends professional emails.
Need help? Email us at support@metorial.com.