Webscrapingai MCP server

Connect Webscrapingai to Claude, ChatGPT, Cursor, Claude Code, Codex, Gemini, GitHub Copilot and any other MCP client, for your whole team. Metorial adds security, governance, and tracing, and gives everyone one Magic MCP URL to connect.

What you can do with the Webscrapingai MCP server

Ask your agent in plain language. Each of these maps to a real tool on the server.

  • Use the ask question about page tool in Webscrapingai
  • Use the scrape html tool in Webscrapingai
  • Use the get account info tool in Webscrapingai
  • Use the select page elements tool in Webscrapingai
  • Use the extract text tool in Webscrapingai
  • Use the extract structured fields tool in Webscrapingai

Supported Tools

ask_question

Ask Question About Page

Ask a natural language question about any webpage and receive an AI-generated answer as plain text. The AI reads the page content and provides a direct answer without requiring HTML parsing. Useful for extracting specific insights, facts, or summaries from webpage content.

scrape_html

Scrape HTML

Fetch the full rendered HTML of any webpage. Supports JavaScript rendering via headless Chromium, proxy rotation, geo-targeting, device emulation, and custom headers. Use POST mode to submit form data or interact with target page APIs. Optionally execute custom JavaScript on the page before retrieving the HTML.

get_account_info

Get Account Info

Retrieve the current WebScraping.AI account status including remaining API credits, credit reset time, and concurrency limits. Useful for monitoring usage and ensuring sufficient credits before running scraping tasks.

select_elements

Select Page Elements

Extract HTML from specific page elements using CSS selectors. Supports single or multiple selectors to retrieve targeted portions of a webpage. Useful when you only need specific parts of a page like headings, prices, navigation, article content, or any element identifiable by CSS selector.

extract_text

Extract Text

Extract visible text content from any webpage, stripped of HTML markup. Returns clean Markdown ideal for feeding into LLMs, text analysis, or content processing pipelines. Supports plain text, JSON (with title, description, and text), or XML output formats. Can optionally include links found on the page.

extract_fields

Extract Structured Fields

Extract specific data fields from any webpage as structured JSON using AI. Define field names and natural language descriptions of what to extract, and the AI interprets the page to return the values. Works well for product details, articles, profiles, contact info, and similar structured content.

search_google

Search Google

Retrieve parsed Google search results for a query, including organic result titles, links, snippets, related searches, and pagination when available.

get_structured_data

Get Structured Page Data

Retrieve structured JSON from a public page on a supported site such as YouTube, TikTok, X, LinkedIn, Instagram, or Reddit. The provider detects the site and page type from the URL.

Connect Webscrapingai to your AI client

Every client uses the same Magic MCP URL, and so does any other MCP client. Add it once and sign in with SSO.

  1. Enable Webscrapingai in Metorial. In your Metorial portal, enable the Webscrapingai MCP server and sign in with the account that owns the data. Access stays governed by your team policies.
  2. Add a custom connector. Open Customize, then Connectors. Select Add, then Custom, then Web.
  3. Paste your Metorial MCP URL. Name it Metorial, paste your Magic MCP URL (<your-magic-mcp-url>) and continue.
  4. Sign in. Select Add and sign in to load Webscrapingai tools.

Run Webscrapingai through the Metorial gateway

One governed layer for every app your agents reach, not a separate connector per tool.

  • Give Webscrapingai and every other app to your whole team through one MCP gateway, behind your SSO.
  • Scope which teams and agents can use Webscrapingai tools with role-based access control, and trace every call.
  • Prefer to run it yourself? Metorial is open source and self-hostable.

More integrations teams use with Webscrapingai

GitHub

Manage repositories, issues, and pull requests. Create and configure branches, star repositories, review code, and merge changes. Automate CI/CD workflows with GitHub Actions, manage workflow runs, secrets, and artifacts. Track issues with labels, milestones, and assignees. Search across code, repositories, issues, and users. Manage organizations, teams, and memberships. Create and manage projects, gists, packages, deployments, and environments. Access security alerts including code scanning, secret scanning, and Dependabot alerts. Read and write file contents in repositories. Manage webhooks, notifications, and codespaces.

Salesforce

Manage CRM data including Accounts, Contacts, Leads, Opportunities, Cases, and custom objects. Create, read, update, and delete records. Query data using SOQL and search across objects using SOSL. Perform bulk data operations for large-scale imports, exports, and migrations. Execute composite requests to batch multiple operations in a single API call. Access analytics, reports, and dashboards. Manage files and attachments associated with records. Interact with Chatter feeds, posts, and groups for social collaboration. Manage org metadata including custom objects, fields, layouts, and workflows. Query data using GraphQL for precise data retrieval across related objects.

Sharepoint

Manage SharePoint sites, document libraries, lists, and files. Create, read, update, and delete lists and list items with custom columns. Resolve site users to numeric Person/Group LookupId values. Upload, download, move, copy, and version files in document libraries. Search across sites, files, folders, lists, and list items using Microsoft Search. Manage permissions at site, list, and item levels with granular access control. Define and manage content types and site columns. Retrieve site properties and search for sites across Microsoft 365.

Airtable

Create, read, update, and delete records in Airtable bases and tables. Manage base schemas including creating tables and fields. Filter records using formulas, sort by fields, and scope queries to specific views. Upsert records to find, create, or update in a single call. Upload attachments to records, read and write record comments, list accessible bases, and receive real-time base change events through webhooks.

Sentry

Track, manage, and resolve application errors and performance issues. List, query, and bulk-update issues and error events with filters like status, assignment, and tags. Create and manage releases, associate commits, and upload source maps. Configure issue alert rules and metric alert rules with notification actions. Set up cron monitors to detect missed or failed scheduled jobs. Build custom dashboards with configurable widgets. Run ad-hoc Discover queries across errors and transactions for performance analysis. Manage organizations, teams, projects, and members. Provision users via SCIM. Access session replay data.

Bitbucket

Manage Git repositories, pull requests, and CI/CD pipelines on Bitbucket Cloud. Create, fork, and configure repositories within workspaces and projects. Create, review, approve, merge, and decline pull requests with inline code comments. Browse source code, list commits, and manage branches and tags. Trigger, monitor, and manage Bitbucket Pipelines. List workspace members, configure repository default reviewers and branch restrictions, create and manage repository webhooks, and search code across repositories.

Technical notes for Webscrapingai

Scrape and extract data from any website with automatic proxy rotation, headless browser rendering, CAPTCHA solving, and retries. Fetch full rendered HTML or plain text from web pages. Extract specific page elements using CSS selectors. Ask natural language questions about webpage content and receive AI-generated answers. Extract structured JSON fields from pages using AI-powered field descriptions. Execute custom JavaScript on loaded pages to interact with dynamic content. Emulate desktop, mobile, or tablet devices. Route requests through datacenter or residential proxies with geo-targeting support. Send custom headers and cookies for authenticated scraping. Check account credit balance and usage limits.

Frequently asked questions

Common questions about connecting Webscrapingai to AI agents with Metorial.

  1. How many tools does the Webscrapingai MCP server have?
    The Webscrapingai MCP server exposes 8 tools, including Ask Question About Page, Scrape HTML, Get Account Info. Each one is governed and traced through Metorial.
  2. Webscrapingai exposes tools your agents call on demand; it does not publish event triggers yet. Everything still runs through one governed MCP URL.
  3. Any MCP client, including VS Code, Windsurf, Zed, OpenCode and Grok. Each one reaches the Webscrapingai MCP server through the same Metorial URL.
  4. Use role-based access control to decide which teams, users, and agents can use Webscrapingai and which of its tools they can call.
  5. Yes. Metorial is open source, so you can run the Webscrapingai MCP server yourself or use the hosted gateway.

Connect Webscrapingai to production AI agents

See how Metorial gives your team governed access to Webscrapingai, with tracing and security controls built in.