Connect Webscrapingai to Claude, ChatGPT, Cursor, Claude Code, Codex, Gemini, GitHub Copilot and any other MCP client, for your whole team. Metorial adds security, governance, and tracing, and gives everyone one Magic MCP URL to connect.
Ask your agent in plain language. Each of these maps to a real tool on the server.
Use the ask question about page tool in Webscrapingai
Use the scrape html tool in Webscrapingai
Use the get account info tool in Webscrapingai
Use the select page elements tool in Webscrapingai
Use the extract text tool in Webscrapingai
Use the extract structured fields tool in Webscrapingai
Supported Tools
ask_question
Ask Question About Page
Ask a natural language question about any webpage and receive an AI-generated answer as plain text. The AI reads the page content and provides a direct answer without requiring HTML parsing.
Useful for extracting specific insights, facts, or summaries from webpage content.
scrape_html
Scrape HTML
Fetch the full rendered HTML of any webpage. Supports JavaScript rendering via headless Chromium, proxy rotation, geo-targeting, device emulation, and custom headers.
Use POST mode to submit form data or interact with target page APIs. Optionally execute custom JavaScript on the page before retrieving the HTML.
get_account_info
Get Account Info
Retrieve the current WebScraping.AI account status including remaining API credits, credit reset time, and concurrency limits.
Useful for monitoring usage and ensuring sufficient credits before running scraping tasks.
select_elements
Select Page Elements
Extract HTML from specific page elements using CSS selectors. Supports single or multiple selectors to retrieve targeted portions of a webpage.
Useful when you only need specific parts of a page like headings, prices, navigation, article content, or any element identifiable by CSS selector.
extract_text
Extract Text
Extract visible text content from any webpage, stripped of HTML markup. Returns clean Markdown ideal for feeding into LLMs, text analysis, or content processing pipelines.
Supports plain text, JSON (with title, description, and text), or XML output formats. Can optionally include links found on the page.
extract_fields
Extract Structured Fields
Extract specific data fields from any webpage as structured JSON using AI. Define field names and natural language descriptions of what to extract, and the AI interprets the page to return the values.
Works well for product details, articles, profiles, contact info, and similar structured content.
search_google
Search Google
Retrieve parsed Google search results for a query, including organic result titles, links, snippets, related searches, and pagination when available.
get_structured_data
Get Structured Page Data
Retrieve structured JSON from a public page on a supported site such as YouTube, TikTok, X, LinkedIn, Instagram, or Reddit. The provider detects the site and page type from the URL.
Enable Webscrapingai in Metorial. In your Metorial portal, enable the Webscrapingai MCP server and sign in with the account that owns the data. Access stays governed by your team policies.
Add a custom connector. Open Customize, then Connectors. Select Add, then Custom, then Web.
Paste your Metorial MCP URL. Name it Metorial, paste your Magic MCP URL (<your-magic-mcp-url>) and continue.
Sign in. Select Add and sign in to load Webscrapingai tools.
Enable Webscrapingai in Metorial. In your Metorial portal, enable the Webscrapingai MCP server and sign in with the account that owns the data. Access stays governed by your team policies.
Add a custom MCP server. On ChatGPT web, open Plugins, select +, then Add custom MCP server.
Paste your Metorial MCP URL. Name it Metorial, paste your Magic MCP URL (<your-magic-mcp-url>), choose OAuth and accept the risk notice.
Create and install the plugin. Select Create as a plugin, install it and sign in to load Webscrapingai tools.
Enable Webscrapingai in Metorial. In your Metorial portal, enable the Webscrapingai MCP server and sign in with the account that owns the data. Access stays governed by your team policies.
Add Metorial to your MCP config. Use ~/.cursor/mcp.json for every project, or .cursor/mcp.json for one.
Sign in. Open Customize in the sidebar, find Metorial and sign in to load Webscrapingai tools.
{
"mcpServers":{
"metorial":{
"url":"<your-magic-mcp-url>"
}
}
}
Enable Webscrapingai in Metorial. In your Metorial portal, enable the Webscrapingai MCP server and sign in with the account that owns the data. Access stays governed by your team policies.
Add the server over HTTP. Add --scope user to use it in every project.
Sign in. Run /mcp, select Metorial and sign in to load Webscrapingai tools.
claude mcp add --transport http metorial <your-magic-mcp-url>
Enable Webscrapingai in Metorial. In your Metorial portal, enable the Webscrapingai MCP server and sign in with the account that owns the data. Access stays governed by your team policies.
Add the server and sign in. The Codex CLI, IDE extension and ChatGPT desktop app share this config.
Check the connection. Type /mcp in Codex to see Metorial and its Webscrapingai tools.
codex mcp add metorial --url <your-magic-mcp-url>
codex mcp login metorial
Run Webscrapingai through the Metorial gateway
One governed layer for every app your agents reach, not a separate connector per tool.
Give Webscrapingai and every other app to your whole team through one MCP gateway, behind your SSO.
Scope which teams and agents can use Webscrapingai tools with role-based access control, and trace every call.
Prefer to run it yourself? Metorial is open source and self-hostable.
Scrape and extract data from any website with automatic proxy rotation, headless browser rendering, CAPTCHA solving, and retries. Fetch full rendered HTML or plain text from web pages. Extract specific page elements using CSS selectors. Ask natural language questions about webpage content and receive AI-generated answers. Extract structured JSON fields from pages using AI-powered field descriptions. Execute custom JavaScript on loaded pages to interact with dynamic content. Emulate desktop, mobile, or tablet devices. Route requests through datacenter or residential proxies with geo-targeting support. Send custom headers and cookies for authenticated scraping. Check account credit balance and usage limits.
Frequently asked questions
Common questions about connecting Webscrapingai to AI agents with Metorial.
How many tools does the Webscrapingai MCP server have?
The Webscrapingai MCP server exposes 8 tools, including Ask Question About Page, Scrape HTML, Get Account Info. Each one is governed and traced through Metorial.
Does the Webscrapingai MCP server support triggers?
Webscrapingai exposes tools your agents call on demand; it does not publish event triggers yet. Everything still runs through one governed MCP URL.
Which AI clients can use the Webscrapingai MCP server?
Any MCP client, including VS Code, Windsurf, Zed, OpenCode and Grok. Each one reaches the Webscrapingai MCP server through the same Metorial URL.
How do I give only some teams access to Webscrapingai?
Use role-based access control to decide which teams, users, and agents can use Webscrapingai and which of its tools they can call.
Can I self-host the Webscrapingai MCP server?
Yes. Metorial is open source, so you can run the Webscrapingai MCP server yourself or use the hosted gateway.
Connect Webscrapingai to production AI agents
See how Metorial gives your team governed access to Webscrapingai, with tracing and security controls built in.