markdownify
Converts a webpage into clean, well-formatted Markdown. Strips ads, navigation, and irrelevant elements while preserving content structure, images, and links. Useful for preparing web content for LLMs, documentation, or content migration.
get_credits
Retrieves the current API credit balance and total credits used for the account.
crawl_website
Crawls multiple pages of a website starting from a given URL, following links with configurable depth and breadth. Two modes: **AI extraction** (structured output with a prompt) and **markdown conversion** (no AI, lower cost). Supports crawl rules for include/exclude paths, sitemap-based discovery, and webhook notifications on completion.
get_request_status
Retrieves the status and results of a previous scraping request by its request ID. Supports all service types: smartscraper, searchscraper, markdownify, scrape, crawl, sitemap, and agentic-scrapper.
discover_sitemap
Extracts and returns the sitemap structure of a website by discovering URLs from sitemap.xml, robots.txt, and common sitemap locations. Useful for understanding site organization, URL discovery, and SEO audits before performing a larger crawl.
web_search
Performs AI-powered web searches using a natural language prompt and aggregates structured results from multiple web sources with source attribution. Two modes: **AI Extraction** (structured data, higher cost) and **Markdown Mode** (raw content, lower cost). Supports geo-targeted search and time range filtering.
raw_scrape
Fetches and returns the raw HTML of a webpage with JavaScript rendering support. Useful when you need the complete page source rather than AI-processed output. Optionally extracts branding information (colors, fonts, typography, UI styles, metadata).
agentic_scrape
Automates browser interactions using a sequence of natural language steps before extracting data. Can click buttons, fill forms, type text, scroll, navigate, and log in to websites. Supports session persistence for multi-step workflows and optional AI extraction with schema support. Without AI extraction, returns raw markdown.
smart_scrape
Extracts structured data from a single webpage using AI and a natural language prompt. Provide a URL (or raw HTML/Markdown content) and describe what data you want extracted; returns structured JSON. Supports custom output schemas for consistent data structure, infinite scroll handling, stealth mode for anti-detection, and proxy routing by country.
Manage repositories, issues, and pull requests. Create and configure branches, star repositories, review code, and merge changes. Automate CI/CD workflows with GitHub Actions, manage workflow runs, secrets, and artifacts. Track issues with labels, milestones, and assignees. Search across code, repositories, issues, and users. Manage organizations, teams, and memberships. Create and manage projects, gists, packages, deployments, and environments. Access security alerts including code scanning, secret scanning, and Dependabot alerts. Read and write file contents in repositories. Manage webhooks, notifications, and codespaces.
Manage SharePoint sites, document libraries, lists, and files. Create, read, update, and delete lists and list items with custom columns. Upload, download, move, copy, and version files in document libraries. Search across sites, files, folders, lists, and list items using Microsoft Search. Manage permissions at site, list, and item levels with granular access control. Define and manage content types and site columns. Subscribe to webhooks for list and library change notifications. Retrieve site properties and search for sites across Microsoft 365.
Manage CRM data including Accounts, Contacts, Leads, Opportunities, Cases, and custom objects. Create, read, update, and delete records. Query data using SOQL and search across objects using SOSL. Perform bulk data operations for large-scale imports, exports, and migrations. Execute composite requests to batch multiple operations in a single API call. Access analytics, reports, and dashboards. Manage files and attachments associated with records. Interact with Chatter feeds, posts, and groups for social collaboration. Subscribe to real-time change events via Change Data Capture and Platform Events. Manage org metadata including custom objects, fields, layouts, and workflows. Query data using GraphQL for precise data retrieval across related objects.
Create, read, update, and delete records in Airtable bases and tables. Manage base schemas including creating tables and fields. Filter records using formulas, sort by fields, and scope queries to specific views. Upsert records to find, create, or update in a single call. Upload attachments to records, read and write record comments, list accessible bases, and receive real-time base change events through webhooks.
Manage Git repositories, pull requests, and CI/CD pipelines on Bitbucket Cloud. Create, fork, and configure repositories within workspaces and projects. Create, review, approve, merge, and decline pull requests with inline code comments. Browse source code, list commits, and manage branches and tags. Track issues with the built-in issue tracker. Trigger, monitor, and manage Bitbucket Pipelines. List workspace members, configure repository default reviewers and branch restrictions, create and manage repository webhooks, and search code across repositories.
Deploy, manage, and scale applications on Heroku's cloud platform. Create and configure apps, scale dynos, provision add-ons (databases, caching, etc.), manage configuration variables, build and release code, add custom domains and SSL certificates, manage collaborators and team permissions, configure pipelines for continuous delivery, set up log drains, and sync data with Salesforce via Heroku Connect. Subscribe to webhooks for real-time notifications on app changes, builds, releases, dyno lifecycle events, and more.
Extract structured data from webpages using natural language prompts and AI. Scrape single pages or crawl entire websites with configurable depth and rules. Perform AI-powered web searches across multiple sources with structured results. Convert webpages to clean Markdown. Fetch raw HTML with JavaScript rendering. Automate browser interactions (clicking, typing, form filling, logging in) before extracting data. Discover website sitemaps. Supports custom output schemas, stealth mode, proxy routing, geo-targeting, and webhook notifications for crawl job completion. Check API credit balance and retrieve past request history.
Common questions about connecting Scrapegraph Ai to AI agents with Metorial.
