delete_configuration
Delete Prompt Configuration
Delete a prompt configuration by its ID.
Ask your agent in plain language. Each of these maps to a real tool on the server.
delete_configuration
Delete a prompt configuration by its ID.
list_runs
List evaluation/experiment runs in a project. Supports filtering by dataset, name, status, and pagination. Use this to find and compare past experiment results.
delete_dataset
Delete a dataset by its ID.
post_feedback
Post user feedback or quality metrics against a specific event or session. Feedback data is used for monitoring, building evaluation datasets, and improving AI outputs over time.
update_event
Update an existing event's metadata, feedback, metrics, outputs, or other fields. Use this to enrich trace data after the fact, such as attaching user feedback or quality metrics.
get_run
Retrieve full details and results of an experiment run by its ID. Includes event IDs, dataset association, configuration, and metric results.
start_session
Start a new tracing session in HoneyHive. A session represents a complete interaction or request and serves as the root of a trace tree. Child events (model, tool, chain) can be attached to it.
compare_runs
Compare two experiment runs side by side. Shows metric differences, common datapoints, and per-event comparison details. Useful for detecting regressions or improvements between runs.
query_events
Search and filter trace events using structured filters. Supports filtering by event type, metadata fields, date ranges, and more. Use for monitoring, debugging, and data exploration.
delete_run
Delete an experiment run by its ID.
update_dataset
Update a dataset name, description, or datapoint membership.
delete_project
DEPRECATED — unavailable with project-scoped data-plane API keys. Manage projects in HoneyHive administration using separate control-plane credentials. This tool reports the limitation without making an API request.
create_metric
Create a new evaluation metric for a project. Metrics can be code-based (custom), LLM-as-a-judge (model), or human annotation (human). They define quality criteria for scoring AI outputs.
delete_metric
Delete an evaluation metric by its ID.
list_datasets
List datasets in a project, optionally filtering by dataset ID. Dataset type filtering is unavailable on the current API. Datasets contain curated input/output pairs used for evaluations and experiments.
list_metrics
List all evaluation metrics defined for a project. Metrics are used to score and evaluate AI application outputs during experiments and production monitoring.
create_configuration
Create a new prompt configuration (managed prompt) in a project. Configurations centralize prompt management with versioning and environment-based deployment.
log_event
Log a single trace event (model, tool, or chain) in HoneyHive. Events are nested within sessions to build a distributed trace. Use this to record individual LLM calls, tool invocations, or chain steps.
log_event_batch
Log multiple trace events in a single batch request. Efficient for high-volume ingestion. All events can optionally share a single session.
get_session
Retrieve a session by ID, including its trace fields and child event IDs.
list_projects
DEPRECATED — unavailable with project-scoped data-plane API keys. Manage projects in HoneyHive administration using separate control-plane credentials. This tool reports the limitation without making an API request.
get_event
Retrieve a single event by its ID, including its trace fields and child event IDs. The connection API key selects the project.
create_dataset
Create a new dataset in a project. Datasets hold curated datapoints for running evaluations and experiments.
add_datapoints_to_dataset
Add raw data records to a dataset with a field mapping. The mapping defines which fields from the raw data should be used as inputs, ground truths, and history.
update_configuration
Update an existing prompt configuration's parameters, environments, or other settings.
update_metric
Update an existing metric's definition, criteria, or settings.
delete_event
DEPRECATED — unavailable: HoneyHive does not document an API for deleting individual events. Retained for existing workflows; invoking this tool reports the provider limitation.
get_run_result
Retrieve detailed evaluation results for an experiment run, including pass/fail outcomes, metric aggregations, and per-datapoint details.
create_project
DEPRECATED — unavailable with project-scoped data-plane API keys. Manage projects in HoneyHive administration using separate control-plane credentials. This tool reports the limitation without making an API request.
create_run
Create a new evaluation/experiment run. Runs associate traced events with a dataset and configuration for evaluation and comparison.
list_configurations
List prompt configurations (managed prompts) in a project. Configurations can be filtered by environment and name. Use this to discover available prompts and their deployment status.
update_project
DEPRECATED — unavailable with project-scoped data-plane API keys. Manage projects in HoneyHive administration using separate control-plane credentials. This tool reports the limitation without making an API request.
delete_session
DEPRECATED — unavailable: HoneyHive does not document an API for deleting individual sessions. Retained for existing workflows; invoking this tool reports the provider limitation.
update_run
Update an experiment run name, status, configuration, metadata, or associated events and datapoints.
update_datapoint
Update evaluation datapoint inputs, expected outputs, history, metadata, or dataset associations.
list_datapoints
List evaluation datapoints in a project, optionally filtering by dataset name or datapoint IDs. The connection API key selects the project.
delete_datapoint
Delete an evaluation datapoint by its ID.
get_datapoint
Retrieve an evaluation datapoint by its ID. Call list_datapoints to discover IDs.
create_datapoint
Create an evaluation datapoint with inputs, expected outputs, conversation history, and optional dataset associations. The connection API key selects the project.
Every client uses the same Magic MCP URL, and so does any other MCP client. Add it once and sign in with SSO.
{"mcpServers": {"metorial": {"url": "<your-magic-mcp-url>"}}}
claude mcp add --transport http metorial <your-magic-mcp-url>
codex mcp add metorial --url <your-magic-mcp-url>codex mcp login metorial
One governed layer for every app your agents reach, not a separate connector per tool.
Trace, monitor, and evaluate LLM applications and AI agents. Create and manage projects, log distributed traces of AI application execution including model calls, tool invocations, and chain events. Run offline experiments against curated datasets with custom evaluators. Create, version, and manage datasets and prompts. Post user feedback and custom metrics against sessions or events. Query and analyze production traces with aggregated cost, latency, token usage, and quality metrics. Manage annotation queues for human feedback collection. Configure prompt deployments across environments.
Common questions about connecting Honeyhive to AI agents with Metorial.