Connect Honeyhive with Hookshot™ and your stack

HoneyHive is a modern AI observability and evaluation platform that enables developers and domain experts to collaboratively build reliable AI applications faster.

Browse integrations Visit website

Tools and triggers

What Agents can do in Honeyhive

Available tools and events after Honeyhive is connected.

  • Add Datapoints to Dataset

    Tool to add datapoints to a dataset. Use when you need to append multiple entries with specified input, ground truth, and history mappings.

  • Batch Create Datapoints

    Tool to create multiple datapoints in a single batch operation. Use when you need to bulk-import events into a dataset or create many datapoints at once. Supports filtering by date range, event IDs, or custom criteria…

  • Compare Experiment Runs

    Tool to retrieve experiment comparison between two evaluation runs. Use when you need to analyze the differences in metrics, datapoints, and events between two runs.

  • Compare Runs Events

    Tool to compare events between two experiment runs side-by-side. Use when analyzing differences in model behavior, performance metrics, or outputs between evaluation runs. Returns matched event pairs with their…

  • Create Batch Model Events

    Tool to create multiple model events in a single request. Use when you need to log a batch of event interactions to HoneyHive.

  • Create Batch Tool Events

    Tool to log a batch of external API calls as tool events. Use when you need to record multiple tool events in one request—use after gathering all event data.

  • Create Configuration

    Creates a new configuration in HoneyHive for managing LLM or pipeline settings. Use this to define reusable configurations with specific models, prompts, and parameters that can be deployed across different…

  • Create Datapoint

    Tool to create a new datapoint with input-output pairs. Use when you need to add a single datapoint with inputs, ground truth, conversation history, and metadata.

  • Create Dataset

    Tool to create a dataset. Use when you need to initialize a new dataset within a project.

  • Create Event

    Tool to create a new event in HoneyHive to track execution of different parts of your application. Use when you need to log a model call, tool execution, or chain step. Events can be grouped into sessions and nested…

  • Create Metric

    Tool to create a new metric in HoneyHive. Use when you need to define how to evaluate model outputs, whether through code (PYTHON), AI evaluation (LLM), human review (HUMAN), or combining multiple metrics (COMPOSITE)…

  • Create Model Event

    Tool to create a new model event to log LLM call data. Use when you need to track a single model interaction including messages, responses, usage, and metadata.

  • Create Tool

    Creates a new tool definition in a HoneyHive project. Use this to register functions or plugins that can be invoked and tracked within HoneyHive. Tools are defined with a JSON Schema for their parameters, allowing…

  • Delete Datapoint

    Tool to delete a specific datapoint by its ID. Use when you need to remove a datapoint from HoneyHive after confirming its identifier.

  • Delete Dataset

    Tool to delete a dataset by ID. Use when you need to remove a dataset after confirming its ID.

  • End Evaluation Run

    Tool to update an evaluation run's status and metadata. Use to mark a run as completed after finishing evaluations, or update run properties like name, metadata, configuration, and associated event/datapoint IDs.

  • Get Configurations

    Tool to retrieve a list of configurations. Use when you need to fetch all configurations for a specific project before making changes.

  • Get Datasets

    Retrieve datasets from HoneyHive for a specified project. Use this tool when you need to: - List all datasets within a project - Find datasets by type (evaluation or fine-tuning) - Retrieve a specific dataset by its ID…

  • Get Evaluation Run Details

    Tool to get details of an evaluation run by its UUID. Use when you need to check the status, configuration, results, or metadata of a specific evaluation run.

  • Get Evaluation Runs

    Tool to retrieve a list of evaluation runs from HoneyHive. Use when you need to: - List all evaluation runs for analysis - Find runs by status, name, or dataset - Get specific runs by their IDs - Paginate through large…

  • Get Events

    Tool to query events with filters and projections from HoneyHive. Use this action when you need to retrieve events with lightweight filtering (limit 1000 results). For bulk exports or more complex queries, use the…

  • Get Events by Session ID

    Tool to retrieve the complete tree of nested events for a specific session. Use when you need to analyze all events (model calls, tool calls, chains) that occurred within a session, including their hierarchical…

  • Get Events Chart

    Tool to retrieve charting and analytics data for events over time. Use when you need aggregated metrics (duration, cost, token usage) grouped by time buckets or fields. Supports percentile analysis (p50, p95, p99) for…

  • Get Metrics

    Retrieves all metrics associated with a HoneyHive project. Returns a list of metrics including their configuration (name, type, description, thresholds, evaluator details) and metadata (creation/update timestamps…

Setup

Connect Honeyhive in Hookshot™

Pick a scope, then confirm what can start a Protege and what it can do.

Trigger Access

What can start a Protege from this app.

Tool Access

What a Protege can do after it starts.

Connection scope

Prefer team for production.

Team

Shared credential for production and unattended runs.

Personal

Tied to one user—only when the workflow needs their account.

Chat

Team-scoped for supported chat surfaces.

Full setup guide

Build with us

See Hookshot™ in your stack.

Request early access or book a live walkthrough with the team.

Request Demo
Hookshot™ setup screen — AI agent workflow configuration with model selection, trigger status, and governance controls.