Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.transluce.org/llms.txt

Use this file to discover all available pages before exploring further.

Inspect (web, GitHub) is an open-source framework for large language model evaluations. We offer helpers that import Inspect .eval logs into Docent.

​
Use this when

Use Inspect ingestion when your source data is already in Inspect .eval format and you want either:
  • a pure conversion step that gives you AgentRun objects back
  • a recursive conversion-and-upload workflow for a directory of .eval files

​
Main helpers

  • convert_inspect_eval_file_to_agent_runs(file_path) converts one Inspect .eval archive into a list of AgentRun objects.
  • convert_inspect_directory_to_agent_runs(root) recursively finds .eval files and returns all converted runs.
  • ingest_inspect_directory(collection_id, fpath, *, upload_agent_run_batch, batch_size=100) batches conversion and upload if you want a lower-level ingestion loop.
  • Docent.recursively_ingest_inspect_logs(collection_id, fpath) is the highest-level client wrapper for recursive ingestion.
from docent.sdk.integrations import (
    convert_inspect_directory_to_agent_runs,
    convert_inspect_eval_file_to_agent_runs,
)

​
Example

To convert a single .eval file: 
from docent.sdk.integrations import convert_inspect_eval_file_to_agent_runs

agent_runs = convert_inspect_eval_file_to_agent_runs("evals/my-run.eval")
To convert every .eval file under a directory: 
from docent.sdk.integrations import convert_inspect_directory_to_agent_runs

agent_runs = convert_inspect_directory_to_agent_runs("/path/to/inspect/logs")
To recursively convert and upload in one step: 
from docent import Docent

client = Docent()
client.recursively_ingest_inspect_logs(collection_id, "/path/to/inspect/logs")
After conversion, upload normally: 
from docent import Docent
from docent.sdk.integrations import convert_inspect_directory_to_agent_runs

client = Docent()
agent_runs = convert_inspect_directory_to_agent_runs("/path/to/inspect/logs")
client.add_agent_runs(collection_id, agent_runs)

​
More on the conversion process

Each Inspect sample becomes one Docent AgentRun.

​
How Inspect data is mapped

The converter:
  • reads archive header metadata such as task and model
  • converts each sample’s messages with parse_chat_message(...)
  • normalizes sample scores into agent_run.metadata["scores"]
  • preserves raw score payloads in agent_run.metadata["scoring_metadata"]
  • includes sample-level fields such as sample_id, epoch, and target in run metadata
  • merges sample metadata on top of header metadata when both are present
Because Inspect messages already fit Docent’s chat schema closely, this conversion is usually low-friction.

​
Recursive ingestion helper

If you want to control uploads yourself, use ingest_inspect_directory(...). It:
  • recursively finds .eval files
  • converts them lazily
  • uploads them in batches through your callback
That lower-level helper is what Docent.recursively_ingest_inspect_logs(...) uses internally.