Manage Collections

A collection is a container for agent runs, rubrics, and evaluation results. See Introduction for more on how collections fit into the Docent workflow.

Create a Collection

from docent import Docent

client = Docent()

collection_id = client.create_collection(
    name="GPT-4 Customer Support Runs",
    description="Production runs from the support agent",
    metadata={"team": "support", "model": "gpt-4"},
)
print(collection_id)  # e.g., "a1b2c3d4-..."

Parameters

str | None

Optional ID for the new collection. If not provided, one is generated automatically.

str | None

Display name for the collection.

str | None

Description of the collection’s purpose.

dict | None

Arbitrary key-value metadata to attach to the collection.

Returns

str

The ID of the created collection.

List Collections

collections = client.list_collections()
for c in collections:
    print(f"{c['id']}: {c['name']} ({c['counts']['agent_run_count']} runs)")

Returns

list[dict]

List of collection summary objects, including ownership, your permission level, and precomputed counts.

Show Collection object fields

str

Collection ID.

str | None

Display name.

str | None

Description.

str

ISO timestamp of creation.

str

User ID of the creator.

dict

Owner of the collection.

Show Owner fields

str

Owner user ID.

str

Owner email.

bool

Whether the owner is an anonymous user.

bool

True if the requesting user owns the collection.

str

Requesting user’s permission level on the collection: "read", "write", or "admin".

dict

Precomputed counts for the collection.

Show Counts fields

int

Number of agent runs in the collection.

Get a Collection

import requests

try:
    collection = client.get_collection("my-collection-id")
    print(collection["name"])
except requests.exceptions.HTTPError as e:
    if e.response.status_code == 404:
        print("Collection not found")
    else:
        raise

Parameters

str

required

ID of the collection to retrieve.

Returns

dict

Collection details with ownership, your permission level, and precomputed counts.

Show Collection object fields

str

Collection ID.

str | None

Display name.

str | None

Description.

str

ISO timestamp of creation.

str

User ID of the creator.

dict

Owner of the collection.

Show Owner fields

str

Owner user ID.

str

Owner email.

bool

Whether the owner is an anonymous user.

bool

True if the requesting user owns the collection.

str

Requesting user’s permission level on the collection: "read", "write", or "admin".

dict

Precomputed counts for the collection.

Show Counts fields

int

Number of agent runs in the collection.

The metadata blob is no longer included on this response. Use get_collection_metadata to retrieve it.

Errors

HTTPError (404) — Collection not found

Update a Collection

client.update_collection(
    "my-collection-id",
    name="Renamed Collection",
    description="Updated description",
)

Parameters

str

required

ID of the collection to update.

str | None

New name. If None, the name is left unchanged.

str | None

New description. If None, the description is left unchanged.

Delete Agent Runs

Remove specific agent runs from a collection.

deleted = client.delete_agent_runs("my-collection-id", ["run-1", "run-2"])
print(f"Deleted {deleted} runs")

Parameters

str

required

ID of the collection.

list[str]

required

List of agent run IDs to delete.

Returns

int

Number of agent runs deleted.

Errors

ValueError — agent_run_ids is empty
HTTPError (404) — Collection not found

Getting Started

Collections

Agent Runs

DQL

Feedback & Labels

More

Legacy

Manage Collections

Create a Collection

Parameters

Returns

List Collections

Returns

Get a Collection

Parameters

Returns

Errors

Update a Collection

Parameters

Delete Agent Runs

Parameters

Returns

Errors

​Create a Collection

​Parameters

​Returns

​List Collections

​Returns

​Get a Collection

​Parameters

​Returns

​Errors

​Update a Collection

​Parameters

​Delete Agent Runs

​Parameters

​Returns

​Errors

Create a Collection

Parameters

Returns

List Collections

Returns

Get a Collection

Parameters

Returns

Errors

Update a Collection

Parameters

Delete Agent Runs

Parameters

Returns

Errors