Skip to main content

LLM output

This module defines data models that are standardized across different LLM providers.

FinishReasonType module-attribute

Possible reasons for an LLM completion to finish.

LLMCompletion

Bases: BaseModel A single completion from an LLM. Attributes:

no_text property

Check if the completion has no text. Returns:

LLMOutput dataclass

Container for LLM output, potentially with multiple completions. Aggregates completions from an LLM along with metadata and error information. Attributes:

non_empty property

Check if there are any completions. Returns:

first property

Get the first completion if available. Returns:

first_text property

Get the text of the first completion if available. Returns:

did_error property

Check if any errors occurred during generation. Returns:

ToolCallPartial dataclass

Partial representation of a tool call before full processing. Used as an intermediate format before finalizing into a complete ToolCall. Parameters:

LLMCompletionPartial

Bases: LLMCompletion Partial representation of an LLM completion before finalization. Extends LLMCompletion but with tool_calls being a list of ToolCallPartial. This is used during the processing stage before tool calls are fully parsed. Attributes:

no_text property

Check if the completion has no text. Returns:

LLMOutputPartial dataclass

Bases: LLMOutput Partial representation of LLM output before finalization. Extends LLMOutput but with completions being a list of LLMCompletionPartial. Used as an intermediate format during processing. Attributes:

non_empty property

Check if there are any completions. Returns:

first property

Get the first completion if available. Returns:

first_text property

Get the text of the first completion if available. Returns:

did_error property

Check if any errors occurred during generation. Returns:

AsyncLLMOutputStreamingCallback

Bases: Protocol Protocol for asynchronous streaming callbacks with batch index. Defines the expected signature for callbacks that handle streaming output with a batch index. Parameters:

AsyncSingleLLMOutputStreamingCallback

Bases: Protocol Protocol for asynchronous streaming callbacks without batch indexing. Defines the expected signature for callbacks that handle streaming output without batch indexing. Parameters:

AsyncEmbeddingStreamingCallback

Bases: Protocol Protocol for sending progress updates for embedding generation.

finalize_llm_output_partial

Convert a partial LLM output into a finalized LLM output. Processes tool calls by parsing their arguments from raw JSON strings, handles errors in JSON parsing, and provides warnings for truncated completions. Parameters: Returns: Raises: