> ## Documentation Index > Fetch the complete documentation index at: https://docs.ag-ui.com/llms.txt > Use this file to discover all available pages before exploring further. # Types > Documentation for the core types used in the Agent User Interaction Protocol (Ruby SDK) # Core Types The Agent User Interaction Protocol Ruby SDK is built on a set of core types that represent the fundamental structures used throughout the system. This page documents these types and their properties. ## AudioInputContent `AgUiProtocol::Core::Types::AudioInputContent` Represents an audio content fragment inside a multimodal user message. ```ruby theme={null} content = AgUiProtocol::Core::Types::AudioInputContent.new( source: { type: "url", value: "https://example.com/audio.mp3", mime_type: "audio/mp3" } ) ``` | Property | Type | Description | | - | - | - | | `source` | `InputContentDataSource, InputContentUrlSource, Hash` | Content source | | `metadata` | `Object` (optional) | Optional metadata. Default: `nil`. | | `type` | `String` (optional) | Identifies the fragment type. Default: `"audio"`. | ## Context `AgUiProtocol::Core::Types::Context` Represents a piece of contextual information provided to an agent. ```ruby theme={null} ctx = AgUiProtocol::Core::Types::Context.new(description: "User locale", value: "es-CL") ``` | Property | Type | Description | | - | - | - | | `description` | `String` | Description of what this context represents. | | `value` | `String` | The actual context value. | ## DocumentInputContent `AgUiProtocol::Core::Types::DocumentInputContent` Represents a document content fragment inside a multimodal user message. ```ruby theme={null} content = AgUiProtocol::Core::Types::DocumentInputContent.new( source: { type: "url", value: "https://example.com/doc.pdf", mime_type: "application/pdf" } ) ``` | Property | Type | Description | | - | - | - | | `source` | `InputContentDataSource, InputContentUrlSource, Hash` | Content source | | `metadata` | `Object` (optional) | Optional metadata. Default: `nil`. | | `type` | `String` (optional) | Identifies the fragment type. Default: `"document"`. | ## ImageInputContent `AgUiProtocol::Core::Types::ImageInputContent` Represents an image content fragment inside a multimodal user message. ```ruby theme={null} content = AgUiProtocol::Core::Types::ImageInputContent.new( source: { type: "url", value: "https://example.com/cat.png", mime_type: "image/png" } ) ``` | Property | Type | Description | | - | - | - | | `source` | `InputContentDataSource, InputContentUrlSource, Hash` | Content source | | `metadata` | `Object` (optional) | Optional metadata. Default: `nil`. | | `type` | `String` (optional) | Identifies the fragment type. Default: `"image"`. | ## InputContentDataSource `AgUiProtocol::Core::Types::InputContentDataSource` Represents a data source for multimodal input content using base64-encoded data. ```ruby theme={null} source = AgUiProtocol::Core::Types::InputContentDataSource.new( value: "base64encoded...", mime_type: "image/png" ) ``` | Property | Type | Description | | - | - | - | | `value` | `String` | Base64 encoded content | | `mime_type` | `String` | MIME type of the content | | `type` | `String` (optional) | Identifies the source type. Default: `"data"`. | ## InputContentUrlSource `AgUiProtocol::Core::Types::InputContentUrlSource` Represents a URL source for multimodal input content. ```ruby theme={null} source = AgUiProtocol::Core::Types::InputContentUrlSource.new( value: "https://example.com/image.png", mime_type: "image/png" ) ``` | Property | Type | Description | | - | - | - | | `value` | `String` | URL string | | `mime_type` | `String` (optional) | Optional MIME type. Default: `nil`. | | `type` | `String` (optional) | Identifies the source type. Default: `"url"`. | ## Interrupt `AgUiProtocol::Core::Types::Interrupt` Represents an interrupt that occurred during agent execution for human-in-the-loop workflows. ```ruby theme={null} interrupt = AgUiProtocol::Core::Types::Interrupt.new( id: "int_1", reason: "input_required", message: "Please provide additional information" ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier | | `reason` | `String` | Reason for the interrupt | | `message` | `String` (optional) | Human-readable message. Default: `nil`. | | `tool_call_id` | `String` (optional) | Associated tool call if applicable. Default: `nil`. | | `response_schema` | `Object` (optional) | JSON schema for response. Default: `nil`. | | `expires_at` | `String` (optional) | ISO timestamp when interrupt expires. Default: `nil`. | | `metadata` | `Object` (optional) | Arbitrary metadata. Default: `nil`. | ## ReasoningMessage `AgUiProtocol::Core::Types::ReasoningMessage` Represents a reasoning message from an agent with chain-of-thought content. ```ruby theme={null} msg = AgUiProtocol::Core::Types::ReasoningMessage.new( id: "reason_1", content: "Let me think through this step by step...", encrypted_value: nil ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the message | | `content` | `String` | Reasoning content (plaintext when not encrypted) | | `encrypted_value` | `String` (optional) | Encrypted reasoning content when in zero-data-retention mode. Default: `nil`. | ## ResumeEntry `AgUiProtocol::Core::Types::ResumeEntry` Represents an entry for resuming an interrupted run. ```ruby theme={null} entry = AgUiProtocol::Core::Types::ResumeEntry.new( interrupt_id: "int_1", status: "resolved", payload: { "answer" => "42" } ) ``` | Property | Type | Description | | - | - | - | | `interrupt_id` | `String` | ID of the interrupt being resolved | | `status` | `String` (optional) | Resolution status (free-form; protocol values are "resolved" or "cancelled"). Default: `nil`. | | `payload` | `Object` (optional) | Response payload for the interrupt. Default: `nil`. | ## Role `AgUiProtocol::Core::Types::Role` Represents the possible roles a message sender can have. ```ruby theme={null} AgUiProtocol::Core::Types::Role # => ["developer", "system", "assistant", "user", "tool", "activity", "reasoning"] ``` ## RunAgentInput `AgUiProtocol::Core::Types::RunAgentInput` Input parameters for running an agent. In the HTTP API, this is the body of the `POST` request. `tools` contains tools provided by the client for this run. Backend-defined tools should remain in the backend agent or framework configuration, and may be advertised separately through agent capabilities. ```ruby theme={null} input = AgUiProtocol::Core::Types::RunAgentInput.new( thread_id: "thread_123", run_id: "run_123", parent_run_id: nil, state: {}, messages: [], tools: [], context: [], forwarded_props: {} ) ``` | Property | Type | Description | | - | - | - | | `thread_id` | `String` | ID of the conversation thread | | `run_id` | `String` | ID of the current run | | `state` | `Object` | Current state of the agent | | `messages` | `Array` | List of messages in the conversation | | `tools` | `Array` | Client-provided tools available for this run | | `context` | `Array` | List of context objects provided to the agent | | `forwarded_props` | `Object` | Additional properties forwarded to the agent | | `parent_run_id` | `String` (optional) | Lineage pointer for branching/time travel. Default: `nil`. | | `resume` | `Array` (optional) | Entries for resuming interrupted runs. Default: `nil`. | ## Tool `AgUiProtocol::Core::Types::Tool` Defines a tool that can be called by an agent. ```ruby theme={null} tool = AgUiProtocol::Core::Types::Tool.new( name: "search", description: "Search the web", parameters: { "type" => "object", "properties" => { "q" => { "type" => "string" } } } ) ``` | Property | Type | Description | | - | - | - | | `name` | `String` | Name of the tool. | | `description` | `String` | Description of what the tool does. | | `parameters` | `Object` | JSON Schema for tool parameters. | ## ToolCall `AgUiProtocol::Core::Types::ToolCall` Tool calls are embedded within assistant messages. ```ruby theme={null} tool_call = AgUiProtocol::Core::Types::ToolCall.new( id: "tc_1", function: { name: "search", arguments: "{\"q\":\"AG-UI\"}" } ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the tool call | | `function` | `FunctionCall, Hash` | Function name and arguments | | `type` | `String` (optional) | Type of the tool call. Default: `'function'`. | | `encrypted_value` | `String` (optional) | Encrypted tool call value for zero-data-retention mode. Default: `nil`. | ### FunctionCall `AgUiProtocol::Core::Types::FunctionCall` Function invocation descriptor used inside tool calls. ```ruby theme={null} fn = AgUiProtocol::Core::Types::FunctionCall.new( name: "search", arguments: "{\"q\":\"AG-UI\"}" ) ``` | Property | Type | Description | | - | - | - | | `name` | `String` | Function name. | | `arguments` | `String` | JSON-encoded arguments. | ## VideoInputContent `AgUiProtocol::Core::Types::VideoInputContent` Represents a video content fragment inside a multimodal user message. ```ruby theme={null} content = AgUiProtocol::Core::Types::VideoInputContent.new( source: { type: "url", value: "https://example.com/video.mp4", mime_type: "video/mp4" } ) ``` | Property | Type | Description | | - | - | - | | `source` | `InputContentDataSource, InputContentUrlSource, Hash` | Content source | | `metadata` | `Object` (optional) | Optional metadata. Default: `nil`. | | `type` | `String` (optional) | Identifies the fragment type. Default: `"video"`. | ## Message Types The SDK includes several message types that represent different kinds of messages in the system. ### ActivityMessage `AgUiProtocol::Core::Types::ActivityMessage` Represents structured activity progress emitted between chat messages. ActivityMessage intentionally does NOT inherit from BaseMessage because its `content` is a structured Hash rather than text/multimodal, which is incompatible with BaseMessage#content's type. It still appears alongside other messages in the stream but is modeled as its own shape. ```ruby theme={null} msg = AgUiProtocol::Core::Types::ActivityMessage.new( id: "activity_1", activity_type: "progress", content: { "pct" => 10 } ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the activity message. | | `activity_type` | `String` | Activity discriminator used for renderer selection. | | `content` | `Hash` | Structured payload representing the activity state. | ### AssistantMessage `AgUiProtocol::Core::Types::AssistantMessage` Represents a message from an assistant. ```ruby theme={null} msg = AgUiProtocol::Core::Types::AssistantMessage.new( id: "asst_1", content: "Hello!", tool_calls: [ { id: "tc_1", function: { name: "search", arguments: "{\"q\":\"AG-UI\"}" } } ] ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the message | | `content` | `Object` (optional) | Text content of the message. Default: `nil`. | | `tool_calls` | `Array` (optional) | Tool calls made in this message; Hashes are normalized to ToolCall instances. Default: `nil`. | | `name` | `String` (optional) | Name of the sender. Default: `nil`. | | `encrypted_value` | `String` (optional) | Encrypted content for zero-data-retention mode. Default: `nil`. | ### BinaryInputContent `AgUiProtocol::Core::Types::BinaryInputContent` Represents binary data such as images, audio, or files. ```ruby theme={null} content = AgUiProtocol::Core::Types::BinaryInputContent.new( mime_type: "image/png", url: "https://example.com/cat.png" ) ``` > **Validation:** At least one of `id`, `url`, or `data` must be > provided. | Property | Type | Description | | - | - | - | | `type` | `String` (optional) | Identifies the fragment type. Default: `"binary"`. | | `mime_type` | `String` | MIME type, for example `"image/png"` | | `id` | `String` (optional) | Reference to previously uploaded content. Default: `nil`. | | `url` | `String` (optional) | Remote URL where the content can be retrieved. Default: `nil`. | | `data` | `String` (optional) | Base64 encoded content. Default: `nil`. | | `filename` | `String` (optional) | Optional filename hint. Default: `nil`. | ### DeveloperMessage `AgUiProtocol::Core::Types::DeveloperMessage` Represents a message from a developer. ```ruby theme={null} msg = AgUiProtocol::Core::Types::DeveloperMessage.new( id: "dev_1", content: "You are a helpful assistant." ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the message | | `content` | `Object` | Text content of the message (required) | | `name` | `String` (optional) | Optional name of the sender. Default: `nil`. | ### SystemMessage `AgUiProtocol::Core::Types::SystemMessage` Represents a system message. ```ruby theme={null} msg = AgUiProtocol::Core::Types::SystemMessage.new( id: "sys_1", content: "Follow the protocol." ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the message | | `content` | `Object` | Text content of the message (required) | | `name` | `String` (optional) | Optional name of the sender. Default: `nil`. | ### TextInputContent `AgUiProtocol::Core::Types::TextInputContent` Represents a text fragment inside a multimodal user message. ```ruby theme={null} content = AgUiProtocol::Core::Types::TextInputContent.new(text: "hello") ``` | Property | Type | Description | | - | - | - | | `text` | `String` | Text content | | `type` | `String` (optional) | Identifies the fragment type. Default: `"text"`. | ### ToolMessage `AgUiProtocol::Core::Types::ToolMessage` Tool result message. ```ruby theme={null} msg = AgUiProtocol::Core::Types::ToolMessage.new( id: "tool_msg_1", tool_call_id: "tc_1", content: "ok" ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the message. | | `content` | `String` | Tool result content. | | `tool_call_id` | `String` | ID of the tool call this message responds to. | | `error` | `String` (optional) | Error payload if the tool call failed. Default: `nil`. | | `encrypted_value` | `String` (optional) | Encrypted tool message value for zero-data-retention mode. Default: `nil`. | ### UserMessage `AgUiProtocol::Core::Types::UserMessage` Represents a message from a user. ```ruby theme={null} msg = AgUiProtocol::Core::Types::UserMessage.new( id: "user_2", content: [ { type: "text", text: "Please describe this image" }, { type: "binary", mimeType: "image/png", url: "https://example.com/cat.png" } ] ) ``` | Property | Type | Description | | - | - | - | | `id` | `String` | Unique identifier for the message | | `content` | `String, Array` | Accepted shapes: a String for plain text, or an Array whose elements are either \*InputContent Models (TextInputContent, BinaryInputContent, ImageInputContent, AudioInputContent, VideoInputContent, DocumentInputContent) OR Hashes with a `type:` key (`text`, `binary`, `image`, `audio`, `video`, `document`); Hash entries are normalized internally to the corresponding Model. | | `name` | `String` (optional) | Optional name of the sender. Default: `nil`. | | `encrypted_value` | `String` (optional) | Encrypted content for zero-data-retention mode. Default: `nil`. | ## State `State` is represented as `Any`. The state type is flexible and can hold any data structure needed by the agent implementation. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.