Eino Guide

Eino Framework Guide

Eino (pronounced "i know") is a Go framework for building LLM applications.

Core Concepts

Component

Standardized interfaces for AI capabilities. Each interface has multiple interchangeable implementations.

Implementations live in eino-ext (OpenAI, Claude, Gemini, Ark, Ollama, Milvus, Redis, Elasticsearch, etc.).

-> Use /eino-component for selecting, configuring, and using components.

Orchestration (Compose)

Three APIs for wiring components into executable pipelines. All compile to a Runnable[I, O] with four execution modes (Invoke, Stream, Collect, Transform).

The compose layer handles type checking, stream conversion between nodes, concurrency, callback injection, and option distribution automatically.

-> Use /eino-compose for building graphs, chains, workflows, streaming, callbacks, and state management.

ADK (Agent Development Kit)

High-level abstractions for building AI agents. Encapsulates the model-tool-loop pattern.

-> Use /eino-agent for building agents, configuring middleware, runners, and human-in-the-loop.

Schema

Shared data types used across all layers:

• schema.Message -- Conversation message (system/user/assistant/tool roles, content, tool calls)
• schema.Document -- Document with content, metadata, and vector embeddings
• schema.ToolInfo -- Tool description with JSON schema parameters
• schema.StreamReader[T] -- Generic streaming reader (always defer stream.Close())

Repositories

Packages at a Glance

eino (core):

eino-ext (implementations):

Choosing Your Approach

Reference Files

• reference/schema.md -- Core data types shared across all layers: Message, Document, ToolInfo, StreamReader
• reference/runnable.md -- Runnable[I, O] interface, four execution modes, runtime options
• reference/quick-start.md -- Three complete working examples (ChatModel, Agent+Runner, Chain)

Instructions to Agent

1. Route to the appropriate skill (/eino-component, /eino-compose, /eino-agent) when the question is specific. Consult the "Choosing Your Approach" table. 2. Always provide Go code examples using real import paths from github.com/cloudwego/eino and github.com/cloudwego/eino-ext. 3. For component implementation details, always read the provider's reference file before generating code. Do not assume constructor or config naming conventions. 4. Prefer ADK (ChatModelAgent + Runner) for agent use cases over manually building ReAct loops with compose graphs. For interactive multi-turn applications needing preemption and lifecycle management, recommend TurnLoop. 5. When showing streaming code, always include defer stream.Close().

Three complete working examples: ChatModel Generate, ChatModelAgent with Runner, and a simple Graph chain.

Prerequisites

• Go 1.18+
• An API key for OpenAI (or Ark, Ollama, etc.)

go get github.com/cloudwego/eino@latest
go get github.com/cloudwego/eino-ext/components/model/openai@latest

Example 1: ChatModel Generate

Create a ChatModel, send messages, get a response. This is the simplest way to use Eino.

package main

import (
    "context"
    "errors"
    "fmt"
    "io"
    "log"

    "github.com/cloudwego/eino-ext/components/model/openai"
    "github.com/cloudwego/eino/schema"
)

func main() {
    ctx := context.Background()

    // Create an OpenAI ChatModel
    model, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
        Model:  "gpt-4o",
        APIKey: "your-api-key",
    })
    if err != nil {
        log.Fatal(err)
    }

    // Construct input messages
    messages := []*schema.Message{
        schema.SystemMessage("You are a helpful assistant."),
        schema.UserMessage("Explain Go interfaces in one sentence."),
    }

    // Option A: Non-streaming (Generate)
    resp, err := model.Generate(ctx, messages)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("Generate:", resp.Content)

    // Option B: Streaming (Stream)
    stream, err := model.Stream(ctx, messages)
    if err != nil {
        log.Fatal(err)
    }
    defer stream.Close()

    fmt.Print("Stream: ")
    for {
        chunk, err := stream.Recv()
        if errors.Is(err, io.EOF) {
            break
        }
        if err != nil {
            log.Fatal(err)
        }
        fmt.Print(chunk.Content)
    }
    fmt.Println()
}

Key points:

• Generate() returns a complete *schema.Message.
• Stream() returns a *schema.StreamReader[*schema.Message] -- always defer stream.Close().
• Switch providers by changing the import and config (e.g., ark.NewChatModel, ollama.NewChatModel).

Example 2: ChatModelAgent with Runner (Multi-Turn)

Use ADK to build a multi-turn conversational agent with tool calling.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/cloudwego/eino/adk"
    "github.com/cloudwego/eino/components/tool"
    "github.com/cloudwego/eino/compose"
    "github.com/cloudwego/eino/schema"
    "github.com/cloudwego/eino-ext/components/model/openai"
    toolutils "github.com/cloudwego/eino/components/tool/utils"
)

// Define a simple tool
func getWeather(ctx context.Context, req *WeatherRequest) (string, error) {
    return fmt.Sprintf("Weather in %s: sunny, 22C", req.City), nil
}

type WeatherRequest struct {
    City string `json:"city" jsonschema_description:"The city to get weather for"`
}

func main() {
    ctx := context.Background()

    // Create ChatModel
    model, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
        Model:  "gpt-4o",
        APIKey: "your-api-key",
    })
    if err != nil {
        log.Fatal(err)
    }

    // Create tool from function
    weatherTool, err := toolutils.InferTool("get_weather", "Get current weather for a city", getWeather)
    if err != nil {
        log.Fatal(err)
    }

    // Create ChatModelAgent with tools
    agent, err := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
        Name:        "weather_assistant",
        Description: "An assistant that can check weather",
        Instruction: "You are a helpful weather assistant.",
        Model:       model,
        ToolsConfig: adk.ToolsConfig{
            ToolsNodeConfig: compose.ToolsNodeConfig{
                Tools: []tool.BaseTool{weatherTool},
            },
        },
    })
    if err != nil {
        log.Fatal(err)
    }

    // Create Runner
    runner := adk.NewRunner(ctx, adk.RunnerConfig{
        Agent:          agent,
        EnableStreaming: true,
    })

    // Multi-turn conversation
    history := make([]*schema.Message, 0)

    queries := []string{
        "What's the weather in Beijing?",
        "How about Tokyo?",
    }

    for _, query := range queries {
        fmt.Printf("\nUser: %s\n", query)
        history = append(history, schema.UserMessage(query))

        // Run agent
        events := runner.Run(ctx, history)
        var content string
        for {
            event, ok := events.Next()
            if !ok {
                break
            }
            if event.Err != nil {
                log.Fatal(event.Err)
            }
            if event.Output != nil && event.Output.MessageOutput != nil {
                if event.Output.MessageOutput.IsStreaming {
                    stream := event.Output.MessageOutput.MessageStream
                    // Note: In real applications, stream should be closed properly
                    for {
                        chunk, err := stream.Recv()
                        if err != nil {
                            break
                        }
                        content += chunk.Content
                        fmt.Print(chunk.Content)
                    }
                    stream.Close()
                } else if msg := event.Output.MessageOutput.Message; msg != nil {
                    content += msg.Content
                    fmt.Print(msg.Content)
                }
            }
        }
        fmt.Println()

        // Append assistant response to history
        history = append(history, schema.AssistantMessage(content, nil))
    }
}

Key points:

• tool/utils.InferTool creates a tool from a Go function with automatic JSON schema inference.
• Runner.Run() returns an AsyncIterator[*AgentEvent] for streaming consumption.
• The agent handles the ReAct loop internally: model -> tool call -> tool result -> model -> response.
• Multi-turn is achieved by maintaining a history slice across Run() calls.

Example 3: Simple Graph (Template -> Model -> Output)

Use the compose package to build a chain: ChatTemplate feeds into ChatModel.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/cloudwego/eino/compose"
    "github.com/cloudwego/eino/components/prompt"
    "github.com/cloudwego/eino/schema"
    "github.com/cloudwego/eino-ext/components/model/openai"
)

func main() {
    ctx := context.Background()

    // Create ChatModel
    model, err := openai.NewChatModel(ctx, &openai.ChatModelConfig{
        Model:  "gpt-4o",
        APIKey: "your-api-key",
    })
    if err != nil {
        log.Fatal(err)
    }

    // Create ChatTemplate
    tpl := prompt.FromMessages(schema.FString,
        schema.SystemMessage("You are a {role} expert."),
        schema.UserMessage("{query}"),
    )

    // Build a Chain: template -> model
    chain, err := compose.NewChain[map[string]any, *schema.Message]().
        AppendChatTemplate(tpl).
        AppendChatModel(model).
        Compile(ctx)
    if err != nil {
        log.Fatal(err)
    }

    // Invoke the chain
    result, err := chain.Invoke(ctx, map[string]any{
        "role":  "Go programming",
        "query": "What are the best practices for error handling in Go?",
    })
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(result.Content)

    // Or use Stream for streaming output
    stream, err := chain.Stream(ctx, map[string]any{
        "role":  "Go programming",
        "query": "Explain goroutines in 3 sentences.",
    })
    if err != nil {
        log.Fatal(err)
    }
    defer stream.Close()

    for {
        chunk, err := stream.Recv()
        if err != nil {
            break
        }
        fmt.Print(chunk.Content)
    }
    fmt.Println()
}

Key points:

• compose.NewChain creates a linear pipeline with type-safe connections.
• AppendChatTemplate and AppendChatModel are convenience methods for adding component nodes.
• Compiled chains/graphs support all four execution modes: Invoke, Stream, Collect, Transform.
• Template variables are passed as map[string]any and rendered using Jinja2 syntax.

Runnable -- The Universal Execution Interface

All compiled Graph, Chain, and Workflow produce a Runnable[I, O]. This is the single interface that downstream code (including ADK agents) uses to execute orchestrated pipelines.

// github.com/cloudwego/eino/compose
type Runnable[I, O any] interface {
    Invoke(ctx context.Context, input I, opts ...Option) (output O, err error)
    Stream(ctx context.Context, input I, opts ...Option) (output *schema.StreamReader[O], err error)
    Collect(ctx context.Context, input *schema.StreamReader[I], opts ...Option) (output O, err error)
    Transform(ctx context.Context, input *schema.StreamReader[I], opts ...Option) (output *schema.StreamReader[O], err error)
}

Four Execution Modes

How It Works

The compose engine automatically converts between streaming and non-streaming at node boundaries. If you call Stream() on a compiled graph but an internal node only supports Invoke, the framework handles the conversion transparently.

// Compile a graph into a Runnable
compiled, err := graph.Compile(ctx)

// All four modes are available on the same compiled object
result, err := compiled.Invoke(ctx, input)
stream, err := compiled.Stream(ctx, input)
result, err := compiled.Collect(ctx, inputStream)
outStream, err := compiled.Transform(ctx, inputStream)

Passing Options at Runtime

Use compose.WithCallbacks and compose.WithCallOption to pass per-request configuration:

result, err := compiled.Invoke(ctx, input,
    compose.WithCallbacks(handler),
    compose.WithChatModelOption(model.WithTemperature(0.7)).DesignateNode("ChatModel"),
)

See /eino-compose for details on call options and callbacks.

Schema -- Core Data Types

The github.com/cloudwego/eino/schema package defines shared types used across all Eino layers (components, compose, ADK).

Message

The fundamental unit of conversation data.

type Message struct {
    Role                     RoleType            // system, user, assistant, tool
    Content                  string              // text content
    UserInputMultiContent    []MessageInputPart  // multimodal input from user (images, audio, etc.)
    AssistantGenMultiContent []MessageOutputPart // multi-part output from model (e.g., reasoning + text, audio + text). When non-empty, prefer this over Content/ReasoningContent.
    Name                     string              // message name
    ToolCalls                []ToolCall          // tool calls requested by model (assistant only)
    ToolCallID               string              // identifies which tool call this message responds to (tool only)
    ToolName                 string              // tool name (tool only)
    ResponseMeta             *ResponseMeta       // model response metadata (token usage, finish reason, etc.)
    ReasoningContent         string              // model's reasoning/thinking content
    Extra                    map[string]any      // customized information for model implementation
}

Roles

Constructors

schema.SystemMessage("You are a helpful assistant.")
schema.UserMessage("Hello")
schema.AssistantMessage("Hi there!", nil)       // content, toolCalls
schema.ToolMessage("result text", "call_id_1")  // content, toolCallID

ToolCall

When the model decides to call a tool, it returns ToolCalls in the assistant message:

type ToolCall struct {
    Index    *int              // used in stream mode to identify chunks for merging
    ID       string            // identifies the specific tool call
    Type     string            // default "function"
    Function FunctionCall
    Extra    map[string]any    // extra information for the tool call
}

type FunctionCall struct {
    Name      string // function name
    Arguments string // JSON-encoded arguments
}

AgenticMessage

Block-based message type for the agentic path, providing lossless multimodal representation aligned with OpenAI Responses, Claude Message, and Gemini APIs.

type AgenticMessage struct {
    Role          AgenticRoleType      // system, user, assistant
    ContentBlocks []*ContentBlock      // ordered typed content blocks
    ResponseMeta  *AgenticResponseMeta // token usage, provider extensions
    Extra         map[string]any
}

Roles

Note: Unlike classic Message, there is no Tool role. Tool results are ContentBlocks within a User message.

ContentBlock (Tagged Union)

type ContentBlock struct {
    Type ContentBlockType  // discriminator

    // Populated based on Type:
    Reasoning          *Reasoning
    UserInputText      *UserInputText
    UserInputImage     *UserInputImage
    UserInputAudio     *UserInputAudio
    UserInputVideo     *UserInputVideo
    UserInputFile      *UserInputFile
    AssistantGenText   *AssistantGenText
    AssistantGenImage  *AssistantGenImage
    AssistantGenAudio  *AssistantGenAudio
    AssistantGenVideo  *AssistantGenVideo
    FunctionToolCall   *FunctionToolCall
    FunctionToolResult *FunctionToolResult
    ServerToolCall     *ServerToolCall
    ServerToolResult   *ServerToolResult
    MCPToolCall        *MCPToolCall
    MCPToolResult      *MCPToolResult
    // ... and more
}

Key ContentBlock Types

Constructors

schema.SystemAgenticMessage("You are a helpful assistant.")
schema.UserAgenticMessage("Hello")

// Type-safe content block construction
block := schema.NewContentBlock(&schema.FunctionToolCall{
    CallID: "call_1", Name: "search", Arguments: `{"q":"go"}`,
})

Streaming

// Concatenate streaming chunks into a single message
fullMsg, err := schema.ConcatAgenticMessages(chunks)

AgenticMessage vs Message

Document

Unit of data for RAG pipelines.

type Document struct {
    ID       string
    Content  string
    MetaData map[string]any
}

Documents flow through: Loader (load) -> Transformer (split/enrich) -> Indexer (embed + store) -> Retriever (search).

ToolInfo

Describes a tool's interface so the model knows how to call it.

type ToolInfo struct {
    Name  string           // unique tool name
    Desc  string           // description for the model (how/when/why to use)
    Extra map[string]any   // extra information for the tool
    *ParamsOneOf           // embedded; nil means no input parameters
}

Parameters can be described in two ways:

// Option A: use params map
schema.NewParamsOneOfByParams(map[string]*schema.ParameterInfo{
    "city": {Type: "string", Desc: "City name", Required: true},
})

// Option B: use raw JSON Schema
schema.NewParamsOneOfByJSONSchema(schemaObj)

StreamReader[T]

Generic streaming reader used throughout Eino. Returned by Stream(), Transform(), and streaming tool calls.

type StreamReader[T any] struct { /* ... */ }

Usage Pattern

stream, err := model.Stream(ctx, messages)
if err != nil {
    return err
}
defer stream.Close() // ALWAYS close to release resources

for {
    chunk, err := stream.Recv()
    if errors.Is(err, io.EOF) {
        break
    }
    if err != nil {
        return err
    }
    fmt.Print(chunk.Content)
}

Creating Streams

// Create a linked reader/writer pair
reader, writer := schema.Pipe[*schema.Message](bufferSize)

// Write from a goroutine
go func() {
    writer.Send(msg, nil)
    writer.Close()
}()

// Create from a slice (useful in tests)
reader := schema.StreamReaderFromArray(items)

Key Rules

• Always `defer stream.Close()` -- failing to close causes resource leaks
• Single consumer -- a StreamReader can only be read by one goroutine
• EOF signals completion -- errors.Is(err, io.EOF) means the stream ended normally