Skip to content

feat(schema): add streaming programming examples#169

Open
shentongmartin wants to merge 2 commits intomainfrom
feat/stream
Open

feat(schema): add streaming programming examples#169
shentongmartin wants to merge 2 commits intomainfrom
feat/stream

Conversation

@shentongmartin
Copy link
Contributor

@shentongmartin shentongmartin commented Feb 4, 2026

Summary

This PR adds comprehensive examples demonstrating Eino's streaming programming capabilities, covering all four streaming paradigms documented in the CloudWeGo documentation.

Changes

Example Lambda Type Pattern Use Case
1_invoke InvokableLambda Ping-Pong Text summarizer
2_stream StreamableLambda Server-Streaming Word-by-word generator
3_collect CollectableLambda Client-Streaming Log aggregator
4_transform TransformableLambda Bidirectional-Streaming Real-time text processor
5_auto_streaming Mixed Auto conversion Pipeline with mixed nodes
6_stream_reader_utils N/A Utilities Pipe, Convert, FromArray, ErrNoValue
7_merge_stream_readers N/A Fan-in Multi-source data aggregator

Key Insights

Streaming Paradigm Coverage

The examples cover all four streaming paradigms in Eino:

  • Invoke: Non-stream input -> Non-stream output (Ping-Pong mode)
  • Stream: Non-stream input -> Stream output (Server-Streaming mode)
  • Collect: Stream input -> Non-stream output (Client-Streaming mode)
  • Transform: Stream input -> Stream output (Bidirectional-Streaming mode)

Practical Use Cases

Each example uses realistic, self-contained scenarios without external dependencies:

  • Text processing and summarization
  • Simulated LLM token streaming
  • Log aggregation and statistics
  • Sensitive data filtering
  • Multi-source event merging

StreamReader Utilities

Example 6 demonstrates essential StreamReader utilities:

  • schema.Pipe[T]: Create producer-consumer stream pairs
  • schema.StreamReaderFromArray: Convert arrays to streams
  • schema.StreamReaderWithConvert: Transform stream element types
  • schema.ErrNoValue: Filter elements from streams

Auto Streaming/Concat

Example 5 shows how Eino automatically handles type conversion between nodes with mismatched stream types.

Testing

All examples compile and run successfully:

go build ./schema/stream/...
cd schema/stream/<example_dir> && go run .

概要

本 PR 添加了全面的示例,展示 Eino 的流式编程能力,涵盖 CloudWeGo 文档中记录的所有四种流式范式。

变更

示例 Lambda 类型 模式 用例
1_invoke InvokableLambda Ping-Pong 文本摘要器
2_stream StreamableLambda Server-Streaming 逐词生成器
3_collect CollectableLambda Client-Streaming 日志聚合器
4_transform TransformableLambda Bidirectional-Streaming 实时文本处理器
5_auto_streaming 混合 自动转换 混合节点管道
6_stream_reader_utils N/A 工具函数 Pipe、Convert、FromArray、ErrNoValue
7_merge_stream_readers N/A Fan-in 多源数据聚合器

关键洞察

流式范式覆盖

示例涵盖了 Eino 中的所有四种流式范式:

  • Invoke:非流式输入 -> 非流式输出(Ping-Pong 模式)
  • Stream:非流式输入 -> 流式输出(Server-Streaming 模式)
  • Collect:流式输入 -> 非流式输出(Client-Streaming 模式)
  • Transform:流式输入 -> 流式输出(Bidirectional-Streaming 模式)

实用场景

每个示例使用真实的、自包含的场景,无需外部依赖:

  • 文本处理和摘要
  • 模拟 LLM token 流式输出
  • 日志聚合和统计
  • 敏感数据过滤
  • 多源事件合并

StreamReader 工具函数

示例 6 展示了必要的 StreamReader 工具函数:

  • schema.Pipe[T]:创建生产者-消费者流对
  • schema.StreamReaderFromArray:将数组转换为流
  • schema.StreamReaderWithConvert:转换流元素类型
  • schema.ErrNoValue:从流中过滤元素

自动流化/合包

示例 5 展示了 Eino 如何自动处理流类型不匹配的节点之间的类型转换。

测试

所有示例编译和运行成功:

go build ./schema/stream/...
cd schema/stream/<example_dir> && go run .
```~

Add comprehensive examples demonstrating Eino's four streaming paradigms:

- 1_invoke: InvokableLambda (Ping-Pong mode) - Text summarizer
- 2_stream: StreamableLambda (Server-Streaming) - Word-by-word generator
- 3_collect: CollectableLambda (Client-Streaming) - Log aggregator
- 4_transform: TransformableLambda (Bidirectional-Streaming) - Text processor
- 5_auto_streaming: Auto streaming/concat between nodes
- 6_stream_reader_utils: StreamReader utilities (Pipe, Convert, etc.)
- 7_merge_stream_readers: MergeStreamReaders for fan-in pattern

Each example is self-contained, runnable, and uses mock data without external dependencies.~

Change-Id: I430c7e75fe2c3c8221bf0515ea7ebfaf19a9a17c
Change-Id: I98ad53b35743d67a9b775358042e916be5e239d6
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant