diff --git a/examples/README.md b/examples/README.md index 90c4c97..a644172 100644 --- a/examples/README.md +++ b/examples/README.md @@ -15,11 +15,11 @@ For a hosted gateway, set the base URL, API key, and model name supplied by the | Example | What it demonstrates | Command | | --- | --- | --- | -| [basic_chat.py](basic_chat.py) | Single-turn and multi-turn chat | `python examples/basic_chat.py` | -| [streaming.py](streaming.py) | Stream chunks and reconstruct content | `python examples/streaming.py` | -| [compare_streaming.py](compare_streaming.py) | First-token and total latency | `python examples/compare_streaming.py` | -| [tool_calling.py](tool_calling.py) | One tool request and a tool-result loop | `python examples/tool_calling.py` | -| [reasoning_mode.py](reasoning_mode.py) | `no_think` versus `high` | `python examples/reasoning_mode.py` | -| [error_retry.py](error_retry.py) | Timeout, network, 429, and 5xx retry policy | `python examples/error_retry.py` | +| [basic_chat.py](basic_chat.py) ([guide](basic_chat.md)) | Single-turn and multi-turn chat | `python examples/basic_chat.py` | +| [streaming.py](streaming.py) ([guide](streaming.md)) | Stream chunks and reconstruct content | `python examples/streaming.py` | +| [compare_streaming.py](compare_streaming.py) ([guide](compare_streaming.md)) | First-token and total latency | `python examples/compare_streaming.py` | +| [tool_calling.py](tool_calling.py) ([guide](tool_calling.md)) | One tool request and a tool-result loop | `python examples/tool_calling.py` | +| [reasoning_mode.py](reasoning_mode.py) ([guide](reasoning_mode.md)) | `no_think` versus `high` | `python examples/reasoning_mode.py` | +| [error_retry.py](error_retry.py) ([guide](error_retry.md)) | Timeout, network, 429, and 5xx retry policy | `python examples/error_retry.py` | Generated text, timings, IDs, tool arguments, and usage counts vary by server and request load. See [`quickstart.md`](../quickstart.md) for API parameters and troubleshooting. diff --git a/examples/basic_chat.md b/examples/basic_chat.md new file mode 100644 index 0000000..f8eb434 --- /dev/null +++ b/examples/basic_chat.md @@ -0,0 +1,38 @@ +# Basic chat + +演示单轮和多轮 Chat Completions。先安装依赖并配置 `HY3_BASE_URL`、`HY3_API_KEY`、`HY3_MODEL`,再运行: + +```bash +python -m pip install --upgrade openai +python examples/basic_chat.py +``` + +脚本发送的核心请求: + +```json +{ + "model": "hy3", + "messages": [{"role": "user", "content": "用一句话介绍 Hy3。"}], + "temperature": 0.9, + "top_p": 1.0, + "max_tokens": 128, + "extra_body": {"chat_template_kwargs": {"reasoning_effort": "no_think"}} +} +``` + +读取 `response.choices[0].message.content` 获取文本,读取 `response.choices[0].finish_reason` 获取结束原因;多轮请求则把历史消息继续放入 `messages`。 + +响应结构示例: + +```json +{"id":"chatcmpl-...","choices":[{"message":{"role":"assistant","content":"Hy3 是腾讯混元团队推出的混合专家模型。"},"finish_reason":"stop"}]} +``` + +示例输出: + +```text +single-turn: Hy3 是腾讯混元团队推出的混合专家模型。 +multi-turn: 清爽且果香自然。 +``` + +实际措辞、ID 和 token 数会因模型与部署而变化。 diff --git a/examples/compare_streaming.md b/examples/compare_streaming.md new file mode 100644 index 0000000..a3be329 --- /dev/null +++ b/examples/compare_streaming.md @@ -0,0 +1,26 @@ +# Non-streaming vs streaming + +比较客户端观测到的总耗时,以及流式请求的首 token 时延: + +```bash +python -m pip install --upgrade openai +python examples/compare_streaming.py +``` + +两次请求使用同一组参数: + +```json +{"model":"hy3","messages":[{"role":"user","content":"用两句话解释什么是 API。"}],"temperature":0.9,"top_p":1.0,"max_tokens":128,"stream":false} +``` + +第二次请求额外设置 `stream: true`。非流式响应从 `choices[0].message.content` 解析;流式响应从每个 `choices[0].delta.content` 解析并拼接。计时从调用开始到响应结束,首 token 计时取第一个非空 delta 的时间。 + +示例输出(仅用于说明格式): + +```text +non-streaming total: 1.842s +streaming first token: 0.241s +streaming total: 1.801s +``` + +网络、并发和服务端批处理会影响结果,应在自己的负载下比较。 diff --git a/examples/error_retry.md b/examples/error_retry.md new file mode 100644 index 0000000..1023037 --- /dev/null +++ b/examples/error_retry.md @@ -0,0 +1,20 @@ +# Error handling and retry + +演示超时、网络错误、429 和临时 5xx 的有限重试与指数退避: + +```bash +python -m pip install --upgrade openai +python examples/error_retry.py +``` + +脚本显式设置 `timeout=30` 和 `max_retries=0`,由应用控制最多 4 次请求。它重试 `RateLimitError`、`APITimeoutError`、`APIConnectionError` 和 5xx `APIStatusError`,优先遵循 `Retry-After`,否则使用带随机抖动的指数退避;400、401 等客户端错误立即抛出。 + +每次成功响应仍按标准结构解析:`response.choices[0].message.content` 是答案,`response.id` 是可记录的请求 ID。示例输出: + +```text +429 rate limit; retrying in 1.17s +answer: OK +request_id: chatcmpl-... +``` + +只对幂等或业务上可安全重复的请求重试;避免记录 API key、完整 prompt 和敏感工具参数。生产系统还应设置总截止时间、监控最终失败率,并处理服务端返回的限流窗口。 diff --git a/examples/reasoning_mode.md b/examples/reasoning_mode.md new file mode 100644 index 0000000..11ad0c0 --- /dev/null +++ b/examples/reasoning_mode.md @@ -0,0 +1,32 @@ +# Reasoning mode + +对比关闭思考和开启较高思考强度时的请求与返回: + +```bash +python -m pip install --upgrade openai +python examples/reasoning_mode.py +``` + +两次请求只改变: + +```json +{"extra_body":{"chat_template_kwargs":{"reasoning_effort":"no_think"}}} +``` + +或: + +```json +{"extra_body":{"chat_template_kwargs":{"reasoning_effort":"high"}}} +``` + +正常答案读取 `response.choices[0].message.content`。部分网关会额外提供 `message.reasoning_content`,脚本用 `getattr` 兼容该字段;不要假设所有部署都会返回它,也不要把它当作稳定的业务协议。 + +示例输出: + +```text +[no_think] content: 最终价格是 150 元。 +[high] content: 先计算 200×0.8=160,再计算 160-10=150,所以最终价格是 150 元。 +[high] reasoning_content: ... +``` + +字段名和内容取决于 serving stack,实际接入时应打印完整 message 做兼容性确认。 diff --git a/examples/streaming.md b/examples/streaming.md new file mode 100644 index 0000000..fa01d09 --- /dev/null +++ b/examples/streaming.md @@ -0,0 +1,27 @@ +# Streaming + +演示流式请求、逐 chunk 解析和最终文本重建: + +```bash +python -m pip install --upgrade openai +python examples/streaming.py +``` + +请求在普通 Chat Completions 参数上增加 `stream: true` 和可选的 `stream_options.include_usage`: + +```json +{"model":"hy3","messages":[{"role":"user","content":"用三点说明流式输出的好处。"}],"temperature":0.9,"top_p":1.0,"max_tokens":256,"stream":true,"stream_options":{"include_usage":true}} +``` + +每个 chunk 先判断 `chunk.choices` 是否为空,再读取 `chunk.choices[0].delta.content`;末尾 chunk 可能没有 choices,不能直接索引。把非空文本片段追加后,用 `"".join(parts)` 得到完整回答,并从 `finish_reason`、`usage` 读取结束状态和用量。 + +示例输出: + +```text +1. 更快看到首字。 +2. 适合实时展示。 +3. 长回答无需等待全部生成。 +full text: 1. 更快看到首字。2. 适合实时展示。3. 长回答无需等待全部生成。 +finish_reason: stop +usage: CompletionUsage(completion_tokens=..., prompt_tokens=..., total_tokens=...) +``` diff --git a/examples/tool_calling.md b/examples/tool_calling.md new file mode 100644 index 0000000..635bdf1 --- /dev/null +++ b/examples/tool_calling.md @@ -0,0 +1,29 @@ +# Tool calling + +演示一次工具请求,以及执行工具后把结果发回模型的多轮循环: + +```bash +python -m pip install --upgrade openai +python examples/tool_calling.py +``` + +请求包含 OpenAI 兼容的函数定义: + +```json +{"model":"hy3","messages":[{"role":"user","content":"北京现在天气怎么样?"}],"tools":[{"type":"function","function":{"name":"get_weather","description":"查询城市当前天气。","parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}],"tool_choice":"auto"} +``` + +解析 `response.choices[0].message.tool_calls`,对每个调用读取 `id`、`function.name` 和 JSON 字符串 `function.arguments`。应用执行工具后追加 assistant 消息和: + +```json +{"role":"tool","tool_call_id":"call_...","content":"{\"city\":\"北京\",\"condition\":\"晴\",\"temperature_c\":26}"} +``` + +重复请求直到没有 `tool_calls`,再读取 `message.content`。生产代码应校验参数、限制循环次数,并将工具错误作为结构化结果返回。 + +示例输出: + +```text +one-shot tool_calls: [... name='get_weather' ...] +final answer: 北京今天晴,26°C,建议穿轻薄长袖,外出可备一件薄外套。 +```