Skip to content

[Bug]: The Claude model specifies that the output array is to be returned as a string. #127

Description

@lban2049

Describe the bug

The following Agent specifies the output as an array

export const pageStructurePlannerAgent = AIAgent.from({
  name: 'Page Structure Planner',
  subscribeTopic: [UserInputTopic, 'revision_request'],
  publishTopic: (output) => {
    logger.info('page structure planner output', JSON.stringify(output.structurePlan));
    return 'validation_request';
  },
  instructions: prompt,
  inputSchema: z.object({
    context: z.string(),
    question: z.string(),
    locale: z.string(),
    outputSchema: z.string(),
    expectedSections: z.string(),
    initialStructurePlan: z.array(
      z.object({
        sectionName: z.string(),
        sectionContentPlan: z.string(),
      })
    ),
    validationFeedback: z
      .object({
        isValid: z.boolean(),
        missingSections: z.array(z.string()),
        duplicateContent: z.array(
          z.object({
            sections: z.array(z.string()),
            description: z.string(),
            score: z.number(),
          })
        ),
      })
      .optional(),
  }),
  outputSchema: z.object({
    structurePlan: z.array(
      z.object({
        sectionName: z.string(),
        sectionContentPlan: z.string().describe('该section的内容规划描述'),
      })
    ),
  }),
  includeInputInOutput: true,
});

However, the model outputs a string resulting in a runtime error

aigne:core Invoke agent ClaudeChatModel succeed with output: {
  aigne:core   json: {
  aigne:core     structurePlan: '[\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "heroSection",\n' +
  aigne:core       '    "sectionContentPlan": "展示AIGNE Framework的核心价值主张,强调其作为AI应用开发框架的优势。标题应突出简化开发过程、无代码开发体验和加速开发的特点。描述应简洁明了地介绍AIGNE的主要功能和价值,包括函数式编程和模块化设计理念。整体内容应吸引开发者,让他们立即理解AIGNE能为他们解决什么问题。"\n' +
  aigne:core       '  },\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "featuresSection",\n' +
  aigne:core       '    "sectionContentPlan": "展示AIGNE Framework的5个核心特性,每个特性配有相应图片。包括:1) TypeScript支持(配图:TypeScript支持图片);2) Blocklet生态系统支持(配图:Blocklet生态集成图片);3) 多种工作流模式支持(配图:多工作流支持图片);4) AIGNE CLI命令行工具(配图:CLI能力图片);5) MCP集成能力(配图:MCP集成图片)。每个特性包含标题、详细描述和相关图片,突出AIGNE的技术优势和开发便利性。"\n' +
  aigne:core       '  },\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "architectureSection",\n' +
  aigne:core       '    "sectionContentPlan": "详细展示AIGNE Framework的4个核心架构组件,每个组件配有相应图片:1) Chat Model(配图:大语言模型LOGO列表图片),描述其作为大语言模型抽象的类,支持多种大语言模型;2) Agent系统(配图:Agent System图片),说明不同类型Agent的功能和用途,包括AI Agent、Function Agent和MCP Agent;3) Workflow系统(配图:多行横线,体现任务流动图片),介绍6种工作流模式及其应用场景;4) Execution Engine(配图:中间有圆球的图片),描述其作为调度Agent的核心引擎。每个架构组件需包含标题、详细描述和对应图片。"\n' +
  aigne:core       '  },\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "workflowSection",\n' +
  aigne:core       '    "sectionContentPlan": "详细介绍AIGNE Framework支持的6种工作流模式:1) 顺序工作流 - 适用于需要按步骤执行的任务;2) 独立工作流 - 适用于可以并行处理的独立任务;3) 交接工作流 - 适用于需要用户参与的交互式任务;4) 反思工作流 - 适用于需要自我评估和改进的复杂任务;5) 代码执行工作流 - 适用于需要执行代码的场景;6) 编辑工作流 - 适用于需要多个专业Agent协同工作的任务。每种工作流需包含标题和详细描述,说明其使用场景和优势。"\n' +
  aigne:core       '  },\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "mcpIntegrationSection",\n' +
  aigne:core       '    "sectionContentPlan": "介绍AIGNE Framework的MCP外部集成能力,展示3个主要集成示例:1) Puppeteer - 用于网页自动化和爬取;2) SQLite - 提供数据库集成能力;3) GitHub - 实现代码仓库交互。每个集成包含标题和详细描述,说明如何通过MCP协议连接外部服务,以及这些集成如何扩展AIGNE的功能。同时提及可以通过MCPAgent连接到相应的MCP服务器或创建自定义FunctionAgent封装API调用。"\n' +
  aigne:core       '  },\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "supportModelSection",\n' +
  aigne:core       '    "sectionContentPlan": "列出AIGNE Framework支持的7种大语言模型,包括各模型名称和对应链接:1) OpenAI (https://openai.com/);2) Claude (https://claude.ai/);3) XAI (https://x.ai/);4) DeepSeek (https://www.deepseek.com/);5) Gemini (https://gemini.google.com/);6) Ollama (https://ollama.com/);7) OpenRouter (https://openrouter.ai/)。强调AIGNE框架的扩展性,允许通过实现自定义的LLMModel类来与任意LLM服务交互。"\n' +
  aigne:core       '  },\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "faqSection",\n' +
  aigne:core       '    "sectionContentPlan": "提供5个关于AIGNE Framework的常见问题及详细解答:1) 支持哪些大语言模型 - 解释AIGNE的扩展性;2) 是否有使用示例 - 提供Github仓库链接;3) 是否有相关的学习资料 - 指向官方站点和Github仓库;4) 如何选择合适的工作流 - 根据任务复杂度、交互需求、并行性等因素选择;5) 工作流可以组合使用吗 - 说明工作流的自由组合特性。每个问题需包含清晰的问题描述和全面的回答,帮助用户快速解决疑问。"\n' +
  aigne:core       '  },\n' +
  aigne:core       '  {\n' +
  aigne:core       '    "sectionName": "callToActionSection",\n' +
  aigne:core       '    "sectionContentPlan": "创建一个引人注目的行动召唤区域,鼓励访问者开始使用AIGNE Framework。标题应强调开始使用AIGNE的简单性,描述应突出快速上手的步骤,如安装、配置和创建第一个Agent。行动按钮文本应简洁有力,如"立即开始"或"免费试用",引导用户进入下一步操作。整体内容应传达使用AIGNE Framework的紧迫感和价值。"\n' +
  aigne:core       '  }\n' +
  aigne:core       ']'
  aigne:core   },
  aigne:core   usage: { inputTokens: 7168, outputTokens: 2851 },
  aigne:core   model: 'claude-3-7-sonnet-latest'
  aigne:core } +51s
  aigne:core Invoke agent Page Structure Planner failed with error: Error: Agent Page Structure Planner output check arguments error: structurePlan: Expected array, received string
  aigne:core     at checkArguments (/Users/lban/arcblock/code/pages-kit/node_modules/@aigne/core/lib/esm/utils/type-utils.js:114:23)
  aigne:core     at processAgentOutput (/Users/lban/arcblock/code/pages-kit/node_modules/@aigne/core/lib/esm/agents/agent.js:280:30)
  aigne:core     at processAgentOutput (/Users/lban/arcblock/code/pages-kit/node_modules/@aigne/core/lib/esm/agents/agent.js:279:30)
  aigne:core     at <anonymous> (/Users/lban/arcblock/code/pages-kit/node_modules/@aigne/core/lib/esm/agents/agent.js:247:39)
  aigne:core     at <anonymous> (/Users/lban/arcblock/code/pages-kit/node_modules/@aigne/core/lib/esm/agents/agent.js:246:64)
  aigne:core     at pull (/Users/lban/arcblock/code/pages-kit/node_modules/@aigne/core/lib/esm/utils/stream-utils.js:82:46)
  aigne:core     at processTicksAndRejections (native:7:39) +6ms
Error in multi-agent page content generation: 109 |         });
110 |     }
111 |     catch (error) {
112 |         if (error instanceof z.ZodError) {
113 |             const message = error.issues.map((i) => `${i.path}: ${i.message}`).join(", ");
114 |             throw new Error(`${prefix} check arguments error: ${message}`);
                            ^
error: Agent Page Structure Planner output check arguments error: structurePlan: Expected array, received string
      at checkArguments (/Users/lban/arcblock/code/pages-kit/node_modules/@aigne/core/lib/esm/utils/type-utils.js:114:23)

Error generating page content for model claude-3-7-sonnet-latest: Agent Page Structure Planner output check arguments error: structurePlan: Expected array, received string

Reproduction Steps

This error doesn't happen every time.

Observed Behavior

Consider adding a check on the output type in the framework, if it's a string but the specified output is structured, try parsing the string to get the final result

Expected Behavior

Can correctly get the output of a given structure

Possible Solution

No response

Additional Information/Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions