Support AI Recommender#13
Conversation
…f environment variable
…nd bullet-formatted titles Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
…, author, and ISBN Co-authored-by: aider (openrouter/anthropic/claude-sonnet-4) <aider@aider.chat>
| println!("Request content: {}", content); | ||
|
|
||
| let request = CreateChatCompletionRequestArgs::default() | ||
| .model("openai/gpt-4.1") |
There was a problem hiding this comment.
Do you think hard-coding the model makes sense here? IMHO we should have this as an argument / config.
There was a problem hiding this comment.
I moved all model variables into a model configuration 👍
| Always respond with valid JSON in the format: `{}`. | ||
| The response itself should be valid json. | ||
| Please do not include any additional text like markdown or explanations."#, titles_bullets, json_format); |
There was a problem hiding this comment.
AFAIK this it not the most up to date version to force an AI to return data in a specific format. Most APIs allow specifying "structured output" or so.
I found this example in the async_openai crate showcasing how to do it.
There was a problem hiding this comment.
Really good point, thank you! I checked the docs while adapting the response format and it says:
/// Setting to `json_object` enables JSON mode. This guarantees that the message the model generates is valid JSON.
///
/// Note that your system prompt must still instruct the model to produce JSON, and to help ensure you don't forget,
/// the API will throw an error if the string `JSON` does not appear in your system message. Also note that the message
/// content may be partial (i.e. cut off) if `finish_reason="length"`, which indicates the generation
/// exceeded `max_tokens` or the conversation exceeded the max context length.
///
/// Must be one of `text` or `json_object`.
pub r#type: ChatCompletionResponseFormatType,
So I'll keep the prompt as is and additionally set the format.
| .model(config.model.identifier()) | ||
| .max_tokens(500_u16) | ||
| .response_format(ChatCompletionResponseFormat { | ||
| r#type: ChatCompletionResponseFormatType::JsonObject, |
There was a problem hiding this comment.
I guess we could be even more detailed here and use this ResponseFormat::JsonSchema specifying the exact format that we want, no?
molikuner
left a comment
molikuner
left a comment
There was a problem hiding this comment.
Let's start with this for now and once there is feedback from users we can iterate on it. 👍
|
@molikuner Thanks for your remarks. I'll leave the json schema/response unresolved and created an issue to check later. |
2 participants
No description provided.