feat(ai_studio): add thinking configuration in request Options#9
Conversation
53113d8 to
e9a5431
Compare
There was a problem hiding this comment.
Pull Request Overview
This pull request adds support for a "thinking" configuration to the Google AI Studio provider, enabling users to control AI reasoning behavior through IncludeThoughts and Budget parameters. To disable thinking mode, users can set the Budget to 0.
- Added
ThinkingConfigstruct withIncludeThoughts(bool) andBudget(*int32) fields - Extended
RequestOptionsto include optionalThinkingfield - Updated request adaptation logic to include thinking configuration in API payloads
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.
| File | Description |
|---|---|
| llms/aistudio/request_options.go | Defines new ThinkingConfig struct and adds Thinking field to RequestOptions |
| llms/aistudio/aistudio.go | Updates request adaptation to include thinking configuration in API payload |
| llms/aistudio/aistudio_test.go | Adds comprehensive tests for thinking configuration scenarios including edge cases |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
I've run some tests: with thinking mode disabled, my request completed in 7 seconds, compared to 15 seconds when thinking mode was enabled. I also checked the metadata—thought token usage was 0 without thinking mode, and a higher value when thinking mode was active. |
|
We can not disable the thinking mode for |
|
Before I get into the code, I wonder if we shouldn't add a top-level, provider-agnostic settings to (at the minimum) enable and disable reasoning. The provider code could then do The Right Thing ™️ to do that with its API (or ignore it if the provider and/or the model does not support it), and accept extra, provider-specific settings if appropriate. My reasoning (see what I did there?) behind that is that the decision to enable or disable reasoning would be lost when switching provider, which arguably is not what one wants to do. |
@apognu For example, For Mistral, OpenAI, and Ollama, they all use the OpenAI provider, but the appropriate configuration depends on the specific model. Since the required settings vary by model, it's difficult to determine in advance which configuration should be applied. |
…type to int in ThinkingConfig
The handling of that setting would remain the responsibility of the adapters (same as you implemented), but the setting itself could be set on the outer Request (same as the temperature setting). The adapter would (if applicable) retrieve that setting and do whatever is appropriate for that provider. We can keep the budget in the provider, since it is highly different across the board. |
…thThinking method; update tests for new functionality
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This pull request adds support for a new "thinking" configuration to the Google AI Studio provider, allowing users to specify options such as
IncludeThoughtsandBudgetin their requests.To disable thinking mode, set the Budget to 0
Support for "thinking" configuration:
ThinkingConfigstruct torequest_options.go, and extendedRequestOptionsto include an optionalThinkingfield. This allows users to specifyIncludeThoughtsandBudgetwhen making requests.aistudio.goto include thethinkingConfigin the request payload when theThinkingoption is provided.Testing enhancements:
TestGoogleAiRequestWithThinkinginaistudio_test.goto verify that thethinkingConfigfields are correctly included or omitted in the payload based on the provided options. The tests cover cases for no config, onlyIncludeThoughts, onlyBudget, both fields, and disabling thinking.github.com/samber/loto the imports inaistudio_test.goto support pointer creation for test values.