Skip to content

feat(ai_studio): add thinking configuration in request Options#9

Merged
OrriMandarin merged 5 commits into
mainfrom
add_thinking_confing_in_aistudio
Sep 18, 2025
Merged

feat(ai_studio): add thinking configuration in request Options#9
OrriMandarin merged 5 commits into
mainfrom
add_thinking_confing_in_aistudio

Conversation

@OrriMandarin

Copy link
Copy Markdown
Contributor

This pull request adds support for a new "thinking" configuration to the Google AI Studio provider, allowing users to specify options such as IncludeThoughts and Budget in their requests.
To disable thinking mode, set the Budget to 0

Support for "thinking" configuration:

  • Added a new ThinkingConfig struct to request_options.go, and extended RequestOptions to include an optional Thinking field. This allows users to specify IncludeThoughts and Budget when making requests.
  • Updated the request adaptation logic in aistudio.go to include the thinkingConfig in the request payload when the Thinking option is provided.

Testing enhancements:

  • Added a new test function TestGoogleAiRequestWithThinking in aistudio_test.go to verify that the thinkingConfig fields are correctly included or omitted in the payload based on the provided options. The tests cover cases for no config, only IncludeThoughts, only Budget, both fields, and disabling thinking.
  • Added github.com/samber/lo to the imports in aistudio_test.go to support pointer creation for test values.

@OrriMandarin OrriMandarin force-pushed the add_thinking_confing_in_aistudio branch from 53113d8 to e9a5431 Compare September 17, 2025 12:24
@OrriMandarin OrriMandarin self-assigned this Sep 17, 2025
@OrriMandarin OrriMandarin added the enhancement New feature or request label Sep 17, 2025
@OrriMandarin OrriMandarin requested a review from apognu September 17, 2025 12:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request adds support for a "thinking" configuration to the Google AI Studio provider, enabling users to control AI reasoning behavior through IncludeThoughts and Budget parameters. To disable thinking mode, users can set the Budget to 0.

  • Added ThinkingConfig struct with IncludeThoughts (bool) and Budget (*int32) fields
  • Extended RequestOptions to include optional Thinking field
  • Updated request adaptation logic to include thinking configuration in API payloads

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 9 comments.

File Description
llms/aistudio/request_options.go Defines new ThinkingConfig struct and adds Thinking field to RequestOptions
llms/aistudio/aistudio.go Updates request adaptation to include thinking configuration in API payload
llms/aistudio/aistudio_test.go Adds comprehensive tests for thinking configuration scenarios including edge cases

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread llms/aistudio/aistudio_test.go Outdated
Comment thread llms/aistudio/aistudio_test.go Outdated
Comment thread llms/aistudio/aistudio_test.go
Comment thread llms/aistudio/aistudio_test.go
Comment thread llms/aistudio/aistudio_test.go
Comment thread llms/aistudio/aistudio_test.go
Comment thread llms/aistudio/aistudio_test.go
Comment thread llms/aistudio/aistudio_test.go
Comment thread llms/aistudio/aistudio_test.go
@OrriMandarin

Copy link
Copy Markdown
Contributor Author

I've run some tests: with thinking mode disabled, my request completed in 7 seconds, compared to 15 seconds when thinking mode was enabled.

I also checked the metadata—thought token usage was 0 without thinking mode, and a higher value when thinking mode was active.

@OrriMandarin

Copy link
Copy Markdown
Contributor Author

We can not disable the thinking mode for gemini-2.5-pro model. The provider returns an 400 error

@apognu

apognu commented Sep 17, 2025

Copy link
Copy Markdown
Collaborator

Before I get into the code, I wonder if we shouldn't add a top-level, provider-agnostic settings to (at the minimum) enable and disable reasoning.

The provider code could then do The Right Thing ™️ to do that with its API (or ignore it if the provider and/or the model does not support it), and accept extra, provider-specific settings if appropriate.

My reasoning (see what I did there?) behind that is that the decision to enable or disable reasoning would be lost when switching provider, which arguably is not what one wants to do.

Comment thread llms/aistudio/request_options.go Outdated
@OrriMandarin

OrriMandarin commented Sep 17, 2025

Copy link
Copy Markdown
Contributor Author

Before I get into the code, I wonder if we shouldn't add a top-level, provider-agnostic settings to (at the minimum) enable and disable reasoning.

The provider code could then do The Right Thing ™️ to do that with its API (or ignore it if the provider and/or the model does not support it), and accept extra, provider-specific settings if appropriate.

My reasoning (see what I did there?) behind that is that the decision to enable or disable reasoning would be lost when switching provider, which arguably is not what one wants to do.

@apognu
It's possible to implement a top-level setting to disable reasoning, but the process varies significantly depending on the provider, making it complex to manage across platforms.

For example,
Gemini's API uses the Thinking config and Budget parameters
Mistral requires a special field for prompt-mode
OpenAI doesn't let you fully disable reasoning, only adjust the "effort." If you want to avoid the reasoning layer with OpenAI, you'd need to use an older model lol (see: https://community.openai.com/t/need-reasoning-false-option-for-gpt-5/1351588/2)
Ollama supports disabling reasoning via the think field.

For Mistral, OpenAI, and Ollama, they all use the OpenAI provider, but the appropriate configuration depends on the specific model. Since the required settings vary by model, it's difficult to determine in advance which configuration should be applied.

@OrriMandarin OrriMandarin requested a review from apognu September 17, 2025 15:41
@apognu

apognu commented Sep 17, 2025

Copy link
Copy Markdown
Collaborator

It's possible to implement a top-level setting to disable reasoning, but the process varies significantly depending on the provider, making it complex to manage across platforms.

The handling of that setting would remain the responsibility of the adapters (same as you implemented), but the setting itself could be set on the outer Request (same as the temperature setting).

The adapter would (if applicable) retrieve that setting and do whatever is appropriate for that provider. We can keep the budget in the provider, since it is highly different across the board.

…thThinking method; update tests for new functionality

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread llms/aistudio/aistudio.go
Comment thread llms/aistudio/aistudio.go Outdated
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Comment thread llms/aistudio/aistudio_test.go Outdated
@OrriMandarin OrriMandarin merged commit 35a7e6d into main Sep 18, 2025
1 check passed
@OrriMandarin OrriMandarin deleted the add_thinking_confing_in_aistudio branch September 18, 2025 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants