feat: Converse API Adapter: enable request metadata traces - #473
Open
AlexFoxalt wants to merge 8 commits into
Open
feat: Converse API Adapter: enable request metadata traces#473AlexFoxalt wants to merge 8 commits into
AlexFoxalt wants to merge 8 commits into
Conversation
adubovik
requested changes
Jul 30, 2026
adubovik
requested changes
Aug 3, 2026
| from typing import Literal | ||
|
|
||
| from aidial_sdk.chat_completion import Request as ChatCompletionRequest | ||
| from aidial_sdk.chat_completion import Request |
| def has_converse_api_configuration( | ||
| request: Request | None, upstream_config: UpstreamConfig | ||
| ) -> bool: | ||
| cond1 = None |
Collaborator
There was a problem hiding this comment.
- Why is boolean condition is assigned
None? Is there a difference betweenFalseandNone? - Why did use overcomplicated
any([cond1,cond2])instead of justcond1 or cond2?
Keep it simple:
configuration = (
cf.configuration
if (request and (cf := request.custom_fields))
else None
)
if configuration and (
"performanceConfig" in configuration
or "guardrailConfig" in configuration
):
return True
return (
isinstance(upstream_config, CloudUpstreamConfig)
and upstream_config.claude_client == "converse"
)
Collaborator
Author
There was a problem hiding this comment.
Replaced with suggested code
| from aidial_adapter_bedrock.utils.log_config import bedrock_logger as log | ||
|
|
||
| CONVERSE_API_REQUEST_METADATA_FIELDS = os.getenv( | ||
| "CONVERSE_API_REQUEST_METADATA_FIELDS" |
Collaborator
There was a problem hiding this comment.
Let's use the env helper:
def get_env_list(name: str) -> list[str] | None:
if (value := os.getenv(name)) is not None:
return [str.strip(s) for s in value.split(",")]
return None|
|
||
|
|
||
| def resolve_paths( | ||
| data: dict[str, Any], config: str | None = None |
Collaborator
There was a problem hiding this comment.
Not really "config", but "paths"
| |AWS_SESSION_TOKEN|NA|AWS session token with an access the Bedrock service| | ||
| |AWS_DEFAULT_REGION||AWS region e.g. `us-east-1`| | ||
| |AWS_CLAUDE_DEFAULT_CLIENT|legacy|Default AWS Claude client mode for cloud credentials path. Supported values: `legacy`, `mantle`.| | ||
| |AWS_CLAUDE_DEFAULT_CLIENT|legacy|Default AWS Claude client mode for cloud credentials path. Supported values: `legacy`, `mantle`, `boto`.| |
| |`userClaims`|User claims, addressable by nested paths, e.g. `userClaims.email`| | ||
|
|
||
| Paths use dot-separated object keys and integer list indices. Unset, empty, or | ||
| whitespace-only configuration disables the feature. Unresolvable paths are |
Collaborator
There was a problem hiding this comment.
Unset, empty, or whitespace-only configuration disables the feature
This is redundant and obvious.
| ignored. The `*` wildcard is not supported. | ||
|
|
||
| Bedrock `requestMetadata` limits apply: at most 16 entries, keys and values up | ||
| to 256 characters, and only characters matching `[a-zA-Z0-9\s:_@$#=/+,-.]`. |
Collaborator
There was a problem hiding this comment.
[a-zA-Z0-9\s:_@$#=/+,-.]
Where is this set of symbols documented?
| _MAX_ENTRIES = 16 | ||
| _MAX_KEY_LEN = 256 | ||
| _MAX_VALUE_LEN = 256 | ||
| _ALLOWED = re.compile(r"[^a-zA-Z0-9\s:_@$#=/+,.\-]") |
Collaborator
There was a problem hiding this comment.
Not _ALLOWED, but _DISALLOWED
| ) | ||
| break | ||
|
|
||
| safe_key = _ALLOWED.sub("", key)[:_MAX_KEY_LEN] |
Collaborator
There was a problem hiding this comment.
Let's replace with a valid placeholder symbol, e.g. "_", to preserve the original length.
|
|
||
| def from_user_info(user_info: UserInfo) -> dict[str, str]: | ||
| data = user_info.model_dump(mode="json") | ||
| return _to_bedrock_metadata( |
Collaborator
There was a problem hiding this comment.
Let's print the data in a debug log.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Support Bedrock request metadata tagging for Converse API-backed deployments:
Model invocation logging
Per-request metadata tagging
Added configurable Converse
requestMetadatapropagation viaCONVERSE_API_REQUEST_METADATA_FIELDS, extracting selected DIALUserInfopaths such asroles.0,project, anduserClaims.email.Added Bedrock-compatible request metadata handling, including entry count, key/value length, and character constraints.
Added
botoas a supported Claude client mode to route Claude chat completions through the Converse API path.Updated docs, env examples, and tests for request metadata tracing and Claude client selection.
Bumped related DIAL/Anthropic dependencies and adjusted ORT config.