[Bug]: OpenAI agent chat traces don't contain input/output messages and other fields

[shivamMg]

Component

General

Description

For agents using openai python package, the chat traces show up when setting the following env vars:

os.environ.setdefault("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT", "SPAN_AND_EVENT")
os.environ.setdefault("OTEL_SEMCONV_STABILITY_OPT_IN", "gen_ai_latest_experimental")

But these chat traces don't contain the following 4 GenAI fields:

gen_ai.input.messages
gen_ai.output.messages
gen_ai.tool.definitions
gen_ai.system_instructions

These fields are vital to debug agents using LLMs.

I've tried my experiments with a version of this sample agent that's the repo: https://github.com/microsoft/opentelemetry-distro-python/blob/9d3b25fa48c8bbd283203aa1757a0aa774458ff7/samples/openai/sample_openai_agents.py

Expected Behavior

Chat traces from openai agents should populate the following 4 GenAI fields:

gen_ai.input.messages
gen_ai.output.messages
gen_ai.tool.definitions
gen_ai.system_instructions

Steps to Reproduce

Use the repo agent with app insights: https://github.com/microsoft/opentelemetry-distro-python/blob/9d3b25fa48c8bbd283203aa1757a0aa774458ff7/samples/openai/sample_openai_agents.py

Environment

Hosted Agent on Foundry

Additional Context

No response

[github-actions]

cc @hectorhdzg @JacksonWeber @lzchen @MSNev @rads-1996 @jeremydvoss

[rads-1996]

@shivamMg Can you please share a screenshot of how you are configuring the microsoft distro?