[SVLS-9189] feat(logs): add config param DD_DURABLE_FUNCTION_LOG_BUFFER_SIZE#1239
[SVLS-9189] feat(logs): add config param DD_DURABLE_FUNCTION_LOG_BUFFER_SIZE#1239lym953 wants to merge 2 commits into
Conversation
Adds `durable_function_log_buffer_size` (env: `DD_DURABLE_FUNCTION_LOG_BUFFER_SIZE`, default: 50) to control the max number of request ID keys held in `held_logs` waiting for durable execution context. Setting it to 0 disables holding entirely, so logs are flushed immediately without durable context enrichment — useful when the tracer is not installed and the buffer would otherwise delay log delivery until full or until extension shutdown. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
lym953
changed the title
Default to 0 (disabled) until tracer-side durable execution support is released. Customers can set DD_DURABLE_FUNCTION_LOG_BUFFER_SIZE=50 to opt into context enrichment once the tracer is available. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Adds a new DD_DURABLE_FUNCTION_LOG_BUFFER_SIZE configuration option that controls the maximum number of request IDs whose logs are held in the durable-function log buffer. The previous hardcoded constant (HELD_LOGS_MAX_KEYS = 50) is replaced by this configurable value, defaulted to 0, which bypasses the hold mechanism entirely so logs are flushed immediately without durable execution context enrichment. This avoids missing-log issues when the tracer is not installed on durable functions.
Changes:
- New
durable_function_log_buffer_sizeconfig field (env + yaml + default), defaulting to 0. LambdaProcessorreads this value, replacing theHELD_LOGS_MAX_KEYSconstant, and short-circuits the hold path when set to 0.- Existing durable-arn tests explicitly set
held_logs_max_keys = 50to preserve prior behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| bottlecap/src/config/mod.rs | Adds durable_function_log_buffer_size: usize field with doc and default 0. |
| bottlecap/src/config/env.rs | Adds env var deserialization, merge call, and updates default-config test. |
| bottlecap/src/config/yaml.rs | Updates yaml test expected struct with new field. |
| bottlecap/src/logs/lambda/processor.rs | Replaces constant with configurable cap; adds zero-disables fast path; updates tests. |
|
|
||
| let processing_rules = &datadog_config.logs_config_processing_rules; | ||
| let logs_enabled = datadog_config.serverless_logs_enabled; | ||
| let held_logs_max_keys = datadog_config.durable_function_log_buffer_size; |
There was a problem hiding this comment.
nit: rename held_logs_max_keys to durable_function_log_buffer_size or sth. similar for better readability.
| /// durable execution context enrichment. Useful when the tracer is not installed. | ||
| /// Default is `0`. | ||
| #[serde(deserialize_with = "deserialize_option_lossless")] | ||
| pub durable_function_log_buffer_size: Option<usize>, |
There was a problem hiding this comment.
Wondering if this should include the LAMBDA prefix, since we're gonna migrate to the config package from serverless-components eventually,
that way it makes it more descriptive that its for the lambda product
Background
For durable functions, the extension holds logs and waits to get the invocation context from the tracer so the extension can enrich the logs.
Problem
When tracing is off (which is not recommended for durable functions), logs are flushed when:
This PR
durable_function_log_buffer_sizeconfig field (env varDD_DURABLE_FUNCTION_LOG_BUFFER_SIZE, default:0) controlling the max number of request ID keys held inheld_logs.0, logs bypass the hold mechanism and are sent immediately without durable execution context enrichment.Test plan
Steps
Result
Before:
After: