kv: add flb_kv_get_all_key_values function

[iypetrov]

Add flb_kv_get_all_key_values function for mk_list.

Addresses #11776 (first and second sub-tasks mentioned in the issue)

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change
• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Run local packaging test showing all targets (including any new ones) build.
• Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **New Features**
  * Added bulk key/value retrieval APIs that return all items as a caller-managed array, provide the total via an out-parameter, and return `NULL` for empty/invalid input.
  * Exposed the same bulk retrieval capability through the configuration and output public interfaces, including wiring into the main API function table.

* **Tests**
  * Added unit tests covering empty vs. populated lists, validating returned counts and the expected key/value entries in order.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

This pull request adds a bulk retrieval API for key-value entries and output properties. It introduces flb_kv_get_all_key_values() to return all entries from a mk_list as a heap-allocated array with count reporting, provides config and output convenience wrappers, registers the capability in the public API struct, and includes comprehensive unit tests.

Changes

Property Bulk Retrieval API

Layer / File(s)	Summary
Core KV Array Function include/fluent-bit/flb_kv.h, src/flb_kv.c, tests/internal/kv.c	flb_kv_get_all_key_values() allocates and returns an array of flb_kv pointers from a mk_list, reporting count via int *out_count. Includes unit test validating empty list (NULL, count==0) and populated list (3 entries with expected keys/values).
Config and Output Wrappers include/fluent-bit/flb_config.h, src/flb_config.c, include/fluent-bit/flb_output.h, src/flb_output.c	flb_config_prop_get_all() and flb_output_get_all_properties() delegate to flb_kv_get_all_key_values(), providing higher-level convenience functions for config properties and output instance properties.
Public API Struct Registration include/fluent-bit/flb_api.h, src/flb_api.c	New output_get_all_properties function-pointer field added to struct flb_api and initialized in flb_api_create().

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Keys cascade in ordered rows,
All properties the function knows,
From lists both shallow, wide, and tall,
Count them right and sum them all,
Bulk retrieval hops with glee!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 27.27% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title accurately and concisely summarizes the main change: adding a new function flb_kv_get_all_key_values to retrieve all key/value elements from a list.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@include/fluent-bit/flb_kv.h`:
- Line 44: flb_kv_get_all_key_values currently returns a pointer array without
length metadata, making callers unsafe to iterate; modify its signature to
include an output count (e.g., add a size_t *out_count parameter) or change the
return to a small struct { struct flb_kv **items; size_t count; } so callers can
know the array length; update the function implementation and all call sites
that use flb_kv_get_all_key_values to set/read the count and to free/iterate
correctly (search for references to flb_kv_get_all_key_values to update usage).

In `@tests/internal/kv.c`:
- Around line 49-61: The test accesses pairs[0..2] without ensuring the
underlying list contains three items; change the check so after verifying pairs
is non-NULL you also assert mk_list_size(&list) == 3 (or equivalent size check
against the list used to build pairs) before indexing pairs[0], pairs[1], and
pairs[2]; reference the list used when calling flb_kv_item_set() and use that
size guard so the subsequent TEST_CHECK(strcmp(...)) validations only run when
the list actually has three entries.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 96ef314f-ede6-4b61-9703-2e217b4ad117

📥 Commits

Reviewing files that changed from the base of the PR and between 7299905 and 24549d4.

📒 Files selected for processing (3)

• include/fluent-bit/flb_kv.h
• src/flb_kv.c
• tests/internal/kv.c

[coderabbitai]

🧹 Nitpick comments (1)

tests/internal/kv.c (1)

44-46: ⚡ Quick win

Check each flb_kv_item_set result explicitly

Please assert each insertion succeeds, so failures are reported at the source rather than only via downstream count/content checks.

Suggested patch

-    flb_kv_item_set(&list, "host", "localhost");
-    flb_kv_item_set(&list, "port", "8080");
-    flb_kv_item_set(&list, "path", "/api");
+    TEST_CHECK(flb_kv_item_set(&list, "host", "localhost") != NULL);
+    TEST_CHECK(flb_kv_item_set(&list, "port", "8080") != NULL);
+    TEST_CHECK(flb_kv_item_set(&list, "path", "/api") != NULL);

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/internal/kv.c` around lines 44 - 46, Each flb_kv_item_set call should
have its return value checked and asserted immediately so insertion failures are
reported at the source; update the three calls to flb_kv_item_set(&list, "host",
"localhost"), flb_kv_item_set(&list, "port", "8080"), and flb_kv_item_set(&list,
"path", "/api") to capture their returns into distinct variables (e.g.,
ret_host/ret_port/ret_path) and add assertions that each return indicates
success (match the test framework's success condition) before proceeding to
later count/content checks.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/internal/kv.c`:
- Around line 44-46: Each flb_kv_item_set call should have its return value
checked and asserted immediately so insertion failures are reported at the
source; update the three calls to flb_kv_item_set(&list, "host", "localhost"),
flb_kv_item_set(&list, "port", "8080"), and flb_kv_item_set(&list, "path",
"/api") to capture their returns into distinct variables (e.g.,
ret_host/ret_port/ret_path) and add assertions that each return indicates
success (match the test framework's success condition) before proceeding to
later count/content checks.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2ccc1540-93cc-41e1-98ac-1dd74658090e

📥 Commits

Reviewing files that changed from the base of the PR and between dd48761 and ff54e1a.

📒 Files selected for processing (1)

• tests/internal/kv.c

[cosmo0920]

Looks good to me.

[iypetrov]

hey @edsiper, do you think this one is ready to be merged?

[edsiper]

hey @edsiper, do you think this one is ready to be merged?

Why do we need this PR?

[iypetrov]

Why do we need this PR?

The main issue is that as a Fluent Bit Go plugin creator currently I have no straightforward way to retrieve all available plugin configuration fields through a single API. Instead, I must manually query individual fields using flb_kv_get_key_value, while also guessing the exact field naming and format (classic, yaml), for example. This PR extends the KV functionality to provide a native mechanism for retrieving all plugin configuration fields in a unified way. As follow-up work, I plan to:

• Extend the Fluent Bit C API to expose this functionality.
• Add support in fluent-bit-go to make plugin configuration access significantly easier for Go plugin authors.

More context is available here.

[iypetrov]

@edsiper wdyt?

[iypetrov]

@edsiper if you want I can merge sub-tasks 1 & 2 from this issue #11776?

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/flb_config.c`:
- Around line 679-682: Add a documentation comment block above the function
`flb_config_prop_get_all` that clearly specifies the memory ownership model and
cleanup responsibility. The comment should document that the returned array is
heap-allocated by `flb_kv_get_all_key_values()`, explain that the caller is
responsible for freeing it using `flb_free()`, describe the return value as a
heap-allocated array of `flb_kv` pointers that may be NULL if the list is empty
or NULL, and clarify that the `out_count` parameter receives the element count
and is set to 0 if a NULL array is returned. This documentation is critical for
API consumers, particularly Go plugin authors, to prevent memory leaks.

In `@src/flb_output.c`:
- Around line 1182-1186: The function flb_output_get_all_properties lacks
documentation about memory ownership and cleanup requirements, which is critical
since it is exposed via the public API in flb_api.h for external plugin
developers. Add a documentation comment above the function definition that
clearly specifies: the returned heap-allocated array must be freed by the caller
using flb_free(), the function returns NULL if the output instance has no
properties, and the out_count parameter receives the element count of the
returned array. Follow the same documentation pattern as
flb_config_prop_get_all() since flb_output_get_all_properties delegates to that
function.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: eaa4c41e-ad6b-435a-9af0-553fee8017a7

📥 Commits

Reviewing files that changed from the base of the PR and between ff54e1a and 10773b2.

📒 Files selected for processing (6)

• include/fluent-bit/flb_api.h
• include/fluent-bit/flb_config.h
• include/fluent-bit/flb_output.h
• src/flb_api.c
• src/flb_config.c
• src/flb_output.c

[iypetrov]

@cosmo0920 @edsiper I included the flb_kv_get_all_key_values to the FLB C API, so to be cleaner why this change is needed. So this PR combines 1 and 2 sub-tasks of the mentioned issue.

[iypetrov]

@cosmo0920 @edsiper wdyt?

[edsiper]

Why do we need this PR?

The main issue is that as a Fluent Bit Go plugin creator currently I have no straightforward way to retrieve all available plugin configuration fields through a single API. Instead, I must manually query individual fields using flb_kv_get_key_value, while also guessing the exact field naming and format (classic, yaml), for example. This PR extends the KV functionality to provide a native mechanism for retrieving all plugin configuration fields in a unified way. As follow-up work, I plan to:

• Extend the Fluent Bit C API to expose this functionality.
• Add support in fluent-bit-go to make plugin configuration access significantly easier for Go plugin authors.

More context is available here.

should we implement instead a way for Golang plugins to register a config map ? I know this is a simple workaround but how do we handle config options which are mistyped ? (actually thats why we implemented config maps) cc: @cosmo0920

[iypetrov]

should we implement instead a way for Golang plugins to register a config map ? I know this is a simple workaround but how do we handle config options which are mistyped ? (actually thats why we implemented config maps) cc: @cosmo0920

If I understand it correctly, the idea is to extend FLBPluginRegister to accept a schema (the same shape as struct flb_config_map[] that native C plugins declare) so the proxy can assign it to plugin->config_map and from there the existing verification with flb_config_map_properties_check() to validate the input from the config, right?

[edsiper]

should we implement instead a way for Golang plugins to register a config map ? I know this is a simple workaround but how do we handle config options which are mistyped ? (actually thats why we implemented config maps) cc: @cosmo0920

If I understand it correctly, the idea is to extend FLBPluginRegister to accept a schema (the same shape as struct flb_config_map[] that native C plugins declare) so the proxy can assign it to plugin->config_map and from there the existing verification with flb_config_map_properties_check() to validate the input from the config, right?

yes, a way for the golang developer to register a config map from Go and get exposed to fluent bit, so fluent bit will not allow to run it if an unknown property is passed by mistake

[iypetrov]

yes, a way for the golang developer to register a config map from Go and get exposed to fluent bit, so fluent bit will not allow to run it if an unknown property is passed by mistake

@cosmo0920 @edsiper I created this PR #12058 as follow-up to this discussion, when you have time can give it a look