s3: document format=parquet option and page-level compression

[rituparnakhaund]

Update S3 output plugin documentation to reflect the new format=parquet option that separates output format selection from byte-level compression.

Documents:

• New parquet value for the format option
• Page-level compression codec control via compression when format is parquet
• Migration path from deprecated compression=parquet syntax
• Configuration examples with and without page-level compression
• Updated existing parquet examples to use new syntax

Related code PR: fluent/fluent-bit#11885

Summary by CodeRabbit

• Documentation
  • Updated S3 output docs with expanded compression options (including zstd and arrow) and clarified how format affects Parquet behavior.
  • Added a dedicated Parquet section covering Parquet output, page-codec mappings, multipart/upload requirements, example configurations, and migration from deprecated compression=parquet.
  • Clarified Arrow guidance and revised Parquet testing examples to use format: parquet with compression: snappy.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR updates S3 output documentation for Parquet configuration, compression codecs, upload requirements, migration from deprecated settings, Arrow output clarification, and testing examples.

Changes

S3 Output Parquet Format Documentation

Layer / File(s)	Summary
Compression and format parameter documentation pipeline/outputs/s3.md	The parameter table documents zstd and arrow compression values, adds parquet to format, and explains Parquet page codec selection and deprecated compression=parquet.
Parquet format section, examples, and format guidance pipeline/outputs/s3.md	Adds Parquet codec mappings, use_put_object requirements, configuration and migration examples, build requirements, clarified Arrow guidance, and updated YAML and Fluent Bit test configurations.

Estimated code review effort: 2 (Simple) | ~8 minutes

Possibly related PRs

• fluent/fluent-bit-docs#2111: Updates overlapping S3 Parquet documentation, codec mappings, requirements, and examples.
• fluent/fluent-bit-docs#2359: Updates overlapping S3 compression and Parquet/Arrow option documentation.
• fluent/fluent-bit-docs#2570: Updates the same S3 format documentation area, including Parquet-related behavior.

Suggested labels: 5.0.5

Suggested reviewers: patrick-stephens, cosmo0920

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: documenting Parquet format support and page-level compression for the S3 output plugin.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[eschabell]

@rituparnakhaund thanks for the docs PR, I pushed a fix for vale issues and now just waiting on code PR merging.

[eschabell]

Code PR merged, fixed small conflict.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)

pipeline/outputs/s3.md (2)

194-199: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Include the required use_put_object setting in the migration example.

The recommended block is incomplete: Parquet requires use_put_object: on, but users replacing only compression: parquet will retain the unsupported multipart-upload mode.

 **After (recommended):**
 
 ```yaml
+use_put_object: on
 format: parquet
 compression: snappy

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @pipeline/outputs/s3.md around lines 194 - 199, Update the recommended YAML
migration example in the “After” block to include the required use_put_object:
on setting alongside format and compression, ensuring Parquet output uses
PUT-object uploads.

</details>

<!-- cr-comment:v1:8fa744ff363a38676b7b83b9 -->

---

`49-49`: _🗄️ Data Integrity & Integration_ | _🟡 Minor_ | _⚡ Quick win_

**Clarify the Parquet compression contract.**

The table omits `uncompressed`, despite the documented uncompressed-page example. It also conflicts with the generic statement that S3 objects are compressed before upload; for Parquet, `compression` selects the page codec rather than an outer object-compression layer. Include `uncompressed` and explicitly qualify the generic compression guidance.

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @pipeline/outputs/s3.md at line 49, Update the compression entry in the S3
outputs documentation to include uncompressed as a supported Parquet page codec,
and clarify that for format parquet the compression value controls the internal
page codec rather than compressing the uploaded S3 object; qualify the generic
object-compression guidance accordingly.

</details>

<!-- cr-comment:v1:115e1aa91ef03949a41bc759 -->

</blockquote></details>

</blockquote></details>

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@pipeline/outputs/s3.md`:
- Around line 194-199: Update the recommended YAML migration example in the
“After” block to include the required use_put_object: on setting alongside
format and compression, ensuring Parquet output uses PUT-object uploads.
- Line 49: Update the compression entry in the S3 outputs documentation to
include uncompressed as a supported Parquet page codec, and clarify that for
format parquet the compression value controls the internal page codec rather
than compressing the uploaded S3 object; qualify the generic object-compression
guidance accordingly.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 6b9664b9-ebee-43ac-b01f-bea86a5aa5ba

📥 Commits

Reviewing files that changed from the base of the PR and between 546958f and 076b3a7.

📒 Files selected for processing (1)

• pipeline/outputs/s3.md