From f2773b5d539bad954b75fb62dd00065af916cf16 Mon Sep 17 00:00:00 2001 From: Hiroshi Hatake Date: Wed, 8 Jul 2026 18:55:32 +0900 Subject: [PATCH 1/2] in_etw: Add a documentation for Event Tracing for Windows input plugin Signed-off-by: Hiroshi Hatake --- .gitbook.yaml | 1 + SUMMARY.md | 1 + pipeline/inputs/event-tracing-windows.md | 192 +++++++++++++++++++++++ 3 files changed, 194 insertions(+) create mode 100644 pipeline/inputs/event-tracing-windows.md diff --git a/.gitbook.yaml b/.gitbook.yaml index 43f895a5f..92c8c8afa 100644 --- a/.gitbook.yaml +++ b/.gitbook.yaml @@ -11,6 +11,7 @@ redirects: input/disk: ./pipeline/inputs/disk-io-metrics.md #inputs/docker: ./pipeline/inputs/ input/dummy: ./pipeline/inputs/dummy.md + input/event_tracing_windows: ./pipeline/inputs/event-tracing-windows.md input/elasticsearch: ./pipeline/inputs/elasticsearch.md input/exec: ./pipeline/inputs/exec.md input/fluentbit: ./pipeline/inputs/fluentbit-metrics.md diff --git a/SUMMARY.md b/SUMMARY.md index 18d8948ee..98ddfa9dd 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -121,6 +121,7 @@ * [TCP](pipeline/inputs/tcp.md) * [Thermal](pipeline/inputs/thermal.md) * [UDP](pipeline/inputs/udp.md) + * [Event Tracing for Windows](pipeline/inputs/event-tracing-windows.md) * [Windows Event logs (winlog)](pipeline/inputs/windows-event-log.md) * [Windows Event logs (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md) * [Windows exporter metrics](pipeline/inputs/windows-exporter-metrics.md) diff --git a/pipeline/inputs/event-tracing-windows.md b/pipeline/inputs/event-tracing-windows.md new file mode 100644 index 000000000..a8bedb0af --- /dev/null +++ b/pipeline/inputs/event-tracing-windows.md @@ -0,0 +1,192 @@ +# Event Tracing for Windows + +{% hint style="info" %} +**Supported event types:** `logs` +{% endhint %} + +The _Event Tracing for Windows_ (`event_tracing_windows`) input plugin collects real-time [Event Tracing for Windows](https://learn.microsoft.com/en-us/windows/win32/etw/about-event-tracing) (ETW) events. + +Use this plugin to define the Windows ETW provider you want to consume, similar to defining an arbitrary Windows Performance Counter. You can select a provider by name or GUID, tune the ETW level and keyword masks, and collect decoded event payload fields through Fluent Bit. + +{% hint style="info" %} +This plugin is only available on Windows operating systems. Some ETW providers and system logger sessions require administrative privileges. +{% endhint %} + +## Configuration parameters + +The plugin supports the following configuration parameters: + +| Key | Description | Default | +|------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------| +| `provider_guid` | ETW provider GUID to enable. Required for `provider` sessions unless `provider_name` is set. | _none_ | +| `provider_name` | ETW provider name to resolve and enable. Required for `provider` sessions unless `provider_guid` is set. | _none_ | +| `session_name` | ETW real-time session name. When `session_type` is `system` and this value is left as the default, the plugin uses `NT Kernel Logger`. | `fluent-bit-event-tracing-windows` | +| `session_type` | ETW session type. Use `provider` for a real-time provider consumer, or `system` for a Windows kernel logger session that uses `kernel_flags`. | `provider` | +| `stale_session_action` | Action when the ETW session already exists. Use `stop` to stop the existing session and retry, or `fail` to return an error without stopping it. | `stop` | +| `level` | ETW provider level. Valid values are `0` to `255`. | `5` | +| `match_any_keyword` | ETW `MatchAnyKeyword` mask. Decimal and hexadecimal values are accepted. | `0xffffffffffffffff` | +| `match_all_keyword` | ETW `MatchAllKeyword` mask. Decimal and hexadecimal values are accepted. | `0` | +| `kernel_flags` | Comma-separated kernel flags used with `session_type system`. Supported names are `process`, `thread`, `image_load`, `cswitch`, `tcpip`, `disk_io`. A numeric `EVENT_TRACE_FLAG_*` mask is also accepted. | `process,thread,image_load` | +| `buffer_size` | ETW session buffer size in kilobytes. Zero uses the Windows default. | `64` | +| `minimum_buffers` | Minimum number of ETW session buffers. Zero uses the Windows default. | `4` | +| `maximum_buffers` | Maximum number of ETW session buffers. Zero uses the Windows default. | `32` | +| `flush_timer` | ETW session flush timer in seconds. Zero uses the Windows default. | `1` | + +For `provider` sessions, set `provider_guid`, `provider_name`, or both. If both are set, the resolved provider name must match the configured GUID. + +For `system` sessions, don't set `provider_guid` or `provider_name`. Use `kernel_flags` to select Windows kernel events. + +The session buffer memory upper bound is `buffer_size` KB multiplied by `maximum_buffers`. With the defaults, the upper bound is `64` KB multiplied by `32`, or `2048` KB. + +## Event record fields + +Each ETW event is emitted as a log record with the ETW timestamp and the following fields: + +| Field | Description | +|-----------------------|-------------------------------------------------------| +| `provider_guid` | Provider GUID from the ETW event header. | +| `provider_name` | Configured provider name, or `null` if unset. | +| `event_id` | ETW event ID. | +| `version` | ETW event version. | +| `channel` | ETW event channel. | +| `level` | ETW event level. | +| `task` | ETW event task. | +| `opcode` | ETW event opcode. | +| `keywords` | ETW event keyword mask. | +| `process_id` | Process ID from the ETW event header. | +| `thread_id` | Thread ID from the ETW event header. | +| `activity_id` | ETW activity ID. | +| `related_activity_id` | Related activity ID when present, otherwise `null`. | +| `payload` | Decoded top-level ETW payload fields as key-value data. | + +## Configuration examples + +### Provider name + +The following example collects events from the `Microsoft-Windows-Kernel-Process` provider by resolving the provider name: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: event_tracing_windows + tag: etw.process + provider_name: Microsoft-Windows-Kernel-Process + level: 5 + match_any_keyword: 0xffffffffffffffff + + outputs: + - name: stdout + match: '*' +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + name event_tracing_windows + tag etw.process + provider_name Microsoft-Windows-Kernel-Process + level 5 + match_any_keyword 0xffffffffffffffff + +[OUTPUT] + name stdout + match * +``` + +{% endtab %} +{% endtabs %} + +### Provider GUID + +The following example selects a provider by GUID: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: event_tracing_windows + tag: etw.provider + provider_guid: '{22fb2cd6-0e7b-422b-a0c7-2fad1fd0e716}' + level: 5 + match_any_keyword: 0xffffffffffffffff + match_all_keyword: 0 + + outputs: + - name: stdout + match: '*' +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + name event_tracing_windows + tag etw.provider + provider_guid {22fb2cd6-0e7b-422b-a0c7-2fad1fd0e716} + level 5 + match_any_keyword 0xffffffffffffffff + match_all_keyword 0 + +[OUTPUT] + name stdout + match * +``` + +{% endtab %} +{% endtabs %} + +### Windows kernel events + +Use `session_type system` to collect Windows kernel logger events. The system session uses `kernel_flags` instead of a provider name or GUID: + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + inputs: + - name: event_tracing_windows + tag: etw.kernel + session_type: system + kernel_flags: process,thread,image_load + + outputs: + - name: stdout + match: '*' +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[INPUT] + name event_tracing_windows + tag etw.kernel + session_type system + kernel_flags process,thread,image_load + +[OUTPUT] + name stdout + match * +``` + +{% endtab %} +{% endtabs %} + +## Requirements and permissions + +The Event Tracing for Windows input plugin uses Windows ETW real-time sessions and Trace Data Helper (TDH) APIs. The following requirements apply: + +- **Operating system**: Windows only. +- **Provider availability**: The configured `provider_name` must exist in the registered ETW providers on the host. If you use `provider_guid`, the GUID must be a valid ETW provider GUID. +- **Permissions**: The Fluent Bit process must have permission to start the ETW session and enable the selected provider. System logger sessions and some providers require running Fluent Bit as an administrator. + +If startup fails with an access error, run Fluent Bit with administrator privileges or use a service account that has permission to create and consume the selected ETW session. From 79b7fb479d9c115511e2893584fca1735b2291a9 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Fri, 10 Jul 2026 14:10:48 +0200 Subject: [PATCH 2/2] docs: pipeline: inputs: event-tracing-windows: fix Vale lint suggestions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Clean up Vale suggestions on the new Event Tracing for Windows input plugin doc: - Use sentence-style capitalization for the page title and sync matching SUMMARY.md navigation entry. - Wrap bare "GUID" references in backticks so the FluentBit.Acronyms rule no longer flags them and table alignments - Classic config keys → Title_Case Applies to #2618 Signed-off-by: Eric D. Schabell --- SUMMARY.md | 2 +- pipeline/inputs/event-tracing-windows.md | 118 +++++++++++------------ 2 files changed, 60 insertions(+), 60 deletions(-) diff --git a/SUMMARY.md b/SUMMARY.md index 98ddfa9dd..cafaf1057 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -121,7 +121,7 @@ * [TCP](pipeline/inputs/tcp.md) * [Thermal](pipeline/inputs/thermal.md) * [UDP](pipeline/inputs/udp.md) - * [Event Tracing for Windows](pipeline/inputs/event-tracing-windows.md) + * [Event tracing for Windows](pipeline/inputs/event-tracing-windows.md) * [Windows Event logs (winlog)](pipeline/inputs/windows-event-log.md) * [Windows Event logs (winevtlog)](pipeline/inputs/windows-event-log-winevtlog.md) * [Windows exporter metrics](pipeline/inputs/windows-exporter-metrics.md) diff --git a/pipeline/inputs/event-tracing-windows.md b/pipeline/inputs/event-tracing-windows.md index a8bedb0af..6c8beaa1b 100644 --- a/pipeline/inputs/event-tracing-windows.md +++ b/pipeline/inputs/event-tracing-windows.md @@ -1,4 +1,4 @@ -# Event Tracing for Windows +# Event tracing for Windows {% hint style="info" %} **Supported event types:** `logs` @@ -6,7 +6,7 @@ The _Event Tracing for Windows_ (`event_tracing_windows`) input plugin collects real-time [Event Tracing for Windows](https://learn.microsoft.com/en-us/windows/win32/etw/about-event-tracing) (ETW) events. -Use this plugin to define the Windows ETW provider you want to consume, similar to defining an arbitrary Windows Performance Counter. You can select a provider by name or GUID, tune the ETW level and keyword masks, and collect decoded event payload fields through Fluent Bit. +Use this plugin to define the Windows ETW provider you want to consume, similar to defining an arbitrary Windows Performance Counter. You can select a provider by name or `GUID`, tune the ETW level and keyword masks, and collect decoded event payload fields through Fluent Bit. {% hint style="info" %} This plugin is only available on Windows operating systems. Some ETW providers and system logger sessions require administrative privileges. @@ -16,23 +16,23 @@ This plugin is only available on Windows operating systems. Some ETW providers a The plugin supports the following configuration parameters: -| Key | Description | Default | -|------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------| -| `provider_guid` | ETW provider GUID to enable. Required for `provider` sessions unless `provider_name` is set. | _none_ | -| `provider_name` | ETW provider name to resolve and enable. Required for `provider` sessions unless `provider_guid` is set. | _none_ | -| `session_name` | ETW real-time session name. When `session_type` is `system` and this value is left as the default, the plugin uses `NT Kernel Logger`. | `fluent-bit-event-tracing-windows` | -| `session_type` | ETW session type. Use `provider` for a real-time provider consumer, or `system` for a Windows kernel logger session that uses `kernel_flags`. | `provider` | -| `stale_session_action` | Action when the ETW session already exists. Use `stop` to stop the existing session and retry, or `fail` to return an error without stopping it. | `stop` | -| `level` | ETW provider level. Valid values are `0` to `255`. | `5` | -| `match_any_keyword` | ETW `MatchAnyKeyword` mask. Decimal and hexadecimal values are accepted. | `0xffffffffffffffff` | -| `match_all_keyword` | ETW `MatchAllKeyword` mask. Decimal and hexadecimal values are accepted. | `0` | -| `kernel_flags` | Comma-separated kernel flags used with `session_type system`. Supported names are `process`, `thread`, `image_load`, `cswitch`, `tcpip`, `disk_io`. A numeric `EVENT_TRACE_FLAG_*` mask is also accepted. | `process,thread,image_load` | -| `buffer_size` | ETW session buffer size in kilobytes. Zero uses the Windows default. | `64` | -| `minimum_buffers` | Minimum number of ETW session buffers. Zero uses the Windows default. | `4` | -| `maximum_buffers` | Maximum number of ETW session buffers. Zero uses the Windows default. | `32` | -| `flush_timer` | ETW session flush timer in seconds. Zero uses the Windows default. | `1` | - -For `provider` sessions, set `provider_guid`, `provider_name`, or both. If both are set, the resolved provider name must match the configured GUID. +| Key | Description | Default | +| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- | +| `provider_guid` | ETW provider `GUID` to enable. Required for `provider` sessions unless `provider_name` is set. | _none_ | +| `provider_name` | ETW provider name to resolve and enable. Required for `provider` sessions unless `provider_guid` is set. | _none_ | +| `session_name` | ETW real-time session name. When `session_type` is `system` and this value is left as the default, the plugin uses `NT Kernel Logger`. | `fluent-bit-event-tracing-windows` | +| `session_type` | ETW session type. Use `provider` for a real-time provider consumer, or `system` for a Windows kernel logger session that uses `kernel_flags`. | `provider` | +| `stale_session_action` | Action when the ETW session already exists. Use `stop` to stop the existing session and retry, or `fail` to return an error without stopping it. | `stop` | +| `level` | ETW provider level. Valid values are `0` to `255`. | `5` | +| `match_any_keyword` | ETW `MatchAnyKeyword` mask. Decimal and hexadecimal values are accepted. | `0xffffffffffffffff` | +| `match_all_keyword` | ETW `MatchAllKeyword` mask. Decimal and hexadecimal values are accepted. | `0` | +| `kernel_flags` | Comma-separated kernel flags used with `session_type system`. Supported names are `process`, `thread`, `image_load`, `cswitch`, `tcpip`, `disk_io`. A numeric `EVENT_TRACE_FLAG_*` mask is also accepted. | `process,thread,image_load` | +| `buffer_size` | ETW session buffer size in kilobytes. Zero uses the Windows default. | `64` | +| `minimum_buffers` | Minimum number of ETW session buffers. Zero uses the Windows default. | `4` | +| `maximum_buffers` | Maximum number of ETW session buffers. Zero uses the Windows default. | `32` | +| `flush_timer` | ETW session flush timer in seconds. Zero uses the Windows default. | `1` | + +For `provider` sessions, set `provider_guid`, `provider_name`, or both. If both are set, the resolved provider name must match the configured `GUID`. For `system` sessions, don't set `provider_guid` or `provider_name`. Use `kernel_flags` to select Windows kernel events. @@ -42,21 +42,21 @@ The session buffer memory upper bound is `buffer_size` KB multiplied by `maximum Each ETW event is emitted as a log record with the ETW timestamp and the following fields: -| Field | Description | -|-----------------------|-------------------------------------------------------| -| `provider_guid` | Provider GUID from the ETW event header. | -| `provider_name` | Configured provider name, or `null` if unset. | -| `event_id` | ETW event ID. | -| `version` | ETW event version. | -| `channel` | ETW event channel. | -| `level` | ETW event level. | -| `task` | ETW event task. | -| `opcode` | ETW event opcode. | -| `keywords` | ETW event keyword mask. | -| `process_id` | Process ID from the ETW event header. | -| `thread_id` | Thread ID from the ETW event header. | -| `activity_id` | ETW activity ID. | -| `related_activity_id` | Related activity ID when present, otherwise `null`. | +| Field | Description | +| --------------------- | ------------------------------------------------------- | +| `provider_guid` | Provider `GUID` from the ETW event header. | +| `provider_name` | Configured provider name, or `null` if unset. | +| `event_id` | ETW event ID. | +| `version` | ETW event version. | +| `channel` | ETW event channel. | +| `level` | ETW event level. | +| `task` | ETW event task. | +| `opcode` | ETW event opcode. | +| `keywords` | ETW event keyword mask. | +| `process_id` | Process ID from the ETW event header. | +| `thread_id` | Thread ID from the ETW event header. | +| `activity_id` | ETW activity ID. | +| `related_activity_id` | Related activity ID when present, otherwise `null`. | | `payload` | Decoded top-level ETW payload fields as key-value data. | ## Configuration examples @@ -87,23 +87,23 @@ pipeline: ```text [INPUT] - name event_tracing_windows - tag etw.process - provider_name Microsoft-Windows-Kernel-Process - level 5 - match_any_keyword 0xffffffffffffffff + Name event_tracing_windows + Tag etw.process + Provider_Name Microsoft-Windows-Kernel-Process + Level 5 + Match_Any_Keyword 0xffffffffffffffff [OUTPUT] - name stdout - match * + Name stdout + Match * ``` {% endtab %} {% endtabs %} -### Provider GUID +### Provider `GUID` -The following example selects a provider by GUID: +The following example selects a provider by `GUID`: {% tabs %} {% tab title="fluent-bit.yaml" %} @@ -128,16 +128,16 @@ pipeline: ```text [INPUT] - name event_tracing_windows - tag etw.provider - provider_guid {22fb2cd6-0e7b-422b-a0c7-2fad1fd0e716} - level 5 - match_any_keyword 0xffffffffffffffff - match_all_keyword 0 + Name event_tracing_windows + Tag etw.provider + Provider_Guid {22fb2cd6-0e7b-422b-a0c7-2fad1fd0e716} + Level 5 + Match_Any_Keyword 0xffffffffffffffff + Match_All_Keyword 0 [OUTPUT] - name stdout - match * + Name stdout + Match * ``` {% endtab %} @@ -145,7 +145,7 @@ pipeline: ### Windows kernel events -Use `session_type system` to collect Windows kernel logger events. The system session uses `kernel_flags` instead of a provider name or GUID: +Use `session_type system` to collect Windows kernel logger events. The system session uses `kernel_flags` instead of a provider name or `GUID`: {% tabs %} {% tab title="fluent-bit.yaml" %} @@ -168,14 +168,14 @@ pipeline: ```text [INPUT] - name event_tracing_windows - tag etw.kernel - session_type system - kernel_flags process,thread,image_load + Name event_tracing_windows + Tag etw.kernel + Session_Type system + Kernel_Flags process,thread,image_load [OUTPUT] - name stdout - match * + Name stdout + Match * ``` {% endtab %} @@ -186,7 +186,7 @@ pipeline: The Event Tracing for Windows input plugin uses Windows ETW real-time sessions and Trace Data Helper (TDH) APIs. The following requirements apply: - **Operating system**: Windows only. -- **Provider availability**: The configured `provider_name` must exist in the registered ETW providers on the host. If you use `provider_guid`, the GUID must be a valid ETW provider GUID. +- **Provider availability**: The configured `provider_name` must exist in the registered ETW providers on the host. If you use `provider_guid`, the `GUID` must be a valid ETW provider `GUID`. - **Permissions**: The Fluent Bit process must have permission to start the ETW session and enable the selected provider. System logger sessions and some providers require running Fluent Bit as an administrator. If startup fails with an access error, run Fluent Bit with administrator privileges or use a service account that has permission to create and consume the selected ETW session.