CEF message is always in 'raw' field

[ualser]

I am new to fluentd configuration. I have spent quite some time now get this going. My goal is to parse following CEF into separate fields as json. Message always ends up in 'raw' field. As I understand it is a default behavior when message is not parsed due to some error. In the end I need all fields like agentID as key value pairs in json. Please assist.

My CEF example:
CEF:0|TEST|HTTPTransactionHandler|ololo|Event Driven|Web Request|2|eventTime=2025-03-29T09:17:12.660 agentId="TEST" eventTraceId="test" eventSpanId="test"

My config example:

<source>
  @type udp
  port 5140
  message_length_limit 10000
  tag waf
  format cef
</source>

<match **>
  @type  stdout
  <format>
    @type json
  </format>
</match>

[sparkmark]

I'm having the same issue. No matter what I try I'm not able to get the CEF parsed. It always ends up as RAW=...

Some sample logs I'm testing with:

<164>9 Apr 10:10:10 testhost1 CEF:0|testVendor|testProduct|testVersion|testDECI|testName|1|msg=test src=1.1.1.1 act=blocked dst=2.2.2.2 dpt=12345
<164>CEF:0|testVendor|testProduct|testVersion|testDECI|testName|1|msg=test src=1.1.1.1 act=blocked dst=2.2.2.2 dpt=12345
<164>1 2024-10-02T14:35:05+02:00 testfw1 - - - - CEF:0|testVendor|testProduct|testVersion|testDECI|testName|1|msg=test src=1.1.1.1 act=blocked dst=2.2.2.2 dpt=12345

I've tried all sorts of configurations as well. Like accepting the data via syslog then using a filter to do the parsing, to make sure it just gets the message field, all with the same result.

In its simplest form, here is my config:

<source>
  @type syslog
  port 5514
  bind 0.0.0.0
  <transport tcp>
  </transport>
  <parse>
    @type cef
    log_format syslog
    parse_strict_mode true
    output_raw_field false
  </parse>
  tag ceflog
</source>
<match ceflog.**>
  @type stdout
</match>

This is my actual config (which was working about 1 year ago)

<source>
  @type syslog
  port 5514
  bind 0.0.0.0
  <transport tcp>
  </transport>
  <parse>
    @type cef
    log_format syslog
    parse_strict_mode true
    output_raw_field false
  </parse>
  tag ceflog
</source>
<filter ceflog.**>
  @type record_transformer
  enable_ruby
  <record>
    in ${record["in"].to_i}
    out ${record["out"].to_i}
  </record>
</filter>
<filter ceflog.**>
  @type aggregate
  intervals 5s
  keep_interval 1s
  group_fields cef_name, src, dst, dpt, act
  aggregate_fields in, out 
  aggregations sum
  aggregate_event_tag aggtest
</filter>
<filter aggtest.**>
  @type record_transformer
  <record>
    baseEventCount 1
    aggregatedEventCount ${record["processed"]}
    name ${record["cef_name"]}
    in ${record["in_sum"]}
    out ${record["out_sum"]}
  </record>
  remove_keys processed, aggregator_id, in_sum, out_sum, cef_name
</filter>
<match aggtest.**>
  @type azureeventhubs_buffered
  connection_string Endpoint=myendpoint
  hub_name test hub
  include_tag false
  type https
</match>
<match ceflog.**>
  @type null
</match>

Out of interest, this is version that got installed of the plugin:
gem 'fluent-plugin-parser_cef' version '0.2.0'

[sparkmark]

Also just to add I tried "gem update fluent-plugin-parser_cef" and I get the following error:

ERROR:  Error installing fluent-plugin-parser_cef:
	fluent-plugin-parser_cef-1.0.0 requires Ruby version ~> 2.1. The current ruby version is 3.0.7.220.

In the docs it say 2.1 or higher, which i have.

[sparkmark]

For anyone else running into this its because the parser does not yet support RFC5424. There is an outstanding pull request from 2020 to have this added. In the end I cloned the repo made my own changes to fix the issues I needed and pushed the gem to rubygems so I can use it now.