Skip to content

jmx-metrics add ability to filter/include metrics by target system and metric maturity #19245

Description

@SylvainJuge

This is part of #19227

This issue describes the implementation proposal of what is described here.

To capture JMX metrics, users currently have to provide one (or both) of the following:

  • their custom definitions in YAML using otel.jmx.config
  • a list of the systems they want to monitor using otel.jmx.target.system

As the 3.0 release is quite close, we can't realistically deprecate and remove the otel.jmx.target.system so it would be wise to preserve it for now.

Implementation proposal

Replace the requirement to have at least one of otel.jmx.target.system or otel.jmx.config with "at least one JMX rule is loaded" (which is equivalent).

I would suggest to add the two following options:

  • otel.jmx.auto.enable defaults to true
  • otel.jmx.auto.maturity defaults to stable` (list of values, with support of wildcards)

The otel.jmx.auto.enable would enable the ability to load all the embedded rules and when the MBeans are available, effectively removing the requirement to set otel.jmx.target.system or providing a custom config.

The otel.jmx.auto.maturity would limit the auto-detection of embedded metrics that have a matching maturity.

This is slightly different of what initially motivated #16016, where the goal was to only capture stable metrics by default, however:

  • if we filter for only stable metrics by default, then it will break existing users if their yaml does not have any stability defined, also it might force us to consider "metrics without stability" as stable metrics, which would definitely be confusing.
  • short-term, only the JVM metrics are stable, and with instrumentation those are not used, when using jmx-scraper it would allow to capture JVM metrics for any remote JVM, this is consistent with the "enable stable metrics by default"
  • this allows to configure otel.jmx.auto.maturity=* to capture all possible embedded metrics, hence removing the need to use otel.jmx.target.system
  • whenever new metrics are promoted to stable, they would be automatically captured by users when updating to a newer version of the library.

In the metrics definition yaml, the only thing we have to add is the ability to define an optional maturity. While this currently duplicates what is being defined in the JMX metrics registry, this is a compromise until the duplication between those files is reduced (allowing to use registry format directly would be a massive refactor).

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions