feat(logging): POC of logger rate limiting

[dehaansa]

Brief description of Pull Request

Proof-of-concept adding log rate limiting to Alloy's logger, backed by github.com/samber/slog-sampling (Threshold sampler: admit the first N identical lines per tick, then drop the rest). Logs are evaluated for uniqueness by component, message, and level.

• New rate_limiting block in the logging config block: enabled, tick, threshold, rate, max_signatures. Defaults: enabled=true, tick="10s", threshold=10, rate=0, max_signatures=1000 (LRU-bounded). Omitting the block keeps limiting enabled with defaults; enabled=false disables it.
• Keys on component identity + log level + message text. Identical repeated lines from the same component are thinned per tick; distinct components/messages are independent. Empty-message records (e.g. keyless go-kit logs) are not rate limited.
• Dropped lines are counted by a new alloy_logging_suppressed_lines_total{level, component_id} metric; the first admitted line of each window carries a slog_sampling.dropped_count attribute.
• Adds dependencies: samber/slog-sampling, samber/slog-multi, samber/slog-common, bluele/gcache, cornelk/hashmap.

Note: this is on by default at the moment.

Pull Request Details

The Alloy team's seen a few different situations where Alloy repeated logging of authentication errors or errors triggered by clock skew cause significant disk usage, as well as reduce the visibility of other errors due to the noise. While log rotation is one part of the solution, causing the logger to deduplicate repeated log messages is also an important aspect. This POC implements rate limiting for deduplication.

PR Checklist

• Documentation added
• Tests updated
• Config converters updated
• This pull request was substantially generated with AI assistance (see the GenAI policy)

[github-actions]

Grafana.GoogleLyHyphens (error)

'recently-used' doesn't need a hyphen.

For more information, refer to Google developer documentation style guide.

---

Reported by Vale using Grafana Writers' Toolkit style. If you believe we can improve the rule, report an issue.

[github-actions]

Grafana.GoogleEnDash (error)

Use an em dash ('—') instead of '–'.

For more information, refer to Google developer documentation style guide.

---

Reported by Vale using Grafana Writers' Toolkit style. If you believe we can improve the rule, report an issue.

[github-actions]

💻 Deploy preview available (feat(logging): POC of logger rate limiting):

• https://deploy-preview-alloy-6813-zb444pucvq-vp.a.run.app/docs/alloy/next/

[Copilot]

Pull request overview

Adds an on-by-default, per-(component, level, message) log rate-limiter to Alloy’s runtime logger using github.com/samber/slog-sampling, with live-reconfigurable options and a Prometheus metric for suppressed lines.

Changes:

• Introduces logging.rate_limiting config block + defaults/validation, and wires it through logger updates.
• Adds a sampling injector handler layer to key and apply per-component sampling without a background goroutine.
• Adds tests/benchmarks plus documentation and a new alloy_logging_suppressed_lines_total{level,component_id} metric.

Reviewed changes

Copilot reviewed 16 out of 18 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
internal/util/metricsutil/metricsutil.go	New leaf helper package for Prometheus registration without internal/util import cycles.
internal/util/metrics.go	Removes the moved MustRegisterOrReturnExisting helper from internal/util.
internal/runtime/logging/sampling.go	Implements signature matching, sampling injector handler, and suppressed-lines metric integration.
internal/runtime/logging/sampling_test.go	Unit tests for matcher/component sniffing and injector behavior.
internal/runtime/logging/rl_bench_test.go	Benchmarks for rate limiting overhead (disabled/admit/drop/churn/parallel).
internal/runtime/logging/options.go	Adds RateLimitingOptions block, defaults, and validation.
internal/runtime/logging/options_test.go	Tests for default rate limiting enablement and validation behavior.
internal/runtime/logging/logger.go	Wires rate limiting into Logger lifecycle with atomic root handler swapping and metric init.
internal/runtime/logging/logger_rl_test.go	End-to-end tests for sampling, budgets across updates, context variants, metrics init, and concurrency.
internal/runtime/logging/logger_event_log_test.go	Disables rate limiting in a delivery-count-sensitive destination flip test.
internal/runtime/logging/deferred_handler.go	Routes the root of the deferred handler tree through the persistent sampling injector.
internal/runtime/internal/controller/node_config_logging.go	Initializes rate-limit metrics registration when logging config nodes are created.
internal/runtime/internal/controller/loader.go	Switches to metricsutil.MustRegisterOrReturnExisting for loader metric registration.
go.mod	Adds github.com/samber/slog-sampling (and indirect deps) to the root module.
go.sum	Records checksums for newly introduced root-module dependencies.
docs/sources/reference/config-blocks/logging.md	Documents the new logging.rate_limiting block and its semantics/metric.
collector/go.mod	Updates generated collector module dependencies (appears incidental to this feature).
collector/go.sum	Updates generated collector module sums (appears incidental to this feature).

[Copilot]

Pull request overview

Copilot reviewed 16 out of 18 changed files in this pull request and generated no new comments.

Suppressed comments (1)

internal/runtime/logging/logger_rl_test.go:241

• require.NoError(t, err) is called from a goroutine. require uses t.FailNow, which only stops the current goroutine and can lead to silently passing tests and/or unsafe test behavior. Prefer reporting the error with t.Errorf (or collecting errors and asserting in the main goroutine).

				require.NoError(t, err)

[Copilot]

Pull request overview

Copilot reviewed 16 out of 18 changed files in this pull request and generated no new comments.

Suppressed comments (2)

collector/go.mod:356

• collector/go.mod is explicitly marked as code-generated by the OpenTelemetry collector builder (see file header). Dependency changes here (like adding github.com/bluele/gcache) should come from updating collector/builder-config.yaml and regenerating, rather than editing this file directly; otherwise the next regeneration will likely overwrite the change.

	github.com/beorn7/perks v1.0.1 // indirect
	github.com/bitfield/gotestdox v0.2.2 // indirect
	github.com/blang/semver/v4 v4.0.0 // indirect
	github.com/bluele/gcache v0.0.2 // indirect
	github.com/bmatcuk/doublestar/v4 v4.10.0 // indirect

internal/runtime/logging/sampling.go:254

• samplingInjector.WithAttrs records the incoming attrs slice directly in ops for later replay. Since this handler retains attrs beyond the WithAttrs call, the slice should be copied to avoid aliasing/mutation by the caller leading to incorrect output or data races.

func (s *samplingInjector) WithAttrs(attrs []slog.Attr) slog.Handler {
	ns := s.clone()
	ns.comp = sniffComponent(s.comp, attrs)
	ns.bgCtx = withComponent(context.Background(), ns.comp)
	ns.ops = append(ns.ops, replayOp{attrs: attrs})
	return ns