Skip to content

feat(vlinsert): add vlinsert stream field limit#1

Open
bearslyricattack wants to merge 3 commits into
labring-sigs:fix-v1.50.0from
bearslyricattack:streamlimits-v1.50.0
Open

feat(vlinsert): add vlinsert stream field limit#1
bearslyricattack wants to merge 3 commits into
labring-sigs:fix-v1.50.0from
bearslyricattack:streamlimits-v1.50.0

Conversation

@bearslyricattack

Copy link
Copy Markdown

No description provided.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an optional per-stream-label value row limiter to the Loki JSON ingest path in vlinsert, preventing more than N log rows per configured label value within a time window.

Changes:

  • Introduces app/vlinsert/streamfieldlimit package implementing a sharded in-memory windowed row limiter driven by CLI flags.
  • Applies the limiter during Loki JSON request parsing to drop rows beyond the per-value quota.
  • Adds unit tests for the limiter itself and an integration-style test for Loki JSON parsing with the limiter enabled.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
app/vlinsert/streamfieldlimit/limiter.go New sharded, windowed row limiter with flag-based configuration.
app/vlinsert/streamfieldlimit/limiter_test.go Unit tests for limiter behavior and window reset.
app/vlinsert/loki/loki_json.go Integrates limiter into Loki JSON parsing to cap ingested rows per configured label value.
app/vlinsert/loki/loki_json_test.go Adds coverage validating row dropping behavior when the limiter is enabled.

Comment thread app/vlinsert/loki/loki_json.go
Comment thread app/vlinsert/streamfieldlimit/limiter_test.go Outdated
Comment thread app/vlinsert/loki/loki_json.go
Comment thread app/vlinsert/loki/loki_json.go Outdated
Comment thread app/vlinsert/streamfieldlimit/limiter.go
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants