perf(loki.process): Reduce goroutines and channels in the stage pipeline

[csmarchbanks]

Brief description of Pull Request

loki.process gave every pipeline stage its own goroutine and unbuffered channel, even though almost all stages are pure synchronous single-entry transforms. For a pipeline built from many stage.match rules, that overhead scales linearly with stage count, and under concurrent load it scales with stream count too.

Stage now splits into SyncStage (Process(Entry) (Entry, bool), the common case — composed as plain function calls) and ChannelStage (Run(chan Entry) chan Entry, only for multiline's wall-clock flush and cri/split_json's fan-out). Pipeline precomputes a single fused function when every stage is a SyncStage; stage.match is one of two concrete types decided at construction depending on whether its nested pipeline is itself fully synchronous.

This PR is now chained on top of #6879, which holds the shared benchmark commit as a common base (that commit also picked up a benchmark-harness fix: the drain goroutine wasn't awaited, so a sub-benchmark's leftover work could bleed CPU into the next one's timing). This branch itself contains only the redesign commit (b18c6744d). git checkout loki-process-benchmarks vs git checkout loki-process-many-rules-bench reproduces the numbers below directly.

benchstat, GOMAXPROCS=2 (representative of a CPU-limited container), -count=10, re-run against the fixed benchmark harness (this supersedes the numbers previously posted here, which predated that fix):

                                                             │       base (#6879)        │        this branch (b18c6744d)       │
                                                             │           sec/op          │    sec/op     vs base                │
PipelineManyRules/rules=1-2                                              708.3n ±  2%   571.5n ± 1%  -19.32% (p=0.000 n=10)
PipelineManyRules/rules=10-2                                            2299.5n ±  2%   915.2n ± 3%  -60.20% (p=0.000 n=10)
PipelineManyRules/rules=50-2                                             9.973µ ±  1%   2.396µ ± 1%  -75.98% (p=0.000 n=10)
PipelineManyRules/rules=100-2                                           19.051µ ±  1%   4.062µ ± 1%  -78.68% (p=0.000 n=10)
PipelineManyRules/rules=500-2                                            84.05µ ±  1%   17.37µ ± 1%  -79.33% (p=0.000 n=10)
PipelineManyRules/rules=1000-2                                          152.10µ ±  1%   34.90µ ± 0%  -77.06% (p=0.000 n=10)
PipelineOneMultilineAmongManyRules/all_sync-2                           149.30µ ±  1%   35.07µ ± 0%  -76.51% (p=0.000 n=10)
PipelineOneMultilineAmongManyRules/one_multiline_at_start-2             151.46µ ±  2%   34.68µ ± 0%  -77.10% (p=0.000 n=10)
PipelineOneMultilineAmongManyRules/one_multiline_in_middle-2            151.25µ ±  2%   19.46µ ± 3%  -87.13% (p=0.000 n=10)
PipelineOneMultilineAmongManyRules/one_multiline_at_end-2               148.66µ ±  2%   34.97µ ± 0%  -76.47% (p=0.000 n=10)
PipelineManyStreamsSaturated-2                                          173.68m ±  2%   25.66m ± 1%  -85.22% (p=0.000 n=10)
geomean                                                                  73.72µ         18.03µ       -75.54%

                               │       base (#6879)         │        this branch (b18c6744d)        │
                               │        entries/sec         │  entries/sec    vs base                │
PipelineManyStreamsSaturated-2              9.212k ±  2%    62.349k ± 1%  +576.83% (p=0.000 n=10)

                               │       base (#6879)         │        this branch (b18c6744d)        │
                               │          B/op / allocs/op   │      B/op / allocs/op    vs base       │
PipelineManyStreamsSaturated-2         6.612Mi / 88.37k      1.053Mi / 8.057k    -84.07% / -90.88% (p=0.000 n=10)

PipelineManyRules is one stream, mostly idle. PipelineOneMultilineAmongManyRules checks that a single stage needing its own channel (nesting stage.multiline) doesn't drag the rest of a 1000-rule pipeline back to the old per-stage-channel cost — it doesn't; one_multiline_in_middle measuring faster than all_sync is a GOMAXPROCS-dependent artifact of an even split enabling incidental cross-core pipelining, not something this design relies on (it disappears at GOMAXPROCS=1). PipelineManyStreamsSaturated models many concurrent streams actually competing for the CPU rather than one idle stream, which is the throughput number that matters for a loaded instance: +577% entries/sec, alongside an 84% cut in memory and 91% fewer allocations per op (fewer goroutines/channels in flight).

Verified with go test -race across internal/component/loki/... and the promtail converter, and golangci-lint (no new findings).

Pull Request Details

Issue(s) fixed by this Pull Request

Notes to the Reviewer

PR Checklist

• Documentation added
• Tests updated
• Config converters updated
• This pull request was substantially generated with AI assistance (see the GenAI policy)

[cla-assistant]

All committers have signed the CLA.