perf: reduce Redis round-trips in pub/sub and batch-job state

[bdshadow]

What

Two independent Redis efficiency improvements, no behavior change:

1. Reuse the singleton ObjectMapper for pub/sub serialization

RedisWebsocketEventPublisher, BatchJobChunkExecutionQueue and BatchJobCancellationManager each constructed a fresh jacksonObjectMapper() on every publish — on the websocket and batch hot paths. They now use the injected singleton, matching how RedisPubSubReceiver already consumes it.

2. Pipeline batch-job state cleanup with Redisson RBatch

RedisBatchJobStateStorage issued one Redis round-trip per key:

• removeJobState deleted the state hash, 7 counters and 2 marker buckets as ~10 sequential calls → now one RBatch.
• clearUnusedStates read each job's hash and deleted it one at a time → now reads every job's state in one pipelined batch and deletes the completed ones in a second. Work is chunked (100 jobs/pass) so peak pipeline-response size and materialized state stay bounded regardless of job count. When a pipelined decode fails (e.g. a codec/enum change makes a value unreadable), it falls back to isolated per-job reads so one bad job can't abort cleanup of the rest.

Tests

BatchJobChunkExecutionQueueTest / ...PerformanceTest updated for the new constructor arg.

New RedisBatchJobStateStorageTest (integration, real Redis) covers the parts with no prior coverage:

• removeJobState pipelines its deletes into a single RBatch (spy asserts createBatch once, never per-key getAtomicLong)
• clearUnusedStates clears completed jobs across multiple chunks while preserving counters, the started marker and unfinished jobs
• clearUnusedStates skips a job whose state cannot be read and still cleans the healthy ones

clearUnusedStates previously had no test coverage (it is only driven by a @Scheduled cleaner); removeJobState's hash-clearing remains covered end-to-end by BatchJobsGeneralWithRedisTest.

Summary by CodeRabbit

• Improvements
  • Improved Redis-backed batch job state cleanup and removal, reducing unnecessary operations while preserving active job information.
  • Enhanced cleanup reliability when individual job states cannot be read.
  • Improved processing across larger sets of completed jobs.
  • Standardized message serialization for Redis-backed batch jobs and WebSocket events, improving consistency across deployments.
• Tests
  • Added coverage for Redis cleanup, deletion batching, chunk boundaries, and partial read failures.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: be14da11-b3c8-4f1c-9be5-85c4898a49b0

📥 Commits

Reviewing files that changed from the base of the PR and between 7c9823d and 70774d4.

📒 Files selected for processing (8)

• backend/api/src/main/kotlin/io/tolgee/websocket/RedisWebsocketEventPublisher.kt
• backend/api/src/main/kotlin/io/tolgee/websocket/WebsocketPublisherConfiguration.kt
• backend/app/src/test/kotlin/io/tolgee/batch/state/RedisBatchJobStateStorageTest.kt
• backend/data/src/main/kotlin/io/tolgee/batch/BatchJobCancellationManager.kt
• backend/data/src/main/kotlin/io/tolgee/batch/BatchJobChunkExecutionQueue.kt
• backend/data/src/main/kotlin/io/tolgee/batch/state/RedisBatchJobStateStorage.kt
• backend/data/src/test/kotlin/io/tolgee/batch/BatchJobChunkExecutionQueuePerformanceTest.kt
• backend/data/src/test/kotlin/io/tolgee/batch/BatchJobChunkExecutionQueueTest.kt

---

📝 Walkthrough

Walkthrough

Redis batch-state removal and cleanup now use Redisson batching and chunked reads, while Redis publishers use injected Jackson ObjectMapper instances. New Redis-backed tests validate deletion batching, completed-state cleanup, preservation of markers and counters, and unreadable-state handling.

Changes

Redis batch state and event serialization

Layer / File(s)	Summary
Injected JSON serialization dependencies backend/api/..., backend/data/..., backend/data/src/test/...	Redis websocket and batch-event publishers now serialize through injected ObjectMapper instances, with configuration and test construction updated accordingly.
Batched Redis state cleanup and removal backend/data/src/main/kotlin/io/tolgee/batch/state/RedisBatchJobStateStorage.kt	State removal uses one RBatch, cleanup reads jobs in chunks with pipelined reads, completed state hashes are deleted without counters, and failed reads fall back safely.
Redis state lifecycle integration tests backend/app/src/test/kotlin/io/tolgee/batch/state/RedisBatchJobStateStorageTest.kt	Redis-backed tests verify batched deletion, chunked cleanup, marker and counter preservation, running-job retention, and unreadable-state skipping.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant RedisBatchJobStateStorage
  participant RedissonClient
  participant Redis
  RedisBatchJobStateStorage->>RedissonClient: read cached job IDs in chunks
  RedisBatchJobStateStorage->>RedissonClient: pipeline execution-state reads
  RedissonClient->>Redis: read job state hashes
  RedisBatchJobStateStorage->>RedissonClient: batch completed-state deletions
  RedissonClient->>Redis: delete state hashes and initialization markers

Loading

Possibly related PRs

• tolgee/tolgee-platform#3578: Modifies the Redis batch job state removal path and its side effects.
• tolgee/tolgee-platform#3593: Changes the ExecutionState payload read by the Redis batch-state layer.
• tolgee/tolgee-platform#3762: Also modifies BatchJobChunkExecutionQueue wiring and queue behavior.

Suggested reviewers: jancizmar, anty0

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main performance change: reducing Redis round-trips in pub/sub and batch-job state handling.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch bdshadow/redis-optimizations

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}