Skip to content

Billing→S3: batch + S3 key builder from logged billing lines #2516

Description

@Hazel-Datastax

Pure logic to turn the stream of billing.events JSON log lines into batched .jsonl content + S3 keys. No I/O — fully unit-testable.

Depends: #2515

What to do

  • For each logged line, parse out only event_type and timestamp (for routing + key); keep the original line verbatim as the .jsonl body (no re-serialization).
  • Batch lines grouped by event_type, sealing a batch on max-events / max-bytes / max-age (whichever first).
  • Build the key <event_type>/<YYYY>/<MM>/<DD>/<HH>/<MM>/<uuid>.jsonl from the batch's event_type + timestamp + a per-file UUID.

Acceptance

  • Unit tests: key format for a fixed timestamp; per-event_type grouping; .jsonl body equals the exact logged lines newline-joined; per-file UUID uniqueness.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions