Skip to content

Correct course of action when the size of index headers exhausts disk #15444

Description

@sinapah

Hello,

I have three Mimir backend pods, running the compactor and store-gateway roles, running with replication factor=3. Each of the pods has a persistent volume of size 1Gi. The problem I'm having is that I see logs like:

2026-05-25T20:14:27.088Z [mimir] ts=2026-05-25T20:14:27.019411838Z caller=lazy_binary_reader.go:182 level=info user=anonymous msg="could not download sparse index-header from bucket; will reconstruct when the block is queried" err="getting sparse index-header from bucket: The specified key does not exist."
2026-05-25T20:14:27.088Z [mimir] ts=2026-05-25T20:14:27.065057295Z caller=bucket.go:450 level=error user=anonymous msg="loading block failed" elapsed=90.539934ms id=01KS4NCRN82HNRMXMV8ZGR0QW3 err="create index header reader: write index header: new binary index header writer: mkdir /data/tsdb-sync/anonymous/01KS4NCRN82HNRMXMV8ZGR0QW3: no space left on device"

I probably could resize the persistent volume, but I wonder if there is a better course of action here or whether this is a symptom that the compactor is failing to offload the sparse index-headers to S3 storage. Could lazy loading help here? Is there another way of reducing the number of index-headers being stored on disk?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions