Skip to content

Flaky tests in Altinity regression suite #94

@CarlosFelipeOR

Description

@CarlosFelipeOR

🧪 Description

This issue tracks flaky or unstable tests in the Altinity regression test suite.

These tests:

  • Are maintained by Altinity
  • Are not part of upstream ClickHouse tests
  • May fail intermittently but pass after reruns

⚠️ Flaky test List

  • Export Part

    • Test case: /s3/minio/export tests/export part/concurrent alter/during export/example=drop table
      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Intermittent
      • First observed: 2026-01-21
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
  • Swarms

    • Test case: /swarms/feature/swarm joins/join clause

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
    • Test case: /swarms/feature/feature/object storage cluster profile events with overloaded node

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
    • Test case: /swarms/feature/node failure/network failure

      • Versions: 25.8.14.20001.altinityantalya
      • Flakiness: Frequent
      • First observed: around 2025-12
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:
  • aggregate functions

    • Test case: /aggregate functions/part 3/merge/
      • Versions: 25.8.16.20002.altinityantalya
      • Flakiness: Rare
      • First observed: Mar 22, 2026
      • Assigned to:
      • Status: Pending investigation
      • 🔗 Links:


📝 Instructions

Please follow the guidelines below when tracking new flaky tests in this issue.

➕ Adding a new flaky test

To add a new flaky test to the list, use the following template:

- **SUITE NAME**
  - Test case: `PATH/TO/TEST`
    - Versions: 
    - Flakiness: 
    - First observed: 
    - Assigned to: 
    - Status: Pending investigation
    - 🔗 Links:
      - CI run
      - Query in history database

📊 Flakiness levels

Use one of the following values for Flakiness:

  • Rare (< 5%)
  • Intermittent (< 30%)
  • Frequent (> 30%)

Choose the level based on how often the test fails in CI.

📌 Status values

Use one of the following Status values:

  • Pending investigation — flaky observed, not investigated yet
  • Under investigation — investigation in progress
  • Fixed (pending confirmation) — fix applied, awaiting CI stability
  • Fixed — confirmed stable after multiple CI runs

🔍 Investigation workflow

For deeper investigation:

  • Create a sub-issue under this issue for the flaky test
  • Use the sub-issue to track analysis, logs, discussions, and fixes

Once the flakiness is fixed and confirmed stable after multiple CI runs:

  • Update the sub-issue accordingly
  • Remove the test from the list in the description

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions