Skip to content

Antalya-26.8: Split TSan CAS stateless lane into 4 shards, move MSan CAS to the large runner - #2926

Draft
CarlosFelipeOR wants to merge 1 commit into
antalya-26.8from
fix/antalya-26.8/cas-stateless-sanitizer-shards
Draft

CarlosFelipeOR wants to merge 1 commit into
antalya-26.8from
fix/antalya-26.8/cas-stateless-sanitizer-shards

Conversation

@CarlosFelipeOR

Copy link
Copy Markdown
Collaborator

Both CAS stateless sanitizer lanes hit the job timeout (12600 s) in MasterCI run 44208230727.

  • TSan, cas s3 storage: each of the 2 shards has ~6.9k tests and did not finish: 5475/6891 and 4403/6934. The plain TSan parallel lane already uses 4 shards, and CAS makes every test slower. This PR splits the lane into 4 shards on the same runner.
  • MSan, cas s3 storage: the job runs on the 30 GiB FUNC_TESTER runner. All 3 shards crossed the memory-pressure threshold of clickhouse-test and dropped to one worker. They stopped at about 1600–2000 of ~4600 tests. This PR moves the lane to AMD_LARGE, the same fix CAS improvements for antalya-26.8 #2831 applied to the ASan CAS lane.

Changelog category (leave one):

  • CI Fix or Improvement (changelog entry is not required)

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Unit tests
  • Performance tests
  • Aarch64 tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • CAS (content-addressed storage; Antalya only)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • OAuth (5m)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

…the large runner

Both lanes hit the job timeout in MasterCI run 44208230727:

- TSan, cas s3 storage: 2 shards of ~6.9k tests each ran out of time.
  The plain TSan parallel lane already uses 4 shards.
- MSan, cas s3 storage: on the 30 GiB FUNC_TESTER runner, clickhouse-test
  crossed its memory-pressure threshold, dropped to one worker, and hit
  the timeout. The ASan CAS lane was moved to AMD_LARGE for the same
  reason (#2831).

Workflow YAML regenerated with praktika.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CarlosFelipeOR CarlosFelipeOR added antalya cicd Improvements and fixes to the CICD process antalya-26.8 labels Nov 19, 2026
@CarlosFelipeOR
CarlosFelipeOR marked this pull request as draft November 19, 2026 19:16
@github-actions

Copy link
Copy Markdown

Workflow [PR], commit [274b0f4]

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

antalya antalya-26.8 cicd Improvements and fixes to the CICD process

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant