Skip to content

Antalya 26.8 Backport of #128217 - Evaluate the PARTITION BY expression of file and object storage writes once per INSERT, not once per block - #2910

Draft
blau-ai wants to merge 3 commits into
antalya-26.8from
backports/antalya-26.8/128217
Draft

blau-ai wants to merge 3 commits into
antalya-26.8from
backports/antalya-26.8/128217

Conversation

@blau-ai

@blau-ai blau-ai commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in an official stable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Fixed PARTITION BY of file, url and object storage writes (table functions and table engines) evaluating stateful and non-deterministic functions anew for every block since 26.1: PARTITION BY intDiv(rowNumberInAllBlocks(), N) restarted its counter on every block and wrote rows into the wrong files, and now64() produced one file per block instead of one per INSERT. Closes ClickHouse#128211. (ClickHouse#128217 by @groeneai)

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

  • Fast test
  • Integration Tests
  • Stateless tests
  • Stateful tests
  • Unit tests
  • Performance tests
  • Aarch64 tests
  • All with ASAN
  • All with TSAN
  • All with MSAN
  • All with UBSAN
  • All with Coverage
  • All Regression
  • Disable CI Cache

Regression jobs to run:

  • Fast suites (mostly <1h)
  • Aggregate Functions (2h)
  • Alter (1.5h)
  • Benchmark (30m)
  • CAS (content-addressed storage; Antalya only)
  • ClickHouse Keeper (1h)
  • Iceberg (2h)
  • LDAP (1h)
  • OAuth (5m)
  • Parquet (1.5h)
  • RBAC (1.5h)
  • SSL Server (1h)
  • S3 (2h)
  • S3 Export (2h)
  • Swarms (30m)
  • Tiered Storage (2h)

alexey-milovidov and others added 2 commits November 18, 2026 15:52
…tateful-function-spans-chunks

Evaluate the `PARTITION BY` expression of file and object storage writes once per INSERT, not once per block

Conflicts committed as-is; the next commit resolves them:
- src/Storages/IPartitionStrategy.cpp (both modified)
- src/Storages/IPartitionStrategy.h (both modified)
Kept antalya's `computePartitionKey(Block&)` overrides, added the new `getPartitionKeyActions()` overrides, and left out upstream's `getPathForRead`/`getPathForWrite` context lines.
@blau-ai

blau-ai commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator Author

Port-AL: cherry-pick conflicts resolved by AI

Conflicted files:

  • src/Storages/IPartitionStrategy.cpp (both modified)
  • src/Storages/IPartitionStrategy.h (both modified)

Commits:

  • 477745a: cherry-pick with the conflicts committed as-is

  • 4ca0a99: resolution

  • IPartitionStrategy.h: the Wildcard and HiveStyle strategies now declare getPartitionKeyActions() const override plus antalya's existing computePartitionKey(Block &) override. I removed the old computePartitionKey(const Chunk &) override.

  • IPartitionStrategy.cpp: the HiveStyle conflict left the old computePartitionKey(const Chunk &) header facing upstream's new getPartitionKeyActions definition. I kept only the getPartitionKeyActions signature.

  • Dropped the getPathForRead/getPathForWrite code that sat inside the conflict hunk. It came from newer upstream refactors and is not in antalya-26.8's header, so it isn't part of this PR's change.

  • The non-conflicting parts of the PR applied cleanly. These are the new IPartitionStrategy::computePartitionKey(chunk, actions) and the PartitionedSink caching of partition_key_actions.

  • StorageObjectStorage.cpp:907 still calls computePartitionKey(Block &), so that overload had to stay.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Workflow [PR], commit [1fcf913]

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants