Repository navigation
Antalya 26.8 - Bump to v26.8.39 - #2870
Conversation
…B (backport of ClickHouse-internal#1) The YAML is regenerated with GITHUB_REPOSITORY=ClickHouse/ClickHouse-internal, which drops the Finish Workflow job.
(cherry picked from commit 60233e8)
…ingFunction` for Multi-Location Disks
…t on `CREATE TABLE` without column definitions
…gregateFunction` in `RawBLOB` output
…nly storage that provides their data (`backup_metadata_only`)
On `master` `StorageTimeSeries::getTargetKinds` is a `static constexpr` returning every possible target kind,
so the backported code called it on the class. On 26.8 it is a non-static const member returning the kinds
this table actually has, which does not compile as a qualified call:
error: call to non-static member function without an object argument
The loop already holds `time_series` and filters every kind through `isInnerTable`, so calling the member on
that table keeps the behavior: it iterates this table's own kinds instead of all of them, and the filter makes
the two sets agree.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The branch decides `should_backup_data` and leaves the loop at once when it is false, while on master the partition checks run first, so that naming a partition of a table that cannot back one up is still refused however the data came to be left out. The cherry-pick put the new `backup_metadata_only` check after that early exit, where it cannot be reached: leaving the data out is exactly what the flag does, so `BACKUP ... PARTITIONS` of such a table was accepted and restored as the whole table. The two checks move above the exit, as on master. `test_backup_restore_s3_plain_backup_metadata_only` and `test_partitions_of_table_not_created_on_replica` assert the refusal; the first of them also left its table behind when it failed there, which is what then made `test_backup_restore_with_s3_throttle` restore that table into a UUID that was still taken. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…` on the table `StorageTimeSeries::getTargetKinds` is a `static constexpr` on master and a non-static const member on this branch, so the cherry-pick did not compile. Calling it on the table would work, but it is the only thing the branch needs it for, and it only produces a warning: `isDataOnBackupMetadataOnlyStoragePolicy` answers `false` for a `TimeSeries` table either way, as the table keeps no data of its own. So the lookup of its inner tables is left out, with the assertion of that warning dropped from the test, and `context` becomes unused here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…helpers for Trimmed to what the original change is about - a table on a `backup_metadata_only` storage policy has its data left out of a backup and a restore - with everything the branch cannot express removed: - `castStorage`, `DeferredTable` and `resolveStorageProxyLoading` come from ClickHouse#121332, which is not on this branch, so the casts are plain `std::dynamic_pointer_cast`. A table behind a `StorageProxy` is no longer resolved, so its data is backed up as usual rather than left out. - `warnBackupMetadataOnlyIgnored` and the lookups of the inner tables of a materialized view and a `TimeSeries` table are gone. They only logged, and the answer does not depend on them: the flag is ignored for both kinds either way. The test asserts the backups still succeed instead of asserting the warnings. `context` is unused in the overload that is left, and the includes the removed code needed are dropped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`test_replicated_merge_tree_ignores_the_flag` asserted the line `warnBackupMetadataOnlyIgnored` writes for an engine of the `MergeTree` family that is not the plain one. That warning was left out of this branch along with the rest of the diagnostics, so the assertion fails while the behavior it guards - the flag ignored, the data backed up as usual, a new UUID after a `structure_only` restore - is unchanged and still asserted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ergeTree variants on empty block after insert deduplication
…f exhausting the stack
…backport to 26.8 The 26.8 branch has neither `REFRESH ... APPEND INCREMENTAL` nor the refresh cursor stored in Iceberg metadata, so this part of the test failed with a syntax error. Keep only the `STREAM CURSOR` check, which exists on 26.8. https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=127635&sha=ff0689119079ed0e62d07208fc18361a84423a02&name_0=BackportPR&name_1=Stateless%20tests%20%28amd_asan_ubsan%2C%20distributed%20plan%2C%20parallel%29 ClickHouse#127635 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ult for nulls in non-nullable Arrow fields
…T FROM` cluster functions ignoring column `DEFAULT` expressions
…t_log.level into account when updating logger levels
…function requests
…tween concurrent embedded client sessions
…ILL QUERY` and `max_execution_time`
…in the Hive metastore catalog
…er-free of memory reservation after DROP WORKLOAD
…1792 Backport ClickHouse#131792 to 26.8: Reject repeated paths in serialized states of `mergedJSONPatch`
…1797 Backport ClickHouse#131797 to 26.8: Bounds-check skipping fully filtered values in the Parquet `PLAIN` string decoder
…w for a key column fixed by `WHERE` and listed in `ORDER BY`
…s without their inputs
…s only of the tables the user can see
…1736 Backport ClickHouse#131736 to 26.8: Reject serialized query plan steps without their inputs
…0893 Backport ClickHouse#130893 to 26.8: Keep the read-in-order virtual row for a key column fixed by `WHERE` and listed in `ORDER BY`
…9818 Backport ClickHouse#129818 to 26.8: Merge tables report rows and sizes only of the tables the user can see
CI triage for
|
| # | Check(s) | Failing test | Cause | Who |
|---|---|---|---|---|
| 1 | Stateless (amd_asan_ubsan, cas s3, 2/2) | UndefinedBehaviorSanitizer: null pointer → server died |
Backport ClickHouse#131054 (622c1227e78), old-analyzer path |
🔴 PR (bump) |
| 2 | Stateless (amd_binary, cas s3) | 05627_prewhere_guard_sibling_subcolumn |
Same backport ClickHouse#131054, fails with query_plan_merge_filters = 0 |
🔴 PR (bump) |
| 3 | Stateless amd_debug / amd_binary cas / amd_binary cas s3 / asan distributed plan | 05576_delta_lake_partition_pruning_null_in_set |
New upstream test vs Altinity's NULL-aware KeyCondition |
🔴 PR (bump), needs a reference fix only |
| 4 | Unit tests (asan_ubsan) | CASUpstreamSlice.SingleAttemptConditionalRemoveIssuesExactlyOneRequestOnThrottle |
Backport ClickHouse#124557 (f319e793209) changes the outer S3 retry loop |
🔴 PR (bump), the Altinity-only test needs an update |
| 5 | Integration (targeted), (db disk, old analyzer, 3/8) | test_create_union_system_log_tables/* (all 7) |
New node4 (clickhouse/clickhouse-server:26.8, with_installed_binary) never starts |
🟡 test added by the bump; same symptom as #6 |
| 6 | Integration (targeted), (db disk, old analyzer, 7/8) | test_backward_compatibility/test_custom_s3_disk_remote_url_allow_hosts.py::* |
Old-version instance never starts | ⚪ pre-existing: same 2 tests fail on #2880 and #2881 |
| 7 | Keeper Stress Tests (PR) | test_scenario[*-no-fault] |
/usr/bin/clickhouse: No such file or directory, because arm build was skipped |
⚪ CI infra, already fixed by #2877 (merged) |
| – | Stateless (amd_asan_ubsan, cas s3, 1/2) | none (job ERROR, log stops mid-run at 5350/6948) |
Job aborted or timed out | ⚪ pre-existing: both shards are ERROR the same way on #2881 |
| – | DCO | – | Merged upstream commits carry no sign-off | ⚪ expected for a version bump |
1. 🔴 UBSan null deref in conditionMayThrow (server crash)
src/Storages/MergeTree/MergeTreeWhereOptimizer.cpp:373:15: runtime error: member access within null pointer of type 'const ActionsDAG::Node'
#0 DB::conditionMayThrow(DB::ActionsDAG::Node const*) MergeTreeWhereOptimizer.cpp:373
#1 DB::MergeTreeWhereOptimizer::analyzeImpl(...) MergeTreeWhereOptimizer.cpp:469
#4 DB::MergeTreeWhereOptimizer::optimize(SelectQueryInfo&, ...) MergeTreeWhereOptimizer.cpp:161
#5 DB::InterpreterSelectQuery::InterpreterSelectQuery(...)
The backport added conditionMayThrow(infos[i].node.getDAGNode()). On the old-analyzer path (InterpreterSelectQuery → optimize()), the RPNBuilderTreeNode is built from the AST, so getDAGNode() is nullptr (RPNBuilder.h:88/147). The crash takes the server down. That produces the Server liveness check failed and the three UNKNOWN 05576… entries in the same job.
The same code is on upstream master, so this is a latent upstream bug that our old-analyzer jobs happen to hit. Suggested fix (src/Storages/MergeTree/MergeTreeWhereOptimizer.cpp:469):
const auto * dag_node = infos[i].node.getDAGNode();
if (!infos[i].viable || infos[i].expensive || !dag_node || !conditionMayThrow(dag_node))
continue;With this guard the old analyzer behaves as it did before the backport. It is worth reporting upstream as well.
2. 🔴 05627_prewhere_guard_sibling_subcolumn with query_plan_merge_filters = 0
The CI flaky-check found it: Runs: 27, Failed: 27 with the randomized settings, Failed: 0 without them, and the culprit setting is --query_plan_merge_filters False.
Code: 6. DB::Exception: Cannot parse string 'n/a' as UInt64 ... while executing 'FUNCTION toUInt64(t.val :: 0)' ...
(query: SELECT count() FROM t_guard_tuple WHERE t.safe != '' AND toUInt64(t.val) > 50;)
I reproduced it locally on the PR's amd_release CI binary by adding SET query_plan_merge_filters = 0; to the test (builder reason: "Reproduce 05627 failure on PR 2870 CI binary with query_plan_merge_filters=0"). It fails the same way every time. So with query_plan_merge_filters = 0, the guard from ClickHouse#131054 does not hold.
There is no follow-up fix upstream: the only later change to the test on master replaces MD5 with sipHash128. Two options:
- Quick (test-only): add
query_plan_merge_filters = 1to the test'sSETline, and open an issue for the real gap. - Proper: find out why the separate (unmerged) filter steps reorder the conditions, either here or upstream.
3. 🔴 05576_delta_lake_partition_pruning_null_in_set (reference mismatch, results are correct)
Fails 42/42, with and without randomized settings, on every build that ran it:
-t: NOT has([NULL, 1], p) [1,3,4] [1,3,4] 0
+t: NOT has([NULL, 1], p) [1,3,4] [1,3,4] 1
The rows returned are identical with and without pruning ([1,3,4]). The only difference is that we prune one more file, the NULL partition. That pruning is correct: has([NULL, 1], NULL) is 1, so NOT has(...) is 0 for that file. Upstream 26.8 expects 0 because it lacks Altinity's NULL-aware min/max logic in KeyCondition (4efc0847aea, 9732cac9ee0, 4086c266e47). Note that the very next line, the ::Array(Nullable(Int64)) variant, already expects 1 upstream.
Fix: set line 3 of tests/queries/0_stateless/05576_delta_lake_partition_pruning_null_in_set.reference to 1. Add a short note in the commit that the extra pruning comes from Altinity's KeyCondition changes.
4. 🔴 Unit test CASUpstreamSlice.SingleAttemptConditionalRemoveIssuesExactlyOneRequestOnThrottle
src/Disks/tests/gtest_cas_upstream_slice.cpp:723
Expected equality of these values:
retrying->deleteRequestTimeouts().size() Which is: 1
3u
It passes on base (#2881: fail: 0, passed: 23966). The bump brings in backport ClickHouse#124557 (f319e793209), which adds this to Client::doRequestWithRetryNetworkErrors:
/// Unless retries are coordinated across threads, the SDK has already retried this error.
if (!client_configuration.s3_slow_all_threads_after_retryable_error)
break;The test's fake ScriptedGetObjectClient overrides the SDK-level DeleteObject virtual (:395), so the SDK's own retry loop never runs. Until now the 3 attempts came from our outer loop. The fixture sets s3_slow_all_threads_after_retryable_error = false (:281), so the outer loop now stops after one attempt. Production still retries inside the SDK. This is a test expectation problem, not a regression. Options:
- Build the
retryingclient withs3_slow_all_threads_after_retryable_error = true, so the outer loop keeps doing the retries the test counts. This needs a fixture parameter; don't change the shared default, other tests rely on it. - Or expect
1uand rewrite the comment: the fake bypasses the SDK attempt loop, so retries are no longer visible here.
The test belongs to the CAS feature, so its owner should pick.
5./6. Old-version instances that never start (with_installed_binary)
Both suites fail in the same way:
Exception: Timed out while waiting for instance `node4' ... to start. Container status: running, logs: Running: bash -c ... clickhouse server --config-file=...
test_custom_s3_disk_remote_url_allow_hosts.py(clickhouse/clickhouse-server:26.8.26.12) is not touched by this PR. It also fails on Antalya 26.8: Fix bug in iceberg manifest compaction when the catalog update fails #2881 and Antalya 26.8: Use the pinned python-bottle tag in integration tests #2880 → pre-existing onantalya-26.8.test_create_union_system_log_tableschanged in this PR: the bump addsnode4withimage="clickhouse/clickhouse-server", tag="26.8", with_installed_binary=True. All 7 tests of the module fail because the module-level cluster cannot start that node. The cause looks like the same environment problem as above, now hitting a test the bump brought in.
I could not open logs.tar.gz here, so I don't know why the upstream-image instances hang on start. Until someone fixes that on base, mark both tests as known-broken or skip them on Altinity CI. Moving test_replaced_during_upgrade and node4 into a separate module would at least keep the other 6 union-log tests running.
7. Keeper Stress Tests: CI config, already fixed
E bash: line 1: /usr/bin/clickhouse: No such file or directory
The bump touches src/Coordination, so the Keeper stress job runs. With ci_exclude_aarch64|arm ticked, Build (arm_binary) is skipped and the job has no binary. #2877 (merged into antalya-26.8) fixes exactly this and cites this PR. Merging the current antalya-26.8 into the bump branch should turn this green.
Suggested next steps for this PR:
- Merge the current
antalya-26.8into the bump branch (picks up Antalya 26.8: Keep Build (arm_binary) when Keeper stress runs with arm tests excluded #2877). - Null-guard
getDAGNode()inMergeTreeWhereOptimizer::analyzeImpl(Update README.md #1). - Pin
query_plan_merge_filters = 1in 05627 and open an issue for the underlying gap (Update README.md #2). - Update the 05576 reference to
1(Update README.md #3). - Adjust the CAS unit test (Update README.md #4).
- Re-run.
I can push 2–4 to this branch if you want. #5/#6 need someone to look at why upstream-image instances don't start in our CI.
Evidence: praktika result_pr.json for 8f782d5, compared with #2880/#2881 on antalya-26.8. 05627 was reproduced locally on the PR's CI amd_release binary. The other verdicts come from CI logs and source reading. CI still has the final word.
Release v26.8.38.9-lts # Conflicts: # cmake/autogenerated_versions.txt
The old analyzer's AST-based `PREWHERE` path has no `ActionsDAG` node. The throwing-condition grouping dereferences that null pointer when either plan-level `PREWHERE` optimization or all plan optimizations are disabled. Skip the DAG-only grouping for AST conditions, keeping the old grouping behavior. #2870 (comment)
With `query_plan_merge_filters = 0`, the 26.8 reader can evaluate a throwing condition before applying its guard's filter. Keep this ordering test on the default until the reader-safety fix and its performance follow-up are ported and verified in this branch. #2870 (comment)
Release v26.8.39.19-lts # Conflicts: # cmake/autogenerated_versions.txt
The scripted client bypasses SDK retries, so disabling coordinated retries makes `Default` issue one request, just like `SingleAttempt`. Enable coordination only in the throttle test, with zero backoff, to restore its three-versus-one check without changing shared defaults. #2870 (comment)
The bumped reader reports higher pruning counters for two predicates, while returning the same rows in all 15 cases. Align the expected counts so the regression test keeps checking `NULL` handling. #2870 (comment)
The fast build has no `Parquet` support. Keep this regression in Parquet-enabled jobs. https://github.com/Altinity/ClickHouse/actions/runs/44141297255/job/134310276886?pr=2870 #2870
CI triage for
|
| Check | Classification | PR-caused? |
|---|---|---|
| Build (amd_release) | Infra: S3 503 on the ThinLTO cache upload | No |
| Build (arm_release) | Infra: S3 GatewayTimeout on the ThinLTO cache upload | No |
| DCO | Process: upstream backport commits have no Signed-off-by |
No |
Integration tests (amd_asan_ubsan, db disk, old analyzer, 3/8): test_s3_cluster/test.py::test_graceful_shutdown |
Known race, fix in open PR #2759 | Very likely no (see note) |
| Regression release swarms | Pre-existing on base, same 2 scenarios fail on #2881 | No |
1–2. Build (amd_release) / Build (arm_release): infra
Compilation succeeded; the downstream integration and regression jobs ran on these binaries. Only the post-build sub-step Upload ThinLTO cache failed (1/5 sub-steps):
Failed to upload .../thinlto_cache.tar.zst to altinity-builds-cache/ci_ch_cache/thinlto/antalya-26.8/amd_release-g0.tar.zst:
An error occurred (503) when calling the UploadPart operation (reached max retries: 4): Service Unavailable
arm_release failed the same way, with GatewayTimeout ... The server did not respond in time.
Next step: nothing in the code to fix. Re-run the two build jobs if you want a green check.
3. DCO: process, expected for a bump
There are 214 commits incorrectly signed off. These are upstream Backport #NNNNN to 26.8: … and Update autogenerated version … commits brought in by merging tag v26.8.37.168-lts. The previous bump PR #2661 failed DCO the same way and was merged.
Next step: none. Don't rewrite upstream history; set DCO to pass manually per the usual bump procedure.
4. test_s3_cluster/test.py::test_graceful_shutdown: known race, very likely unrelated
test_s3_cluster/test.py:2171: in test_graceful_shutdown
assert errors == 0
E assert 2 == 0 (first run)
E assert 1 == 0 (retry)
- The test kills a swarm node (
SYSTEM STOP SWARM MODEfollowed bykill) while 10s3Clusterqueries are running. It counts every exception or result mismatch on the initiator as an error. - Open PR Fixes for graceful shutdown #2759 ("Fixes for graceful shutdown") describes this failure: the initiator only tolerates
ATTEMPT_TO_READ_AFTER_EOFduring node shutdown, notNETWORK_ERROR, and there is a race when swarm mode is toggled. That PR also changes this test. - The bump's only change to
StorageObjectStorageCluster(from upstream backport Fix distributedINSERT ... SELECT FROMcluster functions ignoring columnDEFAULTexpressions ClickHouse/ClickHouse#127657, commita8d7ab4) adds anadd_structure_and_formatflag. It isfalseonly on the distributedINSERT … SELECTpath. The plainSELECT … FROM s3Cluster(...)path this test uses still passestrue, so its behaviour is unchanged. - The test passed on the other recent antalya-26.8 PRs I checked ( Antalya 26.8: Fix bug in iceberg manifest compaction when the catalog update fails #2881, Antalya 26.8 Backport of #131963 - Clear
--jwtand the other credential options from the command line ofclickhouse-clientandclickhouse-keeper-client#2885, Object storage identity cache #2888, Antalya-26.8 - Drop CAS log configs on installed-binary integration nodes #2890).
Caveat: it failed in both attempts of this run, and I couldn't see the exact server-side exception (the job log has none, and I can't unpack the logs.tar.gz artifact here). So I can't completely rule out that the larger upstream delta made the race more likely.
Next step: re-run the shard. If it fails again, look at the s0_0_0 server log in logs.tar.gz for NETWORK_ERROR, which would confirm the case #2759 fixes, and prioritise merging #2759.
5. Regression release swarms: pre-existing on base
The 2 failed scenarios are /swarms/feature/node failure/cpu overload and /swarms/feature/node failure/cpu overload all swarm nodes:
Code: 209. DB::Exception: Received from localhost:9000. DB::Exception: Timeout exceeded while reading from socket
(socket (172.18.0.9:9000), receive timeout 300000 ms): While executing Remote. (SOCKET_TIMEOUT)
#2881 (antalya-26.8, unrelated Iceberg fix) fails the same two scenarios with the same SOCKET_TIMEOUT, plus an additional failure in cluster discovery. The previous bump #2661 also had a red swarms check.
Next step: none for this PR. It's a base-branch issue in the swarms suite under CPU overload.
Verdict: nothing to fix in this PR. Re-run the two release builds and integration shard 3/8, and handle DCO the same way as previous bumps. Evidence comes from the praktika result_pr.json, the shard 3/8 job.log, and the swarms report.html for this PR and #2881.
The fast build has no `Parquet` support. Keep this regression in Parquet-enabled jobs. https://github.com/Altinity/ClickHouse/actions/runs/44141297255/job/134310276886?pr=2870 #2870
With `query_plan_merge_filters = 0`, the 26.8 reader can evaluate a throwing condition before applying its guard's filter. Keep this ordering test on the default until the reader-safety fix and its performance follow-up are ported and verified in this branch. #2870 (comment)
Changelog category (leave one):
CI/CD Options
Exclude tests:
Regression jobs to run: