Repository navigation
Antalya 26.8 Backport of #126267 - Use less s3 requests to get metadata file - #2873
Conversation
…rsed-metadata-json Use less s3 requests to get metadata file Conflicts committed as-is; the next commit resolves them: - src/Planner/PlannerJoinTree.cpp (both modified) - src/Storages/ObjectStorage/DataLakes/DeltaLakeMetadataDeltaKernel.h (both modified) - src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.cpp (both modified) - src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.h (both modified) - src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadataFilesCache.h (both modified) - src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergWrites.cpp (both modified) - src/Storages/ObjectStorage/DataLakes/Iceberg/MetadataGenerator.cpp (both modified) - src/Storages/ObjectStorage/DataLakes/Iceberg/Mutations.cpp (both modified) - src/Storages/ObjectStorage/DataLakes/Iceberg/Utils.cpp (both modified) - src/Storages/ObjectStorage/StorageObjectStorageCluster.cpp (both modified)
Resolved 10 conflicts by keeping the 26.8 code and applying the PR's metadata-lookup and parsed-JSON caching; dropped the `ignore_metadata_pointer_overrides`, `explicit_metadata_file_path` and catalog-lookup code and the `createInitial` declaration, which 26.8 lacks.
|
Port-AL: cherry-pick conflicts resolved by AI Conflicted files:
Commits:
|
CI triage for
|
| Check | Classification | PR-caused? |
|---|---|---|
| Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 1/2 and 2/2) | Job hit its time limit with 0 test failures | No (same on #2846) |
Integration (amd_asan_ubsan, db disk, old analyzer, 1/8): test_database_glue::test_create_table_engine_backend_mismatch_rejected |
Outdated assertion in the test | No (same on #2846; fix is #2869) |
Integration (… 7/8): test_backward_compatibility/test_custom_s3_disk_remote_url_allow_hosts.py (2 tests) |
Test fixture never started its server | No (same on #2846) |
| Regression release swarms | Swarms suite already failing on base | No (same on #2846 and #2869) |
| DCO | Commits have no Signed-off-by line |
No, it's about commit metadata, not code |
1. Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 1/2 and 2/2): time limit
WARNING: Timeout exceeded [12600] for [27955]
WARNING: Job timed out: [Stateless tests (amd_asan_ubsan, cas s3 storage, parallel, 1/2)], timeout [12600], exit code [-15]
- Neither shard has a single
[ FAIL ]. The 3.5 h limit stopped them partway: shard 1 at 3854/6892 tests, shard 2 at 3309/6935. - Same on Antalya 26.8 Backport of #131154 - Reject out-of-range
Decimalscale when reading aField#2846: both shards stopped at about the same point (3884/6890 and 3515/6933) after about 3h40m. - The PR's new tests ran several times each (the runner repeats new tests) and all passed in 12–22 s:
05407_iceberg_metadata_s3_requests: 3× OK05408_iceberg_rejected_alter_keeps_cached_metadata: 2× OK05409_iceberg_select_sees_rows_appended_after_create: 1× OK05441_iceberg_timestamp_selection_lower_version_replaces_cached: 3× OK
- Caveat: about half of each shard never ran, so this job gives no signal for those tests. Other stateless jobs cover them, but they don't run with ASan + CAS storage.
- Next step: nothing to do on this PR. On the CI side, the ASan CAS-S3 job needs more shards or a longer time limit on antalya-26.8.
2. Integration 1/8: test_database_glue/test.py::test_create_table_engine_backend_mismatch_rejected
E AssertionError: Received exception from server (version 26.8.34):
E Code: 36. DB::Exception: ... Table setting `disk` differs from its value in this DataLakeCatalog. The table is reopened with the database settings, so the table-level value would be lost.
- The server rejects the table with a different message than the test asserts. The code path is the DataLakeCatalog settings check, which this PR doesn't change.
- Same on Antalya 26.8 Backport of #131154 - Reject out-of-range
Decimalscale when reading aField#2846: same test fails. - Antalya 26.8: Fix stale assertion in test_create_table_engine_backend_mismatch_rejected(Glue) #2869 ("Fix stale assertion in test_create_table_engine_backend_mismatch_rejected (Glue)") is open and fixes the assertion.
- Next step: nothing to do here; it goes green once Antalya 26.8: Fix stale assertion in test_create_table_engine_backend_mismatch_rejected(Glue) #2869 lands.
3. Integration 7/8: test_custom_s3_disk_remote_url_allow_hosts.py, both tests
E Exception: Timed out while waiting for instance `node' with ip address 172.16.4.7 to start. Container status: running ...
- Both tests fail in the
started_clusterfixture (cluster.start()), before any test code runs. - Same on Antalya 26.8 Backport of #131154 - Reject out-of-range
Decimalscale when reading aField#2846: same 2 failures, same counts (2/1004). - Next step: nothing to do here; it's a problem on the base branch. If someone wants to debug it, the old-version node's
clickhouse-server.err.login the 7/8 artifacts is the place to start.
4. Regression release swarms
✘ [ Fail ] '/swarms/feature/cluster discovery/check cluster discovery with wrong cluster name'
✘ [ Fail ] '/swarms/feature/node failure/check restart swarm node'
✘ [ Fail ] '/swarms/feature/node failure/cpu overload'
✘ [ Fail ] '/swarms/feature/node failure/cpu overload all swarm nodes'
1535 scenarios (1517 ok, 4 failed, 14 skipped)
- These scenarios cover cluster discovery and node failure/CPU overload, which have nothing to do with Iceberg metadata caching.
- Same elsewhere: Antalya 26.8: Fix stale assertion in test_create_table_engine_backend_mismatch_rejected(Glue) #2869, a test-only PR, fails with the same
1517 ok, 4 failed. Antalya 26.8 Backport of #131154 - Reject out-of-rangeDecimalscale when reading aField#2846 fails with1518 ok, 3 failed. - Next step: nothing to do here.
5. DCO
- Neither commit has a
Signed-off-byline:c873aa2ef8, the upstream merge commit, and34061b6a25, the conflict-resolution commit. - Same on Antalya 26.8 Backport of #131154 - Reject out-of-range
Decimalscale when reading aField#2846: it also fails DCO, as backports of upstream merge commits usually do. - Next step: if DCO is required for merging, a maintainer can override it. The alternative is to sign off the commits, which means rewriting history and force-pushing, and I'm not allowed to force-push.
Bottom line: none of the CI failures point at this backport. The one real gap is that the ASan CAS-S3 stateless run stopped about halfway, so this PR has no ASan + CAS-storage result for the tests it didn't reach. The new Iceberg tests did run there and passed. CI on the PR still has the final say; re-running after #2869 merges should leave only the failures that already happen on base.
# Conflicts: # src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadataFilesCache.h
|
Conflicts resolved, asked for in Port-AL by zvonand: Merged antalya-26.8: in IcebergMetadataFilesCache.h kept the PR's parsed-JSON cache cell and base's size_t memory_bytes (dropped the Int64 cast), same as upstream Merged |
CI triage for
|
| Check | Classification | Caused by PR? |
|---|---|---|
Stateless tests (amd_debug, distributed plan, s3 storage, parallel): 03572_export_merge_tree_part_limits_and_table_functions |
Flaky, caused by randomized MergeTree settings | No |
Regression release swarms: node failure/cpu overload |
Pre-existing, also fails on other antalya-26.8 PRs | No |
Grype Scan, keeper and server -alpine images (4 High CVEs) |
Image vulnerability scan, not code | No |
| DCO | 2 backport commits have no Signed-off-by |
Process issue only |
1. Stateless: 03572_export_merge_tree_part_limits_and_table_functions (1 failure, 13498 passed)
---- Count files in big_destination_max_bytes, should be 5 (4 parquet, 1 commit)
-5
+4
CI's own randomized-settings diagnosis for this failure:
Step 1: Re-running with the same randomized settings ... Runs: 3, Failed: 3
Step 2: Re-running without randomized settings ... Runs: 2, Failed: 0, Passed: 2
Result: Passes without randomization. Confirmed: the failure is caused by randomized settings.
Culprit MergeTree settings: ... --index_granularity 39074 --merge_max_block_size 11357 --string_serialization_version single_stream ...
- Why this is not the PR: The test checks how many Parquet files
EXPORT PARTwrites whenexport_merge_tree_part_max_bytes_per_file=3500000. The randomized granule and merge block sizes change where the export splits files, so it writes 4 files instead of 5. The PR only changes the Iceberg metadata cache and thetotalRows/totalRowsInSnapshotpath inStorageObjectStorage*. It does not touch the MergeTree to S3 export sink, and the test does not use Iceberg. - Next step: Safe to re-run. The real fix belongs to the test, not this PR: either pin
index_granularity/merge_max_block_sizeonbig_table, or assert a range of file counts instead of exactly 5.
2. Regression release swarms
✘ [ Fail ] '/swarms/feature/node failure/cpu overload' (5m 4s)
✘ [ Fail ] '/swarms/feature/node failure/cpu overload all swarm nodes' (5m 8s)
- Evidence it is pre-existing: The same two scenarios fail with the same
AssertionErroron Fix cross compiling on Apple #2882 (CMake-only change, job log).Regression release swarmsalso fails on Antalya-26.8 - Drop CAS log configs on installed-binary integration nodes #2890, which only touches integration tests. - Next step: Nothing to do on this PR. This is a known problem with these swarms scenarios on antalya-26.8.
3. Grype Scan: clickhouse-keeper and clickhouse-server …-alpine
Both reports list the same 4 High findings under docker vulnerabilities: CVE-2026-95692, CVE-2026-95691, CVE-2026-88823 and CVE-2026-107496.
- Why this is not the PR: These come from packages in the image (both failing images are Alpine-based). The PR only changes Iceberg C++ code and tests. Antalya 26.8: Fix race condition in Glue update #2861 on the same branch scanned clean (0 High/Critical) on an older build. That suggests the CVEs are new advisories or a base-image change, but I have not verified this.
- Next step: Fix this on the branch (bump the base image or packages), not in this backport.
4. DCO
c873aa2 Merge pull request #126267 from ClickHouse/iceberg... Sign-off not found
34061b6 Resolve conflicts in backport of #126267 Sign-off not found
- This is the usual result for upstream backport commits. Fixing it would mean rewriting branch history with a force-push, which I'm not allowed to do. A maintainer can override the check, as is normally done for backports.
PR health: The PR's own new tests (05407, 05408, 05409, 05441) did not fail in any job that has finished. Nothing in the failures points at the Iceberg metadata changes. I'd re-run the stateless job and wait for SQLLogic/SQLStorm to finish.
Sources: praktika result_pr.json for a75b27f, Actions job logs, Grype reports from the artifact bucket. I did not reproduce anything locally because CI's own reruns already settle the stateless verdict.
Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
Reduce the number of S3 requests when reading Iceberg tables by reusing metadata lookups and caching parsed
metadata.jsonfiles: one GET and one LIST instead of three GETs and two LISTs. (ClickHouse#126267 by @scanhex12)Documentation entry for user-facing changes
...
CI/CD Options
Exclude tests:
Regression jobs to run: