Skip to content

antalya-26.8: amd_msan server crashes in Iceberg manifest-list pruning after int→long promotion #2897

Open

Description

@CarlosFelipeOR

Describe the situation

On antalya-26.8, the amd_msan server dies in test_storage_iceberg_with_spark on every MasterCI run since 2026-11-11. MSan reports use-of-uninitialized-value in accurateLess, called from ManifestListPruner::canBePruned. The rest of the module then fails with Connection refused.

Non-MSan builds don't crash, but they read the same garbage bound, so they may prune Iceberg partitions incorrectly.

Cause

#2758 backported ClickHouse#115817, but left out its commit 083fda43bd9 ("Do not read a partition bound past its buffer"), the only change in IcebergFieldParseHelpers.cpp.

After an int → long promotion, the manifest list keeps a 4-byte bound, and deserializeFieldFromBinaryRepr reads it as an 8-byte Int64. The backport brought the commit's test (test_manifest_list_partition_pruning_after_type_promotion) but not the fix. MSan was excluded from the PR's CI, so the crash first showed up on MasterCI.

Fix

Add the upstream guard to deserializeFieldFromBinaryRepr in src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergFieldParseHelpers.cpp, before insertData:

if (non_nullable_type->haveMaximumSizeOfValue() && str.size() < non_nullable_type->getSizeOfValueInMemory())
    return std::nullopt;

Failed runs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions