You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Iceberg ADD COLUMN of nested types fails commit-unknown recovery with Column already exists #2215
Iceberg ALTER retry after a “commit state unknown” catalog response does not detect that an ADD COLUMN of a nested type (Array / Map / Tuple) already landed. The retry then fails with Column already exists.
isAddColumnApplied rebuilds the expected Iceberg type via getIcebergType(type, last_column_id) from the current last-column-id (already advanced by the successful commit).
For primitives that is a string ("string", "long", …) and the compare succeeds.
For Array / Map / Tuple, getIcebergType allocates new nested field ids. icebergTypesEqual stringifies JSON, so element-id / key-id / value-id / struct field ids do not match → false → retry → Column already exists.
SET allow_insert_into_iceberg = 1;
SET enable_nullable_tuple_type = 1;
SET allow_insert_into_iceberg = 1
Query id: 85e1ee06-cc8e-42ac-a2d2-de6dce7a3a55
Ok.
0 rows in set. Elapsed: 0.002 sec.
SET enable_nullable_tuple_type = 1
Query id: 344f2f8c-797d-4cbb-9380-dcbf57955916
Ok.
0 rows in set. Elapsed: 0.001 sec.
Ubuntu-2404-noble-amd64-base :) SYSTEM ENABLE FAILPOINT iceberg_alter_catalog_commit_reported_as_failed;
SYSTEM ENABLE FAILPOINT iceberg_alter_catalog_commit_reported_as_failed
Query id: a31e3de3-21a9-4b8c-a165-27bde86e1089
Ok.
0 rows in set. Elapsed: 0.002 sec.
Ubuntu-2404-noble-amd64-base :) SELECT name, type, enabled
FROM system.fail_points
WHERE name = 'iceberg_alter_catalog_commit_reported_as_failed';
SELECT
name,
type,
enabled
FROM system.fail_points
WHERE name = 'iceberg_alter_catalog_commit_reported_as_failed'
Query id: df71199d-d00c-44ec-a310-f38a26833580
┌─name────────────────────────────────────────────┬─type─┬─enabled─┐
1. │ iceberg_alter_catalog_commit_reported_as_failed │ once │ 1 │
└─────────────────────────────────────────────────┴──────┴─────────┘
1 row in set. Elapsed: 0.002 sec.
Ubuntu-2404-noble-amd64-base :) alter table ice.`nyc.taxis_p_by_day` add column tuple_col_4 Nullable(Tuple(Int32, String));
ALTER TABLE ice.`nyc.taxis_p_by_day`
(ADD COLUMN `tuple_col_4` Nullable(Tuple(
Int32,
String)))
Query id: 95284155-b76c-4e37-8352-b1838461b145
Ok.
0 rows in set. Elapsed: 0.182 sec.
It looks like this was partially fixed. Nullable(Tuple(Int32, String)) no longer fails, but I was able to reproduce it with Nullable(Tuple(Array(Int32))).
@subkanthi It works fine if the catalog commit is acknowledged, but on the off chance that the client is told the commit failed (which is reproducible using a system failpoint), the retry is supposed to recognize the nested column that already landed. See SQL below:
SET allow_insert_into_iceberg =1;
SET enable_nullable_tuple_type =1;
SYSTEM ENABLE FAILPOINT iceberg_alter_catalog_commit_reported_as_failed;
SELECT name, type, enabled
FROMsystem.fail_pointsWHERE name ='iceberg_alter_catalog_commit_reported_as_failed';
-- enabled must be 1 hereALTERTABLE ice.`<namespace>.<table>`
ADD COLUMN array_col Nullable(Tuple(Array(Int32)));
Activity
subkanthi commentedon Aug 27, 2026
Unable to reproduce in 26.6 @DimensionWieldr
DimensionWieldr commentedon Aug 28, 2026
@subkanthi Try
Nullable(Tuple(Array(Int32))).It looks like this was partially fixed.
Nullable(Tuple(Int32, String))no longer fails, but I was able to reproduce it withNullable(Tuple(Array(Int32))).subkanthi commentedon Sep 15, 2026
@DimensionWieldr not able to reproduce.
DimensionWieldr commentedon Sep 15, 2026
@subkanthi It works fine if the catalog commit is acknowledged, but on the off chance that the client is told the commit failed (which is reproducible using a system failpoint), the retry is supposed to recognize the nested column that already landed. See SQL below:
subkanthi commentedon Nov 17, 2026
@DimensionWieldr #2881 fixes this bug.
Not able to reproduce in this branch
DimensionWieldr commentedon Nov 17, 2026
@subkanthi Nice! Looks like this issue can be closed once that's merged then.