Skip to content

fix: Convert Parquet NANOS footer stats to microseconds in manifest bounds - #228

Merged
subkanthi merged 1 commit into
Altinity:v_24_0_releasefrom
ianton-ru:fix/timestamp-nanos-bounds
Nov 19, 2026
Merged

subkanthi merged 1 commit into
Altinity:v_24_0_releasefrom
ianton-ru:fix/timestamp-nanos-bounds

Conversation

@ianton-ru

Copy link
Copy Markdown

Summary

  • ice insert --no-copy copied Parquet TIMESTAMP_NANOS footer min/max onto an Iceberg timestamp / timestamptz column, so the manifest bounds were about 1000x too large.
  • Divide those statistics by 1000. A remainder widens the bound by less than one microsecond, so the range still contains the original instant. MILLIS is still multiplied by 1000 and MICROS is unchanged.

Fixes #227

Test plan

  • ParquetConversionsTest covers nanosecond widening, an exact microsecond, a negative value, milliseconds, and microseconds
  • PartitioningTest still passes

…ounds

`ice insert --no-copy` and `--force-no-copy` copy Parquet footer min/max
into Iceberg manifest bounds. For a `timestamp`/`timestamptz` column the
bounds must be microseconds, but NANOS statistics were copied unchanged,
so a nanosecond file produced bounds about 1000x too large.

Divide NANOS statistics by 1000. A remainder widens the lower bound down
and the upper bound up, so the range still contains the original instant.
MILLIS is still multiplied by 1000 and MICROS is unchanged.

Fixes Altinity#227

Co-authored-by: Cursor <cursoragent@cursor.com>
@subkanthi subkanthi closed this Nov 10, 2026
@subkanthi subkanthi reopened this Nov 10, 2026
@subkanthi
subkanthi changed the base branch from master to v_24_0_release November 19, 2026 04:10
@subkanthi
subkanthi merged commit 23fccc6 into Altinity:v_24_0_release Nov 19, 2026
11 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Parquet TIMESTAMP_NANOS is registered as timestamptz and no-copy insert keeps nanosecond bounds

2 participants