Skip to content

feat(table): add collect count-limit with Java FieldCollectAgg parity - #1387

Open
PyRSA wants to merge 1 commit into
apache:mainfrom
PyRSA:feat/collect-agg-count-limit
Open

PyRSA wants to merge 1 commit into
apache:mainfrom
PyRSA:feat/collect-agg-count-limit

Conversation

@PyRSA

@PyRSA PyRSA commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Add fields.<col>.count-limit support to the native Collect Aggregator in paimon-rust, aligning its behavior with Java Paimon's FieldCollectAgg.

Changes

  • Support positive, negative, and zero count limits.
  • Preserve Java-compatible distinct semantics during accumulator merges.
  • Align forward aggregation and ignore-retract reverse behavior with Java.
  • Handle NULL inputs, retract operations, and edge cases.
  • Add unit tests covering count limits, distinct merges, reverse aggregation, and boundary scenarios.
  • Document the collect count-limit option in sql.md.

Compatibility

  • Keep the default behavior unchanged when count-limit is not configured.
  • No changes to other aggregators.
  • No new dependencies.

Testing

  • CollectAgg tests: 13 passed.
  • Aggregator regression tests: 115 passed.
  • validation_table_matches_constructors: passed.
  • git diff --check: passed.

Honor fields.<col>.count-limit for ±N/0 and distinct merges, align
ignore-retract reverse and forward hasDuplicate with Java, and document
the option in sql.md.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant