Skip to content

[test] add Java-Rust sketch/bitmap compatibility tests - #1398

Open
PyRSA wants to merge 2 commits into
apache:mainfrom
PyRSA:feat/sketch-bitmap-java-rust-compat
Open

PyRSA wants to merge 2 commits into
apache:mainfrom
PyRSA:feat/sketch-bitmap-java-rust-compat

Conversation

@PyRSA

@PyRSA PyRSA commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Summary

Add bidirectional Java↔Rust compatibility tests for HLL, Theta, RBM32, and RBM64 aggregators, verifying Rust merge results against Paimon production JVM serialization libraries.

Changes

  • Add golden fixtures and Maven-resolved DataSketches 4.2.0 / RoaringBitmap 1.2.1 deps (same versions as paimon-common).
  • Add JVM verifier that deserializes, unions, and extracts results using the same call patterns as Paimon utils.
  • Cover Java→Rust and Rust→Java paths for all four sketch/bitmap types, including empty, multi-merge, and RBM32/RBM64 cross-merge cases.
  • Keep helper APIs behind #[cfg(test)]; document the local generate/verify flow in README_COMPAT.md.
  • Tighten .gitignore for generated lib/, class files, classpath output, and temp rust-written bytes.

Compatibility

  • No changes to production serialization algorithms or storage formats.
  • No new runtime dependencies.
  • No production API visibility expansion outside #[cfg(test)].

Testing

  • table::aggregator::java_rust_compat: 9 passed (includes JVM VerifyRustSketchBitmapCompat).
  • Aggregator regression tests: 116 passed.
  • cargo fmt --check: passed.
  • git diff --check: passed.

AuroraVoyage added 2 commits November 13, 2026 11:24
Verify HLL, Theta, RBM32, and RBM64 serialization and merge
compatibility using Paimon's JVM dependency versions (DataSketches
4.2.0 / RoaringBitmap 1.2.1), Maven-resolved dependencies, and
deterministic Java-generated goldens. Keep test-only APIs behind
#[cfg(test)].
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant