Repository navigation
[c] Expose structured read types for nested projection - #1389
Open
suxiaogang223 wants to merge 1 commit into
Open
suxiaogang223 wants to merge 1 commit into
suxiaogang223 wants to merge 1 commit into
Conversation
Fixes apache#1385. Copy Paimon DataField JSON into the C ReadBuilder and propagate it to scans and reads with core projection precedence.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #1385
Native consumers can currently select root column names through C but cannot pass the structured read types already supported by Rust core. Add a C ReadBuilder entry point for Paimon DataField JSON so callers such as Doris can request nested ROW, ARRAY-element, and MAP-value projections with their field identities intact.
Brief change log
paimon_read_builder_with_read_type_json, copying a JSON array of DataFields into the builder and forwarding it to scan planning and reading.[]selects zero columns, and decoding errors preserve the previous selection.Tests
cargo +1.95.0 test --locked -p paimon-c --lib: 99 passed.-D warnings -A clippy::nonminimal_bool. Strict Clippy encounters the same six pre-existingnonminimal_boolerrors reproduced on unmodified main; no source lint suppressions were added.cargo +1.95.0 fmt --all -- --checkandgit diff --check.cc -std=c11 -fsyntax-only; the new function is present.Local Cargo builds used debug info disabled and incremental compilation disabled. The new end-to-end fixture covers Parquet; other formats reuse the existing core read-type path and were not newly benchmarked.
API and Format
Additive C function; existing exported structs and signatures remain unchanged. The input is the existing Paimon DataField JSON representation, not a new storage format. Explicit read types configure the read without mutating the stored table schema.
Documentation
Add a structured-projection example and configuration/ownership rules to the C integration guide.
AI assistance: Codex helped develop the implementation, tests, and this description.