Skip to content

Enable chunck sizes based on rows (max 1MB by default). - #119

Merged
jorgensd merged 1 commit into
mainfrom
dokken/vtkhdf-chunk-size
Nov 18, 2026
Merged

jorgensd merged 1 commit into
mainfrom
dokken/vtkhdf-chunk-size

Conversation

@jorgensd

Copy link
Copy Markdown
Member

@ottarph raised this issue. Now chunking is set so it contains full rows. Then a read touches at max 2 extra chunks in paralllel, which is not bad. Increases read performance in serial by a factor of four compared to standard chunking on a 120 K dataset.

@jorgensd

Copy link
Copy Markdown
Member Author

Did some checks on larger chunk sizes for uncompressed data, but didn't see any clear benefits of it.

@jorgensd
jorgensd merged commit aec4b01 into main Nov 18, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant