Skip to content

RangeSet: limit index allocations to reduce excessive memory use - #743

Draft
degremont wants to merge 1 commit into
clustershell:masterfrom
degremont:codex/issue-742-rangeset-limit
Draft

degremont wants to merge 1 commit into
clustershell:masterfrom
degremont:codex/issue-742-rangeset-limit

Conversation

@degremont

Copy link
Copy Markdown
Collaborator

Expanding 100 million entries with nodeset can be very slow and consume enough memory to make the CLI unusable.

Replace the range-size assertion with a runtime check that rejects additions when the current RangeSet size plus the incoming index count reaches 100 million. The check accounts for the range step and conservatively counts overlapping indexes again. Document the limit in the class docstring and add library and CLI tests.

Validation: 298 relevant tests passed, plus four focused checks under python -O.

Fixes #742.

Expanding 100 million entries with nodeset was very slow and consumed
enough memory to make the CLI unusable.

Replace the range-size assertion with a runtime check that rejects
additions when the current set size plus the incoming index count
reaches 100 million, accounting for the range step. Document the limit
and add tests.

Signed-off-by: Aurelien Degremont <aurelien.degremont@lelfe.net>
@degremont
degremont force-pushed the codex/issue-742-rangeset-limit branch from 074b466 to 04734b4 Compare November 17, 2026 17:53

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nodeset crashes the whole process when nodeset syntax too long

1 participant