Repository navigation
Conversation
Since #110 turned off git's ignore rules for the walk, it entered every directory on disk outside the top-level ignore_dirs, and dropped the untracked files in them one at a time. A large gitignored directory, like build output or nested git worktrees, cost a full traversal even though nothing in it could end up in the project. The walk now skips a directory unless it holds a tracked file, could hold a team file (those are read before they're tracked, #113), or is the project root. A vendored gem directory with no tracked files no longer gets a CODEOWNERS line, so that line no longer depends on the checkout. Fixes #152
git ls-files lists a submodule as one path, without its contents, so the tracked directory set only held its parents and the walk skipped it. A vendored gem that's a submodule then lost its CODEOWNERS line. The set now holds each tracked path as well as the directories above it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #152.
What
Why
Since #110 turned off git's ignore rules for the walk, it enters every directory outside the top-level
ignore_dirsand drops untracked files one at a time. On a developer checkout with large gitignored directories, the walk took most of the run time. #152 has the details.How
tracked_files::tracked_directoriesbuilds the set of tracked paths and the directories above them. It uses thegit ls-filesoutput the walk already reads. That output lists a submodule as one path, so submodules stay in the set.walk_entersalso keeps any directory inside or above the literal start of a team file glob, such asconfig/teamsforconfig/teams/**/*.yml. Team files are read before they're tracked (Consider new, untracked team config files during validate/generate #113). A glob that starts with a wildcard has an empty literal start, so the walk enters every directory, as it does today.git ls-filesfails, so there's nothing to skip by and the walk is unchanged.Behavior change
A vendored gem directory with no tracked files no longer gets a CODEOWNERS line. Today that line appears only in checkouts that have the directory, so
validatecan pass on one machine and fail on another. Staged files count as tracked, so a gem that's being added gets its line once it'sgit added.Measured
This uses the checkout from #152: 2.0M entries on disk and 145k tracked files, on macOS with 12 cores. The generated CODEOWNERS is byte-identical to 0.7.0's, and to the file committed in that checkout.
The table comes from a prototype of this change. It shows medians of 5 runs, ordered 0.7.0, branch, branch, 0.7.0. The machine was busy, so each case shows both medians.
--no-cache validatevalidate, warm cachevalidate <one file>, warm cacheA single run of the final build agrees:
--no-cache generate: 14.9s down to 5.8s.--no-cache validate: 16.4s down to 5.9s.I didn't measure a clean checkout with this build. A clean checkout has little to skip, so CI times shouldn't change much.
Tests
tracked_directories,literal_glob_prefixandwalk_enters.tests/untracked_directory_test.rs, where each test fails when its half of the change is removed:main.config/teamsis still read. This fails without the team file exemption.cargo test(336 passed, 3 ignored),cargo clippy --all-targets --all-features -- -D warningsandcargo fmt -- --check.