Repository navigation
Ignore Cache-Control when computing the cache key - #369
Open
ahmedalbusaidy wants to merge 1 commit into
Open
ahmedalbusaidy wants to merge 1 commit into
ahmedalbusaidy wants to merge 1 commit into
Conversation
pip 26.2 stopped sending "Cache-Control: max-age=0" on Simple API index requests, so pages recorded with older clients no longer match the request and the cache falls through to the live registry. Cache-Control is a revalidation hint and doesn't change the response the cache replays, so leave it out of the key as with User-Agent. Conditional headers such as If-None-Match stay in the key because they select between a 200 and a 304. Entries recorded with Cache-Control in the key still hit through a legacy key until they are re-cached. A legacy hit isn't recorded again, and a stale legacy entry is replaced under the new key.
Author
|
The six red Smoke jobs (docker, nix, npm, npm-group-rules, npm-group-semver, python-pip-compile) fail the same way on other PRs. I compared every e2e job in this run with the Smoke run for #355 (https://github.com/dependabot/proxy/actions/runs/43385175631):
The same six jobs fail on the smoke-tests hourly run (https://github.com/dependabot/smoke-tests/actions/workflows/smoke.yml), without the proxy change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What are you trying to accomplish?
Refs dependabot/smoke-tests#617.
pip 26.2 (pypa/pip#13955) stopped sending
Cache-Control: max-age=0on Simple API index requests; it now sends it only when revalidating. I checked with real clients against a local index: pip 26.1.2 sendsCache-Control: max-age=0, pip 26.2.1 sends noCache-Control. dependabot-core moved to pip 26.2.1 on 2026-08-28 (dependabot/dependabot-core@4b2a79bb3). Since then the recorded/simple/pages no longer match: in the hourlysmoke (python-pip-compile)run,/simple/responses served from cache went from 115 to 0 on 2026-08-28, and the test now resolves transitives against live PyPI and drifts with each upstream release (smoke-tests#595 found the same key mismatch).This leaves
Cache-Controlout of the cache key, next toConnection,Accept-EncodingandUser-Agent, so a client changing its revalidation hints no longer invalidates recordings.Anything you want to highlight for special attention from reviewers?
Cache-Controlin the hash wouldn't match the new key. On a miss,OnRequestalso trieslegacyKey, which still hashesCache-Control. A legacy hit is served as before and isn't recorded again; a stale legacy entry is dropped and the fresh response is recorded under the new key. This keeps this PR's own Smoke run (which replays the smoke-tests artifacts) and every existing recording unaffected.legacyKeycan go once recordings are re-cached; happy to drop it if you'd rather re-cache straight away.Cache-Control.If-None-MatchandIf-Modified-Sinceselect between a 200 and a 304 (as discussed in Don't cache conditional/bodyless responses; serve conditional requests from cached 200 #204), so they still change the key; a test covers that.Cache-Control: no-cacheormax-age=0can now be answered from a recorded response. The cache is only enabled withPROXY_CACHE=true; in a live CLI run that means requests repeated within the same job, which already share entries when they carry noCache-Control.How will you know you've accomplished your goal?
New tests, each failing without the change:
Test_key: a request differing only inCache-Control: max-age=0gets the same key; addingIf-None-Matchstill changes it;legacyKeydiffers fromkeyonly whenCache-Controlis present.TestCache_RecordedWithCacheControlStillHits: an entry recorded under the legacy key is served, andOnResponsedoesn't record it again (fails with two entries if the key is stored before the fallback).TestCache_StaleRecordingWithCacheControlIsReplacedUnderNewKey: a stale legacy entry is replaced under the new key.TestProxyHTTPSCacheIgnoresCacheControl: through the MITM proxy, a response recorded for a request withCache-Control: max-age=0is replayed for the same request without it (upstream sees 1 request; 2 onmain).go test -race -shuffle=on -count=2 ./...passes (the commandscript/testruns, natively rather than in Docker), and the CI lint steps (yamllint .,go mod tidy -diff, gofmt,go build,go vet) are clean.Checklist