Skip to content

29-rc: add checksum verification - #591

Merged
yosifkit merged 8 commits into
docker-library:masterfrom
thaJeztah:use_checksums
Nov 11, 2026
Merged

yosifkit merged 8 commits into
docker-library:masterfrom
thaJeztah:use_checksums

Conversation

@thaJeztah

Copy link
Copy Markdown
Contributor

This probably fails because we only have the checksums for v29.10.0-rc.1, not for the non-rc releases, but let's see how it goes.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@thaJeztah

Copy link
Copy Markdown
Contributor Author

This probably fails because we only have the checksums for v29.10.0-rc.1, not for the non-rc releases, but let's see how it goes.

Pushed some commits to make it conditional, and CI looks to like it, so can squash if we want to keep that.

Comment thread 29-rc/cli/Dockerfile
Comment on lines 32 to +34
'x86_64') \
url='https://download.docker.com/linux/static/test/x86_64/docker-29.10.0-rc.1.tgz'; \
sha256='d023bcab935f045548c5df79ea96d1f3fccbc091cf9a8347139154859f4767b8'; \

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do have separate downloads now for the CLI and Engine, so we could use the CLI packages here BUT the engine package doesn't contain the containerd binaries, and we do not (yet) have a separate download for containerd (we still need the ctr and shim binaries); https://download.docker.com/linux/static/test/x86_64/

Image

@tianon tianon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Niiiice, I'm so excited to let this TODO finally die ❤️

I've made some code suggestions that should hopefully let us get rid of all changes to 29/ so it stays as-is until we swap over to GA -- I've also done some code golfing suggestions that I wouldn't blame you one bit for not following, and I'm happy to take over from here if you'd rather. 🙇 👍

Comment thread Dockerfile-windows-servercore.template Resolved
Comment thread Dockerfile-windows-servercore.template Resolved
Comment thread shared.jq Outdated
@@ -58,7 +58,9 @@ def download(opts):
wget -O \(opts.target | @sh) \"$url\";"
,
if $notSha256 then "" else "

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if $notSha256 then "" else "
if $notSha256 or ($sha256 == null) then "" else "

@thaJeztah thaJeztah Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This didn't work; not exactly sure why; it complained about the $sha255 not being defined I think 🤔 (need to look closer)

edit; error was this;

./apply-templates.sh 29
processing 29/cli ...
jq: error: $sha256 is not defined at shared.jq, line 60:
	if $notSha256 or ($sha256 == null) then "" else "
jq: 1 compile error

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, down here it would need to be .[opts.sha256Key]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh shoot, or actually more complicated than that because that's per-arch. Give me a sec.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, it's not perfect, but this is the right idea:

diff --git a/shared.jq b/shared.jq
index 372d77d..ed950bb 100644
--- a/shared.jq
+++ b/shared.jq
@@ -57,11 +57,9 @@ def download(opts):
        
        wget -O \(opts.target | @sh) \"$url\";"
        ,
-       if $notSha256 then "" else "
-       if [ -n \"${sha256:-}\" ]; then
-               echo \"$sha256 *\"\(opts.target | @sh) | sha256sum -c -;
-       fi;"
-       end
+       if opts.sha256Key and all(opts.arches[]; has(opts.sha256Key)) then "
+       echo \"$sha256 *\"\(opts.target | @sh) | sha256sum -c -;"
+       else "" end
        ] | add
        | rtrimstr(";")
        | gsub("(?<=[^[:space:]])\n"; " \\\n")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(we should probably also update versions.sh to make it so that if any() and all() don't agree here it's an error, since it doesn't make sense for us to have checksums for one arch but not all of them, but that's a separate thing and is "belt-and-suspenders" so I'm not too worried about it)

Comment thread shared.jq Resolved
Comment thread versions.sh Resolved
Comment thread versions.sh Outdated
Comment on lines +231 to +233
if checksum="$(_curl "$url.sha256")"; then
read -r sha256 filename <<< "$checksum"
if [[ ! "$sha256" =~ ^[[:xdigit:]]{64}$ ]] || [[ "$filename" != "${url##*/}" ]]; then

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm honestly not super worried about the filename (and if the format ever changes to be just the checksum, as I'd personally expect from file.sha256, we don't want to come back to this code):

Suggested change
if checksum="$(_curl "$url.sha256")"; then
read -r sha256 filename <<< "$checksum"
if [[ ! "$sha256" =~ ^[[:xdigit:]]{64}$ ]] || [[ "$filename" != "${url##*/}" ]]; then
if sha256="$(_curl "$url.sha256")"; then
sha256="${sha256%% *}"
if [[ ! "$sha256" =~ ^[[:xdigit:]]{64}$ ]]; then

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm honestly not super worried about the filename

This part definitely was my AI buddy nagging me about "should validate that as well";

and if the format ever changes to be just the checksum, as I'd personally expect from file.sha256, we don't want to come back to this code:

That's actually something I wondered while working on this; what's more common if a checksum is provided per-file? Still follow the sha256sum format (<hash> <filename>), or just a bare hash?

I could see the advantage of the current format be that a sha256sum -c <filename>.sha256 would "just work". But yeah, to capture the hash, you'd have to trim the content.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually; one thing we should fix is that currently it doesn't use binary format;

curl -fsSL https://download.docker.com/linux/static/test/x86_64/docker-engine_29.10.0-rc.1.tgz.sha256
91a4274c4c080076719bcf4633259eaccb26c6ac733f3d3990b88c9323ac525d  docker-engine_29.10.0-rc.1.tgz

I'd expect that to be

-91a4274c4c080076719bcf4633259eaccb26c6ac733f3d3990b88c9323ac525d  docker-engine_29.10.0-rc.1.tgz
+91a4274c4c080076719bcf4633259eaccb26c6ac733f3d3990b88c9323ac525d  *docker-engine_29.10.0-rc.1.tgz

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, IMO it's weird if foo.sha256 contains XXXX foo or XXXX *foo and would expect it to be just XXXX, but both ways are pretty common. Having the checksum available is what's really important.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, agreed that .. it's a bit double to have both filename.shaXX and the filename to be repeated in the file.

I guess there's something to be said for being able to use sha256sum and it to "just work" (both when creating and consuming). Static binaries all seem to be a bit under defined in conventions 😞

Comment thread versions.sh Resolved
@thaJeztah

Copy link
Copy Markdown
Contributor Author

Thanks! I'll probably take most (or all) of your suggestions. My initial motivation of this PR was as a bit of a reminder ("don't forget to update the official images now that we can") and to make sure this could all work (and no fixes needed in our packaging).

Admitted; a bunch was "paint by numbers" and "copy by example", with some of my AI buddy mixed in.

thaJeztah and others added 3 commits November 10, 2026 12:22
Co-authored-by: Tianon Gravi <admwiggin@gmail.com>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
@tianon
tianon marked this pull request as ready for review November 10, 2026 23:31
@tianon
tianon requested a review from yosifkit November 10, 2026 23:31
@yosifkit
yosifkit merged commit c879009 into docker-library:master Nov 11, 2026
8 checks passed
docker-library-bot added a commit to docker-library-bot/official-images that referenced this pull request Nov 11, 2026
Changes:

- docker-library/docker@c879009: 29-rc: add checksum verification (docker-library/docker#591)
- docker-library/docker@f9a1f03: Update 29-rc to 29.10.0-rc.1, buildx 0.38.2, compose 5.7.0
tianon pushed a commit to docker-library/official-images that referenced this pull request Nov 11, 2026
Changes:

- docker-library/docker@c879009: 29-rc: add checksum verification (docker-library/docker#591)
- docker-library/docker@f9a1f03: Update 29-rc to 29.10.0-rc.1, buildx 0.38.2, compose 5.7.0

Co-authored-by: Docker Library Bot <doi+docker-library-bot@docker.com>
@thaJeztah
thaJeztah deleted the use_checksums branch November 11, 2026 00:51
@thaJeztah

Copy link
Copy Markdown
Contributor Author

Thanks for carrying it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants