Skip to content

Keep jspecify off the worker classpath in every build mode - #9267

Draft
reiabreu wants to merge 2 commits into
apache:masterfrom
reiabreu:jspecify-worker-classpath
Draft

reiabreu wants to merge 2 commits into
apache:masterfrom
reiabreu:jspecify-worker-classpath

Conversation

@reiabreu

@reiabreu reiabreu commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

The 3.2.0 candidate has two binary tarballs built from the same tag, and jspecify-1.0.1.jar sits in lib/ in one and in lib-common/ in the other. This makes both give the same layout.

Upgrade notes

storm-client no longer brings org.jspecify:jspecify as a transitive compile dependency (it came through the dependency-reduced pom of storm-shaded-deps). If your topology code uses org.jspecify annotations, declare the dependency yourself, for example org.jspecify:jspecify:1.0.1.

In the binary distribution, jspecify is in lib/ (daemon classpath) in every build, so it is no longer on the worker classpath. It is an annotations-only library and is not needed at runtime.

Details

Cause

  • storm-shaded-deps bundles its libraries with the shade plugin, and the libraries are declared <optional>true</optional>. The plugin's dependency-reduced pom promotes their transitive dependencies and keeps the optional flag.
  • jspecify is the only promoted dependency it left non-optional. error_prone_annotations, j2objc-annotations and failureaccess are optional or daemon-only in both builds.
  • A module that resolves storm-shaded-deps from the Maven repository reads that reduced pom, so jspecify lands on the worker classpath, and dedup-libs.py moves it to lib-common. In a reactor build, as release:perform does, Maven takes the model of storm-shaded-deps from the reactor (ReactorReader.findModel returns MavenProject.getModel(), checked in the bytecode of Maven 3.9.16), so the reduced pom written by the shade step is never read. That model declares Guava as optional and does not declare jspecify, so the worker classpath does not get it and it stays in lib.

Change

  • Declare org.jspecify:jspecify as <optional>true</optional> in storm-shaded-deps/pom.xml, like the libraries around it. The version is already managed in the root pom. jspecify then stays in lib/ in every build mode, as the other Guava annotation jars do.

Verified on a scratch checkout of the v3.2.0 tag with this change, using an isolated local repository so the real one was not touched:

  • The reduced pom of storm-shaded-deps now lists jspecify with optional=true.

  • storm-client-bin built in a reactor and built on its own: lib-worker has 39 jars in both, no jspecify, with identical names and SHA-256 hashes. Before the change the standalone build had 40.

  • final-package daemon jars (copy-dependencies, runtime): 88 jars in both modes, unchanged from before, jspecify included.

  • Workers without jspecify: the full 3.2.0 distribution with jspecify moved from lib-common/ to lib/ (the layout this change produces) passed the smoke test (WordCount, Lambda, MultiThreadWordCount and the Kafka example, 0 failed tuples), and no worker log had a NoClassDefFoundError or ClassNotFoundException. jspecify has only 4 annotation types, and storm-client does not reference them.

Not verified

  • I did not rebuild the full tarballs in both modes and compare them. Build timestamps in the UI files and the branch= line in the version-info files will still differ between any two builds.
  • I did not run this through the smoke tests.

🤖 Generated with Claude Code

reiabreu and others added 2 commits November 8, 2026 22:41
storm-shaded-deps is built with the shade plugin, whose dependency-reduced pom
lists jspecify as a compile dependency. A module that resolves it from the local
Maven repository sees that pom, so jspecify is copied into lib-worker, and
dedup-libs.py moves it to lib-common. In a reactor build, as release:perform
does, Maven reads the source pom, where jspecify is only reachable through JUnit
as a test dependency, so jspecify stays in lib.

The same tag therefore gave two layouts: jspecify in lib/ in the tarball built by
release:perform and in lib-common/ in a rebuild from the tag. Declaring jspecify
in storm-client-bin, with its version managed in the root pom, gives the same
lib-worker in both.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…torm-client-bin

The shade plugin's dependency-reduced pom promotes the transitive dependencies of
the shaded libraries and keeps their optional flag. jspecify was the only one it
left non-optional, so a module that resolves storm-shaded-deps from the Maven
repository got jspecify on its worker classpath, while a reactor build reading the
source pom did not. error_prone_annotations, j2objc-annotations and failureaccess
are daemon-only in both.

Declare jspecify optional in storm-shaded-deps, like the libraries it sits next to,
and drop the dependency this pull request added to storm-client-bin. jspecify is
then in lib/ only, whichever way the distribution is built.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@reiabreu reiabreu changed the title Put jspecify on the worker classpath in every build mode Keep jspecify off the worker classpath in every build mode Nov 8, 2026
@reiabreu reiabreu added this to the 3.3.0 milestone Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant