Repository navigation
Build and test conda packages and wheels - #204
Conversation
|
|
||
| concurrency: | ||
| group: intel-ci-${{ github.ref }} | ||
| cancel-in-progress: true |
There was a problem hiding this comment.
With cancel-in-progress: true keyed on github.ref, two quick merges cancel the first commit's whole run. Jenkins, which waits for that run, gets no packages for it.
| cancel-in-progress: true | |
| cancel-in-progress: ${{ github.event_name == 'pull_request' }} |
There was a problem hiding this comment.
Applied in 6006c16, so a push to master no longer stops the run before it. Every tag is its own ref, so tag runs never cancelled each other either.
| numpy: | ||
| - '2' |
There was a problem hiding this comment.
Agreed, this should land after #202. With the current pin_compatible('numpy', min_pin="x.x"), the 3.15 package built here requires numpy >=2.5,<3.0a0 at run time, which #202 fixes. Once #202 merges I will rebase: numpy: '2' will come from #202, and 3.10 will leave the python: list along with its matrix.
| @@ -1,3 +1,16 @@ | |||
| # The Python versions to build, one per line and without selectors: ci/matrix.py | |||
| # reads them for .github/workflows/intel-ci.yml. NumPy 2, as in conda-forge's | |||
| # global pinning; conda-package-cf.yml passes its own with --python and --numpy. | |||
| "--artifacts", required=True, type=Path, help="the downloaded artifact" | ||
| ) | ||
| parser.add_argument( | ||
| "--work", required=True, type=Path, help="an empty scratch directory" |
There was a problem hiding this comment.
hooks.py doesn't check that --work is empty, and may install conda-index into the user's base environment.
There was a problem hiding this comment.
Fixed in b758764. hooks.py now refuses a --work that is not an empty directory, and it writes the local channel's repodata.json itself, so it installs nothing into base.
|
|
||
| - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | ||
| with: | ||
| python-version: "3.12" |
There was a problem hiding this comment.
Is there any issue to use the newer python?
There was a problem hiding this comment.
No issue: the job only runs ci/matrix.py, which needs nothing beyond the standard library. Moved to 3.14 in 6006c16.
| activate-environment: "" | ||
| conda-remove-defaults: "true" | ||
|
|
||
| - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 |
There was a problem hiding this comment.
Alternatively we can drop setup-python and run with shell: bash -el {0} with base activated.
bash is available on Windows runners too, and setup-miniconda supports this shell there. The step bodies become plain python ci/build.py … and python ci/hooks.py …. The $CONDA path code and hooks.py's re-run of itself would then only matter for local use.
There was a problem hiding this comment.
Done in 6006c16: build and test run in bash -el {0} with base active, without setup-python. On Windows the new steps work: conda-build installs, the package builds and the imports pass. This run is still red on Windows because conda-forge's pytest 9.2.0, published today, dropped colorama again, so the recipe's pip check fails (conda-forge/pytest-feedstock#208). I will re-run once a fixed build is out.
There was a problem hiding this comment.
Re-ran after conda-forge published the fixed pytest build: the run is green on Linux and Windows, test jobs included.
| @@ -0,0 +1,135 @@ | |||
| # Intel Python CI (gh-ci) for mkl_random: builds the conda-forge recipe, conda-recipe-cf, for every | |||
There was a problem hiding this comment.
The same PR is open in three repos at once.
Have you considered moving common things (scripts, workflow) to some common repo like https://github.com/IntelPython/devops-tools ?
That would help to reduce the duplication.
There was a problem hiding this comment.
Yes, we considered it. Duplicated code is a known consequence of the new CI's design, where each repository carries everything its CI needs, and the design's other benefits make up for it.
Build and test steps run in a bash login shell with conda base active, so they call ci/build.py and ci/hooks.py directly and need no setup-python. Pushes to the default branch no longer cancel each other. Same as IntelPython/mkl_random#204.
Build and test steps run in a bash login shell with conda base active, so they call ci/build.py and ci/hooks.py directly and need no setup-python. Pushes to the default branch no longer cancel each other. Same as IntelPython/mkl_random#204.
2ecff7b to
8b5d74a
Compare
vchamarthi
left a comment
There was a problem hiding this comment.
Went through this on the branch. The PR description looks stale: conda-recipe-cf is deleted, macOS and the wheels are built too, and Jenkins is enabled in dry run. There are also no AGENTS.md notes in the diff. Also, conda-package.yml is deleted, so the Intel channel build is no longer tested anywhere. Details inline.
| - wheel >=0.41.3 | ||
| run: | ||
| - python | ||
| - {{ pin_compatible('numpy', min_pin="x.x", max_pin="x") }} |
There was a problem hiding this comment.
pin_compatible('numpy', min_pin="x.x", max_pin="x") resolves to >=2.2,<3.0a0 / >=2.5,<3.0a0 in the build logs, depending on the build. That excludes numpy 2.0 to 2.4 at runtime for the first one, and the minimum follows whatever numpy was in the host env. Is that the intent? A fixed lower bound like the old >=1.26.4 would be easier to reason about.
There was a problem hiding this comment.
Not intended, agreed. #202 fixed it on master, and this is rebased on it: the run requirement is a fixed numpy >=1.26.4, as in pyproject.toml, and numpy's run_exports add the upper bound.
|
|
||
| test: | ||
| name: Test ${{ matrix.name }} Python ${{ matrix.python }} | ||
| needs: [matrix, build] |
There was a problem hiding this comment.
needs: [matrix, build] means a macOS conda failure skips test and so blocks the release, same for the wheel job at :155. If macOS is meant to gate releases that is fine, just checking it is on purpose.
There was a problem hiding this comment.
Yes, a package that fails to build on any platform, macOS included, fails the run. Skipping every test leg wasn't intended, though: in 75f0985 the test jobs use if: ${{ !cancelled() && needs.matrix.result == 'success' }}, as mkl-service does, so every package that built still gets tested and only the failed leg's test fails. Same for test-wheel.
There was a problem hiding this comment.
Changed back since: the test jobs wait for every build again, so a failed build starts no test runner for a run that has failed anyway.
| # macOS runs on Intel runners, as MKL has no Apple silicon build; Jenkins ignores it. | ||
| build: | ||
| name: ${{ matrix.name }} Python ${{ matrix.python }} | ||
| needs: matrix |
There was a problem hiding this comment.
This builds conda-recipe on every push and PR, and I don't see conda-package-cf.yml removed, so the same package may be built twice. Worth deleting or limiting the old workflow if it is now redundant.
There was a problem hiding this comment.
Agreed, it was redundant. 75f0985 removes conda-package-cf.yml; intel-ci.yml builds and tests the recipe.
| """ | ||
| Intel Python CI (gh-ci) test for mkl_random. | ||
|
|
||
| GitHub Actions, through ci/hooks.py, and Intel's Jenkins both run this with one |
There was a problem hiding this comment.
Nit: the docstring says the test runs against a conda package, but the wheel job uses it too. Might be worth saying so.
There was a problem hiding this comment.
Fixed: it says a conda package or a wheel now.
cb71ba5 to
75f0985
Compare
|
Thanks for going through it. The description is updated: it covers macOS, the wheels, the current pins and the test-job change, and links a run on the current head. On AGENTS.md: mkl_random has none on |
c7f5e40 to
1653818
Compare
|
|
||
| [tool.cibuildwheel.linux] | ||
| # The hardening flags conda-forge's compilers set for the conda packages | ||
| environment = { CFLAGS = "-fstack-protector-strong", CPPFLAGS = "-D_FORTIFY_SOURCE=2", LDFLAGS = "-Wl,-z,relro -Wl,-z,now" } |
There was a problem hiding this comment.
mklrand is built as C++, so CFLAGS doesn't reach it: the Linux wheel's .so has no stack protector, unlike the conda one. Could CXXFLAGS = "-fstack-protector-strong" be added too?
There was a problem hiding this comment.
Added: the Linux wheels build the C++ sources with -fstack-protector-strong too.
| - numpy | ||
| - pip | ||
| - wheel >=0.41.3 | ||
| run: |
There was a problem hiding this comment.
mkl is no longer in the run requirements. conda-forge's mkl-devel adds it through run_exports, but the Intel channel's doesn't, so a build against the Intel channel won't require mkl. Could pin_compatible('mkl') stay, as in mkl_fft and mkl-service?
There was a problem hiding this comment.
Restored pin_compatible('mkl') in the run requirements, as mkl_fft and mkl-service have it.
| # which is on CMAKE_PREFIX_PATH thanks to the compiler activation scripts. | ||
| # Do not export MKLROOT: MKLConfig.cmake then searches $MKLROOT/include with | ||
| # NO_DEFAULT_PATH and hard-errors if mkl_version.h is not there. | ||
| $PYTHON -m pip install --no-build-isolation --no-deps . |
There was a problem hiding this comment.
Without -v the build log has no meson or compiler output. Could we add -v -Ccompile-args=-v, here and in bld.bat?
There was a problem hiding this comment.
Added -v -Ccompile-args=-v to build.sh and bld.bat.
| conda-remove-defaults: "true" | ||
|
|
||
| - name: Install conda-build | ||
| run: conda install -y -q -n base --override-channels -c conda-forge conda-build |
There was a problem hiding this comment.
mkl-service pins conda-build=26.9.1 here. Should we pin it too?
There was a problem hiding this comment.
Pinned to 26.9.1, as in mkl-service.
| wheel: "mkl_random {platform} Wheels Python {python}" | ||
| test: | ||
| requires: [pytest] | ||
| benchmark: |
There was a problem hiding this comment.
intel-ci.yml doesn't run benchmark.py or read this section. Could ci/README.md mention that, like in dpnp#3151?
There was a problem hiding this comment.
ci/README.md now says so, in dpnp#3151's words.
1653818 to
ae7d3e0
Compare
| c_compiler: # [linux] | ||
| - gcc # [linux] | ||
| cxx_compiler: # [linux] | ||
| - gxx # [linux] | ||
| cxx_compiler_version: # [linux] | ||
| - '14' # [linux] | ||
| c_stdlib: # [linux] | ||
| - sysroot # [linux] | ||
| c_stdlib_version: # [linux] | ||
| - '2.28' # [linux] | ||
| - '2.17' # [linux] | ||
| c_compiler: # [osx] | ||
| - clang # [osx] | ||
| cxx_compiler: # [osx] | ||
| - clangxx # [osx] |
There was a problem hiding this comment.
The compilers aren't pinned, so the build got gcc 16.2 and clang 23.1, and the packages now need libstdcxx >=16 and libcxx >=23. The feedstock's packages need >=15 and >=21. Could we pin c_compiler_version and cxx_compiler_version to 15 on Linux and 21 on macOS, like mkl-service and mkl_umath?
There was a problem hiding this comment.
Pinned to 15 on Linux and 21 on macOS, for C and C++.
| - "3.10.* *_cpython" | ||
| - "3.11.* *_cpython" | ||
| - "3.12.* *_cpython" | ||
| - "3.13.* *_cp313" | ||
| - "3.14.* *_cp314" | ||
| - "3.14.* *_cp314t" | ||
| - "3.15.* *_cp315" |
There was a problem hiding this comment.
3.10 is still here, though you planned to drop it after #202, and conda-forge no longer builds it. mkl-service has no 3.10 but has 3.15t, which our wheels already build. Should we do the same here?
There was a problem hiding this comment.
Done: the conda packages build 3.11 to 3.15 plus 3.14t and 3.15t, as mkl-service's do.
| # Tags keep them 30 days, pull requests and pushes to master 3. | ||
| retention-days: ${{ startsWith(github.ref, 'refs/tags/') && 30 || 3 }} |
There was a problem hiding this comment.
Pushes to master now keep packages for 3 days instead of 90. Could they keep them 30 days, as in dpnp#3151? Same for the wheels at :154.
| # Tags keep them 30 days, pull requests and pushes to master 3. | |
| retention-days: ${{ startsWith(github.ref, 'refs/tags/') && 30 || 3 }} | |
| # Tags and pushes to master keep them 30 days, pull requests 3. | |
| retention-days: ${{ github.event_name == 'push' && 30 || 3 }} |
There was a problem hiding this comment.
Done, for the wheels too.
| | `test.py` | Runs the installed package's test suite with pytest. | | ||
| | `benchmark.py` | Runs the asv suite in `../benchmarks/` against the installed package. | | ||
|
|
||
| The scripts need nothing beyond the Python standard library and the tools each one's docstring names. macOS packages build and test on Intel runners, as MKL has no Apple silicon build. |
There was a problem hiding this comment.
hooks.py also needs conda, conda-package-handling and ruamel.yaml, and test.py needs pytest. Maybe use the dpnp#3151 wording:
| The scripts need nothing beyond the Python standard library and the tools each one's docstring names. macOS packages build and test on Intel runners, as MKL has no Apple silicon build. | |
| The scripts need the Python standard library and the tools each one's docstring names. macOS packages build and test on Intel runners, as MKL has no Apple silicon build. |
There was a problem hiding this comment.
Took that wording.
| line-length = 80 | ||
|
|
||
| [tool.cibuildwheel] | ||
| build-verbosity = 1 |
There was a problem hiding this comment.
The wheel logs show no compiler commands, unlike the conda logs now. Could we add config-settings = { compile-args = "-v" } here, and compile-args = "-v" next to setup-args on Windows, since that table replaces this one?
There was a problem hiding this comment.
Added: compile-args = "-v" on Linux, and next to setup-args on Windows. The Linux wheels also link with -Wl,--disable-new-dtags now, as in mkl_fft#400, so they get RPATH.
ae7d3e0 to
cdf8220
Compare
|
|
||
| test: | ||
| name: Test ${{ matrix.name }} Python ${{ matrix.python }} | ||
| needs: [matrix, build] |
There was a problem hiding this comment.
This undoes the fix from vchamarthi's thread: one failed build leg, macOS included, again skips the tests of every package that did build. mkl-service keeps it, and antonwolfy asked for the same in dpnp#3151. Could we restore it here and in test-wheel (:156)?
| needs: [matrix, build] | |
| needs: [matrix, build] | |
| # Test every package that built, even when another build failed | |
| if: ${{ !cancelled() && needs.matrix.result == 'success' }} |
There was a problem hiding this comment.
This one is intended. A failed build already fails the run, so the test jobs wait for every build and start no runners for a run that can't go green. That keeps those runners free for other runs in the queue. Once the build is fixed, the next push tests everything.
cdf8220 to
5fbb313
Compare
Adds a ci/ directory and .github/workflows/intel-ci.yml. The workflow builds
conda-recipe/ with conda-build 26.9.1 and the wheels with cibuildwheel, for
every platform and Python version the recipe and pyproject.toml declare, and
tests each package in a clean environment with ci/test.py.
conda-recipe/ is now the only recipe, the conda-forge one that was
conda-recipe-cf/; the Intel-channel recipe, conda-package.yml and
conda-package-cf.yml are removed, as intel-ci.yml builds and tests the same
recipe; its test jobs wait for every build, so a failed build starts none.
conda_build_config.yaml now lists the Python versions, 3.11 to 3.15 plus
3.14t and 3.15t, as mkl-service's does, and numpy: '2', as conda-build
otherwise defaults to 1.26, and gcc and g++ 15 on Linux, clang 21 on macOS.
The recipe requires pin_compatible('mkl'), as mkl_fft's does, since an
mkl-devel without run_exports adds no mkl requirement, and builds with pip
-v and -Ccompile-args=-v, so the log shows meson's and the compiler's
output.
[build-system] requires mkl-devel, and [tool.gh-ci] pin-compatible maps mkl
to it: ci/build.py adds Requires-Dist mkl >= the mkl-devel version python -m
build installed, < its next major, the way conda's run_exports pin it, and
fails a build that logs no version. It takes METADATA's line ending from the
headers. The repair commands exclude MKL's libraries by wildcard, and the Linux
wheels are tagged manylinux_2_28 only, like the mkl wheel.
The Linux wheels build with the hardening flags conda-forge's compilers set:
-fstack-protector-strong in CFLAGS and CXXFLAGS, as mklrand is C++,
-D_FORTIFY_SOURCE=2 and -Wl,-z,relro -Wl,-z,now, plus
-Wl,--disable-new-dtags, so they get RPATH as the conda packages do. The
wheel builds pass compile-args=-v. ci/build.py refuses a wheel whose ELF
files lack full RELRO. It keeps an RPATH or RUNPATH entry only when it is
$ORIGIN or a directory under it, as meson-python keeps the isolated build
env's absolute lib directory, and refuses a file left with none. It rewrites
the wheels in scratch, so a refused wheel never reaches --output, and
refuses two built packages with the same file name, which would overwrite
each other there.
conda-build gets --extra-meta remote_url=... sha=... when GitHub Actions
sets GITHUB_SERVER_URL, GITHUB_REPOSITORY and GITHUB_SHA, so each conda
package records the commit it was built from in info/about.json.
ci/hooks.py tests each wheel in a venv made from conda-forge's Python, as pip
users install it. Every python: entry in conda_build_config.yaml names its
ABI, which ci/matrix.py requires. ci/matrix.py reads x86 as any x86 CPU and
osx64 as undefined, as conda-build does, and refuses a conda-build skip that
uses osx64. A pull request also builds and tests the newest free-threaded
version, and every push and tag run gets a concurrency group of its own.
5fbb313 to
183a1f0
Compare
vlad-perevezentsev
left a comment
There was a problem hiding this comment.
No more comments from my side
LGTM
Thanks @jharlow-intel for all the work on this!
Adds
ci/and.github/workflows/intel-ci.yml, which build the conda packages for Linux, Windows and Intel macOS and the wheels for Linux and Windows, on every declared Python, and test each in a clean environment.conda-recipe/is now the only recipe: the conda-forge one that wasconda-recipe-cf/.conda-package.ymlandconda-package-cf.ymlare removed, asintel-ci.ymlbuilds and tests the recipe.python:list this adds toconda_build_config.yaml. The recipe requiresmklthroughpin_compatible('mkl'), as mkl_fft's does, and builds with gcc and g++ 15 on Linux, clang 21 on macOS, andpip -v -Ccompile-args=-v; the wheel builds passcompile-args=-v. The wheels follow thepyproject.tomlclassifiers: 3.10 to 3.15 plus 3.14t and 3.15t.mklfrom the mkl-devel version they were built with up to its next major, through[tool.gh-ci] pin-compatible, whichci/build.pyadds to each wheel. The Linux wheels build with conda-forge's hardening flags and--disable-new-dtags, so they getRPATHas the conda packages do,-fstack-protector-stronginCXXFLAGStoo, as mklrand is C++, andci/build.pyrefuses a wheel without full RELRO or with a library search path outside$ORIGIN.meson.buildaddsLibraryto CMake's search path on Windows, where mkl-devel puts its CMake config.ci/test.pyrunspytest --pyargs mkl_randomon the installed package. The tests'hashlib.md5calls passusedforsecurity=False.masterand tags build them all.info/about.json.ci/benchmark.pyruns the asv suite inbenchmarks/against an installed package, for running the benchmarks outsideintel-ci.yml, which doesn't run it: one quick sample of each benchmark for a pull request, a full measurement for a tag.Miniforge is pinned to 26.7.2-0, as 26.9.4-0 makes
setup-miniconda'sconda initfail on Windows runners.Validation: run 44579125738 on
cdf8220builds and tests the reduced pull request matrix on Linux, Windows and macOS. Not run yet: the full matrix, which onlymasterand tags build.