Skip to content

[macOS x64] SymSgdNative segfaults in OpenMP barrier with duplicate runtime images #7933

Open

Description

@svick

Summary

A macOS x64 Helix run of Microsoft.ML.Tests segfaulted while NonDefaultColNamesBinaryClassificationOnnxConversionTest was training SymSGD. The captured native stack enters SymSGD's OpenMP parallel region and crosses from libomp.dylib into libiomp5.dylib's __kmp_suspend_64. This is consistent with the Helix payload loading two images of the same OpenMP library under different names. The stack is strong evidence of the interaction, but the precise dyld symbol-binding mechanism and a product-level reproducer still need confirmation.

Reproducible CI evidence

The test was identified from the crash report's thread stacks, not just from the last Starting test: line in the console (xUnit runs tests concurrently).

Crash evidence

The crash report marks native thread 0xc1091 as crashing. Its relevant frames are:

libiomp5.dylib!__kmp_suspend_64<false, true>
libomp.dylib!kmp_flag_64<false, true>::wait
libomp.dylib!__kmp_hyper_barrier_release
libomp.dylib!__kmp_fork_barrier
libomp.dylib!__kmp_launch_thread
libomp.dylib!__kmp_launch_worker

Another captured thread shows the application path entering this OpenMP region:

libiomp5.dylib!__kmp_suspend_64<false, true>
libomp.dylib!kmp_flag_64<false, true>::wait
libomp.dylib!__kmp_hyper_barrier_gather
libomp.dylib!__kmp_join_barrier
libomp.dylib!__kmpc_fork_call
libSymSgdNative.dylib!InitializeState
libSymSgdNative.dylib!LearnAll
SymbolicSgdLogisticRegressionBinaryTrainer.TrainCore
OnnxConversionTest.TestPipeline
Microsoft.ML.Tests.OnnxConversionTest.NonDefaultColNamesBinaryClassificationOnnxConversionTest

The dump lists the two libraries as separate loaded images at bases 0x1a9c38000 (libiomp5.dylib) and 0x1abe17000 (libomp.dylib).

Relevant runtime setup

At the source commit above:

  • eng/helix.proj sets KMP_DUPLICATE_LIB_OK=TRUE, copies Homebrew's libomp.dylib into the test payload, and on macOS x64 copies that file again as libiomp5.dylib.
  • SymSgdNative/CMakeLists.txt links SymSGD against Homebrew's libomp on macOS.
  • The loaded libiomp5.dylib name is needed by MKL's libMklImports.dylib.

This makes the duplicate-runtime setup a plausible cause, but it does not by itself prove why dyld bound this particular call across the two images.

Requested investigation

  1. Reproduce or otherwise confirm the crash and determine whether loading the same OpenMP binary under both libomp.dylib and libiomp5.dylib causes the cross-image call/state mismatch.
  2. Adjust the macOS x64 Helix native-library setup so MKL, SymSGD, and other native dependencies use a compatible single OpenMP runtime, without changing Linux's separate fix in Use MKL's libiomp5 for SymSgdNative OpenMP on Linux x64 #7927.
  3. Consider improving eng/testing/runTests.sh: when a test process exits due to a native crash, print the crashing thread's native frames from the generated crash-report JSON to the console. This would make future failures easier to identify and provide a stable Build Analysis signature even when Helix artifacts are no longer available.

Related context: #7061, #7734, and #5569. #7927 addresses a Linux x64 OpenMP-linking issue and does not change macOS.

Note

This issue draft was prepared with GitHub Copilot; the root-cause mechanism above is a hypothesis to verify.

Activity

  1. added
    bugSomething isn't working
    untriagedNew issue has not been triaged
    area-NativeNative libraries, platform support, and architecture-specific code
    on Nov 20, 2026
  2. MihuBot commented on Nov 20, 2026

    @MihuBot

    I'm a bot. Here is a possible related and/or duplicate issue (I may be wrong):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-NativeNative libraries, platform support, and architecture-specific codebugSomething isn't workinguntriagedNew issue has not been triaged

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions