Skip to content

build: export the NML client headers again - #4785

Closed
hansfbaier wants to merge 3 commits into
LinuxCNC:masterfrom
LiNC-CNC:nml-client-headers
Closed

hansfbaier wants to merge 3 commits into
LinuxCNC:masterfrom
LiNC-CNC:nml-client-headers

Conversation

@hansfbaier

Copy link
Copy Markdown

Fixes #4774, as asked for in #4784: export the headers while NML exists,
instead of adding a client layer on top.

What

  • NML_HEADERS in src/Makefile lists the 22 headers an out-of-tree NML
    client compiles against; they are installed flat into
    $(includedir)/linuxcnc/, next to the HAL and RTAPI headers, which is the
    layout such a client has always built against.
  • The includes inside that set are written as bare names again, the way they
    were in 2.9 ("nml_type.hh", not "libnml/nml/nml_type.hh"). Installed
    flat, the prefixed form does not resolve; the source build is unchanged
    because the four directories are added to the in-tree include path.
  • docs/src/code/nml-messages.adoc gets a "Writing a client outside the
    tree" section: where the headers are, what to link, and that the set is
    exported only for as long as NML exists — no emulation promised for the
    replacement.

Test

tests/build/nml-client-headers compiles and links a client against
${HEADERS} and ${LIBDIR} only, the way linuxcnc-nml is built. It runs
in package builds and skips run-in-place ones, because there the installed
headers are the ones a test can see.

Verification

  • Full tree build at dc1e724 + this: clean.
  • The test passes in system-build mode against a staged flat install; it
    skips under runtests in a run-in-place tree, as intended.
  • make -n RUN_IN_PLACE=no install-kernel-indep lists all 22 headers next to
    the existing ones.

Scope question for review

This is the closure the current out-of-tree NML client needs, not the full
2.9 -dev set (2.9 also shipped nml_srv.hh, cms_srv.hh, shmem.hh,
timer.hh, ...). If the intent is to restore all of 2.9's set, say so and I
will extend NML_HEADERS; adding a header is a one-line change.

Since LinuxCNC#3837 the headers an out-of-tree NML client compiles against are not
installed, so linuxcnc-nml and anything else written at that level cannot
build against 2.10 (LinuxCNC#4774).  The headers a client needs are listed in
NML_HEADERS and installed flat, next to the HAL and RTAPI headers, which is
the layout an out-of-tree client has always compiled against.

The includes inside the exported set are written as bare names again, as they
were in 2.9: these headers are compiled from an installed directory, where
"libnml/nml/nml_type.hh" does not resolve.  The four directories they live in
are on the in-tree include path, so the source build is unchanged.

The set is exported for as long as NML exists.  It is not an interface the NML
replacement has to emulate; when NML goes, these headers go with it, and
docs/src/code/nml-messages.adoc says so where a client author will read it.

tests/build/nml-client-headers compiles and links a client against the
installed set only.  It runs in package builds and skips run-in-place ones,
because there the installed headers are the ones a test can see.
@grandixximo

Copy link
Copy Markdown
Contributor

@hansfbaier Thanks for the quick turnaround. Two questions.

Adding emc/nml_intf libnml/nml libnml/rcs libnml/cms to INCLUDES puts those -I flags on every compile, and #3837 removed them on purpose. I tried a layout that needs neither those flags nor the include changes: install the emc/ headers flat and keep libnml/nml, libnml/cms and libnml/rcs as subdirectories under $(includedir)/linuxcnc/. A client using "libnml/nml/nml.hh" compiled with only -I on that directory (scratch copy, compile only, not linked). The shim would change four includes, but it needs porting for master anyway. Would that work for you, or am I missing something?

interp_base.hh pulls in <boost/noncopyable.hpp>, and I see no boost dependency on the -dev package. As far as I can see the shim includes it without using it. Can it come off the list and out of the test?

@BsAtHome the docs here answer your stop-gap question: the set goes when NML goes, with no emulation. It does not split internals from externals. Is that acceptable for 2.10, or do you want the split first? If it is acceptable, this may not need Sunday's meeting.

@BsAtHome

BsAtHome commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

This is the wrong solution. You cannot just remove the include directory from the #include line. That does not work.

Exported headers need to be moved into the include directory. There is no longer any copying going on and we do not want to reintroduce that problem.

You are not "just" exporting NML headers. You are exporting a whole lot of other interface headers. Most of these are badly designed and contain both internal and external information. That is why I said that they need to get split. We do not want to publish any internals anymore. That will just give all the more problems later on.

Review: adding libnml/nml, libnml/cms and libnml/rcs to the in-tree include
path puts those -I flags on every compile, which LinuxCNC#3837 removed on purpose, and
rewriting the includes of exported headers to bare names changes headers that
are otherwise untouched.

Install the libnml headers under $(includedir)/linuxcnc/libnml/... instead,
which is the layout their includes already expect, and leave both the headers
and INCLUDES alone.  A client still needs a single include path.

interp_base.hh comes off the list: it pulls <boost/noncopyable.hpp>, which the
development package does not depend on, and a client does not need it.
@hansfbaier

Copy link
Copy Markdown
Author

@grandixximo Yes, that works, and it is better than what I had: no -I flags added to every compile, no include changes to headers that are otherwise untouched. The branch now installs libnml/{nml,cms,rcs} under $(includedir)/linuxcnc/ and leaves the emc headers flat; the test client includes "emc_nml.hh" and "libnml/nml/nml.hh" with -I on that one directory, and it compiles and links.

interp_base.hh is off the list and out of the test. It was only there because the shim includes it, and the boost include makes it a bad thing to export.

@BsAtHome Agreed that this publishes headers that mix internals and externals, and that they are not something to keep. That is what the doc section says: the set is exported for as long as NML exists and is not a promise the replacement has to emulate. 21 headers, listed in src/Makefile; no wildcard, no copies — they are installed from where they live.

On "you cannot just remove the include directory from the include line": agreed, that is why that approach is gone. The prefixed includes stay; the directories they name are now installed, so nothing in the headers changes.

The split into internal and external headers is the right end state, but it is a bigger change than this one and it would be a change to 2.10's NML headers, which the in-tree build also compiles. If the split is a condition for 2.10, this PR should wait for it. If a time-boxed set is acceptable, this is the smallest way to keep linuxcnc-nml and the other NML-level clients building.

@BsAtHome

BsAtHome commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Exported headers live in include/. No copying or separate install rules. If a header must be exported, then it must be moved to the include directory and all references in the source-tree must be changed to use the proper <> style include.

That also means that you must cleanup the headers only to export that what you need for interfacing. Just moving around with the current versions will be no better than the situation we had before, and that was a real mess. So, if the you export the headers, then you must go the whole way and start reorganizing the content as well to prevent internal data from being published.

The test harness requires either an expected file or a checkresult script;
test.sh already reports failure through its exit status.
@hansfbaier

Copy link
Copy Markdown
Author

CI was red for a mechanical reason: the harness requires an expected file or a checkresult script in every test directory, and this test had neither, so it failed with "Neither expected nor checkresult existed" on all eight architecture jobs. Added checkresult in 5f2fd2b; the test reports failure through its exit status.

On the structural point from @BsAtHome: what the branch does and does not do is unchanged — it installs 21 named headers, no wildcards and no copies, and it does not split internals from externals. Moving them into include/ and converting every reference to <> is the reorganization the 2.10 NML headers would have to take with them, and it changes code the in-tree build also compiles. That is not a thing I can responsibly add to a stopgap without the maintainers saying the split is required for 2.10. If it is, this PR should wait for that work; if a time-boxed export is acceptable, it is the smaller change. Your call, @rene-dev / @rmu75.

@BsAtHome

BsAtHome commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

On the structural point from @BsAtHome: what the branch does and does not do is unchanged — it installs 21 named headers, no wildcards and no copies, and it does not split internals from externals. Moving them into include/ and converting every reference to <> is the reorganization the 2.10 NML headers would have to take with them, and it changes code the in-tree build also compiles. That is not a thing I can responsibly add to a stopgap without the maintainers saying the split is required for 2.10. If it is, this PR should wait for that work; if a time-boxed export is acceptable, it is the smaller change.

The whole point of exporting headers is that you do it structurally sound. We have bee reorganizing that for quite some time. However, you you are reintroducing the "old ways" by (blindly) installing headers.

Even for a stopgap, the structural problem remains. We have been working on cleaning up the headers and that needs to be done, regardless of this stopgap. Therefore, cleaning it up now is far better than reintroducing "old way" bad patterns. It will haunt us all the rest of the dev cycle and beyond if we revert to old ways. I cannot approve just exporting these headers without being structurally fixed.

@rene-dev

rene-dev commented Nov 8, 2026

Copy link
Copy Markdown
Member

conclusion from our meeting on this issue: @BsAtHome will do a PR that splits the headers, so we can export them.

@hansfbaier

Copy link
Copy Markdown
Author

OK fine, good that a conclusion has been reached.

@BsAtHome

Copy link
Copy Markdown
Contributor

See #4852 for a reexport of the NML headers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Some development headers missing from make install

4 participants