Skip to content

doc: design Lua modulefile support - #742

Open
xdelaruelle wants to merge 2 commits into
envmodules:mainfrom
xdelaruelle:lua-modfile-support
Open

xdelaruelle wants to merge 2 commits into
envmodules:mainfrom
xdelaruelle:lua-modfile-support

Conversation

@xdelaruelle

Copy link
Copy Markdown
Collaborator

Add the lua-modulefile-support design notes, which specify how modulefiles and modulercs written in Lua for Lmod are evaluated by Modules. This is documentation only: no code is changed. Related to #573.

  • Lua modulefile support is always enabled. Rules to locate Lua modulefiles and modulercs, their precedence over Tcl files and their handling in module cache follow Lmod behavior.
  • modulecmd.tcl starts one persistent Lua script on first need and exchanges messages with it over a pipe. The notes detail the start of this script, supported Lua versions, message format, protection of the communication channel, nested evaluations, evaluation end, error reporting, debugging and sandboxing.
  • Only the names of changed environment variables are sent to the Lua interpreter, which fetches their value when a modulefile reads them or runs a command. Sending values with each message makes the cost of the bridge grow with the number of loaded modules.
  • Lmod modulefile and modulerc functions are reviewed: mapping to Tcl modulefile commands, functions handled by the Lua script, behaviors specific to an evaluation mode and way to identify the module tool. hideRegex and forbidRegex map to module-hide-regex and module-forbid-regex, with Lua patterns converted into Tcl regular expressions.
  • Site customization is covered: hooks, site-specific functions and properties file are ported to their Modules counterparts.
  • Implementation stages, installation options, packaging, documentation, linting of Lua files, testsuite and performance benchmark are described.
  • Differences with Lmod are listed.

Performance figures come from a throw-away prototype of the bridge run on the modulefiles of the load2 benchmark: the bridge accounts for about 2 to 5 % of the load time, and load2_lua runs in about 500 ms against 475 ms for load2.

Signed-off-by: Xavier Delaruelle <xavier.delaruelle@cea.fr>
Extend the initial design note on Lua modulefile support into a full
specification:

* Lua modulefile support is always enabled. Rules to locate Lua
  modulefiles and modulercs, their precedence over Tcl files and
  their handling in module cache are defined after Lmod behavior.
* Tcl-Lua bridge is detailed: start of the persistent Lua script,
  supported Lua versions, message format, protection of the
  communication channel, nested evaluations, evaluation end, error
  reporting, debugging and sandboxing.
* Only the names of changed environment variables are sent to Lua
  interpreter, which fetches their value when a modulefile reads them
  or runs a command. Sending values with each message makes the cost
  of the bridge grow with the number of loaded modules.
* Lmod modulefile and modulerc functions are reviewed: mapping to Tcl
  modulefile commands, functions handled by the Lua script, behaviors
  specific to an evaluation mode and way to identify the module tool.
* Site customization is covered: hooks, site-specific functions and
  properties file are ported to the Modules counterparts.
* Implementation stages, installation options, packaging,
  documentation, linting of Lua files, testsuite and performance
  benchmark are described.
* Differences with Lmod are listed.

Performance figures come from a throw-away prototype of the bridge
run on the modulefiles of the load2 benchmark.

Assisted-by: Claude:claude-fable-5-2
Signed-off-by: Xavier Delaruelle <xavier.delaruelle@cea.fr>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant