CUDA=1 on startup

[LordEschatus]

So i have downloaded the prebuilt 1.4 for linux, and when running:

user@host:~/Downloads/colibri-v1.4.0-linux-x86_64$ ./coli chat --model ./GLM-5.2-colibri-int4-g64-with-int8-mtp/ --gpu auto

     ▄▀▀▀▄  ▄        colibri v1.4.0
  ▄▄▄▄▀▀▀▀▄▀▀        tiny engine, immense model
      ▀▀▀▀▀▀▀        GLM-5.2 · 744B MoE · int4 · streaming CPU
        ▀▀▀▀         chat ·  · ram -GB · topp off
          ▀          
  ──────────────────────────────────────────────────────────
  ↺ resuming a saved conversation: 253 tokens, last written just now
    it steers tone, language and topic. :reset clears it · KVSAVE=0 disables saving · delete ./GLM-5.2-colibri-int4-g64-with-int8-mtp/.coli_kv to start clean
**--gpu needs the CUDA build: make colibri CUDA=1 (this binary is CPU-only)**
user@host:~/Downloads/colibri-v1.4.0-linux-x86_64$

whether I run --gpu auto --gpu 0 --gpu 0,1 -r --auto-tier --gpu x,y etc... I continue to encounter the CPU-only warning.

I have tried cloning the repository to make it myself, but Im not sure what command I would pass to setup.sh

I am aware of https://github.com/JustVugg/colibri/actions/runs/30677132862 , but please forgive me , I am not a developer, can read code (badly) , but not sure what the resolution is here.

[bradford-apps]

If you want the Cuda support just like if you want to try the latest updates and bug fixes, you need to compile the executable program yourself. I didn't know how to do that either until I asked AI how to do it.

[basil-k-aji-dev]

Nothing you pass to --gpu can fix this, which is why every variant gave you the
same line. CUDA ?= 0 in c/Makefile:220 means CUDA is opt-in at compile
time, and the released binaries are built without it. The message is telling
you the binary lacks the backend, not that the flag was wrong.

You don't need to be a developer to build it. On Linux:

# 1. toolchain (CUDA toolkit must already be installed: `nvcc --version` should work)
sudo apt install build-essential

# 2. build with the CUDA backend on
git clone https://github.com/JustVugg/colibri
cd colibri
make glm CUDA=1

glm and colibri are the same target (c/Makefile:704-707), so the message's
make colibri CUDA=1 and the above are equivalent.

On setup.sh — there is no argument to pass. It runs
make -s colibri ARCH="${ARCH:-native}", with no CUDA flag. But because the
Makefile uses CUDA ?= 0, an environment variable wins, so this works too:

CUDA=1 ./c/setup.sh

It also checks gcc and OpenMP first, which is useful on a fresh machine.

Three things that commonly bite on the first CUDA build:

• nvcc not at the default path. The build expects /usr/local/cuda; point it
  elsewhere with make glm CUDA=1 CUDA_HOME=/opt/cuda.
• Host compiler too new for nvcc — the usual "unsupported GNU version" error.
  Override just the host compiler: make glm CUDA=1 NVCC_CCBIN=g++-15
  (c/Makefile:241-242).
• GPU architecture. The default CUDA_ARCH=native compiles for the card in
  the machine you build on, which is what you want. Only if you move the binary
  to a different GPU do you need CUDA_ARCH=portable, which emits
  sm_80/86/89/90/120 plus a PTX fallback. An RTX 50-series card (sm_120) needs
  CUDA >= 12.8.

You do not need to make clean first: the Makefile records build-affecting flags
in .build-config precisely so that turning CUDA=1 on after a CPU-only build
rebuilds instead of reporting "up to date" (c/Makefile:709-713).

Once built, --gpu auto will work. There are environment equivalents if you
prefer them: COLI_CUDA=1 with COLI_GPU=N, or COLI_GPUS=0,1 for several
(GPU_BACKENDS.md:131).

One caveat I should state plainly: I read this from the Makefile and
GPU_BACKENDS.md at f028d26 rather than running the CUDA build myself — no
NVIDIA hardware here. If a step fails, paste the error and it should be easy to
place.