Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 56 additions & 91 deletions .github/workflows/update-devpanel-cli.yml
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
name: Update DevPanel CLI binaries

# Build linux/amd64 and linux/arm64 devpanel binaries from
# https://github.com/devpanel/cli/tree/HEAD/cmd/devpanel-client and commit
# https://gitlab.com/devpanel-vn/cli and commit
# base/bin/devpanel-amd64 and base/bin/devpanel-arm64.
#
# Run it manually. The Actions cache records the latest commit that changed
# cmd/devpanel-client. Delete that cache entry to rebuild the same commit.
# Run it manually on a branch. The Actions cache records the upstream default
# branch HEAD after both binaries are committed to this branch. Delete that
# cache entry to rebuild the same commit.
#
# The plan job reads the latest cmd/devpanel-client commit from GitHub GraphQL.
# The build clones devpanel/cli under RUNNER_TEMP, outside this repository.
# GHCR_TOKEN must be able to read that repository. GHCR_USER is the username
# that owns the token. The workflow token commits the binaries.
# The plan job queries GitLab's commits API; the build fetches the commit under RUNNER_TEMP.
# GITLAB_TOKEN needs read_api for the API and read_repository for Git fetches.
# GITLAB_USER is used to authenticate the Git fetch.
# The workflow token commits the binaries.

on:
workflow_dispatch:
Expand All @@ -21,7 +22,8 @@ concurrency:

jobs:
plan:
name: Check cached devpanel-client revision
name: Check cached DevPanel CLI revision
if: github.ref_type == 'branch'
Comment thread
darrenoh marked this conversation as resolved.
Resolved
runs-on: ubuntu-26.04
permissions:
actions: read
Expand All @@ -30,97 +32,48 @@ jobs:
sha: ${{ steps.upstream.outputs.sha }}
changed: ${{ steps.decision.outputs.changed }}
steps:
- name: Require GHCR credentials
env:
GHCR_TOKEN: ${{ secrets.GHCR_TOKEN }}
GHCR_USER: ${{ vars.GHCR_USER }}
run: |
if [[ -z "${GHCR_TOKEN}" ]]; then
echo "::error::GHCR_TOKEN is required to read devpanel/cli."
exit 1
fi
if [[ -z "${GHCR_USER}" ]]; then
echo "::error::GHCR_USER is required. Set it to the username that owns GHCR_TOKEN."
exit 1
fi

- name: Resolve latest cmd/devpanel-client commit
- name: Resolve upstream default branch HEAD
id: upstream
env:
GHCR_TOKEN: ${{ secrets.GHCR_TOKEN }}
GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}
run: |
set -euo pipefail
query='query {
repository(owner: "devpanel", name: "cli") {
defaultBranchRef {
target {
... on Commit {
history(first: 1, path: "cmd/devpanel-client") {
nodes { oid }
}
}
}
}
}
}'
auth=$(printf '%s %s' 'Bearer' "$GHCR_TOKEN")
body=$(mktemp)
code=$(
curl -sS \
-H "Authorization: ${auth}" \
-H "Accept: application/vnd.github+json" \
-H "Content-Type: application/json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
-o "$body" \
-w "%{http_code}" \
https://api.github.com/graphql \
--data "$(jq -n --arg query "$query" '{query: $query}')" \
|| true
)
if [[ "$code" != "200" ]]; then
echo "::error::Unable to read cmd/devpanel-client history from GitHub GraphQL (HTTP ${code:-unknown})."
jq -r '.message // empty' "$body" >&2 || true
rm -f "$body"
if [[ -z "${GITLAB_TOKEN}" ]]; then
echo "::error::GITLAB_TOKEN is required to read gitlab.com/devpanel-vn/cli."
exit 1
fi
if jq -e '.errors | length > 0' "$body" >/dev/null; then
echo "::error::GitHub GraphQL returned an error while reading cmd/devpanel-client history."
jq -r '.errors[].message' "$body" >&2
rm -f "$body"
exit 1
fi
sha=$(jq -r '.data.repository.defaultBranchRef.target.history.nodes[0].oid // empty' "$body")
rm -f "$body"

commits=$(curl --fail --silent --show-error \
--header "PRIVATE-TOKEN: ${GITLAB_TOKEN}" \
"https://gitlab.com/api/v4/projects/devpanel-vn%2Fcli/repository/commits?per_page=1")
Comment thread
darrenoh marked this conversation as resolved.
Resolved
sha=$(jq -r '.[0].id // empty' <<< "$commits")
if [[ ! "$sha" =~ ^[0-9a-f]{40}$ ]]; then
echo "::error::No commit on the default branch changes cmd/devpanel-client."
echo "::error::Unable to resolve the GitLab CLI default branch HEAD."
exit 1
fi
echo "sha=${sha}" >> "$GITHUB_OUTPUT"
echo "Latest cmd/devpanel-client commit is ${sha}"
echo "GitLab CLI default branch HEAD is ${sha}"

- name: Restore the built revision
id: revision-cache
uses: actions/cache/restore@v5
with:
path: ${{ runner.temp }}/devpanel-cli-revision
key: devpanel-client-${{ steps.upstream.outputs.sha }}
key: gitlab-devpanel-client-v1-${{ github.ref }}-${{ steps.upstream.outputs.sha }}
lookup-only: true

- name: Decide whether to rebuild
id: decision
env:
CACHE_HIT: ${{ steps.revision-cache.outputs.cache-hit }}
REF_TYPE: ${{ github.ref_type }}
UPSTREAM_SHA: ${{ steps.upstream.outputs.sha }}
run: |
if [[ "${REF_TYPE}" == "tag" ]]; then
echo "changed=false" >> "$GITHUB_OUTPUT"
echo "Tagged commits are left unchanged."
elif [[ "${CACHE_HIT}" == "true" ]]; then
if [[ "${CACHE_HIT}" == "true" ]]; then
echo "changed=false" >> "$GITHUB_OUTPUT"
echo "cmd/devpanel-client ${UPSTREAM_SHA} is already built."
echo "GitLab CLI ${UPSTREAM_SHA} is already built for this branch."
else
echo "changed=true" >> "$GITHUB_OUTPUT"
echo "cmd/devpanel-client ${UPSTREAM_SHA} is not in the Actions cache."
echo "GitLab CLI ${UPSTREAM_SHA} has not been built for this branch."
fi

build:
Expand All @@ -139,20 +92,21 @@ jobs:
permissions:
contents: read
steps:
- name: Clone devpanel/cli
- name: Fetch GitLab CLI
env:
GHCR_TOKEN: ${{ secrets.GHCR_TOKEN }}
GHCR_USER: ${{ vars.GHCR_USER }}
GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}
GITLAB_USER: ${{ vars.GITLAB_USER }}
UPSTREAM_SHA: ${{ needs.plan.outputs.sha }}
GIT_TERMINAL_PROMPT: "0"
run: |
set -euo pipefail
dest="${RUNNER_TEMP}/devpanel-cli"
mkdir "$dest"
auth=$(printf '%s:%s' "${GHCR_USER}" "${GHCR_TOKEN}" | base64 -w0)
auth=$(printf '%s:%s' "${GITLAB_USER}" "${GITLAB_TOKEN}" | base64 -w0)
echo "::add-mask::${auth}"
git -C "$dest" init
Comment thread
darrenoh marked this conversation as resolved.
Resolved
git -C "$dest" remote add origin https://github.com/devpanel/cli.git
git -C "$dest" -c "http.extraheader=AUTHORIZATION: basic ${auth}" \
git -C "$dest" remote add origin https://gitlab.com/devpanel-vn/cli.git
git -C "$dest" -c "http.https://gitlab.com/.extraheader=AUTHORIZATION: basic ${auth}" \
fetch --depth 1 origin "$UPSTREAM_SHA"
git -C "$dest" checkout --detach FETCH_HEAD

Expand All @@ -162,21 +116,32 @@ jobs:
go-version-file: ${{ runner.temp }}/devpanel-cli/go.mod
cache: false

- name: Build native binary
- name: Build native client binary
working-directory: ${{ runner.temp }}/devpanel-cli
env:
GHCR_TOKEN: ${{ secrets.GHCR_TOKEN }}
GHCR_USER: ${{ vars.GHCR_USER }}
ARCH: ${{ matrix.arch }}
GITLAB_TOKEN: ${{ secrets.GITLAB_TOKEN }}
GITLAB_USER: ${{ vars.GITLAB_USER }}
Comment thread
darrenoh marked this conversation as resolved.
Resolved
GIT_TERMINAL_PROMPT: "0"
GOPRIVATE: gitlab.com/devpanel-vn/*
run: |
set -euo pipefail
git config --global url."https://${GHCR_USER}:${GHCR_TOKEN}@github.com/".insteadOf "https://github.com/"
dest="${RUNNER_TEMP}/devpanel-${{ matrix.arch }}"
(
cd "${RUNNER_TEMP}/devpanel-cli"
env -u GOOS -u GOARCH CGO_ENABLED=0 \
go build -trimpath -buildvcs=false -o "$dest" ./cmd/devpanel-client
)
if [[ ! -d cmd/devpanel-client ]]; then
echo "::error::GitLab source does not contain cmd/devpanel-client; verify the client build path."
exit 1
fi
auth=$(printf '%s:%s' "${GITLAB_USER}" "${GITLAB_TOKEN}" | base64 -w0)
echo "::add-mask::${auth}"
export GIT_CONFIG_COUNT=1
export GIT_CONFIG_KEY_0=http.https://gitlab.com/.extraheader
export GIT_CONFIG_VALUE_0="AUTHORIZATION: basic ${auth}"
dest="${RUNNER_TEMP}/devpanel-${ARCH}"
env -u GOOS -u GOARCH CGO_ENABLED=0 \
go build -trimpath -o "$dest" ./cmd/devpanel-client
unset GITLAB_TOKEN GITLAB_USER GIT_CONFIG_COUNT GIT_CONFIG_KEY_0 GIT_CONFIG_VALUE_0 auth
chmod 0755 "$dest"
"$dest" version
"$dest" app trigger --help

- name: Upload binary
uses: actions/upload-artifact@v7
Expand Down Expand Up @@ -224,7 +189,7 @@ jobs:
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git commit \
-m "Update devpanel CLI from devpanel/cli@${UPSTREAM_SHA}" \
-m "Update devpanel CLI from GitLab devpanel-vn/cli@${UPSTREAM_SHA}" \
-m "Built linux/amd64 and linux/arm64 from cmd/devpanel-client at that commit."
git push origin "HEAD:${GITHUB_REF_NAME}"
fi
Expand All @@ -240,4 +205,4 @@ jobs:
uses: actions/cache/save@v5
with:
path: ${{ runner.temp }}/devpanel-cli-revision
key: devpanel-client-${{ needs.plan.outputs.sha }}
key: gitlab-devpanel-client-v1-${{ github.ref }}-${{ needs.plan.outputs.sha }}
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,10 +129,10 @@ Baseline JSON files in `tests/baselines/` track pre-existing lint violations per

#### Workflows
- **`docker-build.yml`** — triggered on pushes to `main` or `develop` and by manual `workflow_dispatch`; detects changed versions, builds images per-platform in parallel, and merges platform-specific digests into multi-arch manifests.
- **`update-devpanel-cli.yml`** — manual `workflow_dispatch`. Builds `base/bin/devpanel-amd64` and `base/bin/devpanel-arm64` from the latest `devpanel/cli` commit that changed `cmd/devpanel-client`, when the Actions cache does not already contain that commit. A new run cancels one already in progress. The workflow token commits the binaries.
- **`update-devpanel-cli.yml`** — manual `workflow_dispatch` on branches only. Resolves the GitLab `devpanel-vn/cli` default branch HEAD and builds `base/bin/devpanel-amd64` and `base/bin/devpanel-arm64` in parallel on native runners from `cmd/devpanel-client` (the original binary's embedded package path; confirm against GitLab source). The Go version comes from upstream `go.mod`. A branch-specific Actions cache records the revision only after the workflow token commits both binaries. A new run cancels one already in progress. Upstream Git authentication is scoped to GitLab requests, not global configuration.
- **`test.yml`** — runs YAML, shell, and Dockerfile linting plus build/run tests on pushes and pull requests.
- Production images are tagged without suffix (e.g. `devpanel/php:8.3-base`); `develop` branch builds use the `-rc` suffix.
- Required repository secrets: `DOCKERHUB_USERNAME`, `DOCKERHUB_TOKEN`. Optional: `GHCR_TOKEN` (falls back to `GITHUB_TOKEN` for GHCR pushes). `update-devpanel-cli.yml` requires `GHCR_TOKEN` with access to `devpanel/cli`, and the `GHCR_USER` repository variable set to the username that owns that token.
- Required repository secrets: `DOCKERHUB_USERNAME`, `DOCKERHUB_TOKEN`. Optional: `GHCR_TOKEN` (falls back to `GITHUB_TOKEN` for GHCR pushes). `update-devpanel-cli.yml` requires the service account's `GITLAB_TOKEN` with both `read_api` (for the commits API) and `read_repository` (for Git fetches) scopes, plus the `GITLAB_USER` repository variable containing that account's GitLab username. A deploy token with only `read_repository` cannot authenticate the API request.

## Environment Variables (base images)

Expand Down
26 changes: 19 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -291,13 +291,25 @@ for setup instructions.
Native AMD64 builds skip QEMU.
- `update-devpanel-cli.yml` rebuilds `base/bin/devpanel-amd64` and
`base/bin/devpanel-arm64` from the latest
[devpanel/cli](https://github.com/devpanel/cli) commit that changed
`cmd/devpanel-client`. Run it manually. It caches that commit and skips
the build when the cache entry exists. A new run cancels one already in
progress. The workflow token commits the binaries. Delete the cache
entry to rebuild the same commit. The `devpanel/cli` checkout is
placed outside this repository. AMD64 and ARM64 binaries build in
parallel on `ubuntu-26.04` and `ubuntu-26.04-arm`.
[GitLab CLI repository](https://gitlab.com/devpanel-vn/cli) default branch
HEAD. Run it manually on a branch; tags are left unchanged. Each native build
compiles `cmd/devpanel-client` for Linux with CGO disabled. This client path
is based on the existing binary's metadata and must be confirmed against
the GitLab source. Both builds use the same resolved
commit and the Go version from its `go.mod`. The workflow caches the
revision per destination branch only after both binaries are committed,
and skips subsequent builds on a cache hit. Delete that cache entry to
rebuild the same commit. A new run cancels one already in progress.
Set the `GITLAB_TOKEN` repository secret to the service account's GitLab
token with both `read_api` (for the commits API) and `read_repository` (for
Git fetches) scopes. Set the `GITLAB_USER` repository variable to the
service account's username. A deploy token with only `read_repository` is
insufficient because the workflow also queries the GitLab API.
The workflow token commits the binaries. GitLab credentials are scoped
to GitLab Git requests and not stored in global Git configuration. The checkout is
placed outside this repository. AMD64 and ARM64 binaries build in parallel
on `ubuntu-26.04` and `ubuntu-26.04-arm`. This workflow does not change
which binary the Dockerfile copies into images.
- The `advance` variant depends on `secure`, which depends on `base`.
The build chain is enforced via the bake target dependency graph.

Expand Down
24 changes: 8 additions & 16 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,24 +103,16 @@ Use this file to track work that needs to be done. Lint scripts automatically ap
- OPcache remains loaded on every PHP version without enabling a nonexistent PHP 8.5 shared module.
- Focused checks pass; any remaining build or CI blockers are explicitly reported.

# Task: Rebuild the DevPanel CLI from devpanel/cli
# Task: Rebuild the DevPanel CLI from GitLab

## Steps
- [x] Add a manual workflow based on `main`. The workflow token commits the binaries.
- [x] Cache the latest commit that changed `cmd/devpanel-client` and skip the build when that entry exists.
- [x] Build linux/amd64 and linux/arm64 in parallel on native runners.
- [x] Clone `devpanel/cli` outside this repository and commit only the two binaries.
- [x] Resolve and cache the upstream default branch HEAD through the authenticated GitLab commits API, and skip revisions already built for the destination branch.
- [x] Build linux/amd64 and linux/arm64 in parallel on native runners, then commit both binaries.
- [x] Configure GitLab API and source-fetch authentication with the service account's `GITLAB_TOKEN` (`read_api` and `read_repository`) and `GITLAB_USER`.
- [ ] Confirm `cmd/devpanel-client`, Go setup, build dependencies and smoke checks against the actual GitLab source.
- [ ] Run the workflow with GitLab credentials and verify both binaries and revision caching.

## Definition of Done
- A cache miss builds both binaries on `ubuntu-26.04` and `ubuntu-26.04-arm` and commits `base/bin/devpanel-amd64` and `base/bin/devpanel-arm64`.

# Task: Authenticate to devpanel/cli with GHCR_USER

## Steps
- [x] Require the `GHCR_USER` repository variable with `GHCR_TOKEN`.
- [x] Read the latest `cmd/devpanel-client` commit from the commits API as that user.
- [x] Clone and fetch `devpanel/cli` for the build with Basic auth for that username and token.

## Definition of Done
- The plan job records the commit the API returns for `cmd/devpanel-client`.
- The build job authenticates to `devpanel/cli` as `vars.GHCR_USER`.
- A cache miss resolves the latest default-branch commit through the GitLab API, builds both binaries on `ubuntu-26.04` and `ubuntu-26.04-arm`, and commits `base/bin/devpanel-amd64` and `base/bin/devpanel-arm64`.
- GitLab API access uses the service account's `GITLAB_TOKEN` with `read_api` and `read_repository` scopes plus its username in `GITLAB_USER`; the built revision is cached after the binaries are committed.
Loading