Skip to content
pgstyPublic

Repository files navigation

SOW - Software Object Warehouse

Website: sow.pgsty.com Docs Version: 0.7.1 CI Go License: Apache-2.0

sow is an open-source RPM / DEB repository manager from Pigsty. One self-contained binary turns a directory of packages into a working YUM / APT repository, and manages curated repositories with signing, snapshots, audit history, and incremental publication when you need more control.

"Software Object Warehouse": store once, serve everywhere, ship only what changed.

Website | Docs | Get Started | Download | Release Notes | Discuss | Pigsty | 中文

SOW architecture: local RPM and DEB files enter one package pool that stores each package body once, the pool is projected as YUM and APT repository views that hold metadata only, and publication uploads only the changed objects to a filesystem or S3 / R2 target.


Get Started

sow ships in the Pigsty infra repository for mainstream Linux distros on amd64 / arm64:

# APT: Debian / Ubuntu and compatible platforms
sudo tee /etc/apt/sources.list.d/pigsty-infra.list > /dev/null <<'EOF'
deb [trusted=yes] https://repo.pigsty.io/apt/infra generic main
EOF
sudo apt update && sudo apt install -y sow
# YUM: RHEL / Rocky / Alma / Anolis and compatible platforms
sudo tee /etc/yum.repos.d/pigsty-infra.repo > /dev/null <<'EOF'
[pigsty-infra]
name=Pigsty Infra for $basearch
baseurl=https://repo.pigsty.io/yum/infra/$basearch
enabled=1
gpgcheck=0
module_hotfixes=1
EOF
sudo dnf makecache && sudo dnf install -y sow

For mainland China users: consider replacing repo.pigsty.io with repo.pigsty.cc.

Pinned RPM / DEB packages, installer-free tarballs for Linux and macOS, and source builds are covered on the download page and published on GitHub Releases.

Then point sow at a directory of packages:

sow create ./packages                       # emit YUM / APT repository metadata in place
sow create ./packages --sign-with <KEYID>   # also sign currently unsigned RPMs with a GPG key

That directory is now a repository. RPMs become a normal YUM/DNF repository, DEBs become a flat APT repository, and a mixed directory gets both. Serve it with any static web server; --pigsty enables the Pigsty layout conventions, and sow help create documents the complete option contract. Continue with the Quick Start.

For a password-protected signing key, SOW 0.7.0 and later accept --passfile /run/secrets/gpg-password. You can also set SOW_GPG_PASSFILE to that path or configure passfile next to a managed signing key. Priority is CLI, per-key config (including legacy passphrase), then environment. Use one nonempty UTF-8 line: one final LF or CRLF is removed, with no trimming of spaces or tabs. Keep the file outside the repository, normally mode 0600. No default file is searched. For create, --sign-with still selects and enables signing. Signing and recovery commands share this interface; check sow help COMMAND in your installed binary and see password files for the full contract.

Plain sow create treats repository metadata as a deterministic result of the directory: change the inputs and run it again. RPM publication installs and syncs new hash-named indexes before replacing repomd.xml. It retains indexes referenced by the current and immediately previous distinct repomd.xml; unchanged runs preserve that window. After a successful pointer switch, or a rerun that completes interrupted cleanup, it removes older SOW hash-named indexes. Other files, including an external repomd.xml.asc, stay untouched and external signatures must be refreshed separately. An existing repomd.xml that lists metadata SOW does not generate, such as comps groups, updateinfo, or modules, is replaced without it; create reports those data types as a warning and in the JSON dropped_metadata field. prune rejects such an index instead.

The bounded .sow-plain-rpm-history.json file records at most three reference sets so a rerun can identify the predecessor on either side of an interrupted pointer switch. It is a retention list, not a publication journal. Missing or corrupt history resets the window: a changed pointer retains its readable predecessor; an unchanged pointer can only retain its current indexes. Malformed old repomd.xml and corrupt historical indexes do not prevent rebuilding, but their old reader closure cannot be restored. Symlinks at SOW-owned metadata paths are rejected. Rebuilding with only DEB inputs, or an empty --pigsty result, removes SOW RPM metadata and the retention list. Package removal rules, including --pigsty, are unchanged.

To remove old versions from a flat RPM repository, use sow prune:

sow prune ./packages --dry-run            # preview superseded RPMs
sow prune ./packages                      # rebuild the index, then delete old RPMs
sow prune ./packages --protect age_18 --protect age_18-llvmjit

Prune keeps the highest Epoch/Version/Release per package name and architecture. --protect keeps every version of an exact package name and must match a package. It binds replacement RPM digests and indexed bytes to a stable file identity, checks that identity before each deletion and completion, refreshes an existing repo_complete marker, and can be rerun after an interruption. It operates on top-level regular binary RPMs in a local publication source; managed workspaces, mixed RPM/DEB directories, equal-version files, RPM v6 packages, repositories with repomd.xml.asc, and a repomd.xml that lists metadata SOW does not generate (such as comps groups) are rejected. --metadata-timestamp works as for create. --dry-run changes no files. Publishing the result and accommodating clients with cached old indexes remain part of the publication workflow.


Features

  • Flat repositories: sow create replaces createrepo_c, dpkg-scanpackages, and reprepro. One command indexes RPM and DEB packages in place, with optional GPG signing.
  • One pool, many views: a managed Repository stores each package body once and projects metadata-only Dists for EL, Debian, and Ubuntu, so neither disk nor object storage holds duplicate payloads.
  • Explicit state: Desired membership is separate from Built state, every build publishes an immutable Generation, and sow check / status / changes / log keep validation and history inspectable.
  • Incremental publication: sow changes reports the exact delta and sow publish uploads only that, to filesystem or Cloudflare R2 (S3-compatible) targets, with atomic pointer switches and fail-closed recovery.
  • Self-contained: one static Go binary (CGO_ENABLED=0). No daemon, no database service, no language runtime. Linux and macOS, amd64 and arm64.

Managed workspaces

A managed workspace is for repositories that SOW should own over time, with explicit membership, policy, signed metadata, immutable snapshots, audit history, and publication targets:

sow init ./lab
sow repo new local --workdir ./lab
sow dist new stable --format rpm --workdir ./lab --repo local
sow add ./packages/example.rpm --workdir ./lab --repo local --dist stable
sow check --workdir ./lab --repo local
sow status --workdir ./lab --repo local
sow changes --workdir ./lab --repo local
sow log --workdir ./lab --repo local

sow add and sow rm converge the selected Dist to a new Built Generation by default. Configure a filesystem or r2 target in sow.yml, then sow publish TARGET uploads the change set and sow gc collects unreachable local payloads. sow help COMMAND is the authoritative CLI reference shipped with the binary; machine consumers can rely on the closed --json envelopes and the documented exit-code contract.

Upgrading a workspace created by SOW v0.3 or v0.4? Stop writers, back up the workspace, and run sow repo migrate REPOSITORY --workdir DIR once per Repository. Versions 0.5.0 through 0.7.1 use schema v13 to index pool-path ownership without scanning unrelated publication history; configuration and public layout stay unchanged. Then run sow build and sow check: the updated RPM authentication and APT metadata contracts require one rebuild of affected Dists. Later unchanged RPMs reuse matching Built evidence instead of repeated payload verification. Do not reopen the migrated database with an older binary. Upgrading from v0.5.0 through v0.7.0 to v0.7.1 requires no schema migration or public-layout change. If a Repository migrated by v0.5 or v0.6 reports an invalid transition control target after a later build, upgrade to v0.7.0 or newer; no manual repair is needed. A configuration using passfile cannot be loaded by v0.6.1 or older; remove those keys before downgrading. Publication ordering, recovery, target rebinding, the one-copy boundary, and RPM leaf export are specified in Commands and Design Records.


Build

Building from source requires Go 1.27.4 or newer; repository signing additionally requires a usable GPG installation and key.

make build          # write the binary to bin/sow
make test-core      # focused repository-manager tests
make test           # all Go packages plus the patched RPM module
make check          # format, module, vet, staticcheck, deadcode, focused tests
make release-local  # GoReleaser snapshot archives and packages under dist/

GitHub Actions runs regular checks in CI and Docker-backed client / S3 coverage in Integration. Pushing an exact semantic-version tag creates a draft GitHub release after the tag is validated against main, the source version, and the changelog; publishing that draft is a separate manual decision. Maintainers can also run a read-only check against hosted Cloudflare R2 with make test-r2-live (see the SOW_REAL_R2_* variables in internal/r2/cloudflare_integration_test.go).

Before tagging, require successful CI and Integration runs for the exact release commit. Inspect the draft archives and packages, their checksums and sow version output before publishing the Release. Enable the website's pinned download links only after that Release is public and its assets are available.


About

The authoritative user documentation lives in SOW Docs; dated architecture decisions live in Design Records. Historical implementation and verification material remains attached to versioned source tags rather than a second documentation tree in this repository.

SOW is built by the Pigsty team (pgsty), alongside:

  • pigsty — open-source PostgreSQL distribution with HA, PITR, IaC, monitoring, and hundreds of extensions
  • pig — PostgreSQL and extension package manager for EL / Debian / Ubuntu
  • silo — S3-compatible object storage, a community-maintained MinIO fork

SOW is licensed under the Apache License, Version 2.0. Bundled third-party components remain under their respective licenses.

Releases

Packages

Contributors

Languages