ci: move the release workflow's inline Python into tested scripts #479

Merged
natsukium merged 2 commits from ci/release-scripts into main 2026-09-26 12:09:35 +09:00
Owner

The release page and baseline steps in release.yml were heredoc Python between curl calls, and they run only on a tag push. A mistake in them surfaced at the moment a release was being cut, and nothing exercised the safety properties they encode: draft first, never replace a published release, every asset present before publishing, and the baseline job's early exits.

What moved where:

  • release job: tools/release/publish.py release <tag> <notes> <asset>... (existing-draft check and delete, draft creation, uploads, missing-assets check, publish).
  • baseline job: tools/release/publish.py baseline <tag> (newest-release check, asset download and comparison, the "main already carries it" check, BREAKING.md rewrite, branch push, open-PR check, PR creation).
  • darwin-publish: the bundled-client check becomes verify.py --identity --bundled-client.

publish_test.py fakes the Forgejo release and pull API with a local http.server and uses a bare git remote reached through a url.insteadOf rewrite of the push URL. It covers 24 cases, and each safety property was checked by mutating the script and confirming the matching case fails. It runs through publish.py --self-test (also just release-publish-test), and the release and baseline jobs run it before they act. verify_test.py gains the bundled-client cases.

Token handling: every forge call runs curl -fsS -K <conf>, and uploads use -F attachment=@<file>, the same transport the workflow already used. The Authorization header lives in a 0600 curl config, and git authenticates through an extraHeader in a 0600 GIT_CONFIG_GLOBAL file. Both are written to a private temporary directory under RUNNER_TEMP that is removed on exit. Only a 404 on the tag lookup means "no release": that call reads -w '%{http_code}' without -f. The steps run under nix develop .#publish, which gains python3.

Not wired here: pr.yml does not run the new self-test yet. Only the tag jobs and just release-publish-test run it, so a PR-level run needs a line next to the existing verify.py --self-test, or the planned selftest aggregate.

The release page and baseline steps in `release.yml` were heredoc Python between curl calls, and they run only on a tag push. A mistake in them surfaced at the moment a release was being cut, and nothing exercised the safety properties they encode: draft first, never replace a published release, every asset present before publishing, and the baseline job's early exits. What moved where: - **release job**: `tools/release/publish.py release <tag> <notes> <asset>...` (existing-draft check and delete, draft creation, uploads, missing-assets check, publish). - **baseline job**: `tools/release/publish.py baseline <tag>` (newest-release check, asset download and comparison, the "main already carries it" check, `BREAKING.md` rewrite, branch push, open-PR check, PR creation). - **darwin-publish**: the bundled-client check becomes `verify.py --identity --bundled-client`. `publish_test.py` fakes the Forgejo release and pull API with a local `http.server` and uses a bare git remote reached through a `url.insteadOf` rewrite of the push URL. It covers 24 cases, and each safety property was checked by mutating the script and confirming the matching case fails. It runs through `publish.py --self-test` (also `just release-publish-test`), and the release and baseline jobs run it before they act. `verify_test.py` gains the bundled-client cases. Token handling: every forge call runs `curl -fsS -K <conf>`, and uploads use `-F attachment=@<file>`, the same transport the workflow already used. The Authorization header lives in a 0600 curl config, and git authenticates through an extraHeader in a 0600 `GIT_CONFIG_GLOBAL` file. Both are written to a private temporary directory under `RUNNER_TEMP` that is removed on exit. Only a 404 on the tag lookup means "no release": that call reads `-w '%{http_code}'` without `-f`. The steps run under `nix develop .#publish`, which gains `python3`. Not wired here: `pr.yml` does not run the new self-test yet. Only the tag jobs and `just release-publish-test` run it, so a PR-level run needs a line next to the existing `verify.py --self-test`, or the planned `selftest` aggregate.
ci: move the release workflow's inline Python into tested scripts
Some checks failed
bench / Criterion full-suite snapshot (pull_request) Has been skipped
fuzz / cargo fuzz nightly long-run (pull_request) Has been skipped
darwin / detect relevant changes (pull_request) Successful in 14s
darwin / darwin docs-only no-op (pull_request) Has been skipped
darwin / build felis (aarch64-darwin) (pull_request) Successful in 54s
darwin / publish felis (aarch64-darwin) (pull_request) Has been skipped
windows / detect relevant changes (pull_request) Successful in 12s
windows / windows docs-only no-op (pull_request) Has been skipped
windows / cargo nextest (Windows) (pull_request) Successful in 3m8s
windows / frontend smoke (Windows) (pull_request) Successful in 37s
windows / package felis (x86_64-pc-windows-msvc) (pull_request) Has been skipped
bench / Criterion regression gate (pull_request) Has been cancelled
fuzz / cargo fuzz smoke (per target) (pull_request) Has been cancelled
pr / nix flake check (pull_request) Has been cancelled
pr / cargo build / clippy / test / deny (pull_request) Has been cancelled
pr / wire schema is compatible with the base (pull_request) Has been cancelled
pr / frontend smoke (x86_64-linux) (pull_request) Has been cancelled
pr / publish felis (x86_64-linux) (pull_request) Has been cancelled
windows / cargo clippy (Windows cross) (pull_request) Has been cancelled
644c26ed5e
release.yml carried the release-page and baseline logic as heredoc Python between curl calls. Those steps run only on a tag push, so a mistake in them surfaced at the one moment a release was being cut, and nothing exercised the safety properties they encode (draft first, never replace a published release, all assets present before publishing, the baseline's early exits).

- `tools/release/publish.py release <tag> <notes> <asset>...` creates the release as a draft, deletes an unfinished draft for the tag on a rerun, refuses when the tag's release is already published, uploads the assets, re-reads the draft and publishes it only when every asset arrived. It also checks the asset files exist before touching the forge.
- `tools/release/publish.py baseline <tag>` is the baseline job: it stops when a newer final release supersedes the tag, compares the attached felis.proto with the checkout, stops when main already carries the whole baseline, and otherwise regenerates `release/baseline-<tag>`, pushes it and opens its pull request unless one is open.
- `verify.py --identity --bundled-client` replaces the darwin job's inline check that the bundle's client launched and reports the CLI's identity.
- `publish_test.py` runs both subcommands against a local http.server fake of the Forgejo release and pull API and a bare git remote (reached through a url.insteadOf rewrite of the push URL), following the verify_test.py/compat_test.py pattern. `publish.py --self-test` runs it; the release and baseline jobs run it before acting, and `just release-publish-test` runs it locally.

The script does HTTP through urllib rather than curl, so the token never leaves the process: it is read from RELEASE_TOKEN and sent as a header, which removes the need for curl's `-K` config file. A non-default User-Agent is required because Forgejo answers urllib's default one with 403. git still authenticates through an extraHeader in a 0600 GIT_CONFIG_GLOBAL file, now written into a private temporary directory under RUNNER_TEMP and removed when the job ends. Cross-origin redirects (the attachment download) drop the Authorization header, and API calls refuse redirects outright.

Assisted-by: Claude Code
natsukium scheduled this pull request to auto merge when all checks succeed 2026-09-26 10:54:18 +09:00
ci: publish the release page through curl rather than urllib
All checks were successful
bench / Criterion full-suite snapshot (pull_request) Has been skipped
fuzz / cargo fuzz nightly long-run (pull_request) Has been skipped
darwin / detect relevant changes (pull_request) Successful in 14s
darwin / build felis (aarch64-darwin) (pull_request) Successful in 55s
darwin / publish felis (aarch64-darwin) (pull_request) Has been skipped
darwin / darwin docs-only no-op (pull_request) Has been skipped
windows / detect relevant changes (pull_request) Successful in 11s
windows / windows docs-only no-op (pull_request) Has been skipped
windows / cargo nextest (Windows) (pull_request) Successful in 4m13s
windows / frontend smoke (Windows) (pull_request) Successful in 33s
windows / package felis (x86_64-pc-windows-msvc) (pull_request) Has been skipped
bench / Criterion regression gate (pull_request) Successful in 2m12s
pr / nix flake check (pull_request) Successful in 29s
fuzz / cargo fuzz smoke (per target) (pull_request) Successful in 1m15s
pr / wire schema is compatible with the base (pull_request) Successful in 13s
windows / cargo clippy (Windows cross) (pull_request) Successful in 16s
pr / cargo build / clippy / test / deny (pull_request) Successful in 1m57s
pr / frontend smoke (x86_64-linux) (pull_request) Successful in 29s
pr / publish felis (x86_64-linux) (pull_request) Has been skipped
1386c54bb9
The release path only runs on a tag, so replacing the transport the workflow had proven (curl with a -K config file, curl -F multipart, curl's redirect handling) with urllib, a hand-built multipart body and a custom redirect handler traded untested YAML for an untested transport. publish.py now shells out to curl for every forge call, with the Authorization header in a 0600 curl config under a private RUNNER_TEMP directory that is removed when the script exits, the same way the git extraHeader file already was.

The one behavior kept from the urllib version is that only a 404 on the tag lookup means "no release": that call runs without -f and reads the status through -w '%{http_code}', so a forge error there fails the job instead of proceeding to a POST.

The release and baseline steps run under `nix develop .#publish`, which pins curl; python3 joins that shell, the smaller change than adding curl to the default dev shell.

Assisted-by: Claude Code
natsukium deleted branch ci/release-scripts 2026-09-26 12:09:35 +09:00
Commenting is not possible because the repository is archived.
No description provided.