feat: render .fcast recordings through felis's offscreen renderer #2

Merged

natsukium merged 1 commit from poc into main

2026-09-26 00:53:28 +09:00

natsukium commented

2026-09-26 00:13:32 +09:00

Owner

First working felis-reel: it renders a .fcast recording to MP4, WebM, GIF or a PNG sequence through felis's offscreen renderer (natsukium/felis#465).

Why

That offscreen renderer was the one upstream change the design needed. With it merged, the pipeline in docs/explanation/design.md can run end to end.

What changed

Scaffold:
- A Nix flake whose dev shell provides the nightly toolchain, cargo-deny, cargo-nextest, ffmpeg, and the Vulkan loader and fontconfig on the library path.
- Lints mirroring felis, deny.toml with a git-source allow-list, and CI running fmt, clippy, test and deny.
- felis crates are pinned to 5c8b9769. felis-fcast is a git dependency at its own pin; its Cast holds frame bodies as JSON, so the two felis revisions never meet.
timeline:
- The rehydrate burst lands at t = 0.
- --idle-limit caps each gap and --speed scales it.
- Times keep fractions, so a frame due just after an output sample waits for the next sample.
replay:
- Decodes frames through felis-grid's json_v1.
- Applies them to felis-client-core's shadow screen and image shadow.
- Syncs the renderer through a copy of felis-client's GridMsg → renderer mapping (natsukium/felis#442 tracks sharing it).
canvas: the renderer draws at the exact grid size, because it centers a grid in any larger target. Each frame is copied top-left into a canvas as large as the largest grid, with even dimensions. The rest of the canvas is filled with the renderer's clear color, and alpha is forced to 255.
sink: raw RGBA piped to ffmpeg (H.264, VP9, or GIF through palettegen/paletteuse), or one PNG per frame into an empty directory.
CLI checks:
- Rejects a recording whose felis_json this build does not read.
- Rejects --speed values that are not finite and positive.
- Rejects --idle-limit values that are not finite and non-negative.
- Rejects a --font-size outside felis's 4–72 px band.
Config: the base theme and font come from the felis config (--theme selects a file; the client id is reel). The RendererConfig is built here because felis-client's conversion is private. A configured post shader is ignored with a warning.
Docs: README status and usage, and a backlog updated to what is still open.

Verification

fmt, clippy -D warnings, 10 unit tests and cargo deny pass.
By hand, on felis-fcast's gen_hello / gen_scroll samples:
- The PNG frames show the theme colors, bold text and the cursor.
- The MP4 is H.264, 336x144, 101 frames, and a frame grabbed at 2 s shows the expected scroll position.
- The GIF encodes.
- Each rejected option and an unsupported felis_json header exit 1 with a message.

Open in the backlog: GIF duplicate-frame merging, --cursor-blink, an end-to-end test that includes a mid-recording resize, and golden tests (which need pinned fonts from felis).

First working felis-reel: it renders a `.fcast` recording to MP4, WebM, GIF or a PNG sequence through felis's offscreen renderer (natsukium/felis#465). ## Why That offscreen renderer was the one upstream change the design needed. With it merged, the pipeline in `docs/explanation/design.md` can run end to end. ## What changed - **Scaffold:** - A Nix flake whose dev shell provides the nightly toolchain, cargo-deny, cargo-nextest, ffmpeg, and the Vulkan loader and fontconfig on the library path. - Lints mirroring felis, `deny.toml` with a git-source allow-list, and CI running fmt, clippy, test and deny. - felis crates are pinned to `5c8b9769`. felis-fcast is a git dependency at its own pin; its `Cast` holds frame bodies as JSON, so the two felis revisions never meet. - **`timeline`:** - The rehydrate burst lands at t = 0. - `--idle-limit` caps each gap and `--speed` scales it. - Times keep fractions, so a frame due just after an output sample waits for the next sample. - **`replay`:** - Decodes frames through felis-grid's `json_v1`. - Applies them to felis-client-core's shadow screen and image shadow. - Syncs the renderer through a copy of felis-client's `GridMsg` → renderer mapping (natsukium/felis#442 tracks sharing it). - **`canvas`:** the renderer draws at the exact grid size, because it centers a grid in any larger target. Each frame is copied top-left into a canvas as large as the largest grid, with even dimensions. The rest of the canvas is filled with the renderer's clear color, and alpha is forced to 255. - **`sink`:** raw RGBA piped to `ffmpeg` (H.264, VP9, or GIF through `palettegen`/`paletteuse`), or one PNG per frame into an empty directory. - **CLI checks:** - Rejects a recording whose `felis_json` this build does not read. - Rejects `--speed` values that are not finite and positive. - Rejects `--idle-limit` values that are not finite and non-negative. - Rejects a `--font-size` outside felis's 4–72 px band. - **Config:** the base theme and font come from the felis config (`--theme` selects a file; the client id is `reel`). The `RendererConfig` is built here because felis-client's conversion is private. A configured post shader is ignored with a warning. - **Docs:** README status and usage, and a backlog updated to what is still open. ## Verification - fmt, clippy `-D warnings`, 10 unit tests and `cargo deny` pass. - By hand, on felis-fcast's `gen_hello` / `gen_scroll` samples: - The PNG frames show the theme colors, bold text and the cursor. - The MP4 is H.264, 336x144, 101 frames, and a frame grabbed at 2 s shows the expected scroll position. - The GIF encodes. - Each rejected option and an unsupported `felis_json` header exit 1 with a message. Open in the backlog: GIF duplicate-frame merging, `--cursor-blink`, an end-to-end test that includes a mid-recording resize, and golden tests (which need pinned fonts from felis).

natsukium added 1 commit

2026-09-26 00:13:32 +09:00

feat: render .fcast recordings through felis's offscreen renderer

ci / cargo fmt / clippy / test / deny (pull_request) Successful in 54s

Details

0160575756

felis now draws offscreen and reads frames back, which is the one thing
the design needed from upstream, so the pipeline it describes can run:
parse with felis-fcast, decode through felis-grid's json_v1, apply to
felis-client-core's shadows, render and read back each output frame,
pad it into a fixed canvas and hand it to ffmpeg or a PNG sequence.

Every output frame is rendered rather than only damaged ones, so OSC
color overrides, reverse video and image placement changes that dirty
no row still reach the video. The renderer draws at the exact grid
size and the canvas pads top-left, because the renderer centers a grid
in any larger target. The GridMsg-to-renderer mapping is a copy of
felis-client's until felis shares one, and the RendererConfig is built
here because felis-client's conversion is private to its binary.

The felis crates are pinned to the first revision with the offscreen
API; felis-fcast keeps its own, older pin, which never meets this one
because its Cast holds frame bodies as JSON.

Assisted-by: Claude Code

natsukium referenced this pull request

2026-09-26 00:34:10 +09:00

sink: merge identical consecutive GIF frames into one longer frame #3

natsukium merged commit 0115e145d8 into main

2026-09-26 00:53:28 +09:00

natsukium deleted branch poc

2026-09-26 00:53:28 +09:00

natsukium referenced this pull request from a commit

2026-09-26 00:53:28 +09:00

feat: render .fcast recordings through felis's offscreen renderer (#2)