render: draw into an offscreen texture and read frames back

natsukium commented

2026-09-25 22:31:54 +09:00

Owner

The renderer can now draw into a texture of its own and hand the frame back as RGBA, so a consumer with no window (felis-reel, which turns an .fcast recording into video, and screenshot tests) draws through the same code as a felis window. This is the offscreen half of #441; pinned fonts follow separately.

Why

Both constructors took a window handle, chose the adapter and texture format from its surface, and render ended in present. There was no way to build a Renderer without a window. Rasterizing cells a second way on the CPU would drift from what a window shows.

What changed

render:
- Renderer::new_offscreen(width, height, cfg, warmup) picks an adapter without a surface, reusing a warm device when a Warmup supplies one, and draws into a fixed Rgba8UnormSrgb texture. It ignores background_opacity, so the texture holds straight RGB.
- Renderer::read_frame(&mut self, out) replaces out with the last rendered frame as tightly packed RGBA. It copies in row strips sized under the device's max_buffer_size, through one reused staging buffer. It makes no promise about alpha.
- ReadFrameError:
  - NotOffscreen on a window renderer.
  - NoFrame until something has been rendered into the current texture (construction and resize both start a fresh one).
  - RowPastBufferLimit when even one row exceeds max_buffer_size.
  - Poll / Map for device failures.
- The renderer's size and format moved out of SurfaceConfiguration onto Renderer itself, so resize, the viewport uniform and cursor_rect_uv serve both targets. reconfigure_surface does nothing offscreen. The window path behaves as before.
- An offscreen renderer has no post-process stage. new_offscreen returns RendererError::OffscreenPostShader when the config names a shader, and set_post_shader(Some(_)) returns PostShaderError::Offscreen. The stage's uniforms read the host clock, and the bundled trail shader paints the whole frame until the client drives TrailState, so offscreen frames would depend on when they were rendered.
- capture_last_frame keeps re-encoding the cell pass into a scratch texture, since the smoke test calls it on a swapchain texture that cannot be copied from.
Tests:
- On the noop backend with the pinned test font:
  - post-shader refusal at construction and through the setter.
  - NoFrame before the first render and after a resize.
  - Unpadded row lengths across a resize.
  - A two-strip readback under a 64 KiB max_buffer_size.
  - A unit test for the strip size.
- One pixel test needs a real adapter, because the noop backend never executes a pass. It renders three frames (a translucent background_opacity, a background override, then a resize) and checks every pixel of each. It is #[ignore]d in the default run.
ci: just test-gpu runs the crate's ignored tests with Vulkan pointed at lavapipe from the .#smoke shell, and pr.yml's linux-smoke job runs it.
Docs:
- pipeline.md "wgpu surface and resources" records why an offscreen renderer has no post stage, with a Revisit if.
- testing.md "Frontend smoke test" says that the Linux job runs just test-gpu.
- No CHANGELOG entry: no CLI, config, keybinding, wire or default changes.

Verification

just check passes.
just test-gpu passes on lavapipe.
The pixel test also passes on a discrete GPU.
If the change that drops background_opacity offscreen is removed, the pixel test fails.

The renderer can now draw into a texture of its own and hand the frame back as RGBA, so a consumer with no window (felis-reel, which turns an `.fcast` recording into video, and screenshot tests) draws through the same code as a felis window. This is the offscreen half of #441; pinned fonts follow separately. ## Why Both constructors took a window handle, chose the adapter and texture format from its surface, and `render` ended in `present`. There was no way to build a `Renderer` without a window. Rasterizing cells a second way on the CPU would drift from what a window shows. ## What changed - **render:** - `Renderer::new_offscreen(width, height, cfg, warmup)` picks an adapter without a surface, reusing a warm device when a `Warmup` supplies one, and draws into a fixed `Rgba8UnormSrgb` texture. It ignores `background_opacity`, so the texture holds straight RGB. - `Renderer::read_frame(&mut self, out)` replaces `out` with the last rendered frame as tightly packed RGBA. It copies in row strips sized under the device's `max_buffer_size`, through one reused staging buffer. It makes no promise about alpha. - `ReadFrameError`: - `NotOffscreen` on a window renderer. - `NoFrame` until something has been rendered into the current texture (construction and `resize` both start a fresh one). - `RowPastBufferLimit` when even one row exceeds `max_buffer_size`. - `Poll` / `Map` for device failures. - The renderer's size and format moved out of `SurfaceConfiguration` onto `Renderer` itself, so `resize`, the viewport uniform and `cursor_rect_uv` serve both targets. `reconfigure_surface` does nothing offscreen. The window path behaves as before. - An offscreen renderer has no post-process stage. `new_offscreen` returns `RendererError::OffscreenPostShader` when the config names a shader, and `set_post_shader(Some(_))` returns `PostShaderError::Offscreen`. The stage's uniforms read the host clock, and the bundled trail shader paints the whole frame until the client drives `TrailState`, so offscreen frames would depend on when they were rendered. - `capture_last_frame` keeps re-encoding the cell pass into a scratch texture, since the smoke test calls it on a swapchain texture that cannot be copied from. - **Tests:** - On the noop backend with the pinned test font: - post-shader refusal at construction and through the setter. - `NoFrame` before the first render and after a resize. - Unpadded row lengths across a resize. - A two-strip readback under a 64 KiB `max_buffer_size`. - A unit test for the strip size. - One pixel test needs a real adapter, because the noop backend never executes a pass. It renders three frames (a translucent `background_opacity`, a background override, then a resize) and checks every pixel of each. It is `#[ignore]`d in the default run. - **ci:** `just test-gpu` runs the crate's ignored tests with Vulkan pointed at lavapipe from the `.#smoke` shell, and `pr.yml`'s `linux-smoke` job runs it. - **Docs:** - `pipeline.md` "wgpu surface and resources" records why an offscreen renderer has no post stage, with a _Revisit if_. - `testing.md` "Frontend smoke test" says that the Linux job runs `just test-gpu`. - No CHANGELOG entry: no CLI, config, keybinding, wire or default changes. ## Verification - `just check` passes. - `just test-gpu` passes on lavapipe. - The pixel test also passes on a discrete GPU. - If the change that drops `background_opacity` offscreen is removed, the pixel test fails.

natsukium added 2 commits

2026-09-25 22:31:54 +09:00

render: draw into an offscreen texture and read frames back baf0bc139b

A consumer with no window, such as a tool that turns an .fcast
recording into video or a screenshot test, could not use the renderer:
both constructors took a window handle, picked the adapter and format
from its surface, and `render` ended in `present`. Rasterizing cells a
second way on the CPU would drift from what a felis window shows.

`Renderer::new_offscreen` picks an adapter without a surface and draws
into an `Rgba8UnormSrgb` texture; `read_frame` copies it out through a
reused, row-padded staging buffer as tightly packed RGBA. Size and
format now live on the renderer rather than in the surface
configuration, so `resize` and the viewport math serve both targets. `read_frame`
refuses a target nothing has been rendered into since it was created or
resized, so a blank texture never passes for a drawn frame. The copy
runs in row strips sized under the device's `max_buffer_size`: a
texture inside the 2D limit can still need more readback than one
buffer may hold, and an oversized buffer is a validation panic rather
than an error the caller can see.

The offscreen target has no post-process stage and refuses one, at
construction and through `set_post_shader`: the stage's uniforms read
the host clock and the trail shader paints the whole frame until the
client drives its state, so frames would depend on when they were
rendered. `background_opacity` is ignored offscreen because nothing
composites the result. `capture_last_frame` keeps re-encoding the cell
pass, since a swapchain texture cannot be copied from.

Assisted-by: Claude Code

ci, render: run the offscreen pixel test on lavapipe

bench / Criterion regression gate (pull_request) Successful in 8m19s

Details

fuzz / cargo fuzz smoke (per target) (pull_request) Successful in 1m43s

Details

pr / nix flake check (pull_request) Successful in 1m8s

Details

pr / wire schema is compatible with the base (pull_request) Successful in 3m24s

Details

bench / Criterion full-suite snapshot (pull_request) Has been skipped

Details

fuzz / cargo fuzz nightly long-run (pull_request) Has been skipped

Details

pr / cargo build / clippy / test / deny (pull_request) Successful in 2m14s

Details

pr / frontend smoke (x86_64-linux) (pull_request) Successful in 1m11s

Details

pr / publish felis (x86_64-linux) (pull_request) Has been skipped

Details

windows / cargo clippy (Windows cross) (pull_request) Successful in 58s

Details

windows / detect relevant changes (pull_request) Failing after 4m59s

Details

windows / cargo nextest (Windows) (pull_request) Has been skipped

Details

windows / windows docs-only no-op (pull_request) Has been skipped

Details

windows / frontend smoke (Windows) (pull_request) Has been skipped

Details

windows / package felis (x86_64-pc-windows-msvc) (pull_request) Has been skipped

Details

darwin / detect relevant changes (pull_request) Failing after 5m1s

Details

darwin / build felis (aarch64-darwin) (pull_request) Has been skipped

Details

darwin / darwin docs-only no-op (pull_request) Has been skipped

Details

darwin / publish felis (aarch64-darwin) (pull_request) Has been skipped

Details

c757628142

The renderer's unit tests run on wgpu's noop backend, which never
executes a pass, so the one test that reads real pixels back from an
offscreen frame needs a GPU adapter and is ignored in the default run.
Left ignored everywhere, it would never gate anything. The Linux smoke
job already has lavapipe through the smoke dev shell, so `just test-gpu`
points Vulkan at it and runs the crate's ignored tests there.

Assisted-by: Claude Code

natsukium referenced this pull request from natsukium/felis-reel

2026-09-25 22:32:13 +09:00

docs: pin the offscreen-renderer plan to what felis can offer #1

natsukium merged commit 5c8b97690d into main

2026-09-25 23:18:25 +09:00

natsukium deleted branch render-offscreen-target

2026-09-25 23:18:25 +09:00

natsukium referenced this pull request from a commit

2026-09-25 23:18:26 +09:00

render: draw into an offscreen texture and read frames back (#465)

natsukium referenced this pull request from a commit

2026-09-25 23:19:07 +09:00