Share the Kitty image layout with every frontend #161

Merged

natsukium merged 1 commit from graphics/shared-image-layout into main 2026-10-18 13:21:13 +09:00

Conversation 0 Commits 1 Files changed 13 +1361 -821

natsukium commented 2026-10-18 12:56:25 +09:00

Owner

Copy link

Moves Kitty image placement out of felis-render-wgpu into felis-grid::image_layout, so every frontend shares one implementation. That covers anchors and the viewport offset, automatic c=/r=, source clamping, the Unicode-placeholder box fit, and layers and stacking. A frontend without wgpu, such as felis-web-component, gets plain draws (an image-pixel source rect and a screen-pixel destination, per layer in paint order) and paints them with drawImage, with no Kitty logic of its own.

Changes

• felis-grid::image_layout: ImageLayout::rebuild, ImageDraw, and the one-method LayoutImages trait. It sits in felis-grid because the renderer is below felis-client-core and has to name these types.
• felis-client-core: ImageShadow::layout_into, ClientImage::{is_drawable, generation, write_rgba8}. The generation token comes from a process-wide counter, so an async frontend (createImageBitmap) can drop a stale conversion, even across a session switch.
• felis-render-wgpu: only maps draws into atlas UVs. If the slot's size differs from the draw's, the draw is skipped. Clipping for client chrome stays here.

Behavior aligned with kitty

• Placeholder images always draw at z = -1, under the text, whatever z= was recorded.
• An image re-sent under the same id disappears until the new data completes.
• When only the frame on screen is re-sent, it keeps showing until the new pixels complete. An atlas reset mid-transfer re-uploads what was on screen.

Equal-z stacking order does not match kitty yet; that is #155, since fixing it needs a wire change.

Verification

• cargo nextest run --all-features for grid / client-core / render-wgpu / client: all pass. Clippy is clean.
• The GPU tests (--run-ignored only, including the placeholder seam test) pass on a local Metal adapter.
• just check: one failure, felis-cli cli_bridge::tests::a_daemon_write_failure_fails_and_clears_the_link, which times out. It times out on origin/main in this environment too, so it is unrelated to this change.
• I have not checked the rendering visually in the GUI.

Moves Kitty image placement out of `felis-render-wgpu` into `felis-grid::image_layout`, so every frontend shares one implementation. That covers anchors and the viewport offset, automatic `c=`/`r=`, source clamping, the Unicode-placeholder box fit, and layers and stacking. A frontend without wgpu, such as felis-web-component, gets plain draws (an image-pixel source rect and a screen-pixel destination, per layer in paint order) and paints them with `drawImage`, with no Kitty logic of its own. ## Changes - `felis-grid::image_layout`: `ImageLayout::rebuild`, `ImageDraw`, and the one-method `LayoutImages` trait. It sits in felis-grid because the renderer is below felis-client-core and has to name these types. - `felis-client-core`: `ImageShadow::layout_into`, `ClientImage::{is_drawable, generation, write_rgba8}`. The generation token comes from a process-wide counter, so an async frontend (`createImageBitmap`) can drop a stale conversion, even across a session switch. - `felis-render-wgpu`: only maps draws into atlas UVs. If the slot's size differs from the draw's, the draw is skipped. Clipping for client chrome stays here. ## Behavior aligned with kitty - Placeholder images always draw at z = -1, under the text, whatever `z=` was recorded. - An image re-sent under the same id disappears until the new data completes. - When only the frame on screen is re-sent, it keeps showing until the new pixels complete. An atlas reset mid-transfer re-uploads what was on screen. Equal-z stacking order does not match kitty yet; that is #155, since fixing it needs a wire change. ## Verification - `cargo nextest run --all-features` for grid / client-core / render-wgpu / client: all pass. Clippy is clean. - The GPU tests (`--run-ignored only`, including the placeholder seam test) pass on a local Metal adapter. - `just check`: one failure, `felis-cli cli_bridge::tests::a_daemon_write_failure_fails_and_clears_the_link`, which times out. It times out on `origin/main` in this environment too, so it is unrelated to this change. - I have not checked the rendering visually in the GUI.

natsukium added 1 commit 2026-10-18 12:56:26 +09:00

grid, render, client-core, docs, skills: share the Kitty image layout ... 39917c8143

Placing Kitty images (anchor and viewport offset, automatic c=/r=,
source clamp, the Unicode-placeholder box fit, layers and stacking) lived
inside felis-render-wgpu, entangled with atlas UVs. A frontend without
wgpu, such as the browser view, would have had to port it and drift from
the GPU client. felis-grid::image_layout now computes it once as plain
draws in image and screen pixels; the GPU client maps them into its
atlas, and a canvas can hand them to drawImage. It lives in felis-grid
rather than beside ImageShadow because the renderer sits below
felis-client-core and must name the draw type. Client chrome clipping
stays in the renderer: it is that client's UI, not Kitty semantics.

Two behaviors move to kitty's while the logic is in one place:

- Placeholder tiles draw at z = -1 whatever z= the virtual placement
  recorded, as kitty creates every placeholder ref at -1. A z=10
  placeholder image used to cover the text.
- An image re-sent under the same id draws nothing until the new data
  completes, as kitty frees the old pixels when the transmission starts.
  The layout is computed against the shadow's current size, so drawing
  the old atlas slot meanwhile would sample it with the new geometry.
  The client repaints on the header so the image leaves the screen even
  if the transfer stalls. Re-sending only the displayed frame keeps it
  showing instead: the shadow stages the new pixels and swaps them in at
  completion, so an atlas reset mid-transfer re-uploads what was on
  screen rather than a zero fill. The staged copy stays outside the
  session byte cap: the daemon edits the frame in place and admits the
  edit at the cap, and one open transfer bounds the copy to one frame.

The placeholder debug log keeps its line and adds the draws the atlas
could not paint, since a layout tile is no longer proof of a painted one.

ClientImage gains write_rgba8 and a generation token so an asynchronous
frontend (createImageBitmap) can discard a conversion that a newer
frame, replacement, or session switch has superseded; tokens come from a
process-wide counter because a session switch replaces the shadow.

kitty's order for overlapping images of equal z (by image and ref
creation) is not matched: reproducing it needs creation-order metadata
on the wire, which reattach currently loses.

Assisted-by: Claude Code:Opus 5.5

natsukium force-pushed graphics/shared-image-layout from 39917c8143 to f67fa2cc29 2026-10-18 13:04:15 +09:00 Compare

natsukium scheduled this pull request to auto merge when all checks succeed 2026-10-18 13:12:27 +09:00

natsukium merged commit 597f9f6925 into main 2026-10-18 13:21:13 +09:00

natsukium deleted branch graphics/shared-image-layout 2026-10-18 13:21:13 +09:00

natsukium referenced this pull request from a commit 2026-10-18 13:21:13 +09:00

Merge pull request 'Share the Kitty image layout with every frontend' (#161) from graphics/shared-image-layout into main

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

Area/ci

CI workflows (.forgejo/workflows/)

  

Area/cli

felis-cli crate, the felis front-door binary

  

Area/client

felis-client crate, the winit GUI frontend

  

Area/client-core

felis-client-core crate, shadow screen, config, connect_or_spawn_daemon

  

Area/daemon

felis-daemon crate, session pool, IPC server

  

Area/deps

Dependency bumps

  

Area/docs

Documentation tree when no narrower area fits

  

Area/grid

felis-grid crate, cells, scrollback, image store

  

Area/manual

User-facing docs: tutorials, how-to, user-facing reference pages

  

Area/nix

Nix flake, home-manager module, package definitions

  

Area/protocol

felis-protocol crate, felis.proto, frame layout, version constants

  

Area/pty

felis-pty crate, PTY backends

  

Area/render

felis-render-wgpu crate, GPU pipeline, atlas

  

Area/shaping

felis-shaping crate, swash shaper, font fallback

  

Area/skills

Agent skills (skills/felis, .agents/skills/)

  

Area/transport

felis-transport crate, framing, socket/pipe, SSH carrier

  

Area/vt

felis-vt crate, VT parser, dispatch

  

Compat/Breaking

Breaking change that won't be backward compatible

  

Kind/Bug

Something is not working

  

Kind/Documentation

Documentation changes

  

Kind/Enhancement

Improve existing functionality

  

Kind/Feature

New functionality

  

Kind/Performance

Throughput, latency, or memory cost

  

Kind/Security

This is security issue

  

Kind/Testing

Issue or pull request related to testing

  

perf-regression-accepted

Passes the instruction-count bench gate for an intended cost; state why in a PR comment

  

Reviewed

Confirmed

Issue has been confirmed

  

Reviewed

Duplicate

This issue or pull request already exists

  

Reviewed

Invalid

Invalid issue

  

Reviewed

Won't Fix

This issue won't be fixed

  

Status

Blocked

Something is blocking this issue or pull request

  

Status

Need More Info

Feedback is required to reproduce issue or to continue work

  

Status

Upstream

Waiting on a change in an upstream project

No labels

Area/ci

Area/cli

Area/client

Area/client-core

Area/daemon

Area/deps

Area/docs

Area/grid

Area/manual

Area/nix

Area/protocol

Area/pty

Area/render

Area/shaping

Area/skills

Area/transport

Area/vt

Compat/Breaking

Kind/Bug

Kind/Documentation

Kind/Enhancement

Kind/Feature

Kind/Performance

Kind/Security

Kind/Testing

perf-regression-accepted

Reviewed

Confirmed

Reviewed

Duplicate

Reviewed

Invalid

Reviewed

Won't Fix

Status

Blocked

Status

Need More Info

Status

Upstream

Milestone

Clear milestone

No items

No milestone

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

felis-terminal/felis!161

No description provided.