client: expose the terminal to screen readers via AccessKit #240

New issue

Open

opened 2026-10-24 13:25:13 +09:00 by natsukium · 1 comment

natsukium commented

2026-10-24 13:25:13 +09:00

Owner

Target: 0.4.0.

felis draws every glyph on the GPU, so its window has no accessibility tree and screen readers (NVDA, Narrator, VoiceOver, Orca) read nothing. Accessibility is treated as a duty rather than a feature gated on a requesting user: the consumer is the OS's assistive technology, and only the process that owns the window can answer its queries. The design and its rejected alternatives go in docs/explanation/accessibility.md (draft on the docs/accessibility branch), with a fifth platform-shim entry in non-goals.md.

Scope

Build one tree with AccessKit (accesskit_winit supports winit 0.30) on Windows, macOS, and Linux: Window → Terminal role → one text run per visible row, the cursor as the caret, and the selection.
Expose the viewport only. Scrollback enters the tree as the viewport scrolls over it.
The client builds the tree from the screen snapshot it already renders; the daemon is unchanged.
Build nothing until a screen reader first queries the window.
Report text changes as a mechanical diff between consecutive viewport snapshots.

Out of scope (principle 4 or 1)

Suppressing typed-character echo by matching output against keystrokes (decides from output content).
Skipping full-screen/TUI redraws (classifies changes by content). Windows Terminal's maintainers reject this too: microsoft/terminal#13666.
An app-controlled announce/suppress sequence (OSC 200–202 proposal, freedesktop terminal-wg#18): no producer emits it.

Open questions

Whether to push an explicit announcement of new output on top of text-changed events. AccessKit has no standalone announce API; on AT-SPI and macOS an announcement fires only when a live-region node is added. Windows Terminal (UIA notifications, 1000-char chunks) and iTerm2 (announcement after output settles, capped at 256 lines) both push; VTE relies on text-changed events alone. If felis pushes, the cap must be a line or character count, not a content filter.
AccessKit text bugs to avoid or fix upstream: empty TextRun nodes loop or panic (AccessKit#836, AccessKit#837).
Character offsets for wide and multi-codepoint cells (the AT sees codepoints; the grid sees cells).
macOS: avoid AXDocument / settable AXValue and LayoutChanged with element lists, which reset the VoiceOver cursor in iTerm2.
Verification: NVDA and Narrator on Windows, VoiceOver on macOS, Orca on Linux; Accessibility Insights and Accerciser to inspect the tree.

Prior art

Windows Terminal: UIA text provider plus notifications (src/renderer/uia/UiaRenderer.cpp, TermControlAutomationPeer.cpp).
VTE (GTK4): GtkAccessibleText with prefix/suffix diff of viewport snapshots (src/vteaccess-gtk4.cc).
iTerm2: iTermTextViewAccessibilityHelper.m.
Ghostty: macOS NSAccessibility (shipped), GTK GtkAccessibleText (ghostty-org/ghostty#13779), viewport bug ghostty-org/ghostty#9932.
AccessKit author's suggested terminal layout: wezterm/wezterm#913. WezTerm's AccessKit PoC (wezterm/wezterm#5995) stalled on answering AT queries from the window loop.

Target: 0.4.0. felis draws every glyph on the GPU, so its window has no accessibility tree and screen readers (NVDA, Narrator, VoiceOver, Orca) read nothing. Accessibility is treated as a duty rather than a feature gated on a requesting user: the consumer is the OS's assistive technology, and only the process that owns the window can answer its queries. The design and its rejected alternatives go in `docs/explanation/accessibility.md` (draft on the `docs/accessibility` branch), with a fifth platform-shim entry in `non-goals.md`. ## Scope - Build one tree with AccessKit (`accesskit_winit` supports winit 0.30) on Windows, macOS, and Linux: `Window` → `Terminal` role → one text run per visible row, the cursor as the caret, and the selection. - Expose the viewport only. Scrollback enters the tree as the viewport scrolls over it. - The client builds the tree from the screen snapshot it already renders; the daemon is unchanged. - Build nothing until a screen reader first queries the window. - Report text changes as a mechanical diff between consecutive viewport snapshots. ## Out of scope (principle 4 or 1) - Suppressing typed-character echo by matching output against keystrokes (decides from output content). - Skipping full-screen/TUI redraws (classifies changes by content). Windows Terminal's maintainers reject this too: microsoft/terminal#13666. - An app-controlled announce/suppress sequence (OSC 200–202 proposal, freedesktop terminal-wg#18): no producer emits it. ## Open questions - Whether to push an explicit announcement of new output on top of text-changed events. AccessKit has no standalone announce API; on AT-SPI and macOS an announcement fires only when a live-region node is added. Windows Terminal (UIA notifications, 1000-char chunks) and iTerm2 (announcement after output settles, capped at 256 lines) both push; VTE relies on text-changed events alone. If felis pushes, the cap must be a line or character count, not a content filter. - AccessKit text bugs to avoid or fix upstream: empty `TextRun` nodes loop or panic (AccessKit#836, AccessKit#837). - Character offsets for wide and multi-codepoint cells (the AT sees codepoints; the grid sees cells). - macOS: avoid AXDocument / settable AXValue and `LayoutChanged` with element lists, which reset the VoiceOver cursor in iTerm2. - Verification: NVDA and Narrator on Windows, VoiceOver on macOS, Orca on Linux; Accessibility Insights and Accerciser to inspect the tree. ## Prior art - Windows Terminal: UIA text provider plus notifications (`src/renderer/uia/UiaRenderer.cpp`, `TermControlAutomationPeer.cpp`). - VTE (GTK4): `GtkAccessibleText` with prefix/suffix diff of viewport snapshots (`src/vteaccess-gtk4.cc`). - iTerm2: `iTermTextViewAccessibilityHelper.m`. - Ghostty: macOS NSAccessibility (shipped), GTK `GtkAccessibleText` (ghostty-org/ghostty#13779), viewport bug ghostty-org/ghostty#9932. - AccessKit author's suggested terminal layout: wezterm/wezterm#913. WezTerm's AccessKit PoC (wezterm/wezterm#5995) stalled on answering AT queries from the window loop.

natsukium added the

Area/client

Kind/Feature

labels

2026-10-24 13:25:17 +09:00

natsukium referenced this issue

2026-10-24 14:29:13 +09:00

client: announce new output to NVDA and VoiceOver #241

natsukium commented

2026-10-24 14:29:22 +09:00

Author

Owner

Scope update: explicit announcements of new output moved to #241. This issue now covers the tree (viewport text as multi-run lines, caret, selection, geometry) and its text-changed events. The implementation design passed review; the PR will carry the code and the docs together.

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

felis-terminal/felis#240

No description provided.

Rows
Columns