natsukium/felis
Watch
1
Code Issues 27 Pull requests 1 Projects Releases Packages Wiki Activity Actions

[v0.1/P1] Publish CLI and bridge JSON schemas with golden conversations #29

New issue
Closed
opened 2026-09-03 16:18:08 +09:00 by natsukium · 1 comment
natsukium commented 2026-09-03 16:18:08 +09:00
Owner
Copy link

Parent: #12 (P1.8). Related to #3 and #9.

Why

The protobuf socket and TOML config have machine-readable schemas, but the public non-protobuf extension path exists only as Rust serializers and prose. Consumers cannot validate requests or outputs independently.

Scope

  • Commit a schema bundle for bridge envelopes and operation params.
  • Cover every point result/error, stream item/lag/terminal, identifier format, and integer bound.
  • Add golden JSONL conversations for concurrency, cancellation races, daemon loss, stdout loss, and failure before the first item.
  • Keep lifecycle properties such as exactly one terminal in prose and executable conversation tests where JSON Schema cannot express them.

Acceptance criteria

  • Every documented machine object validates against the committed bundle.
  • Invalid ids, unknown fields, and out-of-range integers fail fixtures.
  • Schema regeneration is deterministic and checked in CI.
  • Golden conversations cover reordered concurrent replies and every terminal path.
  • CLI/bridge reference and skills/felis link to the versioned schema.
Parent: #12 (P1.8). Related to #3 and #9. ## Why The protobuf socket and TOML config have machine-readable schemas, but the public non-protobuf extension path exists only as Rust serializers and prose. Consumers cannot validate requests or outputs independently. ## Scope - Commit a schema bundle for bridge envelopes and operation `params`. - Cover every point result/error, stream item/lag/terminal, identifier format, and integer bound. - Add golden JSONL conversations for concurrency, cancellation races, daemon loss, stdout loss, and failure before the first item. - Keep lifecycle properties such as exactly one terminal in prose and executable conversation tests where JSON Schema cannot express them. ## Acceptance criteria - [ ] Every documented machine object validates against the committed bundle. - [ ] Invalid ids, unknown fields, and out-of-range integers fail fixtures. - [ ] Schema regeneration is deterministic and checked in CI. - [ ] Golden conversations cover reordered concurrent replies and every terminal path. - [ ] CLI/bridge reference and `skills/felis` link to the versioned schema.
natsukium commented 2026-09-03 20:35:15 +09:00
Author
Owner
Copy link

Triage plan (2026-09-03)

Source-grounded triage against main at 69076d42, reviewed through seven rounds of an independent reviewer (pi sol/luna) until it passed with no findings. The dependency order that supersedes the tracker's is posted on #12.

Claim check

Accurate at HEAD.

  • Two machine-readable schemas exist, one does not. The wire is crates/felis-protocol/proto/felis.proto (REQ-111, docs/reference/spec.md:65); the config is crates/felis-client-core/felis-config.schema.json, generated by a schemars sync test behind the schema feature (justfile:143-151, crates/felis-client-core/Cargo.toml:26-41, docs/reference/config.md:59-87). The CLI/bridge objects are serde structs only: cli_output.rs:203-706 (envelopes, SessionObject, results, ErrorKind), cli_bridge.rs:334-360 (Envelope, Body), request parsing by hand in parse_request/Params (cli_bridge.rs:398-470). No schema file or schema mention exists for them (grep -n "schema" docs/reference/cli.md docs/reference/ipc.md finds none). felis-cli has no schemars dependency (crates/felis-cli/Cargo.toml).
  • Prose is the only contract. docs/reference/cli.md:87-130 (envelope table, kind list), 615-735 (bridge shapes, ops, params, bounds), ipc.md:1624 ("Non-Rust clients: the stdio bridge").
  • Conversation tests exist but are not golden fixtures. crates/felis-cli/tests/cli_bridge.rs covers correlation (276), two concurrent streams each ending once (314), malformed line (488), stdin EOF cancels (531), daemon loss for point (574) and for every outstanding op (663), id reuse (615), cold socket (712), cancel (902), clean terminal count (949). They are Rust assertions over live output, not committed JSONL. Missing entirely: stdout loss (owned by #21's behavior change), failure before the first item on a bridge stream (the CLI has it at tests/cli_sessions.rs:326, the bridge does not), and any reordering-tolerant golden for concurrent replies.
  • Request-side validation is partial. parse_request checks v, id type, op type, params is object; rows/cols are bounded to 0..=65535 bridge-side (cli.md:686-694); unknown params keys are ignored (the Params accessors read by key). So "unknown fields … fail fixtures" is a new rule, not a documented one.

Nothing already done; nothing wrong in the issue.

Verdict

accept-with-changes.

  1. Generate, do not hand-write. Reuse the config-schema pattern: derive schemars::JsonSchema on the cli_output.rs and bridge types behind a schema feature in felis-cli, with a UPDATE_SCHEMA=1 sync test and a just schema extension. A hand-written bundle would drift from the serde types the way prose already does.
  2. Type the bridge request params. Replace Params<'_> accessors with per-op #[derive(Deserialize, JsonSchema)] #[serde(deny_unknown_fields)] structs. That yields the request schema for free and makes the bridge refuse unknown fields at runtime as malformed_request, so the fixture rule and the binary agree. This is the one behavior change; it is pre-freeze and explicit (principle 4) and should be stated in CHANGELOG.
  3. Golden conversations must be order-normalized. Concurrent stream interleavings and daemon-assigned ids are nondeterministic; a byte-exact golden would flake. Fixtures should be per-id expected sequences plus a whole-conversation multiset, with ids and timestamps masked. "Golden" then means committed JSONL the harness diffs after normalization.

Principle check: no capability is added; the schema is a description of an existing surface (principle 1 pass); the strict-params rule removes a silent no-op (principle 4 pass).

Approach

Schema bundle

  • felis-cli: schema = ["dep:schemars", "felis-protocol/schema"] feature; derive JsonSchema on SessionObject, AttachmentObject, LastNotificationObject, SessionRef, TagResult, SwitchResult, RetargetResult, DaemonStatusResult, ResourceObject, ProtocolVersion, ConfigPathResult, CheckResult, DiagnosticObject, EffectiveConfigResult, DoctorResult, CheckObject, CaptureRow, SearchMatch, NotificationObject, MachineError, ErrorKind, and the private envelope structs (ErrorObject, EndTerminal, ErrorTerminal, LagEvent, Versioned) — plus #23's ListResult. Bridge: Envelope/Body (cli_bridge.rs:334-360) and the new typed Request/per-op params.
  • Constraints schemars cannot derive from serde alone go on the types: id as pattern: ^[0-9a-f]{32}$ (SessionHex), attachment id as ^[0-9]+$ (decimal string, cli_output.rs:492-495), rows/cols 0..=65535 on the bridge request, v as const 1, count/dropped as u64. Use #[schemars(regex(...), range(...))].
  • Output: crates/felis-cli/schemas/felis-cli-v1.schema.json (one document per CLI verb class: point result union, point error, stream item union, lag, end, error terminal) and felis-cli/schemas/felis-bridge-v1.schema.json (request, and the six envelope shapes). Versioned by the SURFACE_VERSION epoch (cli_output.rs:35); the file name carries v1 so a future epoch adds a file rather than rewriting one.
  • Sync test cli_schema in felis-cli mirroring config_schema (UPDATE_SCHEMA=1 writes, otherwise asserts byte equality); extend just schema (justfile:150-151) to run both; CI's --all-features run (.forgejo/workflows/pr.yml:42) executes it, which is the "deterministic and checked in CI" criterion. Pin schemars output stability through the flake's toolchain (already the case).
  • ErrorKind in the schema is an open string with the known tokens as an examples/x-known-values annotation, not an enum: #23 makes the vocabulary additive, and a consumer's validator must not reject a newer kind.

Fixture validation

  • Dev-dependency on a JSON Schema validator (jsonschema crate, or boon; check cargo deny check licenses first). One helper assert_valid(schema_ref, &Value).
  • Every existing e2e test that parses machine output (tests/cli_sessions.rs parse_point/parse_jsonl at 271-299, tests/cli_bridge.rs assert_surface_version at 254) calls the validator, so every documented object is exercised against the bundle by the tests that already produce them (acceptance criterion 1).
  • Negative fixtures under crates/felis-cli/tests/fixtures/schema-invalid/*.json: a 31-hex id, an uppercase id, a numeric attachment id, rows: 65536, an unknown param key, a missing v, a v: 2; the test asserts each fails validation and (for requests) that the bridge answers malformed_request when fed the same line.

Golden conversations

  • crates/felis-cli/tests/fixtures/bridge/<name>.jsonl: each line {"dir":"in"|"out", …object…}; a harness in tests/cli_bridge.rs feeds the in lines, collects stdout until the expected terminal count, masks id-valued session hex and timestamps, groups out lines by request id, and diffs each group's sequence against the fixture; across groups it diffs as a multiset. Scenarios: concurrency (two streams + one point interleaved), cancel racing a terminal (cancel after the stream already ended → malformed_request or no-op, whichever #21/#24 settle), daemon loss mid-stream (fixture daemon killed after N items; expects one daemon_lost terminal per open op and exit 2), stdout loss (Stdio::piped() reader closed early; expects exit code per #21), failure before the first item (sessions.capture on an unknown id → one error terminal, no item), reordered concurrent replies (two point requests whose replies arrive in either order; multiset check).
  • Lifecycle properties JSON Schema cannot state stay as prose in cli.md "Other verbs" and as harness assertions: exactly one terminal per id, id free after terminal, no object after terminal.

Cascade

  • docs/reference/cli.md: "Machine output" and the bridge bullet link the two schema files with a raw URL like config.md:83; a sentence on the fixtures' role. docs/reference/ipc.md:1624 bridge section links the request schema. docs/reference/testing.md gains the fixture layout and the UPDATE_SCHEMA recipe. docs/explanation/architecture/control-surfaces.md "Machine output" records: generated from the serde types (rejected: hand-written, drifts), open kind (rejected: enum), strict request params (rejected: lenient, silent no-op). skills/felis/SKILL.md links the schema for consumers writing validators. CHANGELOG.md: added schemas; bridge now refuses unknown request fields.
  • .claude/skills/add-config-key / extend-ipc skills: extend-ipc must say "regenerate just schema after any cli_output.rs change" (skills go stale like docs).

Dependencies

Must land after every issue that changes a shape it would freeze: #23 (list as point, short_id removal, usage kind, exit table), #24 (retarget result fields), #20 (spawn correlated ops, changes bridge spawn failure shape), #21 (bridge bounds and stdout-loss behavior, which one golden conversation encodes), #26 (daemon status scope semantics), #50 (what unsupported means on the bridge). #12's order (step 10, "after the bridge, CLI, and minor-evolution contracts settle") still holds. #35 (post-v0.1 class rule) is not a dependency; the schema's per-class documents make the classification checkable.

Risk/effort

L. The schemars derive and sync test are S; typing the bridge params is M (touches every op arm in cli_bridge.rs); the golden harness with normalization is the bulk and the flake risk. Main risk: brittle fixtures from timing (mitigated by per-id grouping and multiset comparison, and by using the existing quiet fixture daemon at tests/cli_bridge.rs:89-120). Secondary: a validator crate that fails cargo deny or drags a large tree into dev-deps; check before choosing.

Labels

Keep priority/P1, release/v0.1.0. If the schedule tightens, the schema bundle plus fixture validation is the part that must precede the tag (it is what makes epoch 1 checkable); the reordered-reply and stdout-loss conversations could follow as v0.1.x work without reopening the contract, but only if #21's behavior is at least covered by its own tests.

## Triage plan (2026-09-03) Source-grounded triage against `main` at `69076d42`, reviewed through seven rounds of an independent reviewer (`pi` sol/luna) until it passed with no findings. The dependency order that supersedes the tracker's is posted on #12. ## Claim check Accurate at HEAD. - **Two machine-readable schemas exist, one does not.** The wire is `crates/felis-protocol/proto/felis.proto` (REQ-111, `docs/reference/spec.md:65`); the config is `crates/felis-client-core/felis-config.schema.json`, generated by a schemars sync test behind the `schema` feature (`justfile:143-151`, `crates/felis-client-core/Cargo.toml:26-41`, `docs/reference/config.md:59-87`). The CLI/bridge objects are serde structs only: `cli_output.rs:203-706` (envelopes, `SessionObject`, results, `ErrorKind`), `cli_bridge.rs:334-360` (`Envelope`, `Body`), request parsing by hand in `parse_request`/`Params` (`cli_bridge.rs:398-470`). No schema file or schema mention exists for them (`grep -n "schema" docs/reference/cli.md docs/reference/ipc.md` finds none). `felis-cli` has no schemars dependency (`crates/felis-cli/Cargo.toml`). - **Prose is the only contract.** `docs/reference/cli.md:87-130` (envelope table, kind list), `615-735` (bridge shapes, ops, params, bounds), `ipc.md:1624` ("Non-Rust clients: the stdio bridge"). - **Conversation tests exist but are not golden fixtures.** `crates/felis-cli/tests/cli_bridge.rs` covers correlation (276), two concurrent streams each ending once (314), malformed line (488), stdin EOF cancels (531), daemon loss for point (574) and for every outstanding op (663), id reuse (615), cold socket (712), cancel (902), clean terminal `count` (949). They are Rust assertions over live output, not committed JSONL. Missing entirely: stdout loss (owned by #21's behavior change), failure before the first item on a bridge stream (the CLI has it at `tests/cli_sessions.rs:326`, the bridge does not), and any reordering-tolerant golden for concurrent replies. - **Request-side validation is partial.** `parse_request` checks `v`, `id` type, `op` type, `params` is object; `rows`/`cols` are bounded to `0..=65535` bridge-side (`cli.md:686-694`); unknown `params` keys are ignored (the `Params` accessors read by key). So "unknown fields … fail fixtures" is a new rule, not a documented one. Nothing already done; nothing wrong in the issue. ## Verdict **accept-with-changes.** 1. **Generate, do not hand-write.** Reuse the config-schema pattern: derive `schemars::JsonSchema` on the `cli_output.rs` and bridge types behind a `schema` feature in `felis-cli`, with a `UPDATE_SCHEMA=1` sync test and a `just schema` extension. A hand-written bundle would drift from the serde types the way prose already does. 2. **Type the bridge request params.** Replace `Params<'_>` accessors with per-op `#[derive(Deserialize, JsonSchema)] #[serde(deny_unknown_fields)]` structs. That yields the request schema for free *and* makes the bridge refuse unknown fields at runtime as `malformed_request`, so the fixture rule and the binary agree. This is the one behavior change; it is pre-freeze and explicit (principle 4) and should be stated in CHANGELOG. 3. **Golden conversations must be order-normalized.** Concurrent stream interleavings and daemon-assigned ids are nondeterministic; a byte-exact golden would flake. Fixtures should be per-id expected sequences plus a whole-conversation multiset, with ids and timestamps masked. "Golden" then means committed JSONL the harness diffs after normalization. Principle check: no capability is added; the schema is a description of an existing surface (principle 1 pass); the strict-params rule removes a silent no-op (principle 4 pass). ## Approach ### Schema bundle - `felis-cli`: `schema = ["dep:schemars", "felis-protocol/schema"]` feature; derive `JsonSchema` on `SessionObject`, `AttachmentObject`, `LastNotificationObject`, `SessionRef`, `TagResult`, `SwitchResult`, `RetargetResult`, `DaemonStatusResult`, `ResourceObject`, `ProtocolVersion`, `ConfigPathResult`, `CheckResult`, `DiagnosticObject`, `EffectiveConfigResult`, `DoctorResult`, `CheckObject`, `CaptureRow`, `SearchMatch`, `NotificationObject`, `MachineError`, `ErrorKind`, and the private envelope structs (`ErrorObject`, `EndTerminal`, `ErrorTerminal`, `LagEvent`, `Versioned`) — plus #23's `ListResult`. Bridge: `Envelope`/`Body` (`cli_bridge.rs:334-360`) and the new typed `Request`/per-op params. - Constraints schemars cannot derive from serde alone go on the types: `id` as `pattern: ^[0-9a-f]{32}$` (`SessionHex`), attachment `id` as `^[0-9]+$` (decimal string, `cli_output.rs:492-495`), `rows`/`cols` `0..=65535` on the bridge request, `v` as `const 1`, `count`/`dropped` as `u64`. Use `#[schemars(regex(...), range(...))]`. - Output: `crates/felis-cli/schemas/felis-cli-v1.schema.json` (one document per CLI verb class: point result union, point error, stream item union, lag, end, error terminal) and `felis-cli/schemas/felis-bridge-v1.schema.json` (request, and the six envelope shapes). Versioned by the `SURFACE_VERSION` epoch (`cli_output.rs:35`); the file name carries `v1` so a future epoch adds a file rather than rewriting one. - Sync test `cli_schema` in `felis-cli` mirroring `config_schema` (`UPDATE_SCHEMA=1` writes, otherwise asserts byte equality); extend `just schema` (`justfile:150-151`) to run both; CI's `--all-features` run (`.forgejo/workflows/pr.yml:42`) executes it, which is the "deterministic and checked in CI" criterion. Pin schemars output stability through the flake's toolchain (already the case). - `ErrorKind` in the schema is an open `string` with the known tokens as an `examples`/`x-known-values` annotation, not an `enum`: #23 makes the vocabulary additive, and a consumer's validator must not reject a newer kind. ### Fixture validation - Dev-dependency on a JSON Schema validator (`jsonschema` crate, or `boon`; check `cargo deny check` licenses first). One helper `assert_valid(schema_ref, &Value)`. - Every existing e2e test that parses machine output (`tests/cli_sessions.rs` `parse_point`/`parse_jsonl` at `271-299`, `tests/cli_bridge.rs` `assert_surface_version` at `254`) calls the validator, so every documented object is exercised against the bundle by the tests that already produce them (acceptance criterion 1). - Negative fixtures under `crates/felis-cli/tests/fixtures/schema-invalid/*.json`: a 31-hex id, an uppercase id, a numeric attachment id, `rows: 65536`, an unknown param key, a missing `v`, a `v: 2`; the test asserts each fails validation and (for requests) that the bridge answers `malformed_request` when fed the same line. ### Golden conversations - `crates/felis-cli/tests/fixtures/bridge/<name>.jsonl`: each line `{"dir":"in"|"out", …object…}`; a harness in `tests/cli_bridge.rs` feeds the `in` lines, collects stdout until the expected terminal count, masks `id`-valued session hex and timestamps, groups `out` lines by request `id`, and diffs each group's sequence against the fixture; across groups it diffs as a multiset. Scenarios: concurrency (two streams + one point interleaved), cancel racing a terminal (`cancel` after the stream already ended → `malformed_request` or no-op, whichever #21/#24 settle), daemon loss mid-stream (fixture daemon killed after N items; expects one `daemon_lost` terminal per open op and exit 2), stdout loss (`Stdio::piped()` reader closed early; expects exit code per #21), failure before the first item (`sessions.capture` on an unknown id → one `error` terminal, no `item`), reordered concurrent replies (two point requests whose replies arrive in either order; multiset check). - Lifecycle properties JSON Schema cannot state stay as prose in `cli.md` "Other verbs" and as harness assertions: exactly one terminal per id, id free after terminal, no object after terminal. ### Cascade - `docs/reference/cli.md`: "Machine output" and the bridge bullet link the two schema files with a raw URL like `config.md:83`; a sentence on the fixtures' role. `docs/reference/ipc.md:1624` bridge section links the request schema. `docs/reference/testing.md` gains the fixture layout and the `UPDATE_SCHEMA` recipe. `docs/explanation/architecture/control-surfaces.md` "Machine output" records: generated from the serde types (rejected: hand-written, drifts), open `kind` (rejected: enum), strict request params (rejected: lenient, silent no-op). `skills/felis/SKILL.md` links the schema for consumers writing validators. `CHANGELOG.md`: added schemas; bridge now refuses unknown request fields. - `.claude/skills/add-config-key` / `extend-ipc` skills: `extend-ipc` must say "regenerate `just schema` after any `cli_output.rs` change" (skills go stale like docs). ## Dependencies Must land after every issue that changes a shape it would freeze: **#23** (list as point, `short_id` removal, `usage` kind, exit table), **#24** (retarget result fields), **#20** (spawn correlated ops, changes bridge spawn failure shape), **#21** (bridge bounds and stdout-loss behavior, which one golden conversation encodes), **#26** (daemon status `scope` semantics), **#50** (what `unsupported` means on the bridge). #12's order (step 10, "after the bridge, CLI, and minor-evolution contracts settle") still holds. #35 (post-v0.1 class rule) is not a dependency; the schema's per-class documents make the classification checkable. ## Risk/effort **L.** The schemars derive and sync test are S; typing the bridge params is M (touches every op arm in `cli_bridge.rs`); the golden harness with normalization is the bulk and the flake risk. Main risk: brittle fixtures from timing (mitigated by per-id grouping and multiset comparison, and by using the existing quiet fixture daemon at `tests/cli_bridge.rs:89-120`). Secondary: a validator crate that fails `cargo deny` or drags a large tree into dev-deps; check before choosing. ## Labels Keep **priority/P1**, **release/v0.1.0**. If the schedule tightens, the schema bundle plus fixture validation is the part that must precede the tag (it is what makes epoch 1 checkable); the reordered-reply and stdout-loss conversations could follow as **v0.1.x** work without reopening the contract, but only if #21's behavior is at least covered by its own tests.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
ci/darwin-smoke
No results found.
Labels
Clear labels   
priority/P0
Must be resolved before the v0.1.0 tag

  
priority/P1
Must be settled before the first public contract freezes

  
priority/P2
Explicitly deferred until after v0.1.0

  
release/v0.1.0
Work tracked for the first tagged release

  
status/blocked
Cannot start now: waits on an upstream release, another issue, or a recorded revisit trigger

  
status/planned
Triaged with a source-grounded plan and a passed pi review; ready to implement

  
type/bug
Shipped behavior that contradicts the docs or the code's own intent; needs a product code change

  
type/design
An open scope or trade-off question; needs a recorded decision in the owning explanation doc before code

  
type/test-gap
Documented or reviewed behavior that no test pins; usually split from a review finding, needs no product code change

  
type/tracker
Parent issue that tracks child work

No labels
priority/P0
priority/P1
priority/P2
release/v0.1.0
status/blocked
status/planned
type/bug
type/design
type/test-gap
type/tracker
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
natsukium/felis#29
No description provided.