Skip to content

vector-gpui: native GPUI client (experiment) - #102

Draft
JSKitty wants to merge 511 commits into
masterfrom
gpui
Draft

JSKitty wants to merge 511 commits into
masterfrom
gpui

Conversation

@JSKitty

@JSKitty JSKitty commented Nov 11, 2026

Copy link
Copy Markdown

Requested by JSKitty · project thread

A long-lived experiment branch: Vector drawn natively with GPUI instead of a webview, aiming for exact parity in layout, palette, fonts and features with the Tauri client. It isn't meant to merge any time soon; the PR is here so the diff and discussion stay in one place.

What's in this first cut

  • crates/vector-gpui: its own workspace and lockfile, so it never touches the Tauri client's dependency graph. The only shared-file change is excluding it from crates/Cargo.toml. GPUI is pinned to Zed f8cfc7e.
  • The widescreen shell: window chrome with client-side decorations, the rail (Messages, New Messages, Communities, Collapse, account), the chat list (virtualised uniform_list) and the open chat (virtualised list with date and NEW dividers). The NEW divider stays put while the chat is open.
  • The composer: real text input with selection, clipboard and IME. Enter sends a local echo; the send pipeline isn't wired up yet.
  • Types come straight from vector-core (SerializableChat, SlimProfile, Message). data.rs mirrors the JS derivations (getName, chatPreviewText, timeAgo, unread counts), and each helper names the function it copies.
  • Icons are compiled in from src/icons, so an icon changed in the Tauri client arrives here on the next build. Rubik is cut from the shipped variable font into static weights; Inter is bundled too.

Parity harness (reference/)

capture.mjs boots the real frontend in Chromium through its debug hot-reload path, using mock-tauri.js and the same fixtures.json. It needs no Rust, account or network. It saves a screenshot plus every element's box, font, colour and border. shot.sh renders vector-gpui under Xvfb, and compare.py puts the two side by side with a diff.

On the 1200×760 chat screen, 1.8% of pixels differ by more than 24/255. Most of that is text anti-aliasing; the remaining offsets are about 1px.

Numbers (release build, Linux, software Vulkan)

  • Window on screen 45 ms after launch.
  • About 42 MB of private anonymous memory. Total RSS is 155 MB here, but ~70 MB of that is Mesa's software renderer (libLLVM, lavapipe), which a real GPU driver doesn't load.
  • Binary is 27.8 MB stripped, including 4.4 MB of fonts.

Known gaps

  • Only the widescreen layout so far, not slim or the Centre Console.
  • No live account yet: it reads fixtures, not vector-core's DB or relays.
  • GPUI has no letter-spacing, so the NEW label spaces its letters by hand.
  • Ellipsis truncation differs slightly from CSS: CSS keeps the trailing space before ….

Generated by Claude Code

@JSKitty JSKitty self-assigned this Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Claude encountered an error —— View job


I'll analyze this and get back to you.

JSKitty commented Nov 11, 2026

Copy link
Copy Markdown
Author

The claude check failed, but the cause isn't this PR. The Claude Code workflow runs on every opened PR, and it stops at its environment validation because ANTHROPIC_API_KEY is empty in the job: no API key, OAuth token or federation rule is set for the repo. The same job failed on the fix-audio-crackling PR 40 minutes earlier for the same reason. This PR doesn't touch .github/, and adding a repo secret is beyond what a PR can do, so there's no fix to port here. It goes green when a maintainer adds ANTHROPIC_API_KEY (or CLAUDE_CODE_OAUTH_TOKEN) to the repository secrets. Alternatively, dropping the bare pull_request trigger from .github/workflows/claude.yml stops it running unasked.


Generated by Claude Code

Comment thread .github/workflows/gpui.yaml Fixed
…ple statically [package]

GPUI's wgpu context started Vulkan and GL together; GL is now the fallback for machines
without a Vulkan adapter, which saves 17 MB on Mesa. The static FFmpeg's Opus decoder
needs swresample, which the macOS package build lacked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…PU API

The model renderer's device holds driver memory for the session; after 20 seconds with
no frame asked for, scenes park on the CPU and the device goes, coming back on the next
frame. It no longer starts every backend at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
FFmpeg's frame threading gave each poster probe five decoder threads with their own
buffers, for a single frame. Playback keeps FFmpeg's choice. A chat with two videos
holds 15 MB less.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A wallpaper chat drops from +33 MB to +21 MB, a video chat from +29 MB to +19 MB.
Wallpapers far larger than the pane keep only a cover-sized copy for re-bakes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
WebKitGTK, GTK and libsoup now load only in vector-gpui-miniapp, on the first
Mini App or player window, so the app itself maps 19 MB less and dirties 4 MB
less. The helper's webxdc:// requests are answered by the app over its pipe; a
crashed page closes its windows and leaves the app running. Notifications get
a GLib loop thread of their own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A forked probe names the drivers behind the hardware adapters (or the CPU one
when there is none), checks that a narrowed loader still finds them all, and
sets VK_LOADER_DRIVERS_SELECT. The choice is cached against the GPUs present
and the driver manifests installed. −14 MB RSS, −4 MB private, and the window
appears in half the time once cached. A user's own loader variables win.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
The vendored gpui_wgpu shares its device; the 3D renderer takes it instead
of bringing up a second instance and device. A 3D chat costs 23 MB less.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Profiles that never loaded are queued again, and what sat idle is let go: the
audio engine's decoded sources, the Whisper context and the messages of chats
the timeline isn't showing, then the freed heap goes back to the OS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Undo and Redo (a run of typing is one step), word-wise moves, selection and
deletes, and Shift+Home/End, with macOS's Option and Cmd variants. The
composer's Ctrl/Cmd B, I, E and Shift+X write markdown through a port of the
desktop's cmpToggleMark. Send on Enter is honoured: off, Enter breaks the
line, and Ctrl/Cmd+Enter always sends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A sync that reached no relay waits the connect grace, then says new messages
may be missing, as on desktop. Our own open profile re-reads its badges when
core reports they changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
The roster groups members under the roles shown separately, colours names by their
topmost coloured role, marks bots, and lists only a Private Channel's readers. Each chat
keeps its composer as a draft, an edit sets aside what was typed and gives it back, and a
refused send, failed reaction, a join undone by a kick and a late invite link each say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
The role graph lives on the store, so a channel's authors take their topmost coloured
role and the new roles drop the Owner chip, as on desktop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A release that lists this build's platform (`<os>-<arch>-gpui`) in latest.json is
downloaded with a progress bar, checked against the release minisign key, and put in
place on Restart Now: the .app on macOS, the AppImage on Linux, the NSIS installer on
Windows. Without such an entry, Download Update still opens the release page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…rts; Copy Role ID

Folds are stored under the desktop's own keys and `<community>:<id>` entries. A role's
roster heading offers Copy Role ID in Advanced Mode.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
As on desktop, a manager's tap on the Community icon chooses a new one rather than opening
Community Settings, and the pencil becomes a progress ring while it uploads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Drag the list pane's trailing edge to resize it (200-480px, leaving the
conversation 380px), double-click to restore the default; persisted under
ws_list_width like the desktop. Tooltips on the wallpaper sliders, the
in-use transport dot, the voice stage ping, QR pairing, the SAM password
link, the community icon picker and the custom role colour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…last DM

The last channel per community is kept under ws_last_channel:<npub> as the
desktop does, falling back to the primary when it has gone or turned
private. In the wide layout the Messages door opens the last DM, else the
newest with messages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Stored as ws_members_open in the account's settings, like the desktop, so
hiding the members pane holds across communities and restarts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Builds src-tauri's video.rs and video_compression.rs in, so a clip leaves
either client alike: H.264 Main + AAC, at most 720p30, kept only when
smaller. The preview offers Compress Video above 1 MB with its progress,
and Send mid-encode waits as 'Sending when ready'. Settings gains Video
Quality. Only a hardware H.264 encoder counts as available, so a system
FFmpeg's mpeg4 never ships (VECTOR_VIDEO_TEST_ENCODER allows it locally).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Muted, with the chat's own controls, sized as a picture would be; the clip
is forgotten when the preview closes or sends.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…s asked again

Its retry flipped it to a loading skeleton for a frame, jumping the rows around it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
One drawn but partly hidden is centred in one step rather than scrolled to the top
first and centred a frame later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Both decoders read the same stream; one read head flipped between them, so the start
gate and the fetch window followed whichever read last and playback never began.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
The sound's decoder reads as far ahead as it may, so it always sits at the download's
edge; measured from it, playback never had enough in hand to start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1

JSKitty commented Nov 15, 2026

Copy link
Copy Markdown
Author

The vector-core failure on 7a4ebb0 is community::v2::service::tests::a_mint_whose_delivery_lost_a_part_leaves_nobody_out ("the part a relay dropped served nobody", service.rs:30010), a timing-sensitive relay test. The only vector-core change since the last green run is a defaulted MediaSource::remote() (returns false, used only by the GPUI stream reader), which no community code calls, so this is not from this PR. The next run on 689115a will retry it.


Generated by Claude Code

A click quicker than a frame left the scrub held, so the time stuck on the
spot clicked while the picture played on, and the controls never hid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Each reader's head follows its own thread, so a seek moves it instead of
leaving the old spot counted as buffered: the clock no longer runs over a
frozen picture, and a pause no longer replays the backlog. A seek while
playing waits for the new spot's bytes like a first play. The player keeps
the clip's own aspect (contained, never stretched), and a fetched clip shows
its spinner until it plays rather than the play button.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A host slow to answer each request no longer caps playback at one round
trip's worth of bytes; the requests hold only the host, so letting go of the
player still ends them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A demuxer reads its tracks' samples in turn, jumping back and forth; a head
that followed only the latest read let the other spot's bytes go, so the
picture waited on a range it had just fetched. A head now keeps the lowest
offset its reader touched over two seconds. The sound of a remote file
decodes through the windowed reader rather than segment workers that would
each download the whole file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
… flicker

A head remembers its latest read beside its span's low end, so a seek
forward keeps what it lands on. The buffer is measured from where the
picture reads, a quick host is asked one range at a time, and the clock
holds for the network only after the picture has waited 150 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…clip keeps its seek bar

A waiting picture gets a request lane of its own, and a request unanswered
for 8 s is dropped and asked again. On a narrow player the volume slider,
then the speed, then the time give way before the seek bar does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
One open-ended range reads on from where the decoder needs bytes and stops
reading once the window is in hand, so a host's delay is paid once rather
than per request, nothing in flight competes with the playhead, and a read
elsewhere (a seek, the sound, a header at the end) moves the connection. A
connection that brings nothing for 6 s is reopened where it was, keeping
every byte it already brought.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
}

// SAFETY: `CredReadW` succeeded, so this points to a valid `CREDENTIALW` until `CredFree` below.
let result = unsafe { username_and_password(&*credentials) };

let mut credentials: *mut CREDENTIALW = std::ptr::null_mut();
unsafe { CredReadW(target_name, CRED_TYPE_GENERIC, None, &mut credentials) }.unwrap();
let result = unsafe { username_and_password(&*credentials) };
A file that stores its sound far from its picture has each demuxer read two
distant regions in turn. A reader now keeps a head per region, each region
keeps its own connection (up to three, the least recently used closed), and
fetching serves a waiting read first, then the playhead's regions with the
least in hand. A region in hand to the file's end, such as an index there,
no longer holds back the start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…heir own

A range left unread now holds its host back by TCP alone; on a shared HTTP/2
connection it filled the window the range being read needed, so a seek
could trickle. vector-core gains an opt-in HttpClient::http1_only, pooled
apart from the default clients, which are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1

JSKitty commented Nov 15, 2026

Copy link
Copy Markdown
Author

Windows build on b16933a fails one new test, backend::stream::tests::a_reader_in_two_distant_regions_fetches_each_byte_once: a read there timed out after 45 s, while the test passes on macOS. It is a real intermittent stall in the region-aware stream fetcher (a waiting read that never gets served), not a flake to retry. The single-task fetch loop is being replaced with per-connection tasks, and that change will carry the fix along with a repeated run of this test.


Generated by Claude Code

A slow answer for one range no longer holds up another the playhead needs. The
picture's decoder thread is named the playhead, and a reader holding still keeps
its regions for up to 30 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…reads it

The stretched track is an AIFF made a few seconds ahead of the engine's reads,
from an FFmpeg reader that seeks and leaves the picture unread, so a speed
change no longer downloads and decodes the whole clip. The stretcher now takes
its track in pieces, bit-identical to the one-pass result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A hundredth-apart tie is not transitive, and Rust's sort aborts on a
comparator that is no total order: typing a :shortcode could crash the app.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Leaving a chat for the chat list, Settings or anything else floats what plays,
not only opening another chat. A link's video floats too: it is no attachment,
so the message it was played from is remembered for the floating player.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1

JSKitty commented Nov 16, 2026

Copy link
Copy Markdown
Author

The vector-core failure on a365815 is community::v2::service::tests::control_sync::a_relay_that_caps_open_sessions_is_never_written_off_and_every_plane_still_compares. It isn't from this commit: a365815 changes only crates/vector-gpui/src/emoji.rs, and the same vector-core code passed on the commit before it (7bbd8ea), so it looks like that test is timing-sensitive on the runner. The other 2893 tests passed.


Generated by Claude Code

…onger abort

A byte slice cut through a character panics; the prefix and suffix checks on
link paths and pasted npubs read with get() instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
A check with DOCS_RS left a build script output with no libjxl, which a later
plain build reused and failed to link.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Off-screen pictures and resting video clips cost bytes doubled every 10 s
off screen; past a 128 MB budget the costliest go, swept as soon as decodes
land rather than on the 15 s timer. Idle time shrinks with size. What is in
the frame on screen (Window::image_on_screen) or asked for in the last 1.5 s
stays. Decodes for rows scrolled past are skipped. A shed clip keeps its
position, mute, volume and speed. Linked pictures decode at row size
(1600x800) and the viewer reads its own full-size copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
…s too

A still image unpainted for 5 s and absent from the frame on screen leaves
the atlas while its decoded copy stays cached; drawn again it uploads
again. The 15 s picture sweep runs for a fixtures window as for a live one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Qq6RetJnzXvEeXAvSsgvs1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants