Repository navigation
feat(windows): LCT 2.0 — native capture, on-device Whisper recognition, Ollama pipeline and HUD aligned with macOS - #24
PCcoding666 wants to merge 4 commits into
Conversation
…ic plus tests Platform-independent core for the 2.0 Windows app: settings model mirroring the macOS AppSettings, PCM conversion, 16 kHz resampler, energy VAD and utterance chunker (partial/final decode requests), Ollama client (streaming chat with think:false, keep_alive and options.num_ctx; pull with monotonic progress; load/unload/ps), translation queue (debounce, preemption, dedup, retries), model catalog and hardware-based recommender, recovery controller, session exporter (Markdown/TXT/SRT), stall and silence detectors, diagnostics report and SQLite history store. 97 xUnit tests; the workflow now runs them before the publishes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… Whisper and a HUD UI The app now captures audio itself (WASAPI loopback + microphone as two independent channels), recognizes speech on-device with Whisper.net (whisper.cpp ggml models downloaded on demand), and translates through the LCT.Core Ollama client with streaming output and live draft translation. New terminal-HUD WPF interface (forced dark, emerald accent, monospace labels): status bar with recording timer, live-switchable per-channel language label, model and Ollama indicator, latency; SYS/MIC badged segments with timestamps; level meters; real pause/resume; overlay window (topmost, remembered placement, opacity, font size, click-through); tray icon; global hotkeys; first-run wizard; settings covering every AppSettings key; history window; session export; diagnostics export. OllamaGuardian uses an existing Ollama, starts ollama serve only when nothing answers and stops only the process it started. The IPEX-LLM portable build is an optional wizard path for Intel GPUs. Removes the UI Automation sync loop, updater, telemetry, data migration and enterprise windows. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… scripts Rewrites the READMEs around the new architecture and Windows 10 support, adds a user guide and an updated docs index, updates the release-notes footer and installer text, and removes the Live Captions / forced-IPEX era documents and the diagnostic scripts that killed Ollama processes. VersionPrefix is 2.0.0. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…AD under constant noise - Start the main window before closing the wizard so the Closed handler no longer shuts the app down on completion. - Handle Unchecked on the auto-scroll toggle; don't cancel Closing during a real shutdown. - Overlay: dim the last segment through a bool→opacity converter instead of a bool bound to Opacity; history list shows SYS/MIC badges. - EnergyVad: loud frames lift the noise floor very slowly so a constant hum releases the detector after ~16 s instead of locking it in speech; test added. - Probe hardware (WMI) off the UI thread at startup. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 75fa1a3cd1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (isFinal && trimmed == _lastProcessedText) | ||
| { | ||
| return; |
There was a problem hiding this comment.
When a speaker repeats the last translated sentence, or both capture lanes recognize the same words, this check silently discards the new segment's translation request. AppendSegment has already created a distinct card in the Translating state, and no completion event is emitted for the discarded request, so that card stays untranslated indefinitely and its transcript entry never receives a translation. Scope deduplication to the segment ID, or deliver the cached result to each new segment; the text-based removal in EnqueueLocked needs the same treatment for pending requests.
Useful? React with 👍 / 👎.
| if (flushed != null) | ||
| { | ||
| lane.Recognizer.Submit(flushed); | ||
| var recognizer = lane.Recognizer; | ||
| _ = Task.Delay(4000).ContinueWith(_ => recognizer.Dispose(), TaskScheduler.Default); |
There was a problem hiding this comment.
When recognition takes longer than four seconds or has queued finals, stopping capture cancels recognition before the flushed tail can finish: WhisperRecognizer.Dispose immediately cancels its processing token rather than draining its final queue. If Flush() returns null because the chunker already emitted a final, the other branch disposes the recognizer immediately even though that final may still be decoding. Both paths can lose the last spoken sentences from the transcript. Await completion of queued finals before disposing instead of using the chunker's current buffer or a fixed delay as a completion signal.
Useful? React with 👍 / 👎.
| FinalizeDraftAsSegment(lane, source); | ||
| lane.Draft = ""; | ||
| } | ||
| StopLane(source, flush: true); |
There was a problem hiding this comment.
When a recognition-language, microphone-device, or Whisper-model change restarts a lane with a visible draft, this block appends the draft as a permanent segment and then calls StopLane(..., flush: true) for the same buffered utterance. If the flushed decode finishes, OnRecognition appends its final text as another segment, duplicating the speech in the cards, transcript, and potentially history. Clearing lane.Draft does not clear the chunker's audio. Choose either the draft fallback or the flushed final, or reconcile both using the utterance ID.
Useful? React with 👍 / 👎.
| _vm.Ollama.Configure(OllamaClientOptions.From(Local)); | ||
| await ModelManager.PullAsync(model); |
There was a problem hiding this comment.
If capture is running while the user selects a different model or endpoint in Settings and clicks Download, this reconfigures the same OllamaClient used by the live translation queue for the entire download. Subsequent captions therefore use the unsaved model, target language, prompt, and endpoint; selecting a model that is not downloaded yet causes live translations to fail while it downloads. Restoring the options in finally cannot undo those results. Give the download operation its own client/options snapshot rather than mutating the active session's client.
Useful? React with 👍 / 👎.
Summary
LCT 2.0 for Windows replaces the Live Captions scraper with the macOS architecture: LCT captures audio itself, recognizes speech on-device, and translates through a local Ollama. The Windows app now matches the macOS app feature-for-feature and in design (terminal HUD, forced dark, emerald accent, monospace labels).
windows/is replaced wholesale; the release pipeline, installer script and executable name are unchanged.Design doc:
windows/docs/dev/ARCHITECTURE.md. User-facing notes:## [2.0.0] - Unreleasedinwindows/CHANGELOG.md.Architecture
windows/src/LCT.Corenet8.0AppSettings.swiftkey for key), PCM→float, 16 kHz resampler, energy VAD, utterance chunker (partial every 1.2 s / final after 600 ms silence or 25 s), Ollama client, translation queue, model catalog + hardware recommender, pull-progress accumulator, recovery controller, exporter (MD/TXT/SRT), stall/silence detectors, diagnostics report, SQLite history. Builds and tests on any OS.windows/LiveCaptionsTranslator.csprojnet8.0-windows10.0.17763.0windows/Tests/LCT.Testsnet8.0Data flow:
WASAPI → mono float → 16 kHz → RMS level → EnergyVad/UtteranceChunker → WhisperRecognizer (partial/final) → TranscriptionViewModel → TranslationQueue → OllamaClient (stream, think:false, keep_alive, options.num_ctx) → segment / SessionTranscript / HistoryStore.Recognition engine choice
Whisper.net 1.9.1 (whisper.cpp, ggml models) over sherpa-onnx streaming Zipformer:
Models download to
%LOCALAPPDATA%\LCT\models\whisper\with progress and cancel;smallby default,baseunder 8 GB RAM or 4 cores.Ollama
%LOCALAPPDATA%\Programs\Ollama→PATH→ LCT's IPEX portable). Startsollama serveonly when nothing answers, remembers the PID, stops only that process. Never kills a user-started Ollama.options.num_ctx(default 4096, in Settings),think: false,keep_alive(5m/15m/30m/1h/until quit); unload on quit is a toggle.OllamaEndpoint, same rules as macOS)./api/psshows it gone, announce recovery.Cleanup
Removed:
LiveCaptionsHandler, the UI Automation sync loop,AutoUpdateService/UpdateWindow/VersionSettingsControl,TelemetryService,EnterpriseWelcomeWindow,DataMigrationManager,UpdateUtil, the old pages/windows, WPF-UI,Interop.UIAutomationClient,System.Speech,Newtonsoft.Json, the 1.x docs about the removed systems, and the diagnostic scripts that killed Ollama processes.images/show_livecaptions.pngand the README's Live Captions setup chapter are gone; system requirements say Windows 10 1809+.Version & release
<VersionPrefix>→2.0.0;## [2.0.0] - Unreleasedwritten for users. Workflow, NSIS script (names, keys, paths) andRELEASE.mdmechanics unchanged; only descriptive strings and the requirements footer were updated. One new workflow step runsdotnet testbefore the publishes.Remaining differences from macOS
CaptionSegmenter) is therefore unnecessary and not ported.SpeechSelfTest.swift) is not ported.setting.json) are not migrated; 2.0 writessettings.json.Test plan
CI (
Windows Build and Release):dotnet teston LCT.Core (97 tests) + 5dotnet publishvariants + NSIS installer + version check.Also built locally on macOS with
-p:EnableWindowsTargeting=true(C# and XAML markup compile) anddotnet testrun locally.Needs manual verification on Windows
No Windows machine was available; each item lists the expected result.
Install / launch
LCT-v2.0.0-ci.N-Setup.exeinstalls;LiveCaptionsTranslator.exestarts; the portable x64 single-file build starts (Whisper.net native libs extract from the bundle). Expected: no "DllNotFound" forwhisper.dll/ggml*.dll.%LOCALAPPDATA%\LCT\logs\lct<date>.log.First-run wizard
ollama serve;ollama-serve.logwritten; wizard continues to model check.%LOCALAPPDATA%\LCT\ollamaand proceeds.small(orbase) downloads with progress; Complete;setup-completemarker written; wizard never shows again.Capture
SYS ZH · MIC EN → JA; lines stay distinguishable.Detectors & notices
ollama.exewhile capturing → within ~10–50 s a restart notice, then "Ollama is back" (only if LCT started it; a user-started Ollama is never killed but is restarted after it dies).ollama stop <model>while capturing → model reloads automatically.UI
settings.json; Save disabled with an invalid Ollama endpoint; Reset to Defaults keeps window placement.history.db; search/copy/delete/clear/CSV.LCT Session <date>.<ext>; SRT loads in a player.Exit
ollama psempty), LCT-startedollama serveterminated, user-started Ollama untouched.🤖 Generated with Claude Code