Skip to content

feat(macos): captions without Ollama and bundled local translation - #27

Draft
PCcoding666 wants to merge 1 commit into
masterfrom
codex/issue-25-embedded-translation
Draft

PCcoding666 wants to merge 1 commit into
masterfrom
codex/issue-25-embedded-translation

Conversation

@PCcoding666

Copy link
Copy Markdown
Owner

Original captions previously refused to start when Ollama was missing. This change makes captions the default, independent path and ships a pinned llama.cpp runtime with an optional, app-managed Qwen model download.

  • Captions-first onboarding; system audio does not require microphone permission. Separate speech and translation states.
  • Model progress, cancel/resume/retry, integrity checks, corrupt-model replacement and native streaming inference; no production Ollama startup, probes or recovery path.
  • Preserve source-only export/history, stop-time partial captions and repeated text across independent segments; translation failures retain original captions.
  • Embed/sign the runtime framework and include licenses. Update current user/developer documentation.

Validation: macos/Scripts/run_tests.sh passed Debug, Release, warnings-as-errors and 555 XCTest cases. A development-signed arm64 bundle passed strict verification and launched from Applications outside the build tree.

Draft while real GUI/audio acceptance is completed. First-run captions-first UI is observed; privacy authorizations, actual app model download/inference, offline behavior and formal Developer ID/notarized distribution remain unverified. Intel hardware is untested. The official model host timed out in a CLI probe; that is not an app-download result. Local screenshots/logs are being collected for the GUI acceptance follow-up. See docs/validation/2026-11-04-issue-25.md.

Related to #25. This PR does not close the issue or publish a release.

Copilot AI balanced review requested due to automatic review settings November 4, 2026 07:52

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@PCcoding666

Copy link
Copy Markdown
Owner Author

Benchmark + capture check for this PR (details in #28, benchmark/engine_compare_report.md):

Runtime is not the problem. Same Qwen3.5-4B weights on this M2 Pro / 16 GB: llama.cpp Metal (Q4_K_M) vs Ollama MLX (nvfp4) are within noise — p50 end-to-end 542 vs 534 ms, TTFT 146 vs 160 ms, chrF 55.8 vs 57.6, numbers/names kept 97% vs 92%. Embedding llama.cpp costs nothing in latency or quality.

The bundled model is. The exact qwen2.5-1.5b-instruct-q4_k_m.gguf this PR ships, on the same eval set: instruction compliance 90% (vs 98%), terms kept 89%, chrF 47.7. Concrete failures: 38% → 45% translated as 3.8% → 4.5%; 五十万 as fifty thousand; several English inputs echoed back untranslated; 下周三 became "end of next week". Suggest switching the bundled model to a Qwen3.5 GGUF (4B Q4_K_M is 2.74 GB; unsloth/Qwen3.5-4B-GGUF was reachable, the official Qwen/ repo had no Q4_K_M at the time).

Capture on this branch works. Built b5b50fe with dev-run.sh: headless --speech-selftest en-US and zh-CN both clean (0 errors); GUI dual-lane with system audio (zh) + built-in microphone (en) both produced correct finals at the same time, with translation off and no Ollama installed. Not yet exercised: in-app model download and real local inference.

🤖 Posted by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants