Skip to content

MnemoScript v2 + AI assistant (2.0.0, 2.1.0, Codex) - #1

Open
ehitimum wants to merge 31 commits into
mainfrom
feature/ai-assistant
Open

ehitimum wants to merge 31 commits into
mainfrom
feature/ai-assistant

Conversation

@ehitimum

Copy link
Copy Markdown
Owner

Brings main up to date with everything shipped since it was last updated: the v2 work (released as v2.0.0 from v2_stable_UI) and the AI assistant (released as v2.1.0 from this branch), plus the ChatGPT/Codex connection that came after 2.1.0.

What's in it

v2 (already in v2_stable_UI)

  • Smooth gliding caret on all platforms, data-safety pass (atomic writes, flush on switch/close), separate desktop and phone shells, map studio refinements.
  • Unit + regression test suites and CI; Windows installer and signed Android APK packaging.

AI assistant (phases 0–5)

  • Providers: Claude Code (the writer's own subscription, run locked down), API keys for Claude / OpenAI / Gemini. One AI per project; models switchable per message; sessions stored per project.
  • Chat panel (desktop) and AI tab (phone); Plan / Ask / Auto modes enforced in code.
  • AI edits to chapters (formatting kept), mind maps and fantasy-map names, with Apply/Reject cards and Undo.
  • Inline commands in the editor: Polish, Proofread, Fix grammar, Shorten, Expand, Change tone, Continue writing — always previewed, one undo step.
  • Settings → AI: exact plan limits from claude -p /usage, API keys, default mode, usage through MnemoScript.

Phone pairing

  • A phone uses the PC's Claude Code over Wi-Fi: SPAKE2 pairing code, per-phone keys, XChaCha20-Poly1305 on every message, replay protection, instant revocation. Tool calls run on the phone's own documents.

After 2.1.0: ChatGPT plan through Codex CLI

  • New provider "ChatGPT (your subscription)" driving the writer's signed-in Codex, locked down the same way; shown only when Codex is installed.

Design notes: docs/ARCHITECTURE.md (AI assistant, Phone pairing, Codex). Change log: docs/DEVLOG.md. Roadmap and future connections (Gemini CLI, OpenCode, Ollama): docs/IMPROVEMENT_PLAN.md.

Testing

  • npm run check: lint 0 errors, type check, 116/116 Vitest (unit + regression), production build.
  • cargo test ai:: (35 incl. an end-to-end pairing test over loopback), cargo clippy -- -D warnings clean.
  • Live tests against the real CLIs passed: Claude Code round trip + model switch, /usage, Codex round trip; a probe confirmed Codex cannot read local files.
  • Browser preview checks of the desktop and phone layouts.

Not yet verified on real hardware: the AI features in the installed desktop app, phone pairing on a real phone and Wi-Fi.

🤖 Generated with Claude Code

ehitimum and others added 26 commits June 25, 2026 17:18
… map studio refinement

- Restore the Word-style smooth caret as a ref-driven overlay (SmoothCaret.tsx),
  enabled on desktop, web and mobile with a Settings/palette toggle
- Rewrite Editor.tsx: keyed per document, no per-keystroke re-render,
  static CSS in index.css, readable page column, settings extracted to SettingsPanel
- Flush-save before document/project switches, on app background and window close;
  canvases flush pending edits on unmount/Save and tag updates with their docId
- Rust: atomic writes, content-free project.json, tolerant document loading,
  rename_project / delete_project / export_file, raw-body save_asset
- Choose the phone vs desktop shell by platform (lib/platform.ts), not window width
- Fantasy map: touch strokes, safe PNG export, hillshade + smoothed inked coastline,
  measured/haloed labels, single-shape grid, image cache, hotkeys, phone panels
- ESLint 50 -> 0 errors; docs: IMPROVEMENT_PLAN.md, DEVLOG, ARCHITECTURE

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…release metadata

- Vitest + jsdom + Testing Library: 47 unit tests (platform, proseFlatText,
  heightmap, mapTypes, generator, browser api) and 18 regression tests that mount
  Sidebar / Editor / MindMap / App (shell selection, flush-on-switch, late canvas
  flush routing, smooth-caret overlay, edit echo)
- cargo test: 11 cases for project.rs / lib.rs (atomic write, safe names,
  content-free project.json, registry, corrupt-file tolerance, legacy files,
  concurrent saves)
- Fix: write_atomic used a shared *.tmp name; concurrent writers of registry.json
  renamed each other's temp file away. Unique temp names + a registry mutex.
- CI workflow (lint, typecheck, tests, build, cargo test, clippy) on every push
- Version 2.0.0 across package.json / tauri.conf.json / Cargo.toml; app title;
  README, AGENTS.md, docs/TESTING.md, DEVLOG, roadmap updated

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Windows
- bundle.windows: NSIS per-user install (no UAC), silent WebView2 bootstrapper,
  English-only; installer metadata (publisher, descriptions, copyright, category)
- Default window 1280x820 (min 760x520), centred
- Fix: the installed executable was the Cargo package's generic "app.exe".
  Added [[bin]] name = "MnemoScript" + mainBinaryName so it installs as
  MnemoScript.exe. [lib] name = "app_lib" is unchanged on purpose: gen/android
  links libapp_lib.so by that exact name.

Android
- Release signing wired into gen/android/app/build.gradle.kts from
  keystore.properties; keystore.jks + keystore.properties are gitignored and
  the gradle block degrades to an unsigned build when they are absent.
- arm64 release APK is 59 MB vs 237 MB universal and 692 MB debug.

Docs
- New docs/INSTALL.md: Windows wizard/MSI/portable/silent install, SmartScreen,
  updating, uninstalling; Android sideloading and a back-up-your-keystore warning
- README install sections, ANDROID.md signing note, roadmap item for the
  oversized bundled speech assets, DEVLOG entry
- .gitignore: release/ artifacts and the signing keystore

Verified on this machine: silent install, upgrade over the previous install
(old app.exe removed, shortcuts repointed), Start-menu + desktop shortcuts,
Add/Remove Programs entry, app launches with a visible "MnemoScript" window and
live WebView2 renderers, backend writes ~/.mnemoscript/registry.json.
APK signature checked with apksigner; package id, label and RECORD_AUDIO present.
Tests: 65 Vitest, 11 cargo test, eslint and clippy all clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Spike harness that drives the user's own Claude Code CLI headlessly with all
built-in tools disabled and only MnemoScript MCP tools exposed. Lockdown, ask
mode, plan mode, cross-model handoff and usage-data tests all pass; one session
can span models (plan A). Findings in docs/AI_PHASE0.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Backend (src-tauri/src/ai): provider-neutral turn engine with tool round
trips and cancellation; Claude Code bridge that runs the user's own CLI with
all built-in tools disabled and an in-app loopback MCP endpoint; Anthropic,
OpenAI and Gemini API-key adapters (streaming + tool loops); sessions stored
per project in ai/sessions/; API keys in the OS secret store (desktop) or an
app-private file (Android).

Frontend (src/lib/ai): types, provider/model catalogue with per-message model
switching, document-to-text conversion, read tools with mode enforcement,
context builder that replays other models' turns, session reducer, runTurn.

Debug builds now use ~/.mnemoscript-dev and a separate keychain service so
development never touches the installed app's projects or keys.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Desktop: AI side panel (Ctrl/Cmd+L, top bar button, command palette),
  exclusive with the Format panel; sessions list, new session, connect
  screen, model picker, Plan/Ask/Auto switch, markdown replies, tool chips.
- Phone: AI tab between Write and Tools; stacked layout that hides the tab
  bar while typing and lifts above the keyboard; model picker as a sheet;
  API-key providers only.
- useAiChat hook owns per-project sessions, streaming (rAF batched),
  persistence and provider status; dev-only mock provider (?aimock) for
  the browser fallback.
- Fixes: phone top bar squashed by flex-0, Format disabled on canvases,
  Vite dependency scan scoped to index.html, cold-import test flake.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Write tools: edit_text, write_document, create_document, edit_mind_map,
  edit_fantasy_map. Plan mode never offers them.
- Ask: each change is a card with a word diff and Apply/Reject; the turn
  waits and re-reads the document on approval. Auto: applies at once with
  Undo / Undo all, warning when the document changed afterwards.
- Text edits keep formatting (only changed words are replaced inside a
  paragraph), tolerate curly quotes, spacing and markers, refuse missing
  or ambiguous text, and are all-or-nothing per call.
- App bridge applies edits to open documents live; canvases flush pending
  edits first and remount on change. Pending approvals badge the AI button.
- Claude Code gets MCP_TOOL_TIMEOUT=24h so approvals cannot time out.
- Fixes: word count after external content changes, mind map id clashes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Polish, Proofread, Fix grammar, Shorten, Expand, Change tone (6 tones)
  and Continue writing: one tool-less request each to the project's AI,
  never stored as chat, always previewed before anything changes.
- Triggers: slash menu AI group, Ctrl/Cmd+J and a selection button on
  desktop, AI section in the Format panel and the phone Tools sheet.
- Preview card streams the result, highlights the passage, shows a word
  diff and tone chips; Accept writes only the changed words in one undo
  step (formatting kept), with a plain-paragraph fallback.
- Phone: card docks above the tab bar and the page scrolls the passage
  into view.
- Inline runner lives outside useAiChat (React Compiler memoization).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New ai_cli_usage command runs `claude -p /usage`, which Claude Code
  answers itself (no model call, no cost): session (5-hour) and weekly
  percentages with reset times. Refused if an old CLI sent it to a model.
- Settings -> AI on desktop and phone: Claude account card with limit
  bars (desktop only), API keys add/replace/remove, default mode for new
  projects, and tokens/cost used through MnemoScript today and this week
  (logged on this device from chat replies and inline commands).
- Tests: usage parser fed the real report, local usage totals, default
  mode, settings regression on both shells, Rust parser test and a free
  ignored live test against the installed CLI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- PC: Settings -> AI -> Phone pairing (switch, 8-digit code, paired
  phones with Remove); LAN server with UDP discovery, streamed replies,
  keep-alive that cancels turns of vanished phones.
- Phone: AI -> Connect -> Your PC (discovery or typed address + code),
  Settings -> Your PC (status, the PC's plan limits, Unpair). Tool calls
  from the PC run on the phone's own documents.
- Security: SPAKE2 pairing (5 tries, 5 minutes), per-phone keys,
  XChaCha20-Poly1305 on every message bound to path/device/request/line,
  replay and clock checks, Claude Code turns only, instant revocation.
- Tests: Rust loopback end-to-end + crypto; phone Connect regression.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Version 2.1.0 (package.json, tauri.conf.json, Cargo.toml).
- Roadmap: Gemini CLI, OpenCode (free models), Ollama and Codex noted as
  future connections, after this release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- New provider 'ChatGPT (your subscription)': runs the signed-in Codex
  CLI per message (exec --json, ephemeral, user config and rules ignored,
  read-only sandbox, empty folder, prompt on stdin) with every built-in
  tool family disabled and only MnemoScript's MCP tools, pre-approved.
- Connect shows Codex only when installed, with codex login guidance.
- Live test: Codex calls list_documents and answers correctly; a probe
  confirmed it cannot read local files.
- Notes the OpenCode finding (free-tier limit, retry loop) for later.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ehitimum and others added 3 commits November 12, 2026 23:48
Security and lost edits (items 1-3 shipped in 2.1.0), each with a regression:
- Paired-phone path traversal: per-turn scratch folders get a random name
  under temp/mnemoscript-ai/turns (0700 on Unix) instead of the turn id, and
  Turn::register refuses ids that are not plain tokens. The loopback pairing
  test sends a ../ turn id.
- Inline AI inserted a word at the top of the document when it started a
  later paragraph of the selection (separator position -1 + 1 = 0).
- persistCurrent marked typing done during a save as saved and swallowed
  failed saves; leaving a chapter (switch, close project, close window) now
  asks before dropping edits that could not be saved.

Hardening:
- fs read/write scope narrowed from ** to dialog-picked paths.
- Real CSP instead of csp: null (checked in a browser on the production
  build: no violations; not yet run in the installed app or on Android).
- npm run build regenerates the speech assets first (prebuild).

Version 2.1.1 (not tagged). Docs updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ehitimum and others added 2 commits November 15, 2026 16:15
app/.gitignore already lists public/ort and public/vad as generated. npm run
build regenerates them byte-identically from node_modules (prebuild ->
setup:speech), so CI and release builds still ship them. Files stay on disk.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The flag flips a tick after storage shows the text; under a loaded test run
the synchronous lookup could run first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ehitimum

Copy link
Copy Markdown
Owner Author

Changes for the review (2.1.1)

The must-fix items are addressed in af78421 and 17ae5fe (plus the test tweak 4e4e500). Each fix has a regression test. Items 4 and 5 were already fixed in bdaab93 / 991a20e.

# Finding Change
1 Paired phone path traversal through turnId Scratch folders are now temp/mnemoscript-ai/turns/<random uuid> (Scratch::new, 0700 on Unix) and are never named from the turn id, in both claude_code.rs and codex.rs. Turn::register refuses ids that are not plain tokens (valid_turn_id). The loopback pairing test sends a ../ turn id and asserts the turn never runs and nothing appears outside. Tests: turn_ids_must_be_plain_tokens, scratch_folders_never_take_their_name_from_the_turn, extended phone_pairs_runs_a_turn_….
2 Inline AI inserts at the top of the document insertionPoint in rewrite.ts: after the previous real character, or, when that is the paragraph separator, before the next real one (as plain text, so it doesn't take that word's bold). Test: regression/ai-rewrite.regression.test.ts with the review's example.
3 Edits lost while "Saved" persistCurrent marks saved only if the document object is unchanged after the write, and returns success. leaveDocument asks before switching document, opening or closing a project, or closing the window when the save failed (native dialog), and stays if the writer declines. Tests in regression/app.regression.test.tsx: typing during a save stays unsaved; a failed save asks and stays.
6 96.5 MiB of generated binaries app/public/ort and app/public/vad are no longer tracked (17ae5fe). npm run build regenerates them first (prebuild → setup:speech, from node_modules, no network). All 29 files were checked to be byte-identical to the committed ones. MnemoScript_Datasheet.docx and User Story List.docx are untouched: your call.
7 Whole-disk fs scope, no CSP The ** read/write scopes are gone: only dialog-picked paths are allowed (the dialog plugin adds them to the scope; Android pickers return content URIs that need none). A real CSP replaces csp: null: default-src 'self', wasm-unsafe-eval, blob workers, the asset/IPC hosts and LanguageTool, plus a dev CSP for Vite.

Version bumped to 2.1.1 in package.json, Cargo.toml and tauri.conf.json (not tagged; tagging and release are yours).

Checks

npm run check (125 tests, lint, types, build) · cargo test 47 passed · cargo clippy --all-targets -- -D warnings clean.

The CSP was checked in a browser against the production build with the same policy: chapter editing, mind map, fantasy-map generation, WebAssembly and workers all ran with no CSP violations.

Not verified:

  • The CSP and the narrowed scope in the installed desktop app or on Android (image import, map PNG export, dictation).
  • The native "leave without saving" dialog.

Not addressed: the should-fix list (items 8–16) is unchanged in this push.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant