Skip to content

feat(user-state): handle browser voice connection loss (CAI-8629) - #756

Open
cmullenx wants to merge 11 commits into
webex:nextfrom
cmullenx:chrmulle/webrtcErrors
Open

cmullenx wants to merge 11 commits into
webex:nextfrom
cmullenx:chrmulle/webrtcErrors

Conversation

@cmullenx

@cmullenx cmullenx commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

COMPLETES # https://jira-eng-sjc12.cisco.com/jira/browse/CAI-8629

This pull request addresses

showing the user an error message when mobius connection drops and preventing user state changes to achieve parity with agent desktop

by making the following changes

  • adds dialog for when mobius connection drops
  • prevents changes to user state when mobius connection ot ready

Vidcast: https://app.vidcast.io/share/40b94562-07d1-4abc-aa12-c382443df773

< DESCRIBE YOUR CHANGES >

Change Type

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Tooling change
  • Internal code refactor

The following scenarios were tested

  • The testing is done with the amplify link
    < ENUMERATE TESTS PERFORMED, WHETHER MANUAL OR AUTOMATED >

The GAI Coding Policy And Copyright Annotation Best Practices

  • GAI was not used (or, no additional notation is required)
  • Code was generated entirely by GAI
  • GAI was used to create a draft that was subsequently customized or modified
  • Coder created a draft manually that was non-substantively modified by GAI (e.g., refactoring was performed by GAI on manually written code)
  • Tool used for AI assistance (GitHub Copilot / Other - specify)
    • Github Copilot
    • Other - Codex
  • This PR is related to
    • Feature
    • Defect fix
    • Tech Debt
    • Automation

Checklist before merging

  • I have not skipped any automated checks
  • All existing and new tests passed
  • I have updated the testing document
  • I have tested the functionality with amplify link

Make sure to have followed the contributing guidelines before submitting.

@cmullenx
cmullenx requested a review from a team as a code owner November 5, 2026 12:40
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 5, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-18T19:27:57.882621Z a2c7d47 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cmullenx cmullenx added the validated Indicates that the PR is ready for actions label Nov 5, 2026
@aws-amplify-us-east-2

Copy link
Copy Markdown

This pull request is automatically being deployed by Amplify Hosting (learn more).

Access this pull request here: https://pr-756.d1b38q61t1z947.amplifyapp.com

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2f601f221f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

- Guard voice snapshot initialization when cc is undefined or getVoiceConnectionState is not yet exposed
- Preserve forced Idle state transition during wellness-code lookup
- Synchronize recovery dialog to open only after Idle state transition successfully resolves
- Anchor select overlays in user-state-select-wrapper independently of alert message heights
@github-actions github-actions Bot removed the validated Indicates that the PR is ready for actions label Nov 10, 2026
@brain-frog brain-frog added the validated Indicates that the PR is ready for actions label Nov 10, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e066d4db9b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@mkesavan13 mkesavan13 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll review the rest of the changes during the day. Wanted to send out a quick review

@akulakum akulakum left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local Agent Desktop parity review feedback.

<Button
slot="footer-button-primary"
type="button"
disabled={!onVoiceReconnect}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the host does not pass onVoiceReconnect, this button is disabled. Existing UserState embeds therefore show a Reconnect action the user cannot use; the sample’s regular UserState instance also omits this callback. Agent Desktop wires the action to registerWebCalling. Please provide a host integration example that supplies a real retry callback, or clearly document that reconnect requires host wiring before describing this as full parity.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in a9f8d21 and c018359 — In the sample app, the main embed intentionally omits onVoiceReconnect to accurately reflect a host that delegates reconnect guidance to host calling management. The interactive simulated preview directly beneath it provides full reconnect controls (setVoiceReadinessScenario('ready')). Additionally, the optional host retry lifecycle contract is explicitly documented in widgets/packages/contact-center/user-state/ai-docs/user-state-spec.md.

@akulakum

Copy link
Copy Markdown
Contributor

@cmullenx, could you please address the outstanding Codex review comments? The findings are valid, especially the Wellness Break timeout and lookup edge cases, and the case where the dialog says the agent changed to Idle when no default idle code exists. Thanks.

…(CAI-8629)

- Only open recovery dialog when default idle code exists and transition settles (or was already in default idle)
- Suppress opening recovery dialog with false Idle claim when profile has no default idle code
- Wire onVoiceReconnect on sample UserState embed and document host retry contract in user-state spec
@github-actions github-actions Bot removed the validated Indicates that the PR is ready for actions label Nov 12, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6899699ea8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…ate default codes (CAI-8629)

- Bind recovery dialog specifically to the settled forced Idle request to prevent concurrent in-flight updates from falsely opening the modal
- Show recovery dialog with non-Idle copy when no default idle code exists so Reconnect action remains accessible
- Transition to default idle code if default code arrives after voice-loss timeout has elapsed
- Omit onVoiceReconnect from regular sample embed while preserving interactive preview

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a9f8d21827

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d75561ffcb

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

…low (CAI-8629)

- Read live wellnessIdleCodeLookupPendingRef before forcing Idle to prevent stale timeout closures from skipping pending state registration
- Persist forced Idle transitions out of WellbeingBreak rather than suppressing them as SDK wellness restorations
- Stabilize voice loss timeout against displayedIdleCodes and idleCodes array reference changes
- Prevent rejected forced-Idle requests from retrying on every render
- Keep recovery dialog accessible when forced Idle update fails

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c018359746

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

- Resolve store @webex/contact-center 3.13.0-next.9 and patched node-forge merge conflicts
- Persist forced Idle when matching last confirmed state and preserve deferred forced-Idle intent across recovery
- Make voice readiness preview in React sample opt-in to avoid duplicate state-select Playwright selectors
@brain-frog brain-frog added the validated Indicates that the PR is ready for actions label Nov 16, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5241a823e0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@akulakum akulakum left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the earlier review feedback. Please also address the latest Codex comment about the deferred Idle transition being sent after voice recovers. Could you check the dialog copy as well? When a default idle code arrives late, it can say the agent changed to Idle before the backend confirms the change. Thanks!

…d voice recovery (CAI-8629)

- Track the canceled forced-Idle target in canceledForcedIdleCodeRef when voice
  recovers while a forced Idle transition is still deferred behind a pending
  wellness-code lookup
- Skip the currentState persistence effect for that canceled target so a stale
  render observing the forced-Idle currentState does not call cc.setAgentState
  after the transition was already rolled back
- Add regression test covering voice recovery and wellness-lookup completion
  batched in the same update
@github-actions github-actions Bot removed the validated Indicates that the PR is ready for actions label Nov 18, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7882f3843f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

… spec (CAI-8629)

- Clear canceledForcedIdleCodeRef as soon as the currentState effect observes
  it, regardless of whether currentState matches the canceled target. This
  prevents a stale cancellation token (set when voice recovers while the
  forced render never completed) from later swallowing a legitimate,
  unrelated selection of the same idle code
- Add regression test covering voice recovery while currentState stays on the
  prior confirmed state, followed by a later explicit selection of the same
  idle code
- Correct user-state-spec.md rejected setAgentState description to match
  the implemented behavior: the recovery dialog stays open with neutral
  (non-Idle) copy so onVoiceReconnect remains reachable, rather than
  suppressing/closing it

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fe4142712b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +359 to +360
const isForcedVoiceLoss = forcedVoiceLossIdleCodeRef.current === currentState;
if (prevStateRef.current !== currentState || isForcedVoiceLoss) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve deferred Idle across older SDK state-change events

If an Available request started before wellness lookup is still in flight when the two-second loss timeout defers forced Idle, its subsequent success overwrites the forced selection: StoreWrapper.handleStateChange writes Available back to currentState (packages/contact-center/store/src/storeEventsWrapper.ts:1982), and the older request's continuation advances prevStateRef to Available. This guard then no longer recognizes the forced target, the pending selection is discarded, and completing lookup sends no Idle request. Since hasForcedIdleAttemptedRef also prevents another attempt, the agent remains Available with browser voice unavailable and no recovery dialog. Preserve the forced target independently of SDK-reflected currentState, or serialize it behind the outstanding request.

Useful? React with 👍 / 👎.

…rite (CAI-8629)

- An older in-flight SDK confirmation (StoreWrapper.handleStateChange) can land
  after a forced voice-loss Idle transition was deferred behind a pending
  wellness-code lookup, overwriting currentState with a value other than the
  still-outstanding forced target before lookup completes. The existing guards
  only recognized the deferred selection by comparing against currentState, so
  once lookup finished with currentState reverted, the transition was silently
  dropped and the agent stayed on the stale state with voice unavailable and no
  recovery dialog
- Add a dedicated effect that re-asserts forcedVoiceLossIdleCodeRef once
  wellness lookup completes if currentState still disagrees with it and voice
  remains unavailable, independent of whatever externally overwrote
  currentState in the meantime
- Add a regression test simulating an older SDK confirmation overwriting
  currentState back to Available while lookup is pending, confirming the
  deferred Idle transition is still persisted once lookup completes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a2c7d47159

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +397 to +398
if (forcedVoiceLossIdleCodeRef.current === currentState) {
forcedVoiceLossIdleCodeRef.current = null;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Prevent wellness completion from undoing forced Idle while offline

If browser voice is lost during a wellness break's playing or ending phase, the forced Idle request can succeed here while the break's completion timer remains active. packages/contact-center/ai-assistant/src/wellness/useWellnessBreak.ts:454-463 still invokes performRestore(true), whose requestAvailableState path at lines 292-303 sends Available even after another idle code has taken ownership. That SDK event writes Available into the store, but the forced target is now null and hasForcedIdleAttemptedRef prevents another attempt, so the agent can become routable with unavailable voice and a disabled selector. Coordinate the wellness restoration with voice readiness, or retain an offline guard until readiness returns.

Useful? React with 👍 / 👎.

const WebUserState = r2wc(UserState, {
props: {
onStateChange: 'function',
onVoiceReconnect: 'function',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Update the wrapper module spec for the new public callback

This adds a public r2wc function prop, but packages/contact-center/cc-widgets/ai-docs/cc-widgets-spec.md:84 still lists only onStateChange, and the commit does not update that module's spec. Although the root contracts catalog was updated, review check C1 explicitly requires updating each touched module's ai-docs spec under AGENTS.md's Spec-currency rule. Document onVoiceReconnect and its host-owned retry contract in the wrapper spec in this change so its Web Component surface table also explains how hosts enable Reconnect.

AGENTS.md reference: AGENTS.md:L68-L68

Useful? React with 👍 / 👎.

@mkesavan13 mkesavan13 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a nitpick on moving all text effectively into Constants. Rest looks great.

Approving. Kindly address the nitpick before merging. Thanks for the PR, Christina and Matthew

Comment on lines +58 to +60
const dialogDescription = isAgentInIdle
? "You're no longer connected to Desktop, and your availability state has changed to Idle. Check your internet connection, then try reconnecting."
: "You're no longer connected to Desktop. Check your internet connection, then try reconnecting.";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have this in constants?

Comment on lines +44 to +50
const voiceWarning = isVoiceUnavailable
? isConnecting
? 'Browser calling is connecting. Agent state changes are disabled until voice is ready.'
: isReconnecting
? 'Browser calling is reconnecting. Agent state changes are disabled until voice is ready.'
: 'Browser calling is unavailable. Agent state changes are disabled.'
: undefined;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have these in constants as well?

<Text tagName="p">{voiceReconnectError}</Text>
</div>
)}
{!onVoiceReconnect && <Text tagName="p">Reconnect is managed by the host application.</Text>}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have this and any label texts as well in constants

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants