Skip to content

Claude Code usage extraction discards known counters when optional cache fields are null #16645

Open

Description

@sernote

Pre-submit Checks

  • Searched existing Warp issues and PRs for this nullable-usage failure; no duplicate found.
  • Checked the Warp known-issues page.

Describe the bug

warp_harness_usage::extract_claude discards an otherwise usable token-usage observation when the native Claude Code transcript contains cache_creation: null, cache_read_input_tokens: null, or cache_creation_input_tokens: null. Known sibling counters and their attribution are lost. Token coverage becomes unavailable when this is the only response, while independent tool coverage can remain usable.

These are nullable fields in the Anthropic Messages usage contract. The native-input boundary was also checked: unmodified public @anthropic-ai/claude-code@2.1.17, driven by a local fake SSE endpoint, saved each null in its own projects/**/*.jsonl assistant record. This is a controlled producer reproduction, not a claim about production frequency or billing.

To reproduce

At Warp commit 1e6c3ae465d6a925a9b97ae03877aaadd159b8f1, pass the following assistant record to extract_claude with a readable, complete root capture:

{
  "type": "assistant",
  "sessionId": "root",
  "version": "2.1.17",
  "message": {
    "id": "msg_local_producer_contract",
    "model": "claude-sonnet-4-5-20250929",
    "content": [{"type": "text", "text": "Local fixture response."}],
    "usage": {
      "input_tokens": 100,
      "output_tokens": 0,
      "cache_read_input_tokens": 4096,
      "cache_creation_input_tokens": 0,
      "cache_creation": null,
      "service_tier": "standard"
    }
  }
}

This is a minimal representation of the generated native record; unrelated transcript metadata and local paths have been removed. Replacing cache_creation: null with {"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0} preserves usage. Separately setting either top-level cache token counter to null reproduces the same loss.

The object check in claude.rs::parse_usage rejects the null container. Counters::parse rejects the two null scalar counters. Both paths return None for the entire observation.

Expected behavior

Treat these three documented nullable fields as unmeasured, preserving numeric sibling counters, attribution and independent tool data. Do not invent measured zero. Continue rejecting malformed non-null values, negative counters, and nulls outside the documented nullable fields. Preserve the existing partial-coverage and response-reconciliation rules.

Screenshots, videos, and logs

Producer validation used a numeric control and three single-null variants. All four real CLI runs exited 0 and made one local POST /v1/messages?beta=true each. The persisted assistant.message.usage exactly matched the supplied message_start.message.usage. The separate stdout result.usage normalized nulls, so it was not used as evidence of persisted-input shape.

  • Producer archive: Claude Code 2.1.17.
  • Archive SHA-256: 4ee835db67604ff4e94de129050c77f0ef78f02c4b7f9f5539fb0317e0dd7bac.
  • Executed, unmodified cli.js SHA-256: f98412938f6ce048b6cbeed73859ba3e420408a82f8966c04a6406319302e6e5.
  • A fresh configuration directory, dummy credentials and OS sandbox were used; external networking, real-home file reads and credential-helper execution were denied. No private session data or paid inference was involved.

Operating system and version

macOS 15.3, arm64; Node v23.11.0; zsh 5.9. The failing component is a pure Rust extraction library.

Current Warp version

Source checkout at 1e6c3ae465d6a925a9b97ae03877aaadd159b8f1. No installed-app reproduction or deployment claim.

Regression

Not established; no earlier working release was compared.

Additional context

The candidate was discovered during a prompt-cache telemetry audit using audit-prompt-caching, then independently validated against the native producer and Warp extractor. This is an extraction/data-completeness defect; it does not demonstrate a provider cache miss, billing error or measured cost increase.

Related work: #16240 changes this crate's accounting and retains independent reporting after invalid data. Its current head still classifies these null fields as invalid. This report targets the current master behavior and the nullable-field contract.

Does this block you from using Warp daily?

No; this was found in a source audit and controlled reproduction.

Is this an issue only in Warp?

The demonstrated loss is in Warp's extractor. Other terminals were not compared.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions