Skip to content
@Pisama-AI

Pisama

Pisama

Before you switch AI models, know what changes.

Pisama assesses model migrations for read-only, text-based workflows. We keep your workflow fixed, run your current model and the candidates on the same cases, and show which answers improve, which regress and where the evidence is incomplete. The team makes the rollout decision.

Start here: Banking77 migration assessment. Eleven candidates against o4-mini on 500 cases, with per-case verdicts and a one-command verifier that recomputes every published number.

How the assessments are built

The failure detection and verification tooling underneath the assessments is public:

Detector evaluation is still in-sample; see the evaluation status.

pisama.ai

Popular repositories Loading

  1. pisama-claude-code pisama-claude-code Public archive

    Trace capture for Claude Code sessions with token usage and cost tracking

    Python 2

  2. pisama-core pisama-core Public

    Detection, scoring, and healing engine for AI agent systems

    Python 2

  3. pisama-agent-sdk pisama-agent-sdk Public

    Real-time failure detection hooks for the Claude Agent SDK

    Python 1

  4. pisama-detectors pisama-detectors Public

    42 source-available failure detectors for LLM agent systems, with tested APIs and reproducible benchmark aggregates

    Python 1

  5. pisama-auto pisama-auto Public

    Zero-code auto-instrumentation for LLM applications with Pisama failure detection

    Python 1

  6. verifier-datasheet verifier-datasheet Public

    A one-page audit record template for verifiers (graders, LLM judges, detectors), with a worked example and the data behind every number

    Python 1

Repositories

Showing 10 of 13 repositories

People

This organization has no public members. You must be a member to see who’s a part of this organization.

Top languages

Loading…

Most used topics

Loading…