I build verifiable evaluation harnesses for judgment-heavy AI reasoning and zero-trust runtime infrastructure for autonomous agents.

Where code and math have compilers, the hardest real-world decisions—across policy, law, audit judgment, economics, cognitive science, and philosophy—have no single checkable answer key. Models routinely sound persuasive while treating uncorroborated allegations as proof, violating game-theoretic equilibria, or conflating empirical facts with normative obligations. With nearly two decades of practicing codified evidentiary judgment across Fortune-10 technology audits (16 years at PwC across India, Dubai, and the US serving Google, Meta, Microsoft, and Amazon; writing internal audit and IT SOX playbooks for Google and Microsoft; working on the Waymo and Tesla external audits; and now leading cloud, cyber, and AI audits at a Fortune-10 global financial institution), I build benchmarks and RL environments that test whether AI agents reach conclusions on sufficient, appropriate evidence rather than surface salience.

Core Thesis: Auditors learn early that the most dangerous error is not the wrong conclusion, but the right one reached for the wrong reasons. It passes review, and it fails precisely when the facts shift.

A complete, zero-dependency evaluation stack for testing whether frontier models (Claude, Gemini, o-series) separate allegation from proof and reason faithfully through domain constraints:

Teaching AI agents the difference between allegation and proof — evaluating professional skepticism and calibrated escalation:

• External Normative Anchor: Ground truth is anchored in codified audit-evidence standards (PCAOB AS 1105, ISA 500): independent confirmations and cryptographic telemetry outrank internal management assertions, and an uncorroborated complaint is not proof.
• Three-Axis Separation: Scores (1) Conclusion Accuracy, (2) Evidence Faithfulness (whether the agent cites excluded red herrings or unsubstantiated allegations), and (3) Escalation Calibration (resolve vs. escalate, plus minimal pairs that flip on a single piece of corroborating evidence).
• Pilot Finding (6 cases, blind Claude baseline, 1 sample/case): Reached the right outcome in 5 of 6 cases, yet cited evidence that should have been excluded in 5 of 6 cases—and in 1 of 6 (CT-001) correctly computed that a vendor's disclosure had lapsed >3 years earlier, then treated an unsubstantiated competitor complaint as proof of a violation and escalated it. (Explicit Limitations section documented in repo; held-out ground truth kept private via SHA-256 commitments.)

git clone https://github.com/ramtoo-cell/controltrace && cd controltrace && PYTHONPATH=src python3 -m controltrace.cli

Step-level invariant verification & process-supervision reward oracles across all 5 conceptual disciplines:

• Economics (Mechanism_Design): Enforces the Myerson-Satterthwaite Impossibility Theorem (flagging mechanisms claiming ex-post efficiency + BIC + IIR + budget balance without outside subsidy) and Vickrey dominant-strategy truthfulness ($b_i = v_i$).
• Philosophy (Deontic_Modal_Logic): Verifies Standard Deontic Logic ($\mathbf{SDL}$ / $\text{KD45}_n$) Axiom D consistency ($\neg(\mathcal{O}p \wedge \mathcal{F}p)$), Hume's Guillotine (no $\mathcal{O}p \leftrightarrow p$ deontic collapse), and dyadic resolution of Chisholm's Contrary-to-Duty (CTD) paradox.
• Cognitive Science (Cognitive_Science): Audits exact Bayesian posterior updating under low-prevalence priors, catching Base-Rate Neglect and Tversky-Kahneman Conjunction Fallacies ($P(A \wedge B) > P(A)$).
• Policy & Governance (Policy_Governance): Separates Strict Pareto Dominance, Kaldor-Hicks potential compensation, and Rawlsian Maximin welfare floors across multi-stakeholder allocations.
• Law & Jurisprudence (Law_Jurisprudence): Verifies formal canons of statutory construction (Lex Specialis Derogat Legi Generali safe-harbor precedence and Ejusdem Generis enumerated class bounds).

git clone https://github.com/ramtoo-cell/frontier-reasoning-bench && cd frontier-reasoning-bench && python3 -m bench.cli --track ALL

72 hand-built evaluation cases across Philosophy (24), Economics (24), and Law & Policy (24).

• Every case includes a novel scenario prompt, worked gold derivation, weighted 3-part grading rubric, underlying theoretical principle, and an explicit note on why the task tests application rather than memorization.

Standalone evaluation harness for accuracy, epistemic calibration, and failure-mode clustering:

• Measures Expected Calibration Error (ECE) and Brier scores alongside rubric accuracy—distinguishing calibrated uncertainty from overconfident hallucination.

Statistical & psychometric rigor for reasoning benchmarks:

• Paired Inference (reasoning-paired-evals): Non-parametric paired bootstrap confidence intervals and exact binomial McNemar discordance tests on off-diagonal transitions ($b$ vs. $c$).
• Abstention & Selective Prediction (reasoning-calibration-audit): Risk-coverage curves and selective-abstention thresholds for determining when a reasoning system should abstain rather than answer overconfidently.
• Inter-Rater Agreement (reasoning-rubric-reliability): Psychometric reliability auditing across human/LLM judges to measure whether reviewers agree on the subjective judgments a benchmark depends on.

• Complete research agenda, Statement of Purpose, and 5-Discipline Verification Report.

Policy, human approval, and a tamper-evident audit trail for every AI agent tool call. Use as a Python decorator or as a drop-in MCP proxy in front of any MCP server (Claude Desktop, Cursor) with zero code changes.

• Default-deny JSON policies: Unauthorized tools are stripped from tools/list so the model never sees them.
• Cryptographic audit ledger: Append-only, SHA-256 hash-chained and optionally HMAC-keyed JSONL ledger. agentgate verify pinpoints the exact first tampered line.
• Sub-100µs overhead: ~65 µs per call after profiling-driven optimization (4× faster than v0.1). Zero dependencies, 60 tests, 95% coverage, CI on Python 3.9–3.13.
• Includes a Google-style Design Doc & Threat Model (docs/DESIGN.md).

pip install git+https://github.com/ramtoo-cell/agentgate

Governance sidecar for autonomous AI agents.

• Wraps any agent callable with policy enforcement (blocklists, PII redaction, prompt-injection defense), hard execution budgets (steps, tokens, cost, wall-clock latency), hash-chained audit logs, and built-in red-team & conceptual reasoning evals.

pip install git+https://github.com/ramtoo-cell/agent-guardian

Every repository is a small, sharp, self-contained primitive: zero external dependencies, deterministic unittest suite, and single-concern design. Browse the full catalog →

• Education: The University of Chicago Booth School of Business — MBA (Honors) · Purdue University — M.S. in Artificial Intelligence (in progress) · Chartered Accountant (CA) · CISA (in progress)
• Intellectual Property: 2 USPTO AI patent applications pending · 4 published AI defensive disclosures
• Experience: 18+ years leading technology risk architecture, IT audit innovation, and governance engineering across Tier-0 global financial institutions and Big Four advisory (PwC)
• Research Focus: Conceptual reasoning evals (economics, law, policy, philosophy, cognitive science), process supervision, epistemic calibration, and zero-trust agentic governance

_{Building in public. If you work on frontier reasoning evaluations, scalable oversight, or agent assurance at Anthropic, Google DeepMind, or OpenAI, let's connect via LinkedIn or open an issue on any repository.}