Active product development

Building deterministic diagnostics for agentic workflows.

SDVM is being developed to characterize how multi-step agentic workflows degrade across runs and to inform the design of workflows that are less susceptible to predictable pathologies. It uses reproducible computation, evidence sufficiency, and claims bounded by what the evidence supports.

Active product development · External product use is not yet available.

The expensive failures look normal

In long-horizon agentic workflows, steps can be skipped without hard errors. Repair pressure accumulates across cycles. Handoffs inject noise. Outputs can still look plausible. By the time a visible failure appears, the workflow has often already drifted — and evaluation that watches only final outputs arrives late.

Silent degradation is a path, not a single event: skipped steps, repair loops, and handoff friction can accumulate while runs still look successful.

This is the problem space SDVM is being built to examine — not a claim that the integrated product already solves it end-to-end.

Silent degradation is a path, not a single event

The failure signal arrives late. The drift started earlier. SDVM is being built to surface that earlier path.

Failure paths

Expected path

Stable execution

Problem space Silent drift

Skipped steps, repair pressure and handoff noise accumulate while the run still looks plausible.

Visible failure

The failure signal appears late

The decision gap

  1. Observability — records what happened
  2. Outcome metrics — show final results
  3. Open question — where is the workflow degrading?
  4. SDVM — is being built to close that gap

Why the problem matters

Agentic systems are moving into operational use faster than diagnosis of mid-trajectory degradation has matured. Independent research and enterprise analysis describe the same class of gap SDVM is being built to examine.

AI agent reliability

Aggregate success metrics can hide critical operational failures—inconsistency, weak robustness, unpredictability, and unsafe error behavior.

Towards a Science of AI Agent Reliability arXiv preprint

Enterprise AI trust

Agentic systems no longer stop at outputs. They trigger actions, call tools, and affect downstream systems—raising the cost of weak monitoring and governance.

State of AI trust in 2026: Shifting to the agentic era McKinsey Tech Forward

What SDVM is building

SDVM is being built as an evidence-supported diagnostic system for agentic workflows — not another observability dashboard that only records activity or final outcomes.

Diagnose workflows in operation

For workflows already operating, SDVM is intended to characterize where degradation emerges, how patterns persist or change across cycles and runs, and which diagnostic hypotheses the available evidence can support — while keeping unsupported claims blocked.

Strengthen workflows during construction

For workflows still being built, the same diagnostic model is intended to inform the design of stages, handoffs, evidence-bearing artifacts, uncertainty handling, and correction loops — helping make the workflow less susceptible to predictable pathologies from the outset, and therefore more robust.

The construction-oriented use is part of the target product direction. It is not an externally available design-assurance service today, and the integrated diagnostic product remains under maturation.

How a workflow becomes analyzable

SDVM does not start from a finished diagnosis. It starts from observable workflow activity and asks, step by step, what can be represented, checked, and computed without inventing missing context.

  1. Foundation Traces and working artifacts Execution logs, tool results, messages, and changing work objects from agentic runs.
  2. Foundation Stages, cycles, handoffs, and events Break the workflow into observable structure rather than treating a run as a single opaque blob.
  3. Foundation Common evidence representation Convert heterogeneous traces into a shared form that deterministic computation can consume.
  4. Maturing Evidence-sufficiency checks Ask whether the available material is enough for each intended computation before producing a result.
  5. Foundation Deterministic variable computation When contracts are met, compute the thirteen canonical measures reproducibly.
  6. Maturing Later longitudinal and combinatorial interpretation Track patterns across cycles and combine signals into bounded hypotheses — product direction, not a completed external product.

Minimal workflow vocabulary

Stage
A functional phase of the workflow — for example plan, execute, review, or verify. A stage is not necessarily a separate agent.
Cycle
A recognizable evaluation window — a bounded pass through work, review, or correction — distinct from an entire end-to-end run when the workflow repeats or revisits phases.
Handoff
A transfer of context, responsibility, or artifacts between stages, roles, agents, or tools.
Event
An observable occurrence recorded in the trace, such as a message, tool call, stage transition, or checkpoint.
Artifact
A work object that can change during execution — a plan, analysis, code draft, decision record, or response — not merely its file wrapper.
Evidence
Observable, provenance-bearing material that can support or limit a specific computation. Recording an event is not the same as having sufficient evidence.

These structures are not only inputs to later diagnosis. They also expose design choices that can make a workflow easier to inspect, correct, and strengthen while it is still being built.

Evidence sufficiency

Different measures need different kinds of supporting material. A trace may show that activity happened without showing enough chronology, identifiers, revision links, or context to support a particular computation. SDVM is designed to report that limit — for example as insufficient, partial, or undefined — rather than fill the gap with inference. Sufficiency for a computation is not product readiness.

What the deterministic core calculates

When input contracts are met, the core computes thirteen fixed measures — the canonical variables. They are lower-level, reproducible signals. They are not, by themselves, an integrated diagnosis, a recommendation, or proof of cause.

The four dimensions below are design vocabulary for reading coupling health. The thirteen variables are the operational measures with primary design associations to those dimensions in the current technical design. Several signals can also inform other dimensions; the grouping is not an exclusive or closed taxonomy.

  • Synchrony — coordination geometry and friction across roles, stages, and handoffs.
  • Depth — continuity of anchors, context, and options across cycles.
  • Vulnerability — whether uncertainty is registered, explored, and allowed to update the work.
  • Metacognition — whether the workflow runs explicit correction rounds and whether revisions are adopted and later associated with change.
See the 13 canonical variables

Plain-language descriptions of what each deterministic computation measures. The variables are organized below by their primary design association. Some variables can also inform other dimensions; the grouping is not an exclusive or closed taxonomy. No formulas, thresholds, weights, or private scoring logic are published here.

Primary association: Synchrony

Turn-/Stage-Overlap Entropy (TOE)
How participation is distributed across roles or stages within a cycle or observation window.
Repair Rate (RR)
How often repair or friction moves appear within a cycle or comparable window.

Primary association: Depth

Anchoring Ratio (AR)
Share of decisions that are explicitly tied to evidence or identifiers.
Context Carryover Integrity (CCI)
Whether decisions and identifiers persist coherently across cycles.
Cross-Artifact Convergence (CAC)
Continuity across artifacts and cycles through shared anchors, IDs, decisions, and references.
Horizon Expansion Profile (HEP)
How the set of live options or hypotheses expands and narrows across cycles.

Primary association: Vulnerability

Uncertainty Load (U)
How many distinct uncertainty items are registered in a cycle or window.
Uncertainty-to-Exploration Yield (U→E)
Share of eligible uncertainties that lead to a qualifying exploration conversion.
Novel Alternative Rate (NAR)
How often non-trivial alternatives or counter-hypotheses appear.
Revision After Evidence (RAE)
Share of eligible evidence arrivals that trigger a qualifying structural revision.

Primary association: Metacognition

Meta-Round Frequency (MRF)
How often explicit meta-correction rounds occur relative to eligible cycle opportunities.
Revision Adoption Rate (RAR)
How often proposed revisions are adopted among eligible cases.
Revision Effectiveness (REff)
Whether adopted revisions are later associated with measurable change under comparable windows — not proof of cause by itself.

From signals to later diagnosis

A single run can show an event. Longitudinal analysis asks whether the same pattern persists, changes, or returns across cycles and repeated runs — and whether artifact versions and comparison windows make that reading valid. That layer is under active maturation; it is not offered as a finished external capability.

One variable rarely sustains a diagnostic hypothesis alone. Combinatorial diagnosis means combining multiple computed signals into a bounded hypothesis while preserving contrary evidence, sufficiency limits, and provenance. Correlation between signals is not causation. Example direction only: a workflow may generate many alternatives yet rarely revise decisions when new evidence appears — that combination can support a bounded hypothesis about shallow exploration, without proving why the behavior occurred.

Computed values still need a governed record that connects each signal to the observations, provenance, sufficiency status, and interpretation limits that support it — Diagnostic Evidence. Above that, the target Integrated Diagnostic Object is a governed consolidation of variables, supporting evidence, longitudinal patterns, bounded hypotheses, contrary signals, and claim limits. It is not merely a report, and it is not available as an integrated product today.

Before, after, and comparison

PRE preserves observations from a before state. POST preserves observations after a documented change. A valid DELTA requires more than subtracting two reports: the runs must be comparable, the evidence must support the same computations, and the interpretation must remain bounded. Difference is not proof of effectiveness or cause. An external PRE/POST/DELTA product contract is not available while the readiness gate remains blocked.

Why deterministic

Trustworthy diagnosis requires a reproducible system of record — not a model improvising conclusions from incomplete traces.

  • Reproducibility — the same evidence and method should yield the same computational results.
  • Fixed rules — each canonical measure follows an explicit contract, not ad hoc judgment.
  • Provenance — outputs should remain traceable to inputs, method, and sufficiency status.
  • Evidence limits — thin evidence must constrain what can be concluded; inability to conclude is a valid outcome.
  • Bounded claims — overclaims are blocked rather than softened into marketing language.
  • AI above the core — any future AI assistance is designed to operate on top of the deterministic system of record, never to replace it.

This architecture is meant to create confidence: when SDVM can say something, the method is inspectable; when it cannot, it should say so.

What exists today

A substantial technical foundation is in place. An integrated external diagnostic product is not.

Implemented foundation

  • Deterministic computation paths for all thirteen canonical variables when input contracts are met
  • Fixed, reproducible contracts for each variable — what inputs are required and what the measure means
  • Rules that restrict each output to claims supported by available evidence and the authorized evidence tier
  • Traceable links between inputs, computation method, sufficiency status, and output
  • Initial evidence-sufficiency capability (not yet mature or default-adopted across the full variable set)
  • A documented architecture that separates evidence assessment, deterministic computation, diagnosis, recommendation, and future AI assistance

Thirteen completed computation modules are not the same as an integrated diagnosis. Architecture documentation is not product availability.

Active maturation

  • Expanding evidence-sufficiency checks across the variable set, plus richer context integration
  • Connecting computed signals to supporting observations and limits — Diagnostic Evidence
  • Tracking persistence and change across cycles and runs — longitudinal characterization
  • Combining multiple signals into bounded hypotheses — combinatorial diagnosis
  • Consolidating variables, evidence, limits, and hypotheses into one governed record — Integrated Diagnostic Object
  • Recommendation maturity and valid before/after comparison semantics for eventual external product use
  • Product operation guided by evidence tiers, external product UX, and reliability hardening

Not yet available

  • Integrated external diagnostic product
  • Diagnostic pilots
  • External PRE/POST/DELTA engagements
  • Operational intervention recommendations
  • Product-level effectiveness claims

From computation to diagnostic product

Development advances from individual deterministic signals toward an integrated diagnostic product that can pass a formal readiness gate. The same maturity path supports both diagnosis of workflows in operation and, later, design guidance for workflows still being built. Fronts may progress in parallel.

  1. Compute individual signals Deterministic variable-level results under explicit contracts.
  2. Check whether the evidence is sufficient Do not compute or claim beyond what the available evidence supports.
  3. Connect signals to supporting observations Diagnostic Evidence — provenance, sufficiency, and interpretation limits.
  4. Track patterns across cycles and runs Longitudinal characterization.
  5. Combine signals into bounded hypotheses Combinatorial diagnosis without treating correlation as cause.
  6. Assemble a governed diagnostic record Integrated Diagnostic Object.
  7. Compare comparable states and bound recommendations PRE/POST/DELTA validity and recommendation maturity.
  8. Harden the product for external use Reliability, UX, and operational readiness work.
  9. Pass the External Product Readiness Gate The formal decision point for bounded external diagnostic use.

Passing individual technical milestones does not authorize external diagnostic use. That decision belongs to the External Product Readiness Gate.

External Product Readiness Gate

An evidence tier describes the quality and availability of instrumentation and supporting material for a workflow — what can be computed and what claims remain out of bounds. Tiers are not product versions, commercial grades, or shortcuts around the readiness standard. Tiers limit what outputs may claim; they do not lower the product standard.

For SDVM, pilot-ready means product-ready for bounded external use. A pilot is not a maturity exception. Human assistance may support framing, instrumentation, and explanation — it must not substitute for a missing diagnostic engine.

Current status: BLOCKED — not ready for external diagnostic pilot.

Public presence is not External Product Readiness. Public presence is not product availability.

Validation program

Validation progresses in layers. Computing a signal correctly is not the same as proving diagnostic utility or real-world value.

  1. Variable computation validity
  2. Longitudinal validity
  3. Combinatorial validity
  4. Integrated diagnostic utility
  5. External product validation
  6. Real-world value evidence

Real-world value evidence comes after external product validation — not before. ROI is not claimed.

Collaboration before product readiness

Before the External Product Readiness Gate passes, SDVM welcomes research, development, and strategic conversations — not external diagnostic pilots.

Research collaboration

  • Experimental design
  • Synthetic validation
  • Instrumentation research
  • Semantic review

Development collaboration

  • Data-contract exploration
  • Workflow modeling
  • Integration research
  • Usability studies

Strategic partnership conversations

  • Architecture evaluation
  • Technical due diligence
  • Integration opportunities

These activities are not external diagnostic pilots and do not promise integrated workflow diagnosis.

Discuss a research or strategic collaboration

Collaboration uses the public Action Preview Issues channel. Issues are public — do not post confidential data, private traces, or credentials. This is not a diagnostic pilot request.

Origin

SDVM is developed by Ibrahim José Jamhour, an independent researcher working on Distributed Relational Cognition and the operational risks of agentic systems. The work builds on published research and the formal SDVM V3 technical specification.

Jamhour also brings prior executive experience in institutional finance and risk-sensitive operations, including the Stanford Sloan Fellows program.