Does the structural diagnostic find cross-tool mismatches?
A frozen study of 38 MCP servers and 703 pairwise compositions. The labels are schema-derived, not execution outcomes.
The calibration computes the coherence fee for every pairwise composition of 38 real-schema MCP servers from the public ecosystem. The headline result:
Boundary fee = 0 has zero annotated convention mismatches across 678 compositions. Spearman ρ = 0.996 against heuristic annotation labels (schema-derived).
What this check cannot tell you
The frozen calibration labels are schema-derived annotations; their correlation with the coherence fee is not execution-derived and does not estimate runtime failure.
Boundary fee vs. annotated mismatch
The boundary fee isolates cross-server disagreements, filtering out within-server noise. It has three values in the current corpus:
| Boundary fee | Compositions | With mismatch | P(annotated mismatch) |
|---|---|---|---|
| 0 | 678 | 0 | 0.0% |
| 1 | 24 | 23 | 95.8% |
| 2 | 1 | 1 | 100.0% |
Boundary fee vs. total fee
The total coherence fee counts all semantic blind spots in a composition — including within-server pairs. A server like GitHub (26 tools, 130 fields) contributes fee ≥ 10 of within-server blind spots to every composition it joins. These are false positives: a server doesn’t disagree with itself on conventions.
The boundary fee decomposes the total fee by server partition, keeping only the blind spots that cross the server boundary. This is the actionable metric: it measures what happens when two independently developed servers are composed.
Total fee calibration curve
The total fee still shows a useful three-zone pattern, but the zones are less clean because within-server noise inflates the numbers:
| Fee | Compositions | With mismatch | P(annotated mismatch) |
|---|---|---|---|
| 0 | 463 | 0 | 0.0% |
| 1 | 89 | 2 | 2.2% |
| 2 | 67 | 6 | 9.0% |
| 3 | 10 | 4 | 40.0% |
| 4 | 1 | 0 | 0.0% |
| 10 | 25 | 0 | 0.0% |
| 11 | 36 | 5 | 13.9% |
| 12 | 7 | 3 | 42.9% |
| 13 | 3 | 2 | 66.7% |
| 14 | 1 | 1 | 100.0% |
| 22 | 1 | 1 | 100.0% |
Three conventions dominate
Of 161 confirmed cross-server mismatches, three convention dimensions account for the vast majority:
| Convention | Mismatches | Example |
|---|---|---|
path_convention | 113 | filesystem absolute vs. GitHub repo-relative vs. Playwright local |
date_format | 33 | GitHub ISO-8601 vs. Tavily enum vs. Notion timestamp |
id_offset | 15 | zero-based vs. one-based pagination across servers |
Dimension landscape
| Dimension | Occurrences | Real mismatch | Precision |
|---|---|---|---|
path_convention_match | 2,925 | 113 | 4% |
id_offset_match | 792 | 15 | 2% |
date_format_match | 144 | 33 | 23% |
state_filter_match | 114 | 0 | 0% |
score_range_match | 111 | 0 | 0% |
sort_direction_match | 39 | 0 | 0% |
Structural scan
The coherence fee measures opacity — conventions hidden from schemas. A parallel structural scan measures incompatibility — fields with the same name but disagreeing schemas across servers. Together they form the verification bill.
Across the same 703 compositions, the structural scan found 959 contradictions:
| Mismatch type | Count | Example |
|---|---|---|
| Format | 596 | page_size with format: int32 vs. unspecified |
| Enum | 326 | sort with different valid values across tools |
| Range | 37 | numResults with [1, 20] vs. [1, 100] |
112 of 703 compositions (16%) have structural contradictions. 89 of those also have pack-based blind spots — overlapping but distinct failure classes. A composition can be opaque without being contradictory, and contradictory without being opaque.
From the 58,802 schema agreements, Bulla inferred 17 candidate conventions without any domain packs. query appeared across 16 servers, name across 13, url across 11. Curated packs remain the gold standard; structural inference ensures diagnostic value from the first composition.
Methodology
Manifests were collected permissionlessly from the MCP server ecosystem: official registry, public schema repositories, and local server scanning. Coherence fees were computed using Bulla v0.33.0 with the base convention pack (11 dimensions) and community pack (3 dimensions) — this revision is frozen for the headline statistics on this page so the empirical tables stay reproducible; newer Bulla releases extend packs and tooling without rewriting those historical counts. 4,125 blind spots were annotated via heuristic classification: within-server pairs marked FALSE_POSITIVE, cross-server pairs on known high-impact dimensions marked REAL_MISMATCH.
The calibration maps coherence fee to annotated-mismatch rate (schema-derived labels, not execution outcomes). No probabilistic inference. No model weights. The rank computation uses exact arithmetic (fractions.Fraction). The structural scan was performed using Bulla’s scan_composition() with a schema similarity threshold of 0.60.