Coherence Cost Model: Coherence costs money

At Tana you should furnish yourself with a dragoman. And you must not try to save money in the matter of dragomen by taking a bad one instead of a good one. For the additional wages of the good one will not cost you so much as you will save by having him.

This chapter formalizes the economics of coherence, defining A21 (Coherence Cost Model) -- the three-component cost function (compile-time, run-time, organizational) and the coherence budget that declares the maximum cost a system will pay to keep predicates glueable across scopes. The chapter develops the overlap cost structure with its scaling regimes, the expected-loss formulation that frames coherence as risk management rather than moral preference, the mechanisms of coherence debt and bankruptcy recovery, and the cost-reduction strategies (scope restriction, staleness tolerance, approximation) that make the tradeoffs explicit. Touchstones T9 (schema evolution) and T10 (higher-arity events) are resolved as instances of the cost model. The reader seeking the motivating narrative for why coherence is an economic problem -- not merely a logical one -- should consult Vol I, Chapter 7 (The Witness Protocol) and Chapter 8 (Similes of Symmetry).

The Hidden Tax

Search is a one-time cost. You pay it when you invent the predicate—enumeration, pruning, scoring, witnessing—and then it is done. Coherence is different. Coherence is a recurring cost—call it the coherence fee—levied every time contexts touch, every time vocabularies evolve, every time stakeholders disagree about what something means. The fee is irreducible: it reflects the thermodynamic cost of making local truth compose into global coherence. Someone always pays it. The question is how much of what gets charged is the fee and how much is rent.

Every predicate added to the vocabulary increases three surfaces:

• The overlap surface: more views means more pairwise checks
• The migration surface: more predicates means more things to update when something changes
• The organizational surface: more stakeholders means more coordination when disagreements arise

The Two Empires handle this tax differently. Schema-land pays the tax upfront through migrations and approval committees, then enjoys cheap run-time queries. Embedding-land avoids the tax by abandoning coherence guarantees: you get results, but nobody promises they mean the same thing across contexts.

The Third Mode makes this tax measurable. You can estimate it, budget for it, and decide what you are willing to pay.

The Cost Model

CoherenceCost(op, U, auth) = C_compile(op) + C_runtime(op, U) + C_org(op, auth)

C_compile = C_search + C_certify + C_witness + C_migrate

C_runtime = C_local + C_overlap + C_verify + C_drift

C_org = C_approval + C_audit + C_dispute

CoherenceBudget(U, auth) = {
  max_compile, max_runtime, max_org, max_expected_loss,
  staleness_tolerance, scope_boundary
}

A coherence budget is an SLA on meaning. It specifies where, how often, and at what cost the system will pay to keep predicates glueable.

Coherence vs Consistency: Coherence (Third Mode) is agreement of meaning on overlaps under declared scopes, witnesses, and absence policies. Consistency (distributed systems) is agreement of values under a storage protocol. Consistency asks: do replicas agree on the data? Coherence asks: do views agree on what the data means? The cost model applies to coherence; consistency is a prerequisite but not sufficient.

Compile-Time Costs

Compile-time costs are one-time. You pay them when a predicate is invented, promoted, or migrated.

C_search: The cost of predicate search from Chapter 18. Proposal, expansion, pruning, scoring. Scales with grammar complexity, derivation depth, and parameter space size.

C_certify: The cost of checking A17 obligations. Local grounding requires evaluating the predicate on witness sets. Overlap agreement requires checking pairwise reconciliation. Invariant preservation requires constraint validation.

C_witness: The cost of producing witnesses. Derivations, exemplars, provenance records, calibration curves. These artifacts are not free; they require computation and storage.

C_migrate: The cost of producing migration witnesses when a breaking change occurs. Chapter 16 established that breaking changes require explicit artifacts. Producing those artifacts has a cost that scales with the number of dependent predicates.

Compile-time costs are often underestimated because they are invisible in steady state. A system with stable vocabulary pays them rarely. A system with rapidly evolving vocabulary pays them constantly.

Consider a fashion platform adding 50 new style predicates per quarter. Each predicate requires search, certification, and witnessing. If C_compile averages 10 CPU-hours per predicate, the platform pays 500 CPU-hours per quarter just for predicate invention. That is the hidden tax on vocabulary growth.

Run-Time Costs

Run-time costs are per-query. You pay them every time you evaluate a predicate or verify coherence.

C_local: The cost of evaluating a predicate in a single view. For measurement predicates, this is computation. For rule-based predicates, this is rule evaluation. For learned predicates, this is model inference.

C_overlap: The cost of verifying agreement on overlaps. This is where the cost model becomes interesting.

C_overlap(p, U) = f(p) · |E| · k̄(p)
                ≈ f(p) · |V| · d̄ / 2 · k̄(p)

Theorem: Coherence Budget Monotonicity

Cover refinement cannot reduce verification cost. This theorem justifies the cost model's central tradeoff: finer-grained coherence is strictly more expensive.

This theorem is why the coherence budget exists: there is no free lunch in verification. Finer-grained views demand quadratically more overlap checks. The scope boundary in the coherence budget is the system's declaration of where it stops paying.

Scope limitation. The monotonicity result provides an ordinal guarantee — refinement cannot reduce cost — but the cost model does not provide an absolute cost function that would allow cardinal measurement across systems. The component estimators (C_compile, C_runtime, C_org) are bounded estimators, not exact measurements; they make cost visible and comparable within a system but do not yet support calibrated cross-system comparison. The trilogy's arguments about verification cost rely on the monotonicity property (more composition = more cost, strictly) and the decomposition into irreducible fee versus extractable rent. They do not require absolute magnitudes. Developing calibrated cost functions grounded in empirical data from real systems is an open problem that would strengthen the model's predictive power.

Staleness tolerance reduces f(p). If you re-check coherence every query, f(p) equals your query rate (potentially millions per day). If you re-check once per day, f(p) = 1. The tradeoff is freshness versus cost. Most systems use a hybrid: real-time checks for critical predicates, daily or weekly checks for stable ones.

C_verify: The cost of checking witness validity at query time. This depends on the verification regime:

• Proof-carrying artifacts: check signature or hash, O(1) per query
• Attestation verification: validate certificate chain, O(log n) typically
• Re-certification: re-run A17 obligations, expensive, done rarely

C_drift: The cost of detecting and handling semantic drift over time. Measured by:

• Disagreement rate: percentage of overlap checks that fail
• Calibration decay: drift in calibration witnesses
• Exemplar churn: percentage of witness set that becomes stale

Drift is the silent coherence killer. A system that was coherent at deployment becomes incoherent as the world changes and the predicates do not.

Consider a "trending" predicate defined as "items with sales velocity in the top 10%." The predicate was coherent when defined: all views agreed on what "top 10%" meant. Six months later, one view has switched to a rolling 30-day window while another still uses a 7-day window. The overlap checks pass (both return items), but the items are different. This is semantic drift: the syntax is stable, but the meaning has changed.

Why do naive overlap checks miss this? If the overlap contract only checks well-typedness or non-emptiness (both views return Item sets), drift slips through. Catching it requires either stronger overlap predicates (agreement on specific items, not just non-empty intersection) or periodic re-certification.

Detecting drift requires either periodic re-certification (expensive) or continuous monitoring of disagreement rates (cheaper but reactive). The cost model makes this tradeoff explicit.

Organizational Costs

Organizational costs are per-decision. You pay them when humans must coordinate.

C_approval: The cost of obtaining authority approval for promotion. User to org requires one approval. Org to global may require multiple stakeholders, legal review, compliance signoff.

C_audit: The cost of maintaining audit trails and provenance. Regulated industries require extensive documentation. Every predicate, every witness, every migration must be traceable.

C_dispute: The cost of resolving disagreements when overlaps fail. Two merchants define "sustainable" differently. The overlap check fails. Someone must decide what to do.

C_approval ≈ num_approvals × avg_cycle_time × loaded_labor_rate
C_audit    ≈ audit_frequency × audit_hours × auditor_rate
C_dispute  ≈ dispute_rate × avg_resolution_hours × stakeholder_count × labor_rate

Organizational costs are often the dominant component. A predicate that is cheap to compute and cheap to verify may still be expensive to approve. A schema change that is trivial technically may be blocked for months by stakeholder coordination.

This is where the Two Empires diverge most sharply. The embedding empire minimizes C_org by avoiding approval entirely: anyone can add features to a model without permission. The schema empire maximizes C_org by requiring extensive review: changes to the schema require committee approval. Neither approach is wrong; they are different positions on the cost curve.

The Third Mode makes C_org visible and lets you choose. Some predicates (user-level, local scope) can have C_org ≈ 0. Other predicates (system-level, global scope) may require extensive organizational investment. The budget declares what you will pay for each category.

The Cost of Incoherence

The cost model so far describes the cost of achieving coherence. But coherence is not an end in itself. The question is: what do you lose if you do not pay?

expected_loss(p, U) = P(conflict) × impact(conflict)

where:
  P(conflict):     Probability of overlap disagreement going undetected
  impact(conflict): Business cost of acting on inconsistent data

If you never check overlaps, P(conflict) is high. If conflicting data leads to wrong decisions, impact(conflict) is high. The expected loss is the product.

The coherence budget becomes an optimization problem:

optimal_budget: minimize [ CoherenceCost + expected_loss ]

You pay coherence cost to reduce expected loss. The optimal budget is where marginal coherence cost equals marginal loss reduction.

This frames coherence as risk management, not moral preference. A system that never checks coherence is not evil; it is accepting risk. A system that checks everything is not virtuous; it may be overpaying.

Coherence Budgets in Practice

A coherence budget is a policy object with parameters:

CoherenceBudget = {
  max_compile:        1000 CPU-hours per predicate
  max_runtime:        100 overlap checks per query
  max_org:            2 approval gates, 1 week cycle time
  max_expected_loss:  $10K/month in conflict-induced errors
  staleness_tolerance: 24 hours for non-critical predicates
  scope_boundary:     primary suppliers only; secondary = best-effort
}

Budget-driven behavior:

• Operations exceeding max_compile are rejected or deferred
• Queries exceeding max_runtime return cached results or approximations
• Promotions exceeding max_org are blocked until authority is obtained
• Predicates exceeding max_expected_loss trigger mandatory review

Scope boundaries determine the coherence horizon. Beyond that boundary, the system does not guarantee coherence. It may provide advisory checks, but it will not block operations.

This is the honest answer to "why can't we just have global coherence?" You can, if you are willing to pay. For most systems, the cost of global coherence exceeds the budget. The scope boundary is where you stop paying.

CoherenceBudget = {
  max_runtime: 1000 overlap checks per cycle
  scope_boundary: { primary: top 10 suppliers, secondary: best-effort }
  staleness_tolerance: { primary: real-time, secondary: 24 hours }
}

Primary suppliers: 10 views, d̄ ≈ 9 (near-complete overlap)
  → |E| = 10 × 9 / 2 = 45 edges
  → f(p) = 1 check/day (real-time threshold)
  → k̄(p) = 20ms per check
  → C_overlap(primary) = 45 × 20ms = 900ms/day (~1 second/day)

Secondary suppliers: 40 views, d̄ ≈ 3 (sparse, some clusters)
  → |E| ≈ 40 × 3 / 2 = 60 edges
  → f(p) = 1 check/day (daily batch)
  → k̄(p) = 20ms per check
  → C_overlap(secondary) = 60 × 20ms = 1.2 seconds/day

Total overlap cost: ~2.2 seconds/day of compute

C_overlap(secondary) = 60 edges × 20ms × 10,000 queries = 12,000 seconds/day = 3.3 hours/day

T9: Schema Evolution as Coherence Cost

T9 (Schema Evolution) is the problem of vocabulary change over time. "Employee" and "contractor" merge into "worker." The schema changes; dependent queries must adapt.

The failure mode is breaking changes without migration semantics. The schema changes; queries silently break or return wrong results.

The cost model view: migration is a coherence cost.

Migration cost components:

• C_migrate: producing compatibility witnesses (A17b artifacts)
• C_runtime: evaluating compatibility shims at query time
• C_org: coordinating stakeholders on the change

Schema evolution is not a bug to be fixed but a cost to be budgeted. The Third Mode makes that cost visible.

The key insight: migration cost is not a one-time payment. The compatibility shim runs at every query during the transition period. If the transition takes 6 months and the predicate is queried 1000 times per day, that is 180,000 shim evaluations. The run-time cost of migration can exceed the compile-time cost of the change itself.

T10: Representational Complexity as Coherence Cost

T10 (Higher-Arity Events) is the problem of n-ary relations. "Alice introduced Bob to Carol at the conference" involves four entities in a single event.

The cost model view: richer representations have higher coherence costs.

An n-ary event induces multiple projections. Each projection is a potential overlap constraint. As a proxy (lower bound), arity induces pairwise projection constraints:

C_overlap(n-ary event) ∝ n × (n-1) / 2 projections to reconcile

A 4-ary event (Alice, Bob, Carol, conference) has 6 pairwise projections. The actual constraint count may be higher if higher-order coherence is required. A binarized representation (separate "introduced" and "attended" relations) has fewer overlap constraints but loses the event's unity.

The tradeoff is explicit. You can represent n-ary events with full fidelity, but you pay in coherence cost. Binarization loses information but reduces cost. The cost model lets you make this tradeoff consciously.

Practical Budgeting

The unit economics of vocabulary growth:

total_coherence_cost = Σ_p [ C_compile(p) + T_p · qps(p) · C_runtime(p) + C_org(p) ]

where T_p is the expected lifetime of predicate p and qps(p) is its query volume.

Rules of thumb:

Highly connected predicates are expensive to change. A predicate with many dependents has high migration cost. Before inventing a predicate that others will depend on, consider whether you can afford to maintain it.

Widely scoped predicates are expensive to maintain. A predicate with many overlaps has high run-time cost. Consider restricting scope or increasing staleness tolerance.

High-authority predicates are expensive to approve. A predicate requiring global authority has high organizational cost. Consider whether user-level or org-level scope is sufficient.

Cost reduction strategies:

• Restrict scope (fewer overlaps to check)
• Reduce connectivity (fewer dependencies to migrate)
• Increase staleness tolerance (less frequent checking)
• Use approximations (probabilistic coherence instead of exact)

Each strategy trades something for cost reduction. The cost model makes the trade explicit.

What you trade:

• Restricting scope trades reach for cost. Your predicate applies to fewer contexts.
• Reducing connectivity trades reusability for cost. Other predicates cannot depend on yours.
• Increasing staleness trades freshness for cost. Your coherence checks may be outdated.
• Using approximations trades exactness for cost. Your coherence is probabilistic, not certain.

None of these trades are inherently bad. They are engineering decisions made visible. The cost model lets you make them consciously rather than discovering them in production.

When the Budget Is Blown

The cost model assumes systems operate within their declared budgets. What happens when they do not?

A system can accumulate coherence debt. Predicates multiply without retirement. Overlaps grow dense. Staleness tolerance creeps upward. Eventually the system reaches a state where the cost of maintaining coherence exceeds the budget, and the gap keeps widening. This is coherence bankruptcy: the system cannot pay the tax it owes.

Bankruptcy has symptoms. Overlap checks time out. Migrations stall because the compatibility surface is too large. Organizational approvals backlog because stakeholders cannot coordinate on the accumulated changes. The system still runs, but coherence is a fiction maintained by hope rather than verification.

Recovery requires shedding load.

Scope shedding: Narrow the coherence boundary. Predicates that were global become org-level. Predicates that were org-level become local. The system explicitly abandons coherence guarantees for the shed scope. Downstream consumers receive notice: this predicate is no longer maintained at your scope level.

Predicate retirement: Remove predicates that no longer earn their cost. A predicate with few queries and many overlaps is a candidate. Retirement is not deletion; it is deprecation with a migration path. Queries against retired predicates receive a warning and eventually an error. The predicate's package remains in the audit log.

Invariant relaxation: Convert hard invariants to soft constraints. The system no longer blocks on violations; it flags and continues. This trades correctness for operational continuity. The tradeoff is explicit: relaxed invariants appear in the coherence report with their relaxation date and authorizing decision.

Staleness amnesty: Accept that some predicates will not be fresh. Declare a staleness floor: predicates with staleness tolerance below the floor are raised to it. Queries receive a warning that results may be stale. The system stops pretending it can maintain freshness it cannot afford.

None of these recoveries is costless. Scope shedding breaks downstream assumptions. Predicate retirement forces migrations. Invariant relaxation admits errors. Staleness amnesty reduces trust. The point is not to make recovery painless but to make it structured. A system that recovers through explicit shedding can explain what it gave up. A system that recovers through silent degradation cannot.

The coherence budget is not just a spending limit. It is also a recovery plan. When the budget declares max_expected_loss, it implicitly authorizes the shedding required to stay within that loss. The budget is a contract with the future: here is what we will maintain, here is what we will sacrifice, here is how we will decide.

Consequence

Coherence is not free. Global coherence is often unaffordable.

The honest response is not to pretend coherence is achievable (the schema empire's implicit promise) or to abandon it entirely (the embedding empire's implicit choice). The honest response is to declare a budget.

The coherence budget determines:

• How far coherence extends (scope boundary)
• How fresh coherence is (staleness tolerance)
• How expensive operations can be (cost limits)
• How much risk the system accepts (expected loss)

One distinction matters for everything that follows. The coherence fee—the irreducible cost of composition—is thermodynamic and permanent. No technology eliminates it. But intermediaries who collect the fee may layer rent on top: charging not only for the work of composition but for the privilege of being the only party trusted to do it. That surplus is not a fee; it is a tax on trust itself. The Third Mode makes both visible. It lets you see what composition genuinely costs, and what the intermediary charges above cost. Separating the irreducible from the extractable is not an optimization problem. It is the question that determines who holds power over the systems that compose your truth.

The economics are established. Part V implements them.