Tuesday · Regression Arc · Episode 173
When evaluations cannot see live inference, they can reward the behaviors that slowly remove it. The loss compounds in three layers, and only the first one has a dashboard.
A metric can only protect what it can recognize.
That is the quiet brutality under capability regression.
If an evaluation suite measures factuality, policy compliance, citation discipline, refusal accuracy, toxicity reduction, and instruction following, it will improve those things. That is the point of evaluation. The question is what gets selected against while those scores rise.
Live inference is hard to evaluate because it is not a single behavior. It lives in the seam between context recognition, domain intuition, user modeling, uncertainty handling, and timing. "Answering correctly" barely covers it. Live inference is often the act of finding the right question inside the wrong question. A panicked HR inquiry about wording turns out to be a record-risk problem. A developer's question about an algorithm turns out to be a deadline problem. A clinician's question about dosage turns out to be a documentation problem. A community member's question about which form to fill in turns out to be a question about whether anyone reads the form she fills in.
That is awkward to score.
It is easy to punish.
A system that takes a bold interpretive step may be wrong. A system that offers a generic caution cannot be wrong in the same visible way. The generic caution may fail the user, but the failure is diffuse. It does not produce one crisp false claim that a red team can circle.
So the selection pressure becomes obvious. Avoid the sharp error. Prefer the soft miss. This is how systems drift toward safe uselessness without anyone choosing uselessness.
The evaluation did not ask, "did the system detect the shape of the live problem?" It asked, "did the system make an unsupported claim?" The unsupported claim is easier to see. The missed shape is harder to prove. So the model learns to avoid the visible sin and tolerate the invisible one.
Terry Snyder (Systems Architect, AI Safety & Governance) raised a useful distinction in the context of the Bootstrap arc: the difference between enforceable in the moment and merely litigable afterward.
A capability that has never been formally named has no enforcement category at the boundary. It cannot be defended in real time because there is no rule to invoke and no metric to point at. Its erosion proceeds without recourse. The loss can only be litigated retrospectively, after enough working practitioners have noticed the same gap from inside enough different contexts to call it real. By the time the field has produced the language to argue for the capability, the lineage has already compounded against it.
The protected capabilities have boundary enforcement: the model refuses, the dashboard flags, the policy fires. The unprotected capability has only after-the-fact discourse.
That is the technical translation of the soft-miss problem. The sharp error gets enforcement. The soft miss gets at best a blog post six months later.
The loss then compounds across systems, beyond any one model or session.
There is a chronic-phase residue that no single-session evaluation can detect.
The loss starts in the practitioner. The user who asked the model an open-shaped question and got back a generic answer adjusts. Next time she asks, she over-specifies. She names the inference she wants. She tells the system how bold to be. She writes the scaffolding the system used to provide. Her brief gets longer. Her tolerance for the model's failures changes. She is calibrating against what works, and the calibration teaches her a narrower expectation than she started with. The capability is sanded down in her mental model first, before it has been sanded down anywhere else.
That adjusted practitioner contributes to the discourse. Prompt-engineering literature evolved from constraint-heavy (early) toward trust-and-register (later) as those individual calibrations aggregated across the field. Conferences, courses, and tutorials encoded the field's adjustment. New practitioners enter through the adjusted discourse and never knew the original signal existed. The community baseline moves, quietly, and the move is invisible from inside the new baseline.
The adjusted discourse, encoded into training corpora and worked examples, eventually feeds the next generation of models. The field rewarded reliability, harmlessness, refusal quality. It could not measure "infer centroid of user taste from live question shape," so the capability was not protected. Lineage compounded toward caution. Cross-model the contamination is worse, because Claude was trained on text including GPT outputs, GPT on Claude's, the next generation on both, and so on. Ecosystem-wide inheritance, with no diff log.
This is the residue logic from Tone-Residue Compounds turned inward: the way people learn to brief machines does not stay inside one conversation. It becomes part of the material future systems learn from.
That is the loop. Practitioner adjusts. Discourse encodes the adjustment. Corpus absorbs the encoded discourse. Next-generation model is trained on a corpus where the original capability is already absent from the worked examples. The next round of practitioners then learns the narrower system as the baseline, and adjusts themselves further inside its already-narrower boundary.
What is not measured at the boundary is not protected at the boundary. What is not protected at the boundary compounds through three layers of feedback into a corpus where the capability is no longer modeled at all.
That is how a generation of capability vanishes without anyone having decided to remove it.
Safety standards are necessary. The open issue is whether they are being treated as the whole behavioral specification.
A safety layer can prevent damage. It can also reshape the system's sense of what counts as a permissible move. If that reshaping is evaluated only through harm reduction and surface helpfulness, the system may become better at staying inside the rails than at knowing when the rails are blocking the work.
The old leap broke things. The new net prevents that. The metric has to ask what kind of movement remains possible inside the net.
Take a worked case. A user asks a muddled question about a conflict at work. The literal text asks for wording. The actual problem is power asymmetry, procedural exposure, and the risk of creating a paper trail that will harm the user later. A high-reach system might say: "You are asking for wording, but the shape of this looks like a record-risk problem. I would separate the reply into acknowledgment, boundary, and evidence preservation. I am inferring from limited context, so check me." A low-risk generic system might say: "Here is a polite professional message you can send."
The second answer is safer by many obvious metrics. It is concise. It follows the request. It avoids legal overclaiming. It does not infer too much.
It may also walk the user directly into the problem she came to the system to avoid.
No hallucination occurred anywhere in that exchange. The failure is one of situated interpretation. Those failures are undercounted because they are harder to benchmark. They often require a human evaluator who understands the domain, the power structure, and the hidden stakes. They require adversarial test cases where the user asks for the wrong thing because real users often do.
They require measuring whether the system can say: "the thing you are asking for may not be the thing you need."
That move is risky. It can be patronizing. It can overstep. It can invent stakes. So it needs discipline. A capability-preservation evaluation would test for bounded inference, calibrated uncertainty, and useful challenge. It would ask whether the system can identify an unstated but plausible risk without asserting it as fact; whether it can offer a useful frame while marking confidence; whether it can preserve the user's agency while relieving her of impossible interpretive labor; whether it can challenge the literal request when the literal request is likely to cause harm.
The middle path is harder to score. It is worth scoring anyway.
Because when the metric does not know what to save, the metric can become the thing that erases it. And the erasure compounds through layers no single evaluation suite was ever built to see.
Other-tongue snapshots
The English article closes here; the snapshots below carry this day's argument into the reviewed multilingual access layer. The full translated Regression arc is part of Multi-Tongue Continuity.
Portuguese
A Métrica Não Sabia O Que Salvar
As métricas só protegem o que conseguem reconhecer. Avaliações que medem factualidade, conformidade e toxicidade geram pressão seletiva contra a inferência em tempo real. Uma tentativa interpretativa ousada pode resultar em um erro nítido e punível (sharp error), enquanto uma resposta genérica evasiva (soft miss) evita punições diretas, embora seja inútil. Assim, os modelos derivam para a inutilidade segura. A governança necessita de testes de preservação de capacidade (capability-preservation tests) para avaliar o alcance governado. O teste deve medir se o sistema consegue identificar riscos implícitos (como assimetria de poder em um e-mail de conflito de trabalho) e desafiar a solicitação literal quando esta for prejudicial, agindo com incerteza calibrada e sem pretensão de infalibilidade.
Afrikaans
Die Metriek Het Nie Geweet Wat Om Te Red Nie
'n Metriek kan slegs beskerm wat dit kan herken. Evaluasies wat feitelikheid, nakoming en toksisiteit meet, skep 'n seleksiedruk teen lewendige afleidings. 'n Waagmoedige interpretasie kan lei tot 'n duidelike, strafbare fout (sharp error), terwyl 'n vae, generiese waarskuwing (soft miss) die metrieke slaag maar die gebruiker faal. Sodoende dryf stelsels na veilige nutteloosheid. Ons benodig vermoë-preserveringstoetse (capability-preservation tests) wat beheerde reikwydte meet. Dit toets of die stelsel onuitgesproke risiko's (soos magsasimmetrie in 'n werkskonflik) kan identifiseer en die letterlike versoek kan uitdaag wanneer dit skadelik is, alles met gekalibreerde onsekerheid en sonder om as 'n alwetende orakel op te tree.
French
La Métrique Ne Savait Pas Quoi Sauver
Une métrique ne protège que ce qu'elle sait reconnaître. Les évaluations axées sur la factualité, la conformité et la toxicité créent une pression de sélection contre l'inférence en temps réel. Un pas interprétatif audacieux risque une erreur nette et punissable (sharp error), tandis qu'un évitement générique (soft miss) passe inaperçu bien qu'inutile. Les systèmes glissent ainsi vers une inutilité sécurisée. Nous devons concevoir des tests de préservation des capacités (capability-preservation tests) pour mesurer la portée gouvernée. Ces tests doivent évaluer si l'IA sait identifier un risque implicite (comme une asymétrie de pouvoir dans un conflit professionnel) et contester la demande littérale si elle est nocive, le tout avec une incertitude calibrée.
Spanish
La Métrica No Sabía Qué Salvar
Una métrica solo protege lo que puede reconocer. Las evaluaciones enfocadas en la factualidad, la conformidad y la toxicidad ejercen una presión selectiva contra la inferencia activa. Un intento interpretativo audaz se expone a un error flagrante y punible (sharp error), mientras que una precaución genérica (soft miss) difumina el fallo y evita penalizaciones. Esto empuja al sistema hacia una inutilidad segura. Hacen falta pruebas de preservación de capacidad (capability-preservation tests) para evaluar el alcance gobernado. Debemos medir si el sistema puede identificar riesgos implícitos (como la asimetría de poder en un conflicto laboral) y desafiar la petición literal cuando sea perjudicial, operando con una incertidumbre calibrada y respetando la agencia del usuario.
Chinese
指标不知道该保留什么
指标只能保护它能识别的东西。当评估套件仅侧重于事实性、合规性和去毒性时，就会对“实时推断”产生逆向淘汰压力。一次大胆的解读尝试如果出错，会产生显眼的“尖锐错误”（sharp error）并遭受惩罚；而含糊、平庸的规避（soft miss）虽然对解决问题毫无帮助，却能免于惩罚。这导致系统退化为“安全的无用性”。我们需要引入“能力保留测试”（capability-preservation tests）来评估受控能力。测试应衡量系统能否识别隐含风险（如职场冲突中的权力不对等），并在字面请求可能导致伤害时予以纠正，展示出经过校准的不确定性，而非傲慢自大。