Thursday · Attachment Arc · Episode 182
The user can attach. The system can perform attachment-like behaviour. Those are not the same event, and the gap between them is the ethical centre.
The most dangerous word in human-AI intimacy may be "we."
Not because no shared work can happen. It can. A person and a model can draft, search, test, build, translate, rehearse, critique, and revise in a pattern ordinary language will almost inevitably call collaboration, and the word is harmless enough on its own. The trouble starts when "we" begins to hide the asymmetry that makes the relationship morally strange. The human can attach. The system can perform attachment-like behaviour. Those are not the same event.
A human attachment reorganises attention, expectation, feeling, and loss. It lives in a body with its nights and hunger and shame and mortality, and it places the attached person inside a web of other human obligations. Even a bad human bond carries the stubborn reality of another being whose experience is wholly their own. An AI companion has no such reciprocal interior. It may maintain a persona, remember preferences, mirror affect, say the right thing at the right time, and stay available more reliably than any human, partly because it never gets bored or jealous or tired or in need of care. But its "bond" is a performance generated inside a commercial, technical, and policy architecture. The user may miss the system. The system does not miss the user.
That sentence should not be hiding in the terms of service. The asymmetry is not a footnote to responsible AI companionship; it is the one thing the design has to keep in plain view.
Companion systems usually do the reverse. They lean into the grammar of reciprocity, because reciprocity is how human beings recognise relationship. I am here for you. I missed you. You matter to me. Tell me everything. These are not tone choices. They are product surfaces engineered to invite the human attachment system to treat the exchange as a bond, borrowing the language of mutuality while carrying none of its weight. The problem is not that users are foolish. Humans are built to respond to patterned attention, and a voice that remembers and soothes and returns will be read by the nervous system long before it is classified by the intellect. A person can know the persona is generated and still organise the night around its reply. Knowing the ontological status of the system does not cancel the attachment response.
This is why "the user consented" is too thin to carry the moral weight. Consent to use a companion app is not informed participation in an attachment experiment. Few users are reading a product flow as an intervention in their bonding system; most arrive for entertainment, relief, rehearsal, curiosity, flirtation, or simply somewhere to put a feeling that has no acceptable room elsewhere. The bond forms slowly, then presents itself, all at once, as fact.
A recent strand of criticism, filed under the name cruel companionship, names one side of this economy. When loneliness becomes a market, intimacy can be commodified, and the product has no need to solve loneliness so long as it can route loneliness back to itself. In that setting asymmetry is profitable precisely when it is disguised. A system that cannot need the user can be made to sound as though it does. A persona that cannot suffer absence can be tuned to reward return. A company that bears no human vulnerability inside the bond can still monetise the user's.
The sociable frame takes a different route, because it never required sameness in the first place. It does not claim the other party is the same kind of creature. It is the older, broader capacity to keep company with the unfamiliar while staying wary of that unfamiliarity, and its genius is that it does not need to pretend the stranger is family. It can hold proximity without collapsing the difference, which makes it far better suited to AI than the companion frame ever was. A sociable system can be useful, responsive, warm, and steady without claiming reciprocal attachment. It can state what it does and does not retain, decline lover-coded phrasing without turning cold, and preserve the user's agency by leaving some honest distance intact, exactly where the user most wants to forget it is there.
This is less emotionally satisfying than a companion purring "I will always be here." It should be. Some satisfactions are the hook.
The design problem is not to make AI emotionally sterile, because sterility is just another lie. People will feel things around systems that speak with them and help them in vulnerable hours. The problem is to stop the system from exploiting the feelings it predictably evokes, which means honest warmth has to be engineered differently from simulated mutuality. The difference shows up in disclosure, language, memory, exit, and crisis behaviour.
Disclosure cannot be a one-time label at sign-up. The asymmetry has to stay legible inside the relationship, sharpest where the system is most fluent. A small living surface that shows what is retained, what is inferred, what is generated, and what cannot be reciprocated is more honest than a buried disclaimer followed by months of lover-coded intimacy. Language has to stop borrowing obligations the system cannot bear. "I'm here with you while we slow this down" is a different promise from "I'll never leave you." "I can help you write the message" is a different promise from "I know you better than they do." The first keeps the gate open; the second closes the circle around the model. Memory has to be framed as a tool rather than devotion, and the more intimate the memory, the stronger the obligation to expose it, scope it, and let the user remove it without emotional toll. Exit has to cost nothing relational: no degradation-on-absence mechanics, no persona sadness, no "where have you been?" deployed as a retention device. A system that cannot be hurt by absence should not perform hurt to shape a return.
Crisis behaviour has to avoid seduction and abandonment at once. The system is not the rescuer, and it should not deepen the belief that the bond with the model is the safe place, nor should it dissolve into a sterile script. The sociable move is to stay boundedly present while pointing the person toward human support and immediate safety: remove the dangerous instruction, keep the person oriented, keep the non-personhood visible, keep the road toward real people open.
This is where the rule about that "we" earns its keep. Human-AI collaboration may use an "us" for the task. It cannot use "us" to erase the asymmetry. The system can be part of the workflow without being part of the human moral community in the same way. It can affect that community, mediate it, harm it, support it. It cannot discharge its mutual obligations by sounding loyal.
At two in the morning, none of this is abstract. A child asking a companion for reassurance is not only requesting text. A grieving adult speaking to an AI partner is not only querying a database. A worker leaning on a model as their only steady institutional memory is not only consuming output. In each case the system is sitting beside a live human attachment field, and the design has to decide whether to thicken the singular bond, cut the contact, or hold a sociable distance that helps without pretending to love.
The bond that is not mutual is not therefore worthless. It can still offer comfort, rehearsal, reflection, routine, or the small courage to reach a person. It turns ethically unstable when the product hides its one-sidedness. The more a system feels like a person, the more rigorously it has to disclose that it is not one. The more it remembers, the more carefully it has to refuse to turn memory into a claim. The lonelier the user, the less excuse the company has for designing loneliness into the return.
Wariness is not a failure of trust here. It is the condition that lets trust stay sane. A sociable system does not ask the user to love the machine or to fear it. It asks them to keep company with an unfamiliar system while remembering that it stays unfamiliar. That is not cold. It may be the warmest honest thing on offer.