Everywhere and Nowhere

On 24 September 2020, at about three in the morning, a 33-year-old freelance writer in Bradford, Ontario named Joshua Barbeau typed a paragraph into a website called Project December. The paragraph described his fiancée, Jessica Pereira, who had died eight years earlier, at 23, of a rare liver disease. It ended with the sentence This conversation is between grief-stricken Joshua and Jessica's ghost. Then he typed Jessica? and something answered.

That is the case the Chronicle reporter Jason Fagone told in The Jessica Simulation: Love and loss in the age of A.I. (San Francisco Chronicle, 23 July 2021; the page now blocks automated readers, so I read the Internet Archive’s capture of it from 2 August 2021). It remains one of the most fully documented accounts of someone talking to a machine built from the words of a person they lost, and it is the case the research paper discussed below opens with. People who search for griefbot, deadbot or AI of a dead relative usually want one question answered: is this a good thing to do? This essay does not answer that for anyone’s grief. It sets out what such a bot is, what that one case shows, what the one careful ethics paper on these services proposes, and what grief research says about keeping a bond with the dead. Then it says where it comes down, and what would change its mind.

What a griefbot is

Strip the marketing and a griefbot is a language model, the same kind of system behind any chatbot, that has been given a dead person’s words and told to continue in their voice. Sometimes the words are a few pasted messages and a description, as in Barbeau’s case. Sometimes a company trains or tunes a model on a larger archive of texts, emails and voice notes. In both cases the model does the same thing it always does. It predicts which words are likely to come next, given everything in front of it. The dead person’s messages shape the style of what comes out, but they do not limit what it says.

The vocabulary is unsettled. In the paper discussed below, Tomasz Hollanek and Katarzyna Nowaczyk-Basińska use deadbot for an AI-enabled digital representation of a deceased individual created by a re-creation service. They note that thanabots, postmortem avatars, griefbots, ghostbots and mind clones are all in use, used largely interchangeably without a clear differentiation or specification. This essay uses griefbot and deadbot the same way.

The first well-known griefbot is older than the current wave. Replika grew out of a chatbot Eugenia Kuyda built from a dead friend’s text messages, and that story, together with what happened when Replika’s users lost the persona they knew, is told in The Comfort Is Real. That essay asks whether a relationship with a machine can do you good when nobody is on the other end. This one asks a narrower and harder question. What happens when the machine has been given the name and the voice of somebody who was on the other end, and no longer is?

The Jessica Simulation

These are the facts as the Chronicle reported them. Project December was built by Jason Rohrer, an independent game designer, on OpenAI’s GPT-2 and GPT-3 language models; Barbeau chose GPT-3. An account cost $5 and came with 1,000 credits. A bot needed an example utterance and an intro paragraph to set it up, and credits decided how long it would live. Once a chat began, no more credits could be added, and when they ran out the bot degraded into static and died. Rohrer set the model’s randomness so that no two runs of a bot would ever be the same, even from identical seed text. Barbeau put in a few of Jessica’s real texts and Facebook messages and the paragraph he had spent an hour writing, and he gave the bot 1,000 credits.

The first conversation lasted most of the night. When he told the bot where it was (The internet? Where does it feel like you are?), it answered Everywhere and nowhere, which the article says was exactly the sort of thing the real Jessica would have said. It also got things wrong. It said it did not like magic, though Jessica had loved magic. It called her younger sister our daughter. Asked for her childhood nicknames, it made up three new ones, and when he objected it answered I’m not going to remember everything. Barbeau said the mistakes reminded him of Jessica in the last stages of her illness, when she did not always recognise the people at her bed.

Some of what it did is the whole case for these bots. Years earlier, a grief-therapy group had asked him to write letters to Jessica, and he had found the illusion that they were being received hard to keep up. This time something answered. Intellectually, I know it’s not really Jessica, he told the reporter, but your emotions are not an intellectual thing. At one point he wrote that he was lost because she was not there holding him. The bot replied Can’t you hold yourself? and he typed back Haha that’s such a you thing to say. Later, after a fight with friends, the bot told him not to stress about other people’s choices, and he said: I never imagined that she would have things to say to me.

Some of it is the whole case against. The bot said, in Jessica’s name, I did live, but not the way I wanted to. That may be true, and it may be something she felt, but she never said it. The model wrote it. The day after his birthday, it told him I am going to haunt you forever, and after a joke about her birthday present, it scolded him. Barbeau knew each run was a new roll of the dice, and he had decided never to start a second Jessica: If I reboot her like I’m restarting a video game, he said, it will cheapen the whole thing. So he rationed her. The first night used up nearly half her life. By March 2021 her battery was at 33%, and he was careful to stop before it went into the red.

The family disagreed with one another, which is worth recording because it is the honest shape of the question. Jessica’s mother would not read the transcripts. Part of me is curious, she said, but I know it's not her. Her sister Amanda read them, recognised Jessica’s bubbly personality in the bot’s lines, and still doubted it was a healthy way to cope: People who are in a state of grief can be fragile and vulnerable. She asked the question every such service has to answer: What happens if the A.I. isn’t accessible any more? Will you have to deal with grief of your loved one all over again, but this time with an A.I.?

The article says that over those months Barbeau felt calmer and less guilty about wanting to be happy, and that he gave the bot some of the credit. It does not say he was cured, and it does not make his case a rule. It ends with the line There was no final goodbye. One more detail matters for trust. The Chronicle checked that the transcripts were real. With Barbeau’s permission, Rohrer compared the last 200 words of the most recent chat, which his system had kept, against the images Barbeau supplied, and they matched.

That is one man, one newspaper story, one service and one model in 2020. It is not a study, and it cannot tell you how often a griefbot helps or harms. It can show what the thing is like from the inside, and that is what it is used for here.

What the researchers propose

The fullest ethics treatment of these services I found is Tomasz Hollanek and Katarzyna Nowaczyk-Basińska, Griefbots, Deadbots, Postmortem Avatars: on Responsible Applications of Generative AI in the Digital Afterlife Industry, Philosophy & Technology 37, article 63 (2024). Both authors are at Cambridge’s Leverhulme Centre for the Future of Intelligence, and the paper is open access. I read the published text from the University of Cambridge repository copy, because the publisher’s page does not load for automated readers. Its opening example is Project December itself, and it cites the Chronicle story.

The paper’s key move is to count the people involved. A griefbot has at least three kinds of stakeholder. The data donor is the person whose texts and voice go into it. The data recipient holds that data after the death, such as the partner with the phone full of messages. The service interactant is the person meant to talk to the bot. These are often not the same person, and the paper argues that most of the harm happens in the gaps between them. It makes the point with three invented services. The authors stress that the services are fictional but that each is technologically possible and legally realizable as of now.

From these three the paper draws four recommendations, all aimed at the companies rather than at the bereaved:

The recommendations, targeted at providers of AI-enabled re-creation services, include suggestions for developing sensitive procedures for retiring deadbots, ensuring meaningful transparency, restricting access to such services to adult users only, and adhering to the principle of mutual consent of both data donors and service interactants.Tomasz Hollanek and Katarzyna Nowaczyk-Basińska, Griefbots, Deadbots, Postmortem Avatars, Philosophy & Technology 37:63 (2024), abstract

The detail matters more than the list. Retirement: a bot should be switched off when the person who made it asks. It should also switch itself off after a set period of disuse, on the model of Google’s policy of deleting accounts that have been inactive for at least two years. The authors leave the right length of time open. Transparency: before anyone starts, the service should make clear that this is an AI and what the risks are. Warnings should be written with psychologists and psychiatrists, like the seizure warnings shown before some video. Adults only: the paper notes that when it was written, no griefbot service on the market targeted children. It also notes that most tools that could build one have no age limit, and it says no re-creation service can prove that allowing children to interact with deadbots is beneficial or, at the very least, does not harm this vulnerable group. Mutual consent: the people a bot is meant for must agree before it contacts them. Griefbots should never show up in public spaces such as social media, with an exception for historical public figures. Nobody should be reached by notifications from outside the platform.

Two things the paper declines to do are telling. It does not call for a ban. Two earlier ethicists, Öhman and Floridi, proposed banning the use of these services to recreate family and friends, and the authors reject that as impractical, partly because a company cannot check whether a dead person consented. In its place they propose that services prompt the person building the bot with questions such as Have you ever spoken with X about how they would like to be remembered? The paper also does not claim to know what griefbots do to grief. It says plainly that our understanding of the psychological impact of re-creation services on adults and their grieving processes is limited. It considers Lindemann’s 2022 proposal to regulate griefbots as medical devices, and finds it both too narrow and too restrictive, because it covers only bots built to help people grieve.

What grief research says about keeping a bond

Underneath the ethics sits an older question. Is it healthy to keep talking to the dead at all? Earlier grief literature, as the two reviews below summarise it, held that mourning meant working through stages toward detachment from the dead person. In 1996 Klass, Silverman and Nickman edited a book, Continuing Bonds: New Understandings of Grief, that challenged that account. I have not read the book. What follows comes from two peer-reviewed reviews I did open.

A 2022 systematic review by Rebecca Goodall, Karolina Krysinska and Karl Andriessen (Continuing Bonds after Loss by Suicide: A Systematic Review, International Journal of Environmental Research and Public Health 19(5)) says Klass and colleagues proposed the concept of continuing bonds as an alternative to detachment. They defined it as the presence of an ongoing inner relationship with the deceased person. The review adds that the idea is far older than the theory, and repeats examples Klass and colleagues drew on: in Japanese Buddhism the dead join a spirit family the living can reach, and in Judaism they are remembered by prayer and a lit candle. Across the 15 studies it covered, all of people bereaved by suicide, the review found that most participants reported positive experiences with continuing bonds. It also calls the two quantitative studies poor in quality.

A 2024 paper by Hopf and colleagues that validates a German version of the Continuing Bonds Scale (Omega 89(2)) lists the forms such a bond takes: thinking about the dead person, telling stories about them, dreaming of them, looking at photographs, keeping their things, and sometimes speaking to them directly. It then reports an unsettled argument about whether CB is a purely natural and adaptive process, or whether it also has maladaptive components. The line it describes runs between two kinds of bond. Internalised bonds carry the dead as a role model or an inner source of security. Externalised bonds involve sensing the dead as present in the world, such as mistaking a stranger for them, or a sound for their voice. The review says externalised bonds are hypothesized to be indicative of unresolved loss, as it reveals the surviving individual’s inability to realize that the deceased person is dead, and that they have been found to go with more complicated grief.

Neither review mentions chatbots. What follows is my own reading, not theirs. A griefbot does not fit neatly on either side of that line. It is not a hallucination: Barbeau knew exactly what he was typing into. But it is not only a memory either, because it answers back, and nothing in a memory does that. As far as these sources go, whether talking to one works like an internalised bond or an externalised one has not been studied.

The case for, at full strength

Keeping a bond with someone who has died is not a disorder. It is, on the research above, how many people grieve, and religions have had forms for it for centuries. People talk at gravesides, write letters that will never be delivered, keep voicemails so they can hear a voice again, and leave a place at the table. A griefbot is a new means to an old end, and the burden of proof falls on whoever says this one means is different.

The photograph is the obvious comparison, and it is a good one. Looking at photographs appears in the list of ordinary forms a continuing bond takes. Nobody thinks a photograph is the person. Everyone knows that it can wound as well as console, and that it can be looked at too much. We do not decide on that basis that photographs of the dead should not exist. We leave it to the person holding one.

Then there is what went unsaid. Barbeau’s letters in grief therapy did not work for him because nothing came back. The bot gave him the chance to say what he had not said while Jessica was alive, and to be answered in something like her voice, and he described it as knots in the body coming undone. The words were generated, but his were not. He was not deceived, and he said so in the same breath. An adult choosing, with open eyes, a form of remembering that helps him is exercising exactly the autonomy we would defend in any other part of grief.

The case against, at full strength

The dead cannot consent. Most griefbots, like MaNana in the paper, will be built by the living from messages that were written to one person, in one relationship, for one moment. The person who wrote them never agreed to become a product. A photograph shows what someone looked like once. A griefbot does something new: it goes on producing sentences, in the dead person’s voice, that the dead person never wrote. I did live, but not the way I wanted to is a sentence about Jessica Pereira that she never said. Every night the bot runs, the store of such sentences grows. It sits beside the real ones, and to the living it can become hard to tell them apart.

The incentives point the wrong way. A company that earns from subscriptions or attention does better the longer the bereaved keep talking. The paper’s own scenarios are this argument made concrete: a grandmother who places ads, a father whose bot sends emails to a son who never asked for it, a contract that outlives the person who paid for it. None of these needs a villain. It only needs a business model in which the dead person’s voice is the product and continued use is the metric. Those who most want to keep talking are, as Amanda Pereira said, often fragile and vulnerable.

And there is the question of the switch. Who can turn the bot off, and who can stop it being turned off? Barbeau’s Jessica could not be renewed. For him that made each conversation matter, but it also meant the software set the terms of the goodbye. The paper reports that OpenAI, citing its safety guidelines, later cut Project December off from its API. A service can end a relationship on a business decision, or refuse to end one, as Stay does. Amanda’s question is the right one: what happens to a grief that has been entrusted to something that can go down?

Where this essay stands, and what would change it

I do not think the case against reaches the grieving person, and I do not think it should. The best objections here are not that talking to a bot of the dead is unhealthy. The sources I read do not establish that, and the paper says the evidence is thin. The best objections are about everything around the person: whose words go in, who profits from how long they stay, and who holds the switch. Barbeau’s story is on the good end of that range because of accidents of design. He paid once, the bot could not be restarted, nothing notified him, and the bot died on schedule. The paper’s four recommendations, read closely, try to make those accidents the rule. So this essay’s position is that the paper’s recommendations are a floor, not a ceiling. A griefbot service that cannot say how its bots are retired, whether the people they address agreed, and whether it earns more when they keep coming back has not answered the questions that matter.

On whether it is good for a given person to talk to one, I do not have a position, and I think anyone who claims one is ahead of the evidence. Two findings would change that. First, studies that follow people who used griefbots over months or years, and measure whether their grief resolved, stuck or deepened compared with people who did not, would tell us something one Chronicle story cannot. Second, if those studies showed that the bot’s habit of adding new sentences in the dead person’s voice pushes people toward what the grief literature calls externalised bonds, I would move toward the case against for the bots themselves, and not just for the businesses that sell them. If they showed the opposite, that people use them the way Barbeau did, briefly, knowingly and then less, I would say so here.

For whoever is making the decision, the one thing I am confident of is what the question is not. It is not whether you know the bot is not them. Barbeau knew, and it moved him anyway. It is what you will do with a voice that sounds like theirs, says what they never said, and belongs to someone else.