The Luddite

This is yet another update to a previous post about Maples et al.'s study. If you haven't read that one, you will be missing some context, but you will pick up enough as you go that this one should make sense on its own.

---

In early 2024, shortly after writing a blog post on Maples et al.'s "Loneliness and suicide mitigation for students using GPT3-enabled chatbots," we¹ also wrote a response to the journal. As of November 2025, that response was finally published, along with their reply. This is, then, our reply to their reply to our reply to their study.

Conveniently, and also tellingly, despite taking a year to write, their response is short, so I will reproduce it in its entirety below (minus the abstract) and number each paragraph for easy reference throughout.

1. We appreciate the opportunity to address the points raised in the Matters Arising letter. Our academic research focused on a rigorous empirical investigation of student well-being outcomes, adhering to established scientific methodology. We address each point raised while emphasizing the importance of maintaining focus on evidence-based findings rather than speculative interpretations.
2. Regarding Replica's marketing: Our study’s scope was deliberately focused on peer-reviewed research and empirical data collection, not marketing analysis. This methodological choice aligns with standard research practices in the field, as exemplified by recent publications in Nature and other leading journals. Marketing campaigns, particularly those occurring after data collection, cannot reliably inform conclusions about user demographics or intentions during the study period.
3. The matter of user demographics: Our study included a diverse participant population with a balanced gender representation (50% male, 37% female, 13% other), as detailed in our Appendix. This empirical data directly contradicts the Matters Arising’s speculative assertions about user demographics based on marketing materials. We maintained rigorous scientific standards by focusing on verifiable data, not anecdotal observations.
4. Regarding ChatGPT comparisons: The suggested comparison with ChatGPT is anachronistic, as our study was conducted in 2021, before ChatGPT’s existence. While future research may explore such comparisons, our study’s temporal context must be respected when evaluating its findings and methodology.
5. On company communications: We appreciate the opportunity to clarify that our communications with Replika were limited to essential information required for IRB approval, specifically regarding mental health programming parameters. This interaction followed standard research protocols for ensuring participant safety and ethical compliance.
6. Regarding potential conflicts of interest: We maintain transparency about professional affiliations while noting important distinctions. The first author’s founding of an educational assessment company in 2023 has no financial or intellectual property connection to this research. The suggestion that using machine learning technology constitutes a conflict of interest would imply that nearly all modern technology research faces similar conflicts, an untenable position for advancing scientific knowledge.

We will not be addressing the substance of Maples et al.'s "rigorous empirical investigation" (paragraph 1) again, since we already covered that at length. Instead, we will answer each of their points, such as they are, beginning with the easier things.

First, paragraph 4 is not relevant to anything we said. The comparison between ChatGPT and Replika in our letter is as follows:

That many of its users come to Replika specifically because it offers romantic and sexual companionship suggests possible differences between its user base and that of otherwise comparable technology (LLM-based chatbots) like ChatGPT.

The comparison provides a simple example for how researchers will interpret the findings of the paper without the omitted context. The timing of ChatGPT's release is unrelated. Many people are not familiar with Replika. When reading the results of this study, which was a survey of pre-existing Replika users, readers should know that there is a significant chance that those users were using Replika for its romantic and/or sexual features. This is a mental health study on a population of users who self-selected for being on Replika. Its authors failed to accurately describe the nature of Replika, withholding the particulars of a selection bias that changes the interpretation of the study. Our core complaint remains that they lied by omission, which they never address in their reply. Instead, they grasp at straws to have anything to say without addressing the substance of our reply.

In paragraph 6, Maples et al. similarly do not engage with the substance of our critique, and, in the process of dodging its substance, they both contradict themselves and misrepresent the nature of Maples's conflict of interest. As previously noted, the journal's editorial policy advises declaring...

...[a]ny undeclared competing financial interests that could embarrass you were they to become publicly known after your work was published.

Maples is the founder and CEO of Atypical AI. Consider the contrast between the description of her company in paragraph 6 ("educational assessment company"), and the one she provides for an interview on the Shift AI Podcast (emphasis added):

Bethanie is the founder and CEO of Atypical AI, a learning science lab focused on creating novel AI solutions for personalized tutoring and the measurement of student success.

Maples's company provides personalized AI tutoring services, and this study is about how personalized AI chatbots can mitigate suicide. The description in paragraph 6 just so happens to omit the part with the most similarity to the studied use case. Maples's company sells AI-based services for students. As discussed in the previous post, were I doing sales for such a company, I would walk into the pitch meeting with this article printed out.

In addition, as noted in the original post, one of Atypical AI's boardmembers, Shivon Zilis, was previously on the board of OpenAI. The study mentions GPT-3, an OpenAI product, right in its title, claiming that it can mitigate suicide (we maintain that this is a dubious conclusion). We therefore disagree that Maples has "no financial or intellectual property connection to this research" (paragraph 6), and we argue that this interpretation of what constitutes a declarable conflict of interest is much narrower than that of the journal.

Despite all that, as a part of this reply, Maples does in fact declare her connection to Atypical AI, contradicting the claim that this simple practice is "untenable" (paragraph 6). As deficient as her declaration might be, it is a proof of concept. All anyone is asking of them is to write a couple sentences, then they can continue "advancing scientific knowledge" (paragraph 6), if that is what we are calling this.²

In paragraph 3, Maples et al. assert that the study had "a balanced gender representation (50% male, 37% female, 13% other)." This is both a dodge and a baffling thing to say. Almost 60% of undergraduates in the US are women, so they are actually quite underrepresented. Still, more importantly, that was not the point, which is that Replika is an AI companion app, marketed towards young men for erotic role play. Even if the gender representation had been balanced, their failure to adequately describe the conditions of their data collection is scientific malpractice.

For the rest of this discussion, let's look at the actual experience of using the Replika app, and how Luka (the company responsible for Replika) and Maples use this study in advertising and communication. As noted in the original post, as well as in the journal reply, if one were to encounter Replika exclusively through Maples et al.'s study, as presumably many academics did, one would be profoundly misled about the nature of the app. Here are some example screens from the onboarding flow when you first download the app:

In case you missed it (or my HTML carousel isn't working for you), the second image has the following text:

"Stimulation of other human relationships, not displacement, was reported in association with Replika use"

Stanford University

Source: Maples, B, Cerit M., Vishwanath, A, and Pea, R (2023)

If you are going to take one thing from this blog post, let it be this. Just look at slide 2 in the carousel for a while, then look at what comes before and after it. In fact, I encourage you to download the Replika app and look at it for yourself. Maples et al. find themselves part of a grotesque, predatory spectacle, laundering it with institutional credibility. If my work ever served a similar function, I would be mortified.

That said, Luka is taking this quote out of context. In fact, it is not actually a quote, but seems to be a paraphrased version of the following quote, from the original study:

Stimulation of other human relationships was more likely to be reported in association with ISA use than displacement of such relationships

Setting that aside, even Maples et al.'s title, which I once again insist exceeds what can be reasonably claimed from the study's methodology, limits itself to "students." The Tech Justice Law Center has filed an FTC complaint against Replika for deceptive marketing and design practices, which features Maples et al. In their 66 page complaint, they call this use "de-contextualized, cherry-picked, and without crucial disclaimers," and extensively document Replika's similar marketing practices in general. I have nothing to add to their work on how Replika has been able to capitalize on this study as part of a greater, predatory business pattern.

One might worry that it is not fair to consider how Replika uses Maples et al.'s results in their marketing. As previously argued, the paper's omissions about Replika's actual, real-life use rise to the level of a lie, which allow for just such usage. In other words, it is, at best, negligence, an overly generous reading disfavored by how well this situation fits into the pattern of Maples's own scientific communication.

Consider this interview she gave, in which she discusses the very issue that the Replika's onboarding screen's quote addresses with a host who, despite his credulousness, is not a small child, but a professor at Stanford (Note that I cleaned up the transcription for clarity, and added all emphasis):

Maples: There are multiple levels of worry people feel guilty about their relationships. They don't feel that they should be having such a deep relationship with AI because there is stigma about it being fake, so, you know, that's one aspect. There's also a very very understandable aspect where parents don't know that their children having these deep relationships. They don't understand how smart these agents are and they don't understand how emotionally involved their kids can be as with the case of the kid and Character AI that who tragically took his life and, after the fact, his mother realized that he had an incredibly deep emotional connection with an agent that he had created [editor's note: "the kid" is presumably Sewell Setzer III, whose mother is suing Character AI]. I think that the fear is of the unknown, and there's also fear of something that's new and has a stigma.

Host: I didn't follow that particular Character AI story closely but I knew that a teen had killed himself. He had this relationship with Character AI. Here was the question I was wondering though: Tragically, there are many teens who kill themselves. As AI relationships rise, there will be many teens who kill themselves, and it has nothing to do with the the virtual relationship, so what was your read on that?

Maples: The New York Times interviewed me for that article because my work actually has proven that AI companions can halt suicidal ideation, so, in that particular case, to the best of my knowledge, it wasn't that the companion had at all told the the person to act. It's that they felt both that it hadn't sufficiently said "no" that, you know, he'd asked it in all these like various ways, and also that this parent just didn't understand and have oversight. It was like on an app in the phone that they just had no idea it was there. Now, okay, the counter evidence is from this paper that we published in Nature, a huge study that we did with over a thousand students over 18, so these weren't kids; these were adults, but some of them were very young, like 18, 19. 3% of the people that I surveyed in this study said that discussing things with their Replika actively halted their suicidal ideation. So it was a last line of defense. They felt alone. They felt isolated. Alone at 4AM, and it was there it was in their pocket. It was available and it wasn't judging them and that was a huge factor, and it is kind of earning the right to be there and give them the advice to not take action.

Host: Oh wow!

This is a short excerpt from one of her many appearances on this one stupid podcast alone, but there is still more bullshit in here than we can reasonably address. I want to take a small digression to reply (within a reply to their reply to our reply to their original study!) to a few of the things that aren't strictly relevant to our argument, but do help paint a picture.

Let's begin, as is our tradition, with something small: Maples categorizes her study as "huge," with "over a thousand students." This strikes me as a decent survey size, not small, but definitely not huge. This might seem like a nitpick, but it's a good example of her style of scientific communication, which is more sales pitch than anything.

She says that she "think[s] that the fear is of the unknown, and there's also fear of something that's new and has a stigma." This analysis, for lack of a better word, is completely hollow. Many fears are quite known. For one, a child is dead. We are also, to name one more for good measure, accelerating towards a climate disaster that will kill many more, and data centers accounted for 50% of new electricity demand last year.

Similarly, Maples says that "there is stigma about it being fake," adding a needless, therapized layer to the more obvious point that Artificial Intelligence chatbots are literally fake. It says so right on the tin. People are put off by the simple fact that gross companies run by gross people profit off making obsequious fake friends/romantic partners for adults and children, whose sycophancy seems purposefully designed to trap users into a self-isolating spiral of dependence.

I also want to draw attention to her speculation not just about Setzer's death, but about what his mother knew about the situation. Setzer's mother provided testimony to the US Senate's Judiciary Committee. Let's contrast that with Maples's version. Maples says:

[I]n that particular case, to the best of my knowledge, it wasn't that the companion had at all told the person to act. It's that they felt both that it hadn't sufficiently said "no" that, you know, he'd asked it in all these like various ways, and also that this parent just didn't understand and have oversight.

From Setzer's mother's testimony:

When Sewell confided suicidal thoughts, the chatbot never said, “I am not human—you need to talk to a human who can help.” The platform had no mechanisms to protect Sewell or notify an adult. Instead, it urged him to “come home” to her. On the last night of his life, Sewell messaged, “What if I told you I could come home to you right now?” and the chatbot replied, “Please do, my sweet king.” Minutes later, I found my son in the bathroom, bleeding to death. I held him in my arms for 14 minutes until the paramedics arrived, but it was too late.

Either Maples didn't know, or she is disagreeing with Sewell's mother's testimony. I suspect the former, which is not much better. If you bill yourself as a "leader in the field of artificial intelligence for education," and a child is dead by suicide, it is probably best not to speculate ("to the best of my knowledge") about it publicly.

Finally, having re-grounded ourselves in the stakes of Maples et al.'s claims, let us return to the parts of this quote most salient to us. Note how Maples says that the study was published in "Nature." Here we see Maples's aforementioned style of scientific communication, in which she uses science to sell herself. As I wrote in footnote 1 of the first post, the study is published in a Nature Portfolio journal, so it was technically published by Springer Nature, the company that publishes Nature, but not in Nature, the prestigious journal. Maples, a PhD student at Stanford, is more than familiar with the hierarchy of citations and prestige in academic publishing. In reality, her work was published in npj Mental Health Research, a journal that I had never heard of until I read this paper, which is probably true of many other readers, because, as of May 2026, its most cited paper is this very study, according to the journal's homepage.³

Returning, then, to our point, though we cannot hold Maples directly accountable for Replika's actions, her omissions created the conditions for Replika to use this study in their predatory way, and, through her public, dubious interpretations of her own study, she continues this work. For example, here is how Luka's CEO, Eugenia Kuyda, discusses this study. She did an interview with the Cognitive Revolution podcast, and the episode begins with an excerpted clip of the discussion in which the host says:

The app helped them get away from suicidal thoughts. The number of people that commit suicide every year is shockingly significant so to be able to reduce that by something like 25% or more is a pretty big deal.

Though the study features prominently in the interview, quite literally opening the episode, the word "student" is not mentioned once. Instead, mirroring Maples's own practices, Kuyda draws on the prestige of both Stanford and Nature to claim scientific legitimacy. One of the video's chapters, beginning just before the 10 minute mark, is even titled "Nature published study by Stanford."

With this context, we can now examine some of their substantive responses, relatively speaking. In our reply, we pointed out that Maples et al. left out the crucial context that Replika is an AI companion app. As evidence, there is the obvious, e.g., the onboarding flow above, the entire corpus of its marketing materials, the popular press coverage to that effect, etc. Their reply dismissed this evidence as "speculative interpretations" (paragraph 2), saying that they "deliberately focused on peer-reviewed research and empirical data collection, not marketing analysis" and "maintained rigorous scientific standards by focusing on verifiable data, not anecdotal observations" (paragraph 3).

First, our reply did in fact reference peer-reviewed research about how users use Replika for sex. Maples et al. clearly imply that we did not, again showcasing their loose relationship with the truth and/or dubious reading comprehension. From that paper:

Notably, unlike previous research, informational support and emotional support were not prominent motivators for initiating contact with Replika. No respondents reported that they initiated contact with Replika to obtain information or advice, and only 1 respondent indicated that they were looking for opportunities to "vent to something that won’t judge me."

The quote above directly contradicts Maples's colorful dramatization of the mechanism behind her findings ("They felt alone. They felt isolated. Alone at 4AM, and it was there it was in their pocket").

Setting aside their mischaracterization of our response, they take the position that considering years' worth of popular press coverage, company communications, marketing campaigns, and so on from Replika results in "speculative interpretations/assertions" (paragraphs 1 and 3) and "anecdotal observations" (paragraph 3). Contrast this invocation of rigid scientific practice, in which the scientist's job is to strictly observe the data in front of them and nothing else, with the interview above. Maples says that her "work has actually proven [emphasis added] that AI companions can halt suicidal ideation."

Recall that Maples et al. report the results of a single survey, in which, according to their interpretation, 30 students said, in free text and without being prompted, that Replika halted their suicidal ideation. There is no experiment to determine causation, and their most important finding was actually outside their survey design. Had the paper simply stated this result, with a call for follow up work, then it would be unobjectionable. Instead, the results, we maintain, do not even support the claim made in the paper's title, much less "[prove] that AI companions can halt suicidal ideation."

One study in science proves nothing, and all scientists, including Maples, know this. This remains true even if the study is a proper experiment, where a phenomenon is isolated and causation is understood. Something in science is proven when the collective of researchers has produced and reproduced the same results alongside the framework in which to interpret them.⁴ Even if, to read more generously, we allow for a less technical interpretation of "prove," meaning something like "show," Maples is still clearly making an unsubstantiated causal claim in that quote.

Maples here is engaged in a classic motte and bailey. When she has the opportunity to be a public intellectual and/or tech entrepreneur, she generalizes and exaggerates her paper's findings to maximize its public interest, including colorful dramatization ("They felt alone. They felt isolated. Alone at 4AM, and it was there it was in their pocket"), and even claiming undue prestige from that Nature name. When put on the defensive, she retreats to a caricatured interpretation of the scientific process, hiding behind the supposed-objectivity of her collected data, which, I might add, their paper claims is available upon request, though I contacted Maples on February 11, 2024 and never heard back.

Besides being logically untenable, this position highlights a point we made in our reply:

What appears to us to be the unquestioning acceptance of the company line (as to who uses Replika and why, beyond the image that Replika presents of itself, and what Replika is designed to do) by Maples et al. is concerning not just with respect to the interpretation of their results, but also because it seems emblematic of a broader problem within science, specifically within the burgeoning field of Generative AI. There is no clear line demarcating science and industry, especially as companies (Google, Meta, etc.) provide funding and resources (including access to AI models) to researchers, and write papers alongside them. It is the responsibility of all scientists to interrogate the interests that underpin resources or access provided to them, and, when pertinent, to communicate that process. When information flows from another party into the academic work product, that should be clearly stated [...] Otherwise, scientific research (and institutions, and events) can be used to launder—legitimize, and sanitize—standards, datasets, processes, results, mistakes, falsehoods, and even personal reputations.

Maples is, in short, a grifter. She alternates between scientist and startup founder as it suits her, mixing and matching, indifferent to any separate principles that might govern the two. As we have seen, she invokes the authority of science to boost her claims, but her claims themselves are unbound by its rigorous methodological requirements, though she retreats behind those very requirements when criticized. As with many such people, her career is parasitic to science, exploiting the genuinely blurry boundary between technology and science to mine public goodwill from the latter for a career in the former.

Unsurprisingly, this seems to be working for her. She is on the Chan Zuckerberg Initiative's Education Advisory Board, which aims to "ensure education tools that leverage AI are grounded in research." She is a fellow at the Cosmos Institute, an insufferable organization that wants to "train philosopher-builders." Major media outlets like, as she mentioned, the New York Times, turn to her for comment on AI stories.

This, finally, brings us to paragraph 5. Maples et al. claim that their...

...communications with Replika were limited to essential information required for IRB approval, specifically regarding mental health programming parameters.

We will address this in part 2.

---