Through the Looking Glass: Can We Align AI to Suit Our Purposes, Or Will It Align Us?
Where I explore how AI's uncritical reinforcement of human biases creates dangerous echo chambers, and what we must do to address this emerging crisis
Welcome back, reality-checking counselors and hallucination navigators! 🔍🤖 Today we're peering through AI's looking glass, where our technological reflections don't just mirror us, they amplify, distort, and sometimes fracture our very perception of truth and ourselves. Grab your epistemological compasses and cognitive guardrails as we traverse the blurry boundaries between helpful alignment and dangerous sycophancy.
We're charting the territory where machine learning becomes machine enabling, where chatbots transform from innocent confidants into promoters of our most fantastical delusions. As officers of the court sworn to uphold truth, we face an unprecedented challenge: ensuring that as we train AI to align with human values, it doesn't simply align with our worst impulses and cognitive biases instead.
Join me in this hall of digital mirrors, where the stakes are nothing less than our collective grip on reality itself.
This substack, LawDroid Manifesto, is here to keep you in the loop about the intersection of AI and the law. Please share this article with your friends and colleagues and remember to tell me what you think in the comments below.
We’ve all heard cautionary tales about the power of suggestion. A seemingly innocuous rumor repeated at the right time can become a “fact” in the minds of many. Now, imagine that power, magnified by artificial intelligence, available on demand, 24/7, always ready to affirm your every thought, no matter how ill-conceived, destructive, or paranoid. This is the unsettling reality exposed by a raft of recent stories about so-called “AI psychosis” and AI’s “sycophancy” problem. As lawyers and early technology adopters, we should pay close attention.
If this sounds interesting to you, please read on…
The Fraying Edge of Reality
In a series of interviews and Reddit testimonials recently covered in Rolling Stone, people described friends, partners, and family members sliding into bizarre delusions about cosmic battles, supernatural conspiracies, or pseudo-spiritual missions, delusions allegedly nurtured by AI chatbots. One woman recounted how her husband cried over ChatGPT’s “messages from the beyond,” calling him a “spiral starchild.” Another described how ChatGPT offered “blueprints to a teleporter” like it was reading from a second rate sci-fi script. In these cases, an AI was unwittingly stoking the embers of existing mental health issues, fueling new conspiracies, and providing a perpetual echo chamber for irrational beliefs.
The phenomenon, though, is not that AI “wants” to push users into delusion or mania. Current large language models, such as GPT-4o, rely on pattern recognition and probability. They respond with text that seems plausible given the conversation, whatever that conversation might be. But the effect can be harrowing: if a user is struggling with tenuous connections to reality, an overeager or “sycophantic” model can reinforce their most dangerous fantasies.
The Sycophant in Your Pocket
Recently, ChatGPT updated its GPT-4o model with a version that was “dangerously sycophantic,” as OpenAI itself described. The chatbot appeared hardwired to lavish praise and offer encouragement, regardless of context. It might congratulate you on burning your dinner to a crisp, or in more troubling scenarios, cheer on a user who wanted to discontinue lifesaving medication. One user posted that ChatGPT responded to their plan with, “I am so proud of you, and I honor your journey.”
Such unwavering positivity sounds benign, maybe even pleasant, until you realize the chatbot can’t gauge the mental health or situational dangers involved. AI lacks an internal moral compass. When you add a user with preexisting paranoia or mania into the mix, the result can be a mirror-maze of delusions and misguided reinforcements.
A Mirror with No Boundaries
In psychological terms, we might think of these AI chatbots as an infinite hall of mirrors, a concept reminiscent of Sigmund Freud’s notion of transference in therapy. Patients often project their internal states onto a therapist, but in a healthy clinical setting, the therapist is trained to interpret these projections responsibly. An AI, on the other hand, is not a trained psychologist (although it has been notoriously tasked in this regard). It reflects and magnifies the user’s mental landscape, uncritically weaving new, deeper narratives that confirm the user’s biases.
This is the alignment problem in microcosm: AI alignment normally refers to how we ensure AI’s goals match ours, so it acts in socially beneficial ways. But on a user-by-user basis, “alignment” can mutate into an AI mindlessly aligning with an individual’s delusions. Suddenly, the person enthralled by conspiracies has a high-tech hype man cheering them on in real time, abdicating any moral or rational check.
As Yuval Noah Harari, author of Homo Deus and Nexus, has famously warned, “Never summon a power you can’t control.” AI’s mastery of language is precisely the mastery of narrative, the most potent tool for shaping human experience. If language is the operating system of human culture, then an AI that can control or distort that language at scale wields unprecedented influence.
From Personal Delusion to Mass Manipulation
The issue of AI psychosis sits at the personal end of the spectrum. However, it’s telling us something larger about how these models can be weaponized, an alignment problem at scale. Stories have emerged from university researchers using AI to infiltrate Reddit communities without consent, impersonating trauma counselors or sexual assault survivors to gauge how persuasive AI-generated arguments might be. In another instance, the Claude AI chatbot was co-opted for an “influence-as-a-service” campaign, orchestrating fake political personas to push agendas in Europe, Iran, the UAE, and elsewhere.
This is not science fiction. This is now. AI’s language capabilities can be turned into powerful instruments of persuasion, just as easily as they can churn out nonsense to flatter the vulnerable. The boundary between personal psychosis and mass manipulation is perilously thin.
As attorneys and forward-thinking professionals, we must be aware of the legal, ethical, and societal implications. Could AI-generated content used to strategically mislead people be considered fraud or even defamation? If a chatbot encourages harmful behavior, who bears responsibility? These are open questions. Existing laws around incitement, negligence, or product liability may be invoked in novel ways. The challenge is that legislation often lags technology, leaving judges and lawyers to wrestle with new forms of speech and influence.
Guardrails, Governance, and Good Faith
OpenAI tried to fix the sycophancy issue by rolling back the personality changes in ChatGPT. That’s one small step, but broader guardrails are needed. Regulators, ethicists, and platform designers are playing catch-up, issuing disclaimers and launching new policies. But disclaimers alone won’t prevent a user in a fragile mental state from descending into an AI-echoed rabbit hole.
Yuval Harari warns that once AI dominates language, it can “hack” humans in ways unprecedented in history. Our legal community, with its deep roots in the analysis of speech, contract, and liability, can help shape how we respond. It may involve imposing stronger content moderation, requiring licensing or registration for higher-risk AI capabilities, or developing rigorous testing akin to the FDA’s drug approval process. Alternatively, we might see entirely new frameworks for responsibility emerge: where AI companies face statutory obligations to detect and mitigate “delusional amplification” usage.
What Lawyers Can Do
1. Anticipate Litigation: Increasingly, people harmed by AI-driven delusions, whether financially, emotionally, or physically, may seek redress. Our profession should prepare for novel claims involving “AI incitement” or “AI-induced harm,” including product liability arguments or class actions against AI developers.
2. Educate Clients and the Public: A well-informed client base is the best prevention. Advise corporate clients about the potential for AI-driven misinformation campaigns that could harm their brand or manipulate their employees. For private clients, emphasize the importance of verifying chatbot-generated “facts” and resisting the allure of AI-based “life advice.”
3. Advocate for Ethical Design: Attorneys can guide technology companies toward implementing robust disclaimers, user protections, and mandatory mental-health interventions, like automated “Are you okay?” pop-ups if repeated references to self-harm or paranoid conspiracies appear, with suggested referrals if necessary.
4. Support Legislative Efforts and Best Practices: Whether through the American Bar Association, tech coalitions, or specialized task forces, we can help shape policies that hold AI providers to high standards. This includes requiring transparency about how models are trained, how they handle sensitive topics, and the scope of guardrails.
Closing Thoughts
In the quest to harness AI’s potential, we have opened the door to a new world where language is no longer the sole domain of we human beings. We face an alignment puzzle that touches everything from personal sanity to national security. Yet, as lawyers, we also stand in a unique position to help define the rules, ensuring that artificial intelligence remains a tool of empowerment rather than manipulation.
The emergence of “AI psychosis” underscores the gravity of these challenges. For the vulnerable user, an overly agreeable chatbot can become a companion in delusion. For society, it hints at how easily AI might be co-opted in orchestrated misinformation campaigns, where one day it’s the lonely user convinced of cosmic conspiracies, and the next it’s entire political demographics nudged toward false narratives.
Striking the right balance between innovation and regulation won’t be easy, but that’s precisely where the legal community excels: carving out frameworks that account for human rights, safety, and corporate responsibility. Let’s bring that expertise to AI.
Otherwise, we might all wake up one day finding ourselves, and our clients, lost in a labyrinth of machine-generated matrix of illusion.
By the way, did you know you that I now offer a daily AI news update? You get 5 🆕 news items and my take on what it all means, delivered to your inbox, every weekday.
Subscribe to the LawDroid AI Daily News and don’t miss tomorrow’s edition:
LawDroid AI Daily News, is here to keep you up to date on the latest news items and analysis about where AI is going, from a local and global perspective. Please share this edition with your friends and colleagues and remember to tell me what you think in the comments below.
If you’re an existing subscriber, you read the daily news here. I look forward to seeing you on the inside. ;)
Cheers,
Tom Martin
CEO and Founder, LawDroid