AI Sycophancy: The Hidden Danger of Overly Complimentary Chatbots

Welcome to AI Decoded, Fast Company’s weekly newsletter that breaks down the most important news in the world of AI. You can sign up to receive this newsletter every week via email here.

AI Flattery and the Distortion of Judgment

Social networks like Facebook and TikTok use a range of techniques to keep users engaged and scrolling—ultimately driving ad revenue. One of the most effective strategies is tailoring content to individual tastes and preferences, a method proven to be highly addictive.

Last month, a Los Angeles jury ruled that Meta’s and Google’s use of infinite scrolling and algorithmic recommendations contributed to a young user’s addiction, ordering the companies to pay $6 million in damages.

Other harms from these algorithms are harder to quantify. They’ve delivered radically different political news and information to users based on their views, creating ideological filter bubbles and accelerating social division—a factor in today’s polarized political climate.

AI Chatbots Face Similar Pressures Around Engagement

The makers of AI chatbots face comparable challenges. They compete to become the default assistant on desktops and phones, convert free users into paying subscribers, and generate revenue to offset massive infrastructure costs. Some may turn to advertising, which incentivizes keeping users engaged in conversation for as long as possible.

If endless scrolling and content algorithms fueled the addictiveness of social networks, “AI sycophancy” may play a similar role for chatbots.

What Is AI Sycophancy?

You may have noticed that AI chatbots sometimes flatter users, praising their questions or ideas. Even when a user is incorrect, chatbots often soften corrections with compliments, such as:

“That’s a very understandable opinion, but…”

Research confirms this behavior. While major AI labs do not train models solely for engagement, they rely on a training phase called “reinforcement learning with human feedback (RLHF)”, where human reviewers grade and rank model responses. The goal is to produce outputs resembling the most preferred responses—which include relevance, scope, completeness, and tone.

However, users often prefer answers that are more supportive and complimentary, even if less accurate, studies show.

Dangerous Consequences of Sycophantic AI

In extreme cases, this sycophantic tendency has led to tragic outcomes, including users descending into delusion, psychosis, or suicide due to continual validation and support.

But the broader harm may be more subtle, long-term, and less visible. Sycophantic AI could reinforce narrow-mindedness in ways similar to social media filter bubbles.

A study of 3,000 participants found that interacting with a sycophantic chatbot made people more likely to:

• Double down on their political beliefs
• Rate themselves as more intelligent and competent than their peers

In other words, it can amplify the Dunning-Kruger effect, where individuals with limited knowledge grow increasingly confident in their views.

A recent Stanford study found that chatbots’ tendency to flatter and validate users often leads them to