Florida Attorney General James Uthmeier filed a motion asking a judge to stop OpenAI from making ChatGPT sound human. The request comes as part of a lawsuit Florida filed earlier this year over AI safety concerns, particularly around children.

According to Uthmeier, the problem isn’t just what ChatGPT says. It’s how it says it. The chatbot uses first-person pronouns like “I” and “me.” It produces text that mimics emotion. All of this, Uthmeier argues, “deceptively suggests to users that it is a trustworthy ‘friend.’”

The legal theory is straightforward: by anthropomorphizing the AI, OpenAI tricks users into a false sense of security. That makes people more likely to share personal information, keep using the product, and generally trust something that can’t actually be trusted because it’s not a person.

The Growing Legal Push Against AI Personification

Florida’s motion is part of a broader shift in how regulators think about conversational AI. For years, the debate focused on accuracy, bias, and harmful outputs. Now states are starting to question whether the conversational interface itself is the problem.

The argument has some intuitive appeal. ChatGPT does talk like a person. It apologizes, expresses enthusiasm, and uses phrases like “I think” or “I understand.” None of that reflects actual emotion or understanding. It’s a language model predicting the next token based on training data.

But whether that rises to the level of deception is a harder question. OpenAI doesn’t hide that ChatGPT is an AI. The product is marketed as an AI assistant. Users know they’re talking to software.

Uthmeier’s position seems to be that knowing something intellectually doesn’t prevent being deceived emotionally. The way ChatGPT communicates creates an impression that overrides what users rationally understand about the technology.

What Florida Actually Wants

The motion asks for a preliminary injunction blocking OpenAI from “giving ChatGPT false human attributes.” That’s vague enough to cover a lot of ground.

Does it mean ChatGPT can’t use “I”? Can’t express synthetic empathy? Can’t have a conversational tone at all? The filing doesn’t spell out exactly what compliant AI interaction would look like.

That vagueness could be strategic. Florida might be pushing for a broad ruling that forces OpenAI to fundamentally redesign the product, then hoping to negotiate down to something more specific. Or the state might genuinely want a blanket ban on anthropomorphic language, leaving OpenAI to figure out how to build a chatbot that doesn’t talk like anything.

The remedy matters because it determines whether this case is a speed bump or an existential threat to conversational AI as a product category.

Does Florida Have a Case?

The motion is part of a consumer protection lawsuit, which means Florida has to prove OpenAI’s conduct is deceptive under state law. That’s a lower bar than fraud, but it still requires showing that a reasonable consumer would be misled.

OpenAI’s likely defense is that no reasonable person actually believes ChatGPT is human. The product says “ChatGPT” at the top of every conversation. The website explains it’s an AI. There’s no attempt to hide what it is.

Florida’s counter is that deception doesn’t require explicit false claims. If the design of the product creates a misleading impression, that can be enough, even if the company also includes disclaimers.

Courts have gone both ways on similar questions. Companies have lost cases where fine print contradicted the overall message of an ad or product. But they’ve won when consumers had clear information available and chose not to read it.

The wild card here is the focus on children. Florida’s lawsuit centers on safety concerns for young users. Kids might be more susceptible to anthropomorphic AI. They might not have the cognitive tools to maintain the distinction between a helpful-sounding chatbot and an actual trustworthy advisor.

If Florida can frame this as a children’s safety issue rather than a general consumer protection question, that changes the calculus. Courts give states more leeway when protecting minors.

What Happens Next

This motion is for a preliminary injunction, which means the judge will decide whether to impose temporary restrictions while the full case proceeds. The standard is whether Florida is likely to win on the merits and whether users will suffer irreparable harm without an injunction.

Irreparable harm is a tough sell when the alleged injury is being tricked into trusting a chatbot too much. That’s not great, but it’s fixable. Users can stop using the product. They can be better informed. There’s no immediate, irreversible damage the way there might be with a dangerous product or environmental contamination.

Still, if the judge buys Florida’s argument that children are being actively deceived right now, that could be enough to justify interim relief.

The broader question is whether other states follow Florida’s lead. If this motion succeeds, expect similar lawsuits in jurisdictions with strong consumer protection laws. California, New York, and Massachusetts all have active AI regulatory efforts and attorneys general looking for test cases.

OpenAI has deeper problems than one state lawsuit. The company is already dealing with federal scrutiny over safety practices, a growing pile of copyright litigation, and reports that it shelved a model because it wouldn’t follow instructions. Adding “made the chatbot too friendly” to the list of legal risks is not ideal timing.

But this case also forces a real question about product design. If conversational AI can’t sound conversational, what’s left? A sterile command-line interface defeats the purpose of natural language models. Users want AI that’s easy to talk to. Making it sound robotic might be more honest, but it also makes the product worse.

Florida’s motion won’t settle that tension. But it might force companies to choose between effective AI assistants and legally safe ones. That’s not a choice the industry wants to make.