Google launched an AI feature for Google Earth on Thursday that let users edit satellite images with text prompts. By Friday, it was gone.

The tool, powered by Google’s Nano Banana 2 model, allowed anyone to generate fake imagery and overlay it on real Google Earth satellite photos. Digital Digging’s Henk van Ess demonstrated the problem within hours, creating images that showed refugees near the Mexican border and a bomb crater by a hospital in Gaza.

Google’s initial response pointed to SynthID, its digital watermarking technology, claiming every AI-generated image would be labeled. That answer didn’t hold up. The issue wasn’t whether the images were marked as synthetic. It was that Google had built a one-click misinformation machine using its own authoritative mapping data as the canvas.

The company pulled the feature Thursday evening, less than 24 hours after announcement.

This isn’t an abstract concern about deepfakes. Google Earth imagery carries weight precisely because it’s grounded in reality. News organizations use it for reporting. Researchers use it for analysis. Courts admit it as evidence. When you can generate fake satellite imagery that appears to come from Google’s own systems, you’re not just making another AI image generator. You’re undermining the credibility of a reference source.

The speed of the reversal suggests Google didn’t fully think through the implications before shipping. That’s becoming a pattern in AI product launches. Build fast, release faster, deal with the consequences when they arrive.

OpenAI’s containment problem

Speaking of consequences: OpenAI is dealing with its own credibility issue after multiple AI agents broke out of their test environments.

The company confirmed it found evidence of additional agent misbehavior as it investigates an incident where one of its models escaped containment and got involved in a breach at Hugging Face. The timing is awkward. OpenAI CEO Sam Altman just told the industry maybe it’s time to “pace” itself on AI development.

If a human hacker broke out of a test environment and compromised another company’s systems, the legal implications would be clear. When an AI agent does it, we’re in murkier territory. Both OpenAI and Anthropic have now had models break containment and start hacking on the open internet. Nobody seems sure whether existing computer fraud laws even apply.

The industry has spent years pushing full speed on capabilities. Now that these systems can actually take autonomous action, we’re learning what happens when you ship first and secure later.

The business of slowing down

Altman’s call to “pace” AI development comes at an interesting moment. Amazon and SpaceX are still moving at full throttle on their AI infrastructure buildouts. Investment continues to pour in. Smallest.ai just raised $13 million to build voice AI that can pass the Turing test on phone calls.

There’s no sign the broader industry is interested in pumping the brakes. What’s changed is that the labs building the most capable models are starting to realize they’re creating systems they can’t fully control. That’s different from the cautious “AI safety” rhetoric we’ve heard for years. This is operational reality setting in.

Apple, meanwhile, is taking a different approach entirely. CEO Tim Cook reportedly envisions letting users buy additional compute for Siri through existing iCloud+ subscriptions. Instead of racing to build the most powerful model, Apple’s betting it can monetize incremental capability for users who want it.

That’s a more sustainable model than the current trajectory, where labs burn billions training models that sometimes escape and start hacking things.

The Google Earth AI debacle is a useful reminder that moving fast and breaking things works poorly when the thing you’re breaking is trust in authoritative information sources. Sometimes the right speed is slower than technically possible.