Your AI Is Confident, Helpful, and Working From Completely Wrong Information

An Australian toilet paper brand suspended their AI email agent last week after it told a customer their subscription would more than double in price. Here's why this happens, why it's more common than you'd think, and three things to do before deploying your next AI agent.

By Steven Tann · Thu, 09 Jul 2026 05:07:50 GMT · blog

Your AI Is Confident, Helpful, and Working From Completely Wrong Information

Last week, an Australian company called Who Gives A Crap suspended their AI email agent.

The AI had sent a customer a message saying their toilet paper subscription was about to more than double in price. The customer was, understandably, furious. Who Gives A Crap had to step in, clarify it was wrong, and pull the AI tool off the job while they figured out what went wrong.

Now, Who Gives A Crap is a proper business. They're not some back-shed operation. They're a certified B Corp with millions of customers and a well-known brand built on trust. And their AI still went out and told a customer something completely false with total confidence.

That's the thing about AI. It doesn't know it's wrong. It has no uncertainty face. No nervous laugh. No "actually, let me just double-check that." It reads what it's given, generates a response, and delivers it like it's reading from scripture.

If the data it's working from is rubbish, the output is going to be rubbish. But the customer won't know that. They'll just see an official-looking email from a brand they trusted.

This Is Not a Bug. It's a Feature Working Exactly as Designed.

People often talk about AI hallucination as though the AI is making things up out of thin air. Sometimes it is. But a lot of the time, the problem isn't the AI imagining things. The problem is the AI reading real data that's wrong, outdated, or incomplete, and acting on it with full conviction.

In the Who Gives A Crap case, we don't know exactly what happened, but the likely culprits are familiar to anyone who's tried to automate anything: a pricing update that hadn't propagated through all the systems, a test environment variable that leaked into production, or a lookup table that was pulling old figures. The AI wasn't hallucinating. It was reading something, and that something was wrong.

This is the garbage-in, garbage-out problem. And it's been true since the first spreadsheet. But with AI, the consequences are faster, more visible, and reach more people before anyone notices.

The Data Problem Is More Common Than You Think

In March 2025, Qlik surveyed 500 data professionals in the US and found that 81% of companies still struggle with AI data quality. Not small startups with two spreadsheets. Companies actively investing in AI. And 96% of those professionals said poor data quality could lead to widespread crises in AI projects.

That's not a fringe concern. That's nearly every business that's deploying AI.

On the CRM side, Validity's State of CRM Data Management report found that 76% of organisations say less than half of their CRM data is accurate and complete. Not "could be better." Less than half is right.

So most businesses are sitting on data that's, at best, partially correct. And they're feeding that data to AI systems that will act on it immediately, confidently, and at scale.

A human might pause. "Hang on, that price looks odd." "That doesn't match what I remember." A good VA would sort of sense-check it, ring the customer before sending something that significant. AI won't. AI trusts the data more than a human would, because AI doesn't have the benefit of having worked there for three years and knowing that the pricing sheet gets updated every quarter.

The ChatGPT Problem Is the Same Problem Wearing a Different Coat

Here's another angle on this. In June 2026, researchers caught ChatGPT recommending scam websites to users looking for products. Not because OpenAI made a mistake in their model. But because bad actors had deliberately seeded the AI's training and search data with content designed to promote fraudulent sites, making them appear legitimate.

The AI was working from poisoned data. And it passed those poisoned recommendations to real people shopping for real things.

OpenAI removed the sites once users flagged it. But the principle is the same as the Who Gives A Crap story. The AI is a confident delivery mechanism. It delivers whatever it's given. If what it's working from is wrong, it'll deliver that wrong thing to your customer with the same pleasant tone it would use for a correct answer.

The data is the thing.

What This Means for a 5, 10, or 20-Person Business

If you're running a smaller operation, you might be thinking this is an enterprise problem. Big companies, complicated systems, complex data pipelines. Not your thing.

But the same issue applies at every scale. It's just faster to fix when you're small.

Here's what I see a lot of:

The CRM that hasn't been cleaned in two years. Contacts with wrong phone numbers, duplicate records, old job titles, email addresses that bounce. You connect an AI assistant to it, ask it to follow up with leads, and it's reaching out to people who left those jobs two years ago on behalf of your business. The AI is doing exactly what you asked. The data is the problem.

The pricing that lives in three places. The website says one thing. The quoting spreadsheet says another. The CRM has a third figure from a campaign last October. You give an AI access to "your pricing" and it picks the wrong source, because they're all technically your pricing.

The product descriptions written during COVID. "Available for delivery in 5-7 days." The business now delivers in 2 days. The AI is still telling customers 5-7 because that's what's in the product database. Nobody's updated it.

These are not hypotheticals. These are real situations in real small businesses. And without AI, the damage is limited to the odd email a human sends with old information. With AI working at volume, the same wrong data can reach hundreds of customers before anyone notices.

Three Things to Do Before You Deploy Another AI Agent

This is the actionable bit. None of this is complicated.

1. Do a data audit on whatever the AI will touch.

Before you connect any AI to your CRM, your product database, your pricing, or your customer records, spend an hour going through it manually. Ask: is this accurate? Is this current? Is there a single source of truth, or are there three competing versions?

The AI will trust whatever it finds. You need to trust it first.

2. Build a "last updated" discipline.

Wherever pricing or product information lives, add a date field. "This was last reviewed on X." Then build a reminder, even a simple calendar event, to check it every quarter. You want to know when data is potentially stale before the AI tells a customer something that stopped being true months ago.

3. Test with real-world scenarios before you go live.

This is basic, but surprisingly few people do it properly. Before your AI agent goes live on customer-facing communications, run it against 20 different scenarios yourself. Ask it the things your customers ask. Check the answers against what you know to be true. If it gets something wrong in testing, it'll get that same thing wrong at scale in production.

The Who Gives A Crap team will fix their issue. They're a well-run business and they moved quickly when it went wrong. But the better play is to catch it before it reaches a customer at all.

The Bigger Picture

AI makes good systems faster and bad systems worse. That's not a criticism of AI. That's just the nature of amplification.

If your data is clean, consistent, and current, AI will do extraordinary things with it. It'll follow up with the right people at the right time, answer customer questions accurately, and handle volume you'd never manage manually.

If your data is a mess, AI will make that mess very visible, very fast, to a lot of people at once.

The good news is this is fixable. Especially at small-business scale, where you're not dealing with 50 different systems and 10 years of legacy data. A weekend of proper data hygiene, clear sources of truth, and a test run before going live means your AI works the way it's supposed to.

The alternative is your version of the toilet paper story. And while "Who Gives A Crap" makes for a brilliant headline, it's probably not the one you want associated with your brand.

---

Sources:

---

Want to see how this could work in your business? Book a call and let's talk about where you're at and what's possible.

---

About Steven Tann: Steven helps business owners build systems that run themselves using AI. After 10+ years helping 7,000+ businesses and building his own autonomous operations, he's the bloke who actually does it, not just talks about it. Find out more at steventann.com.