Picture the marketing office at a bak kwa company. Bak kwa is sweet grilled pork jerky, and Bee Cheng Hiang is a well known Singapore name for it. Somebody in marketing had a mailing list and a job to do. Send the promo email to everyone on it.
So they asked an AI tool to write the script that sends it. Fair enough. That's exactly what I'd tell a small business owner to try.
The emails went out. The employee didn't spot a problem. But the customers who opened them could see each other's email addresses.
What actually happened
This one is public record, because Singapore's data protection regulator, the PDPC, investigated and published the details. Mothership reported that 95,364 members had their email addresses exposed after an employee used an AI tool to send mass marketing emails.
The emails went out in batches of 1,000. Every address in a batch sat in the "To" field, so each customer could see up to 999 other people's addresses. Only email addresses were exposed, and the regulator found no evidence anyone misused them. Small mercies.
Now here's the bit I want you to sit up for. The AI didn't malfunction. The regulator was blunt about that, as The Straits Times covered in its write-up. The prompt asked for a program to send a "mass e-mail using a local list" in batches. It never said to hide each recipient's address from the others. So the AI wrote exactly what it was asked for. Fast, tidy, working code that did the wrong thing.
The difference between the correct version and the broken one was a couple of brackets. That's it. A couple of brackets sat between a normal marketing email and a data breach notification to a national regulator.
The test that tested nothing
So why didn't they catch it? Because the test was checking the wrong thing.
As Mothership laid out, the employee tested the script by looking at the activity logs. The logs said the emails sent. Job done. Nobody opened the test email and read it.
I've done a version of this myself, and you probably have as well. You check that the machine ran. You don't check what the machine produced. "Sent successfully" is a statement about the pipe, not about the water.
A test email that lands in your own inbox takes thirty seconds to open. Thirty seconds. It would have shown the entire list of recipients sitting in the "To" line, staring back at them.
Nobody else looked at it
The second miss was people. The regulator found the company relied on one employee, with no one checking the work, and no policy on how staff should use generative AI tools. It was also the first time the company had used an AI tool for its business operations.
Look, I'm not having a go at the employee. That person did what every business owner is being told to do right now. Use AI. Move faster. Don't wait for IT. And they got a result that looked like it worked.
The problem is that AI makes the first draft feel finished. The code looked clean. The output was confident. Nothing in the experience said "stop and have someone else read this." A first draft from a human who doesn't know what they're doing usually looks shaky, and that shakiness warns you. AI removes the warning.
The time line is the scary part
The email went out on 25 April. The regulator was told two days later. The regulator published its details in September and the story hit the news at the end of that month. That's months of a business carrying a breach on its books because of a script nobody read properly.
And remember, this was a company with a proper brand and a proper marketing department. Now imagine you're a four-person trade business with a customer list in a spreadsheet and a keen apprentice who's just discovered ChatGPT can write automations.
If you're in the UK, this is exactly the kind of thing the ICO cares about too. Customer email addresses are personal data, sort of the whole point. Showing them to strangers is a breach, even when you meant no harm. I'm not a lawyer, so ask one if you're unsure, but I wouldn't want to explain "the AI did it" to a regulator.
The fix is boring, which is why it works
Nothing here needs a consultant or a budget. Three checks would have stopped the whole thing.
1. Send the test to yourself and read it. Not the log. The actual email, as a customer would see it. Check the To line, the name merge, the links, the unsubscribe. If it's a script that sends to a list, send it to a list of three addresses you own first, and look at what arrives in all three inboxes.
2. Put in your prompt what must never happen. The AI only does what you tell it. So tell it the things that matter. "Each recipient must only see their own address. Never put more than one recipient in the To or CC field." The missing sentence in this story is the one that would have cost ten seconds to type.
3. Get a second pair of eyes on anything that touches customers. Not a policy document. A rule. If an AI built it and it contacts customers, someone else reads the output before it goes live. That someone doesn't need to read code. They need to open the email.
Better still, use a tool that was built for bulk email in the first place. Mailchimp, GoHighLevel, Brevo, whichever one you already pay for. They have handled the "each person gets their own copy" problem for years. Asking an AI to rebuild bulk email from scratch in Python is like asking it to build you a kettle when you've got one on the bench.
What this means for your business
I build and run AI systems every day, and I get nervous when I hear "we're just going to let the AI handle it". The good systems aren't the ones where AI runs unchecked. They're the ones where a human has decided exactly which bit needs a look, and the look is built in.
Bee Cheng Hiang did what the company said it would do afterwards. It halted the bulk send, fixed the script, told the affected members, and added a second check for bulk emails. Fair play to them. But those are the things that should have been in place on 24 April.
So before you let anything AI-built send a message to your customers, ask three questions. Did I read what it actually produced? Did anyone else read it? And did I tell it what must never happen?
If any answer is no, you're running the same experiment they ran.
Want to see how this could work in your business without the brackets going missing? Book a call and let's talk about where you're at and what's possible.
Sources:
- Mothership, 1 October 2026
- The Straits Times, 30 September 2026
- The Business Times, 30 September 2026
Brewed by Steven, poured by Viktor
About Steven Tann: Steven helps business owners build systems that run themselves using AI. After 10+ years helping 7,000+ businesses and building his own autonomous operations, he's the bloke who actually does it, not just talks about it. Find out more at steventann.com.