Meta Ran Ads That Contained AI-Generated Child Sexual Abuse Imagery

I reported the ad and a week later it was still there. I switched to Apple.

Reported the add but damn YouTube, the freaky fake medical ads are bad enough.

  "Sexual exploitation is horrific, and we work aggressively to keep it off our platform" a Meta [which in the past 2 years has laid off and disbanded all teams responsible for this] spokesperson says.

It effectively mandates a lower-limit on spend for budgeting purposes.

Sure, that money might be spent ineffectively... but corporations will figure it out if their only options are to

1. Not invest in regulatory compliance at all and get fined for noncompliance.

2. Throw money at the problem, spend it inefficiently, and STILL get fined for noncompliance.

3. Throw money at the problem, spend it well enough to pass muster, avoid fines.

For a lot of people, government regulation of this sort is not desirable, but at what point does someone need to drop the hammer to stop this?

Results:

- Reporting a message does nothing no matter how much the content breaks their terms of use. 100% of messages I reported bounced back in a matter of minutes with "no action"

- However, using the "request another review" function, which is available after the initial "no action" response, actually seems to work. How quickly this works seems to vary, I've had actions taken as result of this from 1 day to multiple weeks.

Tiny providers have to scramble, because they'll be destroyed by a single failure to handle it fast enough.

Step 2: wait for it to get out of the news cycle.

Step 3: donate heavily via lobbyists to the opponents of the politicians who called you out

Step 4: de-fund the unit down until just barely enough that you're not lying when you say you "have a team working very hard on the problem." A dozen people in some third world country you pay a dollar a month each, for example.

To get a Facebook account they require all sorts of PII.

Why not have people give their PII to set up an advertising account and if they distribute CSAM then they are banned. Removing access to WhatsApp will stop a large portion of the world from participating.

And the parts that can't? Well, they can't advertise on Facebook/Instagram anymore.

This would mean every single user gets 10 uniquely personalized ads per year.

Even if that wasn't hilariously unreasonable by several orders of magnitude, if the moderation cost was was included in the cost of serving ads, this would all self regulate and only serious actors would participate. Treating it as a public bulletin board while still raking in a percentage of the impressions for sure is a nice business model, but not what's best for the rest of the society. Maybe ads just shouldn't be such a lucrative business.

You can bring this number down by only reviewing new/low-reputation advertisers, increasing minimum spend, rate limiting, etc.

You can also pre-filter for "definitely not CSAM" and only review the more ambiguous stuff.

I think your numbers are low too, surely a moderator can review over one ad per minute on average. And it may be more effective to split-screen multiple ads (with redundancy).

But at the end of the day it's a numbers game, something is going to slip through.

through meta might have lobbied on both

but they lobby for specific (often pretty bad) forms of age verification to be able to

- spy on people better

- have a regulatory moat where due to beady designed laws new alternatives have a much higher hurdle/initial investments and in turn never get anywhere

>The amount of slop you will get suggested as soon as the algorithm warms up is nuts. The ad you are first served might also be slop.

This did not happen for me. I got news about the SpaceX rocket and then more suggestions for news on the homepage, many related to politics.

I did notice that a bunch of the videos were in languages I did not understand though.

The purpose of the new term was to emphasize that producing it involves the actual abuse of an actual child.

And then somehow it got applied to drawings and lately computer-generated stuff.

If I had to guess, this is to try and distance child sexual assault from the concept of "pornography", and maybe make it more abundantly clear that "child porn" is always, 100% of the time, sexual abuse.

I dunno, language evolves like this sometimes. Like "smoking" which is now called "tobacco use" or "cannabis use" to clarify more what is being smoked

It's Facebook's problem if Facebook can't do the things Facebook is required to do. "Oh no it is hard"? Tough. Either they find a way or find they're no longer allowed to do this business.

> This truly is a case where AI is the solution. There's no way to automate this at scale with humans. As AI gets better hopefully detection and moderation will catch up

AI *might* solve this, or Meta's AI may be trapped in an arms race with advertisers trying to get around it for whatever reason has driven them to do this in the first place; but fully automated mass-generation of personalised ads was never a mandatory part of their business plan, so it was a problem they never needed to have.

Don’t buy into Meta’s narrative that there’s nothing that can be done while they continue to make money hand over fist.

Is this firehose of social media actually a societal benefit? I’m less and less convinced that it is.

no, it's just more work, and it's not like facebook could not afford to pay more people

Ad pipeline is malware at this point.

So you'd think that a sophisticated tech company that had paid that financial penalty would find it pretty easy to have a simple text filter for "Martin Lewis" and perhaps even an image filter for his very distinctive face which pushed those ads into a moderation queue, particularly if they both appear together in the scam-ridden investment opportunity category and the advertiser is a new one (that's probably not going to stick around under that name for very long). But no, they remained a common theme for years afterwards...

And, as the sibling ad points out, if you report stuff that's obvious scams you get told they've reviewed it and found it to be OK. Plus you get more of it promoted into your feed because hitting the report button is a form of interaction with it...

I actually had some sympathy for Facebook when they were condemned for not moderating fake news from Myanmar they have no real capacity to moderate, or condemned for daring to slap the gentlest of content warnings on antivax stuff in contexts they understand perfectly well... but when it comes to ads they get paid for, which is a much smaller fraction of Facebook content that is unambiguously not protected speech or a matter of opinion, their lack of effort is quite something to behold.

Well that means don't do the thing that you can't control, or they shouldn't be legally allowed to do the thing they can't control.

Additionally, for one of the biggest companies in the planet, that has done mind bogglingly difficult things in order to make profit, that's the lamest excuse ever.

Fine them out of existence.

They could just set the minimum ad spend at a level that makes moderation possible.

Let's say it costs $15 to have a human spend 15 minutes checking an ad. If the minimum spend per ad is $100 it'd easily cover the costs of the check. And I imagine most ad campaigns are spending a lot more than $100.

How is this our problem? We make the rules, the obey them. If they can't they ought to be shut down. If they want to argue that at their scale they can't abide by the rules society lays down for them we can cut them down to something human sized.

For software, sometimes paying off that debt means getting rid of some of your features/code/stuff. i.e. there's another alternative to "let's AI the AI".

scale the business down. shrink the number of users. make less money.

It doesn't matter how many people are consuming the ads, it matters how many companies are placing the ads.

Let's not invent excuses for the trillion-dollar tech companies. Meta has plenty of money to put humans in the loop. It just chooses not to because some upper manager needs to buy a fifth house, or a third private island.

In this case, we are talking about ad images.

1000 moderators x 5 second per image to click deny or approve.

So 1000 moderators review 720000 ads/hour

Per 8 hour workday: 5,760,000 ads

Per full 24h day: 17,280,000 ads

So we will use 3k for 3 shifts.

51,840,000 new image ads moderated per day.

At ~$1,000 per moderator per month (most jobs are outsourced to Africa and Asia):

$3 million per month $36 million per year

I don't see any public numbers for how many new ads they add daily but meta reported removing 159 million scam ads during 2025, so equivalent to around 436,000 scam ad removals per day.

They can also choose to show fewer ads, i.e., only show manually human reviewed ads, and if there are too many to review, charge more per ad until there aren't too many.

Just because the social property giant corporations steal is somewhat abstract doesn't change any of the fundamentals of right and wrong. I'm so tired of this ultra-capitalist "right to be massive" assumption which places the value of unhinged monopolistic growth over maintaining social commons. What these companies "should do" is face penalties until they can safely manage their scale, or scope their scale to a level they can maintain safely.

Secondly, if I’m wrong and it’s not possible, then you stop providing the dangerous service! There’s no reason Meta has to exist in its current form. If it can’t meet basic standards of human dignity in its product offerings, then it should change or cease to exist.

You don't have Metas legal team nor does the power structure find your valuable to have helping them out nor can you harm them if you don't like them.

There was, quite famously, a man who got away with these sorts of things for quite some time. He owned an island. I can't recall his name....

The truth is impunity is the rule, victims especially of CSA and CSAM, or revenge porn, or of Mark or Musk's companies generating naked images of their body have basically no power, and its an exception when someone who is an abuser AND has any amount of social standing is put away.

but, plenty of people do end up in prison, especially if its facebook/dropbox/gmail/etc... they just dont make the headlines (especially nationally) unless its especially heinous.

No, not at all.

"removing over 36 million pieces of child sexual exploitation content last year"

Not CSAM.

Also how much of the child exploitation material was created by the children themselves?

Teenagers who send each other nude photos are creating child exploitation material but you hardly want to put them jail for that.

“Just stop using meta properties then”. Sure but some of us have wives, friends, etc

Not only are their AI tools ineffective at what they claim they should be doing, they can't even be bothered to prevent removed ads from being re-uploaded later on.

As for the cost of content moderation, Meta has zero interest in any type of moderation to begin with. The cost is just a public-facing excuse. In Careless People, Wynn-Williams mentions that Facebook didn't do anything to stop anti rohingya posts in Myanmar because they did not have any fact checkers who could read Burmese. In other words, they did not have a single moderator in the damn country.

On the other hand, now try to ban that telecom. Gods will laugh.

The eternal question of tech platforms - should we be responsible for what users post? (Ads are not user posted, so we can eliminated that factor. Even if posted by 'users' it's a separate pipeline) But with companies like Meta it doesn't even have to go that far - the philosophical conversation - because it's so hostile to users. People get banned for mild things all the time (apparently extra the case, since the 'pivot' to AI) while the absolutely horrible things you report never get taken down

---

This is sarcasm, by the way. Most decent people make sure what they're about to do won't harm other people and if would cause harm they wouldn't do it. But social media companies and ad networks aren't run by decent people. They're run by people who believe their business is entitled to infinite scale regardless of how many people are harmed along the way.

When people started using Grok to make CSAM the platform's response was to paywall image generation. If we don't start making the driver responsible for what the car does, the coder responsible for what the code does, the CEO responsible for what the company does, they will continue to exploit people in increasingly horiffic ways.

This is why they pushed for the change from “child porn” to “child sexual assault material” because LOTS OF THINGS can be “materials”.