Pentagon says overreliance on AI contributed to missile strike on Iran school

UnitedHealth uses AI model with 90% error rate to deny care, lawsuit alleges - https://news.ycombinator.com/item?id=38299921 - November 2023 (58 comments)

https://litigationtracker.law.georgetown.edu/wp-content/uplo...

At the time a news outlet put out commercially purchased images of the target site before and after -- even a layman could see that it had a children's playground next to the building, and that it was separated from the compound.

https://www.channel4.com/news/factcheck/factcheck-iran-schoo... has the info from around the time, but doesn't show the detailed images in the news report (video) I seem to remember seeing.

I think it was maybe some earlier coverage from NYT, they published this retrospectively - https://www.nytimes.com/video/world/middleeast/1000000107503... (which is also on YouTube, https://www.youtube.com/watch?v=hDcim48W4Io). It shows that even a cursory analysis - without the benefit of enhanced military grade satellite/flyover imagery - indicates the building targeted with pin-point accuracy is a school (eg https://youtu.be/hDcim48W4Io?t=98).

In short, this seems entirely avoidable. I don't think you need any connected system, you just need an image of the target and eyes.

If they trained a model to find targets and didn't train it to avoid civilian targets, and couldn't be bothered to have anyone check them, that's still a war crime. The analyst could probably see the kids playing there with the imaging tech they have available.

This war was unlawfully started by the military at Trump's command, knowing it was not a lawful order; just as they knew firing on shipwrecked civilian sailors was murder. Seemingly it was started to win money betting on fuel prices.

Unfortunately the military all turned their backs on their oath to uphold the constitution (Again! Deploying against USA cities and kidnapping foreign heads-of-state at the whim of Trump is also not lawful). At that point it is hard to see how your democracy can be resurrected.

Just like clowns in charge were caught off guard by the fact that Iran has a drone program, resulting in the loss of dozens of air frames and much of our strategic munition stockpiles. Iran literally jump started Russia's drone program in a high profile way, and yet trump/hegseth somehow completely failed to plan for how to counter those drones and had to resort to lobbing scarce multi-million interceptor missiles at them and watching our bases take heavy damage.

These are just some of the failures we know about. The administration has been extremely tight lipped about these failures and there are surely many more that aren't common knowledge. Who knows how many hundreds of other children have been needlessly killed outside of the school strike.

...

> Hegseth and the commander in charge of the Iran operation, Adm. Brad Cooper, were questioned about the strike before Congress. Hegseth said the U.S. does not target civilians and would investigate

So the administration does not intentionally target civilians, they have just stopped caring about who a civilian is.

When it was just called a "database query" or "algorithmic estimation", it was more implied that the actual decision is left to be made and that no analysis had actually been done yet. But when AI preemptively says "I've analyzed every source of data and come to the conclusion that this ship is carrying nuclear bomb material"; it really sounds urgent doesn't it? It probably doubled down when asked about it.

https://edition.cnn.com/2026/09/18/politics/us-military-ai-f...

And I hedged that. Anything short of 100% responsibility with humans is a failure. I get investigating how it occurred, but that needs be very careful to separate ought from is.

you can also develop pro-social coping mechanisms like open-mindedly talking to someone who opposes you and trying to understand their worldview because you trust that it might expand your own

but, yeah, I doubt pro-social traits are common at the top of any hyperstrict hierarchical organization like the USAF

Replace "AI" with any other tool or software in your scenario, and the outcome is the same. The same people and organizations are accountable, it's just that we currently live in a world without accountability in the highest places where it's most needed, and that's unrelated to AI.

These missiles cost a couple million each.

We have the largest stocks of missiles ever. We can build them faster than we use them and cheaper than anyone else. So you’re a liar saying it would have saved money doing it any other way

/s

No-one mentioned "using ambulances as cover". That is separate from "enemy combatants are acting in a first responder capacity, but are still enemy combatants, therefore the entire first response effort is a viable target". That is a war crime that Israel regularly likes to commit.

> These are the same people who view women as property, homosexuality as a reason for execution, and 13 years olds as viable brides.

Halacha? Acquiring your wife through kiddushin? Only a husband can issue a get (divorce document)? Women in agunahs (dead marriages)? Sounds like property to me.

Rabbinic law makes capital punishment effectively impossible, but the Orthodox sects do everything possible to punish: disownment, both family and social and actual exile from physical communities.

Don't girls reach halachic adulthood at 12?

And lets not throw stones here. Even in the US, you can get married with parental consent or a judge's consent at any age in Mississippi, New Mexico and Oklahoma.

You can get married at 15 in Hawaii and Kansas.

You are "grandfathered" into sexual consent (not as in "marital rape" but "no longer statutory rape") when you do so.

So we may not want to be lording it too much on some guise of moral superiority, particularly given that most of the marriages that happen at those ages are not "young couple got pregnant, want to do the right thing" but "older man in conservative/religious community saw young vulnerable girl and decided he had to have her".

This isn't the case where we are talking about a company doing anything socially good. It's simply there to help authoritarian governments.

If so, I'd say yes, you share some of the responsibility. Legally, I'm sure you'd be fine, but if that were me, I'd feel responsible.

The doctor of course is responsible too; they're the one who didn't double-check that it was an appropriate drug. But multiple people can share responsibility for a bad thing.

Whether you may/should feel responsible depends on how the software works, I guess. Did it automatically recommend some drug, somewhat confidently, with gray text on gray background saying "AI sometimes makes mistakes"?

So I think we should behave responsibly when making software, even more now with AI

If I remember correctly, some of the people responsible for torturing iraqui prisonniers did get sentenced. They were definitely scapegoats of larger organisations. But if the "lampistes" get caught, this may have _some_ effects on otherw potential _lampistes_.

Palantie execs will be fine, of course. I'm.not expecting them to grow a conscience at this stage. So they will shrug it off, and go on with creating shareholders value. Until they're sacked by their board for not meeting some quotas, and have to live with _less money_ - which is the closest thing to the seventh circle of hell for those person, anyway.

https://news.ycombinator.com/item?id=47441817

https://news.ycombinator.com/item?id=36559017

etc.