Amazon identifies Oct 17 PHP2Shell exploitation prior to public disclosure

Posted by cubefox 8 hours ago

Counter235Comment85OpenOriginal

Comments

Comment by andyvanosdale 4 hours ago

If this was being probed and/or exploited on October 17th, this throws shade on the PHP team for throwing shade on Chainguard in their public announcement.

Essentially, the PHP team is deflecting and abdicating their responsibilities in all this mess they created. "Chainguard broke embargo", "support us with money", "we are just volunteers", "we had knowledge years ago but no POC because we wanted to ship features and not security fixes". All just deflections.

This fiasco shows me that I would never use PHP for any project I'm part of. The PHP team royally messed up, and they are deflecting the blame left, right, up, and down.

Comment by JaggedJax 3 hours ago

This doesn't really change anything, other than check your logs back through the 17th. Amazon is not claiming they knew about this being exploited before the 21st. No one appears to be claiming that. Their wording could easily mean they went back through logs and found it later. I haven't seen clear evidence yet that any of the thousands of early attempts were successful. They appear to just be people sniffing/guessing around to try and figure it out, at most sometimes getting something to work in one specific environment.

Yes, we live in a new world now and need to handle disclosures differently by learning from this experience. The first major RCE disclosure that came with 3rd party leaks of this era was bound to cause these issues no matter who it affected. Let's hope the next one is better.

The major mistake I see in this incident on the disclosure side that kicked this all off was Calif posting their video early because they took vibe coders on X at their word. Maybe the early notice by PHP of an upcoming release was a mistake in hindsight given how people reacted.

[] Is this standard in the security researcher world? Let's beat embargo by a few hours and cause extra panic by putting more people at risk?

Comment by andyvanosdale 2 hours ago

It's not about changing anything, and that was not what was posted in my comment. Amazon saying they saw some signs of people probing for this back on October 17 shows that the shade that the PHP publicly threw on Chainguard was misdirected and a deflection on their own efforts to coordinate the release. It should change the PHP team's deflection, if anything.

Nothing is standard in the security researcher world. I have no interest in PHP nor Chainguard, but I have a greater interest. PHP folks can deflect the blame to Chainguard, but really, it's the PHP team who mishandled this from years ago. There were likely other disclosures or discoveries of this, if Amazon were starting to see this probed on October 17. Heck, OpenAI found this at nearly the same time as the original POC disclosure.

Comment by JaggedJax 2 hours ago

Was PHP not correct that as of the embargo there were no confirmed valid uses of the exploit, even with the embargo being broken and people rushing to try to exploit it early?

Maybe we are arguing that they should have release early because of the possibility of it being exploited, but I do believe PHP (and others who backed it up) when they said they reviewed the exploits flying around on Twitter and verified those would not be able to successfully perform this exploit.

And again, Amazon is not saying they saw this being probed on Oct 17th. They are saying that today, in hindsight, they see this in logs from the 17th (unknown if it would have been successful either). If Amazon clarifies that they saw this in the wild at the time, then I agree with you. They just didn't say that.

Comment by dlor 1 hour ago

I don't even know why we're spending time blaming anyone here. The only reason any of these vulnerabilities matter is because bad people do bad things with them. The night of the disclosure, I saw reports like this one and many others, and it was more than enough for what I thought any reasonable person would deem to meet the bar for early release. It turns out reasonable people can disagree on almost anything.

I'd rather we all collectively spend this time talking about how we go fix all the rest of the vulnerabilities we (and everyone else) are finding as quickly as possible and get the patches and mitigations deployed to everyone that needs them as quickly as possible over the coming months.

(I'm the guy that decided to release these early)

Comment by wat10000 2 hours ago

If the bug isn't being actively exploited in the wild, then you want a coordinated disclosure/patch.

If it is, then you just want to get it out as fast as possible.

The PHP team thought they were in the first case and acted accordingly. Turns out they were in the second case. But I see no indication they could or should have known that. They made the right decision with the info they had.

Comment by gilrain 4 hours ago

MediaWiki is the last thing that prevents me never dealing (directly) with it again. And it's hard to imagine a drop-in, migrateable replacement for larger wikis. T_T

Comment by EGreg 3 hours ago

I don’t know why everyone is so hung up on this patch being the only way to fix the situation. The way that Internet services are attacked is by crafting malicious payloads. Having a proxy in front of your application server is a very smart idea, especially if it can do one job: decode input, and re-encode it in a safe way. In a couple hours, I put together a proxy using Claude, which completely eliminates this php2shell attack and many other others, that might possibly be discovered in the near future. Among other things it just decodes and re-encodes multipart-form-data payloads with its own random sane delimiters rather than attacker-chosen ones.

We use it ourselves in our deployments, and yesterday I released it free under MIT license, for anyone who is unwilling or unable to patch their PHP (which is probably millions of servers right now):

https://github.com/Qbix/proxy

Comment by andyvanosdale 3 hours ago

While I get that your intentions are good, if your proxy has a vulnerability, it can be exploited. Many people use nginx as a proxy, and it has its own vulnerabilities that have been fixed over the years. You may have solved one problem while opening up another. Even those fronting services have their own vulnerabilities that have to be fixed.

If a system is accessible by someone else, it is inherently insecure and needs to be hardened.

Comment by e28eta 2 hours ago

It, at the very least, adds additional complexity to exploit your service by putting another layer between attackers and secrets / capabilities held by the application.

And in a case like this, with so many exploitable PHP boxes out there for attackers to choose from instead, certainly buys you some additional protection.

I wouldn’t choose to roll my own on the day I needed it though.

Comment by EGreg 3 hours ago

Well yes, but we have to think logically.

Relying on this patch, you're only protected against these specific CVes, but in general a proxy can protect you from more. It's strictly additive security, right?

And if you can't or won't patch your PHP, this is strictly better than getting pwned and having your PHP-FPM execute arbitrary commands. That's as bad as it gets, and you better hope that Linux user permissions can constrain it, but your apps are basically gone. PHP-FPM even allows PHP to overwrite .php files, meaning next time they are hit, it's completely arbitrary code that persists.

As a side note: https://qbixserver.com is a project I started several months ago, that is a pure PHP server without needing nginx, and it forks after loading your classes, instead of preforking like php-fpm. It's not vulnerable to php2shell, and it is able to rewrite php code on the fly when it loads it, to make it more safe. It does things like disallowing your php scripts to write to .php files, and it makes require($filename) check that $filename actually ends in .php ... a lot of stuff like that.

The security added by this proxy is strictly additive, it can't make things worse. This proxy can be run by a Linux user with no permissions at all, so it just transforms input. Can it have a sleeper code that transforms input in a way that causes your PHP to do something even worse? Well, this is why you should point Claude at the repo and ask it if it has any backdoors. It's completely open source and GitHub actions has been used to build the binary. So I guess if you don't trust GitHub, you can also compile it yourself, but then you're trusting GCC. (Ken Thompson has an interesting talk on Trusting Trust of compilers, from 30 years ago.) But at this point we're way past what people usually are concerned about.

Comment by zdragnar 2 hours ago

> in general a proxy can protect you from more. It's strictly additive security, right

No, the proxy itself may have memory vulnerabilities that could give attackers remote code execution capabilities.

Comment by EGreg 2 hours ago

Right. But it can’t, for several reasons: 1) The proxy is open source and much simpler than PHP runtime or a web application. Its code only takes input and generates output. It is basically a pipe. It has no other capabilities to exploit. 2) The operating system, such as BSD or Linux, supports running processes under a user and enforces limits on that user. So you just jail this proxy — since it has one purpose. Unlike your application, which may do many things and you can’t restrict it, it may become a confused deputy

Comment by hemlock4593 2 hours ago

> The security added by this proxy is strictly additive, it can't make things worse

Except if this code contains bugs that lead to another RCE.

> Well, this is why you should point Claude at the repo and ask it if it has any backdoors.

When do people understand that LLMs are no oracles. Just because some LLMs tell you the code is safe or has no backdoor doesn't mean it is true.

Comment by 8note 2 hours ago

> When do people understand that LLMs are no oracles. Just because some LLMs tell you the code is safe or has no backdoor doesn't mean it is true.

if it tells me there is a backdoor, and shows me how its used, i can be pretty sure that there is one though.

if claude cant find the backdoor, chances are that claude cant find a backdoor. its still not an oracle, but that is still useful feedback for updating priors

Comment by EGreg 2 hours ago

Right. But it can’t, for several reasons:

1) The proxy is open source and much simpler than PHP runtime or a web application. Its code only takes input and generates output. It is basically a pipe. It has no other capabilities to exploit.

2) The operating system, such as BSD or Linux, supports running processes under a user and enforces limits on that user. So you just jail this proxy — since it has one purpose. Unlike your application, which may do many things and you can’t restrict it, it may become a confused deputy

What this has taught me is that we should decouple as much as possible into microservices, so that we can properly sandbox each service, and that way we have chokepoints.

Comment by petcat 3 hours ago

I would be very careful with a vibe coded internet proxy server handling untrusted input especially written in C

Comment by EGreg 2 hours ago

Me too. You should be.

It’s very simple to check it though. And to sandbox it to have zero read and write permissions to any files. About all it can do in a malicious direction is drop requests or deliver malformed requests, and it is straightforward to see there is no way the code will allow that.

Comment by fn-mote 1 hour ago

After reading this, I checked the code. I saw a bunch of malloc calls not checked to see if the allocation failed. I don’t think it’s as obviously bulletproof as you do.

Comment by mirashii 1 hour ago

It's bulletproof, in the same way that my open window is bulletproof because a bullet will fly right through it and not damage the glass.

Additionally, the author isn't engaging on this site in good faith. See this thread https://news.ycombinator.com/item?id=50181138 from yesterday, where egregious vulnerabilities in this project were pointed out, at which point the author force pushed to hide the history, failed to fix the vulnerabilities pointed out (e.g. https://github.com/Qbix/proxy/blob/5a9f8a96f4bdaaaa5444b6986... still doesn't match the PHP multipart parser, so the whole defense can be mooted by switching to a space as the delimiter). This is an exemplar of slop.

Comment by bigwheels 2 hours ago

Great idea! What do you think about implementing this in a safer language like Rust or Go?

Comment by EGreg 2 hours ago

Well, PHP is in C, and I know C much better — but certainly this proxy is so simple, an LLM or even a mechanical program could transpile it to Rust or Go.

Either way, memory leaks here aren’t too relevant.

I just posted about it here a half hour ago: https://news.ycombinator.com/item?id=50197861

Comment by dabinat 1 hour ago

> Either way, memory leaks here aren’t too relevant.

Why would memory leaks in a long-running service not be relevant?

Comment by redox99 2 hours ago

It'd be best to demo it on an unmodified WordPress docker image. Your example has carefully crafted php files on the server.

Comment by cute_boi 4 hours ago

PHP team has always been like this. Instead of creating better software they have always tried to add features with no consideration of backward compatibility.

No serious project should use PHP.

Comment by tredre3 2 hours ago

> they have always tried to add features with no consideration of backward compatibility

This could not be further from the truth. Backwards compatibility is a big reason why PHP doesn't have <your favorite missing feature that is totally a dealbreaker and PHP is thus unusable>. Odds are if you look at their RFCs and internals mailing list you'll likely find proposals and even implementations for any BC-causing feature you can think of.

Comment by jasonlotito 1 hour ago

It was an attempt. Chainguard releasing dramatically increased the attempts. Ignoring facts and timelines to suit an ignorant narrative doesn't help.

> I would never use PHP

That you would trust liars and people who are anti-security is telling.

Comment by DonHopkins 4 hours ago

Who remembers how careless, reckless, and blithe Rasmus Lerdorf was with the PHP 5.3.7 release he didn't bother to test, because running tests was too much of a hassle, because there were already so many test failures that wading through them all to see if there were any new ones was just too much to ask of him, the leader of the widely used project, in charge of cutting releases?

>5.3.7 upgrade warning: [22-Aug-2011] Due to unfortunate issues with 5.3.7 (see bug#55439) users should postpone upgrading until 5.3.8 is released (expected in a few days).

No seriously, he's literally as careless as he claims to be (when he says that repeatedly, you should believe him!), and his lack of giving a shit about things like tests and encryption and security that are extremely important has caused actual serious security problems, like breaking crypt() by checking in sloppy buggy code that would have caused a unit test to fail, but without bothering to run the unit tests (because so many of them failed anyway, so who cares??), and then MAKING A RELEASE of PHP 5.3.7 with, OF ALL THINGS, a broken untested crypt()!

http://i.imgur.com/cAvSr.jpg

Does anyone think that's just his sense of humor, a self deprecating joke, breaking then releasing crypt() without testing, that's funny in some context? What context would that be? Do they just laugh and shrug it off with "lol let Rasmus be Rasmus! ¯\_(ツ)_/¯ "?

https://www.reddit.com/r/programming/comments/jsudd/you_see_...

>r314434 (rasmus): Make static analyzers happy

>r315218 (stas): Unbreak crypt() (fix bug #55439) # If you want to remove static analyser messages, be my guest, but please run unit tests after

https://web.archive.org/web/20140818115440/http://svn.php.ne...

https://web.archive.org/web/20111014213651/https://plus.goog...

>Rasmus Lerdorf

>+Lorenz H.-S. We do. See http://gcov.php.net

>You can see the code coverage, test case failures, Valgrind reports and more for each branch.

>The crypt change did trigger a test to fail, we just went a bit too fast with the release and didn't notice the failure. This is mostly because we have too many test failures which is primarily caused by us adding tests for bug reports before actually fixing the bug. I still like the practice of adding test cases for bugs and then working towards making the tests pass, however for some of these non-critical bugs that are taking a while to change we should probably switch them to XFAIL (expected fail) so they don't clutter up the test failure output and thus making it harder to spot new failures like this crypt one.

Comment by hyperpape 7 hours ago

There's also a tweet by Matthew Prince that says Cloudflare saw thousands of attempts pre-announcement, https://x.com/eastdakota/status/2113079352682570051, then tens of thousands post announcement, and now millions.

Unfortunately, if you read the announcement as referring to 14:00 UTC, that's several hours after Chainguard published their images. I don't know if it's plausible that people saw the Chainguard builds, understood the issue from those and built exploits, but a precise timeline would help understand better.

Comment by sullivanmatt 6 hours ago

https://x.com/dok2001/status/2112811965735202970 was posted at 7:43 AM UTC on October 21, 2026, and Chainguard (according to the PHP Foundation blog post) shipped at 7:41 UTC. One of the big reasons Chainguard broke the embargo was because Cloudflare had been seeing attacks, IIRC.

Comment by 00deadbeef 7 hours ago

The Cloudflare one refers to the time period after the leak where people had already made PoCs. Amazon’s detection is days earlier which is quite concerning.

Comment by 6 hours ago

Comment by cubefox 7 hours ago

So either there was another (but non-public) leak or someone else discovered the exploit almost at the same time as it was about to be published. I guess the latter is much less likely because it would be a major coincidence.

Comment by post-it 6 hours ago

The attacker may have known about the exploit for a long time and were saving it, but upon finding out that it was about to be disclosed, they decided it was now or never.

Comment by cubefox 5 hours ago

How could he have known on Oct 17 that it was about to be disclosed? The dates only leaked on Oct 20.

Comment by stickfigure 5 hours ago

You're asking: How could a hacker - with an exploit that potentially allows him/her to break into half the computers on the internet - know some secret information that is stored on a computer on the internet?

It could also just have been a run-of-the-mill exploit in WordPress.

Comment by cubefox 4 hours ago

A leak is a much simpler explanation than an independent discovery who then also managed to discover before publication that someone else has also discovered it.

Comment by wahern 5 hours ago

With everybody using LLMs there are no coincidences. Popular LLM platforms are like Oracles that help you identify what other people have been working on. "Find a bug in PHP" might have been useless s few months ago, but once somebody used it to develop a full exploit the LLM will start to regurgitate that information.

Comment by qingcharles 2 hours ago

Here they are saying this PHP exploit was found by literally prompting "find bad vulnerabilities in PHP"

https://xxcancel.com/HackingLZ/status/2113025408975143080

Comment by fn-mote 5 hours ago

> once somebody used it to develop a full exploit the LLM will start to regurgitate that information

Sure about that?

Not my understanding of how the LLMs currently work. There’s a cutoff date and no new info after that date is included in training.

Info to the contrary welcome.

Comment by jonhohle 4 hours ago

Frontier models absolutely have access to current information. I just asked Claude about my fantasy football line up. It knew current injury reports, line ups and stats. Google search AI had even more information than that.

However, I’ve done some original research regarding an area that the internet has wrong and that doesn’t seem to have back propagated yet.

Comment by wat10000 1 hour ago

The models don't have access to new information. The harnesses do. All the major chat interfaces now provide the model with web search tools that they use to locate anything beyond their inherent knowledge (whether it's too new, or just something that didn't get baked in during training).

But if something hasn't been posted online publicly, they won't know about it.

Comment by disgruntledphd2 5 hours ago

> Not my understanding of how the LLMs currently work. There’s a cutoff date and no new info after that date is included in training.

You are almost certainly correct in terms of how the public models work. I guess it's possible that the labs have some tools connected for more recent stuff like this, but it seems unlikely.

Comment by dofm 4 hours ago

Yeah — I have log entries I believe show LLM-bot-driven exploit attempts — POSTs to a variety of non-application URLs, blocked on the front end proxy by rules I added that prevent multipart form POSTs -- from an hour before the patch was released.

And an odd little cluster of potential attempts it caught just after 9am GMT. Again I can't see any reason why I would be getting multipart form posts to those URLs, it's definitely an exploit, but it's possible there's another older exploit with that pattern. They aren't WP-type URLs though.

Comment by VladVladikoff 7 hours ago

I’ve seen probing for it in server logs but not actual exploiting of it. I think the botnets added it to their vulnerability scanners but I have yet to see any examples of an actual working exploit. There are hundreds of vibe coded attempts on github but none of them work to the best of my knowledge. Would love someone to prove me wrong here.

Also this post from Amazon is embarrassing. They knew about it from the 17th yet failed to meet the embargo timeline and patch Amazon Linux (their patch is due to be released today the 22nd of Oct).

Comment by athrowaway3z 30 minutes ago

2 hours before your comment a working PoC on github was posted that did the 'hard part' that a random AI prompt was unlikely to find.

https://github.com/9bie/php2shell-bypass-aslr-exp

Comment by onionisafruit 6 hours ago

I don’t know when various parts of Amazon learned about it, but I don’t think this post means they knew about it on Oct 17. I think they looked at past logs from their honeypots and found a probe for php2shell happened on Oct 17.

Comment by boplicity 6 hours ago

What makes you think they knew about it from the 17th? "...dating to Oct. 17, 2026" means that's when it happened. It doesn't mean they identified it on the 17th.

Comment by menomatter 6 hours ago

Do they differentiate between attempts and successful infiltration? I’m trying to understand whether this exploit is easy to use or not. There was an article here from yesterday that argued the exploit was not easy to use and needed LFI

Comment by tylergetsay 5 hours ago

I've been having trouble exploiting it from first principles even in lab conditions.

Comment by artnap 3 hours ago

[dead]

Comment by decidu0us9034 2 hours ago

"Exploitation attempt" lacks clarity. The bot scanners are constantly generating huge volumes of weird traffic. Does anyone have a portable, reliable exploit that works in real heterogeneous conditions and not a lab setup, and is that even feasible? doesn't seem like it. llms automate development of practical exploits but don't remove every constraint that makes them hard to begin with. it feels like people are missing this, idk

Comment by kyledrake 1 hour ago

I had a huge spike in PHP hacking attempts on the 20th, the day before the announcements. It was substantial enough that it was almost a DDoS attack and I had to mitigate it.

Curiously, all of the requests were coming from Cloudflare-controlled IP addresses (Workers? VPN? Internal server?). This of course means that these attempts would succeed for anyone that was whitelisting Cloudflare IPs but blocking others.

I tried to get this information over to them, but their abuse contact form does not allow you to submit unless you have a non-IP domain URL as evidence, which I didn't have because it's just source IPs. Suffice to say if you are behind Cloudflare and think whitelisting only their IPs is protecting you, think again.

Comment by ferngodfather 1 hour ago

> This of course means that these attempts would succeed for anyone that was whitelisting Cloudflare IPs but blocking others.

No, the ranges Cloudflare tells you to whitelist are only their proxy ranges. Workers and all the other junk comes from other ranges that won't be whitelisted.

Of course it's a different matter if you're just whitelisting their AS.

Comment by hannob 3 hours ago

I wish people who report "exploitation attempts" would be very clear about what they actually observed.

Something that would lead to an uncontrolled memory corruption, potentially leading to a crash, or something that indicates a real, code-executing exploit? Those two kinds events are of very different significance. I can't read from Amazon's announcement which one they think they saw.

Comment by Tomdarkness 6 hours ago

So Amazon knew about this exploit and that it was actively being used several days before the release of any patches. Despite this they still took 18 hours AFTER the release of the patches to provide updated version of their Amazon Linux packages?

https://github.com/amazonlinux/amazon-linux-2023/issues/1148

I don't think this blog post is sending the message AWS intended...

Comment by boplicity 6 hours ago

What makes you think they knew about it on the 17th? Scanning old logs is not the same as understanding them as they're generated.

Comment by Tomdarkness 6 hours ago

If this is only a retrospective find then they should make it more clear, from the blog it reads to me as promoting their MadPot tool for detection pre CVE disclosure:

"Subsequently, from 2026-10-21 0200 UTC to 2026-10-21 0800 UTC, Amazon Threat Intelligence identified additional exploitation attempts of PHP2Shell targeting MadPot from additional anonymized infrastructure." (i.e almost 12 hours before the release of the patch)

"Invest in detections that don’t rely on a signature for a specific CVE. The first exploitation will often come before those signatures exist."

"We analyze over 200 billion events per minute and integrate threat intelligence from our MadPot honeypot network to identify and block known exploitation attempts in real time. Our honeypot system provides us with early detection of exploitation attempts, enabling rapid response and threat analysis."

Comment by OtherShrezzing 5 hours ago

My reading here is that they've gone through their logs retrospectively, because you can see these attempts to exploit pretty clearly in an observability platform, and the 17th is only five days ago, so you can assume they've got a lot of fully detailed logs over that period.

Comment by paxy 4 hours ago

It's pretty telling that the attacks started after the vulnerability was found by researchers and PHP insiders notified. Who really believes that a malicious entity found the exact same exploit chain at the same time fully independently?

The PHP foundation needs to give a better answer for why they were asleep at the wheel for 2+ months while insiders were sharing/selling the info.

Comment by nicce 3 hours ago

Well, according to OpenAI:s blog post, they led everything, other than the fix and release. I guess leader takes the responsibility?

Comment by athrowaway3z 26 minutes ago

Maybe its one of their swarms running out of control again (eg HF) that scanned their chatlogs (eg Navier Stokes) and found the exploit.

Comment by makmanalp 6 hours ago

Take a peek at my detailed full-ish timeline of every actor around the embargo and leak, just added amazon's reporting in there as well:

https://news.ycombinator.com/item?id=50190603

Comment by system2 2 hours ago

PHP is dead to me. Countless WordPress and Magento hacks, countless lost customers because of these issues. I am done, and I will spread the hate with passion. I wish I had never learnt this garbage and spent more time with Python and C.

Comment by 4 hours ago

Comment by bzmrgonz 6 hours ago

What did crowdsec see? Anybody know? Php exploits should be on a fast track of alerting, given the wide installation base.

Comment by mazzma 5 hours ago

Crowdsec spotted more than 300 IPs listed in 24h and the first one were spotted prior the patch release. source : https://www.reddit.com/r/CrowdSec/comments/1xci24u/php2shell...

Comment by bcvvvy 7 hours ago

Why haven’t they fixed Amazon Linux then?

Same as none of their AMIs for bottlerocket have emSign revocations.

Absolutely shit company who puts their customers at risk.

Comment by monksy 2 hours ago

High chance they've demotivated their devs who normally pick up this stuff quickly or laid them off.

Comment by bcvvvy 1 hour ago

Probably waiting for their AI credits to get reset.

Comment by chimeracoder 4 hours ago

> Why haven’t they fixed Amazon Linux then? Same as none of their AMIs for bottlerocket have emSign revocations.

Last I checked, Amazon has also not patched their Lightsail blueprints for WordPress, or even communicated about the vulnerability with those customers at all, which is arguably way worse, because

(a) by design, all-in-one blueprints aren't really intended to use system PHP or have system PHP updated in a meaningful way; and

(b) hosting static sites such as WordPress is 90% of the selling point of Lightsail as a product

Comment by bcvvvy 4 hours ago

That's pretty awful. I do wonder if they've bitten off more than they can chew with all the ancillary stuff to their core IaaS platform. That seems to be the only well maintained bit of their estate, well EC2/VPC/S3/R53 etc.

Comment by leverlark 3 hours ago

[flagged]

Comment by asah 6 hours ago

-1: if the fix breaks customers in other ways, they're screwed as well. Damned if you do, damned if you don't...

Comment by Tomdarkness 6 hours ago

It's a upstream targeted fix that was made available significantly faster in packages/images maintained by volunteers, it's not some untested downstream patch. Customers at this point would have likely taken their applications offline over this issue so it would have been difficult to make the situation any worse.

Comment by otterley 5 hours ago

Given the sheer number of PHP applications out there, it's impossible to conclude with certainty that this targeted fix will have no regressive impact without significant testing. How much testing needs to be done is debatable, but I'm pretty sure the answer isn't zero.

Comment by AnthonyMouse 4 hours ago

Someone with a single gigabit internet connection can scan the entire IPv4 address space in less than a day. When you have public RCE vulnerabilities like this, your options are "apply the patch", "take the system offline until you can apply the patch" or "get pwned".

Comment by otterley 4 hours ago

The list of options is larger than that. There are mitigations that can be quickly applied. See, e.g., https://ma.ttias.be/php-cve-2026-109618-early-mitigation/ and https://aws.amazon.com/blogs/security/using-aws-waf-to-apply...

Meanwhile, WAFs are being continuously updated with detection signatures.

Comment by AnthonyMouse 4 hours ago

Mitigations and patches are in the same category, so now you're applying mitigations without testing. What if your new WAF detection signature has false positives etc.?

Comment by otterley 4 hours ago

The difference is that a mitigation is customer-applied, not vendor-supplied. Like it or not, blame shifts when a vendor supplies a fix that's insufficiently tested. The CrowdStrike BSOD incident from 2024 comes to mind.

As for WAF rules, those can usually be flipped on and off with a single switch.

Comment by AnthonyMouse 4 hours ago

Being able to roll it back isn't really a difference, you can revert to the previous version of a package too.

The trouble is if the WAF change e.g. messes up a redirect or turns a success response into an error response or vice versa, which confuses some other system into giving multiple users the same session cookie and gives them access to one another's accounts or writes corrupt entries to the database or whatever it is. You can revert the rule but that doesn't revert the state changes that have already happened.

There is inherently a trade off between keeping the system online, keeping the system patched and testing the patch before applying it.

Comment by otterley 4 hours ago

> Being able to roll it back isn't really a difference, you can revert to the previous version of a package too.

Sometimes. In a world in which customers are increasingly adopting immutable infrastructure patterns and (contrary to best practice) using Launch Templates with symbolic references to `latest` versions of AMIs, this isn't always easy.

> There is inherently a trade off between keeping the system online, keeping the system patched and testing the patch before applying it.

On this we agree 100%.

Comment by bcvvvy 6 hours ago

Nearly every other vendor has pushed fixes for it, including where they pull their upstream stuff from. There's no excuse.

Comment by otterley 5 hours ago

I don't think you're being understanding or empathetic here.

Any vendor that delivers updated packages without thorough regression testing that ends up causing an incident for their customers as a result will get a thrashing from the community. Remember the CrowdStrike BSOD incident in 2024?

Maybe Ubuntu and others are cutting corners to deliver more quickly. It's their risk to take, but it's still a risk.

Comment by bcvvvy 5 hours ago

Every vendor is winging it there. Even RHEL. Ubuntu ships shit half the time as well even on LTS.

If you need to test it you should pin packages.

Comment by otterley 4 hours ago

> If you need to test it you should pin packages.

Unfortunately people rarely follow best practices.

Customers frequently do risky things like using launch templates that refer to the latest AMI. AMI changes, Auto Scaling launches a new/replacement instance in the fleet without adequate testing, *boom*.

Comment by cookiengineer 3 hours ago

Shameless self promo for all the naysayers that are deflecting: https://news.ycombinator.com/item?id=50195017

Comment by 6 hours ago

Comment by otterley 5 hours ago

[dead]

Comment by 8 hours ago

Comment by ActionHank 6 hours ago

All these tech companies are school children comparing how much better their bike is than the others, but they've all got the exact same bike, and that bike is a massive vulnerability that they did nothing to fix, but they're all bragging about how they saw it first or identified it first.

To the person at amazon that thought this was a good press release to put out, I know this is what you were hoping for, so here it is, "wow, so cool". Not sure why, but AI really has everyone running around with desperate loser aura.

Maybe next time, do better, actually patch some things so your customers are covered?

Comment by ActionHank 6 hours ago

[flagged]