Disrupting supply chain attacks on NPM and GitHub Actions

Posted by nyku 5 days ago

Comments

Comment by alpineman 5 days ago

>> high-impact npm accounts are now put into a read-only mode for 72 hours when they change their email or use a 2FA recovery code. This delay allows maintainers time to respond and recover the account before their account can be used to start an attack.

'what time shall we put here?'

'what's the longest hangover you ever had?'

'let's put 72 hours'

Comment by Waterluvian 5 days ago

A weekend plus a day to deal with things seems like a decent minimum duration when it’s somewhat arbitrary what the right number is.

Comment by datakan 5 days ago

I don't think so at all. People go on vacation all the time. This should be 30 days not 3.

Comment by Waterluvian 5 days ago

That's the usual runaway problem, right? Why not 60 then? People go on sabbaticals! etc etc.

I feel like the only way to be wrong for this class of problem is to believe that there's a singular right answer. Just pick something reasonable (like how weekends are a fairly common thing, so don't make it shorter than 48 hours). Start there, then see how much of an issue persists. No matter what, at scale you'll find someone complaining that the number is too little, and people complaining that it's too much. Eventually you just have to tell the complainers to deal with it.

Comment by datakan 5 days ago

There is a big difference between 3 days when someone may be sick or traveling and not looking and 30 which aligns better with 99% of scenarios this may be an issue with. 3 days is absurd. Even password apps with kill switches will default to 7 days.

Cyber attacks will frequently happen on a Friday just for this scenario, they anticipate no ones looking over the weekend.

Comment by tredre3 5 days ago

And 365 days probably aligns with 100% of scenarios. It doesn't mean it's a good idea, because the longer the delay is the more downsides you also get.

Blocking a maintainer from pushing updates for a full month after changing their e-mail would be obviously absurd. So Github would have to add a way of enabling maintainers to super-uper-duper confirm the change and cut the delay short. Which is likely not currently possible, hence starting with a shorter delay.

Comment by Normal_gaussian 5 days ago

It sounds like we should plot 'count of compromised repos' against 'time taken for maintainer to initiate a resolution', and pick something just after the inflection point.

Comment by normie3000 5 days ago

It's often not the maintainer that removes a poisoned release, it's the npm security team/tools. So unless the whole team takes a month off, we're all safe.

Comment by lrvick 4 days ago

I paid $8 for the recently expired email domain of the sole author of NPM package "foreach", so then control to ship any code I wanted to 70k companies was just a support ticket or password reset away. 72 hours would not make a difference here.

Talked to NPM about this when it went viral, and once again all they could say was enabling package signing, even optionally, would discourage inexperienced people from contributing packages as they would be pressured to learn basic security and key management.

Sorry, if you are unwilling to take 10 minutes to learn how to sign your code, you have no business maintaining software releases millions of people depend on. Full stop.

But NPM will not even make it -optional- and rejects all PRs to implement this going back a decade. So no one feels pressured into basic security. Cool.

Comment by e40 4 days ago

> Sorry, if you are unwilling to take 10 minutes to learn how to sign your code, you have no business maintaining software releases millions of people depend on. Full stop.

The crux of the problem is right here.

Comment by 4 days ago

Comment by inigyou 4 days ago

Who chose to depend on "foreach"? Millions of ill-advised people did. The author did not choose that.

Comment by lrvick 3 days ago

If an author is not willing to do responsible release engineering they should lose their publishing ability and have their project marked as unmaintained. This would force people to migrate to better maintained code, or find responsible volunteer maintainers to maintain it. Everyone wins.

Even when you drop off food at a food pantry for consumption by strangers there is a basic expectation it is not spoiled or dangerous.

Comment by summarybot 5 days ago

Github owns NPM. Github has unprecedented access to code analysis tools. Github can run static analysis on nearly everything. Introducing a cooldown period seems like the lowest tech solution to a technological problem I have seen in a longwhile.

Comment by insanitybit 5 days ago

Cooldowns have the benefit of being straightforward. It doesn't rely on updating rules, scanning, keeping up with obfuscation techniques, etc. It just stops installs for a period of time after publish.

I think that cooldowns are mostly dumb but they're very cheap to deploy and they do give other approaches (like scanners) more time to adapt/ do their job.

Comment by summarybot 4 days ago

In my opinion, things like scanners should not need to exist if npm's parent company, GitHub, were to take responsibility for what was published to npm. If Walmart sells you an item that ends up being a handgrenade there would be hell to pay. In this example, Walmart cannot simply say that Walmart is not in charge of what goes into the item and therefore "buyer beware." Yet, this is precisely what GitHub does with every npm package.

One reasonable approach is to increase the scrutiny for new releases based on how many active downloads of the package there are. So for new packages, maybe there will be not a ton of scrutiny, but for things that are downloaded by millions of users, like Axios, there would be significantly more scrutiny per release.

The epidemic of "not my job" and "[we] don't get paid to do that" is a large contributor to why the world is so messed up. I don't know what to call this phenomenon. In the Hawaiian language there is a word Kuleana which means "sacred responsibility" or "sacred task." If more people took on their Kuleana joyfully we would be inviting the ideal [realm] more rapidly.

Comment by insanitybit 4 days ago

Walmart and Github are so different. Walmart doesn't let anyone come in and set up a booth to sell things, Github does. Thinking that Github should take responsibility for what's hosted on it is... well, radical, to say the least.

What you're suggesting is a radical departure from how free software has been distributed historically. That goes well beyond Github and NPM.

Comment by doubled112 4 days ago

Would the metaphor have worked better if it was Amazon?

Walmart does allow other stores to sell through their website.

Comment by insanitybit 4 days ago

I would just drop the metaphor entirely since I don't think they're helpful. If you want to advocate for liability, I'd do so directly

Comment by summarybot 4 days ago

You say "free software" but npm is no longer a small open-source project with a federated rulership, it is owned and operated by a for-profit technological giant. There was a radical departure already, when Github acquired npm. It sounds like your stance is "if something is given away for free, consumer protections are not necessary." But if I get poisoned by free samples given away at a grocery store, who is on the hook? Buyer beware? The onus is on the organization that distributes the food, in the case of the grocery store. It is unreasonable for every person to become a food chemist to be able to ascertain the safety of every single food product. Rather, we as a society agree to put that onus on the distribution. This is no different.

Comment by insanitybit 4 days ago

This isn't about npm, it's about who publishes to npm. This isn't a grocery story.

Comment by summarybot 3 days ago

I don't appreciate a pedantic interpretation of the analogy; consider any marketplace where the sellers are not the platform: Amazon, FB Marketplace, Etsy, eBay, and so forth. When a consumer gets burned, who gets in trouble?

Comment by insanitybit 2 days ago

I don't appreciate analogies at all. If you want to make a point about Github, talk about Github.

Comment by summarybot 55 minutes ago

Already did.

Github owns and controls NPM, npm is no longer a community-controlled project. Therefore, Github should [happily] take on the duties, responsibilities, and obligations of a software publisher to provide either: trust ratings on packages and code, or literally and not figuratively scan every single package-and-version on npm for appropriateness.

But obviously you already got that point and just wanted to exert some modicum of bossiness and control by having me regurgitate my well-formulated thoughts for a third time while painting inside the lines you insist are relevant.

Comment by inigyou 4 days ago

You're suggesting that only stuff billionaires like should be allowed on GitHub or npm. No Monero libraries, for example, or YouTube downloaders.

Comment by lazyasciiart 5 days ago

It's certainly the lowest tech piece of their announcements over the last day, as far as I can tell.

https://github.blog/changelog/2026-07-28-npm-publish-time-ma...

Comment by lrvick 4 days ago

That is because Github is not a thing anymore. We are talking about Microsoft.

Comment by zzo38computer 4 days ago

My only use of GitHub Actions specifies permissions that only allow it to write to the issue tracker, so if the program it executes (which is the "gh" program) is malicious then the only damage it can cause is to the issue tracker and not to the repository itself (unless GitHub Actions itself is compromised). Furthermore, there are no private files (I do not host private files on remote services such as GitHub), so they can't steal my files either.

I think one thing that would help is to support mutual TLS, with X.509 certificate chains for authentication. This is more secure than personal access tokens and allows you to issue them to yourself and others without needing to login first, including to specify whatever permissions you want to set (as a subset of the permissions of the issuer) and to make them expire. (You could also store the private keys of one certificate on a separate computer not connected to the internet, and use that to issue another one to yourself to use that one instead, and can use a passworded private key as well (an alternative of 2FA), so you can recover even if your certificate you are using does become compromised, or something deletes it (intentionally or not), and can revoke the ones that are compromised.)

Mutual TLS (and even ordinary TLS) won't solve everything, but in combination with other things, including ability to restrict access to what is needed, and avoiding needing too many dependencies (since, programs that need too many dependencies is also one thing that can cause many problems), and many other things, it can be an additional step to help with security.

Comment by acdha 4 days ago

> I think one thing that would help is to support mutual TLS, with X.509 certificate chains for authentication. This is more secure than personal access tokens

This adds an large amount of overhead and complexity (i.e. you now need to manage a revocation list) but you didn’t really explain any benefit which isn’t already present except for the possibility of restricting access to the private key. That can have some benefits — e.g. it’d be nice if you could say that releases can only be signed by a particular key stored on a Yubikey with a physical presence check – but I think most of the benefits come not from using x.509 but rather from fine-grained permissions.

Most of the attacks we’re seeing are escalation from an initial minor compromise where an attacker compromised something which had far greater permissions than needed for the task the attacker first exploited, as well as the broken design of GitHub Actions with mutable tags. I don’t think TLS gives us enough to take effort away from working on those.

Comment by zzo38computer 3 days ago

I agree that benefits come from fine-grained permissions; however, for the fine permissions of user authentication (and API authentication), I think it would help to use X.509 certificate chains. The certificate can include a extension for specifying the permissions that it grants (a permission will only be granted if all certificates in the chain grant that permission). This avoids needing to login or use 2FA in order to set up new API keys or access tokens, and prevents problems with misauthentication with the wrong server (personal access tokens don't help much compared with passwords, but X.509 does help). Merely using X.509 won't cause it to have fine-grained permissions, but they can be combined in the way that I described. (This use of X.509 can also be used to allow someone else to operate on your behalf with limited permissions; in this case, a client would issue a certificate to a server, and that server would then also be a client to GitHub.)

If there is a broken design of GitHub Actions, that is another issue, and neither TLS nor X.509 is the issue with that. (Also, GitHub Actions could hopefully be made to allow finer permissions if the existing permissions are not fine enough; that is not related to X.509 at all.)

Avoiding too many dependencies, is another thing to do.

And, signed releases is another thing to do, and is helpful for additional reasons (unrelated to authentication with GitHub, but useful for knowing that you published it rather than someone else (without having to trust GitHub with it)). The key used to sign the release might or might not match that in the X.509 certificate used to authenticate with TLS; there are some benefits if it does (such as if you have published your certificates that someone else can check, or your mention of Yubikey), but it doesn't have to (e.g. because you have multiple certificates, because multiple people are publishing releases, because you have some other reason to not use the same certificate for the other purpose, etc).

Comment by skipants 5 days ago

One thing I don't quite get is how trusted publishing is supposed to be more secure. It still allows publishing if they pwn your workflow.

Is it purely more secure because they can't exfiltrate your secret keys to publish again?

I feel like if your workflow gets pwned you'd be rotating your keys anyways, so I'm not sure if the vendor lock-in is worth it.

Comment by pimterry 5 days ago

Trusted staged publishing helps a lot: you have to independently pwn the workflow _and_ then complete a separate 2FA flow as a maintainer. The workflow never sees any keys that can publish independently.

Comment by flyingshelf 5 days ago

Keys can be reused from anywhere. Trusted publishing means the attacker must trigger the specific workflow on GitHub, which is more difficult and leaves trace of actions on GitHub itself.

Comment by insanitybit 5 days ago

Trusted Publishing pins the publish to the workflow. The next step is to isolate "run untrusted code (`npm build`)" from "publish" in your workflows. Now you have two properties - the publish can only happen from one place and the publish is isolated from untrusted code execution.

Comment by 5 days ago

Comment by MSkill1 5 days ago

I literally am reading this after setting GitHub actions five minutes ago to generate the APK version of my cross platform project for the first time. Great :\

Comment by Rumudiez 4 days ago

[dead]

Comment by hncsiocp9x 5 days ago

Been quietly thinking this for years

Comment by pluto_modadic 5 days ago

the bare minimum award, for the only language and only registry where this regularly happens.

Comment by hinkley 5 days ago

At this rate I'm going to have "npm is by far the worst thing about NodeJS" printed on my gravestone.

Comment by TacticalCoder 4 days ago

> "npm is by far the worst thing about NodeJS"

I'll come and add: "after JavaScript".

Comment by hinkley 1 day ago

That's the cover charge. It's like showing up at a karaoke bar and then complaining about all the drunken covers being terrible.

Comment by lrvick 4 days ago

It is fascinating how far people will go to do anything else at all other than having authors cryptographically sign their packages like every Linux package manager that matters since 1996.

Linux package managers like debian/apt also ship hundreds of signed NPM packages. In other words the unpaid Linux community has once again solved a problem Microsoft fundamentally failed to. Some things never change.

Comment by acdha 4 days ago

What keeps Linux distributions safer is that fewer people are able to push updates and there’s a time delay. The compromises are because people exploit release pipelines and if your build infrastructure is compromised it’ll merrily sign the compromised packages as well.

Comment by lrvick 3 days ago

Which is why automated signatures in release pipelines are usually meaningless. Authors must sign their code on their own hardware before it is published so we can tell when the signing key changes and pay extra special attention to that, and maybe a longer delay.

Comment by effnorwood 4 days ago

Hey Siri, what is a local repo?

Comment by torment-nexus 5 days ago

[flagged]

Comment by receiptincar 4 days ago

[dead]

Comment by DiabloD3 5 days ago

[flagged]

Comment by theF00l 5 days ago

The job can require you to

Comment by DiabloD3 5 days ago

[flagged]

Comment by UqWBcuFx6NV4r 5 days ago

These comments should be removed r from HN, because they in no way actually add to the conversation. They aren’t intelligent, they aren’t insightful, they aren’t actionable, and they don’t invite a genuine reply. All you’re saying is that you happen to not use these technologies yourself – something that I’m sure is only by happenstance – and that you feel superior for it.

This is a blog post by GitHub. what are you suggesting that these employees do? Simply ignore that they exist? Regardless of whether or not you use them, they still exist.

Comment by DiabloD3 5 days ago

It very much is actionable.

You might not know this, but the world of software development existed before NPM and Github, and it continues to exist after them. You are not beholden to either of them.

Comment by anon48293 5 days ago

> This is a blog post by GitHub. what are you suggesting that these employees do? Simply ignore that they exist?