Rust-lang/rust is adopting an LLM policy
Posted by afdbcreid 2 days ago
Comments
Comment by matheusmoreira 1 day ago
https://forge.rust-lang.org/policies/llm-usage.html
https://rustc-dev-guide.rust-lang.org/llm-guidance/writing.h...
> Pre-arranged, non-critical, high-quality, well-tested, and well-reviewed code changes that are originally created by an LLM are allowed, with disclosure.
They clearly put a lot of thought into this and it shows. Huge respect.
> New contributors cannot create a PR using an LLM unless they first talk with a reviewer.
New contributors do not have community connections. I don't know any rust-lang/rust reviewer.
What is the correct procedure for this?
> Harassment is not allowed.
> You may not harass people for using an LLM, regardless of whether or not their use is banned by the policy.
Refreshing take. Not a fan of heavy handed moderation, but I'm still very happy to see that they have this stance given the racism-tier insults I've gotten on forums without it.
I especially like this rule:
> Don’t try to be the police for whether someone has used an LLM. You are not required to “actively look” for whether an LLM was involved.
Comment by jynelson 1 day ago
> What is the correct procedure for this?
there's a #llm-mentoring [0] Zulip stream linked in the dev-guide [1].
[0]: https://rust-lang.zulipchat.com/join/rlfvpemsaacs3pfi6kwqnqj... [1]: https://rustc-dev-guide.rust-lang.org/llm-guidance/writing.h...
Comment by matheusmoreira 1 day ago
Comment by jynelson 1 day ago
Comment by pas 1 day ago
in general to work on something that will get "into the rust compiler" it needs at least an RFC and then a Major Change Proposal [0]
in practice it starts with asking on zulip [1]
[0] https://forge.rust-lang.org/compiler/proposals-and-stabiliza...
[1] https://forge.rust-lang.org/how-to-start-contributing.html
Comment by panstromek 1 day ago
Comment by Cthulhu_ 1 day ago
Get involved with the project before submitting PRs, participate in discussions, etc. But if it's a change you want to contribute, don't use an LLM to write it.
Comment by matheusmoreira 1 day ago
The rules I quoted say LLM code generation is allowed should the developer contact a reviewer first and they agree to review it.
My question was about the exact procedure and etiquette for contacting these reviewers. For example, Linux has a get_maintainer.pl people can use to figure out who to email should they want to contribute something.
Comment by minimaxir 2 days ago
This policy I like.
Comment by feverzsj 2 days ago
Comment by LoganDark 2 days ago
There's a shameboard that curates a list of projects that don't explicitly and unconditionally ban all use of generative AI. Their term for projects that have a policy other than a total ban is "permissive" and it is treated as a sign to avoid that project/community. It's such a mess. Cancel culture is never the answer. I believe it is the right move to ban this completely unproductive and toxic kind of behavior.
Comment by duskdozer 1 day ago
Comment by matheusmoreira 1 day ago
Banishment is just enforcement of boundaries. It gets rid of a problematic person in a specific space where their behavior is not tolerated. It's about the community, not the banished. Once the troublemaker is gone, that's the end of it. Everyone goes back to their normal activities.
Cancel culture consists of people going out of their way to get a person banned everywhere else and even fired by their employers. People launch entire campaigns against the problematic person and make a concerted effort to ruin their life. Not that different from the Kiwi Farms.
Assertiveness and aggression.
Comment by pseudalopex 1 day ago
Comment by LoganDark 1 day ago
What I'm talking about is the practice of following people around for using AI, rooting around for evidence that a project doesn't fully ban AI, etc. and publicly shaming them, harassing them, attacking them. People who do this don't accept neutrality or even mere skepticism, they demand full unconditional bans or else they make themselves a huge nuisance and try to harm you. That's cancel culture, plain and simple.
Comment by imtringued 1 day ago
Comment by zahlman 1 day ago
Comment by matheusmoreira 1 day ago
Comment by andsoitis 2 days ago
Comment by seanhunter 2 days ago
1) Code is the least interesting bit in some ways - they’re trying to build not only rust but a community of experts around rust which is able to maintain and improve it in the long term and the authors of prs need to understand the code deeply and show a willingness to be part of that long run. Llm-generated code even if it looks polished doesn’t indicate a human author who deeply understands it and is willing to be part of that long term maintenance.
2) They don’t have (and don’t want to enforce) a community consensus that llms are good or llms are bad. They just want some rules written down so everyone knows where they stand and people of either camp (and none) can get on with it.
Seems a pretty reasonable approach to me although I don’t really use rust much so don’t have a vested interest.
Comment by nirbendavid 1 day ago
Comment by incrudible 2 days ago
Comment by sensanaty 1 day ago
From my own work experience, the "dinosaurs" are the most valuable people in the whole org, because oftentimes understanding the code isn't nearly sufficient to make good long term choices. You very often need to know the historical context of why certain things were made in order to not shoot blindly in the dark with random changes.
Comment by seanhunter 1 day ago
Outside the world of foss there are for example multiple games that are highly respected that were all entirely or largely developed by a single developer. Rimworld, Songs of Syx, Dwarf Fortress, Ostranauts and many others. In a proprietary context I have myself deeply understood codebases and worked with others who have also.
That being said, you are arguing against a straw man that neither I nor the rust article propose. They are not saying maintainers need to understand all the code they are saying the person submitting a pr needs to deeply understand that pr. Why? Because they want to build a community who deeply understand various aspects of rust so when things need maintenance there are experts around who even if they don’t know a particular piece are invested in the success of the project and know enough to get up the curve and fix whatever needs to be fixed.
Comment by bryanlarsen 1 day ago
Fabrice Bellard is obviously a superhuman, but I expect even he relies on re-analyzing code he previously wrote and has subsequently forgot. This doesn't take him long because he knows his own conventions and proclivities. But I'm confident he doesn't "deeply understand" the entire code base. He has a shallow understanding of the entire code base. He has a deep understanding of crucial parts of it. And he has the ability to quickly re-establish a deep understanding of any piece.
Comment by NobodyNada 1 day ago
Bellard hasn't worked on FFMpeg since 2003.
Comment by zahlman 1 day ago
Comment by incrudible 1 day ago
Now, if you have program that is highly complex, written by a single author who is not you, hinging on the ongoing goodwill and capability of that single author, that is a liability you do not want to have, operationally. Again, that is just how it is.
Comment by duskdozer 1 day ago
>Pre-arranged, non-critical, high-quality, well-tested, and well-reviewed code changes that are originally created by an LLM are allowed, with disclosure.
Comment by matheusmoreira 1 day ago
> Experiment: LLM-created code changes intended for review
> We leave space open to experiment with LLMs to inform future policies.
> This experiment is meant to inform future non-experimental policy, not to serve as the perpetual LLM usage policy.
Comment by ares623 2 days ago
This closes the gap. A key insight. A load bearing difference.
Now, is writing code considered tedium, for a project like Rust?
Comment by thunky 1 day ago
But IME all of the other things are the hard part.
So basically it's fine to get the LLM to think for you and give you the answers as long as you understand Rust syntax well enough to type it all out yourself?
This also seems like it will produce worse code, because it's preventing the LLM from doing the easy/boring part that it's good at.
Comment by cbarnes99 1 day ago
Comment by socalgal2 2 days ago
AI is a jet engine - Banning is for creation is like saying "we require you to go by ship/car/propeller-plane"
Yes, a policy that says AI slop PRs will not be tolerated. But I've had AI track down real hard to find bugs in several projects and provide correct fixes. I've also had AI build new features and new products. The project that ignores this acceleration will eventually fall behind the ones that embrace it.
Comment by Cthulhu_ 1 day ago
Comment by happyweasel 2 days ago
and rust just wants to ensure you are a capable pilot that can actually steer a plane before stealing everyone else's attention
Comment by imtringued 1 day ago
Acceleration makes sense for end user applications, where you're allowed to break whatever you see fit, but I don't see how it helps with core infrastructure like a programming language, where every mistake is multiplied a million fold.
Comment by Cthulhu_ 1 day ago
For projects like Rust, coding velocity is not the bottleneck, speed / productivity is not a priority.
Comment by ares623 1 day ago
Comment by sensanaty 21 hours ago
Comment by weli 1 day ago
Comment by matheusmoreira 1 day ago
However, we should be polite and respect their house rules. They are being reasonable and even made an effort to preempt these exact objections:
> Harassment is not allowed.
> You may not harass people for using an LLM, regardless of whether or not their use is banned by the policy.
> Don’t try to be the police for whether someone has used an LLM. You are not required to “actively look” for whether an LLM was involved.
Comment by ChrisRR 1 day ago
I've seen so many arguments (mainly from non-developers) claiming that anyone who uses an llm to generate code doesn't know how to program. They can't understand the difference between people using an LLM to blindly churn out code, and people developing with an LLM to generate the boring bits
Comment by minimaxir 1 day ago
Comment by matheusmoreira 1 day ago
Yet another thing I won't be able to publish I guess. Sigh.
Comment by m4xp 1 day ago
Comment by matheusmoreira 1 day ago
Comment by IsTom 1 day ago
Comment by orphea 1 day ago
Are you required to tell you used Anthropic™ (or whichever your AI provider of the month is) to generate this code, and not only that *an LLM* was involved?
Comment by matheusmoreira 1 day ago
Generated-by: Claude Fable 5Comment by pseudalopex 1 day ago
Comment by winstonwinston 1 day ago
Comment by weli 1 day ago
"This looks like AI, closed". And you as a new IC with no connections, no community visibility have no recourse to fight back against it. Anti-ai people love boasting about humans > systems and forget about those humans that will surely be negatively affected by this, even if low number.
If you are a new programmer who is just learning and spend time creating a PR for a project you have a lot of respect for and the first interaction is someone falsely accuses you of using LLM's then closes your PR you've now removed someone who could have genuinely grown into your community from ever joining.
Not only that but since now you are by default a suspect of having used an LLM (and you need to prove you didn't use one) why even bother not using one? You are going to get accused of it either way.
Comment by jynelson 1 day ago
Comment by weli 1 day ago
Comment by bendmorris 1 day ago
Are there some potential contributors who might be discouraged because of this policy? Probably. Will maintainers have less time wasted by obvious low effort changes and engagement? Also probably. A user of AI is, in the experience of many maintainers, way more likely to be lazy than engaged.
Comment by DarkNova6 1 day ago
We have focused so much on technological advancement, but our understanding of human systems and building an environment of trust that is not eroded by perverse incentives atrophies.
Comment by xboxnolifes 1 day ago
Comment by nasso_dev 1 day ago
as you probably have seen in TFA, the policy does not ban LLM usage. its just asking for disclosure so that maintainers know how much effort was put in a PR
and, please, open-source maintainers owe us nothing. refusing to merge a patch is not the same as cops searching your car. this is merely maintainers trying to evaluate how they should spend their precious time
Comment by trelane 1 day ago
Your assertion here is irrelevant to the poster's point.
You're focused on the "happy path:" everyone has perfect information and judges correctly. If the submitter used AI, they say so and the commenters on the PR agree. If the submitter did not use AI, they don't say they used AI, and the commenters on the PR agree. This is the diagonal elements on the "used AI" confusion matrix.
The commenter is pointing out that information and judgment are not perfect and they're concerned about the off-diagonal elements of the confusion matrix. Especially the "Didn't use AI, but some commenters believe that they did" element.
Comment by matheusmoreira 1 day ago
There is stigma surrounding LLM usage. There are consequences. Disclosing can lead to harassment and direct harm to one's reputation. I've been called "slop fetishist" and "clanker lover" before.
Comment by jynelson 1 day ago
Comment by matheusmoreira 1 day ago
Comment by ChrisRR 1 day ago
It's just so tiresome
Comment by luciana1u 1 day ago
Comment by piokoch 1 day ago
Comment by duskdozer 1 day ago
Comment by lukasco 1 day ago
The future: SWEs will no longer write code by hand. The era of hand coding is over. We only saw that for sure in the last 6 - 9 months. But it's over.
Many coders have not yet made this transition, true, but it's happening regardless. You may not like it, but as a coder, you will not be hired if you refuse these tools. You will look as ridiculous as an accountant that refuses to use spreadsheets.
For now, we're not there yet.
As a profession, we are learning how to deal with the massive influx of review work that occurs because of LLMs. The bottleneck has moved. And as of now, there aren't good answers. But there will be. We will figure it out, like we figured out CI/CD and agile.
So personally, I'd focus my policies much more on dealing with the issues that this new era presents, and how to make LLM contributions work, and I'd avoid banning LLMs.
Comment by cbarnes99 1 day ago
Comment by lukasco 1 day ago
And later:
"There are very strict guidelines on LLM-generated code changes:
Pre-arranged, non-critical, high-quality, well-tested, and well-reviewed code changes that are originally created by an LLM are allowed, with disclosure."
That's a pretty limited wedge of allowed cases in my opinion.
Comment by satvikpendem 1 day ago
Comment by Cthulhu_ 1 day ago