AI Has No Wisdom and Neither Will You

One interesting comparison is to the history of manufacturing. West/America decided one day that manufacturing would be cheaper to outsource and better (short term) profit was to be made by outsourcing it all to China. The institutional expertise started to deteriorate, to the point that America simply didn't even have the capacity, or expertise anymore to produce stuff (such as grill brush [1])

I feel like you could take all the handwavy comment that are made today to dismiss this caution, and find equal dismissal back then when companies were actively outsourcing the manufacturing.

"I'm coding 10x faster" "look at the output velocity per employee"

"we are producing much more (in China)" "look at profit / number of (manufacturing) employers"

Seems ok if you're American / Chinese but I'm struggling to understand how the rest can be OK with allowing institutional knowledge to deteriorate while having an active dependency to the former two. We already see this with the tech dependency towards USA and manufacturing competition from China.

[1] https://youtu.be/3ZTGwcHQfLY

"Code maintainability and good architecture don’t have good measurements that we can apply"

Who has no wisdom? There are dozens of ways to measure code maintainability. Cyclomatic complexity is just one.

Nothing stops you from wiring up something like SonarQube metrics to your agentic coding workflow.

The first version was built in about two weeks of part time work. Then I started exploring. I learned relational algebra, researched almost every kind of database, reworked the internals, built a small relational algebra layer, a query planner, and an executor, covering everything from the backend storage to the query language. I learned more in those two months than in the previous 20 years.

Did I care what code the agents wrote? No. I read zero lines of generated code. What I cared about was correctness, verified through tests, and the high-level product features. For the first time in my career, I acted as a senior product manager, steering the project along the right roadmap. Without AI, I wouldn't have been able to do that.

When you have superpowers in your hands, you don't need to worry about the laundry. For the first time in my career, I can produce code in C, C++, Java, .NET, or any other language. Sometimes it takes me longer than a senior developer in that language, but does that really matter? Absolutely not. Writing documentation and code by hand in 2026 is like driving a horse and buggy. It doesn't matter how skilled you are with the reins; you'll never compete with a car. My hobby db project isnt opened source yet.

There's so much money in it right now. There's such a momentum. There are zero incentives to slow down for those that are in charge.

I've accepted that in 5-10 years, the vast majority of human devs. and engineers will not touch a single line of code. It'll be small increments, with a couple of big ones here and there.

And there will not be any triumph for those that hold steadfast to the principle of human coding. They'll be tiny boutique shops that do custom stuff, in the same way cobblers are to the mega shoe factories.

I'm going to make my own prediction: this isn't going to happen

I’m sure there are companies who are writing “perfectly maintainable and highly scalable code”. However, for half of my career I’ve been brought into startups to clean up the mess created by engineering teams.

While AI may create an unmaintainable mess (I’m not totally convinced), from my perspective many (not all) engineering teams have been doing that all along. #v2 #refactor

Unless you steer and understand what an LLM will produce, you will end up with something that possible ”works” that has no future plans baked in. Suno generated music has a very unpleasant feeling of sounding like competent music with nothing to say.

I’d say that vibecoded software is similar. My speculation is that current breed of LLMs do not have an I, and I really don’t exactly knows what goes on in those vast arrays of numbers. There’s something there perhaps, but no person.

Still even in the short term someone wants to run a company that expects responsibility of its organisation, how are you going to exact that responsibility if no one actually understands how the thing the organisation makes works.

Maybe a simple crud system can be made fast and loose. But a bank settlement? A pacemaker? Deletion of sensitive data?

I know some companies are betting on that the agent can fix what the agent breaks. It may be true, but up until now everytime I try to relax on strict steering of an agent it tends to go badly rather fast.

Again I don’t know, but I think as long as we don’t invent synthetic persons with their own ideas on what they want to do, which btw opens a massive can of worms, the current situation will persist. However clever the current breeds of systems are.

I do want to state that a find the current trajectory fascinating. I use LLMs daily, it expands the number solutions I can explore. But in order to make something I feel is mine. There’s a choice and the buck stops with me.

Oh boy do I have some news for you about legacy codebases.

I’m not even sure that his axiom is true. A project started with a 2024/5 model can subsequently be worked on by more capable models (who don’t, unlike some humans, have a deep aversion to paying down technical debt). I’ve seldom seen a human written legacy application spontaneously acquire a better engineering team every six months.

We (collectively) were unprepared for a machine that presents itself in human forms. We were the frogs that boiled ourselves. We built a world of images and words on a screen. And then we built a machine that can (increasingly) mirror that world; it does so in a way which most of us are incapable of disambiguating.

It feels like there is indeed a ghost in the machine.

And there is, but that ghost is us. And that ghost is fading surprisingly quickly.

At least I did not find a new thought in that (granted, relatable) rant.

"This is bad and you are bad" requires people to not defend their reality through rationalization, but the point we're at with AI right now is driven by exactly that. So this is at best highly ineffective at reaching the people it claims to want to reach.

That said, the underlying emotion of "you all suck and I hope you lose your jobs you frauds" is relatable and worth screaming from the rooftops of Linkedin dot com for the catharsis alone.

AI is nothing special or new in this regard. It just gives a single guy the velocity to ruin a codebase at the rate of a full enterprise team at double the speed.

A shitty code base still makes money, and that's all that will ever count for the majority of the employed developers, those who don't blog.

Personally for my job I struggle to justify writing code by hand. Agents are much better at typing the code, reading the code, finding patterns and discovering bugs. They don't have my knowledge and my background so the current models still miss things or overcomplicate the code. I don't know if that's going to be true of the next generation of models (or the one after that).

With current models, I cannot fully give in to the vibes. I _have_ to look at code (maybe not 100% of it but at least a majority of the lines). I _have_ to grill the agent to make sure it writes the best code that I deem possible for the problem at hand. This isn't the fastest way to do agentic development, but AI agents have already made us significantly faster than we were in the pre-AI era. I don't want to squeeze every ounce of 'development speed' if that comes at a cost to code quality, maintainability or debuggability.

For my side projects, things are different: either I only care about the outcome (and I give in to the 'vibes') or I care about the journey just as much as the destination in which case the agents are extremely advanced search engines, code reviewers and mentors but they don't get to write any of the code.

Yeah, I guess I personally fall on the whole spectrum: for side projects I'm at the extremes but for my professional work I fall somewhere in the middle.

We've had strong coding agents for less than a year. Anyone making such a definitive statement about how vibe-coded projects progress over time is basing it on guesswork, not evidence.

https://12factor.net/ https://en.wikipedia.org/wiki/Twelve-Factor_App_methodology

Kevin Hoffman expanded on that with the 15-factor app: https://developer.ibm.com/articles/15-factor-applications/

Mind you I may be dating myself as I was first introduced to this paradigm in 2015 working as a Java SpringBoot engineer on an enterprise project that I then migrated (57 microservices) all to Scala, after onboarding two weeks to Scala fresh from no prior Java experience.

I feel like there is so much "wisdom" encoded in books and writings from some of the most prolific engineers and architects over the last several decades.

Look at Matt Pocock's skills with simple primitives like grilling the human, researching through wayfinder maps (a Godsend to my workflow prior to Cursor Projects and orchestrator patterns), and having a solid Domain Driven Design through defining a shared glossary and breaking up work around proper seams.

1) A lot of the time i spent deciding on interfaces (methods, classes, etc.) for humans. E.g. should this be two methods or one, should this method be in this class or moved to utility. Those problems went away. 2) What about performant code? This can be prompted away and when the measurements in your performance tests do not go down, then you can step in. 3) Sad to say but the AI has always been better than me at code-reviews. Maybe this is just me and if so I own that, but to the articles point, it might be harder to fix now. 4) "vibe-coded projects devolve over time into an unmaintainable mess". Preventing and managing this mess is the new skill sets we need to develop as software engineers. 5) Another skill-set we will need to master is how to maintain and grow our coding skills. Some ideas are: a) every once in a while implement a feature yourself. b) no AI Tuesdays! c) Have the AI quiz you on the code base. d) Have the AI develop HTML docs about how the code works.

This takes time away from implementing new features, but that’s true of all code health maintenance.

Putting in massive effort with AI leads to quality exactly like before hand coding.

Vast majority of work was junk before AI because vast majority of people put the minimal possible effort.

The difference now is that AI can make low effort work look high effort at a surface level.

I'm weirdly attached to this line. Physical automation a.k.a. factories are not portable. The manufacturing techniques are, but to "spin up" a new factory requires expertise, effort and capital. In contrast to software, replication can be done quite effortlessly, with containers and such tools, it's easily a one-man job.

Writing, coding and art are not meant to be repetitive, scalable tasks. I think that's what's driving the core of the backlash. If these forms of human expressions can be automated, then why would we need to apply our mental capacity at all?

What worries me more is my own tendency to rely on AI more and more. It is becoming increasingly difficult to choose the harder path of coding by myself instead of taking the easier road, even though I know that road may gradually make me lose some of my skills.

Fortunately, I am closer to the end of my career than the beginning, but I worry a great deal about the new generation of developers.

So now we have a fork: red-pillers who want to regenerate software all the time (esp those with unlimited token budgets) and blue-pillers who want to maintain more code mass per developer-head. Plus, we have software artisans.

That will be an interesting horse race.

[1]: https://news.ycombinator.com/item?id=49774239

Too often people get an idea and don't stop to ask if there is an even better idea. (I'm guilty of this myself). Often it takes a while to figure out what the good ideas are, but people want an answer now.

>In the future we will see more and more companies proudly boasting their “NO-AI” policy as a competitive advantage. And they will be right.

I will very happily take the other side of this bet. Maybe if LLMs stayed as September 2026 LLMs for the next 20 years, I'd grant it's possible. But that's not what's going to happen.

Is it a fact though? Maybe my projects aren't ambitious enough, but about 6 months ago I stopped hitting the point where AI can't maintain what it has written. It's probably unmaintainable by humans, but that might be an increasingly irrelevant quality.

> The proficient developers, the experts, rely on their intuition built with sweat and tears, working long hours trying to debug and fix production issues, swearing to never again be so foolish as to repeat past mistakes. It’s the kind of intuition that can’t really be made into a list of rigid rules, because everything is context-dependent. Experts are incompatible with the same rules and recipes that make beginners more productive. Experts don’t follow the rules, they make the rules.

.... because I have found the same thing - that there's nobody more zealous about some paradigm than those who are recently converted to it and who haven't come to find that everything has its trade-offs. Design is always about evaluating the trade-offs and seeing which ones most suit the given situation.

it just doesn't happend, too much time pressure, and low-hanging fruits must be had, money must be made, and so it goes... that begs the question, if most enterprise software and/or game projects are an unmaintainable mess, yet they still make money and "work" somehow, whats the value in "beautiful" projects/architectures/code over time, if it provably works either way?

We're a few years into a new technology that is still improving. This is a point-in-time critique.

Most code in general is bad by some metric, and likely many metrics. Let’s not pretend that closed-source, in-house code is better.

> There is no fitness function you can define for maintainable code, at least not one that we can discern, otherwise it would’ve been baked into our linters.

This is emotionally appealing because it tells me that my judgement is irreplaceable. But it’s also essentially an appeal to magic. A metric than cannot be defined is either not real or is a matter of taste. And maintainability cannot be a question of taste because it makes concrete claims about the software, not just human perception of it.

All this is to say that we have yet to establish that “vibe-coded projects devolve over time into an unmaintainable mess” is a fact, or that it’s more true of vibe coded projects than hand coded projects. We are still learning how to build the right guard rails on vibe coded software because previously we relied a lot on humans reasoning over the code and saying “this looks about right” which frankly is not a strong engineering practice to start with.

They are often too verbose, or miss capturing an important concept or purpose, or add bullets for parts of the changes that no one cares about.

Capital always prefers machines.

It's appalling that we still hold on to such things that are no longer necessary. Code maintainability is not a problem when you don't have to open a file and inspect how something works anymore. You use english to add to it. You sit on chairs everyday where you don't give a shit how they were created. They fulfill their purpose. hopefully the same can be said for your software.

I absolutely agree with the author that humans need to be in the loop reviewing and understand the code they're merging, and generally take a "Hey, build X like Y utilizing Z" approach when using AI to build instead of the "Hey, solve this problem" approach. Our PE overlords actually mandate the latter, but I'm not doing it.

However, a point the author misses is that with AI, major refactors become relatively quick. Hours instead of months/years.

Yes, AI can and probably will land you with major foundational and architectural problems, but your architecture isn't set in stone anymore. Your entire codebase bends like a leaf in the wind.

But when you can literally generate a whole codebase in a few hours, a lot of these assumptions don’t hold up. That doesn’t mean you don’t need sound logic and maintainability. But a lot of the small things you think would go wrong because of a bloated codebase stop being meaningful objections when you can keep generating, testing, and reworking it until those issues are resolved.

A lot of what we consider good engineering judgment is shaped by the constraints we’ve always worked under. I don’t think people appreciate how much of that changes when those constraints go away. The "slop" problem is ultimately a verification and testability problem.

- Use the code that your agents write in anger.

There you go. Do I know when my agents fuck up? Yes, I absolutely do -- because I'm a user of the code I have my agents write, and I ask things like "why is it taking 50 ms to start this program ..." and then I go in and find stupidity, and excise it. I do this over and over again.

Is it faster than writing it out by hand? Maybe! It's definitely a different perspective.

Start behaving like a baby "why, why, why" and then do a bit of reading, and you'll be fine.

A lot of these blog posts seem like they're aimed at software written by B2B companies who don't even use their own software ...

And, so what? Software as an engineering discipline has long lacked standardization and regulation to be on par with other engineering disciplines, and the fact that code and all its surrounding ecosystems are not "visibile" or "malleable" makes this extremely hard.

You can use terraform and yaml to define infrastructure that literally spins up machines _somewhere_ in the internet. With all its issues, bugs and associated consequences mostly being ignored.

I just don't understand the difficulty in KNOWING what needs to be done: NO LLM usage in university/grad/high schools, NO LLM usage in the first 3 years of your professional career.

Once the basics are solidly grasped, then they can use it at will.

The problem has never been about wisdom, knowledge or LLMs writing good, bad, maintainable or terrible code. It has always been about the skill level of people using it AND on the fact that people start off-loading basic things to these models that they wouldn't before.

If you have the knowledge and "suffered" through experience to learn the fundamentals, than not using LLMs becomes more deterimental than beneficial.

You just CAN NOT skip the trial by fire of learning and absorbing knowledge on your own. That's all.

This book gives a conceptual groundwork for what I believe to be both the promise, and crisis of AI. AI excels at, and either will soon or already has surpassed humans at a kind of reasoning that Horkheimer calls "instrumental reason." This is reason as a tool for achieving ends. AI will, I think, surpass humans at writing maintainable code, as this falls within that domain. The writer is thoughtful, but the obstacles listed in the article are technical, and they will likely fall.

A second kind of reasoning, which he calls "objective reason," is reasoning about which ends are worth pursuing. Our contemporary pragmatic, positivist culture in the West has a hard time reasoning about ends. We tend to just see different perspectives, competing power networks, etc. We are skeptical of capital-J Justice, for example. We tend to see it as historically and culturally contingent, or else as a mask for powerful interests. Horkheimer believed this learned relativism was responsible for new forms of domination and control that emerged in the 20th century in both totalitarian and putatively free societies.

I think the author's thesis would be correct if it had focused on AI's inability to discern ends. But that then raises the question: can we discern ends? Horkheimer never settled on a way to rehabilitate objective reason. He focused on critique, and his critics suggest he ended up with a tangle of negations that couldn't hold up any objective ends. I think rehabilitating objective reason is the task that faces us whether we like it or not, given our moment in history with the advent of AI. It's something practitioners need to think about. Either we recover the philosophical tools of objective reason in light of the critique which caused them to be discarded in the first place, or ends will be imposed arbitrarily by any and everyone with the means to do so.

We could take the Hofstadter idea and say that there is an isomorphism between these two artifacts. Software is a manifestation of the team's knowledge the way an organism is a manifestation of a genome.

With AI, the second artifact is no longer necessarily produced. We don't need to have a team that, as the project progresses, slowly becomes a group of domain experts. Experts that can drive the project direction. Experts that, with time, can see the flaws in their first project and start new breakthrough projects to fix these flaws.

Imagine breeding a new tomato plant. One is robust and tasty, the other is neither, but the genetic code is easier to understand. Picking the legible plant, because legibility is important to you, and may make future tinkering easier, is a dead end. Being able to create a billion different tomato plants and applying selection pressure is more effective, if you have the resources.

The idea that we are applying all this effort to make creating nauseating ads and heinous travel booking sites easier disgusts me, but hey, whatcha gonna do?!? :-)

Source?

False. AI is evolving by leaps and bounds and the outputs are better and better by the day

Personally, repeating what the article says, I haven't coded since may 2026, not a single line, and AI has been delivering exceptional results, improving by the day

Any AI related article necessarily needs to consider future improvements as they will come not by surprise but by steady refinements, and posts like this will look just like the same early AI slop they complain about

Coding IS solved

    > Fact is, vibe-coded projects devolve over time into an unmaintainable mess. The reason is simple, yet hard to fix: code maintainability and good architecture don’t have good measurements that we can apply, because it takes months, years even, to notice the effects of bad architecture or of unmaintainable code.
    > 
    > For one, AI is not trained on what it means for code to be maintainable. For instance, any reinforcement learning done needs a reward signal that can be measured immediately, not in months or years.

I think at the end of the day, it is not impossible to have AI write "good" or "high quality" code. If anything, once the patterns are established, AI will be more likely to adhere to the patterns and rules than any human team. It requires the most experienced engineers on the team to split their time writing the core patterns and documenting them in references/skills.

But it takes a lot of "taste" and a willingness to slow down a bit with AI (to create necessary artifacts), something teams find hard to do when you can ship so fast now.

My experience has been that there is a camp of very senior engineers that are unwilling to adapt to reality and focus on documentation and writing (effectively producing skills and agent guidance which multiplies their effectiveness); they will cling to their knowledge thinking coding a sacred art.

They can't think that a software can be written, designed and maintained in English or higher constructs.

It is binary of either vibe coded app, or worshipping the code that they worship (and I spent 20+ years on)..the industry is moving on faster than most can wrap their heads on, but for those who understand software engineering was always beyond and above code, they will adapt, meanwhile, those who built their entire identity and skillset around managing code will struggle. There will be a place for those people, but it will be niche and specific, and we won't need as many.

I think this is a bad take... if you are going that long without noticing, you are (hopefully) delivering end user value all that time. That is the main driver of code. You can normally dig/hack/rearchitect your way out of an ugly code situation. If you've been building value for the last year based on the hacky code, that's a win.

The truth is, "good code" is relative. It is determined by the specific domain and the composition of the team. Is incomprehensible FP (Functional Programming) code good? No, it isn't. A programmer must assess the team's capabilities and adapt accordingly. Good code is ultimately something that morphs based on the shape of the organization. Once defined this way, good code might share certain commonalities (like readability or a shared mental model), but its actual form varies wildly.

So, what is good code? That definition is missing. To be blunt, the Hacker News posts insisting that we must write "good code" are essentially a form of self-hypnosis.

Just look at paradigms. The mechanics of OOP have changed significantly, FP approaches have evolved, and DOD or DDD are fundamentally different from their early days. Whenever paradigms are discussed, someone claims, "That problem was solved in the past, and nowadays we do X," only for someone else to reply, "I don't think that's actually solved," leading to a fragmented breakdown in consensus. Ultimately, which knowledge remains as tacit knowledge is entirely dependent on the organization's capability.

You could argue that AI is terrible at simplifying code. However, I am skeptical that AI coding needs to be identical to human coding. When you actually code with AI, it often produces structures humans would call anti-patterns, including God Objects. Yet some of those structures can be faster or simpler for machines to navigate. There is no reason to assume that the optimal modularity for AI maintainers must be identical to the optimal modularity for human maintainers.

Of course, I am not denying that the rewards of good architecture are delayed, or that there comes a point where maintenance becomes impossible. But as the AI era ushers in an age of overproduction, software could become disposable, strictly personal, highly tailored to small niches, or ultimately, heavily polarized.

Realistically, programming domains fall into two major categories: "ship it and forget it" (one-offs) and continuous services. I agree with the OP's point that AI struggles to understand boundary delineations. But honestly, you can enforce those boundaries by injecting them into the spec. How those boundaries are drawn in the first place, however, is purely a matter of personal experience.

Personally, I define "good code" as code that allows the entity responsible for the software to achieve its purpose with a sufficiently low cost and error rate, factoring in the software's expected lifespan and future changes.

If you ask an AI to generate work based on this standard of what level of code is "adequate," you might get entirely different results. The biggest problem with discussions around AI is not just that ideological identities prevent proper evaluation (as seen in that article), but that the AI itself scales proportionally to its input. It is an incredibly difficult issue to judge because you don't know an individual's workflow or exactly how they are utilizing the tool.

I do think the value of reading code is important. However, much of what we are discussing in the AI era is actually rooted in the path dependency of how to become a good human senior developer.

Instead, the core focus of AI-driven development might shift toward defining broader abstractions: data semantics, invariant external contracts, and migration strategies.

Ultimately, I believe the paradigm shift of our era should lead us to ask: "How do we write code most economically in a system where AI is the primary maintainer?" The OP might think differently, but at least, that is where I stand.

Let's stop overvaluing human work. Sure, I don't want AI to write the entire codebase without I know what the fuck it did.

But AI is on an equal footing with an average dev.

The agents are trained by contained data. All agents. Similar data sets. Human knowledge packaged into little boxes that all look just the same.

When the boxes are doing all the work for us, the differences between them will come down to color.

There’s yellow ones, and blue ones…. And, they’re all made out of ticky tacky and they all cost just the same…

Well said. The cost of bad architecture imposes a heavy penalty on the project down the line.

> The proficient developers, the experts, rely on their intuition built with sweat and tears, working long hours trying to debug and fix production issues

Long hours of grinding with the problem, the solution, the code and the machine all makes for an intuitive understanding of what a good architecture is in the first place.

LLMs are a tool. A useful tool unlike any we have seen so far. But still a tool. Treating it as such and leveraging it should be prioritized.

This is the labor theory of value; consumers don't care if the code is hand-made, they want the cheap goods (software) that are the output.

Everything is reactionary. The focus is always on short term profits. No one cares about long term effects --- until they start impacting short term profits.

This is an inherent vulnerability that is easily exploited by someone willing to engage in some simple, long term planning --- like China has already done with manufacturing.

For decades, the USA gladly shifted manufacturing to China without a second thought. Now, we lack the expertise to make our own and have no choice but to use China.

Replace "manufacturing" with "software" and "China" with "AI" and it's deja vu all over again.

The author seems to be confusing "I didn't write any code" with "I don't care about software design and maintainability". There exist maintainable and thoughtfully designed software systems for which the designer did not write any code and didn't read most of it. It's not the median, but the median software project has always been unmaintainable before agents.

The argument I'd make in support of the title would be that wisdom is embodied knowledge - knowledge earned from direct experience. Known first-hand. An AI can already have linguistic/technical ability that exceeds a human (at various levels of skill, even) - in coding, copy-editing, what have you. But wisdom? That is the scar tissue we carry from wrong choices. Until an agent has persistent sense of self and a "body" (or pseudo-body) whose existence it is predicated on (thus creating incentivization/outcome alignment), it will not be able to approach "wisdom" in the sense that humans do.

That may seem to have no bearing on the specific subset of tasks related to coding and so on - but it actually does. Humans have a hard limit of attention span, hours in a day, years in a life. An AI does not. So if it fucks things up royally by half-assing an architectural decision, it won't care. And if it doesn't have a persistent memory, it won't even know that it made that mistake before. So it has neither the prerequisite abilities nor the incentives to gain what would be hard-earned wisdom from its mistakes.

In fact, as long as the business model of its creators depends on more usage = more profit, they are actively discouraged from implementing any kind of wisdom-like ability.

For them to be able to offer that without undercutting their profit, they'd need to be able to execute "outcome based billing".

Just to give a hot take, it's funny to look at his builtwith.com. As a developer you have a static site that depends on Cloudflare, Mailchimp, Postmark, Isso ...

Twenty years ago any self-respecting dev would have run the equivalent of all that themselves on their own metal. In 2026 elite neckbeard practice is write the "never reach mastery, because they no longer make choices, they no longer take responsibility" post, hit "publish" and it's magically deployed around the global internet for you. Like a child.

In the future we will see more and more companies proudly boasting their “NO-AI” policy as a competitive advantage. And they will be right.

Yes, a "NO-CLOUD" policy was already so popular, surely this will happen too.

https://builtwith.com/alexn.org

>>"The AI learns rules from rulebooks meant for beginners. The AI notices patterns from code in the wild and let’s be honest, most code in the wild is pretty bad."

Similarly, for self-driving, the AI for driving learns from rulebooks meant for beginners and observes patterns of driving from ordinary drivers in the wild and let’s be honest, most ordinary drivers in the wild are pretty bad. The best the AI self drivers trained in this way can hope to get is the average slop driver minus the catastrophic errors.

Similarly, in legal specialties, there are a few highly expert and wise practitioners, and hordes of average practitioners, and quite a few near-malpractice practitioners, and it is the latter group who write the most stuff on the web because that is how they do marketing, trying to make their ignorance look better — and mostly louder — than what the average lay-person knows. The AIs can NOT tell the difference and 'learns', and dispenses both the good and the actively harmful advice. Acdg to relative who is a top expert in their specialty, AIs can both find key relationships between laws in complicated situations but also readily dispense the actively harmful advice, and you MUST already be a top expert to recognize which is which.

They are great averaging machines, but when average is not good enough, you must be on top of your game.

No. Most rules are there before and will stay after the expert enter the field within its career path, unless the field is bright new territory no one fooled before, which is rare.

Not only experts have to know the rules, otherwise they wouldn’t be expert, but good experts also ideally know why the rules were set, and at least have a fairly well aligned representation of what it would likely lead to to follow or not each rule in this or that situation, which rules are in conflicts and what the tradeoffs are when favoring one on the other.

Everything is context dependant, yes. And LLMs can help to leverage on far wider contexts that a single individual would be able to do on its own. It’s require interest to reach some goal in some social context, and not everyone will use LLMs with the same creativity.

This week-end I was discussing with a friend about our respective use of LLMs. At some point they told me they no longer read the MR, as LLMs can also do great job on that matter now, which is in sharp contrast with what I do. Not that I don’t auto-review with LLMs, but I use that as a first step, be it mine or some other colleague. And then I ask an LLM to prepare me a reading plan to check the MR, taking into account the activity scope and the implementation architecture. To me it was an obvious way to go, but for them it was something they never considered. That’s random sample, of course they would certainly be cases we would switch the "I wouldn’t have thought about it" role.

So yes, LLMs can be used to produce faster giant piles of unmaintainable codebases. Or they can be used to strengthen processes that lead to code quality. The nail won’t prevent us to knock our thumb, to use a nail in reverse side, or to smash our coworker.

I get the paperclip plant issue, but that’s some extreme scenario (which is of course the point of the allegory), and most bad uses will be far more mundane in how they look and what the consequences are. And most good uses will look more and more transparent to users to the point they won’t even wonder about it at each use. Like, most people open the tap, see water fall and they don’t get a sense of wonder, not giving a thought to this masterpiece of engineering and gratitude for all people that works daily in the shadow for this miracle to happen. They don’t leave the toilets thinking "how freaking amazing such a complex system of wastewater is something I can benefit from everyday, unlike so many other of the 100G humans that walked this earth."

Next time you go to WC or tap some water, think about it.

At this point AI is a better developer than most mid-level SMEs I've worked with. I hate this fact, but I don't have a shred of evidence to dispute it anymore. I work on a 20 year old codebase that thousands of developers use and it finds bugs in pre-AI code daily.

Counter prediction: using AI attractive even for employees. Do you really think you’d wanna join such a company? No way.

has the author not started to develop instincts with regard to ai usage and pitfalls?

dumbest take ever. AI is here and not going away, any company that does so will not survive or will be a niche thing for hippies.

i wouldn't know a single engineer who'd want to work at a place like that

Just because it's bad for a human doesn't necessarily mean everything will fall apart - unless a human has to maintain it unaided.