Critical CVE issued for hallucinated SQLite vulnerability

Posted by ymir_e 1 hour ago

Comments

The problem with this kind of thing, is that it reduces the S/N (Signal-to-Noise) ratio, so weeding out the legit CVEs becomes a lot more difficult.

But, on the other hand, I do know that LLMs have been discovering a lot of legit CVEs, and I will lay odds that the blackhats are leveraging them to the max.

Comment by flerchin 35 minutes ago

The vast majority of CVEs are not exploitable, basically noise. I suspect that the overwhelming majority of the CVEs being generated by LLMs are either noise of the sort in the linked article or noise of the sort that is not exploitable.

Comment by cogman10 18 minutes ago

This is what I truly don't like about the CVE system.

One such example is CVE-2023-45853 [1]. Zlib included in it's source an extra set of utilities and add-ons. One such utility, MiniZip, had a buffer overflow vulnerability. BAM, 8.8 CVE (was a 9, looks like they pulled it back a bit). But not one that the 99% of applications using zlib would ever be vulnerable to because almost nobody used the MiniZip utility. It was so unused that the solution for zlib was to simply remove it.

I know about this one particularly because our security policy required us to do a BUNCH of pointless updates for it since zlib is in just about everything.

[1] https://app.opencve.io/cve/CVE-2023-45853

Comment by post-it 3 minutes ago

But could an attacker with access to zip tools exploit it to get a root shell? Probably not, but maybe.

Comment by bluGill 19 minutes ago

The majority these days seems to be real issues, but probably not exploitable. The latest LLMs are very good at reading code and finding issues where the code is wrong. They are somewhat less good at finding exploits - both because finding exploits is harder than finding code that is clearly wrong (even if not exploitable), and also because the better ones are intentionally made to not create exploits (since they don't know if you are a good or bad - they assume bad.

However if I was writing this response just one year ago I would instead be saying: the majority off LLM CVS are noise where the code is correct, and often they are writing up for code that doesn't even exist.

Which is to say I suspect the repo in question was generated with a year-old LLM, since they act like that. The new ones [mostly?] are much better.

Still, if a modern LLM points out something you should fix it. Even if we can't figure out how to exploit it today that doesn't mean we won't figure it out in the future.

Comment by pbronez 10 minutes ago

I once had a customer threaten to reject a delivery over a CVE. I dug into it. The CVE was that an Ocaml PostgreSQL client didn’t implement an authentication feature.

I convinced the customer to accept the delivery by pointing out that (1) our app had zero lines of ocaml and (2) the feature had been implemented in the ocaml driver since the CVE was issued.

Comment by ymir_e 1 hour ago

It seems like we're in a transition period where AI will eventually make all software much more secure than it ever was.

In this period every part of offense and defensive cyber security changes quite rapidly.

Noisy CVEs will probably lead to agents verifying vulnerabilities before humans review them.

The problem with agent reviews from what I can think of is:

- cost to use LLMs to review things

- not necessarily easy to plug-and-play in repos: (domain knowledge + how to look for vulnerability specifically for the stack)

- especially with anthropic: able to use models defensively, without hitting guardrails.

The last one is the most interesting one to me. How does the AI providers know if you're a "good or bad" guy? And does it matter if open source is catching up?

We're in a kind of cyber arms race wether we like it or not.

Comment by rghammt 48 minutes ago

Currently we either get AI promo vulnerability dumps like from Chrome with pretty graphs that no one checks or false positives.

Where is this one now that was hyped everywhere?

https://news.ycombinator.com/item?id=49133889

The GitHub submitter could no longer reproduce the issue and the LKML post has no replies:

https://lore.kernel.org/all/CALCETrXbj__SFQMzPZhES5y6-sh4np-...

Comment by bluGill 17 minutes ago

All the big maintainers I know of disagree. Curl for example is getting overwhelmed with real issues that LLMs are finding. A year ago most LLM found issues where false positives (the code is correct), or hallucinations (the code in question doesn't even exist) - but that has changed and now LLMs are finding real issues in code. Many different projects have reported the same rules.

Comment by inigyou 1 hour ago

This is going to be fun for organizations that are mandated to patch all CVEs, isn't it?

Comment by dgellow 6 minutes ago

One can hope that will put pressure on the industry to design a better system than CVEs. The signal noise ratio was already terrible before LLMs, I cannot imagine that will still be a meaningful system in 10y.

But I’m too cynical to not consider all the middlemen who benefit from the status quo

Comment by lucideer 1 hour ago

I'm very curious what organisations would have such a policy. I can't imagine it being viable for any size of org without significant self-deception (or banning the use of all open source at which point CVEs are moot anyway).

Comment by traceroute66 39 minutes ago

> I'm very curious what organisations would have such a policy.

I would humbly suggest any org of any size that has insurance cover that covers anything tech related (e.g. data loss/recovery, cyber etc.) has a very good look at the small print.

Over the last few years insurers have aggressively been adding "no vulnerability patch, no claim" exclusion clauses.

Comment by clbrmbr 1 hour ago

Many orgs (esp w ISO27000) have a vulnerability management policy that involves patching at least critical CVEs within a short timeline. Tools like trivvy make it possible to do the scans…

Comment by lucideer 46 minutes ago

I've been in such an org, & I've led initiatives to set up automated detection at very large scale. We started by issuing tickets to teams to resolve CVEs within varying timelines - ranging from a 24hr fix to 6 months - connected to the CVSS score. It wasn't viable.

- Firstly, you quickly realise how irrelevant CVSS scores are - initiatives like First's EPSS are designed to fix this but they aren't there yet

- Secondly, you need to begin implementing localised heuristics to determine exploitable code paths. This has generally been incredibly difficult to do reliably - LLMs have started to make it easier, but it's expensive.

- Lastly, you need to factor in consideration of actionable remediation pathways. A dependency upgrade for critical infrastructure might contain breaking changes that take months to fix, or two competing CVEs might be present in interdependent versions of transitive dependencies in your sbom tree.

Most orgs aren't applying any of the above three filters to reduce their CVE remediation burden, & even if they are, it's still too high to make zero a viable target.

In reality, most orgs aren't doing comprehensive detection to begin with - if you haven't discovered all of your CVEs, your remediation burden is going to be a lot more manageable.

Comment by mr_mitm 32 minutes ago

> - Firstly, you quickly realise how irrelevant CVSS scores are

Even if you factor in the environmental score? I realize it's a lot more work, but it basically allows you to tune the score to get any value you want.

Comment by dwedge 25 minutes ago

> Tools like trivvy make it possible to do the scans...

Only if you didn't rip trivvy out of your organisation when it had two supply chain compromises within a month of each other earlier this year

Comment by michaelt 9 minutes ago

SOC2 CC7.1 [1] requires a vulnerability scanner, findings tracked with tickets, assigned severities according to a documented risk-based system, severity-based SLAs for remediation, and that the SLAs mostly be complied with or have tracked exceptions.

However it doesn't mandate any particular SLA, or the details of how risks are to be evaluated.

Organisations get to write their own policy, and they don't need to commit to patching every CVE within 24 hours or anything like that.

[1] https://www.compliancebase.org/controls/soc-2/cc7-1

Comment by 2 minutes ago

Comment by vrighter 10 minutes ago

I have been given a list by security. "We had an automated tool scan that machine. It reported these. Fix anything medium severity and above.

Never mind that some of them involved vulnerabilities in some part of the bluetooth stack (servers in our datacenter don't even have bluetooth). But they just didn't care

Comment by anygivnthursday 1 hour ago

If I remember correctly, we had to patch or provide justification for CVEs flagged by tools like AWS Inspector for SOC2 as well.

Comment by YeahThisIsMe 56 minutes ago

So you didn't have to patch all of them.

Comment by bluGill 14 minutes ago

No, but if you don't patch them you need to convince an auditor that they are not a problem. Often patching is easier.

I'm working on such a problem now - we are using an old web browser (no longer supported) to show help on one system. That is web pages were generate internally, with no links elsewhere, and no provision for the user to enter a URL. It is still easier port to a newer supported browser than to convince the auditors that that we are not exploitable. Sure it is obvious that everything is internal and we won't write html that exploits bugs, but nobody wants to convince an auditor of that.

Comment by jeltz 43 minutes ago

Many large organizations like banks have requirements like this and they solve it through a mix of automatic scanners, e.g. Trivvy, and self-deception as not all systems are actually scanned in any sufficiently large org.

Comment by SirFatty 1 hour ago

ITAR

Comment by lucideer 58 minutes ago

ITAR has no such hard requirements. Might be some orgs that tell themselves they're attempting this under ITAR but they're not doing it in any comprehensive way.

The only thing within ITAR that I'm aware of concerning itself with software supply chain is SP 800-218 requirements & that's just a load of open-to-interpretation weasel words about having CVE detection & automations in place & some defined plans for reducing the number of vulns. Pretty sure that component of it is even eligible for self-assessment.

Comment by bluGill 7 minutes ago

Yes and no. ITAR (and other laws like it) are self assessment and don't specifically say thing thing.

However your interpretation / self-assessment is subject to various reviews/audits. These days the reviewers are not going to be kind to someone who just says "not an issue", they will demand strong justification. Most organizations take the view that is is easier to fix all CVEs than try to pass audits.

Thus by the letter of the law you are correct. However to meet the letter of the law without fixing CVEs is generally seen as harder than thus fixing CVEs. So the effect is ITAR (and similar laws) force you to fix CVEs.

Comment by ignore_prev 32 minutes ago

[flagged]

Comment by ymir_e 1 hour ago

This was my first thought, this could be terrible if used offensively.

The best defense I can imagine is to have an agent reproduce the issues before a human sees it, but even that will cost money.

Comment by smitty1110 1 hour ago

It’s honestly not great. The security guys are completely exasperated at my job, we’re wasting time having with these. You take the scam really, investigate for a bit, write up a DNF with justification, they go and up date records, and we all just kinda hope that someone updates the scans so it stops showing up.

Something is going to give, and I suspect that the optimistic open filling is going to get canceled.

Comment by 1 hour ago

Comment by whatevaa 1 hour ago

Those organizations will have to adapt to new reality, ie, that some CVEs are not real.

Comment by afarah1 18 minutes ago

Nothing new about that

Comment by cleansy 1 hour ago

All organisations also have exceptions to policies. This one would be one

Comment by jeltz 41 minutes ago

Yes, but getting an exception approved can be a huge pain in some places.

Comment by inigyou 43 minutes ago

You'd be surprised...

Comment by dns_snek 17 minutes ago

Steps to patch a hallucinated CVE:

   git stash -m "sigh"
   git commit --allow-empty -m "Patch: CVE-2026-51302"
   git stash pop

Comment by lelanthran 34 minutes ago

Not really. LLMs can hallucinate the patches too :-)

Comment by ape4 1 hour ago

Create the referenced but non-existent file and then fix it /s

Comment by Ekaros 1 hour ago

Not validating submissions seems like avenue for massive attack. Flood the whole system with endless false reports. Thus making it significantly less reliable.

Comment by masklinn 46 minutes ago

That is exactly why many big projects are migrating to becoming CNA, so that randos can’t get assigned unqualified CVEs which nobody has looked at or validated.

Apparently RedHat is a CNA of last resort, so it might be possible to get your project under Redhat’s scope and go through them without having to be a CNA yourself.

Comment by insanitybit 53 minutes ago

This is what the Linux kernel is currently attempting since becoming a CNA. edit: "This" being to sabotage the CVE system via DoS, to be clear.

Comment by inigyou 20 minutes ago

What do you mean? It isn't a DoS to assign every single bug fix a CVE! Every single bug is making someone vulnerable in some way.

Comment by worthless-trash 14 minutes ago

> What do you mean?

No analysis is being done in the linux kernel to assess vulnerability.

> It isn't a DoS to assign every single bug fix a CVE!

On people who care about this, it is, not in the project itself though.

> Every single bug is making someone vulnerable in some way.

Not every bug is making someone vulnerable. (docs bugs, test bugs) behavioral changes, performance improvements, the list goes on.

Comment by delfinom 38 minutes ago

It's been a problem for awhile. Daniel Stenberg has talked about it numerous times on his/curl's blog for the last 4 years. They became their own CNA to try and control it, they opened a hackerone with rewards, but now removed the rewards because it got flooded with AI generated slop daily.

https://daniel.haxx.se/blog/2023/08/26/cve-2020-19909-is-eve... https://daniel.haxx.se/blog/2024/01/16/curl-is-a-cna/ https://daniel.haxx.se/blog/2025/04/24/how-the-cna-thing-is-...

Comment by fabioborellini 24 minutes ago

And this year, some of the reports suddenly got good and meaningful. https://daniel.haxx.se/blog/2026/04/22/high-quality-chaos/

So the agents started doing something useful after a period of filling mailing lists and bug bounties with slop. Sound good, but that's not entirely a good thing. The volume of good reports is a burden as well, and it's likely that long-lasting open source C/C++ projects have legitimate vulnerabilities unpatched. But we don't have any new maintainers, I think.

Comment by gste 15 minutes ago

> The cited code didn't even exist in those versions or referenced unrelated logic. > When testing the PoC payloads they didn’t work (not triggering any crash).

I think the future is pretty obvious, if this isn't being done on projects already: you need to automate these checks and reject automatically

Comment by firesteelrain 19 minutes ago

> This incident demonstrates a systemic issue with automated vulnerability ingestion

But they make X-Ray which does automated vulnerability indexing and matching dependencies to CVEs.

Comment by trashb 39 minutes ago

Could be just me but I would be quite careful running these PoC's, especially since the CVE's mostly seem nonsensical. I suppose these CVE's could be targeting some kind of PoC execution instead of actual bug reporting? (I didn't look at the code in depth).

Why is the repo even mixing CVE's for "schreibfaul1 ESP32-audioI2S" and "SQLite"? Is mixing CVE's for different products in one repo common practice?

Comment by gortok 20 minutes ago

We can chalk this up as another example of over-exhuberance by what folks believe LLMs can accomplish vs. what they actually are.

LLM-based “AI” is able to use its vast corpus of inputs and calculate the most statistically likely output in a given situation. It is probabilistic, and when you are dealing with probabilities in a situation where certainties, not probabilities, matter, you’re going to get dinged on credibility massively when your LLM-based “AI” gets the probabilities wrong at best, or in this case, claims a line of code generates a vulnerability when it is, in fact, a code comment.

LLMs are text-prediction engines. They are not Artificial Intelligence, and shouldn’t not be treated in any form or fashion as if they possess intelligence. What bothers me about this entire situation is that presumably the folks that relied on the LLM-based “AI” to generate these vulnerabilities knew (or should have known) enough about their tool to know this would happen, but did not.

Now, we all pay the consequence, to the tune of hundreds of thousands if not millions of dollars of wasted productivity from teams that have to deal with the resulting fall-out of this usage of “AI”.

Comment by kasperset 25 minutes ago

As a non-expert in this field, I feel we have introduced more “stochasticity”. Before, We had to deal with some human error/ misunderstanding for most part but now we have to deal with power of condensed human expertise along with homogenized flaws.

Comment by Spide_r 1 hour ago

Somewhat related: https://sqlite.org/cves.html

Comment by fl1pper 17 minutes ago

Great article!

We need to further emphasize the importance of responsibility when using LLM tools to produce output for others. It's great to use them for refactoring and bug discovery, but keep in mind that it's your responsibility to analyze it and iterate on it with AI. It makes your code better and develops technical expertise.

The "Hey, analyze that codebase, find all dangerous CVEs, and write a README for the PoCs, so I can post it online for others to analyze, and if I'm lucky, I'll get a paycheck or a title to add to my resumé" approach might work in 0.1% of cases, but it will generate a ton of slop for the community to drown in.

I also think GPTZero and other AI detectors have far more false positives than correct guesses. I tried it on several texts & messages I wrote before 2019, and it flagged them as 80% AI-generated.

In that case, it's reasonable to assume that AI also generated the README text for each discovered CVE. In other cases, however, we should be more cautious.

Comment by dvh 1 hour ago

You're absolutely right...

Comment by mike_hock 23 minutes ago

[dead]

Comment by sabot90260 49 minutes ago

A critical rating on a vuln that doesn't exist is wild. The triage cost still lands on the maintainers.

Comment by trueno 46 minutes ago

just ran this article that ran a cve through gptzero.. through gptzero

-GPTZero AI Detection

-Model 4.8b

-We are moderately confident this text is a mix of AI and human

-63/88 Sentences likely AI generated

Comment by r_lee 7 minutes ago

can anyone explain what value does the ai slop image at the top add to this blog post?

Comment by luciana1u 9 minutes ago

imagine being a maintainer and having to write "this code does not exist" in a CVE response

Comment by account42 42 minutes ago

The irony of writing an article about slop reports and then defacing it with a giant unrelated slop image at the top.

Comment by pelasaco 29 minutes ago

I had the impression that the code review done and text formatting by the jfrog was mostly driven by Claude too..

Comment by throwa356262 48 minutes ago

In the mean time, my very real vulnerability reports are not even acknowledged because maintainers are busy handling this kind of nonsense.

Comment by 1 hour ago

Comment by BedVibe_Studios 1 hour ago

[flagged]

Comment by effnorwood 27 minutes ago

[dead]

Comment by mlvljr 1 hour ago

Honest take, this is a critical CVE.

Comment by awkwardpotato 37 minutes ago

How so? All 6 of the CVEs covered in the article did not actually exist when investigated

Comment by mahogany 22 minutes ago

The comment is making fun of a Claude-ism where it becomes super “honest” about stuff.

It’s a joke but there is an underlying real effect where this type of language is psychologically manipulative and I would guess makes people believe LLMs output more than if it didn’t use “honest” (or “load bearing” or whatever super serious important sounding word).

Comment by mlvljr 25 minutes ago

[dead]

Comment by progval 1 hour ago

> All advisories in this repo seem AI generated when testing them with Gptzero

I pasted this blog post from "Analysis Matrix" to the end in Gptzero, and it also says the blog post was AI-generated (71% chance of AI, 29% chance of AI-Human mix).