AI Just Created Viruses Not Found in Nature

Posted by donohoe 12 hours ago

Counter23Comment12OpenOriginal

Comments

Comment by lrvick 11 hours ago

It is not one angry person shutting down the internet or a massive supply chain attack that worries most as a security researcher. I have come to accept such things are way too easy, and therefore likely, but as a species we will survive without Instagram.

But what actually worries me most is biosecurity.

Someone angry at the world with the $800 of CRISPR gene editing equipment at home and an AI model coaching them through engineering a virus for maximum harm.

Comment by schiffern 11 hours ago

Take your pick, neither sounds great:

1 angry person with $800 in CRISPR and AI

1 angry person with $800 in pangolin cages and time

1 company with a billion chickens and a drive to cut biosafety costs

Comment by sudosysgen 10 hours ago

The genetic sequence for smallpox is publicly available. The bottleneck in causing maximum harm is that you need a large, expensive, well stocked lab and a lot of expertise. LLMs helping people generate a sequence just isn't the bottleneck to bioterrorism, it's entirely about abilities and equipment to deal with the physical world.

It's still worth thinking about it, of course, but by far the way to mitigate risks are things like wastewater surveillance, tracing purchases of physical supplies, etc...

Of course, an LLM could provide advice that makes it easier for a virologist to do something like this, but there is little indication the effect size there is notable.

The life sciences are very hard. A motivated undergrad requires extensive in-person teaching to become useful in far simpler projects. It's not clear how an LLM can in the foreseeable future bridge that gap when human experts can't teach someone without a lot of time and in person interaction, and that is with access to a staffed, resourced lab.

Comment by lrvick 6 hours ago

I would note that we already have groups of hobbyists in the bay area using CRISPr kits to genetically alter frogs, dogs, and in some cases human DNA at home, and even making MNRNA vaccines at home or community labs.

I think underestimating how much more accessible this tech is to randos very recently is a mistake.

Comment by sudosysgen 4 hours ago

I am aware, I personally knew someone who did so a decade ago. This is very different from synthesizing and culturing a virus from a sequence. Also, you wouldn't use CRISPR for this, you'd make a plasmid.

The best defense for these is upstream, so that the plasmid provider refuses to make it for you if it looks like a pathogen or you're not affiliated.

Comment by EA-3167 11 hours ago

They will absolutely not create anything of note with that setup, engineering viruses de novo requires a properly stocked lab with quite a few more zeros at the end of your proposed figure. Guess what happens if you try to assemble all of that as some rando?

Comment by jmpman 11 hours ago

The Colorado movie shooter was a Phd student. https://en.wikipedia.org/wiki/James_Holmes_(mass_murderer)

Someone like him would have access to the "properly stocked lab".

That's the nightmare scenario.

Comment by EA-3167 11 hours ago

I believe you're misunderstanding what I meant, we're talking about a specialized lab with a hell of a lot of privacy (read: zero oversight), lots of time and money, and the freedom to experiment on a range of animal models.

I'm sure that some day, maybe not even that far off, the tech to do all of this will be much cheaper and easier to buy. There will come a time when all that stops people from your nightmare scenario is "used car money", a solid education, and regulatory agencies. Happily that time hasn't arrived yet, but I do appreciate that people are at least talking about it before it actually arrives.

Comment by jmpman 10 hours ago

And I'm pointing out that there are people with access to all of the required equipment today, who are experiencing a schizophrenic break... biology grad students.

Comment by sudosysgen 10 hours ago

Why would they need help from an LLM to design a sequence when there are sequences for many existing highly pathogenic viruses and well understood ways to make them more harmful?

Besides, biology grad students rarely or never have access to a lab with zero oversight.

Comment by eth0up 6 hours ago

Thanks for having the courage to write this. Remember, the YouTube sensation, who made a Covid vaccine in his kitchen?

Your concerns are grievously undermined.

Comment by great_tankard 10 hours ago

This is an absurdly irresponsible way to report this work. Doesn't think to mention that they're working with phage (viruses that infect bacteria, NOT humans) until halfway down the story.