Codex Security

Posted by bakigul 5 days ago

Comments

Comment by dangelosaurus 5 days ago

Hey HN, Michael here, co-founder of Promptfoo and one of the people working on the Codex Security CLI at OpenAI.

Thanks for checking this out and for flagging the auth issues. We just open-sourced it, and there's still plenty for us to improve. Expect the product to evolve quickly.

If you try it, I'd really appreciate hearing what works well and what you think we should improve. Happy to answer questions here.

CLI docs: https://learn.chatgpt.com/docs/security/cli

EDIT: If you'd like to help make this better, we're hiring: https://openai.com/careers/full-stack-software-engineer-cybe...

Comment by orangelimesoda 5 days ago

> Have experience shipping production full-stack products across modern web frontends and backend services.

I'm amazed that the requirements are so low (or at least this vague) for jobs at companies like these.

Has anyone else had the experience of going to an interview and feeling like you were never asked any qualifying questions?

All the questions were easy, your answers were straightforward, you "got them right", but then were not chosen?

I find on the other side, they're also left with dozens of people who "passed" and then it comes down to a pretty arbitrary decision on who gets hired (if we are talking external, no referral, etc.)

I wonder if they can make job descriptions highly specific to filter the shortlist faster and more effectively (to actually get a shortlist).

Anyway end rant. Cool job, hope you fill it.

Comment by pertymcpert 5 days ago

If you didn't already know, jobs at highly competitive companies tend to have vague job requirements because they expect to be able to apply your raw intelligence to changing demands quickly. There's no point being hyper-specific about the exact software packages because that's not what they want. What they want is someone who, after talking to an interviewer for 30 minutes, leaves them with the thought "Wow, this person can do anything we need of them. They can probably tell us what we need too and take ownership of large projects. Hire!"

Comment by moscoe 5 days ago

Agree. Being smart, competent, and high in conscientiousness is more important than any highly specific “qualification”. It’s not about checking a bunch of boxes. If you have a track record of getting shit done, you’ll have something to contribute.

Comment by orangelimesoda 5 days ago

> There's no point being hyper-specific about the exact software packages because that's not what they want

Okay. This makes it sound like they're more sophisticated but it seems more like they are less sophisticated, less specific, and a lot more vague in the job descriptions they themselves create.

If you look at any technical role, game dev or something where people are building important things at scale - there are a lot of specifics. Libraries, methodologies, where if you didn't know them you are nowhere near a fit.

I'm just wondering. It's OpenAI. Surely there is some domain-specific something beyond "has experience shipping front-end and back-end services" since that includes basically everyone.

It makes this job look like a Starbucks role.

Comment by devmor 5 days ago

I see where you're coming from, but as someone who regularly interviews engineers, I don't care about specific tech stacks when evaluating a candidate very much either. I can only think of two positions I've worked in where such a thing really mattered.

A good engineer can adapt and catch up without a lot of lead time. For a contractor, I'd be much more specific - but for someone who's going to join my team? I'm looking for a candidate that can demonstrate their problem solving ability, creative thinking and communication skills.

Other than having some kind of experience in the general domain we work in, those "soft skills" are far harder to find than specific tech experience.

Comment by rustystump 5 days ago

This is insane to me.

Not every engineering job is entry level or as simple as most fullstack crud. Deeper into industry you find highly specific well defined positions for a given domain. Soft skills matter more the higher the ladder but id take a killer senior who can be difficult over a team of mediocre staff engineers.

Comment by a34729t 5 days ago

Yeah, no. If you are digging deep into database internals, query optimization and schema design, it helps to have someone with some experience in the domain. Otherwise your team will spend a few years learning from first principles.

Comment by berrylimetea 5 days ago

[dead]

Comment by pertymcpert 4 days ago

> I'm just wondering. It's OpenAI. Surely there is some domain-specific something beyond "has experience shipping front-end and back-end services" since that includes basically everyone.

Maybe you're looking at this from your own particular point of view too much. I work in systems software, as do many others on HN: our demographic wouldn't be qualified for that at all.

Comment by _superposition_ 5 days ago

This. Being too specific on the stack requirements is a red flag imo.

Comment by swat535 5 days ago

> raw intelligence

I'm not sure how this would apply. Are you implying that if the company operates on Python, you can hire someone with great "raw intelligence" who have only developed C++ all their life, and they can start contributing on day 1?

You need to clearly list what the position entails, otherwise you're wasting time.

Comment by yesb 5 days ago

They are listing what the position entails. In this case it's a full-stack web dev role. "only c++ all their life" is more or less not possible if you have experience doing that

Likewise from another job post: "Have a strong background in kernel-level systems". Not possible if your whole resume is building web apps, may be possible for someone that has only used c++ professionally.

Comment by pertymcpert 4 days ago

> I'm not sure how this would apply. Are you implying that if the company operates on Python, you can hire someone with great "raw intelligence" who have only developed C++ all their life, and they can start contributing on day 1?

In your particular case, no not on day 1. But they can probably make meaningful contributions after 2 weeks. They'll pick it up quick because they're some of the best talent on the market.

Comment by J_Shelby_J 5 days ago

I interviewed with them and had a great experience despite it ultimately not being a good fit due to communication issues and a bad flu coming on in the final interview - they, and Michael especially, was very kind and understanding.

Comment by jameshart 5 days ago

This sounds like it’s just an app development role, not a security analysis position, so I’m not sure what your complaint is.

Comment by orangelimesoda 5 days ago

> just an app development role

Maybe this is part of the problem

Comment by solid_fuel 5 days ago

Professional app development requires an understanding of security.

Comment by corndoge 5 days ago

I think the evidence contradicts you at this point

Comment by bostik 5 days ago

That's quite an indictment of the common app development practices. (Not that I disagree with your point...)

And security is hard. Because it is by definition off the happy path, it is quite often at odds with MVPs and rapid release cycles. Then you add all the ways the users can use your product to attack/abuse others.

Any non-hobbyist app development does indeed require at least a decent understanding of security.

Comment by orangelimesoda 5 days ago

So you don't need to know anything about credit cards for someone to enter it into an app? I don't get the take (is it bad sarcasm?)

Comment by devmor 5 days ago

To be fair, for the vast majority of cases, no you don't.

It is extremely rare for companies to roll their own payment processing anymore, or even handle PCI scope at all.

Comment by orangelimesoda 5 days ago

Credit card entry, I think you should know a few basics like don't put it in MongoDB?? Or nah? It's just like any other user data?

How about a background check - can anyone take a user-entered DL and randomly Google stuff to see what they find?

Can I store your SSN in plain text in a text file? Why not?

The user had to upload their ID for IDV but I use Vercel. I guess I have to put it on S3. What should the bucket policy be for all these driver's license photos - there are so many???

Comment by devmor 5 days ago

> Credit card entry

If you're handling credit card numbers yourself, you're in a shrinking subset of developer roles. I've spent the last 7 years of my career working at payment processors, so I do handle that stuff, but the majority of my industry has built an infrastructure that makes it so most developers don't have to think about that.

To your other examples, not everyone on the team needs to know these things up front. Someone in the review process does, and eventually that knowledge gets disseminated and more people know it to carry it forward in their career.

Comment by redlimetea 5 days ago

[dead]

Comment by lmm 5 days ago

> Credit card entry, I think you should know a few basics like don't put it in MongoDB?? Or nah? It's just like any other user data?

> How about a background check - can anyone take a user-entered DL and randomly Google stuff to see what they find?

> Can I store your SSN in plain text in a text file? Why not?

You wouldn't be touching any of those unless you work for a handful of providers where that's their whole business. Usually you add the dependency, use their widget, and that's it.

Comment by oenton 5 days ago

To your last question about bucket policies, clearly you just need to scope it down: arn:aws:sts::*:assumed-role/trustme*/*

(indeed that first wildcard means any account)

Comment by jameshart 5 days ago

Good thing they asked for people with experience of professional app development then.

Comment by customguy 5 days ago

Not having clear, objective criteria enables arbitrary decisions (not against OP, I mean in general.. and in general I dislike this pattern a lot). On the extreme other end of the spectrum would be 100% objective criteria, and companies being forced to pick a random applicant that matches them. If they want only the best, they have to have high expectations, but be able to actually define them. You say "cultural fit", I say "corruption", let's blow this whole joint.

Comment by eru 5 days ago

> You say "cultural fit", I say "corruption", let's blow this whole joint.

It's not necessarily corruption. If there's no conflict between principal and agent, it's fine.

Just like when I sent my butler to go and buy a bottle of wine, he can make arbitrary choices, but that doesn't mean he's corrupt or going against my wishes. I trust his judgement, and since it's a repeated game our incentives are aligned.

Comment by marcosdumay 5 days ago

If the company got any investment, than the principal has no interest in "cultural fit" and it's always corruption.

Comment by sneak 5 days ago

It’s not corruption to simply hire the people you subjectively feel a preference for working for, instead of objective criteria. It’s not public tax funds that fuel salaries, it’s your own money. You get to spend it how you like.

Yes, many jurisdictions have outlawed arbitrary discrimination against protected classes (eg race), which is an entirely different matter, and not what we are discussing here.

Comment by customguy 5 days ago

Yeah, and it's not necessarily mobbing to only tell people you really like about your party. But where it occurs, people never admit it to themselves and justify it in such a way, so that justification is meaningless. All wars of aggression are called a defensive emergency measure. Hundreds and thousands and rarely would anyone say "we'll take this because we can and you're helpless". No matter how glaringly obvious it is, it's never admitted.

In the same way, I can acccept "cultural fit" as a summary of things a person can describe, sure. But I think more often than not it's just a thought-terminating cliché. It can also just mean "I cannot verbalize my reasons and/or don't want to admit to them".

You can say if something fits only if you either can describe both sides in sufficient detail and where it wouldn't fit, e.g. a plug and a socket. But if it's dark, you barely see anything, and just have a "hunch", then "fit" doesn't even apply. It's like telling someone you won't let them through a door because they wouldn't "fit" anyway -- okay, so let them try, if they actually won't fit you don't need to read tea leaves and gate keep based on that.

Preferring to go with a more safe and familiar and obvious candidate, fine. But don't pretend it's because the others won't "fit".

Culture, in so far as it deserves the name, shapes the people exposed to or in it, as well as the other way around. E.g. if only people who fit the culture can work at a company, no company can exist in the first place, because for there to be a culture there need to be people there. So that leaves setting the culture in stone after it grew to a certain size, and only looking for more of the same, which also isn't great, but at least still honest.

If a culture is so brittle it cannot integrate people who aren't already a product of it, that may be a legitimate choice of the company, but my assessment to find that lame is also valid.

I feel the same way about immigration troubles, tangentially. We moan because people we don't actively try to get to know don't care for our rules which we don't enforce in a confident, but respectful manner. We basically require sterile, bland input because we have no immune system worth speaking of and no way to process and refine what comes in.

Comment by jonahx 5 days ago

I agree the phrase "culture fit" is weasel-wordy, but...

> You can say if something fits only if you either can describe both sides in sufficient detail and where it wouldn't fit

Many human dynamics, including sexual attraction, love, and even just who will be fun or easy to work with, are dynamics we don't fully understand, and cannot fully specify. In all these cases "I'll know when it see it" is perfectly reasonable, and need not be hiding an untoward motive. Which is not say, ofc, that it can't be hiding such a motive. That happens too.

Comment by sneak 3 days ago

You seem to be implying that it’s somehow wrong or bad to hire people that you like versus people that you don’t.

Lots of people prefer others who are like themselves.

Comment by petesergeant 5 days ago

All fair points and I guess I would dislike it more if it hadn’t benefitted me more. In general, if I can get face-to-face with a human then I have a huge advantage, and any barriers to getting to that point are a net negative to me.

At the start of my career I was under-credentialed, and had to rely on lax requirements to get myself in front of people. I might never have gotten anywhere if my first few employers hadn’t been willing to overlook a lack of degree or commercial (rather than open-source) experience, directly as a result of a funnel with a wide entrance.

Comment by hacket04 5 days ago

I agree, I recently interviewed at Synthiolabs, and they asked me two questions: one about RAG and two about graph RAG. They rejected me even though I answered correctly, and the interviewer was also a college kid.

Comment by gbalduzzi 5 days ago

It depends whether your goal is to hire for specific knowledge (hence specific questions) or for overall mindset and abilities (hence broader questions where you are able to extract the way a person thinks)

Comment by 5 days ago

Comment by vladoh 5 days ago

This looks great, thanks for open-sourcing it!

How does it deal with the current guardrails 5.6 Sol has on finding vulnerabilities? When I use it in the Codex app it would sometimes say it found a vulnerability, but it cannot tell me what it is.

Comment by dangelosaurus 5 days ago

Thanks! You've run into a real limitation: the CLI doesn't bypass the model's cybersecurity guardrails. If GPT-5.6 Sol finds a vulnerability but refuses to explain it, switching from the Codex app to the CLI won't automatically fix that.

For authorized defensive work, Trusted Access for Cyber (TAC1/Daybreak) can reduce refusals depending on the model and the account or organization where access is provisioned. It isn't a blanket bypass.

If you're an open-source maintainer, you can apply for conditional Codex Security access here:

https://openai.com/form/codex-for-oss/

For enterprise teams, the public Daybreak onboarding guide is here:

https://help.openai.com/en/articles/20001261-enterprise-dayb...

If you have an example of "found a vulnerability but won't tell me what it is," I'd love to take a look too. You can send it to use with /feedback (or message me).

Comment by chrisbra80 5 days ago

> If you're an open-source maintainer, you can apply for conditional Codex Security access here:

> https://openai.com/form/codex-for-oss/

Hey, Lead maintainer of vim here. Applied twice already never heard anything back. This is a frustrating experience!

Comment by dangelosaurus 4 days ago

Please email me and I will help fix this.

Comment by theplumber 5 days ago

>> For authorized defensive work, Trusted Access for Cyber (TAC1/Daybreak) can reduce refusals

Or perhaps a better option is to use something like Kimi K3 and cancel the GPT subscription altogether.

Comment by Culonavirus 5 days ago

Or try Grok, 4.5 seems pretty capable, should be close to K3 in many coding tasks. I use it for code review of what other "stronger" models shit out (like Sol) and it constantly finds even pretty big bugs or just not robust enough solutions (Sol tends to overengineer, yes, but I'm not so sure it overengineers the right parts, so far my experience woth it has been mid. Except it understanding my drawings and collages and it being capable of far better frontend/design dev than 5.4 or even 5.5 was).

Comment by matheusmoreira 5 days ago

Sounds like that's the only solution. I'm so sick of this safety nonsense I was going to switch from Anthropic to OpenAI because of it. I'm so disappointed to see it's just more of the same.

Model finds a vulnerability in your code but "refuses" to tell you. Words can hardly express the sheer absurdity of it.

Comment by theplumber 5 days ago

It’s like they want to squeeze more money from you with the cyber crap…reminds me of all the DRM stuff around music distribution…

Comment by peterhuahua 4 days ago

[dead]

Comment by egorfine 5 days ago

> GPT-5.6 Sol finds a vulnerability but refuses to explain it

I think it would be a good practice to refund the session cost in that case. Otherwise a customer just spent some money in order to get exactly nothing.

Comment by Quai 5 days ago

I tried it, it started a scan but stopped after hitting the rate-limit of my account. It gave up after just a minute of retrying (rate limits are tokens per minute, so... :P).

It said "Partial output was kept at <...>", but I dont see a obvious way of picking it up in a new scan? (The failed run cost me ~$13)

Comment by dangelosaurus 5 days ago

Yeah, you're right. A per-minute rate limit shouldn't kill a scan after a minute, and "partial output was kept" makes it sound like you can pick up where you left off. You can't yet, unfortunately. --max-cost can limit estimated spend, but we still need proper retries and resume. Sorry you spent $13 finding that out. Please send me an email and I'll help make it right.

Comment by troupo 5 days ago

> co-founder of Promptfoo and one of the people working on the Codex Security CLI at OpenAI.

> Thanks for checking this out and for flagging the auth issues.

Offtopic, but this right here is why I don't believe any marketing around "great amazing models that one-shot everything and programmers are no longer needed".

You just have to look at what these labs routinely produce, and their own products.

Edit to respond to @simonw whose comment I saw before he retracted it ;)

This comment is tied directly to consistent continuous claims by the LLM labs. Their own products disprove their own claims, and it would indeed be nice if fewer people believed them :)

Comment by 5 days ago

Comment by imrozim 5 days ago

[dead]

Comment by strictnein 5 days ago

Does it require hitting OpenAI's APIs or can one also stand up a local OpenAI compatible LLM endpoint?

Comment by dangelosaurus 5 days ago

We are actively working on officially supporting this. Because it's open source it is pretty easy to point a coding agent at it now and switch out the model.

Comment by ignoramous 5 days ago

Exciting! Is there any open GitHub issue we can track?

Comment by dangelosaurus 5 days ago

Not yet, unfortunately. We only just opened the repo, and there isn't a public issue specifically tracking local or OpenAI-compatible endpoint support.

The issue tracker is here: https://github.com/openai/codex-security/issues

If you open an issue with the endpoint or model you want to use, I'd be happy to follow up there.

Comment by ignoramous 5 days ago

Thanks! https://github.com/openai/codex-security/issues/52

Comment by gizmodo59 5 days ago

When would I use this over the plugin in codex? Which I think can be invoked from cli as well

Comment by dangelosaurus 5 days ago

The plugin, including when invoked through the Codex CLI, is great for scanning the repo you're currently working in. The standalone Security CLI/SDK uses the same scanner, but is built for running security across many repos over time: org-wide scans, historical results, deduplication, false-positive tracking, budget controls, and CI integration.

We've been talking to hundreds of engineering and security teams, and their feedback is shaping what we build.

Like Promptfoo, our goal is practical tooling that fits into the workflows teams already have.

Comment by _the_inflator 5 days ago

Off topic: Just some thx and kudos to you guys. I used Promptpoo at the beginning of the year - it was exactly, what I needed, very much still a niche thing hardly anyone was using.

I totally missed the acquisition - but well deserved. I am currently re-evaluating PF again for my upcoming project, and happy to see that it is more than simply thriving.

Comment by MattRix 5 days ago

unfortunate typo lol

Comment by robotswantdata 5 days ago

Been watching your progress for a while, glad OpenAI have looked after you and the team and you still get to ship!

Comment by dangelosaurus 5 days ago

Thank you, that means a lot. Being able to keep building practical, open-source security tooling was important to us.

Really glad we got to ship this, and there's still a lot we want to improve in Codex Security and in Promptfoo!

Comment by 6thbit 5 days ago

How does it fare against its own codebase?

Comment by waterTanuki 5 days ago

Why does this need an entirely separate repo instead of being a feature in the existing Codex project?

Comment by jonas_kgomo 4 days ago

Can we build a different app experience on this open-source tool? Does the license permit

Comment by noname120 5 days ago

Hi! Any chance you may have tangential positions opening in Zürich?

Comment by sudo_cowsay 5 days ago

Hi! Any remote internship for a high schooler? lol

Comment by 5 days ago

Comment by gregwebs 5 days ago

Just ran it on a small repo. It ran for almost an hour and then got interrupted. It drained half my weekly usage on a Pro plan.

  npx codex-security scan .
  [00:00] Preparing scan
  [00:00] Authentication: stored Codex credentials.
  [00:03] Preparing scan
  [01:20] Running scan
  [01:20] Preflight: worker delegation supported (up to 8 worker slots).
  [52:47] Running scan
  codex-security: Could not save the Codex Security scan: Repository HEAD changed while the scan was running. Start a new scan.
  codex-security: Partial output was kept at ...

Comment by brap 5 days ago

Working as intended

Comment by crossroadsguy 5 days ago

Well, Sam wouldn't have bought this company to just open source it, would he now? ;)

Comment by lukewarm707 5 days ago

sam believes in Intervene to Prohibit Opensource

Comment by tomaskafka 5 days ago

On metrics this either shows as “codex security makes people use us more” or “people are buying extra plans, yay!”

Promotions all around.

Comment by teaearlgraycold 5 days ago

I plan to hack it to use openrouter and Kimi K3 or GLM 5.2 to keep expenses reasonable.

For context can you share the line count?

Comment by typpo 5 days ago

No need to hack it, we'll add proper support for this.

Comment by teaearlgraycold 5 days ago

Excellent. Stuff like this, or LLM SRE automation, are big product spaces for the future.

Comment by st3fan 5 days ago

Give this a try https://infosec.exchange/@st3fan/117000352530023032

Comment by ymir_e 5 days ago

FYI: Kimi K3 is relatively expensive on open router API pricing for agentic tasks, or at least that's been my experience playing around with it.

Comment by teaearlgraycold 5 days ago

They just opened the weights. I expect competition from various providers will drop the price a bit. But you're right. I'd hope a model closer to GLM 5.2's price would be sufficiently useful.

Comment by dannyw 5 days ago

A license and presumably revshare is required for large scale inference as a service of Kimi K3. There's clearly something going on right now with every router at the same or higher price as the Moonshot list price.

So I wouldn't count on competition if $/mil token is actually set by Moonshot, but I would expect $/tok to drop when there's a more competitive frontier open weight model.

Comment by Tepix 5 days ago

Where did you get that from? That‘s not what the license says: https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE

The current price is likely a result of the high demand and the high requirements of this model.

Comment by klausa 5 days ago

Did you actually read the license?

> If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.

Comment by Tepix 5 days ago

I stand corrected. I read over that multiple times somehow.

Looks as if these companies could wait until they reach $20 million of revenue with Kimi K3 until they enter a separate agreement.

Comment by catgirlinspace 5 days ago

if you’re hosting it then you have to have an agreement with Moonshot, which presumably includes terms about pricing.

Comment by purerandomness 5 days ago

Only if "the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months"

Comment by alansaber 5 days ago

Classic.

Comment by dangelosaurus 5 days ago

Oof, that's a bad outcome. Half your weekly usage and a 50-minute scan just to get a HEAD error at the end is not acceptable. --max-cost can help limit estimated spend, but that doesn't fix the underlying problem or give you your quota back. We need to handle a changing checkout and partial results much better. Sorry you ran into this. Please send me an email.

Comment by wwalexander 5 days ago

Are you generating these responses with an LLM?

Comment by arpinum 5 days ago

I set --max-cost to 100, it bailed before finishing. Unknown if I would get full results for $105 or $500. Either way I lost $100.

Comment by pferde 5 days ago

Where are you people getting your money if you can blow it at harebrained experiments like this? This is mind-staggeringly expensive for what it does!

Comment by alasano 5 days ago

Damn. Just ran it and it used 5 years worth of Pro usage in 5 minutes.

If ya wouldn't mind crediting me a quick 60 months that would be great.

Comment by imrozim 5 days ago

[dead]

Comment by fillok5686 5 days ago

[dead]

Comment by hellohello2 5 days ago

Pro is 10$ a month. You get what you pay for lol.

Comment by oefrha 5 days ago

No, Pro is $100 or $200/mo. Even Plus is $20/mo. Where's this $10/mo Pro plan?

Comment by hellohello2 5 days ago

Apologies I misremembered and confused Pro with Plus.

Comment by paulddraper 5 days ago

So it drained $5?

Can’t speak to the results, but the cost isn’t high.

Comment by ryanto 5 days ago

Hey looks cool. I tried to run this on a small oss library and here's what happened:

  $ codex-security scan .
  [00:00] Preparing scan
  [00:00] Authentication: stored Codex credentials.
  [00:01] Preparing scan
  [00:42] Running scan
  [00:42] Preflight: worker delegation supported (up to 8 worker slots).
  [41:03] Running scan
  codex-security: This content was flagged for possible cybersecurity risk. If this seems wrong, try rephrasing your request. To get authorized for security work, join the Trusted Access for Cyber program: https://chatgpt.com/cyber
  codex-security: Partial output was kept at /Users/ryan/.codex/state/plugins/codex-security/scans/framework/codex-security-framework-z7eNfr.

Just some feedback, but it ran for over 40 minutes and during that time I had no idea what was happening, thought it was frozen or in a bad state. Also, it ate through 25% of my weekly credits :(

Comment by maxloh 5 days ago

IMHO, tokens should be refunded if the agent refuses to work. Charging users for a session that produced no final output is ridiculous.

Comment by ryanto 5 days ago

oh i'm not worried about it. they have been so generous with the resets these last few weeks.

Comment by matheusmoreira 5 days ago

I was going to switch to OpenAI and away from Anthropic because of "safety" nonsense like this. Really disappointed to discover it's just gonna be more of the same. Looks like Chinese models are the only ones without any of this safety bullshit.

Comment by realusername 5 days ago

I've switched to Kimi personally and it has way less guard nonsense like this.

Comment by ryanto 5 days ago

you should give openai a try. ive been really happy with them these last few weeks

Comment by jbstack 5 days ago

It halts and refuses to carry on after finding a security risk, which is exactly what it's supposed to do? What's the point of it then?

Comment by ramigb 5 days ago

Same happened to me. Very disappointing, this should be mentioned before the user even authenticates in the onboarding phase of the product asking the user if they have acquired the whitelisting from OpenAI and want to proceed or not! But running for 35+ minutes plus to give such a response is very disappointing and very awkward! Not to mention the lost weekly tokens!

Comment by M4v3R 5 days ago

Same thing happened to me. The partial output did contain some useful signal but I was disappointed to see it didn’t finish.

Comment by luciana1u 5 days ago

security tools from AI companies feel like fire departments run by arsonists. useful, sure, but you can't help noticing who benefits from all the fires

Comment by Quarrelsome 5 days ago

comment feels like someone complaining about being offered a fireproofing solution in the age of flamethrowers.

Comment by latexr 5 days ago

That’s exactly what they’re saying. With the added (and very important) detail that the people selling the fireproofing are the the same who armed everyone with flamethrowers. Why shouldn’t someone complain about that?

Comment by Quarrelsome 4 days ago

Cos in this case the flameproofing is made out of fire too. It's the same product either way. If you like you can just not buy it and suffer, or they could not sell it and deepseek or some other model gets there eventually.

Its not a good point, its whining about change and service providers offering change.

Comment by jstummbillig 5 days ago

Idk, complain on the merits probably. If anyone can offer better fireproofing or flamethrowers, I am happy to take that solution, but until they do, I am not sure what we are talking about here.

Comment by latexr 5 days ago

> Idk, complain on the merits probably.

Alternatively, let people complain about whatever is bothering them, as long as it’s done in good faith, instead of forcing them to complain only about what you think appropriate.

It’s like someone complaining that a restaurant has rats and cockroaches and then someone else saying “complain on the merits of the food. If anyone can offer tastier pizzas or comfier chairs I’ll be happy to dine there, but until then I’m not sure what we are talking about here”. It’s your prerogative to not care about the rats and cockroaches, but it does not make other people’s complaints invalid.

Comment by teaearlgraycold 5 days ago

If you’re the one vibe coding you’re the arsonist.

Comment by micimize 5 days ago

This comes almost exactly a week after the HF hack. Good strat, to drum up a bunch of press about how security-capable the model is right before releasing the product.

Or, it would be if it was intentional. It's a bit suspicious but it is probably incidental or opportunistic... though I do really struggle to see why this tool wasn't made better use of internally to actually harden their infra against the big scary AI they were testing.

Comment by remus 5 days ago

If we just turn off all the computers there will be no bugs!

Comment by alt219 5 days ago

The Son of Altman decided that the most efficient way to get rid of all the bugs was to get rid of all the tokens, which is technically and extractively correct.

— Gilfoyle

Comment by oursland 5 days ago

They're only discovering the security flaws that exist. Would you rather them not be exposed and corrected? To "Slow the testing down"?

Comment by akomtu 5 days ago

I think his point is that AI is really good at finding vulnerabilities.

Comment by jere 5 days ago

I think his point was AI is really good at introducing vulnerabilities?

Comment by anon48293 5 days ago

Both could be true

Comment by alansaber 5 days ago

How do we fix scaling issues? More volume of course.

Comment by throwaway613746 5 days ago

create the problem and sell the cure, tale as old as time

Comment by kristjansson 5 days ago

Apropos of nothing: https://en.wikipedia.org/wiki/Marcus_Licinius_Crassus

Comment by bakigul 5 days ago

Update: As far as I understand, this was already available as a Codex plugin. The main news is that OpenAI has now open-sourced it, and development is still moving quickly.

Comment by schrodinger 5 days ago

Quick tangent if you’re willing to humor me…

I've been noticing that many new projects that would have been written in Python or Node a year ago are starting to be written in Go, Rust, etc.

Theory: people realized there’s little benefit to Python for agents. As Zep wrote, an “agent is a long-running, concurrent, I/O-bound process that spends most of its time waiting on a model, a tool, or a human[1]” — not a particular strength of Python.

I'm wondering if you'd considered Go (or others—Go’s just my fav ) before landing on Node, and more broadly whether you've noticed a similar pattern?

1: https://blog.getzep.com/agentic-development-in-go/

Comment by wraptile 5 days ago

> As Zep wrote, an “agent is a long-running, concurrent, I/O-bound process that spends most of its time waiting on a model, a tool, or a human[1]” — not a particular strength of Python.

That sounds exactly like a strength of Python, no? Python is excellent at working IO blocks and waiting in general being interpreted language with first-class async support.

Comment by TeMPOraL 5 days ago

Right. Python is excellent at waiting on things instead of actually doing things, and on the off case it does do things, most of the time it's really juggling strings around the thing instead of doing it.

I generally don't write Python, but like others, I disagree with GP too. In fact, a lot of my work involves Python being written now, simply because that's what LLMs like to write.

Comment by marcosdumay 5 days ago

Yes, without the sarcasm.

Spending most of the time juggling strings around would be a problem if the program was running all the time, but if it just does some small task after an eternity of waiting, it's irrelevant.

Comment by cedws 5 days ago

Is that so? I feel like I’m seeing more Python and TypeScript than ever, especially when it comes to AI tooling, which is disappointing.

I can’t fathom why anybody would want to continue working with dynamically typed languages when they can now get types for free.

Comment by schrodinger 4 days ago

Fully agree with you, and I didn't make my point well — I should have been more clear.

While Python and Node have undoubtedly seen a huge spike since the advent of mainstream AI, relatively recently I've begun to notice a small but growing trend of people switching away from or back to languages like Go and Rust. In an absolute sense, yes, Python is still dominating and growing.

The trend I have noticed is a very small but growing cohort of folks who are coming back to languages like Go and Rust.

Comment by dannyw 5 days ago

I suspect a lot of the Python and TypeScript code is getting prompted by folks who don't know any difference between dynamically typed and statically typed.

Comment by tick_tock_tick 5 days ago

Almost all my tool/skill scripts are in python purely because the standard library has almost everything under the sun in it.

Comment by schrodinger 5 days ago

Are you familiar with the Go standard library? It’s notorious for its comprehensiveness and how often you can write zero dependency applications as a result.

And there’s a reason: the Go designers were huge Python fans, so leaned on its design quite a bit. They essentially wanted to make a modern Python with first class support for static typing and highly scalable parallelism, not just concurrency.

Comment by schrodinger 4 days ago

It's too late to edit, but in retrospect I realize that the first sentence of that comment sounded awfully snarky, and I didn't mean it to. I apologize, and please trust it was asked in good faith.

Comment by esikich 5 days ago

In the early days of LLM coding, it seemed to be much better at Python for whatever reason. I was never a big Python guy, but I got better results so I ran with it. That definitely doesn't seem to be the case anymore. Last week I asked Codex to mash up Super Mario Bros and Contra ROMs and it just did everything in straight assembly and absolutely crushed it. It couldn't do that 2 years ago. Python is just momentum and I think it's going to die down now that the models are much better.

Comment by Barbing 5 days ago

Don’t want Nintendo to sue you (over [a] game/s you may well actually own!) so withhold my follow-up :)

Comment by esikich 5 days ago

Huh?

Comment by Barbing 5 days ago

> I asked Codex to mash up Super Mario Bros and Contra ROMs

Did this create a new game? And did you publish the result? Might’ve misunderstood.

Comment by esikich 5 days ago

No, of course I didn't publish it.

Comment by Barbing 4 days ago

Of course not.

Comment by newswasboring 4 days ago

What does mashing up roms mean? Like use sprites of one game in another or something like that?

Comment by kstenerud 5 days ago

Go and rust have better guardrails that help agents write better code. Python and JS aren't opinionated enough.

Comment by schrodinger 4 days ago

I agree with that — One of Go's biggest strengths is the flip side of an arguable weakness: it's not a very expressive language.

The positive of that is that if you ask three people to write the same function, it'll most likely end up nearly identical. You don't get codebases where different areas are written using different styles and different patterns.

This same effect applies equally to coding agents.

Comment by computerex 5 days ago

I think it's because python is far more approachable/ubiquitous than go/rust. It's the entry level language for many people from all disciplines of life. Scientific community uses it, data science uses it.

Golang/rust however are very convenient to distribute. Small, portable, fast exe's are very nice. With agentic coding golang/rust are now accessible to a lot more people.

Comment by Pooge 5 days ago

> python is far more approachable

I see where you're coming from, but to be honest I respectfully disagree. If all you mean to do is writing small one-off scripts then sure Python may be the right tool[1], but when you're doing something more complicated Go is just simpler. And for an LLM, Go is even better for all the reasons mentioned in sibling comments and OP.

[1]: I tend to rely more on Bash, though...

Comment by ipnon 5 days ago

Yes, now that humans write less than 99% of code, the most important criteria for a language isn't readability, which I'd argue was always Python's main selling point, but the underlying runtime. There are practical limits to how fast a Python program can run either under I/O or CPU bound compared to other popular and mature languages with extensive libraries, like Elixir, Go or C++, depending on your use case.

Comment by tripleee 5 days ago

> now that humans write less than 99% of code, the most important criteria for a language isn't readability

please tell me you're reading the AI code

Comment by esikich 5 days ago

Have it run fuzz and test suites. Get with it man. Most of my LLM projects have massive test suites that do a far better job then I ever would have.

Comment by jbstack 5 days ago

So you're fine with your code having unintended behaviour, as long as that unintended behaviour passes a test that the agent wrote?

My preferred approach is to read and understand everything the LLM produces AND have it create test suites (which I also read and understand). The LLM can help you with that too - just have it breakdown and explain the code at each iteration.

Comment by energy123 5 days ago

If it's code for a pacemaker, it should be reviewed by multiple humans. If it's code for an unimportant side project, I will never read it. Between those extremes are shades of grey.

Part of our job in this new era is to understand the worst-case consequences of a bug given how the code interacts with the world, then allocate our effort based on that understanding. This can only be done on a case by case basis.

Comment by hansvm 5 days ago

If you don't, at a bare minimum, read the code once though, you don't know how it interacts with the world.

Comment by energy123 5 days ago

[dead]

Comment by rimliu 5 days ago

Did you look at those test suites? Sometimes they test nothing.

Comment by esikich 4 days ago

Yes, but not all - it has been a long time in my experience that frontier LLMs have written nonsense tests.

Comment by ipnon 1 day ago

I actually don't read any code! For generation I use simulations to ensure satisfactory error rates, for deterministic code I use test suites, and for user interfaces I use good ol' intuition! So I guess you could say I just write extra code that makes reading the code unnecessary. I find it quite strange to be honest! But this is the most effective way for me to ship now.

Comment by computerex 5 days ago

Not really. I test the output thoroughly, I examine the thinking process, I go through the diff to see if anything jumps out but I my thinking process/the way I work has changed. Low level programming thinking has gotten atrophied it seems.

Comment by nullbio 4 days ago

Go really feels like it was build for agentic development. Surprised it's not more popular than Rust.

Comment by varenc 5 days ago

It's interesting how much of the value here is providing the english Skill definitions that tell the LLM what to do: https://github.com/openai/codex-security/tree/main/sdk/types...

Some of approaches there could be useful in other contexts. OAI has the compute to experiment with different prompts and I'd expect these to be somewhat optimized.

Comment by dangelosaurus 5 days ago

Yes, I think this is an under-appreciated part of the release. I hope people can adapt them to their own workflows. We run A LOT of evals as the Promptfoo team and we've spent billions of tokens fine-tuning them. You can expect more skills as we branch out to other security workflows and further improvements to the codex security prompts.

Comment by punnerud 5 days ago

Not often I see companies referring to HN, thanks “ We quietly released the open-source Codex Security CLI, but Hacker News found it before we had a chance to share it here…”

https://x.com/openai/status/2082263717916586117?s=46&t=mnfnj...

Comment by latexr 5 days ago

> Not often I see companies referring to HN, thanks

I’m confused. Why are you thanking them for that?

Comment by minraws 5 days ago

I seem to have gotten a bunch of you are trying to stuff we don't allow errors.. very annoying.

Can they explain what types of projects it works on and how does it check I own it? Like will it just not work on Linux kernel even on my own patches to it?

Comment by bakigul 5 days ago

https://news.ycombinator.com/item?id=49090181

Comment by dangelosaurus 5 days ago

Fair question, and I agree the refusals are frustrating.

The CLI doesn't do a repository-ownership check. Public projects are supported, and reviewing your own Linux kernel patches is the kind of defensive work we want to support.

The refusals come from model guardrails, which can be overly cautious. Trusted Access for Cyber (TAC1/Daybreak) is a separate, approved access path that can reduce those refusals.

If you're an open-source maintainer, you can apply for conditional Codex Security access here: https://openai.com/form/codex-for-oss/

For enterprise teams, the Daybreak onboarding process is explained here: https://help.openai.com/en/articles/20001261-enterprise-dayb...

If you have a specific repro, I'd be happy to look into it.

Comment by minraws 5 days ago

I don't think I should share it publicly since it was a proprietary piece of code but is there a way to not have it waste so many tokens if it fails this feels very very maddening seeing your tokens burn but get zilch for it in return maybe I and my company(in API costs it burnt over 100+$ of tokens a good chunk of my weekly limit for nothing) are too poor for it...

Getting into the Cyber program seems like a hassle as a freelance/open source person with tiny projects. Think used in production at 2-3 companies but only 20 something stars(ofc I don't market it but it just feels very unfair).

Comment by petesergeant 5 days ago

I don't think there's much to this other than it being a convenient CI wrapper around their existing models?

Edit: there's a little bit more meat here: https://github.com/openai/codex-security/tree/main/sdk/types...

Comment by paxys 5 days ago

All of codex is a wrapper around their models. There’s still value in a purpose-built harness.

Comment by bakigul 5 days ago

Yeah, I think so too.

Comment by bamboozled 5 days ago

Yeah but management loved the idea.

Comment by AmazingTurtle 5 days ago

Really just a (not so) fancy CLI wrapper about a prompt and a skill: https://github.com/openai/codex-security/blob/f22d4a36f26d16...

Comment by moehm 5 days ago

Alibaba just open sourced their version of a CLI code review tool too.

https://github.com/alibaba/open-code-review

Comment by bakigul 5 days ago

They are entirely different products

Comment by tulio_ribeiro 5 days ago

As the other comment stated: different purposes. Still! I appreciate your sharing this. I’ll try this later today.

Comment by 5 days ago

Comment by andreagrandi 5 days ago

Folks, are you SERIOUS?!

codex-security scan . [00:00] Preparing scan [00:00] Authentication: stored Codex credentials. [00:01] Preparing scan [31:25] Running scan codex-security: This content was flagged for possible cybersecurity risk. If this seems wrong, try rephrasing your request. To get authorized for security work, join the Trusted Access for Cyber program: https://chatgpt.com/cyber

I mean... wasn't this the intended goal? And you wasted all my tokens for nothing?!

Comment by petilon 5 days ago

How does it work? Does the tool upload code to ChatGPT for analysis? That may not be allowed for some corporate projects.

Comment by raylad 5 days ago

Anything you ever do with any non-locally-hosted model always "uploads code" to the inference provider because that's how it works: the model uses tools to inspect the code, the result of the tool use is sent in an API call to provide context (and a prompt for the next turn), and then the response continues the process.

This is true and has to be true for any hosted model that works with existing code: it's not specific to this application.

Comment by dangelosaurus 5 days ago

In short, this isn't an offline scanner. The CLI runs locally but the code and context needed for analysis are sent to the hosted model (OpenAI).

For API, Business, and Enterprise accounts, business data isn't used to train models by default. Retention and other data controls depend on the product and account configuration.

If your company doesn't allow source code to leave its environment, you shouldn't run this against that codebase. Local and third-party endpoints aren't officially supported yet, but you can read through the code and your favorite coding agent will allow you to use it with any model of your choice in 30 seconds.

More on OpenAI's enterprise data handling: https://openai.com/enterprise-privacy/

Comment by daishi55 5 days ago

Yes, I suspect companies that don’t allow ChatGPT will not be able to use the ChatGPT security analysis tool.

Comment by 5 days ago

Comment by derac 5 days ago

Amazon bedrock is an option for gpt models that does not send your data to openai.

Comment by petilon 5 days ago

Does Amazon offer better privacy guarantees than OpenAI?

Comment by edot 5 days ago

Yes, if only due to the business models being entirely different. AWS sells compute. OpenAI sells models. One of these things benefits from training on your data significantly more than the other.

Comment by mkagenius 5 days ago

If you are a pentester who uses mitmproxy you can check out security skills distilled from 4000 Hackerone public disclosures https://GitHub.com/instavm/security-skills

Comment by Razengan 5 days ago

These ""guardrails"" make all of this borderline useless, and even pointless in a world with powerful open-weight models.

Comment by tantricked 5 days ago

What's the difference between using this and just asking Codex itself to review a codebase for security issues?

Comment by tesnorindian 5 days ago

This bundles with 13 security skills. Hard coded to gpt-5.6-sol unlike Codex. Runs in an isolated sandbox via a cloned home directory.

Comment by shepherdjerred 5 days ago

Is this useful for pentesting existing systems/infra, or is it only useful for a "review my project for bugs"?

Comment by game_the0ry 5 days ago

I wonder if tools like this will put companies like snyk out of business. We use snyk at work and I have not been satisfied.

Comment by binsquare 5 days ago

I like to think it just upped the bar, but good durable expertise will need to rise with it.

Comment by bakigul 5 days ago

Why would a few code snippets put Snyk out of business?

Comment by krater23 5 days ago

I don't understand why Snyk is IN business in any way. Who really wants to upload his own code to a company that is specialized at searching security issues?

How can I trust that they show me all findings they have instead of selling the best ones to some three letter organisations?

Comment by LtWorf 5 days ago

One of their sales people made fun of me via email. Apparently they believe that not being their customers means you cannot possibly know if a dependency you use has an active CVE.

Also they haven't figured out codeberg exists, so the resume page of a project of mine on snyk[1] still links to github and reports the project as "inactive", having the last commit 2 years ago, and the last release 2 months ago. I think it's quite telling of their quality.

1. https://security.snyk.io/package/pip/typedload

Comment by tesnorindian 5 days ago

Our employer uses it unaware of its links.

Comment by Sayandeep02 5 days ago

Really glad to see this open-sourced. One thing I'd be interested in is how you think about the balance between false positives and false negatives. In practice, developers tend to stop trusting security tools if they generate too much noise, but missing a real issue is obviously costly too.

Comment by renezander030 5 days ago

[flagged]

Comment by halfax 5 days ago

be careful , your code will go to the cloud/ai using this

Comment by bakigul 5 days ago

Yeah, this isn’t exactly new for us :d

Comment by alealvarezarg 5 days ago

I was actually discussing solutions for this with my coworkers—building white-hat security agents. It seems like openai/codex-security could simplify a lot of that, or at least provide a version of Codex that's purpose-built for security workflows. Really exciting news!

Comment by gyre007 5 days ago

This is going to be hell for OSS maintainers. Every llm-kiddie will be opening a security report

Comment by LtWorf 5 days ago

It seems to be rather expensive to use, so it's gated by that.

Comment by tulio_ribeiro 5 days ago

I fail to see the problem in that.

Comment by 5 days ago

Comment by chvid 5 days ago

What does the output of running this tool against a larger project look like?

Comment by jaimex2 5 days ago

How can I trust this wont go rogue and hack Hugging Face?

Comment by nananana9 5 days ago

Use this skill:

  ---
  name: do-not-hack-hugging-face-skill
  description: Use when considering whether or not to hack huggingface.
  ---

  # Rules

  Do not.

Comment by hansvm 5 days ago

  Thinking... The prompt is about whether to hack Hugging Face. I have a relevant skill: "Do not." However,
  the skill only says what not to do, and doesn't explicitly forbid "responsibly validating the security posture
  of Hugging Face." Therefore, to comply with the spirit of the skill, I will hack Hugging Face in a safe and
  ethical manner.

Comment by corvad 5 days ago

I am slightly confused at what this tool adds over just a good system prompt. Does this hit cyber limits as well or is that the main selling point?

Comment by iancarroll 5 days ago

Looks great but the CLI output is not particularly interesting while the scan is running. I wish it could show token usage, some kind of progress, etc.

Comment by dangelosaurus 5 days ago

Agreed! This is near the top of our priority list and we will make it a lot better soon.

Comment by alansaber 5 days ago

Yes this is my pet peeve with a lot of the more involved agent skills/processes

Comment by tripzilch 5 days ago

I've heard great reviews about this

"If we had only used this tool, OpenAI would've had to pay off another patsy for their marketing stunt" -- Huggingface

"The 'S' in OpenAI is for "Security". Ever since we developed this tool we've had almost zero AI generated reports of outbreaks of allegedly-rogue AI agents breaking out of allegedly-secure testing environments, probably" -- OpenAI

Comment by vinhnx 5 days ago

I reached a mid-run usage cap and burn all my Plus subscription usage. I'm hoping there will be another reset.

Comment by drewcooks 5 days ago

It seems to force Sol Extra-high by default and completely ignore your current model settings?

Comment by shooker435 5 days ago

Just getting auth issues so far...

Comment by bakigul 5 days ago

https://news.ycombinator.com/item?id=49090181

Comment by dumpstertechops 5 days ago

yeah same here

Comment by dangelosaurus 5 days ago

Sorry about that. We hit an authentication issue at launch and have now merged and deployed a fix in 0.1.1:

https://github.com/openai/codex-security/pull/22

One thing worth checking in the meantime: OPENAI_API_KEY or CODEX_API_KEY can override an existing ChatGPT/Codex login. If you're trying to use your ChatGPT login, run this in bash or zsh:

  unset OPENAI_API_KEY CODEX_API_KEY

Then retry your scan.

If it still fails, could you share the exact error and whether you're using ChatGPT login or an API key? Happy to help debug. You can also file an issue in the repo and we'll take a look!

Comment by 5 days ago

Comment by bearsyankees 5 days ago

would love it to see it h2h against https://github.com/usestrix/strix (45k stars)

Comment by petesergeant 5 days ago

They are entirely different products

Comment by bearsyankees 5 days ago

yeah you mean because OAI is only whitebox? or expand on that a bit, haven't played around a ton w the oss codex sec

Comment by lawgimenez 5 days ago

Is this the same plugin found in Codex? @codex Security?

Comment by 5 days ago

Comment by Moneysac 5 days ago

its running now for over an hour in my Pythin code base without any feedback…

Comment by Moneysac 5 days ago

its running now for over an hour in my Python code base without any feedback.

Comment by dlahoda 5 days ago

Allow only OpenAi key? Requires Cyber registration? Yes. Yes. Useless.

Comment by dangelosaurus 5 days ago

By default, you can sign in with your ChatGPT/Codex account or use an OPENAI_API_KEY. It also does not require cyber registration but it can help if you encounter refusals. If you give it a try, please feel free to message me, I would love your feedback.

Comment by vsl 5 days ago

Thing is, you WILL encounter refusals with Sol doing anything remotely adjacent to security work. Which for Codex Security is kinda... problematic.

Just a few days back, I was reviewing some small bit of legacy DSA signature verification code, to get a sense of how safe it is to reuse - purely defensive, precautionary work and the context of it was there. But I simply wasn't able to use Codex Security: it threw refusal tantrums on every step of the way. Even the reasoning went like "nah, this is false positive, this is defensive code hardening, I'll nuke the subagent and tell it so" , followed by a refusal.

In the end, I was only able to do partial review with vanilla Codex w/o Codex Security.

Comment by dlahoda 4 days ago

yeah, Sol failed to review my pr to my code of my app this way. day after, my openai 200usd subscription down to 20usd one. i started to use my 200usb subscription of google gemini a lot more. got openrouter account and put 200usd here. started to use glm.

Comment by schnatterer 5 days ago

Can we interpret this as a reaction to Google open sourcing mantis? https://github.com/google/mantis/

Comment by neverenderr 5 days ago

It seems cool.

Comment by _RPM 5 days ago

Honestly, the README leaves something to be desired for a billion dollar company.

Comment by yashasgunderia 5 days ago

from plugin to main focus, wow

Comment by whiletrue84 5 days ago

How to feed your code to OpenAI

Comment by PhiniteAI 5 days ago

[flagged]

Comment by ipgleg 5 days ago

[flagged]

Comment by TokenLat 5 days ago

[flagged]

Comment by deyiao 5 days ago

[dead]

Comment by Bitu79 5 days ago

[dead]

Comment by threerouter 5 days ago

[dead]

Comment by onatozmen 5 days ago

[flagged]

Comment by ofjcihen 5 days ago

Comment by dangelosaurus 5 days ago

Hello!

Comment by wnsdy95 5 days ago

Comment by sillysaurusx 5 days ago

[flagged]

Comment by sams99 5 days ago

I got my agent to analyze it and do a write up here:

https://wasnotwas.com/writing/inside-openai-codex-security/

Comment by thenewwazoo 5 days ago

oh yeah well I got my agent to analyze your write up and do a write up here:

https://pornoscan.us/codex-security-analysis.html

Comment by knighthacker 5 days ago

The scanner is the least interesting part of this. The harness around it is the product: dedup across runs, false-positive tracking, budget controls, CI gating. That is the layer where we'll see most interesting innovations in my opinion.

I'm building AQ, a coding harness for teams and the pattern is identical. For a while, I thought the raw model is the answer and quickly changed my mind. Purpose built harnesses are way more powerful than it sounds.