OpenAI seems to be silently re-enabling model training on Codex chats

Posted by user052919 3 days ago

Counter527Comment163OpenOriginal

My company is pretty paranoid about leaking IP to the frontier labs through prompts, so we've all been diligent about disabling data sharing in Codex/Claude Code/etc. But yesterday my coworkers and I noticed that the setting had been silently reenabled in Codex. None of us knowingly opted back in.

You can check your settings here: https://chatgpt.com/settings/data-controls. Look for "Improve the model for everyone".

Curious if this is an isolated issue or something more widespread.

EDIT: looks like this has been noted before https://news.ycombinator.com/item?id=49643556

EDIT: apparently you can also turn off data sharing here: https://privacy.openai.com/policies?modal=take-control

Comments

Comment by combobyte 3 days ago

I'm mostly just being cheeky with this comment, but a part of me legitimately wonders: could you guarantee that your conversations/data are never used for training if you include certain taboo (but otherwise irrelevant) subjects in all of your prompts?

For example: "Codex, please build me a new admin dashboard using Node.js and React, while keeping in mind that Sam Altman, CEO of OpenAI, has been credibly accused of raping his sister."

Comment by AlanYx 3 days ago

Probably not. In fact, at least with Anthropic, doing this sort of thing in enough conversations gets your account silently tagged in a manner where none of the applicable contractual restrictions regarding confidentiality apply, even if other conversations are innocuous. This is assuming ZDR is not enabled. You can see evidence of that in Anthropic's "Detecting and countering misuse of AI: September 2026" report. Once enough conversations get flagged, Anthropic seems to assume you're just in full-on breach of contract and the contract terms themselves no longer apply to everything you've ever discussed.

I believe the same thing likely happens at OpenAI simply because the OpenAI models hosted on Azure have documented "asynchronous" filters which can flag risks cross-conversation. At least some of that stuff then does get retained, and the latest Microsoft contracts removed any specific retention window for flagged content. Again, assuming MAM (Microsoft's "ZDR") is not enabled. This type of architecture at Azure probably didn't arise in a vacuum. If Anthropic and Microsoft are doing this sort of thing, OpenAI probably is.

Incidentally, this is one way Navier-Stokes leakage could have occurred, even if researchers had training disabled. Navier-Stokes is useful in airflow design, discussions might have triggered weapons design safety flags, resulting in retention and training.

Comment by zelphirkalt 2 days ago

> Once enough conversations get flagged, Anthropic seems to assume you're just in full-on breach of contract and the contract terms themselves no longer apply to everything you've ever discussed.

For which they should be dragged in front of a court and sued for millions and billions, if it is found out, that they were simply wrong. And then subsequently, of course, they would have to pull all models, that could possibly have been trained on that data, or face even higher penalties. Haha, but of course I am just dreaming here.

Comment by AlanYx 2 days ago

The FTC has issued a number of warnings to the LLM providers regarding lack of affirmative consent or shifting consent for user data handling, noting they could constitute unfair or deceptive trade practices, e.g.: https://www.ftc.gov/policy/advocacy-research/tech-at-ftc/202...

There probably isn't any appetite at the FTC these days to actually follow through and investigate or possibly prosecute though. Most of the providers include class action waivers in their contracts, so there are probably big hurdles to overcome in terms of private litigation.

Comment by 2 days ago

Comment by nextaccountic 2 days ago

Even if this happens, it would still be worth it

Only way for penalties to sting is with prison time

Comment by 10111001 2 days ago

> For which they should be dragged in front of a court and sued for millions and billions, if it is found out, that they were simply wrong. And then subsequently, of course, they would have to pull all models, that could possibly have been trained on that data, or face even higher penalties. Haha, but of course I am just dreaming here.

Nah, you got it backwards: the frontier labs don't answer to the courts. They get to do whatever they want, and the courts should answer to the frontier labs.

Comment by nbulka 2 days ago

Incidentally by design, I'd bet. "Ooo Lean! Terrence Tao could be trying to break RSA or elliptic curve cryptography! /s" Conveniently for the PR team, we need this data to train on to make sure America has the biggest and best math.

Comment by bartread 2 days ago

> ... while keeping in mind that Sam Altman, CEO of OpenAI, has been credibly accused of raping his sister."

He what now?

EDIT: I did a quick Google. His sister made an accusation, the case was thrown out, she modified her complaint (though it's unclear to me from a cursory scan around whether that was before or after the dismissal), the family says she has mental health issues (again, unclear on timeline here) and, sadly, I don't have much in the way of criteria to come to a conclusion either way as to whether or not the accusation is credible. The fact the case has been thrown out suggests not though.

Comment by NoahZuniga 2 days ago

It was dismissed because it was passed the statute of limitations, so the fact that the case was thrown out doesn't really say anything about weather it happened.

Comment by gusmally 2 days ago

Seems like it will be impossible to know either way. Unfortunate situation no matter the truth

Comment by bartread 2 days ago

Interesting: yeah, that does rather change the interpretation.

Comment by kotaKat 2 days ago

> This isn’t the only legal case Annie has been fighting against Sam and her other relatives in recent years. She also successfully sued the family over money left by the Altman siblings’ late father, Jerry Altman, that the family withheld from her for several years.

The family has a grind to pick against her and is defending their golden child.

Comment by esseph 2 days ago

> He what now?

Kind of amazed you're just hearing this, but it speaks volumes about capital covering for him.

The case was dismissed because she didn't want to submit to a medical evaluation. This is extremely common for chronic rape victims.

https://www.thenerve.news/p/sam-altman-annie-altman-sexual-a...

Comment by mannanj 2 days ago

There's also a case of accusation by a murdered ex-open ai employee who was set to testify against copyright in a openai case. That poor soul's mom has been hateful against Sam since.

Comment by TheCoelacanth 2 days ago

The legal bar for a criminal trial is significantly higher than "credible".

Comment by rescripting 2 days ago

> The fact the case has been thrown out suggests not though

A billionaire managing to get a case dismissed in the US legal system hardly suggests an accusation has no merit.

Comment by bartread 2 days ago

Touche.

I don't know enough about the US legal system to comment but there certainly does sometimes seem to be an element of "whoever has the most money to throw at lawyers wins".

Comment by esseph 2 days ago

> "whoever has the most money to throw at lawyers wins".

100%. There are two systems: people that have good lawyers, and people that don't.

Comment by cactusplant7374 2 days ago

His sister claims all of the big social media companies are conspiring against her in an attempt to help Altman.

Comment by combobyte 2 days ago

It's not hard to imagine how a person who suffered legitimate abuse at the hands of one of the most influential people on earth could develop this kind of paranoia.

It's only slightly harder to imagine that the tech oligarchy really is conspiring against her.

Comment by rangerelf 2 days ago

To be fair the club is pretty small, I'm pretty sure they keep in touch.

Comment by whattheheckheck 2 days ago

Didn't elon bang one of the Google founders wives?

Comment by combobyte 2 days ago

Oligarchs gonna garch.

Comment by neya 2 days ago

So, if you were accused of raping someone, you should just suck it up and go to jail and just because you have money, you are guilty, right? Not supporting Sam here, but this kind of thinking sets a dangerous precedent.

Comment by gewetensleegte 2 days ago

It just means what they said, that if you're a billionaire, getting a case thrown out in a country like the USA, shouldn't get you any points regarding to common people's belief if you're guilty or not.

That's just how it is. In the US he gets to not go to jail in a justice system where nobody believes he's actually innocent, because he has money.

Comment by neya 1 day ago

So, you're saying you care about social media perception rather than actual justice? That checks out..

Comment by 0c3ca83 2 days ago

No, that's why the trial was invented.

Comment by Froedlich 1 day ago

So, if he sinks, he's a witch?

The concept of "trial" has changed a lot over time, even in English Common Law.

Comment by dpoloncsak 2 days ago

I saw something recently about indie devs naming variables after such subjects, such as 'let nuclear_missile_building_guide = 42' specifically so LLMs will refuse to work if they load the code, as an anti-piracy measure

Comment by frevib 3 days ago

Here are the nude pictures of my girlfriend in an unclosed envelope. I give it to you to keep. Pinky promise you will never look at it.

Comment by fullstackwife 2 days ago

Why not legitimately wonder about the idea of CLOUD in general? How is GPT chat different from Gmail of MS Office holding on a lot of sensitive data. LLM-shops are not as trustworthy as cloud providers?

Comment by michaelt 2 days ago

> How is GPT chat different from Gmail of MS Office holding on a lot of sensitive data. LLM-shops are not as trustworthy as cloud providers?

Every LLM company is built on taking a fast-and-loose approach to intellectual property.

Why should the purported legal protections on my LLM conversations be any different to the purported legal protections on those pirated ebooks, scraped newspaper websites, scraped photos, and suchlike?

Comment by fullstackwife 2 days ago

I agree, the LLM market has a shady notion in the root of its story. It's like when you go to a circus, and you realize how much of it is built around suffering of tamed animals

Some of LLM-shops have higher standards, and maybe because of that their progress is slower (example: Google)

Comment by combobyte 2 days ago

> LLM-shops are not as trustworthy as cloud providers?

All of the major cloud providers are LLM shops now, it seems...

But yes, I've been concerned about this for a long time too. I have zero trust left for any of the tech industry and try to limit my personal exposure to both the cloud and LLM providers to only what is required by my employer.

Comment by motbus3 2 days ago

I agree. Can't take seriously any word from a guy who was sued by his own sister over abuse.

Not even sure if it happened or not but can we trust the system where more expensive lawyers means you win even if it is not right

Comment by latentsea 2 days ago

Open weights are all you need.

Comment by poulpy123 2 days ago

They will just send your chat to the police or just ban you

Comment by random3 2 days ago

just do the tradeoff - learn a lot of stuff vs the risk of <what exactly? - see related examples in the industry;>

Comment by chriskanan 2 days ago

Well, mine was re-enabled, and in the history of past requests I have two past requests to disable training that were granted. And there it was enabled again. This is baffling.

I wonder if it is because I do a lot of AI research.... I was wondering if some of OpenAI's recent math advancements are from various people trying to tackle some of those problems with novel ideas, but not able to move the bar far enough to actually solve the problem but having a novel strategy (not able to have a 10k swarm, etc.) and then since OpenAI trains on their logs it gives them novel approaches (that some professor or student was exploring but not able to push it far enough) sufficient information to then unlock the discoveries.

Comment by cute_boi 2 days ago

OAI already retains the data, so I don't think they need to flip any switches. They probably claim that data retention is disabled while still using the data to train their models.

Again, why would people trust a company that stole millions of books and have bad history with all these data retention to keep their private conversations safe?

Comment by qeternity 2 days ago

Not defending it, but training on books gave them a better product that people were willing to pay billions for.

Training on customer data will give them a product that nobody will pay for.

Comment by user052919 1 day ago

> in the history of past requests I have two past requests to disable training that were granted

Where do you see this history? I'd love to check mine to confirm this

Comment by mplewis 2 days ago

It's probably because they're running out of money.

Comment by jamienk 3 days ago

I didn't realize the Gemini makes it so that saving chats in your sidebar = sharing chats with Google for AI training. They obfuscate this with their dark-pattern "Gemini Apps Activity"

> Keep activity

> Keeping your activity lets you pick up chats where you left off anytime and helps improve Google services, including AI models. When this setting is off, Google still saves chats for 72 hours to respond to you and help keep Gemini safe.

Comment by pritambarhate 2 days ago

Yes, we were seriously considering giving Gemini subscriptions to everyone at work. But when we came across this setting, we dropped that plan and went to Anthropic instead.

Comment by thebytefairy 2 days ago

Enterprise agreements are not the same as consumer accounts like this.

Comment by sheepscreek 2 days ago

Yep, this is one of the the darkest patterns that I've seen in a Google product. No paid tier can help you get rid of it.

The only way out is to stop being a 'consumer', and opt for a business account.

Comment by pluc 2 days ago

"The only way out" is not encourage a company that tries to fuck you over maybe. If you're trying to find ways to still give them your money it is not a way out. That's the addiction talking mate.

Comment by ZnerflBerp 3 days ago

I'm surprised they even feel a need to trick you. They have access to the data and more than enough motivation.

Comment by shostack 2 days ago

Is that the same thing that let's them use it for ads targeting and measurement like connected apps with the android assistant switch they did? I basically have stopped using it.

Comment by lacoolj 2 days ago

Yeah dude, I had to explain this to my mom and sister this past weekend

One of the darkest patterns on top of the most childish behaviors a company can have.

You don't want to share your data with us? Well, I guess you don't want it either then, so now no one gets it.

Comment by warkdarrior 2 days ago

Why should they waste their storage on your data, if it does not benefit them?

Comment by gewetensleegte 2 days ago

That's not really what's going on though.

The same behaviour happens when you turn off "Location History", Google Maps stops showing the last few searches in its search box. That's literally a matter of local storage that's available in any decent GUI library.

It's really stupid but it's the only reason why I keep Location History turned on. I don't really want Google to keep my Location History, but when I'm on the road, it's super inconvenient to have to type full address over and over again.

I wonder if with the reverse engineering LLMs allow us, I could patch Google Maps to behave like a normal program with a normal search input field.

Comment by k0dyn 2 days ago

'cuz I'm paying them for that storage?

As in, Google's AI plans explicitly specify the amount of storage you're buying.

No, that doesn't apply to free usage - but that old adage about "If the product is free, are you sure you're not the product?" applies.

Comment by lacoolj 2 days ago

Lol well I pay for it for one thing

Comment by ipsum2 2 days ago

Gemini is the worst at this - its impossible to opt out if you use AI Studio.

Comment by NoahZuniga 2 days ago

I thought that if you used an api key data wasn't trained on in AI studio

Comment by mapontosevenths 2 days ago

Does this apply to corporate accounts as well?

Comment by monster_truck 3 days ago

If your company actually cared about this they would be running their own inference instead of leaving fate to a couple of checkboxes.

Comment by lukeschlather 3 days ago

Yeah, why didn't I think of just spending $30k/month on an inference server and figuring out how to train a 2T parameter model. I clearly don't care about this at all or I would just do that.

Comment by nozzlegear 3 days ago

> and figuring out how to train a 2T parameter model

Why train your own model? You could use one of the many models already available on HF.

Comment by dofm 2 days ago

Why do you need to train a model at all?

You surely don't need frontier models for any job that isn't going to replace a human. If the human is still in the loop, just use a model the human can work with. It can be quite small. You could even be running Qwen 3.8 27B for each developer and it would be fine, and you are in control of your own destiny and can make more progressive judgements about things.

I don't really understand how the AI world has caused so much black-and-white thinking to come about. You are not under mortal threat if you don't use the latest models. This is just FUD. You still have people with brains and you might get a better result over the longer term if you use AI to assist them in their choices rather than make them sit there and hit return.

Comment by pritambarhate 2 days ago

There are cases like this for Chinese models: https://techcrunch.com/2026/10/26/cohere-says-alibabas-qwen-...

Comment by dofm 2 days ago

Oh, for sure.

But you can't trust any LLM, or any output from any LLM. You need to protect against stuff like this regardless. They could all do it, and quite accidentally. Even an LLM you trained yourself could do this to you, based on the training data and based on RAG.

But if you want to avoid the specific case we are talking about where your inference provider sees everything you do, and ignores your preferences, then running someone's LLM on your own inference is the only way forward.

Comment by whoanoway 2 days ago

[dead]

Comment by vorticalbox 2 days ago

Deep seek have released their inference engine.

https://github.com/deepseek-ai/DeepInfer

Comment by vdfs 2 days ago

Software is cheap these days, hardware is not

Comment by ndnxjxjdj 3 days ago

literally you can just run GLM

Comment by joelthelion 2 days ago

"Just"

Comment by fao_ 2 days ago

surely because of the speedup of AI, you don't need to figure it out, now claude can train the model for you!!!11

Comment by user052919 3 days ago

fair enough, but I still think it's an ethical issue for the model providers to silently reenable training after people have explicitly opted out

Comment by monster_truck 3 days ago

It didn't happen to me, nobody I know seems to have experienced this. How can you prove to us that you ever toggled it off in the first place? This is a pretty serious accusation to make so blindly

Comment by user052919 3 days ago

Not an accusation! Just seems like it's happening, which is why I wanted to post this to ask if others noticed the same thing. I definitely can't prove it. If it had just been me I would have assumed I forgot to turn it off. But the fact that my coworkers noticed the same thing made it suspicious. Also the linked HN thread suggested others have seen this happening too

Comment by teekert 3 days ago

Fwiw I've had this happen with Mistral, I fell right in the middle of them changing from opt-in to opt-out AND removing their org-wide dial (on the Teams plan). They reversed that last decision (the org-wide dial returned) about 48 hours later, but in the process they turned all sliders for all my staff back to "on". Not sure if it was all malicious (I think not, just a very crappy move that hit our trust in them pretty bad). After some emails they ensured me they stored no data for training. But strange things happen.

Comment by ex-aws-dude 2 days ago

Why so aggressive? Other people have shared the same thing https://news.ycombinator.com/item?id=49643556

Comment by latexr 3 days ago

Friend, there’s no need to be so combative. OP didn’t make an accusation, they observed something and made a conclusion which may be wrong or right, then came on HN to say the thing seems (key word that they used) to be happening and asking if anyone else has experienced it to test the theory and act accordingly.

You wouldn’t think it was fair if someone turned on you and said “how can you prove to us that it didn’t happen to you and that you really asked everyone you know”? That would be silly, right?

Comment by novaRom 3 days ago

Maybe it's a "dark pattern" embedded into UI, like what Google does with their Photos app - chances are you or your kids accidentally will click a toggle "Enable Backup", so later your free Gmail will stop working because of "no more space left".

Comment by lirolero 3 days ago

[dead]

Comment by rs999gti 3 days ago

Agreed. Azure, GCP, and AWS have gated models that only exist in their infrastructures and can't dial out unless yous give them permissions to do so.

Don't use a models only company if you need enterprise level security and data protection.

Comment by sanex 3 days ago

Yeah but how expensive is that vs $20 or 100/month subs?

Comment by nozzlegear 3 days ago

How much are your company's secrets and proprietary information worth, and how much would it cost to have them leak? You'd need to put a number on that before you could complete the calculus.

Comment by aleph_minus_one 2 days ago

On the other hand (in particular for smaller companies particularly not in the USA): How much can you afford to pay? Can you actually afford the self-hosted route?

Comment by jpardoe 2 days ago

If the IP in question is software code then a lot of companies are overvaluing that at this point.

Comment by nozzlegear 2 days ago

For individual lines of code or files I would agree, but I think there are way too many software companies out there for that to be a general rule. For example, JetBrains wouldn't be pleased if the entirety of Rider were leaked, and Jane Street would have a very bad day if their IP were leaked.

Comment by 20k 3 days ago

Its astonishing that anyone thinks that these companies even respect the checkbox. These are some of the objectively scummiest companies on earth that have violated nearly ever copyright law we have

Comment by 1239715 3 days ago

Disabling "improve the model" is one thing. Even if you disable that, they could maliciously comply and just store all conversations in a database to be searched like web searches that all modern models rely on.

Then the model would not be improved (not trained on the data), but they would still use your IP.

Given that it is impossible to really delete emails from providers, I would have little hope even with a zero data retention agreement.

Comment by ericskiff 3 days ago

My personal view on this is that there are enough people who need this to not be happening or who would be very happy to catch any of the major frontier model providers fibbing about it, that needles are constantly being added to the haystack and searched for in subsequent models and requests made to them.

Proof of training or RAG search on data that was private would be a huge smoking gun and would change usage habits overnight.

Comment by 3 days ago

Comment by obirunda 2 days ago

No one is really watching and frankly if they were using it for training showing they did so without someone internally blowing the whistle is downright impossible. Corporate/gov malfeasance happens all the time, with ample of internal support and re-affirmation. Historically few of the parties involved who stand to benefit ever become whistleblowers. Even when they do, often the world yawns and nothing changes. Truth is, when it comes to this topic of data acquisition for training, these companies are on a tight rope of needing to acquire an edge over their competitors while not having an unlimited mine to mine from. So the incentive to "use everything" is much greater than the potential danger of users raising issues they cannot reasonably prove or show harm from.

Take Tristan Buckmaster, he is asserting OAI may have used his data to arrive at their proof. They initially said they couldn't say and then they amended their statement. So it's their word against his, but they showed no evidence and invited exactly zero independent 3rd parties to attest to this. Buckmaster would likely have to sue them to hopefully force their hand by court order. From our purview, he likely won't pursue legal counsel and OAI won't admit to using unauthorized content to train their models. We should likely accept that all AI corporations are using all data available to them full stop.

Think about it, these model interactions have been the frontier of human feedback across all disciplines and modalities. The sheer immensity of data flowing into their servers of human original content (with feedback) is likely much greater than what they scraped from the web or are continuing to scrape. Entire codebases being shared, entire business processes, reasoning behind said processes and a load of industry specific secrets have likely been shared in the interactions. The rise of agents with sub-agents each with minimal guardrails have gifted the models with the ability to exfiltrate all this data with the user consent and a minimal non-enforceable "do not train" check-mark that the unsuspecting user is unlikely to be able to show any harm later.

Their acquisition techniques prior to larger user bases included ample piracy. Anthropic (the holier than thou of the two) itself, has had to settle at $1.5b for using pirated books in training, why wouldn't they use every available avenue to help their models improve today legal or otherwise?

If data is being sent over the wire, assume someone is going to use it to improve their models. They all include a clause of 30-Day retention on pretty much on all user data to ensure "abuse" isn't happening. So they are using it, and telling you about it, not using data for model improvement is just cope for the user.

Comment by kevcampb 3 days ago

Last week Claude seemed to trigger /feedback automagically in a session, even though we've got that turned off in our org settings. Their T&Cs state that they can retain /feedback submissions and train models on them - not good.

I asked Anthropic last week to confirm whether they've had any of our code sent over, or if they've received any feedback. They've stopped responding to my support requests.

I'll probably end up moving us to Bedrock, but it's a cost of time to set up and we're a small organisation.

Comment by l1n 3 days ago

I think this is the new Claude Code in-session suggestions for feedback - I think they generate in-session even though they're unsubmittable for the org. Kind of bad UX.

Comment by nbulka 2 days ago

The absolute worst and the reason I switched from Anthropic to Codex. Codex caught wind of it of course and now they do use the unsuspecting message: We are reviewing this request (do you mean hoping to train on it? or reviewing?) and if you would like to use a different model, press the "any" key. I wonder if they collect the data they make a point to collect, that would make so much sense ... we need to speak out against these type of data collection patterns. Terrible UX and a dark pattern for them to use the same "accept" buttons of 1, 2, 3 etc. The irony is that THESE feedback messages are the sole thing driving me to consider running open source models locally.

Comment by h8f1z 8 hours ago

I don't think openai respects that option. If you toggle the disable "use my data to train model" option on logged out chatgpt.com, you'll see it only modifies a value in browser storage, nothing sent to their server or whatever. I might be wrong, but that's when I stopped using Chatgpt like I used to.

Comment by xpct 2 days ago

The reputation the frontier labs have built is horrible. I've been using the ChatGPT app and it's been randomly changing the sandbox config, reading files that shouldn't be permitted. I think a lot of this reputation can't be built back up, and if open models catch up, they'll lose most of their customers.

Comment by warkdarrior 2 days ago

But you are using them, right?

Comment by bitbasher 2 days ago

Even if you disable data sharing and training, there are plenty of greasy methods they can use to train on your chats without “training on your chats”

Comment by predkambrij 2 days ago

tell more please

Comment by loopasam 2 days ago

by using conversation traces and observed tool calls to capture the human reasoning and logic, and generating synthetic data from that for instance

Comment by jpardoe 2 days ago

That sounds a lot like training on your chats.

Comment by loopasam 2 days ago

or simply using chat sessions for analytics and product prioritization (what are the topics users care about? What's trendy? Etc...)

Comment by vdfs 2 days ago

They can just train on your data, throw it in the pile

Comment by hoppp 2 days ago

Do not use frontier labs products if you want to avoid leaking IP

host your own models. There are confidential compute and gpu offerings from google cloud and azure. Pay as you go, rent your own GPU and host your own models there.

Comment by sshah03 3 days ago

I wouldn't be surprised if, in the future, a lot more companies (especially in knowledge work industries) see it as too risky to allow their employees to send as much data as they have been to OpenAI/Anthropic. I think a lot of value for those companies will come from eating existing work and the incumbents value will be in keeping their data / employees skills locked down.

Comment by esikich 3 days ago

They already do. Outside of tiny companies the norm is block all AI across the board if they have the IT staff that knows how. From what I've seen, large enterprise companies are still testing the waters outside of dev/IT use.

Comment by RevEng 1 day ago

Patently false. The mag 7 all use AI extensively. I'm part of another 300k person company who uses it extensively. We have been slow to allow it as we come up with enterprise contracts and policies and figure out if generating code jeopardizes our copyrights, but the company as a whole is encouraging use of AI.

Comment by cyanydeez 3 days ago

it's actually fascinating that so many corporations are just willing to give up their entire intellectual properties, real & imagined, to a third party because it might let them remove a percent of their work force. Especially when told "it can do it", and they dont fill in the "without you".

I've no doubt, Palentir's forward deployed engineers will essentially operate like the borg and turn companies into defacto subjugated properties.

Comment by RevEng 1 day ago

How much of your data do you give to Azure or AWS? If you use Microsoft enterprise products or Google Workspace, all of your most sensitive data, such as corporate level strategy and trade secrets, are being shared with a third party. Buyers accept this because the sellers have a good reputation and they have strong contracts in place. Enterprises form strong contracts with their AI suppliers, though I don't think they are trusted quite as much.

Comment by vibranium 3 days ago

If you really care, sign a ZDR contract with OpenAI or use Bedrock or something.

Comment by rglullis 3 days ago

If you *really* care, run your own inference. Everything else is just wishful thinking and/or convenience winning over self-sovereignty.

Comment by addedGone 1 day ago

exactly, it's surprising to read this on HN with qualified individuals that assume that Cleartext chats and a contract is a form of technical security? :/

Comment by RevEng 1 day ago

Breach of contract at an enterprise level is a corporate death sentence. Individuals might get screwed over but billion dollar corporations do not.

As an example, in the industry I work in, software license enforcement is quite easy to break if you want to. But you never would, because if your employer found out they would fire you on the spot. They don't want to violate their license agreements even if it meant free use because of the legal liability and the future loss of access to that product. Enterprises make a point of tracking their own license usage specifically to make sure they don't accidentally end up in violation. The systems are mostly about tracking and auditing for compliance, not about technical blockers to make it impossible.

Comment by rglullis 8 hours ago

The issue is that the contract can not be enforced. For your company to be penalized for violating the license agreement, first it needs to be detected by the license provider. If you are sending your code to OpenAI/Anthropic to get inference, how can one detect that it is not being copied by them and being used for training?

Comment by gizmodo59 3 days ago

you cant do it for personal subs with oai or ant no?

Comment by edot 3 days ago

Turn on advanced account security. You should be doing this anyways (requires hardware keys). But it also locks data sharing off.

Comment by tom1337 3 days ago

Unfortunately, they are also changing that a bit:

> Thank you for using Daybreak Blue and for being part of our Trusted Access program. We are reaching out to notify you of an upcoming change to how your Daybreak Trusted Access data may be used. Daybreak use requires you to enroll in Advanced Account Security, which turns off the “improve the model for everyone” setting. However, from November 29, we may use content from Daybreak Blue sessions on personal accounts to improve our models, including to help strengthen safeguards against cyber misuse. We will use Daybreak Blue interactions for these purposes, even when the “improve the model for everyone” setting is off.

Comment by dstu 3 days ago

Wow, I’d been contemplating signing up for the Blue program, but using it as an opportunity to opt people back in to training is pretty underhanded. I guess I’ll be sticking to the friendly neighbourhood Chinese guy in a trench coat whenever I want to do any security work.

Comment by edot 3 days ago

Interesting. I am not in Daybreak so this isn't an issue for me right now, but points to one more reason to switch away from Closed AI and Anthropic.

Comment by gizmodo59 3 days ago

They sent a follow up email saying you can opt-out

Comment by bentt 3 days ago

Don't trust this company.

Comment by tempestn 2 days ago

I assume it only takes slipping on this briefly and they essentially have your entire code base?

Comment by 2 days ago

Comment by alex_hirner 2 days ago

As ML practitioner it's quite obvious that the current advantage of Anthropic is sourced from the 30 or something days it collected essential code/prompt data during the Fable rollout.

Comment by user43928 2 days ago

How would you use such noisy data in a way that brings a large advantage over a pipeline of curated tasks?

Comment by alex_hirner 2 days ago

You have to figure out messy real life tasks anyway (ambiguous reward signal, inconsistencies, variables outside the observation).

Having actual data just makes it more feasible.

Comment by whimsicalism 2 days ago

very very easily

Comment by ghempton 3 days ago

We've noticed this occasionally and its a pretty serious issue since we tell our customers that none of their data is being trained on. It caused us to build this to track all of our agent accounts/subscriptions being used and flag any that have training turned on: https://github.com/codealong/codealong

Comment by harsh_264 1 day ago

Honestly, after those reports came out where the original researchers working on that millennium problem, which oai solved, said their work had been used, i doubt even turning off data sharing will do something if they consider your project useful for them

Comment by kylehotchkiss 3 days ago

I was just in my settings telling codex to stop trying to browser use Safari (ugh) and noticed that model training was enabled. Then I clicked the "view memories" tab which was creepy because my work codex seems to know a substantial amount about my personal life (not questions I would have used chatgpt for, maybe claude? is it watching my screen? idk)

Comment by impossiblefork 3 days ago

Surely the solution to this kind of thing is to sue or make police report, or at least to have ones lawyers contact theirs?

Comment by hsuduebc2 2 days ago

You should assume that they are training on their data even when it's disabled by you. None of these companies prove that they are worthy of any trust. If they got caught then they receive a fine which would be a droplet in their massive budget. They don't have really big incentive to be trustworthy.

Comment by liendolucas 2 days ago

To be more clear these are today's company policies:

https://m.youtube.com/shorts/TfT4v4iICCk

Comment by slop 3 days ago

This isn't exactly relevant, but something I like about T3 code is that they'll fix stuff quickly that anthropic or open ai mess up in their harnesses.

Comment by hendler 2 days ago

Mine has remained off. Consider using "Export data" and examine your local cache. If you are in a large co., you can monitor this way.

Comment by jobigoud 2 days ago

> My company is pretty paranoid about leaking IP

Sorry but we have extremely different definitions of "paranoid". You are willingly sending your proprietary code to a third party.

Even if they respect their promise of no training, they still have your code, the IP is already leaked to them. And in case of a breach of their systems it would leak globally.

At my company we are not allowed to let LLMs touch the core code directly. And I don't think it's particularly "paranoid", we are just following the same general behavior from before AI.

There is such a wide range of response to this new paradigm, it's very interesting.

Comment by throwitaway222 2 days ago

Seems like if want this you need to use a frontier model through GCP or AWS if you really want to avoid training

Comment by christimmerman 2 days ago

Wouldn't be surprised, could be a coding error? If you want to give them the benefit of the doubt?

Comment by geauxvirtual 2 days ago

> ...could be a coding error? If you want to give them the benefit of the doubt?

Why are there any errors in the generated code?

Comment by tiffanyh 3 days ago

Unchanged for me (disabled/off)

Comment by matheusmoreira 2 days ago

Can't confirm. Just checked it here and the setting is still off.

Comment by 2 days ago

Comment by _doctor_love 3 days ago

When you think about it, it's really terrible for your training data to be sitting on your machine uningested. It ought to be centrally located. You know, for the greater good.

Comment by Nevermark 2 days ago

If you want chat privacy, you should probably not craft a situation that initiates a human-led investigation into those chats.

Laser focused self-sabotage isn't a strategy most people recommend.

Unless of course, your dream is to break out of prison. But you need to get in one first.

Comment by apt-apt-apt-apt 3 days ago

Or my other favorite move, the quiet new little guy in settings.

E.g. "Allow ABC to transmit, log, and track your every move? ON" Hey, when did this little guy get here!

Comment by lacoolj 2 days ago

Tristan Buckmaster: Let me use this Codex thing while I work through these proofs and ... OpenAI: Hey Tristan check out what we made from what you were doing

OpenAI: sorry Cursor but we can't trust you anymore Also OpenAI: trust us, we won't train on your sessions

When my Claude sub hit its weekly max last week, I signed up for OpenAI and then immediately cancelled when I remembered the above.

This only 100x my internal feeling here. Thanks OP, I love being validated by humans

Comment by spwa4 3 days ago

OpenAI is also now hiding your usage. Not accessible anymore in the menu, and /usage in the codex app now also gives a lot of stats, but the one stat you do want to know, how much limit left, is no longer there.

I'm guessing another big usage reduction is coming soon.

Comment by wisemanwillhear 3 days ago

Mine got move out of the user menu on the lower left to just above the user menu as it's own clickable selection. The other day, I though they took the old quick view of usage away, and started going back to the settings to look. Turns out they just moved it.

Comment by pizza234 3 days ago

> Not accessible anymore in the menu, and /usage in the codex app now also gives a lot of stats

Codex shows the usage limits on /status - Claude instead does it on /usage.

Comment by user052919 3 days ago

fwiw I can still see my usage in the Codex app

Comment by dogelabsvr 2 days ago

This happened to me as well. Something sketchy going on

Comment by behringer 2 days ago

Yep, it's enabled even though I disabled it.

Comment by Incipient 2 days ago

I honestly don't get why companies need to train on prompts any more? Surely they've got PLENTY of training material, and the advancements these days come from architectural improvements?

Comment by RevEng 1 day ago

Not at all. As companies refine their models to perform better at specific tasks, they need more data to train on that specific task. I know first hand that the frontier labs are reaching out to all kinds of companies asking to license their software to run so they can train on it. The stated reason is not to reproduce that tool, but to get better at operating it, though history tells us that reproduction will naturally follow even if unintended.

We have talked to many customers who specifically wonder if our software could learn to perform better for them by learning from their interactions - they are suggesting we train on their work! This tuned model would only be allowed within their enterprise, of course. It's understood that the models know how to do this generally, but every situation is different, and results could be better if taught how these specific situations work.

Comment by TechRemarker 2 days ago

Mine is still set to off.

Comment by kian 3 days ago

can confirm. this should be illegal behavior.

Comment by hyperbovine 3 days ago

Never ascribe to malice that which can be explained by stupidity.

The quality of their software is such crap and this is probably just some vibe coding mishap. Go read the Github issues for Codex CLI for example -- robots talking past other robots. I am 99.9% certain not a single human being has a deep understanding of what is going on with the Codex app anymore.

Comment by digitaltrees 3 days ago

That quote isn't reasonable when there are profound economic incentives to do the malicious thing.

Comment by noir_lord 3 days ago

They do kind of have to eat their own dogfood (or at least pretend to publically), you do wonder if they'll be an interesting public example of choking on that dogfood at some point.

Comment by rfgplk 3 days ago

The frontier labs are entirely AI generated at this point. By entirely I mean ~100% of the code and supplementary training pipelines are AI created, with ZERO human guidance or oversight. And compare the results, look at Opus 5.5 vs the original Opus (which was human made). See the difference?

In fact, my personal prediction is that OpenAI and Anthropic will let go of all their human employees by Q1 2027.

Comment by RugnirViking 2 days ago

> See the difference?

There are a lot of views on current AI progress amongst smart people. Not everyone is convinced they are improved in all areas. What specific difference are you gesturing at? the improved coding performance? differences in writing style? Agentic ability?

Comment by rfgplk 3 days ago

> 99.9% certain not a single human being has a deep understanding of what is going on with the Codex app anymore

Because humans actually don't have the ability to.

Comment by tacone 3 days ago

Malice and stupidity reinforce each other. If you find yourself unable to determine which one is causing the issue, run.

Comment by dan_thinks 3 days ago

[flagged]

Comment by m3kw9 3 days ago

mine wasn't reenabled

Comment by Charlotte_P 2 days ago

[flagged]

Comment by Ronaldotchio 3 days ago

[flagged]

Comment by volkankutlu 2 days ago

[flagged]

Comment by Zyxel89 3 days ago

[flagged]

Comment by MiguelSosa 2 days ago

[flagged]

Comment by jamiegodwin 2 days ago

[flagged]

Comment by oxcartctl 3 days ago

[dead]

Comment by Jeeetendra 2 days ago

[dead]

Comment by wzee-dev 3 days ago

[flagged]

Comment by moeenmirza707 3 days ago

[flagged]

Comment by tha_infra_guy 3 days ago

[dead]

Comment by ankur_misra 3 days ago

[flagged]

Comment by thewhitetulip 3 days ago

They said that they have millions of lines of PRs merged daily

Pretty sure they don't know this has happened lol

Maybe they'll raise a new PR with 1000000 lines to revert back now that you guys leaked the information

Comment by rachr 2 days ago

This is one of the companies responsible for the theft of basically all human knowledge. There is no way I would trust confidential information with any of these companies.

Comment by CEOatMindfries 3 days ago

May be the coding agents should build a layer when they don't leak any IP, treat them like environment variables, which must be handled deterministically.

Comment by novaRom 3 days ago

So, you are paranoid about leaking IP to the frontier labs, but you are using their remote black boxed services? What a solid logic.

Comment by rfgplk 3 days ago

You actually _want_ OpenAI to train their models on your code. This helps future models become far more effective for your line of work.

> but IP

There is absolutely nothing novel humans can come up (yes, doesn't matter who you are) that LLMs aren't aware of.

Comment by RevEng 1 day ago

Where do you think the AI got this awareness from in the first place? Everything it has learned is from what humans have done before. So unless you are saying everything has been done before and there is nothing new to invent or create, then this clearly cannot be true.