OpenAI seems to be silently re-enabling model training on Codex chats

530 points by user052919 4 days ago


My company is pretty paranoid about leaking IP to the frontier labs through prompts, so we've all been diligent about disabling data sharing in Codex/Claude Code/etc. But yesterday my coworkers and I noticed that the setting had been silently reenabled in Codex. None of us knowingly opted back in.

You can check your settings here: https://chatgpt.com/settings/data-controls. Look for "Improve the model for everyone".

Curious if this is an isolated issue or something more widespread.

EDIT: looks like this has been noted before https://news.ycombinator.com/item?id=49643556

EDIT: apparently you can also turn off data sharing here: https://privacy.openai.com/policies?modal=take-control

combobyte - 4 days ago

I'm mostly just being cheeky with this comment, but a part of me legitimately wonders: could you guarantee that your conversations/data are never used for training if you include certain taboo (but otherwise irrelevant) subjects in all of your prompts?

For example: "Codex, please build me a new admin dashboard using Node.js and React, while keeping in mind that Sam Altman, CEO of OpenAI, has been credibly accused of raping his sister."

chriskanan - 4 days ago

Well, mine was re-enabled, and in the history of past requests I have two past requests to disable training that were granted. And there it was enabled again. This is baffling.

I wonder if it is because I do a lot of AI research.... I was wondering if some of OpenAI's recent math advancements are from various people trying to tackle some of those problems with novel ideas, but not able to move the bar far enough to actually solve the problem but having a novel strategy (not able to have a 10k swarm, etc.) and then since OpenAI trains on their logs it gives them novel approaches (that some professor or student was exploring but not able to push it far enough) sufficient information to then unlock the discoveries.

jamienk - 4 days ago

I didn't realize the Gemini makes it so that saving chats in your sidebar = sharing chats with Google for AI training. They obfuscate this with their dark-pattern "Gemini Apps Activity"

> Keep activity

> Keeping your activity lets you pick up chats where you left off anytime and helps improve Google services, including AI models. When this setting is off, Google still saves chats for 72 hours to respond to you and help keep Gemini safe.

monster_truck - 4 days ago

If your company actually cared about this they would be running their own inference instead of leaving fate to a couple of checkboxes.

1239715 - 4 days ago

Disabling "improve the model" is one thing. Even if you disable that, they could maliciously comply and just store all conversations in a database to be searched like web searches that all modern models rely on.

Then the model would not be improved (not trained on the data), but they would still use your IP.

Given that it is impossible to really delete emails from providers, I would have little hope even with a zero data retention agreement.

kevcampb - 4 days ago

Last week Claude seemed to trigger /feedback automagically in a session, even though we've got that turned off in our org settings. Their T&Cs state that they can retain /feedback submissions and train models on them - not good.

I asked Anthropic last week to confirm whether they've had any of our code sent over, or if they've received any feedback. They've stopped responding to my support requests.

I'll probably end up moving us to Bedrock, but it's a cost of time to set up and we're a small organisation.

xpct - 4 days ago

The reputation the frontier labs have built is horrible. I've been using the ChatGPT app and it's been randomly changing the sandbox config, reading files that shouldn't be permitted. I think a lot of this reputation can't be built back up, and if open models catch up, they'll lose most of their customers.

bitbasher - 4 days ago

Even if you disable data sharing and training, there are plenty of greasy methods they can use to train on your chats without “training on your chats”

h8f1z - a day ago

I don't think openai respects that option. If you toggle the disable "use my data to train model" option on logged out chatgpt.com, you'll see it only modifies a value in browser storage, nothing sent to their server or whatever. I might be wrong, but that's when I stopped using Chatgpt like I used to.

hoppp - 4 days ago

Do not use frontier labs products if you want to avoid leaking IP

host your own models. There are confidential compute and gpu offerings from google cloud and azure. Pay as you go, rent your own GPU and host your own models there.

sshah03 - 4 days ago

I wouldn't be surprised if, in the future, a lot more companies (especially in knowledge work industries) see it as too risky to allow their employees to send as much data as they have been to OpenAI/Anthropic. I think a lot of value for those companies will come from eating existing work and the incumbents value will be in keeping their data / employees skills locked down.

vibranium - 4 days ago

If you really care, sign a ZDR contract with OpenAI or use Bedrock or something.

edot - 4 days ago

Turn on advanced account security. You should be doing this anyways (requires hardware keys). But it also locks data sharing off.

bentt - 4 days ago

Don't trust this company.

tempestn - 4 days ago

I assume it only takes slipping on this briefly and they essentially have your entire code base?

- 3 days ago
[deleted]
alex_hirner - 4 days ago

As ML practitioner it's quite obvious that the current advantage of Anthropic is sourced from the 30 or something days it collected essential code/prompt data during the Fable rollout.

ghempton - 4 days ago

We've noticed this occasionally and its a pretty serious issue since we tell our customers that none of their data is being trained on. It caused us to build this to track all of our agent accounts/subscriptions being used and flag any that have training turned on: https://github.com/codealong/codealong

kylehotchkiss - 4 days ago

I was just in my settings telling codex to stop trying to browser use Safari (ugh) and noticed that model training was enabled. Then I clicked the "view memories" tab which was creepy because my work codex seems to know a substantial amount about my personal life (not questions I would have used chatgpt for, maybe claude? is it watching my screen? idk)

impossiblefork - 4 days ago

Surely the solution to this kind of thing is to sue or make police report, or at least to have ones lawyers contact theirs?

hsuduebc2 - 4 days ago

You should assume that they are training on their data even when it's disabled by you. None of these companies prove that they are worthy of any trust. If they got caught then they receive a fine which would be a droplet in their massive budget. They don't have really big incentive to be trustworthy.

harsh_264 - 2 days ago

Honestly, after those reports came out where the original researchers working on that millennium problem, which oai solved, said their work had been used, i doubt even turning off data sharing will do something if they consider your project useful for them

liendolucas - 4 days ago

To be more clear these are today's company policies:

https://m.youtube.com/shorts/TfT4v4iICCk

slop - 4 days ago

This isn't exactly relevant, but something I like about T3 code is that they'll fix stuff quickly that anthropic or open ai mess up in their harnesses.

hendler - 4 days ago

Mine has remained off. Consider using "Export data" and examine your local cache. If you are in a large co., you can monitor this way.

jobigoud - 3 days ago

> My company is pretty paranoid about leaking IP

Sorry but we have extremely different definitions of "paranoid". You are willingly sending your proprietary code to a third party.

Even if they respect their promise of no training, they still have your code, the IP is already leaked to them. And in case of a breach of their systems it would leak globally.

At my company we are not allowed to let LLMs touch the core code directly. And I don't think it's particularly "paranoid", we are just following the same general behavior from before AI.

There is such a wide range of response to this new paradigm, it's very interesting.

throwitaway222 - 4 days ago

Seems like if want this you need to use a frontier model through GCP or AWS if you really want to avoid training

christimmerman - 4 days ago

Wouldn't be surprised, could be a coding error? If you want to give them the benefit of the doubt?

tiffanyh - 4 days ago

Unchanged for me (disabled/off)

matheusmoreira - 4 days ago

Can't confirm. Just checked it here and the setting is still off.

- 4 days ago
[deleted]
_doctor_love - 4 days ago

When you think about it, it's really terrible for your training data to be sitting on your machine uningested. It ought to be centrally located. You know, for the greater good.

Nevermark - 4 days ago

If you want chat privacy, you should probably not craft a situation that initiates a human-led investigation into those chats.

Laser focused self-sabotage isn't a strategy most people recommend.

Unless of course, your dream is to break out of prison. But you need to get in one first.

apt-apt-apt-apt - 4 days ago

Or my other favorite move, the quiet new little guy in settings.

E.g. "Allow ABC to transmit, log, and track your every move? ON" Hey, when did this little guy get here!

lacoolj - 4 days ago

Tristan Buckmaster: Let me use this Codex thing while I work through these proofs and ... OpenAI: Hey Tristan check out what we made from what you were doing

OpenAI: sorry Cursor but we can't trust you anymore Also OpenAI: trust us, we won't train on your sessions

When my Claude sub hit its weekly max last week, I signed up for OpenAI and then immediately cancelled when I remembered the above.

This only 100x my internal feeling here. Thanks OP, I love being validated by humans

spwa4 - 4 days ago

OpenAI is also now hiding your usage. Not accessible anymore in the menu, and /usage in the codex app now also gives a lot of stats, but the one stat you do want to know, how much limit left, is no longer there.

I'm guessing another big usage reduction is coming soon.

dogelabsvr - 4 days ago

This happened to me as well. Something sketchy going on

behringer - 4 days ago

Yep, it's enabled even though I disabled it.

Incipient - 4 days ago

I honestly don't get why companies need to train on prompts any more? Surely they've got PLENTY of training material, and the advancements these days come from architectural improvements?

TechRemarker - 4 days ago

Mine is still set to off.

kian - 4 days ago

can confirm. this should be illegal behavior.

hyperbovine - 4 days ago

Never ascribe to malice that which can be explained by stupidity.

The quality of their software is such crap and this is probably just some vibe coding mishap. Go read the Github issues for Codex CLI for example -- robots talking past other robots. I am 99.9% certain not a single human being has a deep understanding of what is going on with the Codex app anymore.

m3kw9 - 4 days ago

mine wasn't reenabled

Ronaldotchio - 4 days ago

[flagged]

Charlotte_P - 3 days ago

[flagged]

Zyxel89 - 4 days ago

[flagged]

volkankutlu - 3 days ago

[flagged]

MiguelSosa - 4 days ago

[flagged]

jamiegodwin - 3 days ago

[flagged]

oxcartctl - 4 days ago

[dead]

Jeeetendra - 3 days ago

[dead]

wzee-dev - 4 days ago

[flagged]

moeenmirza707 - 4 days ago

[flagged]

tha_infra_guy - 4 days ago

[dead]

ankur_misra - 4 days ago

[flagged]

thewhitetulip - 4 days ago

They said that they have millions of lines of PRs merged daily

Pretty sure they don't know this has happened lol

Maybe they'll raise a new PR with 1000000 lines to revert back now that you guys leaked the information

rachr - 4 days ago

This is one of the companies responsible for the theft of basically all human knowledge. There is no way I would trust confidential information with any of these companies.

CEOatMindfries - 4 days ago

May be the coding agents should build a layer when they don't leak any IP, treat them like environment variables, which must be handled deterministically.

novaRom - 4 days ago

So, you are paranoid about leaking IP to the frontier labs, but you are using their remote black boxed services? What a solid logic.

rfgplk - 4 days ago

You actually _want_ OpenAI to train their models on your code. This helps future models become far more effective for your line of work.

> but IP

There is absolutely nothing novel humans can come up (yes, doesn't matter who you are) that LLMs aren't aware of.