Tell HN: Gemini trains on chat history, even for subscribers (personal accounts)

For an AI system to be “completely safe”, like a self-driving car, it must map the full environment it operates in and be taught the “right way” to handle every conceivable scenario.

So when you present it with novel data or a situation, that constitutes an “AI safety event”, which per their ToS is a carve out for training on your session.

Further, go read the ToS carefully about what you opt out of. You’ll note there’s nary a mention of tool calls. You’re only opting out inputs (what you type) and outputs from the model from being trained on. All the content of the tool calls made while running on your system are fair game for training.

And this goes for all cloud providers btw.

But there's two reasons to bring cautionary attention to it anyway:

1. As a user, I may not be okay with those terms as they stand and may not have the legal insight to have intuited them. In as much as I have a choice, because of those terms, I may want to use Gemini less/differently or look at other providers that have crafted less shrewd and defensive legalese and processes.

2. As a executive or manager at Google, whether working directly in feature management, or advertising, or government compliance, these same terms define the framework for anything I want to develop. What "makes sense" from a defensive legal standpoint today can the open door for an exploitative or otherwise unexpected use later.

I get what you're saying here in terms of plausible or justifiable "innocence", but that probably shouldn't get in the way of some critical personal reflection for users.

> They're being retained for legal purposes.

It’s a significantly wider usage than that:

A subset of chats are reviewed by human reviewers (including Google’s trained service providers) to help improve Google services. These include Gemini models, other generative AI models that power Gemini Apps, and technologies that help reduce unintended activations.

Reviewers assess if Gemini Apps’ responses were low-quality, inaccurate, or harmful. They also suggest better responses. We use their suggestions to help our models respond better in the future.

That’s in addition to the liability language:

Chats are also reviewed to keep Gemini safe and secure for everyone and to address violations of the Google Terms of Service, including the Generative AI Prohibited Use Policy. Reviewed chats are retained for up to three years.

https://support.google.com/gemini/answer/13594961?hl=en#huma...

Some important things to take away from this:

* They’re claiming to use chat transcripts for both training and for human review to improve training, not just for liability.

* They don’t define “subset”. It could be most chats.

* They make no promises of anonymizing content, just “disconnecting” it from the account.

* This is happening for free and paid user subscriptions, by default, and the only way to stop it is to turn off chat history altogether.

Some chats are saved for 14-30 days IF they trigger a safety classifier, but they are never used to train future models or retained after that period, the objective is to verify safety. Simple as that.

If you are a business user, like in the case of Azure OpenAI, the analysis and priorities are quite different though, but the mechanism seems to be the same.

Don't care for what, store it away audit save if you have to, talk to laywers in case you need to read through them but don't let random people read my stuff.

And in this case stuff = AI discussion which feels to me similiar to my google docs documents im hoping/expecting Google also to not read

This doesn’t make sense. In such cases data could be reviewed by court order. That’s not something you’d have to agree to in tos.

But this would require an admission of limitations of the technology.

Do people sue Google if search links to a page that has misleading info that results in someone taking a wrong drug? Nope.

Do we sue the browser maker when the info on a given Web page is malicious? Also no.

So, I don't understand why we treat chatbot providers any different. Everyone should be expected not to believe a word they say without corroboration.

The primary purpose has always been improving search overall -- traditionally this has meant updating query interpretation, ranking, presentation, etc for smaller and smaller query slices ("Ah, when we show the mayor and the population in response to [Barcelona] people just revise their query to [Barcelona FC], so we should make sure to include latest match result on the top. Not so for [New York]."), and running tests on every product change to see what people like. In this case nobody cares about any _individual_ user's data, only the collective. This is basically the original "collective intelligence" contract of PageRank, albeit hillclimbing on billions of searches and clicks instead of using link structure. (Old question: does original PageRank count as machine learning, or not?)

Second is personalization -- things like "Oh, my flight or my delivery is expected on date X, handy that gmail popped a reminder on my calendar, good work Google." Or searching for a restaurant in Maps and seeing results that are biased towards the kinds of restaurants you already like, that kind of thing.

When it comes to revenue the purpose of these isn't to target ads, it's just to have you get value from and use the product, and when you are ready to spend money this is where you come to search for the thing, whatever it is.

Finally there is limited use of a limited set of data for ads relevance. There's higher CPC for ads you might actually be interested in (and if you are looking to spend money, is finding a willing seller who has compelling terms via an ad such a bad thing?) But ad revenue is mostly downstream of users being satisfied with the core product and using it more, whatever that product is. (Search, Maps, AI chat, etc)

(Bonus. My Search pro tip re: ads is leverage the modern stack and spell out your intent with a few more words when you query. We're pretty good at not serving ads for non-commercial queries. But old-school super short "searchese" queries make it hard to guess whether your intent is commercial or not.)

actually even gemma locally is great for a lot of chat purposes.

Google has this pattern of all or nothing approach.

Maps refuses to show your *current* location if you don't enable location accuracy i.e. if you won't allow it to spy on your WiFi or Bluetooth it refuses to show your current location at all. GPS, what's that?

We can disable notifications and even phone access but physical activity is the one permission that cannot be disabled for play services !?

Google apps have a lot of WTFs!

They are not transparent about it either. Anything you link to it to enable basic functionality you had on device before gets them the ability to use it for not just training but ads and measurement.

Thanks for the reminder OP!

You’d maybe want to counter with something like 'But Google/Amazon/whoever just wants to sell me stuff!'. What makes you think that Chinese B2C would just pass on that opportunity?

Just like how Kim Jong Un killed his brother using dumb pawns to actually carry out the murder.

So if your head has not grasped this reality, then they are already two steps ahead of your "What could they possibly do".

This is also the reason why congress keeps banning Chinese enterprise. It's not because they care about privacy, it's because they care about people being turned into pawns.

eg Gemini enterprise has a note under each chat session explicitly saying it’s not used to improve the models

Local models are also a thing. The Qwen 3.8 I'm running on a spare computer is definitely not training on my inputs.