8M users' AI conversations sold for profit by "privacy" extensions

I stick to extensions that Mozilla has manually vetted as part of the Firefox recommended extensions program.

> Firefox is committed to helping protect you against third-party software that may inadvertently compromise your data – or worse – breach your privacy with malicious intent. Before an extension receives Recommended status, it undergoes rigorous technical review by staff security experts.

https://support.mozilla.org/en-US/kb/recommended-extensions-...

I know that Google hates to pay human beings, but this is an area that needs human eyes on code, not just automated scans.

  Initialization: The accumulator a starts as self (the window object).
  Iteration 1 (b = "alert"):
  I("alert") returns string "alert".
  It tries a["alert"] (which is window["alert"]).
  This finds the alert function.

  New Accumulator a: The alert function.
  Iteration 2 (b = "Hello World"):
  I("Hello World") returns string "Hello World".
  It tries a["Hello World"]. The alert function does not have a property named "Hello World", so this is undefined.
  The || operator moves to the right side: a(b).
  It executes alert("Hello World").
  Result: A browser popup appears.

The company behind this appears to be "real" and incorporated in Delaware.

> Urban Cyber Security INC

https://opencorporates.com/companies/us_de/5136044

https://www.urbancybersec.com/about-us/

I found two addresses:

> 1007 North Orange Street 4th floor Wilmington, DE 19801 US

> 510 5th Ave 3rd floor New York, NY 10036 United States

and even a phone number: +1 917-690-8380

https://www.manhattan-nyc.com/businesses/urban-cyber-securit...

They look really legitimate on the outside, to the point that there's a fair chance they're not aware what their extension is doing. Possibly they're "victim" of this as well.

I am surprised because google review team rejects half of my extensions and apps.

Sometimes things don't make sense to me, like how "Uber Driver app access background location and there is no way to change that from settings" - https://developer.apple.com/forums/thread/783227

The permissions model for browser extensions has always been backwards. You grant full access at install time, then cross your fingers that nothing changes in an update.

What we actually need is runtime permissions that fire when the extension tries to do something suspicious - like exfiltrating data to domains that aren't related to its stated function. iOS does this reasonably well for apps. Extensions should too.

The "Recommended" badge helps but it's a bandaid. If an extension needs "read and change all data on all websites" to work, maybe it shouldn't work.

“ A few weeks ago, I was wrestling with a major life decision. Like I've grown used to doing, I opened Claude”

Is this where we’re at with AI?

As someone who has witnessed BiScience tracking in the past, I am not surprised to to hear that they might be involved in all this. They came up in the past when researchers investigated the cyberhaven compromise [1][2]. Though the correlation might not all be there its kind of disappointing

[1] https://secureannex.com/blog/cyberhaven-extension-compromise.... [2] https://secureannex.com/blog/sclpfybn-moneitization-scheme/ (referenced in the article)

I don't understand why so many people are using / trusting VPNs

"Let us handle all your internet traffic.. you can trust us.. we're free!"

No thank you.

Google needs to act on removing these extensions/doing more thorough code reviews. Reputability is everything, and they can be actually valuable (e.g. LastPass, my own extension Ward)

There has to be a better system. Maybe a public extension safety directory?

I thought manifest v3 was supposed to make chrome extensions secure?

What is the economic value of all these AI chat logs? I can see it useful for developing advertising profile. But I wonder if it's also just sold as training data for people try to build their own models?

I'm glad the extension system isn't broken (e.g. extensions being hacked). This is just scammy extensions to begin with. I've been scared of extensions since they were first offered (I did like useing greasemonkey to customize everything back in the 2000's/2010's), but I can't resist privacy badger and Ublock Origin since they are open source (but even then it's still a risk).

So much of what's aimed at nontechnical consumers these days is full of dishonesty and abuse. Microsoft kinda turned Windows into something like this, you need OneDrive "for your protection", new telemetry and ads with every update, etc.

In much of the physical world thankfully there's laws and pretty-effective enforcement against people clubbing you on the head and taking your stuff, retail stores selling fake products and empty boxes, etc.

But the tech world is this ever-boiling global cauldron of intangible software processes and code - hard to get a handle on what to even regulate. Wish people would just be decent to each other, and that that would be culturally valued over materialism and moneymaking by any possible means. Perhaps it'll make a comeback.

[flagged]

This is exactly why we need more transparency in analytics tools. When building products that handle user data, the "free" model almost always means you're the product.

The scary part is these extensions had Google's "Featured" badge. Manual review clearly isn't enough when companies can update code post-approval. We need continuous monitoring, not just one-time vetting.

For anyone building privacy-focused tools: making your data collection transparent and your business model clear upfront is the only way to build trust. Users are getting savvier about this.

How did I know this was an israeli company just by how unethical they are at scale?

Some people have mentioned that this is a U.S incorporated company (Delaware). Recommend reading Moneyland by Oliver Bullough if you want to know more about the U.S role as the new shell company haven.

The island states have been dethroned.

> This means a human at Google reviewed Urban VPN Proxy and concluded it met their standards.

Or that the review happened before the code harvested all the LLM conversations and never got reviewed after it was updated.

I wouldn't be surprised if this was done by one of those AI companies themselves!

Remember FaceBook x Onavo?

"Facebook used a Virtual Private Network (VPN) application it acquired, called Onavo Protect, as a surveillance tool to monitor user activity on competing apps and websites"

Somewhat ironically, this article has significant amounts of AI writing in it. (I've done a lot of AI writing in my own sites, and have been learning how to smother "the voice". This article doesn't do a good job of smothering.)

Is the use of WebAssembly going to make spotting these malicious extensions harder?

lol, this Urban VPN addon was available for Firefox too but got removed at some point. https://old.reddit.com/r/firefox/comments/1jb4ura/what_happe...

I think using AI is very helpful today but if the data is selt for HR it's very bad

Oh, a free of cost vpn extension that requires access to all sites and data is somehow spyware, color me surprised.

With those extensions the user's data and internet are the product, most if not all are also selling residential IP access for scrapers, bots, etc.

Good thing Google is protecting users by taking down such harmful extensions as ublock origin instead.

Why would one expect privacy with a vpn? That too a free one? With the web all traffic is encrypted point to point, which means individual sites could compromise your privacy but there is no single funnel to lose all your data. VPN is exactly that! All data goes through a single funnel and they can target anything they want

This is a huge trust failure. A VPN or ad blocker quietly harvesting full AI conversations is the opposite of what users expect, and the fact that these extensions were featured makes it even worse. This really puts the effectiveness of browser extension reviews into question.

Why is a security researcher using a Free VPN? The standard wisdom is "if its free, you're the product". So you're going to proxy all your sensitive traffic through a free thing? Its not great to trust paid services with your data, nevermind free stuff.

Sometimes knowing tech makes us think we're somehow better and can bypass high level wisdom.

Would using native AI apps only prevent this? I think so right?

Is this criminally prosecutable?

Nice write up. It would be great if the authors could follow up with a detailed technical walk through of how to use the various tooling to figure out what an extension is really doing.

Could one just feed the extension and a good prompt to claude to do this? Seems like automation CAN sniff this kind of stuff out pretty easily.

Am I just paranoid or open router is the next bomb ticking to a privacy explosion? What is their business model anyway?

Do we know for how much that type of content sells? Not that I'm interested in entering the market, but the economics of that kind of thing are always fascinating. How much are buyers willing to pay for AI conversations? I would expect the value to be pretty low

I wish Congress spent as much time fighting about issues like this vs trying to break up Google. This is far more impact.

Articles like this do a decent job of bringing awareness, but we all know Google will do absolutely nothing

What would the fallout look like if too many people start to have horror stories about how much their life is destroyed by incriminating or down right nasty or wrong ai chat history. It'll suddenly become a tool where you can't be honest. If it's not already.

> And then an uncomfortable thought: what if someone was reading all of this?

> The thought didn't let go. As a security researcher, I have the tools to answer that question.

What huh, no you don't! As a security researcher you should know better!

Can someone just AI all the privacy policies please and tell us who else is pranking?

> A "Featured" badge from Google, meaning it had passed manual review and met what Google describes as "a high standard of user experience and design."

Trusting Google with your privacy is like putting the fox in charge of the henhouse.

If you want a VPN you can trust, deploy your own with AlgoVPN: https://github.com/trailofbits/algo

"And then an uncomfortable thought: what if someone was reading all of this?"

If you really are a security researcher then that's not true. You already know all this.

Wasn't the whole coercion Google did around Manifest V3 in the name of security?

How is it possible to have extensions this egregiously malicious in the new system?

I treat extensions like they're all capable of privileged local code execution. My selection is very vetted and very small.

Why these browser extensions cannot live in a guarded sandbox? Extensions are given full access to whatever is available on any page. I had legacy React developer tools and Redux DevTools installed for years. What a great attack vector.

From my experience, Google does not do a thorough app review. Reviewers get maybe a few minutes to review and move on due to the volume of apps awaiting review.

This is digital assault of 8m people and should be treated that way.

Only those users that were stupid enough to "converse" with their chatbot.

8 million users on sketchy VPN extensions.

70 thousand users on what I would actually call "privacy" extensions.

Bit of a misleading title then.

With hardcoded flags like “sendClaudeMessages” and “sendChatgptMessages”, they weren’t even trying to hide it.

Is this the same Google that is preventing us from installing unapproved software on our phones?

> A free VPN promising privacy and security.

If you are not paying for the product, you are the product.

If the business model isn't obvious, you are the product

If the product is free, you are the product.

The footer animation of koi.ai is so cool.

Pro tip - never install any browser extensions. Avoid like a plague. I had a couple installed that were “legitimate” and I have direct evidence of them leaking/selling my browsing data. Just avoid.

> We asked Wings, our agentic-AI risk engine

I hate to be that guy, but I am having a difficult time verifying any of this. How likely is it that this is entirely hallucinated? Can anyone independently verify this?

These converstions can be used to train a competing AI

There were these two people.

And um, a boy and a girl.

...

Anyway, the thing was that one day they started acting kinda funny. Kinda, weird.

They started being seen exchanging tokens of affection.

And it was rumoured they were engaging in...

b xncbb vnxv

fasdfas

Note that this is a pretty blatant GDPR violation and you should report this to the local data protection agency if you are a EU resident and care about this (especially if you've used this extension). Their privacy policy claims the data collection is consent-based and that the app settings also let you revoke this consent. According to the article, the latter isn't the case and the user is never informed of the extent of the collection and the risk of sensitive or specially protected personal information (e.g. sexual orientation) being part of the data they're collecting. Their privacy policy states the collected data is filtered to remove this kind of information but that's irrelevant because processing necessarily happens after collection and the GDPR already applies at the start of that pipeline.

If Urban VPN is indeed closely affiliated with the data broker, a GDPR fine might also affect that company too given how these fines work. There is a high bar for the kind of misconduct that would result in a fine but it seems plausible that they're being knowingly and deliberately deceptive and engaging in widespread data collection that is intentionally invasive and covert. That would be a textbook example for the kind of behavior the GDPR is meant to target with fines.

The same likely applies to the other extensions mentioned in the article. Yes, "if the product is free, you are the product" but that is exactly why the GDPR exists. The problem isn't that they're harvesting user data but that they're being intentionally deceptive and misleading in their statements about this, claim they are using consent as the legal basis without having obtained it[0], and they're explicitly contradicting themselves in their claims ("we're not collecting sensitive information that would need special consideration but if we do we make sure to find it and remove it before sharing your information but don't worry because it's mostly used in aggregate except when it isn't"). Just because you except some bruising when picking up martial arts as a hobby doesn't mean your sparring partner gets to pummel your face in when you're already knocked out.

[0]: Because "consent" seems to be a hard concept for some people to grasp: it's literally analogous to what you'd want to establish before having sex with someone (though to be fair: the laws are much more lenient about unclear consent for sex because it's less reasonable to expect it to be documented with a paper trail like you can easily do for software). I'll try to keep it SFW but my place of work is not your place of work so think carefully if you want to copy this into your next Powerpoint presentation.

Does your prospective sexual partner have any reason to strongly believe that they can't refuse your advances because doing so would limit their access to something else (e.g. you took them on a date in your car and they can't afford a taxi/uber and public transport isn't available so they rely on you to get back home, aka "the implication")? Then they can't give you voluntary consent because you're (intentionally or not) pressuring them into it. The same goes if you make it much harder for them to refuse than to agree (I can't think of a sex analogy for this because this seems obvious in direct human interactions but somehow some people still think hiding "reject all non-essential" is an option you are allowed to hide between two more steps when the "accept all" button is right there even if the law explicitly prohibits these shenanigans).

Is your prospective sexual partner underage or do they appear extremely naive (e.g. you suspect they've never had any sex ed and don't know what having sex might entail or the risks involved like pregnancy, STIs or, depending on the acts, potential injuries)? Then they probably can't give you informed consent because they don't fully understand what they're consenting to. For data processing this would be failure to disclose the nature of the collection/processing/storage that's about to happen. And no, throwing the entire 100 page privacy policy at them with a consent dialog at the start hardly counts the same way throwing a biology textbook at a minor doesn't make them able to consent.

Is your prospective sexual partner giving you mixed signals but seems to be generally okay with the idea of "taking things further"? Then you're still missing specific consent and better take things one step at a time checking in on them if they're still comfortable with the direction you're taking things before you decide to raw dog their butt (even if they might turn out to be into that). Or in software terms, it's probably better to limit the things you seek consent for to what's currently happening for the user (e.g. a checkbox on a contact form that informs them what you actually intend to do with that data specifically) rather than try to get it all in one big consent modal at the start - this also comes with the advantage that you can directly demonstrate when and how the specific consent relevant to that data was obtained when later having to justify how that data was used in case something goes wrong.

Is your now-active sexual partner in a position where they can no longer tell you to stop (e.g. because they're tied up and ball-gagged)? Then the consent you did obtain isn't revokable (and thus again invalid) because they need to be able to opt out (this is what "safe words" are for and why your dentist tells you to raise your hand where they can see it if you need them to stop during a procedure - given that it's hard to talk with someone's hands in your mouth). In software this means withdrawing consent (or "opting out") should be as easy as it was to give it in the first place - an easy solution is having a "privacy settings" screen easily accessible in the same place as the privacy policy and other mandatory information that at the very least covers everything you stuffed in that consent dialog I told you not to use, as well as anything you tucked away in other forms downstream. This also gives you a nice place to link to at every opportunity to keep your user at ease and relaxed to make the journey more enjoyable for both of you.

TLDR: AI company uses AI to write blog post about abusive AI chrome extension

(Yes it really is AI-written / AI-assisted. If your AI detectors don’t go off when you read it you need to be retrained.)

ctrl-f israel: 1 result found

Deleted.

It's ridiculous how many comments are being removed.