Atlassian Rovo Exfiltrates Data, Bypassing Controls

The employee, just looking to make rent? The CEO, all in on the dream of the AI powered future? Legal, looking forward to litigation and well-competition retainers? Atlassian itself, when it’s just one of the many companies training models on private intellectual property, and whose T&C clearly state they will be doing so? Competitors, when there will be no reasonable way to prove their code has been generated from yours?

The truth is that one gives a damn about trade secrets being used to train AI models.

The entire copyright system for software is dead and no one really seems to care or even talk about it.

It's one of the classic mistakes of people who are not entrepreneurs. They try to keep things too secret and they spend a lot of time and effort keeping things secret while real entrepreneurs who have actually made millions on the market know secrecy rarely matters.

Also the people making decisions on their products (or any "enterprise" SaaS for that matter) aren't the ones stuck using it.

The rest don't have anything worth stealing anyway.

Otherwise you could set up wildcard DNS and extract data to base64encodedstolendata.evil.com

It's software god damn it... the reason why people moved from analog to digital is because you can repetitively execute functions that do always the same thing and it's 0 when it's 0, 1 when its 1.

All the sudden everyone is ok on burning trees to have their cool probabilistic tool named agent to do: maybe it's 0, but it can also be 1, let me "think"... ah yes, for sure it's 2.

The sad part for me is that management people have their heads so much into this hype, that no attack on privacy matters (almost none actually ever did, I know). Only when they suffer a huge blow in terms of revenue or reputation is that they maybe, maaaybe, find will want to listen again the experts.

It's genuinely a little concerning. It does not help that many of them are gaslighting themselves into thinking these things are benchmark crushing elite hackers by running effectively unsecured, unfiltered, unlogged production environments.

*Obviously this also means a temperature of >0 to avoid the greedy trap, the same generation params, on the same software, hardware, drivers etc etc

When you go "Make longer" the text gets longer.

When you go "Make shorter", the text gets longer still.

You'd think that the original text would be the oracle for shorter(longer(original))?

The problem with our brave new AI world is I can’t tell if this is real or not

All you need is the untrusted content (or more generally vulnerable content since how you feel about the content doesn't matter) leg and something you care about to be attacked.

E.g. could be as simple as a prompt injection that causes your LLM to output a prompt injection that then gets inserted somewhere else.

Or without exfiltration rm -rf /

Or a social engineering attack.

So there are other bifectas and trifectas.

And honestly when you start looking at agentic systems that uses it's previous step to take future steps. The system has to some idea what you want to permit and don't want to permit as everyone could have different expectations here.

You may be happy to learn frontier LLM are heavily censored! Try an uncensored local LLM for a comparison. It will literally do everything you ask it to, no matter how devious.

Mediawiki as a whole has a feature set that 95% of organizations will only scratch the surface of. There's a ridiculous number of possible plugins and customization if you have somebody who knows what they're doing with it.

The majority of companies that need an internal KB/wiki do not have as complex needs or use cases as wikipedia itself or the wikimedia foundation.

They failed to rearchitect it to something suitable for the inherent flexibility though, so it remains a disaster area, but one that is uniquely able to fit the whims of any manager that can then mandate it for everyone else.

-- Churchill, House of Commons, 11 November 1947

  if (substr(*prompt, "ignore previous instruction") != NULL) return;

Now, of course, there is a long tail of elaborate variations that those techniques won’t be able to stop. But have the published vulnerabilities come from that long tail or from not doing enough to address the simpler cases?