Discovering Cryptographic Weaknesses with Claude

(I'll caveat that by saying I think machine learning fundamentals are useful for evaluating any estimator. And an ML background can be good to give one an appreciation of how hard some tasks are to estimate, such as machine translation, summarization, code generation, and others)

Focusing on a better prompt is likely to get to the correct result faster than incomplete prompts and lots of "no change this ..." replies.

Also, I've heard anecdotally that LLMs will underweight the earliest prompt text once context gets too long, so reminding the LLM of the most important aspects of the prompt seems to be perhaps valuable and certainly what lots of humans attempt.

> Typical users run software written by atypical users.

https://news.ycombinator.com/item?id=49084936

This extends to everything. Anthropic has a few thousand engineers, but millions of (also engineer) users. Entire business can be built on niches that are at most a few week pet project for a team there, that can inevitably and significantly outperform them, despite being the people behind the thing.

I'm sure I'm not the only one here who jumped into this whole agentic stuff, built some tooling to make things comfy, only to see that tooling all be increasingly introduced as prim and proper features in the various harnesses weeks later.

The token cost is amortized for longer conversations, but I find it bothersome that there's all this implicit instruction I didn't write or am now obligated to understand.

I make a custom agent prompt with "Defer to the user." and little else.

Skills, CLAUDE.md/AGENTS.md should only ever be used if the model struggle at something or doesn't know how to use something. Vast majority of project should never need a skill or CLAUDE.md. If you writing React apps you don't need these.

Give a LLM a bash tool and a prompt and it will outperform your complex setup with skills and tools.

Two triggers: random and some half-reliable spiral / loop detection.

The spined off has instructions to check what the agent is doing, compare it to what it’s supposed to do, and either offer suggestions, refocus it, or do nothing. And its response then gets injected in the agent’s context.

Not perfect, but surprisingly effective for such a simple thing.

Context management is still important, though. If you get to a certain amount of context, things start performing really badly.

> we anthropig fire employes makr company run no mistkaes

> Importantly, this is just one of many (autonomous) sessions where Claude worked on discovering new ideas. Many sessions resulted in no new discoveries; other follow-up sessions improved on the insight developed in this one. This document was produced by having Claude rewrite the chain of thought to include more detail to make it easier to read.

1. I'm glad the second kind works too;

2. First kind is where I find my overall throughput to be literally constrained by my typing speed;

3. Most importantly: those prompts you quote aren't just "half-assed" like sibling comment states; they're different. The style of writing, and the typos, capture emotional valence. It's a signal.

Again, I too produce such prompts - including the exact same typos - when under pressure and irritated by the direction the model is taking.

Humans may remain superior in spatial / non-verbal reasoning for a while longer yet, and, in the meanwhile, computers may aid us in collaborating to put that to use better.

2. AI-assisted, computer-verified proofs could further democratize mathematics by reducing the power of connections to get a reviewer to look at a journal submission. We can then also decouple the two tasks of

  a. Verifying a statement is true
  b. Explaining it

Business folks riding the hype train? Maybe.

I predict the same will happen with AI: certainly the latest and greatest will still command a steep price (yes, supercomputers are still a thing) but for most people who just need something reasonably fast and powerful, cheap (or free) AI will do the trick, especially when run locally.

So no, the aristocracy won't have a lock on the technology because tech is always being democratized. Until arbitrary computation itself is outlawed (and yes, I know, governments and industry are always inching us closer to that), we'll be ok.

Many of those researchers, particularly the primary researchers and the individual(s) driving the prompts behind these big stories, have very advanced math degrees and experience. What this shows more than anything is how ML can augment expertise, the searching of solution spaces, and the connecting of dots between existing almost-there research.

But also what's left out is all the time wasted pursuing dead-ends. There's an obvious publication bias at play here, though we can't know how extreme without transparency.

In a way LLMs are, after all, trained to LARP people, including fictional characters and their tropes - this was actually exploited for jailbreaking to good effect in the late pre-agentic era (read: some two years ago). C.f. Waluigi effect. Not sure if it still holds for current models, but I can't imagine why it would not.

First, is it reproducible consistently at ~50% of workers? If not, what is the rate.

Second, are there any lessons to be learned here to increase the rate of success by changing models/weights/training?

The news by itself isn't really good news. But it could lead to good news. Maybe.

Although if RSA had a vulnerability I would be very very shocked probably because I still haven't learnt post quantum encryption algorithms enough to really feel like they should be unbreable...

If there is a researcher or someone in space how should I feel about it. Is it as bad as RSA being completely broken open?

I do understand that AI will get better, and a lot actually at very easily verifiable tasks but this one I find it hard to wrap my head around because of my ignorance.

It is standard in cryptography to analyze ciphers under this kind of attack, which is stronger than normal attacks, because a cipher that resists to a stronger attack will also resist to weaker attacks, so using the strongest possible attack increases the confidence in a cipher.

While using the strongest attack for testing a cipher remains the correct method, chosen-plaintext attacks are no longer realistic today, so even when a cipher appears somewhat vulnerable to such attacks that does not imply that it is vulnerable in normal use.

The reason is that the modes of operation for ciphers where the base cipher can be attacked with chosen plaintexts are obsolete. The most frequently used modes of operation are now modes like the counter mode (e.g. in AES GCM), where it is impossible to perform a chosen plaintext attack (i.e. where you must trick the victim to encrypt a text that you choose, but in counter mode the cipher only encrypts a sequence of numbers chosen by the intended victim, which cannot be influenced by the attacker).

https://hawk-sign.info/

PS: Never heard of LEA, looks like a Korean cryptography standard equivalent to AES. I don't know what was the previous best attack on it. Maybe weak or strong depending on that, since it's an attack on chosen-text.

I feel like there is a pattern emerging regarding the type of novel discoveries LLMs are good at finding, but it will take some more data points to see if the trend solidifies.

It’s the difference between having an original thought or the ability to extrapolate one based on data vs the ability to ingest someone else’s thought and validate/expand on it.

That is a huge difference.

HAWK is a signature algorithm, not encryption.