Using secondary school maths to demystify AI

I do hope you're able to remember what you had for lunch without incessantly repeating it to keep it in your context window

it doesn't sound like you really understand what these statements mean. if LLMs are like any humans it's those with late stage dementia, not healthy adults

The definition of "thinking" in any of the parent comments or TFA is actually not defined. Like literally no statements are made about what is being tested.

So, if we had that we could actually discuss it. Otherwise it's just opinions about what a person believes thinking is, combined with what LLMs are doing + what the person believes they themselves do + what they believe others do. It's entirely subjective with very low SNR b/c of those confounding factors.

Is it only humans that have this need? That makes the need special, so humans are special in the universe.

We don't fully understand how brains work, but we know brains don't function like a computer. Why would a computer be assumed to function like a brain in any way, even in part, without evidence and just hopes based on marketing? And I don't just mean consumer marketing, but marketing within academia as well. For example, names like "neural networks" have always been considered metaphorical at best.

Unicorns are not bound by the laws of physics - because they do not exist.

edit : Thinking is undefined, statements about undefined cannot be verified.

It ties into another aspect of these perennial threads, where it is somehow OK for humans to engage in deluded or hallucinatory thought, but when an AI model does it, it proves they don't "think."

Right back at you, brochacho. I'm not the one making a positive claim here. You're the one who insists that it must work in a specific way because you can't conceive of any alternatives. I have never seen ANY evidence or study linking any existent AI or computer system to human cognition.

>There's no need or benefit for searching for unnecessarily complicated alternative interpretations.

Thanks, if it's alright with you I might borrow this argument next time somebody tries to tell me the world isn't flat.

>It simply doesn't look

That's one of those phrases you use when you're REALLY confident that you know what you're talking about.

> like we need anything more, in principle, to fully explain the nature of biological intelligence, and consciousness, and how brains work.

Please fully explain the nature of intelligence, consciousness, and how brains work.

>Mind as interdimensional radios, mystical souls and spirits, quantum tubules, none of that stuff has any basis in a ruthlessly rational and scientific review of the science of cognition.

well i definitely never said anything even remotely similar to that. If i didn't know any better i might call this argument a "hallucination".

This is the point, we don't know the delta between brains and AI any assumption is equivalent to my statement.

For one thing, yes, they can, obviously [1] -- when's the last time you checked? -- and for another, there are plenty of humans who seemingly cannot.

The only real difference is that with an LLM, when the context is lost, so is the learning. That will obviously need to be addressed at some point.

that they can't perform simple mathematical operations without access to external help (via tool calling)

But yet you are fine with humans requiring a calculator to perform similar tasks? Many humans are worse at basic arithmetic than an unaided transformer network. And, tellingly, we make the same kinds of errors.

or that they have to expend so much more energy to do their magic (and yes, to me they are a bit magical), which makes some wonder if what these models do is a form of refined brute-force search, rather than ideating.

Well, of course, all they are doing is searching and curve-fitting. To me, the magical thing is that they have shown us, more or less undeniably (Penrose notwithstanding), that that is all we do. Questions that have been asked for thousands of years have now been answered: there's nothing special about the human brain, except for the ability to form, consolidate, consult, and revise long-term memories.

1: E.g., https://arxiv.org/abs/2005.14165 from 2020

A lot of people seemingly haven't updated their priors after some of the more interesting results published lately, such as the performance of Google's and OpenAI's models at the 2025 Math Olympiad. Would you say that includes yourself?

If so, what do the models still have to do in order to establish that they are capable of all major forms of reasoning, and under what conditions will you accept such proof?

Sometimes, because of the consequences of otherwise, the order gets reversed

There are people confidently claiming they can’t and then other people expressing skepticism at their confidence and/or trying to get them to nail down what they mean.

... someone else points out that the same models that can't "think" are somehow turning in gold-level performance at international math and programming competitions, making Fields Medalists sit up and take notice, winning art competitions, composing music indistinguishable from human output, and making entire subreddits fail the Turing test.

So long as you accept the slide ruler as a "new kind of intelligence" everything will probably work out fine, it's the Altmannian insistence that only the LLM is of the new kind that is silly.

"Can not be measured", probably not. "We don't know how to measure", almost certainly.

I am capable of belief, and I've seen no evidence that the computer is. It's also possible that I'm the only person that is conscious. It's even possible that you are!

The argument that was actually made was "LLMs do not think".

Check out eg Dennett.... or ... his opionions about Searle; Have fun with eg... this:

"By Searle’s own count, there are over a hundred published attacks on it. He can count them, but I guess he can’t read them, for in all those years he has never to my knowledge responded in detail to the dozens of devastating criticisms they contain;"

https://www.nybooks.com/articles/1995/12/21/the-mystery-of-c...

If you were a mind supervening on the behavior of some massive time/space scale computer, how would you know? How could you tell the difference between running on a human making marks with pen and paper and running on a modern CPU? Your experience updates based on information transformations, not based on how fast the fundamental substrate is changing. When your conscious experience changes, that means your current state is substantially different from your prior state and you can recognize this difference. Our human-scale chauvinism gets in the way of properly imagining this. A mind running on a CPU or a large collection of human computers is equally plausible.

A common question people like to ask is "where is the consciousness" in such a system. This is an important question if only because it highlights the futility of such questions. Where is Microsoft Word when it is running on my computer? How can you draw a boundary around a computation when there are a multitude of essential and non-essential parts of the system that work together to construct the relevant causal dynamic. It's just not a well-defined question. There is no one place where Microsoft Word occurs nor is there any one place where consciousness occurs in a system. Is state being properly recorded and correctly leveraged to compute the next state? The consciousness is in this process.

The question is, are the people in the simulated universe real people? Do they think and feel like we do—are they conscious? Either answer seems like it can’t possibly be right!

It's been kinda discussed to oblivion in the last century, interesting that it seems people don't realize the "existing literature" and repeat the same arguments (not saying anyone is wrong).

That simulated candle is perfectly melting wax in its own simulation. Duh, it won't melt any in ours, because our arbitrary notions of "real" wax are disconnected between the two simulatons.

Thanks for stating your views clearly. I have some questions to try and understand them better:

Would you say you're sure that you aren't in a simulation while acknowledging that a simulated version of you would say the same?

What do you think happens to someone whose neurons get replaced by small computers one by one (if you're happy to assume for the sake of argument that such a thing is possible without changing the person's behavior)?

Build a simulation of creatures that evolve from simple structures (think RNA, DNA).

Now, if in this simulation, after many many iterations, the creatures start talking about consciousness, what does that tell us?

It might if the simulation includes humans observing the candle.

The connotation is simply an internal process of indeterminate length rather than one of reflexive length. So they don’t apply it when a GPU is slinging out 120 FPS in a first person shooter.

Very much like this effect https://www.reddit.com/r/opticalillusions/comments/1cedtcp/s... . Shouldn't hide complexity under a truth value.