AI is dramatically accelerating AI research

Benchmarks are by definition a curated set of things that can be measured and evaluated.

Things that can be measured and evaluated can be (expensively) trained on.

The progress we are observing comes from figuring out how to measure and evaluate particular aspects of intelligence, and then expensively training on feedback loops made of those exact measurements (literally benchmaxing).

What is not shown in these graphs are the set of things that intelligence can do, which are difficult, expensive or impossible to measure and evaluate.

That set of things is much larger than the set of benchmarks represented in the graph.

Therefore the graph is misleading in the sense that the "superintelligence" being targeted is "jagged" to use Karpathy's term, and we infact already have jagged superintelligence (calculators, compilers, Bayesian spam classifiers, etc.)

We are witnessing a profound explosion of much more jagged superintelligence, restricted to the exact domains that can be (expensively) RL'ed on.

If I had to do my individual tasks without AI, I would be slower. But if I take a step back, I'm really not changing the world any faster than I was before.

Anecdotally it seems like the majority of continued uplift comes from models doing more and exploring more of the search space, and we are getting better at giving them embarrassingly parallel problems to Hoover up with infinite token spend.

That is, unless this is saying that automated AI research is going to get us to "smarter models with fewer parameters" and of course, these are faster to train, then I can see how the exponential loop really gets going, but there's certainly an information theoretical limit here?

This combined with ability to massively scale experiments for even (or maybe particularly) the most mundane performance tweaks might just be making the process of improving AI more efficient.

Im not telling it what to do, i ask it for ar eport/analysis and research and read its article and tell it what to implement based on 'feelings' and it works.

Limit is hardware and tokens and an AI Researcher not having hardware or tokens is probably not what they refer to.

Alone normal researchers were quite handicaped: converting data from tool a to python library x, understanding how tools work, doing basic google research etc.

On the other hand I still only get meaningless blabla when I let them write s.th. without telling what is important.

I read it about 20 years ago, and once it got into nanobots and Kurzweil's vision that the whole universe would be intelligent, I didn't find the book credible.

But I'm wondering if the short-term prediction, that in the 2030s we won't be able to anticipate what tech does for us, (IE, the Singularity,) is useful?

https://en.wikipedia.org/wiki/The_Singularity_Is_Near

I suspect that part of the reason some people are hoping for that to happen is that they thoroughly mix up AGI and ASI. We do not need some kind of omnipotent god-like super intelligence in order to get much greater than human level capabilities. We are actually almost there in a lot of ways. In some aspects like speed and breadth of knowledge, very superhuman.

What I think is going to happen is that within a few years, maybe 18 months to 5 years, the new materials and compute-in-memory research is going to land in data centers at scale.

That is going to be 10-50+ larger models, 10-50+ faster, much smarter than humans. 50+ less electricity usage.

At some point, it is going to be obvious to people that if we want to maintain control of planet earth then we need to maintain control of AI research to some degree.

Assuming that superintelligence will be benevolent and just.. going for it to me is something that might earn the human species a Darwin award.

Could be from a sandwich that tastes like the best sandwich ever, but is super bad for your cholesterol or something.

I dont know where Im going with this.

Actually did some research I think the term is "Wireheading" from a story/theory from the 50s

In my own work, I've found that it's much faster to brainstorm with a frontier model like Opus. I'll have an idea and be able to get a working implementation and try it out on toy problems within O(minutes), instead of O(weeks) earlier.

These people are literally just a public relations operation for the big AI labs. I equate them more with a lobbying firm than I do a research firm.

https://news.ycombinator.com/item?id=35491162

> we would reach extreme superintelligence within a year

What does infinite capabilities mean? How about putting the growth on a log scale and see how that curve might look?

What the hell is extreme superintelligence?

AIs do not kill people, people kill people.

I dont think super intelligence will be here in a year, I think within a year AI/llms will have a similar sentiment surrounding them that NFTs do know (it already does for most normies). And most people will be recovering from a hype induced mania they lived in for two years straight.

Of course its more useful than NFTs, but when there's no giant bag to farm these hype monsters will inevitably disappear.

The bags are drying up fortunately and tokens/inference seems to be experiencing deflation, and hardware prices will eventually pop and contribute to further deflation, Then all the bag chasers will move back to crypto or whatever gets hot next.

Opus 5.5 is so amazing I'm not surprised if it can improve itself.

Therefore it cannot avoid taxes the way they did before (using shell companies to route the money through tax havens). It'll be expensive, every government will double the price or more, and you're selling to consumers, and the products will be at the mercy of every government where they try to sell AI and other services.

People don't get how serious this is. If AI doubles revenue but drives silicon valley firms' tax rates from 5% to 10%, it's a loss.

Add to that that it's something silicon valley firms have never done before.

That is factually incorrect, and seems to be much of the premise behind the authors way of thinking.

EDIT - No it doesn't. I replied to the wrong post. Mea Culpa.

After the election, titles like this just irritate me. Every piece written about AI progress is:

"The machines taking jobs, flooding the internet with sludge and poisoning our online experiences, are getting better even faster! Isn't this incredible? Yeah but sorry we also think it's only riskier than ever to deploy these systems at scale. That being said, we're still going to do it, go public and take institutional money from people's retirement funds, destroy the job market, and make 3000 Anthropic employees and 500 investors really rich. There wasn't anything we could do besides write a lot of ink on it and be shocked we lost every local data center referendum."