Gemini Robotics 2 brings whole body intelligence to robots

Google has never been good at supporting developers, and it absolutely loathes end users. Support for everything that is not their ad ecosystem is miserable. There's never any long term plan, everything gets deprecated and bitrots.

After all this time it's still a pain point just to pay them for a Gemini API key.

Don't forget: resources

Yeah, something surprising to me was that I can't even add a custom MCP to gemini. It has tons of missing features.

Everything would be perfect if only you were CEO of both companies!

    “I also see that I have two Microsoft Outlooks, and neither one of those are working if you want to remote in and check"

Wait? What are you talking about? We're already drowning in them. There's literally more than Google messenger apps. An incomplete list of Google AI chatbots currently online:

    Gemini
    Gemini Live
    Gemini Code Assist
    Gemini for Google Cloud
    NotebookLM
    Gmail
    Google Docs
    Google Sheets
    Google Slides
    Google Drive
    Google Chat
    Google Meet
    Google Calendar
    Google Forms
    Google Vids
    AppSheet
    Workspace Studio
    Google Search
    Google app
    Google Chrome
    Google Lens
    Google Maps
    Google Flights
    Google Hotels
    Google Shopping
    Google Photos
    YouTube
    Google Messages
    Google Translate
    Android
    Circle to Search
    Google Home
    Google Play
    Pixel apps and features
    Google Flow
    Google AI Studio

> I mean they did practically invent LLMs

Didn't all those authors leave Google?

Maybe it was just using Google Search, and not finding anything ...

I won't comment much on this, but I'd say these articles are 2-3 years late and reduces the agency of the people, who in many cases moved to Gemini because they wanted to work on LLMs

> probably the first real-world deployment of a self-improving system that beat humans.

Isn't that Checkers right from the dawn of reinforcement learning?

I think that credit might go to IBM:

https://en.wikipedia.org/wiki/TD-Gammon

If you had asked HN 3 years ago what would code-generating market would be, they would have used the exact same reasoning and given us the opinion "Why no one will pay for glorified auto-complete"

People are going to be surprised by how obscenely profitable both will be in a couple of years

I like how someone can ignore that DeepMind invented AlphaFold, while OpenAI and Anthropic have zero experience shipping AI protein folding.

This entire industry might be hopelessly tribalist, nowadays.

I love how HN ignores past execution results

the word delightful has lost all meaning to me

I'm not sure.

I think house cleaning/chores require at least a "part-time" job's work of work for the average household (especially with children). We may be atypical, but my partner and I don't want to hire out a cleaning service and deal with the whole human component. But, I think we'd gladly pay 15k+ if it allowed us to take on the additional gainful workload. Even at a 20k price tag, I suspect we would likely have made up the difference in under a year.

Robot lawnmowers are mostly over $1000 already and they only do one thing. Heck, a lot of lawnmowers are over $1000.

You can pay over $500 for an automatic cat litter box. Robot vacuums can be cheap but run to $600 or so. Household appliances is a robust sector where people have a proven track record of spending a lot of money.

> For homes, it's a very different bar. You'd have a hard time convincing most American families to purchase anything with a >$1000 price tag.

There are plenty of white collar employees in cities paying $500+/month for someone to come clean their home. As one of those people, I could easily see paying low five figures for something like this if it actually worked well and could replace most household labor.

>You'd have a hard time convincing most American families to purchase anything with a >$1000 price tag.

Like cars, they'll likely purchase them through finance deals, with a monthly payment. Or perhaps even rent/lease them.

Another huge blocker (that is essentially just cost too) is maintenance. These things are like 1000x more complex than a car, need to work in a diverse range of environments, handle water, household chemicals, different humidities, kids, etc… maybe eventually robots will service robots but that seems very hard!

And talking of kids, these will have to be exceedingly safe, a 6ft machine falling on a child is probably a worry most households could do without.

> You'd have a hard time convincing most American families to purchase anything with a >$1000 price tag.

They all have cars. Turns out you can finance things, and turn them into forever debts. Paying $30K to never have to do chores again, or maybe even cook again, and live in a clean neat environment is not only an amazing proposition that opens up a lot of free time for people, but will also have a lot of social pressure behind it. We have social pressure that causes people to buy >$1000 phones to virtually no marginal benefit over $20 phones.

They will be paid off at $500/mo over 10 years, and most renters might just use the one that came with their apartment. If they're more like $80K or $100K I could see having a problem selling them. But if they basically turned apartment buildings into hotels, they would be a bargain for landlords; just give them the keys.

edit: of course, sci-fi has already rehearsed this. You can watch any number of movies and tv shows with families walking through a robot showroom guided by a guy in a cheap suit offering to give them the best deal. I'm sure you could find written examples from the 40s.

Unitree's R1 humanoid robot is only about $6000, and it's still a nascent, smallish scale technology. They will come down.

If the future home robots are any good, it saves you from buying a dishwasher and robot vacuum. It can replace a maid/cleaning service and gardener/lawn-mowing service. For anyone paying for those services, it pays for itself.

$1000 isn't that high. Probably about a third of American households have an appliance over $2000, which is still a massive market.

They could be shared, and work practically non-stop (minus nights, as that would wake people up). That'd bring the per-consumer cost down by a large fraction. The sharing would likely be organized by a commercial third-party, but still, it could be much cheaper than human cleaners.

Having said that, it could still be decades till the robot is ready to be released into human homes unsupervised. The variation in human homes, what people store in them etc. is not easy. However, it could be ready for hotels much sooner.

If someone can make a robot that can keep a house and surroundings clean and pretty, it is going to sell like hot cakes..

You never know the future but imagine the people who were bearish on personal computers when it wasn't a thing yet but was being discussed. Imagine Email, Internet etc etc. Now they are commodities. I am sure one day AI Robots will be a commodity that we could buy for $1000. May be 10 years from now. But it is coming.

Price point: they’ll be very low volume and very high price. There’s enough inequality and plenty of cash at the top to support such luxury robots. Everyone else will be doing their chores the old fashioned way, with their own hands.

I think domestic robots will make a lot of incremental progress and that few, if any, will be humanoid. We'll see a laundry folder and sock picker-upper and they'll be more like roombas or just big cubes. And tethered.

Yeah if you can afford the probably 10x it will actually cost, you can also just afford a cleaner to come twice a week and be able to do a wider variety of tasks (anyone going to trust this with ironing?)

if you give me a robot that cleans the house and can do dishes and laundry I'll easily pay the cost of a compact car for that. I know a LOT of working families that would feel similarly.

I mean, a twice-monthly house-cleaning service is running you ballpark 5k/year and people treat that shit as non-negotiable. I tried to claw back that line item after our second baby was born (wife didn't want them in the house anyway when we had a newborn), and it was completely unsuccessful. She claimed misery and all our friends were on her side.

So the lifestyle stayed inflated.

if all it could do was operate a lawn mower and a shovel, the annual value to a homeowner in the north east is ~1-2k depending on lawn size assuming they value their free time at the same amount as their hourly wage.

what you have to keep in mind is homeowners salary is way above avg salary so anything you can do to effect them has a much larger impact than it would seem at first.

I think ~car is a more likely range, there's a lot of value in cooking / cleaning, Americans buy cars pretty frequently

Just slap an Apple logo on it and you can charge whatever you want.

Self-driving car brings a robot cleaner to your door step once a week.

[dead]

> Hell, the only ones that can afford them will be the same that would hire human household help.

But maybe they'd rather deal with a robot than a human?

The dishwasher is out of the way because you made space for it out of the way. We'll probably adapt our spaces for robots too eventually.

For example, you could store far more things out of easy human reach.

This was my thought too. An unsupervised robot can cause 10s of thousands in damage by accidentally ripping a water line and flooding your house or just breaking all of your dishes. There’s no level to how dumb and awful LLMs can hallucinate

This is not the bar for societal adoption, and we can already see a demonstration that it is false.

A waymo is basically an industrial strength robot. It operates among inherently unpredictable humans already. There is no guarantee that it will never, ever hurt a human.

And yet in many cities around the world, you can call a waymo and ride it and society accepts it.

I don't think that there's any inherent difficulty to do with physical bodies, it's just that the learning techniques used by the big labs, that is to say, various forms of Reinforcement Learning, don't transfer to the real world. Because in the real world if you try stuff at random until you figure out the right policy, you end up destroying too many robot bodies. Imitation learning that went wild a couple years ago with ALOHA looks like a great way to get around this on paper, but then you realise that you have to train your robot for hours on every single task you want it to perform and it still won't generalise even between similar tasks. The sim-2-real gap also remains a big obstacle and you can't just train in simulation and transfer to the real world seamlessly; and then the seams end up being more like chasms.

Then there's the problem of generalisation. RL is insane in figuring out the dynamics of any arbitrary environment you drop it in at any time. Unfortunately once you have a policy trained in one environment (or one task) you have to train again for the next environment (or task) you want to deal with. The number of different environments, situations or tasks in the real world ends up being overwhelmingly large.

So for example DeepMind has been trying for ever to train various robots to do stuff like grasp arbitrary objects by training in the real world or in simulation and still grasping is an unsolved problem, in the general case; and in fact in most cases.

With LLMs it was different in that there was a www of text to train on and the stakes are lower because there's no sim-2-real gap and there's no need to deal with the real world, in training or deployment, it's just text on a screen. That's very hard to replicate in robotics.

There are major breakthroughs still to be made and they aren't currently even on the horizon. And no, I don't have the answer with my super secrete robotic AGI homebrew project either, just saying :P

I agree

I only wish I could convince some humans in my life to slow down and do things with care.

They didn't mention anything about folding clothes so I suspect this comment is just an ad

No, the above is absolutely exactly true. Robots folding clothes and tying shoelaces etc is nothing but a tech demo at this point.

It's difficult to grok this because if you watch a human folding a t-shirt, you can reliably predict that the same human will fold a different t-shirt just as well, and in fact be perfectly capable of folding a wide variety of other clothes items as well. Not so for robots. With robots, what you see is precisely what you get. If you see a robot folding a t-shirt, all that means is that that particular robot can fold that particular t-shirt. The state of the art today is that the same robot cannot be expected to be able to fold a different t-shirt.

For example, see this article about Mobile ALOHA at Google. There's a passage where the visiting, awe-struck, journalist asks whether the robot he's just seen tying up a pair of shoelaces can tie up his own shoe.

“If I gave it my shoe,” I ventured, “would it just totally fail?”

“We could try,” Tompson said. I removed my right sneaker, with apologies to anyone forced to handle it. Tompson gamely placed it on the table, while Driess reloaded the policy.

“To set expectations,” Driess said, “this is a task that is thought of as being impossible.”

Tompson eyed his new experimental subject with some trepidation. “Very short shoelaces,” he said.

The policy booted up, and the claws set to work. This time, they poked at the shoelace without getting a grip. “Do you give consent for your shoe to be destroyed?” Driess joked, as the hands grabbed at the tongue. Tompson let them try for a few more seconds before hitting the Failure pedal.

https://archive.ph/CiJJG#selection-1887.0-1915.291

(Original: https://www.newyorker.com/magazine/2024/12/02/a-revolution-i...)

As to ACT-2 which basically uses the same techniques as ALOHA (imitation learning) far as I can tell, that's a commercial product and the information they give on their site is difficult to parse. E.g. they say they have 99.1% ±0.3 success rate, 778 successful folds and 9 garment types which is low enough to engender some trust they're not trying to inflate their numbers, but they don't say whether they trained on the garments used in evaluation or not. Chances are they did, because that's the current limit of the technology, i.e. if the garment being folded is unseen (as opposed to the environment, which they tout) then performance is basically random. So either ACT-2 have a major breakthrough that is a few leaps and bounds away from the current state of the art, or you've just watched a tech demo.

The fact that they only advertise "9 garment types" though is a big hint: they have the same problem with generalisation as everybody else at this point in time.

This blog post

Actuators on modern humanoids use cycloids or planetary reducers, not strain waves. They are much more compliant and backdrivable because of the huge reduction in reflected inertia.

> this is a tangent but you should revisit your assumptions about giraffe anatomy

I don't know. This is the scenario i was thinking of: I imagine a giraffe taking a palette of goods (maybe 1200kg) into its mouth and trying to lift it off from a high shelf. I don't think that will go well for the giraffe. Totally normal load for a forklift, impossible for a giraffe.

The other scenario I was thinking of: Giraffe picking items from totes, but the totes can be very high. Each individual item is light enough for a giraffe to not immediately break its neck. But I would not be surprised if the repeated stress of raising and lowering its head hundreds per hour would break something in the giraffe's body. If you ever seen a giraffe bend down to the ground you see that it is a whole operation. Plus the items will be all covered in giraffe saliva.

> their necks actually weigh a tremendous amount

I didn't doubt that for a second. What I doubt is that they have any useful capacity to carry extra weight besides their own neck/head. Like economically useful capacity. What work could a giraffe do in a warehouse which doesn't break the giraffe if it is doing that work all day every day for years.

It was Richard Dawkins dissecting it IIRC in front of a lecture hall where he makes a point that if there were a God designing animals, he wouldn’t have made this insane thing.

What innovations specifically?

yes, for heavy lift and bulk manipulation high gear ratios are still necessary. Maybe someday cable drives will be feasible. No question that we are not at human performance yet or that there isn't a ways to go. This is the specific claim I was arguing against:

> There has been no innovation in robotic actuators since Honda's Asimo.

I believe we are 100% making tangible progress and the technology is improving faster than it ever has.

I mean, there's a time in history when the population successfully brought out the guillotines and made real change against centralized power.

> If all goes well, imagine a world in which you work because you want to not because you have to.

And if all goes bad, imagine a world in which you and your family are homeless and starve to death.

Luckily the politicians and business leaders in place today who are going to be responsible for navigating us to one of these outcomes are the adults in the room, very ethical, even keeled and not the least bit corrupt. So... we should be fine! /s

I think this is way too optimistic. I suspect these things will be only available to the already-quite-wealthy. While the poor (i.e., not billionaires) are left to rot and starve without jobs, property, or livelihoods. Then, when the poors (i.e. us) become too much of a headache, these things will be deployed to murder the poors (i.e. me).

I am imagining a world somewhere between the movie Elysium and Oblivion.

I'm really curious - are the people who keep saying this in denial or lacking in a basic education in economics? You can imagine all you want, but this is absolutely not a possible outcome.

"Disturbing" to be able to economically provide cooking, cleaning, laundry, yard maintenance, snow shoveling, etc., for an elderly parent who lives alone (by her choice)? Why is that disturbing?

have you ever bathed or changed diaper on old people?

Yeah, kinda. Waymos have been on the road since 2023.

If it can make any robot then it’s not very specialized.

We have Google, Figure, Tesla, Boston Dynamics, and literally dozens of Chinese companies building humanoid robots and making huge improvements in the tech. Not there yet by any means as a general-purpose home assistant, but I don't think it will be too surprising if at least one of them succeeds

16 MHz is the chip's clock speed. Arduino PID is more like 100 Hz to 1 kHz.

Human deliberate movement runs around 2-10 Hz I think.

For this operating domain the latency comparison is human reaction time and not arduino reaction time.

Also the process that’s running at 16MHz is not the same as a full VLA. VLA is much more expressive. You think the processor that is running the VLA system is running at 10Hz or some GHz?

Why is a Roomba or automated rubbish disposal system more "ethically right" than a humanoid robot?

roombas having cameras, then also uploading them, was a misstep for sure

Second chart (it's to the right of the first chart and clicking it to scroll is easier than otherwise)

I don't think it's the dream, I think it's the cynical market-driven logic.

Automation has been rolling out for a long while now, and most of the low-hanging fruit of things that can easily be redesigned in a cost-effective way has been dealt with already. The role of these robots is to address all of the stuff that would have been automated by now if they could have.

Because every country wants cheap, expendable, tireless soldiers.

Level 5 automomy has been "only five years away" for 15 years

...and do not forget Support, Maintenance and that you will need an forever Cloud-connection...

You can point to the bitter lesson to support your claim, but on the other hand I can point to the massive investment in capital and time to get self-driving cars viable to support my more bearish view.

At this point the bitter lesson has become a meaningless shibboleth. It seems very few people have actually read the original article:

The Bitter Lesson Rich Sutton March 13, 2019

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

And even fewer are aware of the author's follow up on what his article says about the current trend in AI:

Silicon Valley Doesn't Understand The Bitter Lesson – Richard Sutton

https://youtu.be/QMGy6WY2hlM?si=0aOmgKiPfGEEXoK9

But how does LLMs help in making chat bots better, help with this "multi-sensory" data. Has this anything to do with the current AI surge with LLMs?

Has it been demonstrated? Or is it just that such things now get a lot of funding now, for no good reason?

Yeah, look carefully at that demo of tying the garbage bag strings: it's done in a very peculiar style that suggests a very specific, very precise, "algorithm" taught in an imitation learning session, which has no chance to transfer to other tasks, or even other garbage bag strings.

As usual with robot tech demos: WYSIWYG.

> political commentary from someone

No it depends on what they choose to answer. canyon289's account self-describes as Bayesian, a hallmark of rationalists, one of whose taglines is "politics is the mind-killer". Nonetheless https://www.lesswrong.com/posts/iKm2FhpWkuuBojm82/why-i-left... is one of the most upvoted posts on LessWrong (the big rationalist site) of all time. Most of the comments don't touch on the politics of the situation at all.

> someone just going about their job

They're doing a little bit more than that by actively recruiting (even more so than the usual "if this interests you we're hiring" bit at the end of a post). So I'm doing a little bit more by asking them about a recent high profile departure. It's a little bit different from the usual way employees chime in on threads.

Because we're nearing the endgame...perfect total surveillance and control will be possible soon

"Just following orders"

is how some of the worst shit in human history went down.

They're literally CREATING the tech and tools and you're saying don't even question them about potential and proven misuse?? lol

They're the first people that should be asked this question.

> Believe me, it's very common they tell me that something is bad idea

In my experience, the most they dare is to suggest better alternatives, which is the polite and useful version of telling me something is a bad idea.

But in general, they do not tell you "this whole idea is complete bogus". So far, only Gemini did. Don't even remember what about.

True, but their margins only exist because of protectionist US laws (i.e. 100% tariff on Chinese competitors).