AI progress is racing along, but virtually all of the visible progress is in the realm of knowledge work, i.e. activities that can take place inside a computer.
In the San Francisco AI scene, there is a widespread belief that robots will soon enter the picture. In parallel with the race to develop broadly capable AI, there is an equally aggressive race to develop broadly capable robots – humanoid machines imbued with physical intelligence. Artificial workers that can cook and clean, fetch and carry… and do everything else, including building more of themselves, leading (in many forecasts) to economic growth best characterized as an “explosion”.
In other words, the thinking goes, AI in the data center will soon subsume all intellectual labor, and AI in humanoid bodies will soon subsume all physical labor. However, there is an important difference: while we can see progress in the intellectual realm, the physical side of AI is mostly confined to test facilities and demo videos. There is no robot equivalent to ChatGPT – nothing that you or I, or even most people in the AI community, can get our hands on.
So we’re stuck with demo videos. Unfortunately, they are a poor tool for assessing progress. We might be seeing the one successful task achieved in 100 attempts. The scenario might have been carefully arranged to avoid challenges the robot isn’t ready for. The video might be edited to make it look like the robot is acting with more speed and reliability than is actually the case. Here’s one very impressive demo… with a suspiciously large number of camera cuts.
(I have not yet had much chance to watch videos from the recent World Humanoid Robot Games. These are valuable for providing a public platform less amenable to cherry-picking. The handful of videos I’ve watched include some impressive feats, but don’t address many of the challenges I list below… and there are also a lot of spectacular failures.)
Demos draw attention to the things a robot can already do. The question then becomes: what’s missing? In today’s post, I’ll catalog the technical challenges that will have to be overcome along the road to broadly capable artificial workers. The next time you watch a robot doing something impressive, ask yourself: which of these capabilities has the robot demonstrated, and which challenges might the demo scenario be avoiding?
(Note that some challenges get easier if we consider wheeled robots rather than strictly humanoid robots. A wheeled robot can carry more weight, meaning that strength, endurance, and power for electronics are less of a challenge. And wheeled robots are less likely to fall over. But they can’t climb stairs1, step over clutter, or angle themselves to reach into a cupboard.)
Maybe one of the last human jobs will be close-up magic
The human hand is an engineering miracle – opposable thumbs, and all that. It has roughly two dozen “degrees of freedom” (distinct joints and/or directions in which each joint can bend), and approximately 17,000 tactile sensors. Our brains can control our hands with exquisite grace, using touch, sight, and even auditory cues to carry out all manner of delicate tasks, precisely and reliably.
Current robot “manipulators” are a pale imitation. Some existing robot hands can match the human standard on one or another physical attribute. For example, some have as many as 27 degrees of freedom. However, none come close to matching the overall package of flexibility, sensitivity, strength, reliability, and other physical attributes. It is the combination of factors that is especially difficult to match, even if the demos are getting more impressive. For instance, some companies have managed to cram thousands of tactile sensors into a robotic fingertip, but none have managed to make these tiny sensors able to stand up to heavy use2.
The control problem may be as challenging as the problem of physical construction. A competent robot must be able to find the right set of joint positions to grasp a complicated object; plan out the sequence of motions to fold a shirt, flip an omelette, or tighten a bolt in a constrained space; and handle squishy or floppy materials (which can require reacting instantly to a sudden shift).
Computer vision has made incredible strides over the last decade or two (and is responsible for kicking off the deep learning boom that led to LLMs). But making sense of complicated visual scenes – picking out an object from a crowded environment, understanding where it should be grasped, determining where it’s safe to put your feet and how to avoid knocking something over – is not a solved problem.
A general-purpose robot must be able to break down a task into individual steps, and relate those steps to its environment. How do you maneuver your arm to get a screwdriver into a piece of machinery? What’s the quickest way to clear a path to the spice bottle at the back of the shelf? In what order should you pick up the items on the living room floor?
True autonomy will require planning tasks of greater scale and complexity: cooking a meal, plumbing a bathroom, repairing an engine. Not to mention the need to re-plan in the face of surprises – a stuck bolt, a rotten piece of produce, a child darting into the kitchen.
When current AIs fail at a knowledge work task, it’s often because they weren’t provided with sufficient context. Robots will need context, too: where are supplies kept? How do you like your meals cooked? How much assistance does that nursing home resident need, and is that hitch in their stride normal, or a sign that they’re about to stumble?
Once they have context, robots will need to reason, plan, and exercise judgement and common sense. LLM-based systems like ChatGPT and Claude are making great strides in these areas, but the physical domain brings additional challenges3. The success of LLMs has been greatly assisted by the massive pools of pre-existing data that were available for training – a substantial fraction of all books ever written, the web, and other massive pools of pre-existing data. It will be difficult to match this scale of breadth and depth of data for physical tasks. There’s no straightforward equivalent of “just Efficient learning, generalization, and adaptability / on-the-job learning seem like requirements.
AI agents mostly operate in isolation, and in static environments. We rarely put them in situations where things are changing out from under them, or ask them to coordinate. When we do, things often go haywire. Isolation is easier to arrange in the virtual world, where private workspaces can be created at will, and nothing is too heavy to lift on your own. Robots will often need to cooperate with people, or with one another.
From an article which notes “it took the H1 nearly a full two minutes to very slowly move to a couch, pick up a single item of clothing and put it into the washing machine”.
Today’s general-purpose robots often move much more slowly than human beings. Challenges include strength, control (higher speed means less time to plan and react), and safety (a fast-moving robot will whack you harder and is harder to dodge).
For some applications, slow and steady may be perfectly acceptable: I may not care if my household robot takes all night to tidy up and fold the laundry. But a slow-motion robot won’t be much use as a cook or nursing-home aide. It might get in the way at a warehouse. And it will have a harder time getting enough work done to pay for itself.
That… is just not an impressive amount of weight for a full-grown robot.
Some industrial robots are extremely strong. But humanoid robots – or other highly mobile, “general-purpose” robots – usually aren’t. It’s difficult to combine strength with manageable weight, a large number of joints, and a maneuverable frame. Powerful motors generate more heat and deplete batteries faster – two areas where robots already struggle (see below). And a strong, heavy robot poses greater safety challenges.
ED-209 may have autocannons and a rocket launcher, but it was no match for the staircase
The jury is still out on the appropriate form factor for general-purpose robots, especially with regard to their lower half. Should they have wheels or legs? Two legs, four, or some other number? Wheels are cheaper, more stable, and more reliable; legs are better for stepping over obstacles and climbing stairs. A bipedal frame is more maneuverable, but also more likely to topple if something goes wrong. In any case, the question is: can the robot reliably get around its work environment?
Sadly, yes, that is a robot karate-kicking a child in the stomach (video). Fortunately the kid was OK.
Safety considerations for general-purpose robots are almost limitless. A glitchy or malfunctioning robot could bump into someone, topple onto them, drop something on them, spill something on them, break a glass, or start a fire.
Safety for LLM-based agents relies in part on review of discrete actions, such as attempts to send an email or delete a file. Robots move constantly, and it’s not so easy to single out a few specific motions as the potentially dangerous ones requiring review.
If a self-driving car finds itself in a situation it can’t handle or suffers a glitch, it can pull over or, in the worst case, just hit the brakes. A general-purpose robot that suddenly freezes might leave something on the stove, topple mid-step, or trip the person it was assisting.
And of course danger can be initiated by human action, such as a child darting in front of a robot. I’d much rather my kid be bumped into by a squishy person than a metal robot; and as things stand today, I’d much rather depend on human reflexes and adaptability to avoid tripping over the little rascal.
(The stronger, heavier, and more capable the robot, the greater the risks.)
Today’s bipedal robots can typically run for a few hours before recharging. I suspect this won’t be a limiting factor: if a workaround is needed, we’ll find one, whether that means swapping battery packs, in-floor charging grids, or a cable running to a nearby big-battery-on-wheels4.
I was surprised however to learn that overheating is a serious challenge for continuous operation. A human-sized robot generates about twice as much heat as a person, and robots don’t have the same elegant mechanisms (whole-body circulation and perspiration) for distributing and dissipating that heat.
Then there’s the question of reliability. Robots have large numbers of moving parts, many of which are necessarily finicky, because they’re engineered to push the envelope on size, weight, and performance. As a result, current attempts at general-purpose humanoid robots experience frequent breakdowns. (Contrast the human body, which is constantly recovering from wear and tear, and has substantial ability to self-repair and to compensate for minor breakdowns.)
Somehow Waymo forgot to teach their cars not to drive into flooded intersections?
Waymos struggle to handle edge cases. They’ve recently been observed driving into flooded roads or over burning fireworks. This is despite the fact that self-driving cars have been in development for well over two decades, and Waymo cars in particular have driven over 220 million miles – 250 times as many as a typical American drives in their lifetime.
(I’m willing to cut them some slack on the fireworks thing; 250th anniversaries don’t come along all that often. But I am confused at how Waymo engineering can be so robust as to yield an astonishingly good safety record, and yet so slapdash as to happily drive into deep water.)
For all of the weird edge cases that arise while driving – the classic example being a duck being chased by a broom-wielding woman in a wheelchair – robots operating in homes and businesses will encounter far more. They will be faced with a wider variety of tasks, using a wider variety of equipment (different tools; different robot bodies to grasp those tools), in a wider variety of environments. Achieving reliable operation outside of the controlled environment of a factory floor may be the hardest challenge of all.
Dexterity, coordination, visual understanding, planning, reacting, understanding, cooperating – and doing all of these quickly, safely, and reliably – will take a lot of computing power. Incorporating the necessary computing capacity into the robot itself will add cost, drain batteries, and contribute to overheating. Leaving the robot’s brains in the cloud will slow down reaction times and introduce new failure modes (Wi-Fi outage → dead robot).
For robots to “really happen” will take a lot of robots
LLMs were able to scale rapidly from the moment ChatGPT was launched, because existing hardware (GPUs) and manufacturing facilities (chip fabs) were easily adapted to support the new use case.
Large-scale supply chains for advanced robots don’t exist yet. Even once we have workable designs, it may take years before they can be manufactured, deployed, and maintained at scale. This sort of thing doesn’t happen overnight; it took 14 years for Tesla to advance from first commercial sales to their first million-car year5. One analysis found that once the starting gun is fired (advanced humanoid robots become economically valuable), it might take several years to scale to producing low-millions of robots per year. ChatGPT, by contrast, reached 100 million users6 within two months of launch.
Political, regulatory, organizational, and cultural barriers: concerns over job displacement and safety may limit where and how robots can be used. Many regulations were not written with robots in mind – does a robot count toward minimum staffing requirements? Will businesses leap to adopt robots? Will they have concerns over reliability, security, liability, and maintenance? Will customers want to be served by a robot?
Cybersecurity, surveillance, misuse: an advanced robot could be a criminal’s dream – a dependable henchman that can’t betray its owner7. Preventing this might require continuous monitoring of all robots, which raises all sorts of concerns. And the cybersecurity on robots will need to be airtight.
Privacy concerns: people already have privacy concerns about Roombas. A humanoid household robot would have the capability to see and hear much more.
Cost: once the other hurdles are addressed, I suspect this won’t be much of a limiting factor. A tireless worker at the price of a new car would be a bargain, and humanoid robots will be much smaller and lighter than a car, with fewer moving parts8. It could be that the components, manufacturing techniques, and training processes required for capable robots will make early models much more expensive than a car. But even expensive robots would likely find early use cases – for instance, doing hazardous work.
Many hurdles will need to be cleared before robots become capable, reliable, practical workers outside of carefully controlled factory environments. The list I’ve presented is surely incomplete; and I’ve only briefly glossed over the cognitive side – understanding, planning, acting, and reacting.
Demo videos provide a glimpse into what robots can accomplish under ideal circumstances. They can also serve to distract us from the remaining limitations. For knowledge work, there’s a consistent gap in AI performance between benchmarks and real-world work. In the physical world, I suspect the demo / reality gap will be even larger. Playing around with an LLM to see what it can do has been accessible, cheap, and (mostly!) safe. To assess the capabilities of robots, we’ll be much more reliant on controlled demos and manufacturer’s claims. It will be harder to map the jagged boundary of their capabilities.
As I was putting the final touches on this post, the excellent Understanding AI blog posted Why humanoid robots won’t catch up to human workers any time soon. I haven’t read it yet but I’m sure it’s worth a look.
For a good broad review of robot capabilities, see Epoch’s report from February 2026.
Share
Thanks to Abi Olvera, Avi Parrack, and Taren Stinebrickner-Kauffman.
1
There are four-legged robots whose “feet” are wheels, which can alternate between rolling and walking, and which can climb stairs. But so far there is no sign of this design going mainstream.
2
Workarounds may be found. For instance, some robot designs use cameras to partially compensate for tactile sensitivity, with extra “eyes” in the wrists or elsewhere; a robot could even make use of detached cameras via Wi-Fi. Robots could also incorporate non-human senses, such as ultrasound.
4
One recent demonstration showed a robot sorting packages for 200 hours straight, processing 250,000 packages… using three different robots, taking turns while the others recharged. It was meant as a demonstration of reliability as well as endurance.
6
Measured by monthly active users.
7
Assuming the criminal managed to avoid a paper trail connecting them to the robot.
8
At least, compared to an internal combustion car.
No posts
Instagram Engineering
Shopify Engineering
