Original text: Training a Humanoid Robot for Hard Work

Authors: Alberto Rodriguez, Director of Robotics Behavior at Atlas; Shane Rozen-Levy, Research Engineer; and Vinay Kamidi

Compiled by: Felix, PANews

This humanoid robot is unlike any you've ever seen before. Some things are obvious in the latest video: the Atlas robot rotates its torso 180 degrees, crouches down, lifts a mini-fridge, and delivers it to an engineer taking a break. But there are also less obvious details, such as how the robot makes full use of its arms, legs, and torso to perform lifting tasks that are difficult for humans, and details that are completely impossible to show in the video, such as the speed at which the robot develops and the precision of its actions.

This is certainly refreshing, but why do it?

 Atlas Mini Fridge

Boston Dynamics' other robots are designed to automate the heaviest tasks. The Stretch robot can autonomously unload boxes weighing up to 23 kilograms from trucks in extremely hot environments. The Spot robot performs the same measurements along the same inspection route at exactly the same time every day, detecting signs of problems on the factory floor early. These jobs, though monotonous, require a high degree of attention to detail, and Stretch and Spot provide this kind of service every day.

Atlas aims to provide a wide range of functionalities for scenarios requiring extreme strength, endurance, and dexterity, such as factories, warehouses, and construction sites. Boston Dynamics is working to develop Atlas into a general-purpose tool for manual labor. Achieving the performance and reliability required for real-world environments necessitates significant advancements in both hardware and behavioral control.

The following is a series of meticulously designed experiments showcasing significant advancements in both hardware and behavior. Within just weeks of Atlas's public debut in January, it demonstrated the humanoid robot's performance in strength, dexterity, and full-body control.

Physical intelligence for the real world

Over the past few years, the market has witnessed a fundamental shift in behavioral architectures, driven by demonstration data and exhibiting increasingly enhanced generalization capabilities. These are key elements in realizing the promise of humanoid robots: adaptability, rapid learning, and ease of task reassignment. These architectures can drive not only the behavior of desktop robotic arms but also fully functional humanoid robots performing real-world tasks.

While the most advanced mainstream methods can produce excellent behavior, they also have some limitations: they rely too heavily on continuous camera feedback , not only for understanding the world but also for guiding control loops; their interaction with the environment is limited to a very limited number of robot surfaces, usually fingers, and often only fingertips; and they are almost entirely focused on lightweight tasks.

Real work, especially heavy physical labor, requires a broader definition of "physical intelligence." When moving objects, teams utilize any part of their body to bear the weight and use touch to adapt to the object's shape, mass, and stiffness.

You can't lift a refrigerator just by looking at it and using your hands. You have to be prepared, anticipate its weight, lean forward to adjust your body to its shape and weight, and judge whether you can lift it. The real work happens during interaction. A humanoid robot should be able to grip a box with its forearms and biceps, lift heavy objects from the ground to its thighs with its knees, and carry long, heavy objects on its shoulders, just as easily as it can lift a refrigerator.

Atlas uses reinforcement learning (RL) to learn how to lift a refrigerator, practicing a massive number of refrigerator-lifting exercises in a simulated environment. The most challenging part isn't seeing a refrigerator or knowing how to lift it, but learning to adapt to refrigerators of any shape Atlas might encounter in the real world. This is a problem combining control and perception, where perception is implicitly accomplished through the body's proprioception. The strategies driving these behaviors have learned to adapt to various changes, such as the refrigerator's position, mass, ground friction and grip, or its placement among the torso, arms, and hands. This level of adaptation is one of the most fundamental building blocks of physical intelligence.

Robots carrying heavy loads

The hardware showcased today is also unique. This generation of Atlas robots is designed not only to meet the flexibility and strength required for practical work, but also to combine the simplicity and reliability needed for mass production. While the humanoid robot's shape has its advantages, strategic breakthroughs can significantly improve its performance and efficiency.

Here are some highlights that may not be immediately obvious:

Minimalist actuators: Only two types of actuators are used on the robot's body. This allows the focus to be placed on manufacturing more efficient and powerful actuators on a larger scale, ultimately reducing costs. All of these are rotary actuators, which are easier to accurately represent in simulations, crucial for the high-performance reinforcement learning using proprioceptive feedback mentioned earlier.
Highly repetitive components: The same sub-components are reused as much as possible on the body. The two legs and two arms are identical. The shoulder-to-shoulder and pelvis-to-pelvis structures are also exactly the same.
Infinite Rotation Joints: These actuators can rotate indefinitely. This is achieved by eliminating all cables between the joints, thus eliminating a major factor leading to actuator hardware failure. In turn, this reduces costs for Atlas customers and gives Atlas a uniquely efficient mode of movement.
Symmetrical feet: Because Atlas has equally excellent forward and backward movement, its feet are symmetrical.
Easy to maintain: The arms, legs, hands, and head are all field-replaceable units that can be replaced in just a few minutes.

The mobile mini-fridge demonstrates strength, full-body coordination, and the use of proprioceptive feedback. This has become a benchmark for industrial work: moving heavy objects in manufacturing environments that typically require two people to work together.

However, some less practical tasks are also meaningful. For example, a 90-kilogram robot can perform handstands and backflips because it has an excellent thermal management system, meaning Atlas can work in hot environments. Moreover, these behaviors can train other transferable skills: such as how to move nimbly and with balance, how to move fully in confined environments, and how to recover from slips and falls.

Training process

As a product and research platform, one of Atlas's goals is to train and deploy new behaviors within a day. While this demonstration didn't reach that speed, Atlas's ability to reliably move the refrigerator far exceeded expectations.

Here are the methods for training robots:

Reference trajectory: To train new behavior, a reference trajectory is used, which is data that tells the policy what to do. This can be a remote operation demonstration, an animated trajectory, or a description of a more abstract goal. For the refrigerator moving task, a simple animation was used first to fully utilize Atlas's superhuman range of motion.
Incentives: Next, set a goal to guide the robot to complete the task by following the animation trajectory as closely as possible. Reinforce the desired behavior (keeping the weight in the Atlas's grippers and maintaining the same position and orientation) by establishing a reward mechanism, while also applying push-pull interference to the robot and the refrigerator so that they can remain focused on the main task even when disturbed.
Simulation: Atlas ran simulation programs in parallel on a graphics processing unit (GPU), performing millions of hours of motion practice. Through extensive simulation experience, Atlas learned to adjust its behavior according to various changes in the refrigerator.
Real Robot : After the simulation results were satisfactory, hardware testing was conducted. Simulation can only help to a certain extent, while hardware testing is the fundamental way to continuously improve.
Iteration : Once real data on the performance of the strategy on a real robot is obtained, the training process can be returned to make adjustments and strengthen the behavior.

Narrowing the gap between simulation and reality

One of the most significant improvements in the enterprise version of Atlas is the high fidelity of its simulation environment . Atlas's simulations are very close to reality; they can be easily trained, tested, and iterated upon rapidly. Generally speaking, if a behavior looks good in simulation, it will also perform well on the robot.

The simulation-to-reality gap refers to the difference between the performance of a strategy in a simulation environment and its performance on real hardware. The assumptions and mathematical simplifications in simulations cannot capture the complexities of the real world. Subtle variations and variables, such as friction, latency, or sensor noise, accumulate and cause malfunctions in the physical world.

While it may never be possible to completely eliminate this gap, we are very close. The Atlas team has established a rigorous pipeline and system support for testing and development. A policy trained today can be tested on a robot tomorrow with the mature policy, and data collected to drive the next iteration and the development of new behaviors.

What makes the difference between simulation and reality so small?

High-fidelity hardware: Unlike previous platforms, this platform uses only two powerful and efficient actuators, which are completely symmetrical. This simple design and structure, along with the efficiency of the actuators, means that the robot can be modeled with extremely high accuracy in simulations. Because the robot model closely resembles the actual hardware, there are fewer fidelity issues when deploying trained strategies. Simulation results are completely consistent with actual results.

Domain randomization: To make the strategy more robust, the robot was not trained in an ideal environment. Domain randomization was used to fine-tune parameters such as the refrigerator's weight, floor friction, or motor power throughout the training process. These small random variations during training make the final behavior more resilient to real-world variables. For example, the strategy for moving the refrigerator was initially trained for loads of 50-70 pounds, but the robot successfully moved a refrigerator full of items weighing over 100 pounds. The team also did not test under perfect conditions. They placed a variety of items from the lab into the refrigerator; the weights were inconsistent, the distribution uneven, and items shifted within the refrigerator during movement. With a well-developed strategy, all these interfering factors can be eliminated by Atlas, rather than being handled by engineers.

People and Processes: Finally, the processes and operations are designed to streamline training, testing, and experimentation. The team has established a rigorous set of processes, with a large number of people working behind the scenes. The team collaborates closely with numerous teams responsible for the actual operation of the robots, including the hardware design team, maintenance technicians, and robot captains. The entire organization works together to make Atlas as reliable and efficient as possible, while continuously pushing the limits of new functionalities.

UBTECH: Pre-orders for the first batch of the world's most advanced bionic humanoid robot have exceeded 2,110 units.

Popular Articles