LATENT

Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data

Zhikai Zhang1,3*, Haofei Lu1,3*, Yunrui Lian1,3*, Ziqing Chen1,3, Yun Liu1,3, Chenghuai Lin3, Han Xue1,3, Zicheng Zeng3, Zekun Qi1,3, Shaolin Zheng3, Qing Luan3, Jingbo Wang5,
Junliang Xing1, He Wang2,3, Li Yi1,4†

* Equal contribution † Corresponding author

1Tsinghua University, 2Peking University, 3Galbot, 4Shanghai Qi Zhi Institute, 5Shanghai AI Laboratory

Abstract

Human athletes demonstrate versatile and highly-dynamic tennis skills to successfully conduct competitive rallies with a high-speed tennis ball. However, reproducing such behaviors on humanoid robots is difficult, partially due to the lack of perfect humanoid action data or human kinematic motion data in tennis scenarios as reference. In this work, we propose LATENT, a system that Learns Athletic humanoid TEnnis skills from imperfect human motioN daTa. The imperfect human motion data consist only of motion fragments that capture the primitive skills used when playing tennis rather than precise and complete human-tennis motion sequences from real-world tennis matches, thereby significantly reducing the difficulty of data collection. Our key insight is that, despite being imperfect, such quasi-realistic data still provide priors about human primitive skills in tennis scenarios. With further correction and composition, we learn a humanoid policy that can consistently strike incoming balls under a wide range of conditions and return them to target locations, while preserving natural motion styles. We also propose a series of designs for robust sim-to-real transfer and deploy our policy on the Unitree G1 humanoid robot. Our method achieves surprising results in the real world and can stably sustain multi-shot rallies with human players.

Multi-shot Rallies

Rally 1

Rally 2

Rally 3

Rally 4

Reactive footwork

Footwork 1

Footwork 2

Footwork 3

Footwork 4

Try to beat different human players

Human Player 1

Human Player 2

Human Player 3

Self play (simulation)

Human Player 3