Dance Skill Feat. Nick Kosir

TLDR: We built a custom dance for world famous weatherman, Nick Kosir.

The Vision

Portia on our team saw one of Nick Kosir's legendary dances. She reached out to say hi.

Adam loves learning new moves, and Nick graciously offered to be Adam's dance partner!

Easy + fun. Exactly what Adam needs for his first time dancing.

Teaching a robot how to dance is relatively simple if you have all the right raw materials. Here's how we did it.

Rohan took the dance video that Nick shared and used a video analysis tool into joint motions. Then we used Mujoco to convert the motion into join and root positions that correspond to Adam's body in a CSV format. Then converted the CSV into npz format which is compatible with Nvidia's Isaac Lab. Then used Isaac Lab with a reward function that considers a few variables.

Baby steps. Something like 150MM+ iterations…the equivalent of ½ the people in the U.S. learning how to dance like Nick.

Sometimes we keep Adam on a gantry - easier to dev this way.

We encountered some wobbliness - we then used a Booster training pipeline (thanks Booster) instead of using our custom pipeline. Much better.

Then we amped up our GPUs to train faster.

We ran a bunch of concurrent training runs on RunPod until the dance looked sexy…

If you'd like to meet Adam or collaborate, feel free to reach out. We usually check DMs on Sundays.