Thread by @sterlingcrispin

My #bostondynamics #spotmini is almost ready for it's machine learning boot camp , on the left… instagram.com/p/BhIj8waADof/

spent part of the day baking textures + aggregating meshes + setting up the rigid bodies & colliders in Unity so this thing will be able to walk the walk

the physics simulation is going a little haywire and throwing my robots around like an angry God but its sort of beautiful

yeah uhh, I guess that's one way of moving forward but walking would be nice

YAAAAS MY PRETTIES YAAAAASSS
(at least their sort of walking)

a few hours later, they're being rewarded for moving to the left of the screen (X axis) I had reduced the punishment for moving in the Z+Y axis I think they need stricter guidelines so they stop crazy-spider-crawling

Spotmini is now training to go #supersayian in the hyperbolic time chamber

@punchesbears @awjuliani I know there's a whole copyright infringement thing to contend with but it would be awesome to merge this into the main repo

lots of physics settings changed and a different reward function trained to about 100k steps

the steep red line is this last video with the green/pink earlier training, you can see its successful much quicker, if I let it train as long it would probably settle down and figure it out

this last video has a "tap dancing penalty" where its rewarded less the more frequently it starts/stops touching the ground, and a "touch the damn ground reward" that increases the velocity reward by a percentage per foot that's touching the ground

I thought I was making progress with these agents following a moving target but these JERKS just found a flaw in the physics simulation and they're using it to glide across the floor like total cheaters

I scaled up all the objects, masses, torque by 10x to squash the weird bug earlier they were exploiting but now I think they need a bit more torque? Or maybe adjusting the beta hyperparmeter so they explore more random motions earlier on.

#UnityMLAgents #MLAgents #madewithunity

Manthese little jerks have ALMOST got it figured out, you can see their front legs reaching for a proper step with such delicate hesitation

TOO funny! Im going to retrain them on a randomly bumpy terrain that gets more difficult the further from the center of the world, but none of these agents have ever seen anything but a perfectly flat floor so their perception+actions have no concept of it and theyre freaking out

I think if this agent could talk it would be whispering "please end me" , maybe it needs more torque in its legs ? It looks a little too weak to move the weight of its legs

#UnityML #UnityMLAgents #MachineLearning #SpotMini #bostondynamics

adding more torque was not the answer, this was identically trained as the last one with 50% more torque available to it and its even lower to the ground and more pathetic, maybe I need to retrain with 50% LESS torque so it learns to be more efficient?
#UnityML #UnityMLAgents

Orange=150% torque, I was excited at first around 200k-500k steps it was out performing the other settings great, but then did worse and worse, @awjuliani @punchesbears is it true the Beta hyperParameter has no effect on continuous actions? Shouldn't the entropy be decreasing?

trying out a new reward function that I'm hoping will let me "drive" SpotMini:

scale how much the Agent is moving in the goal direction * how much its facing the goal direction * how fast its moving in any direction

then penalize that value by how much torque is used

yeah you know what great, I wanted you to cross your back legs and do some weirdo pushups for hours

I think I need to replace the magnitude of the velocity in the reward by total distance traveled or something like this, because technically it's velocity is alright and it might be thrusting in the right direction

Not Bad! The back legs are acting up a little so I added a punishment for the lower legs touching and I'm retraining now but its getting there!

#UnityML #UnityMLAgents #MachineLearning #ReinforcementLearning #MadeWIthUnity

newly reworked reward function! It's heading toward the randomly moving goal pretty well so long as the goal isn't behind it or directly sideways! I'm going to reward it for keeping it's body more stable and retrain it tonight

#UnityMLAgents #UnityML #bostondynamics #spotmini

my #SpotMini robot is finally starting to walk more and more like the real robot using just reinforcement learning! Tweaking the reward was the key. I've still got a long way to go but there's been tons of progress

#UnityMLAgents #UnityML #bostondynamics #ReinforcementLearning

Rough terrain performance on the same neural network as shown above

This kin of omni-directional locomotion is going to be key for my long term goals for this agent

So excited to start training with this robot arm in place and eventually solve complex pick + place tasks for assembling sculpture and architecture

Redoing weight of bodies in sim., making Spot's body heavier, legs lighter, based on @BostonDynamics reported 25KG for Spot Mini and the volumes calc'ed in @netfabb. I think the weight + proportions are close, but I'm bothered the density is off, comments?docs.google.com/spreadsheets/d…

Using the mesh I had for the body was under estimating (pic right), and a convex hull is an overestimate, so I added this white form on the inside to approx batteries + other unknowns. I'm at 0.68g/cm^3 total seems low wouldn't it float in water? SpotMini must weigh > than 25KG?

If you're conservative and drop half the body weight then assume the robot weighs something between steel and carbon fiber you'd end up at 66KG/145LB I think the reported 25KG for #spotmini must be aspirational and not operational. I could be wrong but I dont think I'm 2x off

Check out the back two legs alternating between being used for walking and being used to keep the rest of the body in balance, its only half way done training so it may resolve this into a nice gait (the choppiness and low resolution is because this is a capture during training)

here's the result of that last training session, I'm going to retrain and adjust the weights of each reward so its more motivated to keep its body off the ground and aligned to the floor, I also added a "jerk term" to reduce jerkiness check the next tweet

the "jerk term" is the result of looking at all the agents limbs linear and angular velocities, normalizing them, getting the difference of that over time, summing those differences, and squaring it, then punishing the agent according to that measurement, this is the result

@punchesbears @awjuliani @erwincoumans have you built in jerk minimizing rewards into anything you've tried? I noticed a large improvement and it seems like a heavily researched approach toward motion planning and theories of how animals move

Progress, trained here with a fairly high beta parameter causing the agent to explore the space of possible movements and learn a decent gallop although the beta is so high that its not a stable solution

overclocking my CPU up to 4ghz base speed 4.6-4.8ghz boost and RAM up to 2933Mhz and I'm training 8 simultaneous agents 🙏 thoughts and prayers to my CPU temp

This network had a height punishment factor 10x greater than past networks and it looks like it improved the behavior so I'm going to retrain with higher weights on the height punishment and see if I can get it to walk fully upright

My PC had an electrical failure two weeks after I tweeted this and has been in disrepair since, the 1600W power supply had a meltdown of some kind and was arcing in the case. Thankfully it doesn't seem to have taken anything else with it. Hoping it doesn't happen again 🙏🤐👼⚡️