I made my first ACT policy! And it worked..!
I finally settled on a marker pickup task, primarily because Luciano loves throwing markers into the air and watching them fall. You can imagine how that game ends.
A marker-picker-upper would be a genuinely useful use-case. It would really need a mobile base to do it thoroughly, but baby steps..
I kept trying to come up with something more useful the arm could do while attached to a table, where stray markers don’t tend to collect. Some crafty little task that might actually be valuable. Like stringing beads, but that would need a special gripper. Could train it to put away tools, but I don’t want the first task to involve anything too bulky.
So I kind of gave up on making the first task overtly practical.
Instead, I trained it to pick up 2 markers – a blue & a pink – and place them into a container. I couldn’t even find a proper container and nearly 3D printed one, but ended up simply using a scrap of cardboard instead.
It’s just practice, to see if it works. And it did! Spectacularly..!
I’ve got my little interface now where I press a button and it starts teleoperation with recording running. It definitely took practice, and I also had to vibe-code a “don’t save that episode because I messed up” button, which turned out to be pretty important.
Then, after spending ten minutes trying to get Cursor on my Windows desktop to SSH into the Raspberry Pi and grab the data, I gave up, and humbly used a USB drive.
My desktop has 12 GB of VRAM, which turns out to be plenty to create and act policy. It took less than an hour.
After that, I copied the policy back onto the Pi, got Cursor to vibe-code a run policy button, laid out the markers, and fired it up.
It wasn’t perfect, but it succeeded pretty consistently: about as often as I did when I first tried teleoperating the pickup.