Leveraging foundation models in Stringman
Nathaniel Nifong
New features in the Stringman control panel make it possible to run dataset recording, policy evaluation, and model training jobs in the cloud.
This means that even the most advanced features of a Stringman robot can now be used with nothing more than an 8 core laptop.
I’ve been recording datasets in my own home picking up laundry and toys and fine-tuning a model called Pi 0.5 from Physical Intelligence ( a research company at the forefront of robotic intelligence with one of the best foundation models available) I even got a callout from the founder!
I think it is within reach this summer to get Pi 0.5 to run reliably enough on stringman that we can ask it to do arbitrary tasks with a prompt like “Arrange all the toys in color order on the floor”
I look forward to posting any intriguing behavior that comes out of this
But of course the intended interface for Stringman is that of an appliance. A thing you turn on that does a job and turns itself back off and that remains my core focus. Clearing floors with minimal human interaction and staying out of the way.
Collecting toys with stringman
Operations
Neufangled Robotics will be hiring Anirudh Bhaskar, an EE student from NC State for a summer internship. He will design a final revision of the PCBs that go in Stringman's anchors and gripper as well as helping with data collection and testing.
I'm focused mainly on behavior now that the Arpeggio hardware is pretty stable and reliable. It's still 3D printed for the foreseeable future, but we will cross the bridge to injection molding only if an when sales justify it.
Come to the 79°West Coworking Space & Innovation Hub Wednesday May 6th at 11:00 to 1:00 for a Stringman demo and lunch!
I will be showing the newest hardware as well as giving a talk on Imitation learning with 3D printed robots.
Add the event to your calendar