ROS Discussion Group, Tonight! June 30 @ 7pm Pacific Time

[camp .]

  It's Tuesday again! Whether you're a beginner or an expert, please join us if you're interested in ROS.  :-]  

Join Zoom Meeting

https://us06web.zoom.us/j/88983478865?pwd=bRdjh1GJpslwHJBANe46VCTlSewyNy.1 

Meeting ID: 889 8347 8865, Passcode: 935511

Thanks,

Camp

[Anthony Malary]

I've been checking this out for a while is it ok if I join starting next week ? 

Best Regards, AM

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/684669321.255369.1782850746829%40mail.yahoo.com.

[camp .]

> I've been checking this out for a while is it ok if I join starting next week ? 

    Sure! We meet every Tuesday. - cp

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/CABzOT0yAHkAX_UzpOERVtxti_kmbFx%3DVUPkALpb6NdGy6%2B83%3DQ%40mail.gmail.com.

[camp .]

Tonight! - cp

[Hitesh VS]

Thank you all for having me yesterday! It was great to see everyone’s current work.

@Scott, I’m really intrigued by your project. I’ve been studying VLAs for a while, and I’m currently putting together a local dev environment for similar models. I'd love to hear your perspective on the pipeline. For anyone else in the group also interested in this space, could you share a bit more on your setup? Specifically:

1. What infrastructure are you using for fine-tuning and inference for large VLA models?

2. Which model are you currently exploring?

3. What is the primary aim of your project?

I’d love to hear your insights, compare notes and I'm sure others would find your approach to these challenges valuable as well!

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/55199072.336050.1782867370060%40mail.yahoo.com.

[Scott Horton]

Hello Hitesh,

I'm just getting started exploring VLAs.  I built the Lerobot so-101 arm and follower.  I've done some initial training and eval to work thru the steps and get a feel for whether the 'juice is worth the squeeze' given the training effort involved (since my purpose is mostly for learning and fun).

Policies

I tried the ACT policy which worked sometimes if you evaluated using the same cases used for training.  I was able to train it on a local computer that has an older Nvidia gpu.

I also tried using smolVLA.  I tried training it on the local computer with 50 training episodes with 10 episodes covering 5 different object positions/orientations.  It took more than a day for the recommended 20k steps, and those steps used very small batch sizes (12) given the limited GPU memory (8GB).   As such, the overall number of training samples was somewhat low.  The results were poor as probably expected.

I tried again yesterday using Colab with a Pro subscription so I could use an A100 GPU.  For that case I used just 25 episodes covering 2 different object positions.  That took 6.5 hours (about $4.50).  While the overall number of training samples was much higher and the training curves looked better, the resulting performance was poor.  I was expecting at least better consistency for the training cases.  I'm guessing more training data is needed.

I have considered trying some of the more advanced models.  Unfortunately the training time/cost goes up, and from what I have read, the execution performance is slow on the Orin.

Inferencing

I'm using a Jetson Orin AGX.

Since I'm currently without a real purpose for the arm, it's somewhat hard to focus on where to spend the training time/dollars.   I'm more inclined to get MoveIt working for it and stick with old-school control.   Since my use case will probably only involve picking-up objects, I have wondered whether a hybrid approach would work better.  That is, use a policy for the grasping phase and use MoveIt for overall control.   (Potentially less training data would be required since the policy scope would be smaller?)  Not sure whether you can partition the problem in that manner.

Also, since I figured I would eventually be using MoveIt, I have modified my arm to include a wrist_yaw joint to give the arm 6 DOF to make it easier for motion planning.  Note sure if that is complicating the training aspects.  I'm fairly confident the necessary changes where made for that change.  If you are interested you can find that change and other notes I made on my efforts here: https://github.com/rshorton/lerobot.git   (depthcamerasupport branch)

Let me know if you have any questions.  I would also find it interesting to hear other folks experience with this stuff.

Scott

[Michael Wimble]

I’m very interested in what you are doing, Scott. I watched a dozen or so YouTube videos of people trying the recommended so101 training and not a single one achieved any useful results, in my opinion. Unless you are just trying to pick small onions up from a small table and move them a small distance where the whole environment has very low variability. 

On Jul 1, 2026, at 11:27 AM, Scott Horton <horton...@gmail.com> wrote:

Hello Hitesh,

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/c65764c4-8534-44f9-9f09-0d38e5ab4809n%40googlegroups.com.

[Scott Horton]

Micheal, I'm with you on your assessment.  Here's a video where I evaluated it for a few minutes:  https://www.youtube.com/watch?v=YYZwJSEdrEw

[Hitesh VS]

Hi Scott,

Thank you for the detailed explanation of your setup and the insights into your experiments with LeRobot! It is interesting to know that you're considering the challenges of decoupling motion control from grasping; that's something worth exploring.

I’ve previously explored setting up fine-tuning and inference within an Isaac Sim environment using the LeRobot EnvHub (https://huggingface.co/docs/lerobot/envhub_leisaac) during my time at university, though I unfortunately never found the time to fully dive into it.

Thanks again for sharing your experience. I will definitely go through your GitHub too.

Best,
Hitesh

On Wed, Jul 1, 2026 at 1:15 PM Scott Horton <horton...@gmail.com> wrote:

Micheal, I'm with you on your assessment.  Here's a video where I evaluated it for a few minutes:  https://www.youtube.com/watch?v=YYZwJSEdrEw

--
You received this message because you are subscribed to the Google Groups "HomeBrew Robotics Club" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hbrobotics+...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/hbrobotics/3890d7bd-144a-4a6a-807c-312f329896cbn%40googlegroups.com.

[Scott Horton]

Hi Hitesh,

Please take my insights with a grain of a salt.  <g>  I'm way early on the learning curve for this stuff.

Thanks for pointing-out the EnvHub.  That looks interesting.

Scott