Using Robotpy/wpilib on VMX-pi

[Charlie Peppler]

I've been using the Java based VSCode development test environment successfully with the new WPIlib 2020.

Thank you  Kauai Labs for your work on this.

I know that you have Python examples that speak directly to the HAL, however, I'm curious if the
VMX-pi supports RobotPy/WPIlib.

If you do support RobotPy, what is the recommended development environment for writing, downloading and testing?

For example, do you recommend:

• using a straight text editor to write and test the program directly on the VMXpi, or
• using VSCode (Python support) to write on a Windows machine, then deploy to the VMXpi (I'm guessing not
  since Java and C++ Precompile and assemble packages that are pushed down, while Python is interpreted)

Thanks in advance for any insight.

[Jacob Roth-Ritchie]

Hello,

As far as I am aware, not yet.  I asked over in Chief Delphi, and they said that in order to support robotpy, it would need to be rebuilt.

I want to try to do it, but it is definitely currently over my head.  We are actually currently a python team, so if this is a possibility, it would be an incredible asset.  We jumped all in with the assumption that one of two things would happen...

1) Robotpy/ pyfrc would become compatible with the VMX-pi either through my hands, one of the dev teams', or a combination effort.  I assume Kuai labs has bigger fish to fry.

2) The combination of the VMX-pi and mi summer projects will help the team move to java.

I have begun focussing on option #2.

Here are the dev directions for robotpy.  What they said it would entail is forking the repo and replacing the wpilib hal with the VMX-pi one.

I am not exactly certain how to do most of what is necessary, but many of my summer plans have been canceled, so...

As for the second part to your question, here is what we do with our competition bot.

On our development machine, 

• We install the current vscode and the full wpi suite, 
• Then, we install all of the libraries for robotpy (Including the 3rd party libraries we need)
• Ensure for certain Python is on the path
• Create an environment for robot py (specifically one folder for the current project)
• Create a new vscode project and add the folder.
• Create a robot.py file and begin coding.
• We open a new terminal in vscode, and run it by typing 
  py robot.py deploy
  

You can try it yourself without a robot by installing robotpy and using the simulator.  

I would recommend looking into their example projects first.  They have some excellent physics support, but encoders do not really work well. 

py robot.py sim

~Jacob

[Jacob Roth-Ritchie]

Edit:  I realize, I had a typo.  In the beginning, I wrote...

Hello,

As far as I am aware, not yet.  I asked over in Chief Delphi, and they said that in order to support robotpy, it would need to be rebuilt.

What I meant to say was that robotpy would need to be rebuilt in order to support the VMX-Pi HAL.  Sorry for any confusion.

[Charlie Peppler]

What you say makes sense.  I remember seeing somewhere in RobotPy comments that

they are shifting the way they are supporting the wpilib devices (building C++ wrappers),

so that environment might not be the right way to go in the short term on the VMXpi.

I had another thought.  I have used Python scripts on a straight RasPi before, and

communicated with it through networktables.  If I wanted to add a separate task 

(like communicating to a non wpilib device) using Python, then all that would be 

necessary would be to launch a python script from Java, then talk to it via networktables,

then kill it when the robot deactivates.

Just thinking out loud....make sense?

[Scott Libert]

My intuition, somewhat grounded in experience, is that *since the VMX-pi is binary-compatible with the WPI HAL*, nothing on the VMX-pi side should need to be recompiled, and it's possible that even the python side doesn't need to be rebuilt (assuming that there's a linux raspbian compilation of the python bindings).  The WPI LIbrary HAL dll is the same name as that used in the RoboRIO, so the dll is basically identical, except that it's compiled using the linux raspbian compiler.

VMX-pi's binary compatibility with the WPI-HAL has taken us quite awhile to get working well - and we think it's a key design investment that we've made, with the goal of achieving maximum compatibility.  So surely I'm biased, but finding a way to reuse that investment is the approach I'd recommend.  Further, I think pursuing this approach is better than a loosely-coupled approach using Network Tables (what I'll call the "coprocessor model").  Network Tables use a "blackboard" approach (multiple writers, multiple readers, but no transactions) that could be challenging for "quick turnaround" processing like reading a value from a sensor and then turning around and taking action upon it in a timely manner.  Not impossible, but likely a lot of work to utilize in a general way that also enables rich interoperability, and prone to frustrating latency-related behaviors and really no way to know for sure that the "receiver" actually received what the "sender" sent without inventing a "return receipt" concept, and then an API on top of that which would likely end up looking a lot like the WPI Library API.....

Two other considerations:

- If WPI were to announce support for Python (or C#) with the WPI Library, we might change our position.

- While it's tempting to focus on compilation/linkage issues when considering the porting to a new language, there are a few additional key challenges to solve:  (a) reliable deployment from a remote development computer to the VMX-pi and (b) support for third-party libraries.  Breakthroughs in these areas have the potential to change the equation.

Kauai Labs remains open to supporting whatever efforts others may take to get Python working for the classroom.  While in the near term we're focusing our resources on what we see as "bigger fish" (Artificial Intelligence, autonomous navigation) - if there's anything anyone things we can do to help or support the Python efforts please let us know.  Starting with navX and now with VMX-pi, our philosophy is to embrace collaboration for the benefit of students everywhere.

Hope that helps,

- scott

[Scott Libert]

> using VSCode (Python support) to write on a Windows machine, then deploy to the VMXpi (I'm guessing not
since Java and C++ Precompile and assemble packages that are pushed down, while Python is interpreted)

On this part, it touches on an important design consideration.  RobotPy (I must assume this, it's the only thing that could possibly work) must depend upon the "native" shared libraries (dynamic link libraries with a .so extension) which are deployed to /usr/local/frc/third-party/lib.  Then I assume that there is a Python-to-native binding layer of some sort which allows python code to access those shared libraries.

In that model, some process has to push the native shared libraries down to the raspberry pi (VSCode does that already).  But Python has it's own package management/deployment scheme, and it seems likely this would be used to push down the pure-python packages (and perhaps the python-to-WPI bindings) down to the raspberry pi.

My intuition is that becoming as expert as possible in how this deployment would/should work is likely the fastest way to understanding the ideal architecture, and finding what (if any) dragons like waiting.

[Charlie Peppler]

OK, I tried a quick test and made some progress.  I installed the latest pynetworktables onto the VMXpi,
and wrote a short script that connected to the NetworkTables instance, using the fixed IP

address.

The problem I'm wrestling with now is....I start (enable) the java program on the
robot.  Whenever I run the python script (that connects successfully to the NetworkTables instance), 

it forces the Java program to disable. So, they're definitely connected somehow...just not
sure how to connect them _together_ yet.

This could be useful if we need to tie in some type of sensor that does not have a wpilib
connection.

[Charlie Peppler]

On Wednesday, May 27, 2020 at 4:56:44 PM UTC-4, Scott Libert wrote:

[...] finding a way to reuse that investment is the approach I'd recommend. 

I can't agree more.  The problem I am considering (more below) is the proper way to tie in devices (particularly sensors) that are not already a part of the WPIlib world.
This may not be as critical for FIRST only training (since the ones that are available should meet the needs of the FIRST challenges), but as you note below, may not be
sufficient for further areas of study.
 

Further, I think pursuing this approach is better than a loosely-coupled approach using Network Tables (what I'll call the "coprocessor model").  Network Tables use a "blackboard" approach (multiple writers, multiple readers, but no transactions) that could be challenging for "quick turnaround" processing like reading a value from a sensor and then turning around and taking action upon it in a timely manner. 

Complete agreement.  The problem I have is, how does one tie in new devices that are not already in the WPIlib stable?  There seem to be plenty available for actuators (motors, pneumatics etc.).
but in a larger environment, there will be a need for more sensing capability (RFID as one example, precise low cost distance measurement another, IoT devices a third).
 

Not impossible, but likely a lot of work to utilize in a general way that also enables rich interoperability, and prone to frustrating latency-related behaviors and really no way to know for sure that the "receiver" actually received what the "sender" sent without inventing a "return receipt" concept, and then an API on top of that which would likely end up looking a lot like the WPI Library API.....

Completely agreed.  Having built device drivers before, from an architectural perspective, loosely coupled does not belong in the sensor level feedback and control loop.
If the robot is to communicate with higher level systems  (UIs, manufacturing applications, or networks of robots working together), loosely coupled comms have their place (MQTT, etc)

Two other considerations:

- If WPI were to announce support for Python (or C#) with the WPI Library, we might change our position.

- While it's tempting to focus on compilation/linkage issues when considering the porting to a new language, there are a few additional key challenges to solve:  (a) reliable deployment from a remote development computer to the VMX-pi and (b) support for third-party libraries.  Breakthroughs in these areas have the potential to change the equation.

Kauai Labs remains open to supporting whatever efforts others may take to get Python working for the classroom. 

 

While in the near term we're focusing our resources on what we see as "bigger fish" (Artificial Intelligence, autonomous navigation)

Ironically, I was thinking about autonomous navigation when the need for new sensors came up, and thinking about how to integrate a new
one.  For example, a low cost laser distance sensor such as this or this would be critical for autonomous navigation.

I guess from a marketing perspective, Kauai Labs/Studica are caught between making FIRST level robotics easy for teaching entry level
robotics, and building a more advance platform for research applications.

Has Kauai Labs/Studica ever reached out to Carnegie Mellon?  I was curious because I saw that Jacobs school had engaged
CMUs program for Lego based training for the robotics classes at their school.  That would be a great place for you guys to get
connected, and they would completely understand what you are trying to do.

At the end of the day...what is best practice for integrating a new sensor into the WPIlib world?
 

- if there's anything anyone things we can do to help or support the Python efforts please let us know.  Starting with navX and now with VMX-pi, our philosophy is to embrace collaboration for the benefit of students everywhere.

Amen for collaboration.  You guys are doing a great job, and would love to help out. 

Hope that helps,

Extremely helpful, thank you for your time and thoughts.
 

[Charlie Peppler]

With the exception of the scripts themselves, Python "environments" (including libraries
and bindings) are tightly coupled to the platform on which they run (pip etc).

I used a very simple approach on the VMX-pi.  Instead of "deploying", I simply setup a 

mounted drive (using Samba) on the development computer using the fixed IP address 

of the VMX-pi.  Then I simply opened the directory with VSCode where the Python files 

reside, as though it was on the dev computer.

To execute, I used SSH to run the script from the command line.  Pretty easy.

It's the beauty of Linux and Samba!

[Scott Libert]

> Whenever I run the python script (that connects successfully to the NetworkTables instance), 

it forces the Java program to disable.

Any info in the logs that might suggest what's underlying this?  We know that the network tables connection from a remote IP address works (that's what the smart dashboard or shuffleboard on the driver station do).  Without looking into it, is it possible that Network Tables requires that the client be a different IP address than the server?

[Scott Libert]

> Whenever I run the python script (that connects successfully to the NetworkTables instance), 

it forces the Java program to disable.

Well, that's a brilliant question, and I'd agree that NetworkTables is a great answer for low-bandwidth sensors.

It's not as great an answer for high-bandwidth sensors like the Intel RealSense.  For those types of sensors, one approach is a more general file-descriptor-based approach.  With this approach (assuming the device is running on the same Raspberry Pi as the VMX-pi WPI Library code) there could be a generic WPI Library class that reads from a file descriptor, and waits on file descriptor changes (some notification mechanism needs to be defined).  At an implementation level, the file descriptor could be implemented as shared memory.

WIth that approach, there would then need to be device-specific file descriptor parsing classes (subclassed from the file descritor-based device class).  The other downside would be that to support both C++ and Java with a single source code base, a JNI interface (not fun code, but doable because examples exist) could be developed.

But perhaps that's worth exploring?

[Scott Libert]

> Has Kauai Labs/Studica ever reached out to Carnegie Mellon?  

Actually, we visited the NREC at CMU, including meeting w/the Robotics Academy and also with their graduate students, that was a little over two years ago.  Nothing solid ever came out of it, but perhaps timing wasn't right.

I'd be happy to reach out again, if folks think there's a good fit please share your thinking and we'll see what we can do.

More generally, VMX-pi is definitely heading towards college-level usage because it's being used starting this year with the college-level WorldSkills mobile robotics tournament.

One of the avenues as well for research we've been looking into is supporting ROS, however it's yet another mountain to climb we haven't focused our resources on yet.

Given our heavy investment in the WPI Library, I think we want to maximize what can be done in this environment - so this concept of defining an architecture to enable it is very interesting and compelling.

On Wednesday, May 27, 2020 at 2:54:51 PM UTC-10, Charlie Peppler wrote:

[Jacob Roth-Ritchie]

Our school year is beginning its accelerated (yet also somehow drawn-out) race to the end so I am a bit more preoccupied with that these days.  Sorry I have been out of the loop.

This discussion is wonderful, and though you are both significantly more well-versed than me in these matters, it gives me hope that the python library may be doable.  A few observations.

As for robot py, it can indeed be programmed right on the Rio itself.  We have done this at events before.  What robot.py deploy seems to do is grab all the dependencies and send them to the Rio.  I do no think more is involved.  I think the big compile pieces happen at install (for both robot py and the vendor libraries) and at run time. However, I may be mistaken.  I seem to remember a PR on building Robotyp for raspbian, so that could be a sticking point.

As for non-frc devices.  Wpilib seems to have generic communication protocol for all of their IO (Analog, DIO, PWM, I2C, SPI, UART, etc).  Yet, they also seem to not always be consistent.  If you peruse Chief Delphi, you can see many people having difficulty with many of the functions for I2C and SPI specifically. The other option is of course to use network tables.

As for robotpy and third party, that is always the rub.  In theory, the fact that both robotpy would be similarly implemented on the VMX as the Rio and that (c++)VendorLibs are similar as well, it should work with few modifications.  Yet, things may not be so simple.  

I did get the PixyCam2 working with Java on the VMX-Pi both through network tables (by using a second raspberry pi), and directly through I2C (many thanks to the Kauai team for routing out and quashing that bug) using the PseudoResonance library.  I am incredibly excited about this because it means I can work on learning targeting (one of the many things we never get to every season) without using the powercell (which my dog loves) or our limelight (which I do not want to bring home for fear of damage).

I am not sure if I added much but just wanted to jump in and say I am excited about this discussion and with the prospects that lie herein.  When I get some time, I will attempt to tackle robotpy and see how quickly things go awry.

Thank you again for your vision and discourse.

~Jacob