WPIlib, KauaiLabs/Studica robot and Websockets

[Charlie Peppler]

I am doing a research project that involves linking real time data coming from
the Kauai Labs NavX and a "Digital Twin" in Unreal Engine.

I have been looking for an interprocess communications mechanism
to speak between the two worlds with the least amount of development
and might have a decent lifecycle if I can get the two to work together.

My first thought was MQTT, which I've already tried on the VMXpi/Java
side, but the support is weak on the UE side.

I found another option using Websockets with JSON, but figured I would post
here to see if anyone knows of a wpilib/Java friendly interface for wpilib.

The simplest approach would be to open a socket between the two, and
send one way messages for now.

I'm definitely not there yet, but I have found someone in the wpilib world
who has been using Websockets and what looks like a solid API.

Here

https://github.com/wpilibsuite/allwpilib/tree/v2021.1.1/simulation/halsim_ws_core

and here:
https://github.com/wpilibsuite/allwpilib/blob/v2021.1.1/simulation/halsim_ws_core/doc/hardware_ws_api.md

If anyone has guidance as to how I can implement a simple instance of this
connection, I would appreciate it.  (Wary because I see the 2021 on the 
wpilib version, and know it's not time to roll forward yet :-)

Best,

Charlie

[jrothr...@hdsd.org]

Hello, I hope all is well.

I think there is a chance that the websockets will require v2021, but I do not know for certain. If you want to see an example of how they work, you can take a look at the Romi. In addition, the websockets interface tends to be a simulator interface not with a full robot (at least as far as I understand). One of the things that makes the VMX-Pi better than the Romi is that it does not rely upon the simulator for implementation. The way Wpilib usually handles this is through Network Tables. If I were you, that is where I would start (though there is a chance NT may not be fast enough).

If you want to see some examples of the Network tables implementation, you can look at the custom web dashboard and the PikitLib. The latter is a similar project to the VMXpi except it is much more limited in scope. It also uses Python rather than Java or CPP.

[Charlie Peppler]

Allow me to clarify.

I am familiar with NT, and did see the Romi.  Unfortunately, 
the game engine tool I'm using (Unreal Engine - UE4) does
not have an interface for NetworkTables.  I am not trying
to run a robot simulation, just send a message to a UE4 project
that has a WebSockets connector. 

(much like you would be able to do with MQTT).

I am trying to find a way to use a Java library from a
wpilib robot program that will allow me to connect with 

websocket server, and send a message to the UE4 client.

[jrothr...@hdsd.org]

Ah, that makes sense. I still think in your position, I would use network tables. I feel, I would write an interface that moves the data from the network table instance to the UE instance. I think most of the websocket data (at least with the Romi) is passed using a .json object. 
Here is the documentation for the websockets. I do think it is reliant on both the simulator and v2021 of WPILib. The pikitlib library Created its own Puthon driver station. If you look at that, it may be trivial to grab the network table entries, package them as a .json and move them to UE. However, I would have difficulty with this as much of it is beyond my current programming skills. The other suggestion is take a look at Chief Delphi for other teams that have used the Unreal Engine in projects before. Most of them are for the simulator, but it may help to see how they implemented it. Here is one such example. 

[jrothr...@hdsd.org]

One other thought. You could set up a separate websocket on the Pi which reads either the network table instance or live data, and create the interface there. That way, you would avoid installing a bunch of dependencies on the host machine. The WPILibPi image has an interface for this. If you started with that image, then installed the VMX-Pi software over it, you may get the best of both worlds. With that, you get the added benefit of writing a simple Java program and uploading it like you would a cameraserver. I am currently doing this to get Photon Vision working on the Romi. It works well.