New DeepLabCut models not compatible with Bonsai

[Sebastian Wittekindt]

Hello,

I trained a new model with the 2.2 release of DeepLabCut, but when I attempt to run inference in Bonsai using the DetectPose node I get an error telling me that my interpreting binary is incompatible. Some googling shows that this has to do with two different versions of TensorFlow being used. Bonsai seems to use 1.15 whereas DeepLabCut is using TensorFlow 2.0+.

Any idea how to fix this or whether there are plans to update Bonsai’s node to work with newer versions of TensorFlow?

Thank you!!

-Sebastian

[Gonçalo Lopes]

Hi Sebastian,

Bonsai DeepLabCut package uses the TensorFlowSharp library, a bridge to interface natively C# and TensorFlow:

NuGet Gallery | TensorFlowSharp 1.15.1

Unfortunately, this library seems to be stuck at 1.15.1 for a while now. To support TensorFlow 2.0+ would require a move to a different library, for example TensorFlow.NET:

NuGet Gallery | TensorFlow.NET 0.60.2

TensorFlow.NET only seems to support TensorFlow 2.0+ so it would have to be one or the other. It is on our list of dependencies to review, but unfortunately not as easy to resolve as one would hope.

--
You received this message because you are subscribed to the Google Groups "Bonsai Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bonsai-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/8093ecfd-f784-476b-9b6b-673640e54419n%40googlegroups.com.

[Sebastian Wittekindt]

Hi Goncalo,

Thanks for the fast reply! TensorFlow 2.0+ seems to be compatible with the latest DeepLabCut release so would TensorFlow.NET would be a good solution for the DLC integration?

Please do let me know if you think this is something that could realistically be fixed in the next couple of weeks.

Otherwise do you know which version of DeepLabCut is compatible with the Bonsai node as they currently stand? I could try to roll back DeepLabCut to that version.

Cheers,

-Sebastian

[Gonçalo Lopes]

For the short term, I believe it is best to rely on Tensorflow 1.15.1.

There is a short DLC guide focused on Bonsai compatibility here:

GitHub - bonsai-rx/deeplabcut: A Bonsai interface for real-time markerless pose estimation using DeepLabCut

It should be fine as long as your GPU is compatible.

Hope this helps!

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/6a2aaaa1-e315-4aa0-8262-3176c8fed8bdn%40googlegroups.com.

[Sebastian Wittekindt]

Hi Goncalo,

Any chance of getting an update for the DLC node sometime soon?

Cheers,

-Sebastian

[Gonçalo Lopes]

Hi Sebastian,

This needs new bindings for Tensorflow 2.x. We have a tool that can generate them quickly, but I need time to generate it, clean up the code, publish and only then can test and upgrade. Because of this I was thinking it will probably happen together with the first preview of Bonsai running in .NET 5 by mid next year.

My understanding was that it was possible to train DLC with older versions of Tensorflow and then just run those trained models against the binary, again, as long as GPU compatibility holds. Do you have a specific need that would require Tensorflow 2.x?

To view this discussion on the web visit https://groups.google.com/d/msgid/bonsai-users/f753cd76-ba08-4dca-905f-4c3b0156694cn%40googlegroups.com.