Interoperability working group: PyTorch, TensorFlow, CLI and Python API

[Samuel Marks]

PyTorch, TensorFlow, and tigers (Oh my!)

Looking to start a working group for interoperability. Expected deliverables:

• New libraries and PRs to major repositories for cross-ML-framework interoperability
  
• Learning modules, including FAQs around moving from one framework to another
• Best-practice guides and example repos for sharing models & research (i.e., os.path.join, setuptools, &etc.)
• TensorFlow & TensorFlow ecosystem build guides for more platforms and Linux distributions than is officially supported
  

To elaborate on the first two bullet points:

Of the many popular machine-learning frameworks out there, PyTorch and TensorFlow are leading the fray. It would be great to enumerate the differences for beginners moving from one (or the other) framework.

Once that's done, it would be pertinent to build tools and libraries to combine experiments—models &etc.—from multiple machine-learning frameworks.

There is already work here, e.g.,

• Keras (TensorFlow, CNTK, Theano);
• MMdnn (Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML);
• ONNX (many)
  

…but I think there is something majorly missing, even when you just look at sharing between PyTorch and TensorFlow.

Proposed solutions to literally sharing code—without conversion—between TensorFlow and PyTorch:

• Build a new Keras backend for PyTorch
• Build a new ontology abstraction that people should use instead of Keras for PyTorch and TensorFlow
• Work at the IR level, ensuring, e.g., that both frameworks use MLIR
  
• Fill in the PyTorch API using TensorFlow imports
• Fill in the TensorFlow API using PyTorch imports
  

The last two here I am most leaning towards… as these can be monkeypatched in for seamless interoperability across codebases*

*expecting some caveats to apply

Along the way various helpers will need to be ported out of TensorFlow, e.g., tf.data (and not with trivial .as_numpy() solutions).

PS: Don't think that PyTorch is the only framework in scope… but we should start somewhere =)

PPS: Cross-posted this to ML GDE group

If you're interested reply to this email thread. I'll send through a calendar invite for the inaugural meeting once sufficient interested has been shown.

Let's stop competing and start collaborating

SAMUEL MARKS
Sydney Medical School | Westmead Institute for Medical Research | https://linkedin.com/in/samuelmarks
Director | Sydney Scientific Foundation Ltd | Offscale.io, Sydney Scientific Pty Ltd
SYDNEY SCIENTIFIC FOUNDATION and THE UNIVERSITY OF SYDNEY

[Samuel Marks]

Plan to start on the first of next month. I am going to take what I've built for my PhD—https://github.com/SamuelMarks/ml-glaucoma—and turn it into something which would be useful to others.

Although it's open-source, I very much doubt anyone else would find it useful (in its current form). My next version I want to choose a better ontology (if you look at the number of options to the `train` CLI subcommand you would start to understand!).

Also want to open it up to not just TensorFlow, Keras, and PyTorch; but other frameworks also.

Following on here—with most of the relevant intricacies exposed on both the CLI and the Python API—I will add various forms of categorical and continuous optimisation of parameters and hyperparameters.

Including: ray tuning, bayesian methods, genetic algorithms, NN, and others. The categorical optimisation should not be dismissed so easily as some do, but instead this should be opened up also, even if it's just for the simple switcher between different transfer learning models that I currently do.

Finally, opening this up to non-binary—i.e., multi-class—classification problems, and expanding from the image-only focus to other foci (text analysis, video analysis, &etc.), including through unsupervised—and hybrid—learning.

The risk of doing this all myself is that it becomes another library that is really only useful to one engineer (or one team of engineers). Would really like contributions—even if just in idea form!—from others, so that it is generalised sufficiently.

Thanks for your consideration,

Samuel Marks

Charity | consultancy | open-source | LinkedIn

Let's stop competing and start collaborating 🚀

SAMUEL MARKS
Sydney Medical School | Westmead Institute for Medical Research | https://linkedin.com/in/samuelmarks
Director | Sydney Scientific Foundation Ltd | Offscale.io, Sydney Scientific Pty Ltd
SYDNEY SCIENTIFIC FOUNDATION and THE UNIVERSITY OF SYDNEY

--
You received this message because you are subscribed to the Google Groups "SIG Addons" group.
To unsubscribe from this group and stop receiving emails from it, send an email to addons+un...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/addons/6c18f143-7d14-4286-bfac-923901381619%40tensorflow.org.