Announcing support for native SavedModel execution in Node.js

[Brijesh Krishnaswami]

Hello TF.js community,

We are excited to bring you native SavedModel execution in Node.js. This is now available for use with the @tensorflow/tfjs-node package version 1.3.2 and later, on both CPU and GPU. 

Native execution enables direct execution of TensorFlow models saved in SavedModel format (from both TensorFlow 1.x and 2.0) in Node.js. Key features include:

• No conversion needed to the TensorFlow.js model format when deploying models in Node.js (with tfjs-node and tfjs-node-gpu)

• Native execution means that any TensorFlow SavedModel can execute, even if the model uses ops that are not supported in TensorFlow.js yet. 

• Performance improvements over TensorFlow.js models: See below a comparison of inference latency of MobileNet v2 between native execution in Node.js against converted execution and core Python TF on both CPU and GPU.  

Note: This feature is only available in the Node.js server environment, and not in the browser. Native execution depends on the libtensorflow C library, which is only included in the tfjs-node and tfjs-node-gpu packages, and not available in the tfjs union package.

Here is a short code snippet of how to load and run a TensorFlow SavedModel:

const model = await tf.node.loadSavedModel(path, [tag], signatureKey); const output = model.predict(input);

You can get the model tags and signatures information (aka MetaGraphs) through a helper API:

const modelInfo = await tf.node.getMetaGraphsFromSavedModel(path);

We would love to hear your feedback and/or questions. 

thanks,

Brijesh (on behalf of the TensorFlow.js team)

--

Brijesh Krishnaswami | Technical Program Manager, TensorFlow JS | br...@google.com | 650-943-3836

[Edoh Kodjo]

Hi,

That's an exciting news. Big thumb for all working hard to bring all these features. I wonder since tfjs-node can now read tensorflow saved-model would it offer to train from scratch object detection model ? 

Thanks

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/CA%2Bv6sqEoiqetC4ppmeMzTQ%3DDfTVyPcuj-1-Y_b_9-2TO7F3kYg%40mail.gmail.com.