Failed to compile fragment shader

Eduardo Ballesteros

unread,

Jul 17, 2018, 6:00:57 PM7/17/18

to TensorFlow.js Discussion

Hi everyone,

I am having an issue similar to #177. When running on chrome I get the error "Failed to compile fragment shader" when I call model.execute({input: myTensor}).

Tensorflowjs 0.12.
Chrome 67 on macOS (High Sierra).
myTensor size: 76800 shape: [1,160,160], dtype: "float32"

My script looks like this:

generate webmodel:

tensorflowjs_converter 
--input_format=tf_frozen_model
--output_node_names='output_node0, output_node1'
--saved_model_tags=serve
frozen_model.pb 
webModel/

JS

model =  loadFrozenModel(webModel/tensorflowjs_model.pb, webModel/weights_manifest.json)
//generate myTensor
model.execute({input: myTensor})

I can work around this error disabling hardware acceleration but the the model is too slow.

Thanks in advance.

Eduardo Ballesteros

unread,

Jul 17, 2018, 11:26:54 PM7/17/18

to TensorFlow.js Discussion

* myTensor shape: [1,160,160,3]

Nikhil Thorat

unread,

Jul 18, 2018, 9:55:02 AM7/18/18

to eballeste...@hawk.iit.edu, TensorFlow.js Discussion, Ping Yu

Hi Eduardo -- would you host your model online somewhere for us to look at?

Ping can you look into this?

On Tue, Jul 17, 2018 at 11:26 PM Eduardo Ballesteros <eballeste...@hawk.iit.edu> wrote:

* myTensor shape: [1,160,160,3]
--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/6fb1a7cc-7ab4-4287-a341-6462e92f5e8c%40tensorflow.org.

Message has been deleted

Andrew Sigurdson

unread,

Jul 18, 2018, 10:23:04 AM7/18/18

to TensorFlow.js Discussion

Hello Nikhil, I'm working with Eduardo on this issue. We can't host this model anywhere because we don't want to release the model publicly. But we can share with you our index.js file and index.html file. Our code is very simple. It just has an index.html, index.js, and web model folder that contains our model and weights. If you still cant figure out the issue from those files than we will make a model that hopefully replicates this issue. We hope these files are enough to figure out our error. Thanks. Andrew Sigurdson

Message has been deleted

Eduardo Ballesteros

unread,

Jul 18, 2018, 10:53:52 AM7/18/18

to TensorFlow.js Discussion

index.html

<html>
    <head>
        <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tf...@0.12.0"></script>
        <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-converter"></script>
        <script src="index.js"></script>
    </head>
    <body>
        <img src="images/sample.jpg" id="picture" height="160" width="160">
        <button id = "loadMod"> Load </button>
        <button id = "runMod"> Run </button>
        <script>
            var loabButton = document.getElementById('loadMod');
            var runButton = document.getElementById('runMod');
            loabButton.addEventListener('click',loadMod);
            runButton.addEventListener('click',runMod);
        </script>
    </body>
</html>

index.js

var model;
const path_to_models = 'http://someIP:8080/';

async function loadMod(){
    console.log("Loading model")
    model = await tf.loadFrozenModel(path_to_models + 'tensorflowjs_model.pb', path_to_models + 'weights_manifest.json');
    console.log('Model loaded')
}

function runMod() {
    imageElement = document.getElementById("picture");
    const image = tf.fromPixels(imageElement);
    const batchedImage = image.expandDims(0);
    try{
        output = model.execute({
            "import/input_1": batchedImage.toFloat()
        });
    }
    catch(runModelError){
        console.log(runModelError)
    }
    console.log(output[1].dataSync());
}

Daniel Smilkov

unread,

Jul 18, 2018, 11:15:55 AM7/18/18

to eballeste...@hawk.iit.edu, TensorFlow.js Discussion

Hi Eduardo and Andrew,

Can you send us a screenshot of the console log and the stack trace when the error happens? To give you an idea of what we are looking for, I purposefully introduced a bug in our relu shader and I'm attaching the screenshot of the error that would be helpful to us:

Daniel

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/cdae9783-c319-45bb-a310-ac5d5d1430b1%40tensorflow.org.

Eduardo Ballesteros

unread,

Jul 18, 2018, 11:39:49 AM7/18/18

to TensorFlow.js Discussion, eballeste...@hawk.iit.edu

I attach the full message. Thank you very much.

Screen Shot 2018-07-18 at 10.35.28 AM.png

Screen Shot 2018-07-18 at 10.35.47 AM.png

Screen Shot 2018-07-18 at 10.35.59 AM.png

Screen Shot 2018-07-18 at 10.36.21 AM.png

Nikhil Thorat

unread,

Jul 18, 2018, 12:51:24 PM7/18/18

to eballeste...@hawk.iit.edu, TensorFlow.js Discussion

Thanks for the screenshots, these are perfect -- we'll debug this today!

On Wed, Jul 18, 2018 at 11:39 AM Eduardo Ballesteros <eballeste...@hawk.iit.edu> wrote:

I attach the full message. Thank you very much.

--

You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/26e57556-1ae4-4553-9d42-a4bdaaf0ce24%40tensorflow.org.

Andrew Sigurdson

unread,

Jul 18, 2018, 3:06:41 PM7/18/18

to TensorFlow.js Discussion, eballeste...@hawk.iit.edu

Perfect, thank you so much.

Daniel Smilkov

unread,

Jul 18, 2018, 5:39:45 PM7/18/18

to as...@umich.edu, TensorFlow.js Discussion, eballeste...@hawk.iit.edu

Turns out that the a param passed to tf.div(a, b) is a 4d tensor with a zero in its shape (e.g. [0, 1, 1, 1]) and TF.js doesn't support zeros in a shape. Now, since you are using the converter, the call to tf.div() originates from the converter, which just evaluates the python graph. It turns out that TF Python explicitly supports zeros in a shape for good reasons, so we'll add that support in TF.js too.

In the meantime, you might want to investigate where the 0 originates from in your python code, and, if possible, tweak the code to get around that problem.

Hope this helps!

Daniel

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/1182b609-0c53-47f3-9c52-de4078bd5f21%40tensorflow.org.

Daniel Smilkov

unread,

Jul 18, 2018, 6:00:18 PM7/18/18

to as...@umich.edu, TensorFlow.js Discussion, eballeste...@hawk.iit.edu

I'm also very curious to understand if that 0 is a side-effect of a bug on the python side, or a completely valid case. Let me know! Thanks!

Daniel

Andrew Sigurdson

unread,

Jul 18, 2018, 6:32:56 PM7/18/18

to Daniel Smilkov, TensorFlow.js Discussion, eballeste...@hawk.iit.edu, Saeid Bagheri

Hey Daniel, thanks for all your help! I’m very new to tensor flow to be honest and someone else wrote the python code. I’m not really sure why the shape is (0,1,1,1) so I will consult the person who wrote the python tf with your question tomorrow and let you know.

Andrew Sigurdson

unread,

Jul 20, 2018, 10:58:02 AM7/20/18

to TensorFlow.js Discussion, as...@umich.edu, eballeste...@hawk.iit.edu

Daniel, is there a date in mind that you think this will be added to tensor flow js? And apparently, we are unable to tweak the python code to get around this. So I have to wait for you guys to make the patch. On a separate note, as I understand, the model can easily be "stolen" so do you have any advice for me for keeping the model private? Thank you,

Andrew Sigurdson

Daniel Smilkov

unread,

Jul 20, 2018, 11:16:53 AM7/20/18

to as...@umich.edu, TensorFlow.js Discussion, eballeste...@hawk.iit.edu

Hi Andrew,

Before we can't commit to a date, we need to better understand how often and why this happens in practice and if there are workarounds. If you can share why you are unable to tweak the python code, it will help us plan and re-prioritize.

As for the model security, we've been thinking about this a lot. There is no way to protect a model at the moment. If you want to protect the model for all but a small set of users, you can use auth tokens and require users to authenticate.

Daniel

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/f54ef53f-d0cf-4228-8e22-21bc1923b59d%40tensorflow.org.

Eduardo Ballesteros Grande

unread,

Jul 23, 2018, 3:51:24 PM7/23/18

to smi...@google.com, as...@umich.edu, tf...@tensorflow.org

Hi Daniel,

That 4th dimension arises from the model being prepared to process batches of images. The first dimension corresponds to the number of images in the batch. I don't know why that 0 appears, I would be expecting a 1 since we are passing 1 image (and when printing the tensor we see 1,160,160,3 shape). Also I find it curious that this does work if we disable WebGL (although it's really slow).

Sorry for not being able to throw any more light into this. We will continue investigating and keep you guys posted.

Best,

Eduardo

Daniel Smilkov

unread,

Jul 24, 2018, 1:07:19 PM7/24/18

to eballeste...@hawk.iit.edu, as...@umich.edu, TensorFlow.js Discussion

Thanks! There is definitely a 0 at some point. To help debug, you can short-circuit the computation by specifying explicitly an output node name:

output = model.execute({"input": batchedImage.toFloat()}, outputNodeName);

See signature here for details.

I would specify outputs that are near the input and move towards the final nodes - this will help reveal the culprit operation where the error shows up, which can help us debug.

Daniel

Reply all

Reply to author

Forward