tensor flow internal memory leak?

Andrew Sigurdson

unread,

Aug 7, 2018, 6:47:36 PM8/7/18

to TensorFlow.js Discussion

Have you guys noticed that every two predictions theres a spike in CPU usage and a very long frame where the UI freezes? We did some debugging on chrome and found that alot more things are going on during the second prediction than the first. But basically during the entirety of the webcam capturing frames, the ui freezes on alternating predictions. We have been looking for memory leaks on our end but nothing found so far.

Andrew Sigurdson

unread,

Aug 8, 2018, 11:17:45 AM8/8/18

to TensorFlow.js Discussion

Screen Shot 2018-08-07 at 5.43.54 PM.png

Screen Shot 2018-08-07 at 5.45.09 PM.png

Tyler Hill

unread,

Aug 8, 2018, 5:56:07 PM8/8/18

to TensorFlow.js Discussion

Do you have some code you can share that causes this?

Andrew Sigurdson

unread,

Aug 9, 2018, 1:10:45 PM8/9/18

to TensorFlow.js Discussion

Well now we are getting a very slow frame rate. I think we have cleaned up the memory issues. the FPS is like 2 if that. We want to understand why its so low for us on the phone. On the computer its about 8 frames per second. The pacman example is alot of frames per second on the phone and we are trying to accomplish that.

index.js

async function init() {
    console.log("VERSION 6");

    // const canvas = document.createElement("CANVAS");
    const canvas = document.getElementById("output");
    canvas.width = 224;
    canvas.height = 224;

    async function loadModel() {
        console.log("MODEL LOAD");
        document.getElementById("messages").innerHTML += "<br>Loading model";
        const model = await tf.loadModel(path_to_models + 'tfjs_model/model.json');
        document.getElementById("messages").innerHTML += "<br>Model loaded<br>";
        model.predict(tf.zeros([1, 224, 224, 3])).dispose();
        return model;
    }

    const model = await loadModel();

    async function loadVideo() {
        console.log("VIDEO LOAD");

        let constraints = window.constraints = {
            audio: false,
            video: true
        };
        if (/Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(navigator.userAgent)) {
            constraints = {video: {facingMode: {exact: 'environment'}}}
        }

        try {
            const stream = await navigator.mediaDevices.getUserMedia(constraints);
            const video = document.querySelector('video');
            video.srcObject = stream;

            return new Promise((resolve) => {
                video.onloadedmetadata = () => {
                    resolve(video);
                };
            });
        } catch (e) {
            console.log('navigator.getUserMedia error: ', e);
        }
    }

    const video = await loadVideo();
    var isPredicting = true;

    function showResult(result, duration, interval) {
        const index = argMax(result);
        document.getElementById("messages").innerHTML = "<br>PR:" + labels[index];
        document.getElementById("messages").innerHTML += "<br>CO:" + result[index];
        document.getElementById("messages").innerHTML += "<br>running model: " + duration + " ms";
        document.getElementById("messages").innerHTML += "<br>draw interval: " + interval + " ms";
    }

    let lastDraw = performance.now();

    async function processFrame() {
        while(isPredicting) {
            const now = performance.now();
            const interval = now - lastDraw;
            lastDraw = now;
            const start = performance.now();
            // -------- INFERENCE ---------
            const prediction = tf.tidy(() => {
                console.log("INFERENCE");
                // document.getElementById("messages").innerHTML += "<br>Running model";
                const pixels = tf.fromPixels(video);
                canvas.getContext('2d').drawImage(video, 0, 0, 224, 224)
                const batchedImage = pixels.expandDims(0);
                try {
                    return tf.tidy(() =>
                        model.predict(batchedImage.toFloat().div(tf.scalar(127)).sub(tf.scalar(1))));
                }
                catch (runModelError) {
                    document.getElementById("messages").innerHTML += "<br>" + runModelError;
                }
            });
            prediction.data()
            .then((result) => {
                const duration = performance.now() - start;
                showResult(result, duration, interval);
            })
            prediction.dispose();
            // ----------------------------
           
            await tf.nextFrame();
        }
        

        //const result = [0];
        


    }

    console.log("VIDEO START");
    video.play();
    processFrame();


}

index.html

<!DOCTYPE html>
<html>
<style>
    .container {
        position: relative;
        color: red;
    }

    .top-left {
        position: absolute;
        top: 8px;
        left: 16px;
    }

    .top-right {
        position: absolute;
        top: 8px;
        right: 16px;
    }

    .bottom-left {
        position: absolute;
        bottom: 8px;
        left: 16px;
    }

    .text-size {
        font-size: 28px;
    }

</style>
<head>
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.3/jquery.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tf...@0.12.0"></script>
    <script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-converter"></script>
    <script src="index.js"></script>
    <title>Home</title>
</head>
<body onload="init()">
<div class="container">
    <!--<video autoplay playsinline width='100%' height='100%'></video>-->
    <video id="video" playsinline style="
        display: none;
        " width="224" height="224">
    </video>
    <canvas id="output"></canvas>
    <div class="top-left">
        <span class="text-size" id="messages"></span>
    </div>
    <!--<div class="top-right">-->
    <!--<button onclick="stopVideo()"> Stop Video</button>-->
    <!--<button onclick="runVideo1()"> Run Video</button>-->
    <!--</div>-->
</div>
</body>
</html>

Rohan

unread,

May 6, 2019, 2:13:27 AM5/6/19

to TensorFlow.js Discussion, as...@umich.edu

did you manage to fix it!?? I'm stuck on a similar issue.

Kangyi Zhang

unread,

May 6, 2019, 1:13:06 PM5/6/19

to Rohan, TensorFlow.js Discussion, as...@umich.edu

Hi Rohan,

Can you provide some more details of your use case?

Thanks

On Sun, May 5, 2019 at 11:13 PM Rohan <rohan1...@gmail.com> wrote:

did you manage to fix it!?? I'm stuck on a similar issue.

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/3e12681c-155c-40b7-ae91-65c99a441434%40tensorflow.org.

J Decker

unread,

May 9, 2019, 11:36:32 PM5/9/19

to TensorFlow.js Discussion, rohan1...@gmail.com, as...@umich.edu

This other message about slow performance creating tensors also mentions a memory leak...

https://groups.google.com/a/tensorflow.org/forum/#!topic/tfjs/CVYBwBRdUZg

On Monday, May 6, 2019 at 10:13:06 AM UTC-7, Kangyi Zhang wrote:

Hi Rohan,

Can you provide some more details of your use case?

Thanks

On Sun, May 5, 2019 at 11:13 PM Rohan <rohan1...@gmail.com> wrote:

did you manage to fix it!?? I'm stuck on a similar issue.

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tf...@tensorflow.org.

Reply all

Reply to author

Forward