predicting many images ends up OOM error of GPU.

[Eddie Lee]

When I load saved_model once to detect objects and predict large amount of images.
I used tf.tidy, tf.dispose everywhere it can possibly help to clear memories. 

However, I still get the oom message after processing 400~500 images. 

I have to close the electron app and restart to continue. Is there a way to flush gpu memory without closing the process. 

[Jason Mayes]

Are you disposing of your images or keeping them in memory eg in the DOM or in some array?

Have you checked the TF Tensor count to ensure if the tensors in memory is the cause or the image data itself?

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/46367005-23d8-4e01-a22c-2ded572cd64cn%40tensorflow.org.

--

•  Jason Mayes   •  Senior Developer Advocate, TensorFlow.js   •  Google   •   Follow me or get inspired with #MadeWithTFJS demos

This email may be confidential and privileged. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Na Li]

Hi Eddie,

Can you share your code?

On Wed, Jan 13, 2021 at 6:49 PM Eddie Lee <eunmi...@gmail.com> wrote:

--

[Eddie Lee]

I used Worker initiate the tfjs. Then, 

const detectionModelSync = await loadDetectionModel({ modelsDir });

const classificationModelSync = await loadClassificationModel({ modelsDir });

for loop {

 I run each image to get bounding box and class

}

[Eddie Lee]

+ I printed out logs of tf.memory every step in the for loop. I checked that the tensor size didn't increase.

[Na Li]

I see that the local detector will return a tensor, when is that tensor disposed?

[Eddie Lee]

There is tf.dispose(tensor) in the for loop.

[Na Li]

Hi Ping and Ann,

Do you know how come this can happen. If our memory tracking doesn’t show changes, could it be TF library has memory leak?

[Mark Strefford]

I've found that tf.dispose() doesn't remove all tensors created by tf.js, it seems there are intermediary tensors created during many tfjs calls. I now wrap any tf.js code in tf.tidy(). It's worth a try, although to be fair this doesn't explain why your tf.memory values don't change.

Also, as an aside, I notice you create buffers on each pass through your detection loop. It could be that these aren't being cleared up properly, so perhaps create the buffer once and reuse it each time. I don't know how efficient Javascript's memory deallocation processes are.

[Mark Strefford]

Here's an example of how I've used tf.tidy():

```let preds;
tf.tidy(() => {
x = this.preprocessInput(x, state);
const predictions = this.net.predict(x);
preds = this.getLocationAndRotation(predictions, state, debug);
})
return preds;```