Uncaught (in promise) Error: Cannot start training because another fit() call is ongoing. - after moving from tfjs@0.12.3 to tfjs@0.12.5

[Roberto Stelling]

I've the following code that was, apparently, working fine in tf...@0.12.3 (model converging and no error messages)

When I've switched from tf...@0.12.5 the following messages show in the console:

tf...@0.12.5:2 Uncaught (in promise) Error: Cannot start training because another fit() call is ongoing.

    at t.<anonymous> (tf...@0.12.5:2)

    at tf...@0.12.5:2

    at Object.next (tf...@0.12.5:2)

    at tf...@0.12.5:2

    at new Promise (<anonymous>)

    at __awaiter$1 (tf...@0.12.5:2)

    at t.fit (tf...@0.12.5:2)

    at t.<anonymous> (tf...@0.12.5:2)

    at tf...@0.12.5:2

    at Object.next (tf...@0.12.5:2)

I don't see no other changes during execution except for the console messages.

Any pointers?

[Shanqing Cai]

Do you know which line of the code snippet you showed this error happens at? 

--
You received this message because you are subscribed to the Google Groups "TensorFlow.js Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tfjs+uns...@tensorflow.org.
Visit this group at https://groups.google.com/a/tensorflow.org/group/tfjs/.
To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/bd7326ea-44c5-4ee4-be0d-2bdbed32dcd6%40tensorflow.org.

--

---

Shanqing Cai

Software Engineer

Google

ca...@google.com

[Roberto Stelling]

Hi Shanqing Cai,

When I `unfold` the error, it shows the call to

  requestAnimationFrame(trainAndMaybeRender);

as the starting point.

Cheers,

Roberto Stelling

[Shanqing Cai]

I think your call to requestAnimationFrame() is causing the error. Because of that call, there will be multiple calls to your trainAndMaybeRender() function happening in an time-interleaved way,

which in turns causes Model.fit() to be called more than once concurrently, giving this error. In 0.12.5, we made a change that disallows concurrent calls to Model.fit(). 

Can you restructure your code to make sure that Model.fit() calls don't happen in an concurrent way?

To view this discussion on the web visit https://groups.google.com/a/tensorflow.org/d/msgid/tfjs/7a51f614-22e8-4ace-b663-c6e2afdb053e%40tensorflow.org.

[Roberto Stelling]

Then, it should work if I move the call to requestAnimationFrame to the end of trainAndMaybeRender() function.
Will give it a try and let you know.

Thanks a lot!

[Roberto Stelling]

Yes, it did the trick! Actually it is enough to place the call to requestAnimationFrame after the calls to train1Batch, to prevent concurrent calls to model.fit.
Thanks a lot for the help!