node.js server side cache

[Emil Anticevic]

Hi all,

How does the JS library cache behave when running in a Node.JS process?

It seems like it grows indefinitely and never clears. Does this mean that a server trying to process jobs for a million users generating 1 job every few minutes would eventually just die due to insufficient memory because finished jobs still live in memory?

Is there a way to run the Firebase JS library server side in such a way that it doesn't grow in RAM indefinitely or to disable/clear the local in-memory cache?

Best,

Emil

[Itzy]

If what you're saying is correct, you should be able to clear the cache by logging off (unauth()) and then logging the server back in again. You may have to wait for the ongoing jobs to finish and stop new ones from being created before logging out, though. 

Is there a better way?

Itzy

[Emil Anticevic]

Hi Itzy, so based on my experiments deep into last night the cache DOES eventually clear but it's sort of opaque when (It seems to be ready for garbage collection on the NEXT tick of the server process). What's not clear is when Firebase is no longer going to use this cache and will instead force a read from the server.

What you describe with unauth() SHOULD work, but that's not how Firebase programs their own examples and libraries on github. Also, it's very inefficient since it requires a redundant logout/login for EACH individual process. As far as I've always been able to tell Firebase team suggests you stay authenticated and keep creating / reading / writing to references. (see here for example: https://github.com/firebase/firebase-queue)... so it's potentially really easy for a developer to shoot themselves in the foot with cache and get strange behavior / runaway RAM growth on a worker process operating on lots of records.

Also, this issue is sort of related to the transaction issue here:

https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/firebase-talk/camly5PSaU4/8GGn9yKRfmUJ

In that case, the transaction will keep using the local cache even after the next tick even though there are no listeners etc. on that data, and we have no way to force the transaction to do a read before attempting the update, and we have no way to instruct the transaction to skip any attempts to call the update function on the locally cached value.

I'm still hopeful that the Firebase team will provide some insight short of "don't use our JS library server side" :)

[Jacob Wenger]

Hi Emil,

As you've found out, the cache does not grow indefinitely and will eventually clear. If it doesn't, then that is a bug.

One thing that probably contributed to what you were seeing here (as well as with the transaction issue) was the fact that you were using v0 of our Firebase client, which was released over two years ago. We've made lots of improvements and bug fixes since then! Make sure to upgrade to the latest version of our Node.js client (which can be found at the top of the page here). We just released 2.2.7 as of this comment. Note that the Firebase client-side JavaScript SDK and Node.js SDK are one and the same.

For what it's worth, we use the Firebase Node.js library to run several of our services, so it is most definitely possible to use it in production. We haven't seen any of the caching issues you reported.

Jacob

--
You received this message because you are subscribed to the Google Groups "Firebase Google Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firebase-tal...@googlegroups.com.
To post to this group, send email to fireba...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/41b13782-5e36-4362-b4ea-9db3be0c41c5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Emil Anticevic]

Thanks Jacob,

I believe you are 100% correct.

We're running a lot of code on Node.JS + Firebase JS SDK at this point, so I'm excited to learn all the details and will let you folks know if we run into any edge cases.

Best,

Emil

--
You received this message because you are subscribed to a topic in the Google Groups "Firebase Google Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/firebase-talk/4PDZsG1kctM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to firebase-tal...@googlegroups.com.

To post to this group, send email to fireba...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/firebase-talk/CAGcMwsA-xPrpkp3XXY3Y1GyRCn_3j7-%2BiZ4hvfzirUL98CG8tg%40mail.gmail.com.