Random grpc "Stream removed" in cloud function

[Florian Lherbette]

Hello,

I've started to get weird errors in my Firebase app a day ago. One of my functions is randomly failing with a cryptic "Stream removed" error (see stacktrace below). I have difficulty pinpointing where the functionin fails, but I think it's happening on a Firestore read.

Does anyone have an idea about what's happening here? Oddly this started to happen after a deployment (but with no changes on the failing function).

Error: Stream removed
    at ClientReadableStream._emitStatusIfDone (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client.js:255:19)
    at ClientReadableStream._receiveStatus (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client.js:233:8)
    at /user_code/node_modules/firebase-admin/node_modules/grpc/src/client.js:705:12

-- 

Florian

[sg]

Does this happen with local emulation as well?

firebase serve

(Make sure your firebase-tools and grpc modules are up to date) Also verify that your node version is supported by firebase-tools.

[ma...@snowitapp.com]

Hello,

I'm getting this error too! I guess it does not depend on the function.

I asked to support but no answer till now.

There's also an open question on stack overflow:

https://stackoverflow.com/questions/47856259/firebase-functions-get-error-stream-removed-oncreate-firestore-event

Any help is appreciated. Thanks.

[Jelmer Feenstra]

I'm getting the same error since a few (3?) days. I thought it could be related to me now reading firestore in parallel, but now I'm not so sure.

The error happens randomly for me as well, but doesn't seem to happen with local emulation. My firebase-admin version is 5.5.1.

[Davide Biagini]

I have a https endpoint cloud function that will trigger a Firestore query

exports.extractWinners = functions.https.onRequest((req, res) => {
    let time = Date.now()
    const DELTA = 60000
    admin.firestore().collection('wishes').where('expiration', '>=', time - DELTA).where('expiration', '<=', time + DELTA).get().then((querySnapshot) => {

 If I go to the endpoint through the browser, execution goes correctly

When I'm calling it from www.cron-job.org I get

{ Error: Stream removed
    at ClientReadableStream._emitStatusIfDone (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client.js:255:19)
    at ClientReadableStream._receiveStatus (/user_code/node_modules/firebase-admin/node_modules/grpc/src/client.js:233:8)
    at /user_code/node_modules/firebase-admin/node_modules/grpc/src/client.js:705:12 code: 2, metadata: Metadata { _internal_repr: {} } }

[Sebastian Schmidt]

Hi,

Thank you for your report. We have seen a number of these errors lately and are investigating internally.

Sebastian

[Francis Au-Yeung]

Thanks much for looking into this! 

We are also experiencing the exact error message (and stack trace) for all of our Cloud Function deployments accessing Firestore.

THANK YOU!

-francis

[gr...@net-conversion.com]

Just chiming in that we've seen this too. Particularly if we've deleted a sub-collection then try to re-add it.

[ken...@carnivorestudios.com]

We are getting the same errors with our app. 

On Monday, December 18, 2017 at 12:08:58 PM UTC-8, Sebastian Schmidt wrote:

[Eric Poulin]

I'm getting the same issue. Quite sure my functions are setup correctly and returning a promise. This is also affecting functions I have barely changed.

[ma...@snowitapp.com]

Any News?

Il giorno domenica 17 dicembre 2017 19:58:14 UTC+1, Florian Lherbette ha scritto:

[sapphire.m...@engineerbetter.com]

We're experiencing exactly the same error, it started around three days ago. Any updates on this?

[Gabriel Bessa]

Also getting this error

[Gerard Niemira]

Hi Firebase Team -- Any update on this? Without a path to resolution it's going to become a real problem for us soon. Thank you!

[Laurent Pellegrino]

I am also getting the error. At least 2 days since the last update. The status page shows no problems...

https://status.firebase.google.com/

Is it normal?

[Sebastian Schmidt]

Hello all,

Sebastian from the Firestore SDK team here. We believe this issue is related to the recent update of the GRPC Client SDK and have been running tests with GRPC 1.7.1. So far, we have not been able to reproduce this issue with this older GRPC version.

@google-cloud/firestore is now at 0.10.1. If you update your dependencies, you will be able to pull in this release.

Thanks for your patience.

Sebastian

[Mong Kok Seong]

Hi Sebastian,

I'm accessing firestore from cloud functions, hence i am using: https://github.com/firebase/firebase-admin-node (version 5.7.0 currently)

and it is currently just depending on @google-cloud/firestore 0.10.0, is the fix already available in this version? Or is it only available starting from version 0.10.1 ?

Cheers,

Mong

[Francis Au-Yeung]

I believe as of version 5.6.0 of firebase-admin, it depends on version "^0.10.1" of "@google-cloud/firestore".

And firebase-admin v.5.8.0 just became available.

-francis

[Mong Kok Seong]

Please have a look here: https://github.com/firebase/firebase-admin-node/blob/master/package.json

It is still "^0.10.0"