Larger MongoDB Query from App Engine neither succeeding nor failing nor throwing exception

[Bradley Barrows]

Hi

This weekend we ran into an issue. A function, using Mongoose (a NodeJS MongoDB library) was making a pretty simple query, which when run locally either by running the query directly or when running the entire Node server directly ran fine.

However, when deployed to Google App Engine we noticed that logging was logged out up until we awaited the query to MongoDB and then never continued. No exception was ever logged as well.

It is as if the Mongo query was taking too long and App Engine just killed the Node process all together.

The server did return a 500 error. I believe it was a 503 specifically.

We ended up realizing by using limit or projection we could decrease the response size of the query and get a result and the execution of the NodeJS application would continue.

Now if Mongo was an HTTP server I would see that there is a 32MB maximum request size and be able to find that there are only so many records I can query at once. However, since this is a TCP connection to the Mongo server I cannot find any quotas or limitations on how much data a TCP connection from an App Engine server can send or receive.

As we know that decreasing the size of the response from MongoDB fixed our issue, I believe it is safe to say the issue is related to the amount of data sent/received or more specifically received over TCP when communicating with an App Engine server.

Does anyone have any details on known limitations of App Engine communicating with other services over TCP?

Thank you so much!
Brad

[Angel (Google Cloud Platform Support)]

Hello Brad,

to my knowledge there are no direct data limits in App Engine when used over TCP. There are, however, some architectural considerations that are worth being aware of for Google Cloud Platform. You can find the related article in [1]. It details some use cases in which a final user might experience latency issues or network inefficiencies. I believe it might give you some insight as to what might be happening in your case.

Kind regards.

______________________________

[1] - https://cloud.google.com/architecture/tcp-optimization-for-network-performance-in-gcp-and-hybrid

[Bradley Barrows]

This is exactly the article I have been looking for. Thank you so much!

I will respond if I can find any specific information related to this issue in case anyone else runs into a similar problem in the future as well.

Thank you so much,
Brad

--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/Lj6qaT2lC90/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/6d0562a1-a9e7-4baf-ba03-0335ffa9568dn%40googlegroups.com.