Google Cloud Run Container Status

[Rodrigo Silveira]

I realize this is not the Cloud Run group, but that's the closest to it.

I've asked my question on Stackoverflow, but the solution(s) posted there were not helpful.

My issue: I have a Flask (Python 3.6) REST API that I'm deploying on Cloud Run. The first thing my server does at startup is to load two large Pandas dataframes into memory. Until those two global objects are loaded, the API is not useful. I have a health check defined in my Dockerfile that helps with this - until the two dataframes are loaded, the health check is not healthy. The problem is that when I deploy a new revision on Cloud Run, it will start delivering traffic to my container before it's actually fully loaded, and thus, the first several requests to the API will take 10-15 seconds.

I've optimized this startup step about as much as it can before things become completely ridiculous and unreasonable, and yet, this is the best I can do.

My question: is there a way to help Cloud Run know that the container is still loading and shouldn't be taking traffic yet? Something along the lines of: load the container, but don't add it to the load balancer yet, since the status is not yet "healthy"?

[Jason Collins]

I believe you can just delay listening on $PORT until you're ready to receive traffic.

[Rodrigo Silveira]

Update: after posting the above and stating that I'd optimized everything that could be optimized, I decided to take another stab at it. Now my cold starts are at 2 seconds. Great improvement, but I'm curious what else I might be able to do on the Docker side. Thanks!

[Elliott (Cloud Platform Support)]

Hello Rodrigo,

What do you have in mind to optimize on the Docker side? Although this may not be the right forum for that, maybe we can help.

[Rodrigo Silveira]

What I'm trying to accomplish is to have Cloud Run not drive traffic to my container until it's fully loaded. I have Docker health checks that do that, but Cloud Run seems to ignore them. I also delay the port binding (as suggested by Jason Collins), but again, Cloud Run seems to ignore that as well, as I get crazy high latencies right after deploying a new revision.

Thanks

On Thu, May 7, 2020 at 11:07 AM 'Elliott (Cloud Platform Support)' via Google App Engine <google-a...@googlegroups.com> wrote:

Hello Rodrigo,

What do you have in mind to optimize on the Docker side? Although this may not be the right forum for that, maybe we can help.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/24199233-1038-430b-ab8a-43d2636f7c05%40googlegroups.com.

[David (Cloud Platform Support)]

You mentioned you tried delaying port binding and it did not work, can you explain what happens exactly? As Cloud Run should start sending traffic to your application once you start listening on the port number so if you are delaying the port binding, it should work in your benefit.