Very slow connection to Cloud storage and Cloud SQL

301 views
Skip to first unread message

Leonard Cremer

unread,
Jul 10, 2020, 12:32:48 PM7/10/20
to Google App Engine
Problem you have encountered:
When I download a 83kb file from cloud storage it takes between 5 and 12 seconds where from my local machine it takes between 2 and 5 seconds.
My cloudSQL inserts are also much slower than when tested from another server to CloudSQL so it seems to be related. 

What you expected to happen:
I expect this to be very fast as it is both on the us-central servers

Steps to reproduce:
  let filename = `${routeKey}.json.gz`;
  logger.info(loggerKey, `filename:${filename}`);
  const file = storageBucket.file(filename);
  logger.info(loggerKey, `have file:${filename}`);
  const [contents] = await file.download()
  logger.info(loggerKey, `downloaded:${filename}`);

Other information (workarounds you have tried, documentation consulted, etc):
Here is my stackdriver logs linked to the code above. The download step takes 12 seconds in this instance.
I 2020-07-10T08:18:15.188999891Z downloaded:159414332187374700443664:ATL:BRU:economy.json.gz
I 2020-07-10T08:18:03.111999988Z have file:159414332187374700443664:ATL:BRU:economy.json.gz
I 2020-07-10T08:18:03.111000061Z filename:159414332187374700443664:ATL:BRU:economy.json.gz
I 2020-07-10T08:18:03.111000061Z Starting processSkyJSONFile  

Juan Carlos Gomez

unread,
Jul 13, 2020, 5:12:51 AM7/13/20
to Google App Engine
Hello,

Thank you for your post. I understand that you have a Cloud SQL instance and Cloud Storage bucket in the us-central region and the access to both of this servers from your local machine is slow.

This could be for many reasons external to the Cloud Platform scope such as:
+ Temporary bad ISP bandwidth.
+ Another application(s) in your local machine are using the network bandwidth and it taking the larger part of it.
+ Another application(s) is using to much CPU power which is slowing the connection processes to your GCP resources.
+ other local network issues.

There was no known incident related to this as it can be seen in [1]

In order to help you further I would suggest that you consider all the other possibilities. And then, check with the gsutil tool to check the speeds with the a standard tool.

Leonard Cremer

unread,
Jul 13, 2020, 12:32:35 PM7/13/20
to google-a...@googlegroups.com
Hi Juan

It is actually not slow from my local machine but slow from an Google App Engine nodejs application running in the same region. If I do the same things from my local machine or Glitch it works fine but from the GAE app it is very slow. 

Thanks
Leonard


--
You received this message because you are subscribed to a topic in the Google Groups "Google App Engine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/wm9l1i3jeRU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/cacf578f-9861-48fa-b09a-d72a2579bf6eo%40googlegroups.com.


--
Leonard Cremer
Founder

Johannesburg, South Africa

Juan Carlos Gomez

unread,
Jul 14, 2020, 5:55:30 AM7/14/20
to Google App Engine

You are right, I twisted the effects.

Is this happening consistently for the requests in your App Engine?
If so, I suggest you create a ticket in the Cloud Support [1] so you can share information such as your project-id, bucket names, ... through a private channel.

Best regards,
Juan Carlos


Reply all
Reply to author
Forward
0 new messages