Skip to first unread message
TH TH
Apr 4, 2016, 10:59:19 AM4/4/16
to Google App Engine
So I have 25,000 .py scripts running at the same time every 2 minutes as CRON jobs (distributed equally over 1000 servers)
They all have to access the same DB at the same time, ONLY to READ from db, not to write.
So I was wondering if Google Datastore (for example) could handle this? 25000 requests to the same DB at the same time just to read?
I couldnt find this in Google's AE docs.
Best Regards
Nick (Cloud Platform Support)
Apr 4, 2016, 2:49:43 PM4/4/16
to Google App Engine
Hey TH,
The Datastore does have restrictions on concurrent modifications, but as far as read operations, there is an unlimited quota when billing is enabled. As long as you can provision enough reading apps, you should be able to successfully run the script without hitches, although you'll want to experiment and profile with the system to determine what bottlenecks might exist in your script's logic. You could look into Memcache as well, as a potential middle layer to speed up the reads.
Let me know if you still have questions and I'll be happy to help.
Sincerely,
Nick
Cloud Platform Community Support
The Datastore does have restrictions on concurrent modifications, but as far as read operations, there is an unlimited quota when billing is enabled. As long as you can provision enough reading apps, you should be able to successfully run the script without hitches, although you'll want to experiment and profile with the system to determine what bottlenecks might exist in your script's logic. You could look into Memcache as well, as a potential middle layer to speed up the reads.
Let me know if you still have questions and I'll be happy to help.
Sincerely,
Nick
Cloud Platform Community Support
TH TH
Apr 4, 2016, 5:08:04 PM4/4/16
to Google App Engine
Is there a way to use Datastore inside Python 3? If so, how can install and load modules?
Nick (Cloud Platform Support)
Apr 5, 2016, 3:27:53 PM4/5/16
to Google App Engine
Hey TH,
I'm not sure what you mean by "install and load modules". Perhaps this docs page on libraries in python27 will help you with that question, although feel free to clarify if I missed your meaning. There shouldn't be any differences for python3, given the differences between 3.x and 2.x wouldn't impact anything in the doc.
As to using Datastore with python, this is definitely possible, via the Cloud Datastore REST API, or via the gcloud python library, or via the google api client library for python to connect to the API, or via the ndb package. There are many means of connecting to Datastore.
I hope this has been helpful. Feel free to let me know if you have any further questions.
Regards,
I'm not sure what you mean by "install and load modules". Perhaps this docs page on libraries in python27 will help you with that question, although feel free to clarify if I missed your meaning. There shouldn't be any differences for python3, given the differences between 3.x and 2.x wouldn't impact anything in the doc.
As to using Datastore with python, this is definitely possible, via the Cloud Datastore REST API, or via the gcloud python library, or via the google api client library for python to connect to the API, or via the ndb package. There are many means of connecting to Datastore.
I hope this has been helpful. Feel free to let me know if you have any further questions.
Regards,
Nick
Cloud Platform Community Support
Reply all
Reply to author
Forward
0 new messages