google app engine and big file generating

34 views
Skip to first unread message

Jukka Hautakorpi

unread,
Jan 27, 2020, 1:35:07 AM1/27/20
to Google App Engine
I have a gae php+sql -app
that I want to extend to generate big set of files. Php script will make these files from few 2 mb templates.

These files will be 100-500 mb total, and then zip them and push / return to user's browser, same way as google drive "zip files" seems to work, so i can assume that
1.  To generate such big set of files, it can't just be in the gae's ram, so we can save the files to Cloud Storage. I see that I can also zip files in cloud storage and then return this, so should not be a problem

2.  If there are 20 users generating files at the same time, we need a simple queue so that backend is processing some kind of job queue
what is the technique of making such a job queues and then run them with fifo-principle ?

Aref Amiri (Cloud Platform Support)

unread,
Jan 27, 2020, 3:08:16 PM1/27/20
to Google App Engine
You may want to consider using Cloud Tasks, as it is mainly used to manage the execution of a large number of distributed tasks. It allows you to execute tasks asynchronously and control those requests to better manage the load on your services. 
Reply all
Reply to author
Forward
0 new messages