Jago Kapok
Jul 12, 2017, 11:22:40 PM7/12/17
to Fat-Free Framework
I want to build web app that handle big data (data that store in database is over than 1000 data).
When I tried to load data in my web app (over than 200 data), the page is shown error "page isn't working" (below that app runs normally).
I understand that F3 is microframework, but is there any way to do that?
When I tried to load data in my web app (over than 200 data), the page is shown error "page isn't working" (below that app runs normally).
I understand that F3 is microframework, but is there any way to do that?
ikkez
Jul 15, 2017, 10:46:39 AM7/15/17
to Fat-Free Framework
well memory can be a problem when you try to iterate through all at once.. depending on the technique and the amount of data, this can happen more or less quickly.
the mapper does consume more memory than working with the DB->exec method directly. I would suggest to use some kind of pagination or batch processing when possible. If not, you probably need to get your hands down on raw PDO and work with a Cursor.
the mapper does consume more memory than working with the DB->exec method directly. I would suggest to use some kind of pagination or batch processing when possible. If not, you probably need to get your hands down on raw PDO and work with a Cursor.
Nuwanda
Jul 17, 2017, 5:33:00 PM7/17/17
to Fat-Free Framework
This is right.
You should never load data without, at the very least, pagination. That is, without restricting the volume of data returned. You should also restrict the fields returned if only a subset of data is required, and apply filter criteria where possible. And use COUNT to check total results.
It seems irresponsible to simply allow your code to return all rows without any limiting. It falls into the same category as not validating or sanitizing user data from forms: you're just asking for trouble and unexpected conditions that crash your code.
You should never load data without, at the very least, pagination. That is, without restricting the volume of data returned. You should also restrict the fields returned if only a subset of data is required, and apply filter criteria where possible. And use COUNT to check total results.
It seems irresponsible to simply allow your code to return all rows without any limiting. It falls into the same category as not validating or sanitizing user data from forms: you're just asking for trouble and unexpected conditions that crash your code.
Reply all
Reply to author
Forward
0 new messages