Methods for processing large data amounts from database

4 views
Skip to first unread message

Alexander Obuhovich

unread,
Oct 17, 2011, 9:31:47 AM10/17/11
to In-Portal Development
In-Portal nice $this->Application->Conn->Query method, that accepts SQL and returns an array of data to used right away. This proved very useful for small sets of data.

However, when you need to process 2000 or more records, then this method will eat up a lot of memory while transforming sql result into PHP array.

Attached methods contains kDBConnection::QueryRaw and kDBConnection::GetNextRow methods, that basically does the same what mysql_query and mysql_fetch_assoc methods. Also kDBConnection::Destroy method was changed to accept incoming recordset resource, that needs to be destroyed.


Here is usage example:

$rs = $this->Conn->QueryRaw($sql);

while ( $row = $this->Conn->GetNextRow($rs) ) {
echo $row['userid'];
echo $row['fullname'];
echo $row['userstatus'];
}

$this->Conn->Destroy($rs);



Ready for testing.


--
Best Regards,

http://www.in-portal.com
http://www.alex-time.com
db_class_for_large_data_set_512.patch
db_class_for_large_data_set_513.patch

Dmitry A.

unread,
Oct 23, 2011, 2:23:46 PM10/23/11
to in-por...@googlegroups.com
Thanks for the patch!

Do I test it on 5.1.3 or 5.2.0 (no patch yet)?


DA

Alexander Obuhovich

unread,
Oct 23, 2011, 3:47:45 PM10/23/11
to in-por...@googlegroups.com
You can apply to 5.2.0 too, there are 2 new methods and a small change for 1 method, that can be applied manually, when auto-apply won't work.
Reply all
Reply to author
Forward
0 new messages