Re: Printing results from PHP MongoDB query problem

[Jesse]

Any tips or hints?

Thanks!

On Thursday, December 13, 2012 6:00:31 PM UTC+2, Jesse wrote:

Hi,

I'm having a problem with fetching and printing results from MongoDB (2.2.1) using PHP. 

I can print the results just fine if I can match the 100 results (limit) in below 60 seconds (timeout). The problem is that if I have matches here and there in database, and the 60 second timeout is reached then nothing is printed.

I was wondering if and how it is possible to print (even the part of the results) when the timeout is reached. The database is quite large with two db's with over 37 million entries each, running on 4 core xeon and 16gb RAM. 

I'm not worried about the fact that I might not reach the end of database with the limited time, it is not really crucial. I would just like to output the results already found when the timelimit is reached...

Below are main points of the PHP-script:

<?php

/////////////////////////////////

// Connect to database

$conn = new Mongo();

// Select a database

$db = $conn->messages;

// Select a collection

$collection = $db->logging;

/////////////////////////////////

/////////////////////////////////

// Define searchbase

$search_base = array('SDATA.destination-port' => $search_destinationport);

$search_base['SDATA.destination-address'] = $search_destinationip;

$search_base['SDATA.policy-name'] = $search_policyid;

$cursor = $collection->find($search_base)->sort(array('$natural' => -1))->limit(100)->timeout(60000);

/////////////////////////////////

try {

        foreach ($cursor as $obj) {

                   echo $obj['DATE'];

                        foreach ($obj['SDATA'] as $key => $result) {

                                 echo $result;

                        }

        }

}

catch(MongoCursorTimeoutException $e) {

        echo "No match in database";

}

// Close MongoDB connection

$conn->close();

?>

[Raxit Sheth <Mobile 4 Mumbai>]

hint:

$cursor = $collection->find($search_base)->sort(array('$natural'
=>-1))->limit(100)->timeout(60000);

[Jesse]

I was looking and looking at the documentation + forums but did not really see it. batchSize parameter allowed to return amount of results wanted at a time.

Thanks for the hint ;)

- Jesse

[Sam Millman]

Why are you manually adding a timeout? Also why are you manually searching your data and not using the querying power of Mongo?

To answer your question better, instead of using a try catch block on the whole lot which will try and do the entire cursor unless you get a timeout you can either:

- Wrap the catch around the getMore command

- Detect that the cursor returns a timeout via the return of the getMore command.

--
You received this message because you are subscribed to the Google
Groups "mongodb-user" group.
To post to this group, send email to mongod...@googlegroups.com
To unsubscribe from this group, send email to
mongodb-user...@googlegroups.com
See also the IRC channel -- freenode.net#mongodb

[Jeremy Mikola]

On Thursday, December 13, 2012 11:00:31 AM UTC-5, Jesse wrote:

 

I'm not worried about the fact that I might not reach the end of database with the limited time, it is not really crucial. I would just like to output the results already found when the timelimit is reached...

MongoCursor's timeout is useful for triggering exceptions for long running queries or, by setting 0 or -1, allowing the cursor to remain active past the 30-second default. In your case, if you simply want to stop processing results after 60 seconds, why not just track the time in PHP and abort your iteration after a set time?

If you find yourself only accessing the "DATE" and "SDATA" fields of your document, you may want to consider using result projection to limit the document data returned. That would allow you to scale up your batch size while still staying below the 16MB message limit (between server and driver).

Lastly, there should be no need to call Mongo::close(). The driver does this on its own upon shutdown, and manual closing defeats any benefit of connection pooling/re-use when working with things like FPM or Apache workers. On a lesser note, "?>" is also unnecessary if the entire file is PHP source code :)