run php daemon

[Antonio Delli Carpini]

is there a way to run files get tweets and parse tweets directly from a single php file?

[Adam Green]

Hi, Antonio. In theory you could certainly rewrite get_tweets.php and parse_tweets.php into a single script. The basic goal, however, is to insert the tweets as fast as possible, so you can keep up with the API when the flow is high. That is why I grab the tweets and insert them as simply as possible in get_tweets.php. Then parse_tweets.php can work with them at any rate without stalling the collection. 

On Wed, Apr 30, 2014 at 11:42 AM, Antonio Delli Carpini <anto.del...@gmail.com> wrote:

is there a way to run files get tweets and parse tweets directly from a single php file?

--
You received this message because you are subscribed to the Google Groups "140dev Twitter Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 140dev-twitter-fra...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Adam Green
CEO, 140dev.com
CTO, UniteBlue.org
ad...@140dev.com
781-879-2960
@140dev

[Antonio Delli Carpini]

This I have understood. But if I wanted to run the script remotely on a web server, I would make sure that within a php file recall the two files: get and parse. In this way, there are two processes running on the server; While the other way processes are running on your computer.

Il giorno mercoledì 30 aprile 2014 17:50:00 UTC+2, Adam Green ha scritto:

Hi, Antonio. In theory you could certainly rewrite get_tweets.php and parse_tweets.php into a single script. The basic goal, however, is to insert the tweets as fast as possible, so you can keep up with the API when the flow is high. That is why I grab the tweets and insert them as simply as possible in get_tweets.php. Then parse_tweets.php can work with them at any rate without stalling the collection. 

On Wed, Apr 30, 2014 at 11:42 AM, Antonio Delli Carpini <anto.del...@gmail.com> wrote:

is there a way to run files get tweets and parse tweets directly from a single php file?

--
You received this message because you are subscribed to the Google Groups "140dev Twitter Framework" group.

To unsubscribe from this group and stop receiving emails from it, send an email to 140dev-twitter-framework+unsub...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Antonio Delli Carpini]

I apologize for my English. But from the beginning I wanted to say if there was a way to retrieve the two files and get the parser from a php script instead of from the Command line

[Adam Green]

You can run a background process from a script with:

exec('nohup php parse_tweets.php > /dev/null &');

To unsubscribe from this group and stop receiving emails from it, send an email to 140dev-twitter-fra...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Antonio Delli Carpini]

I 've tried several times this code in localhost , but not working. I have Xampp on Windows. I also tried with other appropriate codes for windows , but I do not work. The only code that I work is as follows:

$ command = ' start / BC: \ xampp \ php \ php.exe ' . dirname ( __FILE__ ) . ' / get_tweets.php >> ' . dirname ( __FILE__ ) . ' / outlog - file.log ' ;

pclose ( popen ( $ command , ' r '));

$ command = ' start / BC: \ xampp \ php \ php.exe ' . dirname ( __FILE__ ) . ' / parse_tweets.php >> ' . dirname ( __FILE__ ) . ' / outlog - file.log ' ;

pclose ( popen ( $ command , ' r '));

But the code above me only works for a file, only for the first running and then send that in this way the process is activated on the machine and not remotely.

I have also tried with the following code:

$ logFile = ' app- output.txt ' ;

$ command = ' % COMSPEC % / c ' ;

$ command . = ' C :/ xampp / php / php.exe ' . dirname ( __FILE__ ) . ' / get_tweets.php ' ;

$ command . = ' > " ' . $ logFile . ' " 2> & 1 ' ;

$ WshShell = new COM ( " WScript.Shell ");

$ oExec = $ WshShell -> Run ( $ command , 1, true);

But still does not work .... At this point I asked for help from you.

Il giorno mercoledì 30 aprile 2014 22:01:54 UTC+2, Adam Green ha scritto:

You can run a background process from a script with:

exec('nohup php parse_tweets.php > /dev/null &');

On Wed, Apr 30, 2014 at 3:56 PM, Antonio Delli Carpini <anto.del...@gmail.com> wrote:

I apologize for my English. But from the beginning I wanted to say if there was a way to retrieve the two files and get the parser from a php script instead of from the Command line

Il giorno mercoledì 30 aprile 2014 19:52:33 UTC+2, Antonio Delli Carpini ha scritto:

This I have understood. But if I wanted to run the script remotely on a web server, I would make sure that within a php file recall the two files: get and parse. In this way, there are two processes running on the server; While the other way processes are running on your computer.

Il giorno mercoledì 30 aprile 2014 17:50:00 UTC+2, Adam Green ha scritto:

Hi, Antonio. In theory you could certainly rewrite get_tweets.php and parse_tweets.php into a single script. The basic goal, however, is to insert the tweets as fast as possible, so you can keep up with the API when the flow is high. That is why I grab the tweets and insert them as simply as possible in get_tweets.php. Then parse_tweets.php can work with them at any rate without stalling the collection. 

On Wed, Apr 30, 2014 at 11:42 AM, Antonio Delli Carpini <anto.del...@gmail.com> wrote:

is there a way to run files get tweets and parse tweets directly from a single php file?

--
You received this message because you are subscribed to the Google Groups "140dev Twitter Framework" group.

To unsubscribe from this group and stop receiving emails from it, send an email to 140dev-twitter-framework+unsubscr...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Adam Green
CEO, 140dev.com
CTO, UniteBlue.org
ad...@140dev.com
781-879-2960
@140dev

--
You received this message because you are subscribed to the Google Groups "140dev Twitter Framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email to 140dev-twitter-framework+unsub...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Adam Green]

Sorry. I have little experience using Windows as a server. I do all my server work with Linux. Maybe someone else has some ideas. 

To unsubscribe from this group and stop receiving emails from it, send an email to 140dev-twitter-fra...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Antonio Delli Carpini]

Ok thanks anyway. Maybe I learn to use linux :)

[Antonio Delli Carpini]

which linux distribution do you recommend?

[Adam Green]

I tend to use Centos, but that is because my sysadmin and all my various webhosts prefer it. Personally, they all are functionally the same for my work. 

To unsubscribe from this group and stop receiving emails from it, send an email to 140dev-twitter-fra...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Hugh Fuve]

ignore_user_abort(true);

If you just add this to the top of a PHP page, you can then load it from a web browser and it will keep running when you close it.

Be careful with these files though, you want some kind of authentication on them, because a malicious person could load up multiple copies of the file, and bring your server to it's knees. It pays to put some kind of abort option inside the main loop, something that tracks active threads and limits them, and something that looks for a file or mysql based abort command. Like you could create a lock file, and check for it on load.. if you find the lock then you just abort.. and if the lock should disappear while executing, then you abort the script.. this way you can just delete a lock file if you need to shut it down.

you should probably also try to hide /db/get_tweets.php and parse_tweets.php from exposure to the web for this reason, because multiple instances can currently be fired by anyone who knows you're running this code. Some kind of basic authentication would probably be fine.

Here's another way to create background PHP processes that can be fired from javascript or ajax on your pages. without using command lines, and in a way that will work on windows or linux.(and you can also use this technique to create multi-threaded php scripts that run in the background)

add this to the top of the scripts that you want to be persistent, (get_tweets.php) (parse_tweets.php) this will allow the page to keep running even if the user shuts the page down.

ignore_user_abort(true);

Then create a small 1 pixel image and place it in the same folder as your PHP scripts somewhere, the script will load this, the image could be anything so you could even make a 16*16 animated gif to show there is work in progress, I like 1 pixel because it lets you hide whats going on.

                1pixel.gif

Then add this to the top of get_tweets and parse_tweets it will load the image onto the page.

Header("Content-Type: image/gif");

          $fn=fopen("1pixel.gif","r");

          fpassthru($fn);

        flush();

Now you can load get_tweets or parse_tweets from any page with html like this..

<img src="get_tweets.php?vars=whatever">

which means you could take any mouseover or onclick event and load the php script from a web page, for example by changing the javascript src property of the image object. Or by just loading an <img> from inside a <span> by placing a span somewhere with an id assigned .. using something like

<span id=blaa><span>

and then you can create events within other objects of the page, like buttons and images and add this to it 

onclick=document.getElementById("blaa").innerHTML ="<img src='get_tweets.php'>"

This will cause the contents of the <span> to load with your image, and will activate the PHP file to start it's scan.

again when doing stuff like this, you would want to add a lock and abort mechanism and some kind of thread limit.

these types of tricks (with adequate credentials and processing power) will let you create 100+ simultaneous fsockopen() connections to different servers, using different user credentials, and show a live streaming progress report of what is going on, all from a single web page without having to drop to a command prompt.