Php client

[Adrenalin]

has anybody seen a php client for beanstalk?

[Keith Rarick]

On Sun, Aug 10, 2008 at 11:47 PM, Adrenalin <adren...@gmail.com> wrote:
> has anybody seen a php client for beanstalk?

I haven't seen one. It would be a welcome contribution!

kr

[Tim Gunter]

I have written one. I am currently testing it.



On Aug 11, 3:11 pm, "Keith Rarick" <k...@causes.com> wrote:

[Tim Gunter]

I just did some benchmarking and the speed is on-par with the ruby
client.

There is also client-side support for distributed servers.

[Keith Rarick]

On Fri, Aug 15, 2008 at 12:37 PM, Tim Gunter <icyl...@gmail.com> wrote:
> I just did some benchmarking and the speed is on-par with the ruby
> client.
>
> There is also client-side support for distributed servers.

Awesome. I'm really looking forward to this, Tim. Thanks.

kr

[Tim Gunter]

Well folks, here it is.

http://sourceforge.net/projects/beanstalk/

On Aug 19, 5:36 pm, "Keith Rarick" <k...@causes.com> wrote:

[Keith Rarick]

On Wed, Aug 20, 2008 at 11:43 AM, Tim Gunter <icyl...@gmail.com> wrote:
> Well folks, here it is.
>
> http://sourceforge.net/projects/beanstalk/

Thanks! I'll add a link from the beanstalkd page right away.

After a quick glance, I noticed something that looks like a typo on
lines 532 and 533. I think that

$this->mode = self::DRAINING;
return self::DRAINING;

ought to say

$this->mode = self::MODE_DRAINING;
return self::MODE_DRAINING;

Otherwise it looks great!

Thanks again.

kr

[Tim Gunter]

On Aug 21, 4:24 pm, "Keith Rarick" <k...@causes.com> wrote:
> Thanks! I'll add a link from the beanstalkd page right away.

That's great, thanks :)

> After a quick glance, I noticed something that looks like a typo on
> lines 532 and 533. I think that
>
>              $this->mode = self::DRAINING;
>              return self::DRAINING;
>
> ought to say
>
>              $this->mode = self::MODE_DRAINING;
>              return self::MODE_DRAINING;

Well spotted! I checked the fix into SVN and rolled a new release.
It's up now. Sorry!

Tim

[Erich]

Awesome! Thanks Tim. Just before I sat down to catch up on my groups,
I was thinking how useful a PHP beanstalk client could be for me this
week. One question tho: how did you know? :)

Regards,
Erich

[Tim Gunter]

On Thu, Aug 21, 2008 at 5:47 PM, Erich <soph...@gmail.com> wrote:

Awesome! Thanks Tim. Just before I sat down to catch up on my groups,
I was thinking how useful a PHP beanstalk client could be for me this
week.  One question tho: how did you know? :)

Can't let out all my secrets ;)

Well I'm glad there will be at least 1 person testing this monster :p Let me know of any bugs or things that aren't working as you would expect.

[Ludovic Levesque]

Hi Tim,

I spotted another typo:

$res[$servername] = $this->_do_my_sever($server, $in_method,
$in_args);

'sever' instead of 'server'

Thanks for the lib.

Ludo

On Aug 22, 1:39 am, "Tim Gunter" <icyliq...@gmail.com> wrote:

[Tim Gunter]

Hey Ludo,

Thanks for spotting that. I actually encountered this early yesterday
and committed it to SVN hehehe. I just haven't had the time to roll a
new package.

Is it working well in other respects?

Tim

--
I may have lost my way now
Haven't forgotten my way home

[Ludovic Levesque]

Yes, it works well, with some tubes.

But no stress tests for the moment.

Ludo

[FaceySpacey.com]

Hey everyone, I'm looking to hire someone to help me use beanstalkd
(php library) to speed up some heavy numerical calculations in a
Mortgage Loan search application/site I've built...I'm still trying to
find the right solution and was thinking map reduce type stuff (maybe
Pig) might be the right way to go. Currently, we're all setup on
Amazon EC2, and was also thinking Amazon SQS, but heard that this
might be faster...So I still need to find the solution that offers the
most value to implement in terms of cost and speed.

Anybody want to check out the scenario with me to help me make this
decision? If Beanstalkd is the way to go, I definitely have a job for
someone.

On Sep 23, 11:18 am, "Ludovic Levesque" <lud...@gmail.com> wrote:
> Yes, it works well, with some tubes.
>
> But no stress tests for the moment.
>
> Ludo
>

[avip]

Hi Tim,

Not sure if you've already spotted this typo or not but in function
__call(..) you have
"do_my_server" instead of "_do_my_server" (underscore in front).

Also, I'm not sure if this is my misunderstanding of PHP, the way you
planned it, or an actual error but: peek_ready() accepts only a write
back variable, not a server name; in the code you do an array_shift()
in hopes of getting the server name. So, elsewhere, if someone tried
doing just $beanstalk->peek_ready($nextJob); it would fail...

On Oct 14, 5:53 pm, "FaceySpacey.com" <jamesgillm...@gmail.com> wrote:
> Hey everyone, I'm looking to hire someone to help me use beanstalkd

> (phplibrary) to speed up some heavy numerical calculations in a

> > >>> > I was thinking how useful aPHPbeanstalk client could be for me this

[Tim Gunter]

Hi,

Thanks for pointing that out. I need to roll a new release soon, just
havent had the spare time.

Regarding your questions, I think it's just a misunderstanding.

$beanstalk refers to a pool of servers, and is represented by the
BeanStalk class. Calling peek_ready() on $beanstalk expects a server
name and optional writeback.

$beanstalk then routes the peek_ready() call internally to an actual
server instance, represented by a BeanQueue object. THIS peek_ready()
does not need a servername parameter, and therefore only accepts the
writeback.

Since array_shift modified the passed array, the top level
peek_ready() only sends the writeback to the internal peek_ready, as
the servername as already been shifted off.

Does that make sense?

- Tim

[avip]

Hi,

Wow, that was a fast turn-around! I thought it would take days :)
Thanks for both explanations, now that I read them (and after poking
in the code for a while), it seems pretty "obviously necessary" to
tell the function which server to perform the peek on given there's a
pool.

I've been trying some very basic PHP scripts to learn the internals, I
have one script that puts a job onto the server (I manually verify
it's there with a telnet session). Then the following:
<code>
require('bin/Others/BeanStalk.class.php');
$beanstalk = BeanStalk::open(array('servers' =>
array('127.0.0.1:11300'),'select' => 'random peek'));

$beanstalk->use_tube('test');

$nextJob = '';
$ret = $beanstalk->peek_ready('127.0.0.1:11300',$nextJob);

if ($ret === 1)
{
echo 'PEEK OPERATION OK<br/>';
echo 'Next ready job='.$nextJob.'<br/>';
}
else {
echo 'PEEK OPERATION FAILED<br/>';
}
</code>

I also turned debugging on and some simple debugging echos to
peek_ready(). Result:

Normal output from "if (BeanStalk::DEBUG) echo __METHOD__."\n";":
----------------------------------------------------------------------------------------------------
BeanStalk::__call(use_tube)
BeanStalk::_do_my_server
BeanQueue::use_tube
BeanStalk::__call(peek_ready)
BeanStalk::_do_my_server

Output from extra echo statements I added:
----------------------------------------------------------------------------------------------------
peek_ready(): in_writeback=127.0.0.1:11300 <--first line in function
JOB=I was here <--output right after "$job = $this-
>read_message($data[1]);"
Writeback variable provided <--from inside of "if (!
is_null($in_writeback))"
Output from code pasted above:
----------------------------------------------------------------------------------------------------
PEEK OPERATION OK
Next ready job=

Not sure why this is happening, could be some type of misunderstanding
on my end again. Any ideas as to what I may be doing wrong?

Thanks for a great lib and the feedback by the way! Much
appreciated :)
--Avi

On Nov 7, 3:06 pm, "Tim Gunter" <icyliq...@gmail.com> wrote:
> Hi,
>
> Thanks for pointing that out. I need to roll a new release soon, just
> havent had the spare time.
>
> Regarding your questions, I think it's just a misunderstanding.
>
> $beanstalk refers to a pool of servers, and is represented by the
> BeanStalk class. Calling peek_ready() on $beanstalk expects a server
> name and optional writeback.
>
> $beanstalk then routes the peek_ready() call internally to an actual
> server instance, represented by a BeanQueue object. THIS peek_ready()
> does not need a servername parameter, and therefore only accepts the
> writeback.
>
> Since array_shift modified the passed array, the top level
> peek_ready() only sends the writeback to the internal peek_ready, as
> the servername as already been shifted off.
>
> Does that make sense?
>
> - Tim
>
>
>
> On Fri, Nov 7, 2008 at 2:53 PM, avip <avital.pek...@gmail.com> wrote:
>
> > Hi Tim,
>
> > Not sure if you've already spotted this typo or not but in function
> > __call(..) you have
> > "do_my_server" instead of "_do_my_server" (underscore in front).
>

> > Also, I'm not sure if this is my misunderstanding ofPHP, the way you

[Tim Gunter]

Hey,

Yeah this is an issue with beanstalk clients like mine, and beanstalkd
1.0. I was trying to implement a polling check that could pull the
next ready job from a pool, but peek() didn't turn out to work as I
initially expected.

Avoid the peek modes for now. Stick to 'sequential wait' and 'random
wait' select values.

I will release v1.1 of my client shortly, and then the peek modes will
work again, as expected.

[avip]

Ah! Of all the functions, I choose this one to test first with :)

Will avoid the peek modes until next version, as suggested.

Thanks again!
--Avi

On Nov 7, 4:15 pm, "Tim Gunter" <icyliq...@gmail.com> wrote:
> Hey,
>
> Yeah this is an issue with beanstalk clients like mine, and beanstalkd
> 1.0. I was trying to implement a polling check that could pull the
> next ready job from a pool, but peek() didn't turn out to work as I
> initially expected.
>
> Avoid the peek modes for now. Stick to 'sequential wait' and 'random
> wait' select values.
>
> I will release v1.1 of my client shortly, and then the peek modes will
> work again, as expected.
>
>
>
> On Fri, Nov 7, 2008 at 4:11 PM, avip <avital.pek...@gmail.com> wrote:
>
> > Hi,
>
> > Wow, that was a fast turn-around! I thought it would take days :)
> > Thanks for both explanations, now that I read them (and after poking
> > in the code for a while), it seems pretty "obviously necessary" to
> > tell the function which server to perform the peek on given there's a
> > pool.
>

> > I've been trying some very basicPHPscripts to learn the internals, I

[avip]

Hi again Tim,

Running into some basic problems: I have a script that loads the queue
with some jobs using put(), this works fine and is verified manually
via telnet. In a different script, I attempt to reserve a job using:

<code>
require('bin/Others/BeanStalk.class.php');

$beanstalk = BeanStalk::open(array('servers' =>

array('127.0.0.1:11300'),'select' => 'random wait'));

$beanstalk->use_tube('test2');

$job = $beanstalk->reserve();
echo "job=".print_r($job,true);

BeanStalk::delete($job);
</code>

Almost straight out of the example, only difference is that put() and
reserve() are spread out across two files. I've tried every select
mode as well with similar results.

BeanStalk::__call(use_tube)
BeanStalk::_do_my_server
BeanStalk::reserve
BeanQueue::reserve
Fatal error: Maximum execution time of 30 seconds exceeded in /bin/
Others/BeanStalk.class.php on line 989

Since I added some echos to original, my line 989 is: $data =
@fread($this->socket,$in_buf_size);

Is this related to the problems from yesterday? Also, you mentioned
v1.1 will be released soon, could you be more specific? (weeks?
months?) I'm just trying to figure out whether or not I should try the
python or other libs for now.

Thanks!
--Avi

[avip]

Hi yet again!

Scratch the last post. I spent more time experimenting with a telnet
session and reading the protocol itself more carefully.

For any other people who are completely new to beanstalkd and are like
me - wanting to go from installing server to writing code - take note:
the "use <tube>" command is *just* for producers. Attempting to
"switch tubes" using "use <tube>" and then reserve will not work and
actually block if the default tube is empty, not to mention confuse
you like it did me. The command you're actually looking for when you
wish to *consume* jobs is "watch <tube>"

Very important-and much farther down the protocol doc-is the
following:

The "watch" command adds the named tube to the watch list for the
current
connection. A reserve command will take a job from any of the tubes in
the
watch list. For each new connection, the watch list initially consists
of one
tube, named "default".


Take my code from above as example, seems like it should work because
it's almost straight from the example. My first mistake was assuming
that. I ran the example first, it worked fine without any error, but
running it several times in a row after my own code failed, I noticed
it was erratically timing out. When it wasn't timing out, it behaves
the way I described in the post prior to this one, just silently
returns with no actual action taking place.

Both the example or the example code above will timeout not because of
any PHP-specific error but because doing "use_tube('test2')" doesn't
actually *switch* tubes in order for you to consume from that job-
queue - that is managed by the daemon for you provided you tell it you
wish to *watch* that tube. Not the most obvious thing to realise upon
initial inspection!

So, in the above piece of code I pasted, reserve() is actually trying
to ask for a job from the tube "default" (created automatically),
since it is empty (because in a previous piece of code I, correctly,
used "use_tube('test2')" to add jobs), the function just blocks...in
other words, the would-be consumer waits for a job; however, PHP times
out the script based on its own timer.

If you wish to consume jobs using PHP scripts in the background
elsewhere without this time limit simply add "set_time_limit(0)" to
the top of the relevant PHP script to allow for infinite time - this
will allow your consumer scripts to run in the background indefinitely
until there is a job to be done.

Anyway, using "watch" fixes the problem for me, I hope this post saves
someone a few hours of confusion.
Tim, if you could correct me on anything I got wrong... :)

Thanks!
--Avi

On Nov 8, 12:30 pm, avip <avital.pek...@gmail.com> wrote:
> Hi again Tim,
>
> Running into some basic problems: I have a script that loads the queue
> with some jobs using put(), this works fine and is verified manually
> via telnet. In a different script, I attempt to reserve a job using:
>
> <code>
> require('bin/Others/BeanStalk.class.php');
>
> $beanstalk = BeanStalk::open(array('servers' =>
> array('127.0.0.1:11300'),'select' => 'random wait'));
>
> $beanstalk->use_tube('test2');
>
> $job = $beanstalk->reserve();
> echo "job=".print_r($job,true);
>
> BeanStalk::delete($job);
> </code>
>
> Almost straight out of the example, only difference is that put() and
> reserve() are spread out across two files. I've tried every select
> mode as well with similar results.
>
> BeanStalk::__call(use_tube)
> BeanStalk::_do_my_server
> BeanStalk::reserve
> BeanQueue::reserve
> Fatal error: Maximum execution time of 30 seconds exceeded in /bin/

> Others/BeanStalk.class.phpon line 989

[avip]

Hi Tim,

I was wondering if you could post some example consumer/worker code
using your class?

I've managed to daemonize a php worker script but inevitably there's a
polling while(1) loop somewhere attempting to reserve() a job. This
works fine when there are actually jobs on the queue, but for an empty
queue cpu-usage shoots up to 99%.

To test out whether or not this is a task or invocation issue, I tried
replacing the job reservation/worker code with a simple print
statement. The result was a well-behaved background worker. When I
switched back to trying to reserve and delete a job, cpu-usage shot
back up on an empty queue.

Which obviously leads me to suspect there's some other loop somewhere
in the class responsible for reading or blocking or...not sure
what...that's causing this.

If you could shed some light on this?

Thanks very much!
--Avi

> anyPHP-specific error but because doing "use_tube('test2')" doesn't

> actually *switch* tubes in order for you to consume from that job-
> queue - that is managed by the daemon for you provided you tell it you
> wish to *watch* that tube. Not the most obvious thing to realise upon
> initial inspection!
>
> So, in the above piece of code I pasted, reserve() is actually trying
> to ask for a job from the tube "default" (created automatically),
> since it is empty (because in a previous piece of code I, correctly,
> used "use_tube('test2')" to add jobs), the function just blocks...in
> other words, the would-be consumer waits for a job; however,PHPtimes
> out the script based on its own timer.
>

> If you wish to consume jobs usingPHPscripts in the background

> elsewhere without this time limit simply add "set_time_limit(0)" to

> the top of the relevantPHPscript to allow for infinite time - this

> ...
>
> read more »

[Ludovic Levesque]

On Wed, Dec 3, 2008 at 9:49 PM, avip <avital...@gmail.com> wrote:
>
> Hi Tim,
>
> I was wondering if you could post some example consumer/worker code
> using your class?
>
> I've managed to daemonize a php worker script but inevitably there's a
> polling while(1) loop somewhere attempting to reserve() a job. This
> works fine when there are actually jobs on the queue, but for an empty
> queue cpu-usage shoots up to 99%.
>
> To test out whether or not this is a task or invocation issue, I tried
> replacing the job reservation/worker code with a simple print
> statement. The result was a well-behaved background worker. When I
> switched back to trying to reserve and delete a job, cpu-usage shot
> back up on an empty queue.
>
> Which obviously leads me to suspect there's some other loop somewhere
> in the class responsible for reading or blocking or...not sure
> what...that's causing this.

Hi,

I ran into same problems at the beginning. Then I switched to
sequential peek method:

$beanstalk = BeanStalk::open(
array(
'servers' => array(
'192.168.1.40:11300',
),
'select' => 'sequential peek',
'peek_usleep' => 2500, // 2.5 ms
)
);

and no 100% CPU problem now.

Hope it helps,
Ludo

[avip]

Hi Ludo,

That's what I was suspecting, that peek-ready blocks without the aid
of a while loop on the server, which eases the burden on PHP. I'll try
it out tonight, thanks for your input :)

Cheers!
--Avi

On Dec 3, 5:50 pm, "Ludovic Levesque" <lud...@gmail.com> wrote:

> On Wed, Dec 3, 2008 at 9:49 PM, avip <avital.pek...@gmail.com> wrote:
>
> > Hi Tim,
>
> > I was wondering if you could post some example consumer/worker code
> > using your class?
>

> > I've managed to daemonize aphpworker script but inevitably there's a

> ...
>
> read more »

[jsia]

I tired to run the php consumer in the background but it won't work

I tired running it in the php cli
php <scriptsource> &

the script has a while (1) loop , I tired running it php
<scriptsource> not running from the background and it works fine but
when i try to run it in the background it won't work. I also added the
set_time_limit(0) in the php script.

Please Advice

On Dec 4 2008, 7:07 am, avip <avital.pek...@gmail.com> wrote:
> Hi Ludo,
>
> That's what I was suspecting, that peek-ready blocks without the aid
> of a while loop on the server, which eases the burden on PHP. I'll try
> it out tonight, thanks for your input :)
>
> Cheers!
> --Avi
>
> On Dec 3, 5:50 pm, "Ludovic Levesque" <lud...@gmail.com> wrote:
>
> > On Wed, Dec 3, 2008 at 9:49 PM, avip <avital.pek...@gmail.com> wrote:
>
> > > Hi Tim,
>
> > > I was wondering if you could post some example consumer/worker code
> > > using your class?
>
> > > I've managed to daemonize aphpworker script but inevitably there's a
> > > polling while(1) loop somewhere attempting to reserve() a job. This
> > > works fine when there are actually jobs on the queue, but for an empty
> > > queue cpu-usage shoots up to 99%.
>
> > > To test out whether or not this is a task or invocation issue, I tried
> > > replacing the job reservation/worker code with a simple print

> > > statement. The result was a well-behavedbackgroundworker. When I

> ...
>
> read more »

[avip]

Hi jsia,

If your script is consuming the queue as you expect when you run it
from a web page or console, then great, the only thing you should need
from console is: php -q filename.php &

You could also try "php -q filename.php > beanstalkWorker.log 2>&1 &"
to run in the background and redirect any output from the worker to a
log file.

Note the "-q" flag after the php command in both cases.

Hope this helps!
--Avi


On Jan 1, 1:16 am, jsia <jsi...@gmail.com> wrote:
> I tired to run thephpconsumer in the background but it won't work
>
> I tired running it in thephpcliphp<scriptsource> &

>
> the script has a while (1) loop , I tired running itphp
> <scriptsource> not running from the background and it works fine but
> when i try to run it in the background it won't work. I also added the
> set_time_limit(0) in thephpscript.
>
> Please Advice
>
> On Dec 4 2008, 7:07 am, avip <avital.pek...@gmail.com> wrote:
>
> > Hi Ludo,
>
> > That's what I was suspecting, that peek-ready blocks without the aid

> > of a while loop on the server, which eases the burden onPHP. I'll try

> ...
>
> read more »