Way to send request to Specific Server?

[Aaron Raddon]

I am investigating use of Gearman in some system administration type
functions, when I add a new node to a cluster of servers (for this
example adding a new memcached node), I want to be able to perform
some tasks such as updating configuration for my web apps to be able
to utilize it. I thought Gearman is close by providing cross-language
support, status, queuing, etc but I need the ability to perform a
method on Each machine instead of only One machine. Is there a way
to do this? Or would I need to have different methods/functions on
each server and my own catalog keeping track of all servers?
Gearman mush have some internals that keep track of all servers, and
all servers that offer a function so it seems I would be able to build
on top of that.

Thanks

Aaron

[Pedro Melo]

Hi,

You can do this by having a worker on each server register a function
named after the hostname or IP of the primary interface, and then
submit a job with that name.

You can use the admin interface to listen all the queues, so this
provides you with a poor-man catalog.

But I would recommend that you look at other systems to configure your
services. Puppet comes to mind.

Bye.

[Eric Day]

Hi Aaron,

You might want to check out how others have done similar things:

http://datacharmer.blogspot.com/2009/06/remote-replication-setup-with-gearman.html

http://blogs.tokutek.com/tokuview/using_gearman_for_nightly_build_and_test/

You con have each host register a unique function, and then possibly
another worker that keeps a list of those workers that are registered
so you can query to see what machines are available. Or you can grab
the list from the job server, a few different ways to go about it.

-Eric

[bergbrains]

I introduced my manager to Gearman and he asked me to set it up to
send the same command to a group of servers. For this application
each of a group of hosts processes the logs on its local disks and
returns the output. Currently, it's done by ssh'ing to one host which
uses Parallel::ForkManager to ssh out to the group of hosts to run the
local log processor. The outputs get combined and returned.

So, as a first pass I wrote a light wrapper for client that allows me
to pass a list of hosts to $taskset->add_task in addition to the other
standard parameters. A job with the specified job name is submitted
for each host; but each with a job name with "_$host_name" appended to
it. My workers submit their jobs with the job name and "_$host_name"
appended to it. as in:

$worker->register_function( $job_name . '_' . $host_name, {...} );

So, multiple similar jobs are submitted that will only be run on
specific hosts.

Another application is one that we use to run a job on each of our
hosts that collects stats about the box. Submitting one job with a
list of all of our hosts would do the trick. SSHing all over the
place seems messy these days.

What do you guys think about this? Are there tool that are better
suited to this? I already plan to run Gearman in a more conventional
way for other things.

This was just a proof of concept. I expect I'll get started on this
in earnest soon, but am starting to think about monitoring and
reporting as well.

Eric

[Giuseppe Maxia]

I did something similar to install different applications on remote
servers. I solved the problem by installing one Gearman server on each
host and by giving knowledge of these servers to the client, which was
able to invoke the wanted tasks accordingly.

http://datacharmer.blogspot.com/2009/06/remote-replication-setup-with-gearman.html

If you don't want to burden yourself with this overabundance of
servers, you can set the workers with different function names. For
example, workers on server 1 will register my_first_task_1 and
my_second_task_1, workers on server 2 will register my_first_task_2
and my_second_task_2.

Giuseppe

--
The Data Charmer
http://datacharmer.org/