libpq configuration on ubuntu

[Brian Cassidy]

Hey All,

I'm having trouble getting gearmand configured to use libpq on ubuntu.
FWIW, things worked as described when i ran gearmand from the command-
line. However, once I tried to use the /etc/default/gearman-job-server
file to specify options i got an error.

My config looks like this:

PARAMS="-q libpq --libpq-conninfo=\"dbname=gearman user=gearman\""

I get this error:

gearman_conf_parse_args:Unknown option: user=gearman"

Seems like it balks on the space. I've worked around this by
specifying these options directly in the init.d script --but I'd much
rather keep things tidy.

Whilst I'm at it, I was wondering if this is a valid bug:

https://bugs.launchpad.net/gearmand/+bug/510297

I suppose I could just modify the schema manually to work around it?

Thanks in advance,

--Brian Cassidy

[Tim Uckun]

>
> PARAMS="-q libpq --libpq-conninfo=\"dbname=gearman user=gearman\""
>
> I get this error:
>
> gearman_conf_parse_args:Unknown option: user=gearman"
>

This works for me.

--libpq-conninfo='user=blah dbname=blah_db host=host.com port=5432'

[Brian Cassidy]

Tim,

On Wed, Nov 17, 2010 at 3:52 PM, Tim Uckun <timu...@gmail.com> wrote:
> This works for me.
>
> --libpq-conninfo='user=blah  dbname=blah_db host=host.com port=5432'

Thanks for your reply, however, trying the following:

PARAMS="-q libpq --libpq-conninfo='user=gearman dbname=gearman
host=localhost port=5432'"

gives me:

gearman_conf_parse_args:Unknown option: dbname=gearman

I should mention that this is gearman 0.12 from the launchpad ppa.

--
Brian Cassidy (brian....@gmail.com)

[Tim Uckun]

Can you post your entire script?

[Brian Cassidy]

Tim,

On Wed, Nov 17, 2010 at 4:09 PM, Tim Uckun <timu...@gmail.com> wrote:
> Can you post your entire script?

Seriously, that's literally everything. the
/etc/default/gearman-job-server file gets slurped in by the init.d
script.

Here's what the daemon launch code of the init.d script looks like
(unmodified from the version that ships with the ppa deb):

start()
{
log_daemon_msg "Starting Gearman Server" "gearmand"
if ! test -d ${PIDDIR}
then
mkdir ${PIDDIR}
chown ${GEARMANUSER} ${PIDDIR}
fi
if start-stop-daemon \
--start \
--exec $DAEMON \
-- --pid-file=$PIDFILE \
--user=$GEARMANUSER \
--daemon \
--log-file=/var/log/gearman-job-server/gearman.log \
$PARAMS
then
log_end_msg 0
else
log_end_msg 1
log_warning_msg "Please take a look at the syslog"
exit 1
fi
}

For completeness, the /etc/default/gearman-job-server file:

# This is a configuration file for /etc/init.d/gearman-job-server; it allows
# you to perform common modifications to the behavior of the gearman-job-server
# daemon startup without editing the init script (and thus getting prompted by
# dpkg on upgrades). We all love dpkg prompts.

# Examples ( from http://gearman.org/index.php?id=manual:job_server )
#
# Use drizzle as persistent queue store
# PARAMS="-q libdrizzle --libdrizzle-db=some_db
--libdrizzle-table=gearman_queue"
#
# Use mysql as persistent queue store
# PARAMS="-q libdrizzle --libdrizzle-host=10.0.0.1 --libdrizzle-user=gearman \
# --libdrizzle-password=secret --libdrizzle-db=some_db \
# --libdrizzle-table=gearman_queue --libdrizzle-mysql"
#
# Missing examples for memcache persitent queue store...

# Parameters to pass to dbus.
PARAMS="-q libpq --libpq-conninfo='user=gearman dbname=gearman'"

--
Brian Cassidy (brian....@gmail.com)

[Tim Uckun]

I have a vague memory of having problems with ubuntu startup scripts,
gearman, and libpq. I am willing to bet that if you didn't use the
$PARAMS variable and instead inserted the params individually it would
all work.

Try it.

[Brian Cassidy]

Tim,

As mentioned in my initial message, I am able to shove the options
directly into the init.d script and bypass the "defaults" file
entirely and everything works great (which is what I'm doing now, in
fact). However, the point is, I'd like to use the mechanism in place
for doing custom options rather than dirtying up the init.d script --
if at all possible.

Cheers,

--
Brian Cassidy (brian....@gmail.com)

[Tim Uckun]

> As mentioned in my initial message, I am able to shove the options
> directly into the init.d script and bypass the "defaults" file
> entirely and everything works great (which is what I'm doing now, in
> fact). However, the point is, I'd like to use the mechanism in place
> for doing custom options rather than dirtying up the init.d script --
> if at all possible.

Sorry it's a little late here so I didn't read your first email
carefully enough.

I just looked at my config files and I have all the params in the
init'd script and not the PARAMS variable. Clearly start-stop-daemon
is doing some crazy escaping thing. Have you tried escaping the
spaces?

[Brian Cassidy]

On Wed, Nov 17, 2010 at 5:00 PM, Tim Uckun <timu...@gmail.com> wrote:
> I just looked at my config files and I have all the params in the
> init'd script and not the PARAMS variable.  Clearly start-stop-daemon
> is doing some crazy escaping thing. Have you tried escaping the
> spaces?

I've tried the standard "\ " escaping; no luck (same error as before).
I've even tried to get cheeky with \x20 -- that didn't complain with
an error, but gearman didn't actually load.

Suggestions welcome! :)

--
Brian Cassidy (brian....@gmail.com)

[Matt Harrington]

I was never able to get this to work as expected, so I just used the env variables.  You can prolly put something like this into your config:

export PGDATABASE=gearman
export PGUSER=gearman

PARAMS="-q libpq"

These variables are consumed by libpq and added to your connection string.  I believe the password could also be handled in this fashion to keep it out of the process list.  You can find the list of env variables here:

http://www.postgresql.org/docs/8.4/static/libpq-envars.html

[Brian Cassidy]

Matt,

On Wed, Nov 17, 2010 at 8:58 PM, Matt Harrington <quer...@gmail.com> wrote:
> I was never able to get this to work as expected, so I just used the env
> variables.  You can prolly put something like this into your config:
>
> export PGDATABASE=gearman
> export PGUSER=gearman
>
> PARAMS="-q libpq"

Awesome! This at least lets me leave the init.d script in pristine
order. Thanks.

> These variables are consumed by libpq and added to your connection string.
> I believe the password could also be handled in this fashion to keep it out
> of the process list.  You can find the list of env variables here:
>
> http://www.postgresql.org/docs/8.4/static/libpq-envars.html

I'm actually using a .pgpass file in the /var/lib/gearman/ folder for
that. Seems to work okay.

Thanks again.

--
Brian Cassidy (brian....@gmail.com)

[saltos]

Hi guys someone know how to pass password into PARAMS?

I tried PARAMS=".... --libpq-conninfo=hostaddr=xxx port=5432 dbname=xxx user=xxx password=xxx"

but i got this error:

ERROR [  main ] PQconnectdb: fe_sendauth: no password supplied



Thanks in advance,

salvo

[Jozef Ševčík]

--queue-type Postgres --libpq-conninfo='user=postgres password=SECRET dbname=gearman host=localhost port=5432' --libpq-table=gearman

mind the quotes

--
You received this message because you are subscribed to the Google Groups "Gearman" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gearman+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[saltos]

Into terminal this command works:

gearmand -L 127.0.0.1 --libpq-conninfo 'hostaddr=127.0.0.1 port=5432 dbname=gearman user=XXX password=XXX' --libpq-table=queue123 -q Postgres

But i don't know why in /etc/default/gearman-job-server with the same parameters doesn't work

PARAMS="--libpq-table=queue123 --libpq-conninfo='hostaddr=127.0.0.1 port=5432 dbname=gearman user=XXX password=XXX' -q Postgres"

i have errors like this:

PQconnectdb: invalid connection option "'hostaddr"#012

where am I doing wrong?  quotes?

[Brian Aker]

Hi,

If you get the syntax right for Postgres, would you reply to this thread with the answer?

Thanks,

-Brian

[Nat Makarevitch]

EpicFail is right.

On Debian edit /etc/default/gearman-job-server and add at the end:

export PGHOST=TheHostname
export PGPORT=5432
export PGUSER=gearman
export PGPASSWORD=ThePassword
export PGDATABASE=gearman
PARAMS="--verbose -q libpq --libpq-table=gearmanqueue1 --verbose"

[Brian Aker]

Hi,

Is there anything that needs to be done to setup the gearman user?

Thanks,

-Brian

[Nat Makarevitch]

Hi

Is there anything that needs to be done to setup the gearman user?

Much... First check that the Debian package you use is linked with libpq (in order to have it talk with PostgreSQL ).

It is OK for Sid ( http://packages.debian.org/fr/sid/gearman-job-server ) , but apparently not for Wheezy ( http://packages.debian.org/fr/wheezy/gearman-job-server ). Or is it statically linked?  I doubt so and no time to check.  In any case apply EpicFail's approach (export environment variables in /(etc/default/gearman-job-server ) the check the log. If an ERROR message about "libpq init" appears, you are out of luck.

Then you need driver(s) for the programming language(s) used. This, in turn, implies a recent libgearman. AFAIK all Debian stable packages are too old. In a word: better build it locally then create a package thanks to checkinstall.

To do so (here with SQL3 and PostgreSQL drivers): aptitude install libboost-program-options-dev libboost-dev libevent-dev checkinstall php-pear php5-dev libpq-dev

... if libhostile cannot be compiled:

libhostile/function.c: In function 'print_function_cache_usage':

libhostile/function.c:54: error: 'for' loop initial declarations are only allowed in C99 mode

libhostile/function.c:54: note: use option -std=c99 or -std=gnu99 to compile your code

make[1]: *** [libhostile/libhostile_libhostile_la-function.lo] Error 1

make[1]: *** Waiting for unfinished jobs....

libhostile/poll.c: In function 'poll':

libhostile/poll.c:93: error: 'for' loop initial declarations are only allowed in C99 mode

libhostile/poll.c:93: note: use option -std=c99 or -std=gnu99 to compile your code

make[1]: *** [libhostile/libhostile_libhostile_la-poll.lo] Error 1

make[1]: Leaving directory `/home/guest/tmp/gearmand-1.0.5'

... then invoke configure this way: CFLAGS=-std=gnu99 ./configure

Then invoke make

It may compile this way.  In such a case:

As root invoke checkinstall make install

You then have locally installed it, and also obtained a package

To create a system "gearman" user use adduser

To create a PostgreSQL user log in as the "postgres" system user then (sudo -i   then  su - postgres ) then invoke createuser

Using psql as the "postgres" user authorize the PG "gearman" user to create a database: ALTER USER gearman CREATEDB;

Then as the "gearman" user invoke "createdb gearman"

Using psql as the "postgres" user restore the PG "gearman" user normal rights: ALTER USER gearman NOCREATEDB;

OTH

[Rodney Chupp]

Brian,

I was able to get this to work using a connection uri instead of the connection string. For example:

PARAMS="--queue-type=Postgres --libpq-table=queue123 --libpq-conninfo=postgresql://username:password@host:port/database_name"

-- Rodney