[ANN] beanstalkd 1.5

[Keith Rarick]

This is beanstalkd version 1.5. A lot of changes went into this release,
including some big improvements.

As always, there will be no incompatible protocol changes until version 2.0. A
client written for version 1.5 will work unmodified with any later 1.x
release of beanstalkd.

Highlights
----------

- removed dependency on libevent
- no more autotools; to build beanstalkd, just type "make"
- incremental binlog compaction; fixes #43
- removed flag -d; alternatives are in the "adm" directory

More News
---------

- switch to MIT license
- remove DEBUG compile flag
- more interesting make examples in README
- change default CFLAGS
- verbose mode; closes #52
- properly timeout reserve-with-timeout; fixes #78
- document that EPOLLRDHUP was introduced in Linux 2.6.17.
- read beanstalkd 1.4.6's log format for compatibility
- report size of attempted read for WAL read errors
- accurately report position of WAL file errors
- add file index to job stats
- clarify behavior of cumulative stats; closes #81
- adding support for counting deletes on a tube
- delete command now deletes delayed jobs
- fix segfault; closes #71
- bsd build fix; PATH_MAX is defined in limits.h
- bsd build fix; echo -n is not portable
- do not read in STATE_WAIT; fixes #22
- document the new binlog stats
- redo the WAL code, improving style
- more careful binlog read checking
- in log recovery, recompute delay with current time; fixes #62
- provide and document more system admin tools
- support systemd socket activation

Full list of changes in this release (includes authorship information):
<http://github.com/kr/beanstalkd/compare/v1.4.6...v1.5>

Our Urls
--------

Download the 1.5 tarball directly:
<https://github.com/downloads/kr/beanstalkd/beanstalkd-1.5.tar.gz>

Learn all about beanstalk:
<http://kr.github.com/beanstalkd/>

Talk about beanstalk development or use at:
<http://groups.google.com/group/beanstalk-talk>

Bugs
----

Please report any bugs to:
<http://github.com/kr/beanstalkd/issues>

kr

[Keith Rarick]

Please note that the format of log files produced by
flag -b (the “binlog”) has changed. Beanstalkd 1.5
can read older files, but earlier versions can’t read
files produced by beanstalkd 1.5 or later, so use
caution when migrating.

kr

[Peter Kieltyka]

Awesome work man! But, can you please put back the -d option? This is
still convenient at times.

[Keith Rarick]

On Thu, Feb 2, 2012 at 3:36 PM, Peter Kieltyka
<peter.k...@nulayer.com> wrote:
> Awesome work man! But, can you please put back the -d option? This is
> still convenient at times.

Can you give an example of what you're trying to do?
That'll help us look for a better alternative.

kr

[Fred Liangjing Wang]

I think many users enjoy use this option and some other tools also used -d option to start a daemon.

> --
> You received this message because you are subscribed to the Google Groups "beanstalk-talk" group.
> To post to this group, send email to beansta...@googlegroups.com.
> To unsubscribe from this group, send email to beanstalk-tal...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/beanstalk-talk?hl=en.
>

[Chad Kouse]

I agree it's handy. I run dedicated beanstalkd servers so I can see
the value. Ops team would appreciate only having one thing to monitor.
If its a big issue though it's no problem. We will make it work.
Thanks for the great work.

--chad

[Keith Rarick]

On Sat, Feb 4, 2012 at 6:17 PM, Fred Liangjing Wang <i...@1e20.com> wrote:
> I think many users enjoy use this option and some other tools also
> used -d option to start a daemon.

This has been discussed here before.

For development purposes, one can run beanstalkd in the foreground
or use the shell's "&" notation to run in the background. It's just as
easy to type "beanstalkd &" as it is to type "beanstalkd -d".

For production deployment, popular monitoring tools like launchd,
systemd, upstart, supervisord, god, monit, runit, daemontools, etc
all work *more reliably* without a flag such as -d in beanstalkd. There
are a few examples of how to properly run beanstalkd in production in
https://github.com/kr/beanstalkd/tree/master/adm (and I'd really love
contributions for other monitoring tools that aren't in that dir yet).

Please read http://dustin.github.com/2010/02/28/running-processes.html
for a more thorough discussion of how to monitor processes in
production. This comes up often enough, maybe I should think
about writing a blog post to discuss this specifically in the context
of beanstalkd.

kr

[Keith Rarick]

On Sat, Feb 4, 2012 at 6:56 PM, Chad Kouse <chad....@gmail.com> wrote:
> I agree it's handy. I run dedicated beanstalkd servers so I can see
> the value. Ops team would appreciate only having one thing to monitor.
> If its a big issue though it's no problem. We will make it work.
> Thanks for the great work.

Thanks!

It really shouldn't be any more difficult to run beanstalkd without the
-d flag. I've tried to make switching more straightforward by including
several examples of configuration for production process-monitoring
tools. If there's anything else I can do to make life easier, I'd love to
hear about it.

Ops teams in particular should be pleased that -d is no longer there
to confuse the issue. It's easier to monitor a process that doesn't
daemonize itself.

kr

[Peter Kieltyka]

Hey Keith,

What you're saying makes sense. It's better practice to let the OS's
init/launch mechanisms daemonize a process. Not to keep beating a dead
horse.. -d is convenient for quick deploys, without having to write
your own init script. Each variant of the OS has their own nuances and
it's annoying to have to fiddle with those scripts to boot the daemon.
beanstalkd & isn't a good option because you'll have to do some extra
work to handle the HUP on exit of the shell... something like
beanstalkd & disown, but depends on the OS.

It's really just a convenience thing.. and keeping -d doesn't take
away from sysops who want the OS to daemonize/manage the process.
Anyways, I'm done now :) whatever you decide is cool... beanstalkd is
awesome, good work on 1.5, its solid.

Peter

On Feb 5, 1:12 am, Keith Rarick <k...@xph.us> wrote:

[Thomas Parrott]

I too agree with using a proper process manager, however it would be
really useful to have the -d flag back, as everything else in our
stack daemonizes itself (apache, nginx, mysql, puppet, redis etc) and
moving to a process manager is more of a separate project (not one I
have time to look at atm). This leaves me with having to use & at the
end of the command line and this looks messy and does not play nicely
with Redhat init scripts.

[Thomas Parrott]

As an aside, the example init script in the beanstalkd repo uses the -
d flag.

Maybe I am missing some feature in the Redhat init scripts that allows
daemonising a process that doesn't support it.

On Feb 13, 9:14 am, Thomas Parrott <thomas.parrott%infinity-

[Thomas Parrott]

I opened issue 100 for this:

https://github.com/kr/beanstalkd/issues/100

On Feb 13, 9:16 am, Thomas Parrott <thomas.parrott%infinity-

[Keith Rarick]

>> As an aside, the example init script in the beanstalkd repo uses the
>> -d flag.
>

> I opened issue 100 for this:

Yes, that's an error. Thanks.

kr