How to write a Daemon Process of Linux in golang?

[feeling4t]

Hi gays, I want to write a server program in Linux, and it can run in daemon without shell

does golang has any pkg to implement it?

I new how to write it on C, but I don't know how to implement it on golang, such as signal, PID, fork()

[Dave Cheney]

This is possible, but not easy, currently. This statement may not be
cross platform.

I recommend you write your application to _not_ daemonise and use a
tool like upstart, systemd or daemontools to manage the process
externally.

Cheers

Dave

> --
>
>

[feeling4t]

I know in Unix it may have compatible problem, but in posix linux it will OK

It seems all server program run in daemon in this way, like

httpd

httpd stop

httpd restart

在 2012年9月17日星期一UTC+8下午5时42分04秒，Dave Cheney写道：

[minux]

On Mon, Sep 17, 2012 at 5:49 PM, feeling4t <feel...@gmail.com> wrote:

I know in Unix it may have compatible problem, but in posix linux it will OK

In general daemonize is not possible for Go.

this is issue 227.

the problem is complex interaction between fork(2) and threads, see the

issue tracker for details.

[tomwilde]

For future reference:

"guys" = people in general

"gays" = homosexual people

Better to explain directly and save you a couple of embarrassing moments.

Cheers.

[Kyle Lemons]

Use a shell, start-stop-daemon, cron, etc to start the binary in the background.  I have done backend and daemon programming in Go practically since its public release and have not wanted for fork/daemonize.

--
 
 

[feeling4t]

It's very amazing and strange.

It is said that golang is design for large-scale programs, such as multithreading servers, such as http server or other

But it seems all server in linux can run daemon

在 2012年9月17日星期一UTC+8下午5时54分11秒，minux写道：

[feeling4t]

can you give me more detail information?

Thanks a lot

在 2012年9月18日星期二UTC+8上午12时06分16秒，Kyle Lemons写道：

[David Anderson]

Starting processes in unix systems (notably linux, but Mac OS and
FreeBSD also follow this) is fairly simple: fork() and immediately
exec() in the child process. If you do anything but call exec() in the
child process you risk deadlocking your program due to multithreading
interactions. Having a program daemonize itself requires doing stuff
other than plain exec() in the child process.

As a result, the classroom "fork, setsid, fork again" algorithm is not
applicable if your program is operating in a multithreaded
environment. Unfortunately, Go pretty much always operates in a
multithreaded environment, because modules can define init() and start
goroutines before your main() executes.

The current state of the art in systems like upstart, systemd (linux)
and launchd (Mac OS), is to write your program without daemonizing
features, and assume that the system management software will start
your process already daemonized.

In other words, don't include daemonization into your program;
instead, when you're ready to deploy it, configure the system
management daemon (sysvinit, upstart, systemd, launchd, windows
services) to start your program with the correct daemonization
procedure.

- Dave

> --
>
>

[Sergey Shepelev]

You may be amazed even more. A number of service management tools was created in past years, namely: daemontools, launchd, runit, supervisord, systemd, upstart and quite a few more. They are very different but share one thing: all recommend running service in foreground, NOT as daemon. They have various solutions to work with programs daemon programs for backward compatibility, but recommended way is no daemon.

And all programs in Go are automatically compatible with modern view on running services.

For example, Upstart is being used in Ubuntu 8.04+ and Fedora 14-, so if your server runs that, you don't need daemonization.

Otherwise, programs such as daemonize and start-stop-daemon help to make normal program appear as daemon with pidfile.

http://software.clapper.org/daemonize/

http://man.he.net/man8/start-stop-daemon

http://blog.n01se.net/blog-n01se-net-p-145.html

[feeling4t]

Thank you very much Dave, I'll try a shell

在 2012年9月18日星期二UTC+8下午12时21分50秒，David Anderson写道：

[Rory McGuire]

On Wednesday, 19 September 2012 11:43:31 UTC+2, feeling4t wrote:

Thank you very much Dave, I'll try a shell

for a quick test I think you can use:

nohup somecommand &

I wonder if you could use StartProcess with nil stdin, stdout, stderr and call setsid from the new process?

[Rory McGuire]

On Friday, 21 September 2012 21:39:31 UTC+2, Rory McGuire wrote:

I wonder if you could use StartProcess with nil stdin, stdout, stderr and call setsid from the new process?

Just tried this on jra's upgradable.go and it works in gnome-terminal. jra uses exec.Command to start the new process.

[Iñaki Baz Castillo]

El martes, 18 de septiembre de 2012 06:46:41 UTC+2, Sergey Shepelev escribió:

You may be amazed even more. A number of service management tools was created in past years, namely: daemontools, launchd, runit, supervisord, systemd, upstart and quite a few more. They are very different but share one thing: all recommend running service in foreground, NOT as daemon. They have various solutions to work with programs daemon programs for backward compatibility, but recommended way is no daemon.

And all programs in Go are automatically compatible with modern view on running services.

For example, Upstart is being used in Ubuntu 8.04+ and Fedora 14-, so if your server runs that, you don't need daemonization.

Otherwise, programs such as daemonize and start-stop-daemon help to make normal program appear as daemon with pidfile.

http://software.clapper.org/daemonize/

http://man.he.net/man8/start-stop-daemon

http://blog.n01se.net/blog-n01se-net-p-145.html

Just wonder if somebody has realized that, for example using "daemonize" in the first link above, there is no way for the script to know wheter the daemonized process has ended due to some error (i.e. cannot bind into an already used port) since it cannot check the exit status code of the daemonized process.

I don't want that my init script returns 0 (OK) even when the daemonized process has failed to start.

 

[Iñaki Baz Castillo]

El martes, 12 de noviembre de 2013 13:33:39 UTC+1, Iñaki Baz Castillo escribió:

http://software.clapper.org/daemonize/

http://man.he.net/man8/start-stop-daemon

http://blog.n01se.net/blog-n01se-net-p-145.html

Just wonder if somebody has realized that, for example using "daemonize" in the first link above, there is no way for the script to know wheter the daemonized process has ended due to some error (i.e. cannot bind into an already used port) since it cannot check the exit status code of the daemonized process.

I don't want that my init script returns 0 (OK) even when the daemonized process has failed to start.

Said that, the very same occurs for start-stop-daemon and even for the new Upstart. From  http://upstart.ubuntu.com/cookbook/:

----------

If your daemon has a "don't daemonize" or "run in the foreground" mode, then it's much simpler to use that and not run with fork following. One issue with that though, is that Upstart will emit the started JOB=yourjob event as soon as it has executed your daemon, which may be before it has had time to listen for incoming connections or fully initialize.

----------

So IMHO all those telling that "daemonize mechanism is not required at all just because it is hard to achieve within a Go program" are just wrong. Of course I can do a bash script that waits a few seconds, checks the existence of a PID file and whether there is a running process with that PID value. But the reason for coding a real daemon (double fork and so on) is exactly not to do that fragile stuff.

[Tamás Gulácsi]

I think you arrived to the wrong conclusion: how will you start/stop/reload ... etc your daemonized (Go or any other) program? The easiest is to send a proper signal to it. But to which pid? Pidfile IS fragile, just as you wrote. Not to mention ps -ef|grep ...

The easyest, simplest, most rational solution is to have your program run in the foreground, and have a "runner" which runs it, takes care of logs, stops (restarts, sends email... etc), restarts, log rotation... etc). One such program is daemontools/runit which does just the minimum, but dutifully. More heavyweights are Upstart and systemd.

This also allows you to leave running and monitoring to other programs, and allows the administrator run your program as she whishes.

[Iñaki Baz Castillo]

2013/11/12 Tamás Gulácsi <tgula...@gmail.com>:

> I think you arrived to the wrong conclusion: how will you start/stop/reload
> ... etc your daemonized (Go or any other) program? The easiest is to send a
> proper signal to it. But to which pid? Pidfile IS fragile, just as you
> wrote. Not to mention ps -ef|grep ...
>
> The easyest, simplest, most rational solution is to have your program run in
> the foreground, and have a "runner" which runs it, takes care of logs, stops
> (restarts, sends email... etc), restarts, log rotation... etc). One such
> program is daemontools/runit which does just the minimum, but dutifully.
> More heavyweights are Upstart and systemd.

While I agree with the above I also consider that we have what we
have, and if I want to build a server/daemon, for example for Debian,
and build a DEB package, I want that users can install it and have a
init script that just works as expected. I've been using Debian
services (apache, mysql, SIP proxies, etc etc) which come with a init
script, and have experimented no problems at all. For sure the
mechanism you suggest is much appropriate but that is not what
typically comes with today's *nix distributions, and I would not like
to require users installing my DEB package to configure one of the
tools you suggest above in order to properly run my software as
daemon.

And AFAIK using Upstart for managing a software that just runs in
foreground is not very nice since it seems that it cannot wait to
realize whether the program has correctly started or not (but may be I
should check Upstart a bit more).

Thanks a lot!


--
Iñaki Baz Castillo
<i...@aliax.net>

[Robert Johnstone]

It is becoming more common to use a wrapper even when working with a normal init script.  Add the package daemontools as a dependency to your package, and you can use daemonize inside your init scripts.  Good luck with which ever approach you use.

[Iñaki Baz Castillo]

2013/11/12 Robert Johnstone <r.w.jo...@gmail.com>:

> It is becoming more common to use a wrapper even when working with a normal
> init script. Add the package daemontools as a dependency to your package,
> and you can use daemonize inside your init scripts. Good luck with which
> ever approach you use.

It will take still long time until I deal with this issue, but yes, I
will have to :)

[Lars Seipel]

On Tue, Nov 12, 2013 at 10:26:10PM +0100, Iñaki Baz Castillo wrote:
> It will take still long time until I deal with this issue, but yes, I
> will have to :)

Well, in that case Debian might have switched its init system to one of
Upstart, systemd or OpenRC anyway. ;-)

See:
https://wiki.debian.org/Debate/initsystem

[David Arroyo]

On Nov 12, 2013, at 7:33 AM, Iñaki Baz Castillo <i...@aliax.net> wrote:
> I don't want that my init script returns 0 (OK) even when the daemonized process has failed to start.

Programs that fork themselves into the background can fail after forking as well. I often see this problem with Java programs all the time. We know from CS that a program can never decide if it will stop or not :P

The most common use for this I see in init scripts is failing early if configuration is incorrect. It's not 100% the same, but in Upstart, you can reproduce this behavior with a `pre-start` stanza:

$ cat /etc/init/httpd.conf
pre-start script
apachectl configtest || { stop ; exit; }
end script

exec /usr/sbin/httpd

[Matt Harden]

systemd, at least, can bind the socket(s) for you and pass them to your program at startup. It also offers a mechanism for the program to notify it when startup is complete, either via dbus or unix domain socket.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Iñaki Baz Castillo]

2013/11/13 Matt Harden <matt....@gmail.com>:

> systemd, at least, can bind the socket(s) for you and pass them to your
> program at startup. It also offers a mechanism for the program to notify

> it when startup is complete, either via dbus or unix domain socket.

That's the way! Good to know, thanks.

[Iñaki Baz Castillo]

2013/11/13 David Arroyo <dr...@aqwari.net>:

> Programs that fork themselves into the background can fail after forking as well.

What I do is to communicate the "master" process (afther the double
fork) with the "grandparent" process (the executed one) via a PIPE, so
the grandparent waits a bit in order to receive some data from the
"master" process via the PIPE and then the "grandparent" process
exists with success or error status depending on that data.

[Konstantin Khomoutov]

On Tue, 12 Nov 2013 22:15:22 +0100

Iñaki Baz Castillo <i...@aliax.net> wrote:

[...]

> > The easyest, simplest, most rational solution is to have your
> > program run in the foreground, and have a "runner" which runs it,
> > takes care of logs, stops (restarts, sends email... etc), restarts,
> > log rotation... etc). One such program is daemontools/runit which
> > does just the minimum, but dutifully. More heavyweights are Upstart
> > and systemd.
>
> While I agree with the above I also consider that we have what we
> have, and if I want to build a server/daemon, for example for Debian,
> and build a DEB package, I want that users can install it and have a
> init script that just works as expected. I've been using Debian
> services (apache, mysql, SIP proxies, etc etc) which come with a init
> script, and have experimented no problems at all. For sure the
> mechanism you suggest is much appropriate but that is not what
> typically comes with today's *nix distributions, and I would not like
> to require users installing my DEB package to configure one of the
> tools you suggest above in order to properly run my software as
> daemon.

Write your init script so that it uses daemon [1] to run and control
your Go process, and make your package depend on the daemon package.
This does not introduce a dependency on another init system or
something resembling an init system as this program is just a sort of
proxy/wrapper which "hosts" a normal foreground program while does
appear to the world as a true Unix daemon.

The daemon program also provides streaming what your hosted program
writes to its standard output streams into syslog messages which is
also convenient as the hosted program could be developed without any
thought of being a daemon.

1. http://packages.debian.org/wheezy/daemon

[Joan Llopis]

I've been using supervisord. Now we are migrating to Docker (easy as we run Ubuntu). No need to worry about daemonize. Just build the appropiate image and run.

BTW, still using Upstart to be sure the container will be restarted when appropiate.

[ba...@xaprb.com]

I agree with previous posters in this thread: don't daemonize unless you have to. The only thing I'll add is that if you have to, you can do it via exec instead of fork, with https://github.com/VividCortex/godaemon.

[Iñaki Baz Castillo]

Great.

2013/11/14 <ba...@xaprb.com>:

> I agree with previous posters in this thread: don't daemonize unless you
> have to. The only thing I'll add is that if you have to, you can do it via
> exec instead of fork, with https://github.com/VividCortex/godaemon.
>

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Alexandre Fiori]

Daemons are more than programs that double fork and detach from terminals. Besides that, they usually change their own permission to another user and group (setuid/setgid currently not possible with Go), change the runtime directory, write pid and log files, and recycle the log file on SIGHUP.

Most of those things can be done outside your program (in Go, some have to), and that's why there's an infinity of utilities to do that for you. A while ago I published one that I rolled out for some projects that are packaged as both deb and rpm, and therefore have to be "start and stopped" like other system daemons. I also use it with upstart to run freegeoip.net, with this script: https://github.com/fiorix/freegeoip/blob/master/README.md#running-with-upstart

The utility can be found at https://github.com/fiorix/go-daemon and does not have any external dependencies, it's just a minimal C program to be as less intrusive as possible.

[Jakob Borg]

2013/11/16 Alexandre Fiori <fio...@gmail.com>:

>
> Daemons are more than programs that double fork and detach from terminals.
> Besides that, they usually change their own permission to another user and
> group (setuid/setgid currently not possible with Go),

http://golang.org/pkg/syscall/#Setreuid and friends handle this just fine.

> change the runtime
> directory, write pid and log files, and recycle the log file on SIGHUP.

I would argue that all of this is better handled externally; i.e. the
program in question writes log output to stderr, the runner manages
log files, pid files if necessary, etc.

(You say much the same further down, so I'm not disagreeing with you).

//jb

[Alexandre Fiori]

On Nov 16, 2013, at 7:47 AM, Jakob Borg <ja...@nym.se> wrote:

> 2013/11/16 Alexandre Fiori <fio...@gmail.com>:
>>
>> Daemons are more than programs that double fork and detach from terminals.
>> Besides that, they usually change their own permission to another user and
>> group (setuid/setgid currently not possible with Go),
>
> http://golang.org/pkg/syscall/#Setreuid and friends handle this just fine.

Not really, check this out: https://code.google.com/p/go/issues/detail?id=1435

>
>> change the runtime
>> directory, write pid and log files, and recycle the log file on SIGHUP.
>
> I would argue that all of this is better handled externally; i.e. the
> program in question writes log output to stderr, the runner manages
> log files, pid files if necessary, etc.

That's right.

[Jakob Borg]

2013/11/16 Alexandre Fiori <fio...@gmail.com>:

>
> On Nov 16, 2013, at 7:47 AM, Jakob Borg <ja...@nym.se> wrote:
>
>> 2013/11/16 Alexandre Fiori <fio...@gmail.com>:
>>>
>>> Daemons are more than programs that double fork and detach from terminals.
>>> Besides that, they usually change their own permission to another user and
>>> group (setuid/setgid currently not possible with Go),
>>
>> http://golang.org/pkg/syscall/#Setreuid and friends handle this just fine.
>
> Not really, check this out: https://code.google.com/p/go/issues/detail?id=1435

Heh, that's awesome. Thanks for bringing that to my attention. :)

//jb

[Baron Schwartz]

To be clear, VividCortex's godaemon library does more than double fork and detach from the terminal.

On Sat, Nov 16, 2013 at 6:20 AM, Alexandre Fiori <fio...@gmail.com> wrote:

Daemons are more than programs that double fork and detach from terminals.

[sergey....@gmail.com]

Take a look at repository https://github.com/sevlyar/go-daemon

[Konstantin Khomoutov]

On Mon, 13 Jan 2014 10:04:32 -0800 (PST)
sergey....@gmail.com wrote:

> Take a look at repository https://github.com/sevlyar/go-daemon

Some points:

1) Russian commit messages in a project advertised to a general
community (which naturally uses English for communication
and documentation) is a really strange pick.
Suppose someone wants to fork your project to fix a bug and submit a
pull request -- how will they read the commit log?
It's better to use bad English than good Russian in this case.

2) Reborn() is a bad name because it's an adjective but you need
a verb. It should have been named Rebirth() instead, which
leads us to another idea -- why not just Clone() then?
By the way, Linux has the clone() syscall which AFAIK underlies
fork() and vfork().