Best option for running on port 80

[Simon Smithies]

Next problem ... one day I'll get this site live ...

I've read http://zotonic.com/documentation/871/running-on-port-80-and-port-443
and tried both the authbind and setcap approaches.

authbind doesn't seem to work, I think because zotonic's binding to
ipv6 ips. I read somewhere that setting listen_ip to a specific ip in
zotonic config will solve that problem, but I don't know how to
determine what ip that should be.

In trying (and failing) to answer that question, I found a
recommendation to use setcap rather than authbind (in
http://groups.google.com/group/zotonic-users/browse_thread/thread/90c854bc1f11f95d,
Alain O'Dea) and am now trying that instead. I'm running zotonic on a
VPS, so at a hardware level I'm sharing the machine, but in reality at
the vm level where I think it matters, I hope I'm not.

However the problem I get with setcap is when I try and run these
commands:
$ setcap 'cap_net_bind_service=+ep' /usr/lib/erlang/erts-5.8.4/bin/
beam
$ setcap 'cap_net_bind_service=+ep' /usr/lib/erlang/erts-5.8.4/bin/
beam.smp
I get this response:
unable to set CAP_SETFCAP effective capability: Operation not
permitted

So, two questions:
1. What is the recommended approach for running zotonic on standard
ports on a VPS?
2. If authbind, then how to determine what IP to listen on instead of
'any'; if setcap, what to do about the Operation not permitted error?

Thanks!

- Simon

[Scott Finnie]

Hi Simon

I have my site running on port 80 on a vps using authbind.

I had similar symptoms to you and the problem was indeed that zotonic was trying to bind to ipv6 ports.

There's a paragraph in the cookbook entry you link to that solves this (setting ZOTONIC_IP, ZOTONIC_PORT etc).  The important thing is these need to be set in the environment you lanch zotonic from.

I posted an example init.d script for this to the forum a while ago.  You might want to try that (sending this from phone & can't find link).  Let me know if you can't find it & want it. I'll send when I get back to laptop.

Hth,
Scott.

[Scott Finnie]

Simon, here's the details.  Works for me on debian & linux mint with zotonic v0.7.  Haven't tried on other configs.

1. Determine which interface your server runs off.  Running ifconfig should give you something like this:
eth0      Link encap:Ethernet  HWaddr 00:16:3e:0c:39:af 
          inet addr:109.109.237.2  Bcast:109.109.237.7  Mask:255.255.255.248
            [...]

lo        Link encap:Local Loopback 
          inet addr:127.0.0.1  Mask:255.0.0.0
            [...]

If you're just testing on the local machine you can use the loopback interface (lo).  If it's for access anywhere else use eth0 (or whatever your equivalent is).

2. Per the cookbook, create ~zotonic/.zotonic_defaults with the following entries.  (Note: if you use another user to start zotonic, put the .zotonic_defaults file in that user's home directory).  Substitute 'eth0' below as required from step 1 above.

export ZOTONIC_PORT=80
export ZOTONIC_PORT_SSL=443
public_interface=eth0
export ZOTONIC_IP=`/sbin/ifconfig $public_interface | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}'`
export ERL="authbind --deep erl"

3. create /etc/init.d/zotonic.  See link below for file contents. Set owner as root:root and permissions as 755

4. Patch bin/zotonic to read .zotonic_defaults (again see link below for modified file)
   ** Note **: the patch was for v0.7.  bin/zotonic may have changed since then so best check.

5. Start zotonic as follows:
     sudo /etc/init.d/zotonic start

Hopefully that should do it, shout if not.

hth

--
Files
-----
Sample .zotonic_defaults, init.d script & patched bin/zotonic on bitbucket here:
https://bitbucket.org/sfinnie/zotonic/src/6a5560adc4f4/scripts/

All work with zotonic v0.7 on debian & mint, not tried elsewhere.

[Simon Smithies]

Hi Scott,

Thanks for this. Still problems - though I feel like I'm getting closer.

Zotonic now doesn't start. The debug trace begins well, then goes downhill:

$ sudo /etc/init.d/zotonic debug

Calling zotonic debug:Reading config from /home/zotonic/.zotonic_defaults

ZOTONIC_PORT=80

ZOTONIC_PORT_SSL=443

ZOTONIC_IP=127.0.0.1

Erlang (BEAM) emulator version 5.6.3 [source] [smp:4] [async-threads:0] [kernel-poll:true]

Eshell V5.6.3  (abort with ^G)

(zoton...@vps200.lnx.vps.isx.net.nz)1> ** /home/zotonic/zotonic-0.8/zotonic/deps/webzmachine/ebin/webmachine_deps.beam hides /home/zotonic/zotonic-0.8/zotonic/deps/webmachine/ebin/webmachine_deps.beam

** /home/zotonic/zotonic-0.8/zotonic/deps/webzmachine/ebin/webmachine_request.beam hides /home/zotonic/zotonic-0.8/zotonic/deps/webmachine/ebin/webmachine_request.beam

** /home/zotonic/zotonic-0.8/zotonic/deps/webzmachine/ebin/webmachine_mochiweb.beam hides /home/zotonic/zotonic-0.8/zotonic/deps/webmachine/ebin/webmachine_mochiweb.beam

<heaps of this sort of thing snipped out, then ... >

** Found 17 name clashes in code paths 

{"init terminating in do_boot",{{badmatch,{error,{bad_return,{{zotonic_app,start,[normal,[]]},{'EXIT',{{case_clause,{error,{"no such file or directory","public_key.app"}}},[{zotonic_app,ensure_started,1},{zotonic_app,start,2},{application_master,start_it_old,4}]}}}}}},[{zotonic,start,1},{init,start_it,1},{init,start_em,1}]}}

Crash dump was written to: erl_crash.dump

init terminating in do_boot ()

Something about starting in this way is preventing public_key.app from being found. I don't see this when starting normally.

My Erlang purports to be R14B04:

$ erl

Erlang R14B04 (erts-5.8.5) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

But my erlang directory only contains erts-5.6.3: 

$ ls /usr/lib/erlang

bin  erts-5.6.3  lib  man  releases  usr

5.6.3 is referenced at the start of the trace.

Found this when following the setcap instructions in the cookbook. Is this normal? Seems wrong to me ... so I guess my next step is to try and repair my erlang install. However given it works fine with zotonic running on port 8000, am reluctant to do that until I'm sure erlang is a problem.

Finally, .zotonic_defaults results in ZOTONIC_IP=127.0.0.1.  To access from the internet, I think it needs to be the machine's public IP address. Can we override that in zotonic/priv/config (once it's been successfully created)?

- Simon

[Andreas Stenius]

Get rid of old deps/webmachine (it was renamed to webzmachine to
better reflect that it is an adapted version for zotonic).

//Andreas

2012/2/23 Simon Smithies <simon.s...@gmail.com>:

[Simon Smithies]

Well that got rid of all those "hides" errors. I still have the
public_key.app error though. Complete trace is now:

$ sudo /etc/init.d/zotonic debug
Calling zotonic debug:Reading config from /home/
zotonic/.zotonic_defaults
ZOTONIC_PORT=80
ZOTONIC_PORT_SSL=443
ZOTONIC_IP=127.0.0.1
Erlang (BEAM) emulator version 5.6.3 [source] [smp:4] [async-threads:
0] [kernel-poll:true]

Eshell V5.6.3 (abort with ^G)

(zoton...@vps200.lnx.vps.isx.net.nz)1> ** Found 0 name clashes in
code paths
** Found 0 name clashes in code paths

{"init terminating in do_boot",{{badmatch,{error,{bad_return,
{{zotonic_app,start,[normal,[]]},{'EXIT',{{case_clause,{error,{"no
such file or directory","public_key.app"}}},
[{zotonic_app,ensure_started,1},{zotonic_app,start,2},
{application_master,start_it_old,4}]}}}}}},[{zotonic,start,1},
{init,start_it,1},{init,start_em,1}]}}

Crash dump was written to: erl_crash.dump
init terminating in do_boot ()

- Simon

On Feb 24, 11:10 am, Andreas Stenius <andreas.sten...@astekk.se>
wrote:

> Get rid of old deps/webmachine (it was renamed to webzmachine to
> better reflect that it is an adapted version for zotonic).
>
> //Andreas
>

> 2012/2/23 Simon Smithies <simon.smith...@gmail.com>:

>
>
>
>
>
>
>
> > Hi Scott,
>
> > Thanks for this. Still problems - though I feel like I'm getting closer.
>
> > Zotonic now doesn't start. The debug trace begins well, then goes downhill:
>
> > $ sudo /etc/init.d/zotonic debug
> > Calling zotonic debug:Reading config from /home/zotonic/.zotonic_defaults
> > ZOTONIC_PORT=80
> > ZOTONIC_PORT_SSL=443
> > ZOTONIC_IP=127.0.0.1
> > Erlang (BEAM) emulator version 5.6.3 [source] [smp:4] [async-threads:0]
> > [kernel-poll:true]
>
> > Eshell V5.6.3  (abort with ^G)

> > (zotonic...@vps200.lnx.vps.isx.net.nz)1> **
> > /home/zotonic/zotonic-0.8/zotonic/deps/webzmachine/ebin/webmachine_deps.bea m
> > hides
> > /home/zotonic/zotonic-0.8/zotonic/deps/webmachine/ebin/webmachine_deps.beam
> > **
> > /home/zotonic/zotonic-0.8/zotonic/deps/webzmachine/ebin/webmachine_request. beam
> > hides
> > /home/zotonic/zotonic-0.8/zotonic/deps/webmachine/ebin/webmachine_request.b eam
> > **
> > /home/zotonic/zotonic-0.8/zotonic/deps/webzmachine/ebin/webmachine_mochiweb .beam
> > hides
> > /home/zotonic/zotonic-0.8/zotonic/deps/webmachine/ebin/webmachine_mochiweb. beam

>
> > <heaps of this sort of thing snipped out, then ... >
>
> > ** Found 17 name clashes in code paths
> > {"init terminating in

> > do_boot",{{badmatch,{error,{bad_return,{{zotonic_app,start,[normal,[]]},{'E XIT',{{case_clause,{error,{"no
> > such file or
> > directory","public_key.app"}}},[{zotonic_app,ensure_started,1},{zotonic_app ,start,2},{application_master,start_it_old,4}]}}}}}},[{zotonic,start,1},{in it,start_it,1},{init,start_em,1}]}}

>
> > Crash dump was written to: erl_crash.dump
> > init terminating in do_boot ()
>
> > Something about starting in this way is preventing public_key.app from being
> > found. I don't see this when starting normally.
>
> > My Erlang purports to be R14B04:
> > $ erl
> > Erlang R14B04 (erts-5.8.5) [source] [smp:4:4] [rq:4] [async-threads:0]
> > [hipe] [kernel-poll:false]
>
> > But my erlang directory only contains erts-5.6.3:
> > $ ls /usr/lib/erlang
> > bin  erts-5.6.3  lib  man  releases  usr
>
> > 5.6.3 is referenced at the start of the trace.
>
> > Found this when following the setcap instructions in the cookbook. Is this
> > normal? Seems wrong to me ... so I guess my next step is to try and repair
> > my erlang install. However given it works fine with zotonic running on port
> > 8000, am reluctant to do that until I'm sure erlang is a problem.
>
> > Finally, .zotonic_defaults results in ZOTONIC_IP=127.0.0.1.  To access from
> > the internet, I think it needs to be the machine's public IP address. Can we
> > override that in zotonic/priv/config (once it's been successfully created)?
>
> > - Simon
>

> > On Fri, Feb 24, 2012 at 4:48 AM, Scott Finnie <scott.fin...@gmail.com>

> >> On Feb 23, 2012 9:01 AM, "Simon Smithies" <simon.smith...@gmail.com>

> >> wrote:
>
> >>> Next problem ... one day I'll get this site live ...
>
> >>> I've read
> >>>http://zotonic.com/documentation/871/running-on-port-80-and-port-443
> >>> and tried both the authbind and setcap approaches.
>
> >>> authbind doesn't seem to work, I think because zotonic's binding to
> >>> ipv6 ips. I read somewhere that setting listen_ip to a specific ip in
> >>> zotonic config will solve that problem, but I don't know how to
> >>> determine what ip that should be.
>
> >>> In trying (and failing) to answer that question, I found a
> >>> recommendation to use setcap rather than authbind (in
>

> >>>http://groups.google.com/group/zotonic-users/browse_thread/thread/90c...,

[Scott Finnie]

Hi Simon, sorry to hear you're still having problems.  The app error I'll have to leave to Andreas etc. I'm afraid, don't know about that one.

Re. IP address though: easiest fix is to set it manually in the .zotonic_defaults file:

ZOTONIC_IP=<your IP>

I.e. replace the current shell command. Means you'd need to change manually too if server ip changes but since it's a public server i assume the ip will be static.

It's strange the script is returning the loopback address though. If you run ifconfig does it definitely show another address?

- Scott.

[Simon Smithies]

I see what's going on with the IP address - I used the wrong for interface name in .zotonic_defaults (venet0). I've changed it to use venet0:0 and now the script gives the right IP.

That leaves the public_key.app error to sort out.

- Simon

[Scott Finnie]

Ok good, one problem out of the way.

Don't really have a clue about the public_key problem so this is speculation...

Did you just copy over bin/zotonic from the link i sent you? I don't know if the base script has changed since 0.7. If so that could (?) be a source of errors. In the absence of something more meaningful (and assuming you haven't done it already) you might want to try:

1. Patching the 'read preferences' code into the 0.8 version of bin/zotonic
2. Trying to start by directly running bin/zotonic from the command line for the zotonic user.

The former should remove anything arising from changes in the script. The latter should remove any issues arising from different user environments (root for init.d/zotonic, zotonic for bin/zotonic).  I wonder if the latter is the source of the '2 erlang versions' problem you mention.

Bit of a 'blunt weapon' suggestion I know, sorry...

- S.

[Andreas Stenius]

I think Simon might be asleep at the moment, so I'll fill in for him (for now).

We managed to resolve his remaining issues during the day today.

There were a number of things. First, he had two different versions of
erlang installed. One of which seems to have been a broken install.
One was found when he ran erl from the command line, another when
invoking the zotonic script (I guess he used sudo while doing that,
otherwise I don't see why that would be the case).

The other was that he had apache (or something) already running on port 80.

I'll let simon fill in the rest, if he wants.

//Andreas

2012/2/24 Scott Finnie <scott....@gmail.com>:

[Simon Smithies]

Thanks for filling in Andreas ... had to have that sleep!

Scott, I had patched bin/zotonic rather than copying it over, so that wasn't the problem. 

I think you're right, my two versions of erlang were the issue (apart from still having apache on port 80!). Prior to setting up authbind, I was running zotonic as the zotonic user, simply with bin/zotonic, and everything did go fine. 

Hey, a couple of low priority questions, for understanding as much as anything:

I've just tried '/etc/init.d/zotonic start' as zotonic without sudo. That seems to workfine. Do you see any issues with that?

Also, what purpose does '/etc/init.d/zotonic' serve? ... why not just use 'bin/zotonic start' -- that seems to work too. 

Finally, a big thank you! Great authbind instructions Scott -- putting aside my environmental issues, they worked perfectly. And thanks again to Andreas and Arjan for getting me through those environmental issues.

- Simon

[Scott Finnie]

Hi Simon, glad to hear you're up & running (and thanks for update Andreas).

re. your questions:

On 24/02/2012 22:04, Simon Smithies wrote:

Hey, a couple of low priority questions, for understanding as much as anything:

I've just tried '/etc/init.d/zotonic start' as zotonic without sudo. That seems to workfine. Do you see any issues with that?

Can't think of any issues.  If there were, I would've expected them to be permissions-related - so preventing it from working.  So if it works fine that's fine.

Also, what purpose does '/etc/init.d/zotonic' serve? ... why not just use 'bin/zotonic start' -- that seems to work too.

It allows zotonic to start automatically when the server boots - rather than having to manually start.  If you're just running from the command line, then using bin/zotonic directly is fine.

Finally, a big thank you! Great authbind instructions Scott -- putting aside my environmental issues, they worked perfectly.

You're welcome - thanks for saying thanks :-)

 - Scott.