"mix deps.get" doesn't seem to work on OpenShift/AWS

[Martin Rauscher]

Hello,

I'm trying to create a custom cartridge for running Elixir/Phoenix on OpenShift (OpenStack by RedHat), but I'm running into trouble:

[diy-sharown.rhcloud.com diy]\> mix deps.get
Registry update failed (http_error)
{:failed_connect, [{:to_address, {'s3.amazonaws.com', 443}}, {:inet, [:inet], :eacces}]}
** (Mix) Failed to fetch registry

Erlang 17.4, Elixir 1.0.2

The script I'm using to setup the runtime Environment Looks like this:

#!/bin/bash
# The logic to start up your application should be put in this
# script. The application will work only if it binds to

DEPS_DIR=${OPENSHIFT_HOMEDIR}app-root/dependencies/

export PATH=$PATH:${DEPS_DIR}erl/usr/lib/erlang/bin:${DEPS_DIR}elixir/bin

export MIX_HOME=${DEPS_DIR}mix_home
mkdir MIX_HOME

export HEX_HOME=${DEPS_DIR}hex_home
mkdir HEX_HOME

export HOST=$OPENSHIFT_DIY_IP
export PORT=$OPENSHIFT_DIY_PORT

# Install Erlang if needed

if ! which erl; then
 cd $DEPS_DIR
 wget http://packages.erlang-solutions.com/site/esl/esl-erlang/FLAVOUR_1_general/esl-erlang_17.4-1~centos~6_amd64.rpm -O /tmp/erlang.rpm
 mkdir erl
 cd erl
 rpm2cpio /tmp/erlang.rpm | cpio -idmv
fi
cp -f $OPENSHIFT_REPO_DIR/install/* ${DEPS_DIR}erl/usr/lib/erlang/bin/

# install Elixir if needed

if ! which iex; then
 wget https://github.com/elixir-lang/elixir/releases/download/v1.0.2/Precompiled.zip  -O /tmp/elixir.zip
 cd $DEPS_DIR
 mkdir elixir
 cd elixir
 unzip /tmp/elixir.zip
fi
mix local.hex --force

cd $OPENSHIFT_REPO_DIR
elixir --version

cd diy
mix deps.get
mix deps.compile
nohup ./runAsProd.sh |& /usr/bin/logshifter -tag diy &

Any ideas on how I can debug this further?

I already tried to compile Erlang from source, but that didn't help either.

- Martin

[Eric Meadows-Jönsson]

I don’t know what the eaccess error means in the context of inet / httpc. It’s usually some kind of permission error, anyone knowledgable about OTP that knows?

Can you try to hit https://s3.amazonaws.com/s3.hex.pm/registry.ets.gz from the machine in the same context you’re running your script to see if that works? Maybe just add wget https://s3.amazonaws.com/s3.hex.pm/registry.ets.gz to your script and check if that command is able to fetch the registry.

​

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Eric Meadows-Jönsson

[Dave Cottlehuber]

> I don’t know what the eaccess error means in the context of inet / httpc.
> It’s usually some kind of permission error, anyone knowledgable about OTP
> that knows?
>
> Can you try to hit https://s3.amazonaws.com/s3.hex.pm/registry.ets.gz from
> the machine in the same context you’re running your script to see if that
> works? Maybe just add wget
> https://s3.amazonaws.com/s3.hex.pm/registry.ets.gz to your script and check
> if that command is able to fetch the registry.

> > I'm trying to create a custom cartridge for running Elixir/Phoenix on
> > OpenShift (OpenStack by RedHat), but I'm running into trouble:
> >
> > [diy-sharown.rhcloud.com diy]\> mix deps.get
> > Registry update failed (http_error)
> > {:failed_connect, [{:to_address, {'s3.amazonaws.com', 443}}, {:inet, [:
> > inet], :eacces}]}

A+, Dave
— sent from my Couch

Martin,

Does any internet access work from within this cartridge? AFAICT eacces should
only appear in sockets when firewall is blocking this, for example.

I spelunked briefly in the OTP source and didn’t come back any wiser, up to
https://github.com/erlang/otp/blob/master/erts/preloaded/src/prim_inet.erl

A+
Dave

[Martin Rauscher]

I could wget the registry file without any issues, so it must be some other problem.

The OpenShift machines have very restricted filesystem access, but I have set MIX_HOME and HEX_HOME to directories to which I can write. But still, maybe it tries to store the download somewhere else temporarily... (/tmp/ is writable.)

[José Valim]

What does:

elixir -e "IO.puts System.tmp_dir"

Return for you?

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Lead Developer

[Martin Rauscher]

I found another way to debug this: strace

$ strace -o trace.txt -f -v mix deps.get
$ grep EACCES --color -C5 trace.txt

457820 setsockopt(7, SOL_SOCKET, SO_PRIORITY, [0], 4) = 0

457820 write(4, "!", 1)                 = 1

457819 <... poll resumed> )             = 1 ([{fd=3, revents=POLLIN|POLLRDNORM}])

457820 bind(7, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("0.0.0.0")}, 16 <unfinished ...>

457819 read(3,  <unfinished ...>

457820 <... bind resumed> )             = -1 EACCES (Permission denied)

457819 <... read resumed> "!", 32)      = 1

457820 futex(0x7f8d63fea428, FUTEX_WAIT_PRIVATE, 2, NULL <unfinished ...>

457819 read(3, 0x7f8d5bc3ea60, 32)      = -1 EAGAIN (Resource temporarily unavailable)

457819 futex(0x7f8d63fea428, FUTEX_WAKE_PRIVATE, 1 <unfinished ...>

457820 <... futex resumed> )            = 0

--

457819 clock_gettime(CLOCK_MONOTONIC, {113128, 430242609}) = 0

457819 read(8, "\0\0\0b\0\0\0\1\4\0\0\0\0016\347@\230\0\0\0\4s3-1.amazon"..., 65536) = 102

457819 socket(PF_INET, SOCK_STREAM, IPPROTO_TCP) = 9

457819 fcntl(9, F_GETFL)                = 0x2 (flags O_RDWR)

457819 fcntl(9, F_SETFL, O_RDWR|O_NONBLOCK) = 0

457819 bind(9, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("0.0.0.0")}, 16) = -1 EACCES (Permission denied)

457819 close(9)                         = 0

457819 futex(0x7f8d63740350, FUTEX_WAKE_PRIVATE, 1) = 1

457815 <... futex resumed> )            = 0

457819 poll([{fd=3, events=POLLIN|POLLRDNORM}, {fd=0, events=POLLIN|POLLRDNORM}, {fd=8, events=POLLIN|POLLRDNORM}], 3, 0 <unfinished ...>

457815 stat("/var/lib/openshift/5492b7595973cad763000203/app-root/runtime/dependencies/elixir/bin/../lib/mix/ebin/httpc_response.beam",  <unfinished ...>

Seems like mix tried to BIND a socket...?
This machine only allows binds to happen on specific IPs and Ports.
But I don't get why it binds...

[Martin Rauscher]

"/tmp/", so that seems to be OK.

[Martin Rauscher]

OK, it seems to be a problem with Erlang's httpc module. For some reason it binds the socket whereas wget doesn't seem to do it.

Once I force the module to use the local interface, it works (although of course it doesn't find the remote host on the loopback device...):

Interactive Elixir (1.0.2) - press Ctrl+C to exit (type h() ENTER for help)

iex(1)> :inets.start

:ok

iex(3)> :httpc.request(:get, {'http://www.erlang.org', []}, [timeout: 10000], [])

{:error,

 {:failed_connect,

  [{:to_address, {'www.erlang.org', 80}}, {:inet, [:inet], :eacces}]}}

iex(4)> {:ok, ip }=:inet.parse_address('127.2.103.129')

{:ok, {127, 2, 103, 129}}

iex(5)> :httpc.set_options([ip: ip])

:ok

iex(7)> :httpc.request(:get, {'http://www.erlang.org', []}, [timeout: 10000], [])

{:error,

 {:failed_connect,

  [{:to_address, {'www.erlang.org', 80}},

   {:inet, [:inet, {:ip, {127, 2, 103, 129}}], :timeout}]}}

[diy-sharown.rhcloud.com 5492b7595973cad763000203]\> curl -s http://www.telize.com/ip

54.211.71.173

iex(1)>  :inets.start

:ok

iex(2)> {:ok, ip }=:inet.parse_address('54.211.71.173')

{:ok, {54, 211, 71, 173}}

iex(3)> :httpc.request(:get, {'http://www.erlang.org', []}, [timeout: 10000], [])

{:error,

 {:failed_connect,

  [{:to_address, {'www.erlang.org', 80}}, {:inet, [:inet], :eacces}]}}

But now I really have no idea how to fix this... :(