Rancher IO server and agent on same machine

[Peter Br.]

I have started Rancher server and registered one host by starting rancher agent on it. 

I have secured rancher server with OAuth.

Now I want to add the same host rancher server is running on.

If I copy-paste code from Add Host page and start new container with agent, it fails to connect. Logs:

INFO: Attempting to connect to: http://serveraddress:8088/v1

ERROR: http://serveraddress:8088/v1 is not accessible

The problem is that host address is not accessible from container. 

I tried to add --net=host to docker run so that rancher agent can access server address.

But no luck again:

INFO: Attempting to connect to: http://serveraddress:8088/v1

INFO: http://serveraddress:8088/v1 is accessible

Rancher State

Traceback (most recent call last):

  File "./resolve_url.py", line 9, in <module>

    r = requests.get(url)

  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 60, in get

    return request('get', url, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 49, in request

    return session.request(method=method, url=url, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request

    resp = self.send(prep, **send_kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 569, in send

    r = adapter.send(request, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 407, in send

    raise ConnectionError(err, request=request)

requests.exceptions.ConnectionError: ('Connection aborted.', error(110, 'Connection timed out'))

curl: no URL specified!

curl: try 'curl --help' or 'curl --manual' for more information

Traceback (most recent call last):

  File "./register.py", line 8, in <module>

    client = from_env(access_key=os.environ['CATTLE_REGISTRATION_ACCESS_KEY'],

  File "/usr/lib/python2.7/UserDict.py", line 23, in __getitem__

    raise KeyError(key)

KeyError: 'CATTLE_REGISTRATION_ACCESS_KEY'

[Bill Maxwell]

Hello,

 When registering the host that the rancher server is running on you need to set the -e CATTLE_AGENT_IP='<host ip>'.  You just need to add that option to the command line provided by the UI for registration. Adding the the variable will force the container to listen on a specific IP address. You will want it to be accessible from the workstation you are using. You don't need to run --net=host, the rancher-agent startup scripts will launch bound to the right ip.

 We use this script: https://github.com/rancherio/cattle/blob/master/tools/development/register-boot2docker.sh to register boot2docker VMs on our workstations. It is essentially the same process. 

Let us know if this helps.

-Bill

[Peter Br.]

Greetings,

Thanks for your help. 

Without --net=host I can only reach host from container by IP 172.17.42.1.

So I started agent like this:

docker run -d -e CATTLE_AGENT_IP=172.17.42.1 --name rancheragent --privileged -v /var/run/docker.sock:/var/run/docker.sock rancher/agent:v0.5.2 http://172.17.42.1:8088/v1/scripts/<key1>:<key2>:<key3>

In container logs I can see:

INFO: Attempting to connect to: http://172.17.42.1:8088/v1

INFO: http://172.17.42.1:8088/v1 is accessible

Rancher State

Traceback (most recent call last):

  File "./register.py", line 9, in <module>

    secret_key=os.environ['CATTLE_REGISTRATION_SECRET_KEY'])

  File "/usr/local/lib/python2.7/dist-packages/cattle.py", line 34, in from_env

    return gdapi.from_env(prefix=prefix, factory=Client, **kw)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 601, in from_env

    return _from_env(prefix=prefix, factory=factory, **args)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 620, in _from_env

    return factory(**result)

  File "/usr/local/lib/python2.7/dist-packages/cattle.py", line 12, in __init__

    super(Client, self).__init__(*args, **kw)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 194, in __init__

    self._load_schemas()

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 306, in _load_schemas

    response = self._get_response(self._url)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 257, in _get_response

    headers=HEADERS)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 469, in get

    return self.request('GET', url, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 457, in request

    resp = self.send(prep, **send_kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 569, in send

    r = adapter.send(request, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 407, in send

    raise ConnectionError(err, request=request)

requests.exceptions.ConnectionError: ('Connection aborted.', error(110, 'Connection timed out'))

I'm running docker on Ubuntu 12.04.

Docker version is 1.5

If I run curl from rancheragnet container I get the following:

root@f1cb8131ca36:/# curl http://172.17.42.1:8088/v1/scripts/<key1>:<key2>:<key3>

#!/bin/sh

export CATTLE_REGISTRATION_ACCESS_KEY="registrationToken"

export CATTLE_REGISTRATION_SECRET_KEY="<key1>:<key2>:<key3>"

export CATTLE_URL="http://<external server address>:8088/v1"

export DETECTED_CATTLE_AGENT_IP="172.17.42.1"

Regards,

Peter

[Peter Br.]

The problem was that my iptables rules on the server blocked communication between containers.

I added a rule to table filter chain INPUT to accept packets from interface docker0:

iptables -I INPUT 10 -i docker0 -j ACCEPT

Now it works fine.

Peter

[Eduar Tua]

Hi Peter,

I have the same issue, running the management server server and Agent on the same machine (My laptop). But cannot create custom hosts. Your solution didn't work for me. I and running a Fedora 21.

Output: 

INFO: Running Agent Registration Process, CATTLE_URL=http://192.168.0.3:8080/v1

INFO: Checking for Docker version >= 1.6.0

INFO: Found Server version: 1.6.0

INFO: docker version: Client version: 1.6.0

INFO: docker version: Client API version: 1.18

INFO: docker version: Go version (client): go1.4.2

INFO: docker version: Git commit (client): 4749651

INFO: docker version: OS/Arch (client): linux/amd64

INFO: docker version: Server version: 1.6.0

INFO: docker version: Server API version: 1.18

INFO: docker version: Go version (server): go1.4.2

INFO: docker version: Git commit (server): 350a636/1.6.0

INFO: docker version: OS/Arch (server): linux/amd64

INFO: docker info: Containers: 8

INFO: docker info: Images: 126

INFO: docker info: Storage Driver: devicemapper

INFO: docker info: Pool Name: docker-253:1-3018032-pool

INFO: docker info: Pool Blocksize: 65.54 kB

INFO: docker info: Backing Filesystem: extfs

INFO: docker info: Data file: /dev/loop0

INFO: docker info: Metadata file: /dev/loop1

INFO: docker info: Data Space Used: 3.402 GB

INFO: docker info: Data Space Total: 107.4 GB

INFO: docker info: Data Space Available: 36.39 GB

INFO: docker info: Metadata Space Used: 6.619 MB

INFO: docker info: Metadata Space Total: 2.147 GB

INFO: docker info: Metadata Space Available: 2.141 GB

INFO: docker info: Udev Sync Supported: true

INFO: docker info: Data loop file: /var/lib/docker/devicemapper/devicemapper/data

INFO: docker info: Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata

INFO: docker info: Library Version: 1.02.93 (2015-01-30)

INFO: docker info: Execution Driver: native-0.2

INFO: docker info: Kernel Version: 4.0.4-202.fc21.x86_64

INFO: docker info: Operating System: Fedora 21 (Twenty One)

INFO: docker info: CPUs: 8

INFO: docker info: Total Memory: 7.693 GiB

INFO: docker info: Name: localhost.localdomain

INFO: docker info: ID: WLKH:MUUW:33OQ:VCA7:F6RY:M3IP:ALWT:VT2M:445D:OU5X:HSOU:BU42

INFO: Attempting to connect to: http://192.168.0.3:8080/v1

INFO: http://192.168.0.3:8080/v1 is accessible

INFO: Inspecting host capabilities

INFO: System: true

INFO: Host writable: true

INFO: Token: xxxxxxxx

INFO: Running registration

Traceback (most recent call last):

  File "./register.py", line 11, in <module>

    secret_key=os.environ['CATTLE_REGISTRATION_SECRET_KEY'])

  File "/usr/local/lib/python2.7/dist-packages/cattle.py", line 34, in from_env

    return gdapi.from_env(prefix=prefix, factory=Client, **kw)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 604, in from_env

    return _from_env(prefix=prefix, factory=factory, **args)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 623, in _from_env

    return factory(**result)

  File "/usr/local/lib/python2.7/dist-packages/cattle.py", line 12, in __init__

    super(Client, self).__init__(*args, **kw)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 195, in __init__

    self._load_schemas()

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 309, in _load_schemas

    response = self._get_response(self._url)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 258, in _get_response

    headers=self._headers)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 477, in get

    return self.request('GET', url, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request

    resp = self.send(prep, **send_kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send

    r = adapter.send(request, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 415, in send

    raise ConnectionError(err, request=request)

requests.exceptions.ConnectionError: ('Connection aborted.', error(111, 'Connection refused'))

[Bill Maxwell]

Hi Eduar,

 Can you post the Rancher server version you are using? Also, can you post the rancher/agent image the command tells you to use? 

-Bill

[Eduar Tua]

Hi Bill,

This is my configuration:

I am using the latest server image --> "CATTLE_RANCHER_SERVER_IMAGE=v0.23.0-rc6"

Agent server image --> rancher/agent:v0.7.9

I have running the host rancher server on my laptop ip 172.17.42.1:8080. Also, I have three VM running with Vagrant with a Fedora box.

The main problem comes out when I try to connect the agent on these VMs. I got this with the docker logs command:

INFO: Attempting to connect to: http://172.17.42.1:8080/v1
ERROR: http://172.17.42.1:8080/v1 is not accessible

After a while:

ERROR: Could not reach http://172.17.42.1:8080/v1. Giving up.

[Bill Maxwell]

Hmm.. It looks like you are using the docker bridge to register that agent. Based on the error, it doesn't look like the agent is able to communicate with the server.

Try starting the agent with -e CATTLE_SCRIPT_DEBUG=true

You should see an env var CATTLE_URL. Try using curl to see if the headers show more information. Sometimes, its actually getting a 401 unauthorized response.

 I'm not sure of the networking setup, but your VMs will need to be able to access the rancher server. Typically, this is done by publishing the port 8080 (adding -p 8080:8080 to the docker run command) to the host, then you could register to your eth0 interface.

-Bill

[Luca G. Soave]

I'm having the same issue, but my version is "rancher/agent:v0.7.10" and Rancher server v0.28.0:

$ docker ps -a
CONTAINER ID        IMAGE                   COMMAND                CREATED             STATUS                      PORTS                              NAMES
c19e2c59cd2f        rancher/agent:v0.7.10   "/run.sh http://127.   11 minutes ago      Exited (1) 11 minutes ago                                     
7672518c30d4        rancher/server:latest   "/usr/bin/s6-svscan    About an hour ago   Up About an hour            3306/tcp, 0.0.0.0:8080->8080/tcp   prickly_wozniak             

the browser is showing the "admin" at  http://127.0.0.1:8080/static/infra/hosts/add/custom where I'm adding a "custom host" with the following command as suggested at step 4 (Copy, paste, and run the command below to register the host with Rancher: ...):

sudo docker run -d --privileged -v /var/run/docker.sock:/var/run/docker.sock rancher/agent:v0.7.10 http://127.0.0.1:8080/v1/scripts/13B39B052954624120AF:1437825600000:Up6EEPb2XAlI3ucRU0MrQgqqY

Starting the agent with -e CATTLE_SCRIPT_DEBUG=true I get :

CATTLE_URL=http://127.0.0.1:8080/v1/scripts/13B39B052954624120AF:1437822000000:DfnsZAKPkLmPwOulQYfGbuSA

then running curl on that I get :

$ curl http://127.0.0.1:8080/v1/scripts/13B39B052954624120AF:1437822000000:DfnsZAKPkLmPwOulQYfGbuSA
#!/bin/sh

export CATTLE_REGISTRATION_ACCESS_KEY="registrationToken"
export CATTLE_REGISTRATION_SECRET_KEY="13B32k4ke9d4624120AF:14356722003300:DfnsZAKPkLPlKJIulQYfGb"
export CATTLE_URL="http://127.0.0.1:8080/v1"
export DETECTED_CATTLE_AGENT_IP="172.17.42.1"

The only thing is that agent is on 172.17.42.1 and server on 127.0.0.1 ...
moreover Docker log say that the agent try (succesfully) to connect to http://10.36.36.90:8080/v1 and not to http://127.0.0.1:8080/v1/ :

INFO: Attempting to connect to: http://10.36.36.90:8080/v1
INFO: http://10.36.36.90:8080/v1 is accessible

then fails:

...
INFO: http://10.36.36.90:8080/v1 is accessible
INFO: Inspecting host capabilities
INFO: System: false

INFO: Host writable: true
INFO: Token: xxxxxxxx
INFO: Running registration
Traceback (most recent call last):
  File "./register.py", line 11, in <module>
    secret_key=os.environ['CATTLE_REGISTRATION_SECRET_KEY'])

  File "/usr/local/lib/python2.7/dist-packages/cattle.py", line 37, in from_env
    return gdapi.from_env(prefix=prefix, factory=Client, **kw)
  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 608, in from_env
    return _from_env(prefix=prefix, factory=factory, **args)
...


Any suggestion ?

[Luca G. Soave]

Done.

Solved following this tutorial: http://docs.rancher.com/rancher/quick-start-guide/

[Rafael Gomes]

I followed the documentation, but didn't work

The command:

docker run -d -e CATTLE_AGENT_IP=192.168.8.185 --privileged -v /var/run/docker.sock:/var/run/docker.sock rancher/agent:v0.8.1 http://192.168.8.185:8080/v1/scripts/C5DCB11ADE66CF48D45B:1440532800000:oDz1ZRmBuY0u8QSJlb8mAKZnuWU

The same error:

INFO: Running registration
Traceback (most recent call last):
  File "./register.py", line 11, in <module>
    secret_key=os.environ['CATTLE_REGISTRATION_SECRET_KEY'])
  File "/usr/local/lib/python2.7/dist-packages/cattle.py", line 37, in from_env
    return gdapi.from_env(prefix=prefix, factory=Client, **kw)
  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 608, in from_env
    return _from_env(prefix=prefix, factory=factory, **args)

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 627, in _from_env

    return factory(**result)
  File "/usr/local/lib/python2.7/dist-packages/cattle.py", line 12, in __init__
    super(Client, self).__init__(*args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 195, in __init__
    self._load_schemas()

  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 313, in _load_schemas
    response = self._get_response(self._url)
  File "/usr/local/lib/python2.7/dist-packages/gdapi.py", line 262, in _get_response

    headers=self._headers)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 477, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 415, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(111, 'Connection refused'))

--
You received this message because you are subscribed to the Google Groups "Rancher" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rancherio+...@googlegroups.com.
To post to this group, send email to ranc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/rancherio/20d8fa08-6b91-4be9-85a7-94cda17ae9f8%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Rafael Gomes
Consultor em TI
LPIC-1 MCSO Zabbix Specialist

Atenção: Este e-mail pode conter anexos no formato ODF (Open Document Format)/ABNT (extensões odt, ods, odp, odb, odg). Antes de pedir os anexos em outro formato, você pode instalar gratuita e livremente o BrOffice (http://www.broffice.org).

[Denise Schannon]

You have some containers that were launched when you were running Docker 1.5. Please delete those containers and re-run Rancher. This should fix your issue.

Denise

To view this discussion on the web visit https://groups.google.com/d/msgid/rancherio/CAMJX_-axKxipedEdRQzoNsgLB3EpVS2Kn-wqF7a0neBMWJeB8w%40mail.gmail.com.

[Denise Schannon]

Also, we’ve moved away from google groups to http://forums.rancher.com/ for questions/help.

Thanks,

Denise

From: <ranc...@googlegroups.com> on behalf of Rafael Gomes <linux...@gmail.com>
Date: Tuesday, August 25, 2015 at 1:23 PM
To: "Luca G. Soave" <luca....@gmail.com>
Cc: Rancher <ranc...@googlegroups.com>, "bill.t....@gmail.com" <bill.t....@gmail.com>
Subject: Re: Rancher IO server and agent on same machine

To view this discussion on the web visit https://groups.google.com/d/msgid/rancherio/CAMJX_-axKxipedEdRQzoNsgLB3EpVS2Kn-wqF7a0neBMWJeB8w%40mail.gmail.com.

[Rafael Gomes]

Thanks! Your advice worked here :)