JupyterHub + Gitlab on docker - Gitlab OAuth fails

[Frédéric Collonval]

Hello everyone,

I'm trying to set a docker environnement containing an instance of Gitlab + JupyterHub.

As a starting point, I use that example: https://github.com/jupyterhub/jupyterhub-deploy-docker
It works like a charm with GitHub OAuth. I then switch to GitLab OAuth and works by using an gitlab.com account.

So I moved on update the GITLAB_HOST variable in the JupyterHub image to use the docker image gitlab locally. And there is the strange behavior, if I set GITLAB_HOST with the local IP address of the server exposing port 80 and 443. I can connect to JupyterHub and launch notebooks using Gitlab account. But if I set the domain, it fails (error 500) the log of JupyterHub being:

[I 2017-12-17 18:50:01.384 JupyterHub oauth2:82] OAuth redirect: 'http://jupyterhub.example.com/hub/oauth_callback'
[I 2017-12-17 18:50:01.386 JupyterHub log:122] 302 GET /hub/oauth_login?next= → http://gitlab.example.com/oauth/authorize?client_id=...&state=...&response_type=code&redirect_uri=http%3A%2F%2Fjupyterhub.example.com%2Fhub%2Foauth_callback (@192.168.1.60) 2.51ms
[E 2017-12-17 18:50:01.616 JupyterHub web:1590] Uncaught exception GET /hub/oauth_callback?code=..&state=... (192.168.1.60)
    HTTPServerRequest(protocol='http', host='jupyterhub.example.com', method='GET', uri='/hub/oauth_callback?code=...&state=...', version='HTTP/1.1', remote_ip='192.168.1.60', headers={'X-Real-Ip': '192.168.1.60', 'Host': 'jupyterhub.example.com', 'Connection': 'close', 'Upgrade-Insecure-Requests': '1', 'X-Forwarded-For': '192.168.1.60,::ffff:172.18.0.2', 'Referer': 'http://jupyterhub.example.com/hub/login', 'Accept-Encoding': 'gzip, deflate', 'X-Forwarded-Host': 'jupyterhub.example.com', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0', 'X-Forwarded-Proto': 'http,http', 'Cookie': 'oauthenticator-state="2|1:0|10:1513536601|20:oauthenticator-state|120:ZXlKdVpYaDBYM1Z5YkNJNklDSWlMQ0FpYzNSaGRHVmZhV1FpT2lBaVpUQTJOamRoWXpOa1l6RXpOR00zTmpoa09EZ3lPVE16TUdJek5qZ3labVFpZlE9PQ==|c5873a9b1dbf77f8f39eaa1383ff46f31a25a8bb1833022f47f4b4a3ac0fa8bc"', 'X-Forwarded-Port': '80', 'Dnt': '1', 'Accept-Language': 'fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3'})
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.5/site-packages/tornado/web.py", line 1511, in _execute
        result = yield result
      File "/opt/conda/lib/python3.5/site-packages/oauthenticator/oauth2.py", line 182, in get
        user = yield self.login_user()
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/handlers/base.py", line 328, in login_user
        authenticated = yield self.authenticate(data)
      File "/opt/conda/lib/python3.5/site-packages/jupyterhub/auth.py", line 227, in get_authenticated_user
        authenticated = yield self.authenticate(handler, data)
      File "/opt/conda/lib/python3.5/site-packages/oauthenticator/gitlab.py", line 95, in authenticate
        resp = yield http_client.fetch(req)
    tornado.curl_httpclient.CurlError: HTTP 599: Failed to connect to gitlab.example.com port 80: Connection refused
   
[E 2017-12-17 18:50:01.631 JupyterHub log:114] {
      "X-Real-Ip": "192.168.1.60",
      "Host": "jupyterhub.example.com",
      "Connection": "close",
      "Upgrade-Insecure-Requests": "1",
      "X-Forwarded-For": "192.168.1.60,::ffff:172.18.0.2",
      "Referer": "http://jupyterhub.example.com/hub/login",
      "Accept-Encoding": "gzip, deflate",
      "X-Forwarded-Host": "jupyterhub.example.com",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
      "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0",
      "X-Forwarded-Proto": "http,http",
      "Cookie": "oauthenticator-state=\"2|1:0|10:1513536601|20:oauthenticator-state|120:ZXlKdVpYaDBYM1Z5YkNJNklDSWlMQ0FpYzNSaGRHVmZhV1FpT2lBaVpUQTJOamRoWXpOa1l6RXpOR00zTmpoa09EZ3lPVE16TUdJek5qZ3labVFpZlE9PQ==|c5873a9b1dbf77f8f39eaa1383ff46f31a25a8bb1833022f47f4b4a3ac0fa8bc\"",
      "X-Forwarded-Port": "80",
      "Dnt": "1",
      "Accept-Language": "fr,fr-FR;q=0.8,en-US;q=0.5,en;q=0.3"
    }
[E 2017-12-17 18:50:01.631 JupyterHub log:122] 500 GET /hub/oauth_callback?code=...&state=... (@192.168.1.60) 20.01ms

I'm not familiar with networks and docker. But if you could point me to a solution or a way to get more useful information to figure out the source of the error, I would appreciate.

Kindly

Frederic


For info the docker-compose.yml looks like that:

version: "2"                                                                                                        
                                                                                                                   
services:                                                                                                          
  web:                                                                                                              
    image: 'gitlab/gitlab-ce:latest'                                                                                                    
    restart: always                                                                                                  
    container_name: gitlab                                                                                          
    environment:                                                                                                    
      GITLAB_OMNIBUS_CONFIG: |                                                                                      
        external_url 'http://gitlab.example.com'                                                                    
        unicorn['worker_processes'] = 2                                                          
        nginx['custom_nginx_config'] = "include /var/opt/gitlab/nginx/conf.d/jupyterhub-http.conf;"
    ports:
      - '80:80'
      - '443:443'
    volumes:
      - '/srv/gitlab/config:/etc/gitlab'
      - '/srv/gitlab/logs:/var/log/gitlab'
      - '/srv/gitlab/data:/var/opt/gitlab'

  hub-db:
    image: postgres:9.5
    container_name: jupyterhub-db
    restart: always
    environment:
      POSTGRES_DB: ${POSTGRES_DB}
      PGDATA: ${DB_VOLUME_CONTAINER}
    env_file:
      - secrets/postgres.env
    volumes:
      - "db:${DB_VOLUME_CONTAINER}"

  hub:
    depends_on:
      - hub-db
    build:
      context: .
      dockerfile: Dockerfile.jupyterhub
      args:
        JUPYTERHUB_VERSION: ${JUPYTERHUB_VERSION}
    restart: always
    image: jupyterhub
    container_name: jupyterhub
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:rw"
      - "data:${DATA_VOLUME_CONTAINER}"
    links:
      - hub-db
    environment:
      DOCKER_NETWORK_NAME: ${DOCKER_NETWORK_NAME}
      DOCKER_NOTEBOOK_IMAGE: ${LOCAL_NOTEBOOK_IMAGE}
      DOCKER_NOTEBOOK_DIR: ${DOCKER_NOTEBOOK_DIR}
      DOCKER_SPAWN_CMD: ${DOCKER_SPAWN_CMD}
      POSTGRES_DB: ${POSTGRES_DB}
      POSTGRES_HOST: hub-db
    env_file:
      - secrets/postgres.env
      - secrets/oauth.env
    command: >
      jupyterhub -f /srv/jupyterhub/jupyterhub_config.py

volumes:
  data:
    external:
      name: ${DATA_VOLUME_HOST}
  db:
    external:
      name: ${DB_VOLUME_HOST}

networks:
  default:
    external:
      name: ${DOCKER_NETWORK_NAME}