Error at edx requirements installation (installing development environment on Ubuntu 12.04)

[Plinio Gañi]

Hello,

I have done a successful installation of the development environment in the past (on a different machine) by following the directions found here, but in this new installation attempt I am facing an error at the point when edx requirements are being installed. Here I copy the error information:

pip install -q --exists-action w -r requirements/edx/base.txt
  Could not find a tag or branch 'd89aae2a82f2b', assuming commit.
  Could not find a tag or branch '88ec8a011e481918fdc9d2682d4017c835acd8be', assuming commit.
  Could not find a tag or branch 'cd0b2b31997afccde519fe5b3365e61a9edb143f', assuming commit.
  Could not find a tag or branch 'cccc3978ad2df82a78b6f9648fe2e9baddd22f88', assuming commit.
  Could not find a tag or branch '776973740bdaad83a3b029f96e415a7d1e8bec2f', assuming commit.
  Could not find a tag or branch 'd53fe0e81b623f084e91776bcf6369f8b7b63879', assuming commit.
  Could not find a tag or branch '42cb1bb1dc680a60c6452e8bb2b843c2a0382c90', assuming commit.
  Could not find a tag or branch '95e69341cebf5a6a9fbf7c4f5439d458898bdc3b', assuming commit.
  Could not find a tag or branch '6dd8a9223cae34184ba5e2e1a186f36c4df1e080', assuming commit.
  Could not find a tag or branch 'e3d98f9455', assuming commit.
  Could not find a tag or branch '823a102e48', assuming commit.
  Could not find a tag or branch 'f0211d702d', assuming commit.
  Could not find a tag or branch '62de7b576a08f36cde5b030c52bccb1a2f3f8df1', assuming commit.
  Could not find a tag or branch '9965a53c269666a30bb4e2b3f6037c138aef2a55', assuming commit.
  Could not find a tag or branch '459aff7b63db8f2c5decd1755706c1a64fb4ebb1', assuming commit.
  MD5 hash of the package https://pypi.python.org/packages/source/w/watchdog/watchdog-0.6.0.tar.gz#md5=6e8fb84aa6bdeb2054906bdf40048eba (from https://pypi.python.org/simple/watchdog/) (3f8ad1f7f9e32a61b6fb00d68fd9dbc8) doesn't match the expected hash 6e8fb84aa6bdeb2054906bdf40048eba!
Bad MD5 hash for package https://pypi.python.org/packages/source/w/watchdog/watchdog-0.6.0.tar.gz#md5=6e8fb84aa6bdeb2054906bdf40048eba (from https://pypi.python.org/simple/watchdog/)
Storing complete log in /home/plinio/.pip/pip.log
rake aborted!
Command failed with status (1): [pip install -q --exists-action w -r requir...]
/home/plinio/edx_all/edx-platform/rakelib/prereqs.rake:35:in `block (2 levels) in <top (required)>'
/home/plinio/edx_all/edx-platform/rakelib/helpers.rb:47:in `when_changed'
/home/plinio/edx_all/edx-platform/rakelib/prereqs.rake:32:in `block in <top (required)>'
Tasks: TOP => install_prereqs => install_python_prereqs
(See full trace by running task with --trace)

    !! ERROR !!

    The last command did not complete successfully,
    For more details or trying running the
    script again with the -v flag.


I have attached the full log file to this post for additional information.

Can someone direct me on the right way to resolve this?

Thanks a lot in advance for your support.

Best regards,

Plinio.

[Yarko Tymciurak]

The instructions you link to look different enough from 

https://github.com/edx/configuration/wiki/edX-Ubuntu-12.04-Installation#installing-edx-on-a-single-ubuntu-1204-precise-server-from-scratch

that you might want to start over, follow the instructions above.

[Plinio Gañi]

Dear Yarko,

I'm trying to install the production environment following the instructions at the link you mentioned (following the manual procedure instructions), and I am receiving the following error after doing:

sudo ansible-playbook -c local ./edx_sandbox.yml -i "localhost,"

Everything goes fine until it reaches this point:

TASK: [common | pip install virtualenv] *************************************** 
changed: [localhost] => (item=virtualenv==1.10.1)
failed: [localhost] => (item=virtualenvwrapper) => {"cmd": "/usr/local/bin/pip install -i https://pypi.python.org/simple virtualenvwrapper", "failed": true, "item": "virtualenvwrapper"}
msg: stdout: Downloading/unpacking virtualenvwrapper
  Running setup.py (path:/tmp/pip_build_root/virtualenvwrapper/setup.py) egg_info for package virtualenvwrapper
    Download error on https://pypi.python.org/simple/pbr/: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol -- Some packages may not be found!
    Couldn't find index page for 'pbr' (maybe misspelled?)
    Download error on https://pypi.python.org/simple/: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol -- Some packages may not be found!
    No local packages or download links found for pbr>=0.5.19
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip_build_root/virtualenvwrapper/setup.py", line 7, in <module>
        pbr=True,
      File "/usr/lib/python2.7/distutils/core.py", line 112, in setup
        _setup_distribution = dist = klass(attrs)
      File "build/bdist.linux-i686/egg/setuptools/dist.py", line 265, in __init__
      File "build/bdist.linux-i686/egg/setuptools/dist.py", line 289, in fetch_build_eggs
      File "build/bdist.linux-i686/egg/pkg_resources.py", line 618, in resolve
        cannot be resolved.
      File "build/bdist.linux-i686/egg/pkg_resources.py", line 862, in best_match
    
      File "build/bdist.linux-i686/egg/pkg_resources.py", line 874, in obtain
        cache_path
      File "build/bdist.linux-i686/egg/setuptools/dist.py", line 339, in fetch_build_egg
      File "build/bdist.linux-i686/egg/setuptools/command/easy_install.py", line 617, in easy_install
    
    distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pbr>=0.5.19')
    Complete output from command python setup.py egg_info:
    Download error on https://pypi.python.org/simple/pbr/: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol -- Some packages may not be found!

Couldn't find index page for 'pbr' (maybe misspelled?)

Download error on https://pypi.python.org/simple/: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol -- Some packages may not be found!

No local packages or download links found for pbr>=0.5.19

Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip_build_root/virtualenvwrapper/setup.py", line 7, in <module>

    pbr=True,

  File "/usr/lib/python2.7/distutils/core.py", line 112, in setup

    _setup_distribution = dist = klass(attrs)

  File "build/bdist.linux-i686/egg/setuptools/dist.py", line 265, in __init__

  File "build/bdist.linux-i686/egg/setuptools/dist.py", line 289, in fetch_build_eggs

  File "build/bdist.linux-i686/egg/pkg_resources.py", line 618, in resolve

    cannot be resolved.

  File "build/bdist.linux-i686/egg/pkg_resources.py", line 862, in best_match



  File "build/bdist.linux-i686/egg/pkg_resources.py", line 874, in obtain

    cache_path

  File "build/bdist.linux-i686/egg/setuptools/dist.py", line 339, in fetch_build_egg

  File "build/bdist.linux-i686/egg/setuptools/command/easy_install.py", line 617, in easy_install



distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pbr>=0.5.19')

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_root/virtualenvwrapper
Storing debug log for failure in /root/.pip/pip.log


FATAL: all hosts have already failed -- aborting

PLAY RECAP ******************************************************************** 
           to retry, use: --limit @/home/plinio/edx_sandbox.retry

localhost                  : ok=8    changed=6    unreachable=0    failed=1  

Do you have ideas why is this happening or suggestions on how to fix it?

Thanks a lot in advance for your help.

Plinio

[Yarko Tymciurak]

On Friday, March 14, 2014 3:55:12 PM UTC-5, Plinio Gañi wrote:

Dear Yarko,

I'm trying to install the production environment following the instructions at the link you mentioned (following the manual procedure instructions), and I am receiving the following error after doing:

sudo ansible-playbook -c local ./edx_sandbox.yml -i "localhost,"

Everything goes fine until it reaches this point:

TASK: [common | pip install virtualenv] *************************************** 
changed: [localhost] => (item=virtualenv==1.10.1)
failed: [localhost] => (item=virtualenvwrapper) => {"cmd": "/usr/local/bin/pip install -i https://pypi.python.org/simple virtualenvwrapper", "failed": true, "item": "virtualenvwrapper"}
msg: stdout: Downloading/unpacking virtualenvwrapper
  Running setup.py (path:/tmp/pip_build_root/virtualenvwrapper/setup.py) egg_info for package virtualenvwrapper
    Download error on https://pypi.python.org/simple/pbr/: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol -- Some packages may not be found!

Hi Plinio -

Sorry you are having so much trouble.

When I go to https://pypi.python.org/pypi/pbr/  this is all ok / all there;

Also when I go to https://pypi.python.org/simple/pbr/  (a imple directory listing in a browser).

My suggestions would be:

• it might be a temporary network accessibility glitch - simply re-try
• if not, then try to get from your facility manually to those two addresses (pypi/pbr, and simple/pbr/)
• if that works, then go to your machine, and confirm that it works from your machine of interest:  try either curl or wget, something like:
• curl -O https://pypi.python.org/simple/pbr/
• if that fails, see if this (http instead of https) works:
• curl -O http://pypi.python.org/simple/pbr/

[Plinio Gañi]

Dear Yarko,

Tanks a lot for your prompt reply. I already checked the avalability from another computer just browsing the repository using Firefox and everything is fine, also tried with Firefox from the machine I am performing the installation and everything is ok. I could even download one of th files without problem with wget from a terminal both using http and https as follows:

wget http://pypi.python.org/packages/source/p/pbr/pbr-0.5.19.tar.gz
wget https://pypi.python.org/packages/source/p/pbr/pbr-0.5.19.tar.gz

What could be happening? I am not an expeinced linux user and really appreciate your help.

Plinio.

[Plinio Gañi]

Update:

Although both urls https://pypi.python.org/simple/pbr/ and https://pypi.python.org/simple/ open fine in Firefox, I noticed that while the former opens very quickly, the latter takes various seconds (around 20) to resolve and finally open... May this be important?

Thanks a lot.

Plinio.

[Yarko Tymciurak]

yes - it may be causing a timeout (it's simply a text directory listing) -  just retry when this happens, and server and more local caches are likely to get it a second time.

This is similar to calling a mobile phone number when traveling.   The first time someone tries you, it may not ring because the network is busy trying to resolve the location of the phone;  a similar effect happens from DNS lookups (and more things besides lookups too - but effect is similar).

Just retry.

When 2 or 3 times of retry doesn't work, then be suspicious.

[Plinio Gañi]

Dear Yarko,

You are right, it seems to be timing out and keeps giving the same error no matter how many times I retry. It looks like the proxy may be involved and there are also redirections in the middle and then the text directory takes a while to fully open, thus the problem of timing out. Maybe I can manually download the requirement (in this case pbr vesion >=0.5.19) and put it somewhere so the setup script can find it and use it instead of trying to download it? Do you know where I should place the file and if I should decompress it first or just to put the .gz as downloaded? Any advice is very welcome!

Thanks a lot,

Plinio.

[Yarko Tymciurak]

On Monday, March 17, 2014 12:10:43 PM UTC-5, Plinio Gañi wrote:

Dear Yarko,

You are right, it seems to be timing out and keeps giving the same error no matter how many times I retry. It looks like the proxy may be involved and there are also redirections in the middle and then the text directory takes a while to fully open, thus the problem of timing out. Maybe I can manually download the requirement (in this case pbr vesion >=0.5.19) and put it somewhere so the setup script can find it and use it instead of trying to download it? Do you know where I should place the file and if I should decompress it first or just to put the .gz as downloaded? Any advice is very welcome!

This is not an edx question now, but if you search, or try these for starters you might find help:

http://stackoverflow.com/questions/4806448/how-do-i-install-from-a-local-cache-with-pip

http://www.pip-installer.org/en/latest/user_guide.html#fast-local-installs

Regards,

- Yarko

[Plinio Gañi]

Hi Yarko,

Many thanks for your reply.

I apologize if my last question was not EdX specific, but please understand I am not an educated linux user and sometimes it is hard fro me to tell the boundaries between the EdX platform and the tools it intimately depends on (for its installation or running)

I managed to manually download and install pbr and retried the installation successfully passing the previous error point, but now I am facing a different error which does not look related with a failed downloading attempt, but something about an abnormal termination of devpi-server.

Could you please take a look at the error and tell me if this is an error to be posted here or not?

Thanks a lot in advance for your always kind support.


Error output:

TASK: [devpi | ensure devpi is started] ***************************************
failed: [localhost] => {"failed": true, "item": ""}
msg: devpi-server: ERROR (abnormal termination)

FATAL: all hosts have already failed -- aborting

Regards,

Plinio.

[Plinio Gañi]

UPDATE:

Nevermind, I installed devpi-server manually (wasn't even installed) and could pass through the error, it seems I will have to apply the same solution every time the installation show an error of this type. I will return with news if I can finish a successful installation.

Best regards and again thank you very much for all your help.

Plinio.

[Apssdc Gov]

hi 

i a new to edx , when i am trying to install i am getting the error 

TASK: [devpi | install devpi pip pkgs] ****************************************

failed: [localhost] => (item=devpi-server) => {"cmd": "/edx/app/devpi/venvs/devpi/bin/pip install devpi-server", "failed": true, "item": "devpi-server", "rc": 13}

msg: [Errno 13] Permission denied

failed: [localhost] => (item=eventlet) => {"cmd": "/edx/app/devpi/venvs/devpi/bin/pip install eventlet", "failed": true, "item": "eventlet", "rc": 13}

msg: [Errno 13] Permission denied

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************

INFO:ansible.callback_plugins.datadog_tasks_timing:server_utils | Install system packages ------------------------------------------- 9.64s

INFO:ansible.callback_plugins.datadog_tasks_timing:edxapp_common | Install system packages ------------------------------------------ 9.59s

INFO:ansible.callback_plugins.datadog_tasks_timing:mongo | install mongo server and recommends -------------------------------------- 9.58s

INFO:ansible.callback_plugins.datadog_tasks_timing:nginx | Install the nginx package ------------------------------------------------ 9.52s

INFO:ansible.callback_plugins.datadog_tasks_timing:common | Install role-independent useful system packages ------------------------- 9.48s

INFO:ansible.callback_plugins.datadog_tasks_timing:mongo | install python pymongo for mongo_user ansible module --------------------- 6.81s

INFO:ansible.callback_plugins.datadog_tasks_timing:common | pip install virtualenv -------------------------------------------------- 5.69s

INFO:ansible.callback_plugins.datadog_tasks_timing:mongo | wait for mongo server to start ------------------------------------------- 2.13s

INFO:ansible.callback_plugins.datadog_tasks_timing:supervisor | install supervisor in its venv -------------------------------------- 1.89s

INFO:ansible.callback_plugins.datadog_tasks_timing:common | Update CA Certificates -------------------------------------------------- 1.63s

INFO:ansible.callback_plugins.datadog_tasks_timing:

Playbook edx_sandbox finished: Mon Feb 15 02:59:20 2016, 137 total tasks.  0:01:26 elapsed.

           to retry, use: --limit @/home/ghouse/edx_sandbox.retry

localhost                  : ok=102  changed=12   unreachable=0    failed=1

can any one help me out in resolving out