Successfully installed edx-platform on Ubuntu 14.04 LTS

[Omar Al-Ithawi]

Hi All,

For hardware compatibility I had to install Ubuntu 14.04 LTS. After many trials and hacks I managed to install it. The following is log of what I have done. If I missed something, please tell me!

Install some dependencies to make sure that you get ruby with SSL support:

$ sudo apt-get install build-essential bison openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev libssl0.9.8 libssl-dev

The install ruby to get bundle support (actually not sure if this is required):
$ sudo apt-get install ruby

Install testing requirements:

sudo apt-get install memcached

Install the repository:

$ cd ~
$ mkdir -p ~/edx_all/
$ cd ~/edx_all/
$ ln -sd ~/edx-platform edx-platform
$ cd ~
$ git clone g...@github.com:Edraak/edx-platform.git
$ cd edx-platform
$ git apply
# Paste the patch in it to make it accept Ubuntu 14.04 LTS (trusty).
$ ./scripts/create-dev-env.sh

Trusty Patch

diff --git a/scripts/create-dev-env.sh b/scripts/create-dev-env.sh
index 7f7c9f9..7d83e1e 100755
--- a/scripts/create-dev-env.sh
+++ b/scripts/create-dev-env.sh
@@ -252,7 +252,7 @@ case `uname -s` in

         distro=`lsb_release -cs`
         case $distro in
-            wheezy|jessie|maya|olivia|nadia|precise|quantal)
+            wheezy|jessie|maya|olivia|nadia|precise|quantal|trusty)
                 if [[ ! $noninteractive ]]; then
                     warning "
                             Debian support is not fully debugged. Assuming you have standard
@@ -442,6 +442,7 @@ if [[ -n $compile ]]; then
 fi

 # building correct version of distribute from source
+true || (
 DISTRIBUTE_VER="0.6.28"
 output "Building Distribute"
 SITE_PACKAGES="$WORKON_HOME/edx-platform/lib/python2.7/site-packages"
@@ -461,7 +462,7 @@ else
   error "Distribute failed to build correctly. This script requires a working version of Distribute 0.6.28 in your virtualenv's python installation"
   exit 1
 fi
-
+)
 case `uname -s` in
     Darwin)
         # on mac os x get the latest distribute and pip
diff --git a/scripts/install-system-req.sh b/scripts/install-system-req.sh
index b4e5bb0..ad6b3e8 100755
--- a/scripts/install-system-req.sh
+++ b/scripts/install-system-req.sh
@@ -31,7 +31,7 @@ case `uname -s` in
         distro=`lsb_release -cs`
         case $distro in
             #Tries to install the same
-            squeeze|wheezy|jessie|maya|lisa|olivia|nadia|natty|oneiric|precise|quantal|raring)
+            squeeze|wheezy|jessie|maya|lisa|olivia|nadia|natty|oneiric|precise|quantal|raring|trusty)
                 output "Installing Debian family requirements"

                 # add repositories

Now your journey begins...

After many many trial and errors you should have pip installed, among so many dependencies. You'll need to do the following hacks:

$ sudo apt-get install libmagickwand-dev
$ pip install -r requirements/edx/paver.txt
$ workon edx-platform
$ pip install distribute==0.6.29

Manually (as in manually extract the files) install TinyUrl-0.1.0-py2.5.egg in

$ cd ~/.virtualenvs/edx-platform/lib/python2.7/site-packages/
$ wget https://pypi.python.org/packages/2.5/T/TinyUrl/TinyUrl-0.1.0-py2.5.egg
$ mv TinyUrl-0.1.0-py2.5.egg TinyUrl-0.1.0-py2.5.zip
$ unzip TinyUrl-0.1.0-py2.5.zip
$ mv EGG-INFO TinyUrl-0.1.0-py2.5.egg-info

$ cd /usr/include
$ sudo ln -s freetype2 freetype

Keep running $ ./scripts/create-dev-env.sh
Until $ workon edx-platform && cd ~/edx-platform && paver run_all_servers --setting_lms=cms.dev work well.

Ruby SSL Issue

If ruby complains about SSL you should run this command: $ cd ~/edx-paltform $ rbenv install
This should solve it.

Configuring and Running the Platform

While the [] guide covers almost all you need. It misses few points. You need to configure your local dev. environment:

$ gedit lms/envs/private.py

And put the following code in it:

from os.path import expanduser
from path import path
home = path(expanduser("~"))

CONTENTSTORE = {
    'ENGINE': 'xmodule.contentstore.mongo.MongoContentStore',
    'DOC_STORE_CONFIG': {
        'host': 'localhost',
        'db': 'xcontent',
    },
    # allow for additional options that can be keyed on a name, e.g. 'trashcan'
    'ADDITIONAL_OPTIONS': {
        'trashcan': {
            'bucket': 'trash_fs'
        }
    }
}

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': home / "db" / "edraak-edx.db",
    }
}

And for cms $ gedit ./cms/envs/private.py:

from os.path import expanduser
from path import path
home = path(expanduser("~"))

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': home / "db" / "edraak-edx.db",
    }
}

You'll have two dummy warnings about missing local configuration files which
can be easily turned off by putting an empty JSON in each:

$ echo '{}' | sudo tee ~/env.json /home.json

Run The Servers

$ workon edx-platform
$ cd ~/edx-platform
$ paver run_all_servers --settings_lms=cms.dev

Run the Test Successfully

$ git checkout benp/fix-js-tests # release and master had failing JS tests so I 
picked up this branch with a passing tests
$ paver test

Non-English Locales

If you're like me and your locale is ar_JO.UTF-8 $ ls will display the Arabic
names of the months. This will cause the lms tests to fail. Changing the
locale using $ unity-control-center language and set everything to
English USA. This should solve the issue.

Thanks,

[Omar Al-Ithawi]

I'm attaching the markdown source of the previous post to avoid copy & paste formatting issues:

Install some dependencies to make sure that you get ruby with SSL support: 

```

$ sudo apt-get install build-essential bison openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev libssl0.9.8 libssl-dev

```

The install ruby to get `bundle` support (actually not sure if this is required):

```

$ sudo apt-get install ruby

```

Install testing requirements:

```

sudo apt-get install memcached

```

Install the repository:

```

$ cd ~

$ mkdir -p ~/edx_all/

$ cd ~/edx_all/

$ ln -sd ~/edx-platform edx-platform

$ cd ~

$ git clone g...@github.com:Edraak/edx-platform.git

$ cd edx-platform

$ git apply

# Paste the patch in it to make it accept Ubuntu 14.04 LTS (trusty).

$ ./scripts/create-dev-env.sh

```

###Trusty Patch

Manually (as in manually extract the files) install [TinyUrl-0.1.0-py2.5.egg](https://pypi.python.org/packages/2.5/T/TinyUrl/TinyUrl-0.1.0-py2.5.egg#md5=d9f8873ff61f5a428532bbd949431c76) in 

```

$ cd ~/.virtualenvs/edx-platform/lib/python2.7/site-packages/

$ wget https://pypi.python.org/packages/2.5/T/TinyUrl/TinyUrl-0.1.0-py2.5.egg

$ mv TinyUrl-0.1.0-py2.5.egg TinyUrl-0.1.0-py2.5.zip

$ unzip TinyUrl-0.1.0-py2.5.zip

$ mv EGG-INFO TinyUrl-0.1.0-py2.5.egg-info

```

```

$ cd /usr/include

$ sudo ln -s freetype2 freetype

```

Keep running `$ ./scripts/create-dev-env.sh`

Until `$ workon edx-platform && cd ~/edx-platform && paver run_all_servers --setting_lms=cms.dev` work well.

###Ruby SSL Issue

If ruby complains about SSL you should run this command:

```

$ cd ~/edx-paltform

$ rbenv install

```

This should solve it.

#Configuring and Running the Platform

```

##Run The Servers

```

$ workon edx-platform

$ cd ~/edx-platform

$ paver run_all_servers --settings_lms=cms.dev

```

##Run the Test *Successfully*

```

$ git checkout benp/fix-js-tests # release and master had failing JS tests so I 

picked up this branch with a passing tests

$ paver test

```

###Non-English Locales

If you're like me and your locale is ar_JO.UTF-8 `$ ls` will display the Arabic

names of the months. This will cause the `lms` tests to fail. Changing the 

locale using `$ unity-control-center language` and set everything to 

English USA. This should solve the issue. 

[Han Su Kim]

You should not need to install 14.04 to continue to get HW support, going to 12.04.5 will work and is what we do if we run into the issue (generally we don't since we use AWS)

https://wiki.ubuntu.com/Kernel/LTSEnablementStack

That all being said, the only difference to get 14.04 the last time I tried was to pre install

sudo apt-get install -y build-essential software-properties-common python-software-properties curl git-core libxml2-dev libxslt1-dev python-pip python-apt python-dev nodejs

See instructions https://github.com/edx/configuration/wiki/edX-Ubuntu-12.04-64-bit-Installation

[Omar Al-Ithawi]

Hi Han,

Thank you for the tips, I will definitely try it the next time I'm going to install it on 14.04.

BTW I'm installing a dev. environment not sure how the edx_sandbox.yml will be suitable for that. If it is the same please advice :).

Thanks,

--

Omar Al-Ithawi
Senior Engineer, Edraak.org
Queen Rania Foundation
for Education & Development
t.   +962 6 4016464 ext. 407
m. +962 7 90754407
e. oit...@qrf.org

 

[Michael Shamberger]

Hi,

I tried into installing into an ubuntu 14.04 server.  I get the following error.   Ran it twice and received the same again:

(I am using https://github.com/edx/configuration/wiki/edX-Ubuntu-12.04-64-bit-Installation on a 14.04 box.

TASK: [ora | ensure ora_celery is started] ************************************ 

failed: [localhost] => {"failed": true, "item": ""}

msg: ora_celery: ERROR (abnormal termination)

FATAL: all hosts have already failed -- aborting

Anyone else see this?   I am trying to run 14.04 since the company that provides my dedicated server dropped support in their provisioning tools for new installs of 12.04.

I look into the /edx/var/log/supervisor/ora_celerytderr.log and see the following error:

  File "/edx/app/ora/venvs/ora/local/lib/python2.7/site-packages/scipy/linalg/lapack.py", line 14, in <module>

    from scipy.linalg import flapack

ImportError: cannot import name flapack

[Han Su Kim]

The dev enviorment scripts in edx- platform have basically been deprecated. They arent updated any more.

Look at the configuration repo for the new hotness. 

—
Sent from Mailbox

<image001.png>

[Brian Wilson]

Although it's not supported, I went through a similar process of getting edx-platform running on 14.04.  A couple of observations:

* The issue with freetype/freetype2 should go away once Pillow gets upgraded from 1.7.8 to 2.5.1.
* The issue with rbenv install also required a rbenv rehash afterwards.  (e.g. if sass isn't found)
* Make sure to create the log and db directories.
* I found it was adequate to use cms.dev for settings, though there may be a more recent PR to fix dev as well to define CONTENTSTORE.  I didn't need a private.py.

[Omar Al-Ithawi]

Hi Han,

We're using edx/configuration to deploy to master. But I'm not sure if edx_sandbox.yml will create a deb environment. Does it?

Thanks,

 

[Sarina Canelake]

Hi Omar,

In the future, questions like this should really be posted to the Operations mailing list. https://groups.google.com/forum/#!forum/openedx-ops

Best,

Sarina