Eucalyptus 1 RC1 Prerelease

[Ned Batchelder]

Hello Open edX adopters,

The next Open edX release will be Eucalyptus. We have a prerelease available now for limited use. Eucalyptus 1 RC 1 is the first release candidate of the first Eucalyptus release (we've changed the numbering scheme slightly since Dogwood).

This prerelease is only available for fullstack and native installs.  Devstack is not yet supported.  An overview of the installation options is here: https://openedx.atlassian.net/wiki/display/OpenOPS/Open+edX+Installation+Options

If you want to try this pre-release, please do not do it on a live machine, or risk live data with it.  The OPENEDX_RELEASE tag name to use is open-release/eucalyptus/1rc1 .  If you want to upgrade from a previous install, the existing instructions work, but the migrate.sh script has been renamed to upgrade.sh to better match its function.

Before we release a real Eucalyptus, we will be adding a few more fixes, and want to hear from you if you discover problems with this pre-release.  We even want to hear if you don't encounter problems! :)

Thanks,

--Ned.

[Ovnicraft]

Hi Ned, 

thanks amazing work around edX, so there is any page where we can found all pending issues locking eucalyptus release ? 

 

Thanks,

--Ned.

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CAGtJPNNv_3QnXO%2Bp7AYt0QMR%3DJMJNbtXMSkK6tNu9qABmy%2BoOA%40mail.gmail.com.

--

		Cristian Salamea about.me/ovnicraft

[Joel Barciauskas]

Hi Cristian,

I'm sorry, can you clarify what you mean? Are you asking what the "few more fixes" are that we will be making?

[Ovnicraft]

On Tue, Jul 26, 2016 at 9:55 AM, Joel Barciauskas <jo...@edx.org> wrote:

Hi Cristian,

I'm sorry, can you clarify what you mean? Are you asking what the "few more fixes" are that we will be making?

Yes.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/9fa45ce2-38c3-4a67-bf9a-aec2b8bd5076%40googlegroups.com.

[Omar Al-Ithawi]

Hi,

One thing that I would like to have in this, or future releases.

As a non-English fork maintainer I would like to have enough time before the release cut date to get my Transifex translation to something satisfactory (i.e. 100% reviewed for the important resources).

The thing is, it is very hard for me to know when new strings are being pushed to Transifex and when they're getting pulled. 

The time window most of the time is either too narrow for us to react, or it is unknown and that's why we tend to miss it.

Could this be communicated clearly? Especially around the release cut dates?

Thanks,

[Ovnicraft]

On Tue, Jul 26, 2016 at 10:09 PM, Omar Al-Ithawi <oit...@qrf.org> wrote:

Hi,

One thing that I would like to have in this, or future releases.

As a non-English fork maintainer I would like to have enough time before the release cut date to get my Transifex translation to something satisfactory (i.e. 100% reviewed for the important resources).

The thing is, it is very hard for me to know when new strings are being pushed to Transifex and when they're getting pulled. 

The time window most of the time is either too narrow for us to react, or it is unknown and that's why we tend to miss it.

Could this be communicated clearly? Especially around the release cut dates?

Hi, i we are using edX in spanish version and when something changes in transifex, its notify about changes in any resources updated.

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/a7d35b80-69ba-4812-822a-990ef999d9d2%40googlegroups.com.

[Pierre Mailhot]

In order to check if the problems I encountered after the upgrade are with the rebase of my branch/fork or general to open-release/eucalyptus/1rc1 , I tried installing a fresh copy on a new ubuntu 12.04 EC2 instance using the instructions from https://openedx.atlassian.net/wiki/display/OpenOPS/Native+Open+edX+Ubuntu+12.04+64+bit+Installation

Let's jusy say it complained heavily when it reached the following task:

TASK: [edxapp | code sandbox | Install sandbox requirements into sandbox venv] *** 

Looks like atlas, numpy and scipy problems. I haven't encountered those in a while...

[t...@opencraft.com]

Hi Ned, everybody,

First, congratulations on the pre-release!

Using OpenCraft.Hosting, we've been testing the process of

* deploying production instances that run Eucalyptus RC1, and
* upgrading production instances from Dogwood to Eucalyptus RC1.

Deploying Eucalyptus RC1 worked really well! With our setup, we only needed to add a small number of changes on top of the open-release/eucalyptus/1rc1 release. For most of these changes, we already have PRs open to submit them to the configuration and edx-platform repos (they are not specific to Eucalyptus; we were using them for our Dogwood instances as well). The remaining changes are minor and specific to our setup.

Upgrading from Dogwood to Eucalyptus RC1 went mostly fine, but we did have to run some of the commands from the update.sh script manually to complete the process (see below).

For context, our upgrade process is to provision a new server for an existing instance from scratch using the edx_sandbox.yml playbook (duplicating the instance's settings), and then switching over the DNS. The instance uses MySQL and Mongo databases located on a different server.

With this approach, provisioning goes fine up to the oauth_client_setup role, which fails with django.db.utils.OperationalError: (1054, "Unknown column 'oauth2_client.logout_uri' in 'field list'"):

https://gist.github.com/itsjeyd/f824918df495586dd21225985cd67ee1

To end up with a working server, we need to run

/edx/bin/edxapp-migrate-lms --fake oauth2_provider zero
/edx/bin/edxapp-migrate-lms --fake-initial

on the new server and then provision another VM for the same instance.

---

Another thing that was nice to see was that switching from using courseware_studentmodulehistory to coursewarehistoryextended_studentmodulehistoryextended for CSMH data was really easy. For the setup where existing data remains in courseware_studentmodulehistory and new data gets written to coursewarehistoryextended_studentmodulehistoryextended, we only had to add

EDXAPP_FEATURES:
  ENABLE_CSMH_EXTENDED: true
  ENABLE_READING_FROM_MULTIPLE_HISTORY_TABLES: true

to an instance's settings before provisioning a new server. No manual intervention necessary! :)

---

Thanks for the great work on this release!

--
Tim
@OpenCraft

[TJ Keemon]

Hey Pierre-

I think we're working through the same problem as you, but we attempted to install on GCloud. Here's the full error that we saw:

https://gist.github.com/tkeemon/868f320dc94a4ed825aee84047c0501f

Ned-

Do you want us to keep this discussion here or should we spin it off into another thread? 

-TJ

[Pierre Mailhot]

Hey TJ,

I have tried a second time and got the same error.

So since it looked like a numpy / scipy issue, I decided to try another route.

I tried this base on what I found on http://stackoverflow.com/questions/19693302/numpy-install-under-ubuntu-12-04-causes-python-importerror

sudo apt-get install python-numpy python-scipy libblas-dev liblapack-dev gfortran python-dev
sudo pip install numpy --upgrade
sudo pip install scipy --upgrade

I then restarted the installation script. It takes a long time to install on the m3.large instance I am using right now, but I will be updating you soon if it works or not...

[Pierre Mailhot]

TJ, that fixed it for me.

Ned, you may want to consider upgrading numpy and scipy as part of the upgrade from Dogwood to Eucalyptus.

My quick fix installed numpy-1.11.1 and scipy-0.18.0.

[TJ Keemon]

Hi Pierre-

I ran the first line:

sudo apt-get ...

Then reran the sandbox.sh script and it looks like I'm past the task that was crashing. But the provisioning hasn't finished, so I'm not home free yet. 

I wouldn't think those two upgrade commands for numpy and scipy would do anything. Those are upgrading the OS versions of the packages, while the Ansible task was installing numpy and scipy into venvs in /edx/app/edxapp/venvs/*

For my deployment, the versions installed in the venvs (edxapp and edxapp-sandbox) are:

scipy==0.14.0

numpy==1.6.2

Regardless, something in that apt-get command fixed my situation. 

Thanks for your help!

[Ned Batchelder]

Omar,

Our plan is to make a separate pair of Eucalyptus resources on Transifex.  These would stay in-sync with the Eucalyptus code, while the rest of the resources would continue to follow master.  Does that help with the translation?

--Ned.

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/a7d35b80-69ba-4812-822a-990ef999d9d2%40googlegroups.com.

[Omar Al-Ithawi]

Ned,

> Our plan is to make a separate pair of Eucalyptus resources on Transifex.  These would stay in-sync with the Eucalyptus code, while the rest of the resources would continue to follow master.  Does that help with the translation?

Initially this solution looked promising and thought it would help me, but in practice it required me and re-translate all the strings in Transifex which is a very costly process. I thought that Transifex would consider a 100% matched string as one. But that's not the case, and it turned out that you'd have to re-translate the same string in dogwood and non-dogwood.

Although we're using dogwood now, we're still depending on the latest translations and using some magic to help with that.

We're still looking for other options, since translations are a major part of supporting non-English learners.

Thanks,

--
You received this message because you are subscribed to a topic in the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CAGtJPNPUkT7bbw04x8_%2B7D3HiO%2B1o6VXDCUgaXN3Z-RCmPCuUQ%40mail.gmail.com.

--

Omar Al-Ithawi

Senior Engineer, Edraak.org

Queen Rania Foundation

for Education & Development

T +962 6 4016464  Ext. 700

M +962 7 90574405

E oit...@qrf.org

[Pierre Mailhot]

Tobias,

Thank you for the explanation and the PR.

As we do have our own fork at EDUlib, I did not encounter this issue after my rebase of the various repositories and upgrading from Dogwood to Eucalyptus.

I've only encountered this issue when installing a fresh install with the edX repositories when I wanted to check a problem I was dealing with after the upgrade.

It's good to know I was not imagining things :-)

On Wednesday, August 3, 2016 at 4:54:45 PM UTC-4, Tobias Macey wrote:

So, to provide a bit more detail here...

I ran into this issue last week when doing a clean rebuild of some EdX boxes. The root of the issue is that SciPy released version 0.18 last Tuesday, which requires Numpy version >=1.7.2. EdX is pinning Numpy to 1.6.2, but the dependencies listed in local.txt in requirements/edx-sandbox/local.txt have scipy in their `install_requires` attribute of setup.py without any version specifier. This causes 0.18 to be pulled in which fails to build because numpy is already present but at a version that is too old.

My solution was to issue this PR: https://github.com/edx/edx-platform/pull/13136

I agree, however, that as many dependencies as possible should be updated as part of the Eucalyptus release.

[Tobias Macey]

So, to provide a bit more detail here...

I ran into this issue last week when doing a clean rebuild of some EdX boxes. The root of the issue is that SciPy released version 0.18 last Tuesday, which requires Numpy version >=1.7.2. EdX is pinning Numpy to 1.6.2, but the dependencies listed in local.txt in requirements/edx-sandbox/local.txt have scipy in their `install_requires` attribute of setup.py without any version specifier. This causes 0.18 to be pulled in which fails to build because numpy is already present but at a version that is too old.

My solution was to issue this PR: https://github.com/edx/edx-platform/pull/13136

I agree, however, that as many dependencies as possible should be updated as part of the Eucalyptus release.