docker container for edx (devstack, prodstack)

[Kiran Patil]

Hello Friends,

It is been one week or so trying to install devstack on Fedora 20 and still not yet finished.

Vagrant has its own niche but I am looking leap further from devops perspective.

How about having a docker container for both dev(devstack) and ops(prodstack) ?

I am not saying this as naive user but I am seeing lots of big Freesoftware applications are moving to Docker for devstack and prodstack.

I am seeing how they are benefiting from that just one click/command and done attitude. For example Discourse is one of the finest software I have seen and they have migrated to Docker and I am seeing its benefit as a devops, https://github.com/discourse/discourse_docker and https://github.com/discourse/docker_manager

I believe with the rate edx is developing we need a technology, where a single command/click installs either devstack or prodstack.

IMHO, we should go for Docker support, since it will save hell lot of time for once and all and make devstack and prodstack bringup less painfull.

Thanks.

[Nate Aune]

Hi Kiran,

We have created a Docker image using the latest edx codebase (probably less than a week old) which you can find here:

https://registry.hub.docker.com/u/appsembler/edx/

I would love to see greater support for Docker in edX!  Also, just a plug for the Open edX launcher service which lets you spin up a fully functioning Open edX site in about 5-10 seconds. http://launcher.appsembler.com/openedx   Thanks to everyone who has already given us feedback about it!

 

Nate

[Claude Coulombe]

Greetings,

docker containers are very cool... indeed

docker has just reached version 1.0 and Google has announced the support for docker image in App Engine yesterday

docker could probably be a solution for the future... 

Claude

[Ahmad Abd Arrahman]

It would be great if you can share the Dockerfile used for this image,
and want to know If I need to use separate data image for mysql?
and any advice is very appreciated

Thanks

[Ahmad Abd Arrahman]

Can you check the repo, I cannot pull it!!

using docker pull appsembler/edx not working with this image, although it is working well elsewhere 

On Tuesday, June 10, 2014 1:11:45 AM UTC+2, Nate Aune wrote:

[Nate Aune]

Sorry, can you use this URL instead? https://registry.hub.docker.com/u/appsembler/edx-docker-base/

So the command you would run is:

docker pull appsembler/edx-docker-base

[Ahmad Abd Arrahman]

Thanks Nate,
but this one is outdated and using old release of edx as I tried it a month ago.

It would be appreciated if there is a more recent one.
And I want your advice for handling data "mysql&mongo"

Thanks in advance
Ahmad

[Nate Aune]

I wanted to let you know that we've updated our Docker image to the latest official Aspen release. You can find this Docker image here:

https://registry.hub.docker.com/u/appsembler/edx-lite/

Or if you're just looking to try Open edX, you can use our Docker-powered testdrive service. It takes about 30 seconds to spin up a new site.

Go to http://openedx.appsembler.com and click on "Get a free demo site"

Tomorrow and friday at the Open edX hackathon, we're going to be working on a multi-container Docker solution that will have the Open edX app server in one Docker image and the datastores (MySQL and MongoDB) in separate Docker images, which will make it easier to upgrade and scale.

https://openedx.atlassian.net/wiki/display/OPEN/Ideas+for+Open+edX+Conference+Hackathon

Feedback most welcome on the Docker image and the test-drive service!

Nate

[Yarko Tymciurak]

I'm (sadly) not at there this week, but if there's some way to help with this particular (docker) effort, tmux-ing in or whatever, I'm happy to give some hours to this effort.

[Yarko Tymciurak]

...actually, now that I've had a moment to glance at some of the projects, I see quite a few interesting ones:

• docker;
• upgrade django (1.7 ==> then python 3.4+ would be much fun!)
• bok-choy
• x2go (I've repeatedly dreamed of being able to take a "box" into a workshop, and present quizes live)

[Andy Sio]

Have you tried to use Amazon Beanstalk to run the docker? I tried it and installed successfully according to Amazon. But, I am not able to run.  This is the Dockerrun.json file that I use in Amazon beanstalk.  What's your thoughts?

{

"AWSEBDockerrunVersion" : "1",

"Image": {

"Name": "appsembler/edx-lite"

},

"Ports" : [

{"ContainerPort":"8080"}

] 

[Nate Aune]

Hi Andy,

Thanks for trying to get the edx-lite Docker image working on Amazon ElasticBeanstalk. I started looking at Beanstalk a few weeks ago, but didn't get very far. In looking at your port specification, it appears you have the wrong port defined.

Look at the wiki page for how to make an edX Docker image:

https://github.com/appsembler/configuration/wiki/Creating-a-Docker-image

The port specs are as follows: "PortSpecs":["80", "18010", "18020"]

So try changing 8080 to 80 and see what happens.

Nate

--

Nate Aune - nate...@gmail.com

http://www.nateaune.com

+1 (617) 517-4953

[Andy]

Great.  I changed the port to 80 and it looks like I can connect to the image.  But, looks like there is some error.  edx-env.elasticbeanstalk.com.  I am new to the docker so that I need to know how to see the exact error.  But, it's pretty easy. Once i create my json. AWS just automatically downloaded your docker image and setup the whole thing for me.

Andy

[Andy]

So, I am able to get into the container to see what's going on.  This is what i got after attaching the container to take a peek from AWS.  See below.

2014-12-03 15:23:02,628 INFO success: lms entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-12-03 15:23:02,628 INFO exited: lms (exit status 1; not expected)
./run: 22: ulimit: error setting limit (Operation not permitted)
./run: 6: ulimit: error setting limit (Operation not permitted)
2014-12-03 15:23:03,446 INFO spawned: 'lms' with pid 6729
2014-12-03 15:23:03,447 INFO success: forum entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
./run: 6: ulimit: ./run: 22: ulimit: error setting limit (Operation not permitted)
error setting limit (Operation not permitted)
2014-12-03 15:23:04,460 INFO success: lms entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-12-03 15:23:04,530 INFO exited: lms (exit status 1; not expected)
2014-12-03 15:23:04,560 INFO spawned: 'lms' with pid 6742
2014-12-03 15:23:04,590 INFO exited: forum (exit status 1; not expected)
./run: 6: ulimit: error setting limit (Operation not permitted)
./run: 22: ulimit: error setting limit (Operation not permitted)
2014-12-03 15:23:05,157 INFO spawned: 'forum' with pid 6751
2014-12-03 15:23:05,594 INFO success: lms entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-12-03 15:23:05,594 INFO exited: lms (exit status 1; not expected)
./run: 6: ulimit: error setting limit (Operation not permitted)
./run: 22: ulimit: error setting limit (Operation not permitted)
2014-12-03 15:23:06,429 INFO spawned: 'lms' with pid 6834
2014-12-03 15:23:06,429 INFO success: forum entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
./run: 22: ulimit: error setting limit (Operation not permitted)
./run: 6: ulimit: error setting limit (Operation not permitted)
2014-12-03 15:23:07,443 INFO success: lms entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-12-03 15:23:07,510 INFO exited: lms (exit status 1; not expected)
2014-12-03 15:23:07,533 INFO spawned: 'lms' with pid 6847
2014-12-03 15:23:07,561 INFO exited: forum (exit status 1; not expected)
./run: 22: ulimit: error setting limit (Operation not permitted)
./run: 6: ulimit: error setting limit (Operation not permitted)
2014-12-03 15:23:08,133 INFO spawned: 'forum' with pid 6856
^C2014-12-03 15:23:08,403 WARN received SIGTERM indicating exit request
*** Shutting down runit daemon (PID 11)...
2014-12-03 15:23:08,404 WARN received SIGTERM indicating exit request
2014-12-03 15:23:08,404 INFO waiting for devpi-server to die
2014-12-03 15:23:08,406 INFO waiting for lms, cms, forum to die
2014-12-03 15:23:08,409 INFO stopped: devpi-server (terminated by SIGTERM)
/usr/bin/mysqladmin: connect to server at 'localhost' failed
error: 'Access denied for user 'debian-sys-maint'@'localhost' (using password: YES)'
2014-12-03 15:23:08,541 INFO success: lms entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2014-12-03 15:23:08,610 INFO exited: lms (exit status 1; not expected)
2014-12-03 15:23:10,434 INFO stopped: forum (exit status 1)
2014-12-03 15:23:10,604 INFO stopped: cms (exit status 0)

[Nate Aune]

Hi Andy,

It seems as though AWS Elastic Beanstalk's Docker is more strict about ulimit settings, which controls the number of open files you can have.

Here is someone who ran into the same problem - no response yet on Stackoverflow:

http://stackoverflow.com/questions/27007769/how-to-increase-ulimit-in-docker-in-elastic-beanstalk?lq=1

Nate

[Andy]

I have to say using the Docker is the best testing, developing, and deploying strategy for EdX.  I tried to use your docker images locally and they work flawlessly.  On the other hand, I tried to follow the development stack instruction and just couldn't to get it work.  Kind of frustrating.  Thank you for the docker image!!! 

[Nate Aune]

Aww.. thanks Andy! I'm glad that our Docker images worked out for you. Did you ever manage to get it working on AWS Elastic Beanstalk?

Nate

[Andy]

I followed your instruction up to the SSH point.  here

But, I am not able to ssh to the container. It doesn't give me any warning, it just hangs there.  I tried to use "docker attach containerID" But, I couldn't get there. Anythoughts?

Andy   

[Andy]

The problem is probably caused by this.

http://webiphany.com/technology/2014/06/12/what-ip-do-i-access-when-using-docker-and-boot2docker.html

[Nate Aune]

Yeah, those instructions were written for someone not using boot2docker. We should probably make a version of the docs for folks using boot2docker.

The short answer is that if you're on a newish Docker client, you can use the 'docker exec' command to "ssh" into the container.

https://docs.docker.com/reference/commandline/cli/#exec

What were the steps to get edx-lite running on AWS Elastic Beanstalk?

Nate

[Andy]

I haven't successful get the edx-lite running on AWS Elastic Beanstalk yet.  In fact, I am trying to create the docker image and try it again.  There are just some problem with using boot2docker.  I switched to use Vagrant to start the phusion/baseimage to build your edx-lite.  Working under Vagrant is a lot better since it supports docker.  

[Andy]

I decided to document mystep. This may help people in the future.

1. Download the Vagrant file from here to setup your phusion/baseimage in the Vagrant

2. Vagrant up

3. Vagrant ssh

4. Follow your steps starting from "Inside the Docker container"

5. Install the edXLite.  I have to use sudo to run "ansible-playbook -vv -c local -i "127.0.0.1," docker_lite.yml"

6. Things go pretty well, except i get the following error.  But, looks like i get some error in the nginx.

TASK: [nginx | make sure nginx has started] *********************************** 

<127.0.0.1> REMOTE_MODULE service name=nginx state=started

failed: [127.0.0.1] => {"failed": true, "item": ""}

msg: service not found: nginx

FATAL: all hosts have already failed -- aborting

[longso...@gmail.com]

Hi, after run the docker， what should I do to get all the service up and use edx???

在 2014年6月10日星期二 UTC+8上午7:11:45，Nate Aune写道：

[JITHIN SHA]

Hello Nate,

The docker pull commands were failing for me.

docker pull appsembler/edx

Using default tag: latest

Error response from daemon: manifest for appsembler/edx:latest not found

docker search edx

NAME                        DESCRIPTION                                     STARS               OFFICIAL            AUTOMATED

appsembler/edx-full         Open edX is an open source MOOC platform tha…   20                                      

appsembler/edx-lite         Open edX is an open source MOOC platform tha…   17    

docker pull appsembler/edx-lite

Using default tag: latest

Error response from daemon: Get https://registry-1.docker.io/v2/appsembler/edx-lite/manifests/latest: Get https://auth.docker.io/token?scope=repository%3Aappsembler%2Fedx-lite%3Apull&service=registry.docker.io: net/http: TLS handshake timeout

docker pull appsembler/edx-full

Using default tag: latest

Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: TLS handshake timeout

Can you please share the exact place to get the image.

Thanks

J

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/060932c5-b515-4016-8547-d124be19b582%40googlegroups.com.

--

JITHINSHA
ജിതിന്‍ഷാ കെ എസ്

[Nate Aune]

Hi Jithin, That Docker image was created in 2014 (5 yrs ago), so I would not recommend using it anymore.  If you want to run Open edX in Docker, I recommend that you use one of these projects: https://github.com/edx/devstack https://github.com/regisb/tutor Nate

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CANtXGwfKmgJyvmJpH5DHmTM_BK%2BGO%3DSQ-bj8BzAy4jUdrb_xFQ%40mail.gmail.com.

Website: http://www.appsembler.com

Blog: http://www.appsembler.com/blog

Twitter: http://twitter.com/natea

LinkedIn: http://linkedin.com/in/natea

Phone: +1 (617) 702-4331

[JITHIN SHA]

Hello Nate,

Thanks Man... I tried seeing the followup query. In fact now i checked the date when you made the post. So sorry man....

Yeah am trying a little bit of openedx using native installation for the time being. Will try the containers as well. 

Have a good day.

Regards

J

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/06e8263d-19f7-dfcf-0dd7-1e024bff126d%40mixmax.com.