Docker Devstack Beta

[Brian Beggs]

Today we would like to announce the rollout of the Docker Devstack Beta to the edX Open Source community. The team is close to feature parity with Vagrant Devstack and we would like to being the process of rolling out Docker Devstack and gather feedback.

What do you need to do?

Over the next few weeks/sprints we are asking everyone to stop using Vagrant Devstack and start using Docker Devstack.

How do you get started with Docker Devstack?

Instructions on installing docker and getting an edX Devstack built can be found here:

https://github.com/edx/devstack#edx-devstack-

Need help?

Issue the commands:

make stop

make pull

and try what you were doing again.

Need more help?

The slack channel will be the best place to get help: #docker

Found an issue?

Please file any issues you find in the PLAT project in JIRA https://openedx.atlassian.net/projects/PLAT/issues. You may also submit a pull request to the github repository: https://github.com/edx/devstack

What is going to happen to Vagrant Devstack?

We are in the process of phasing out vagrant devstack. It will still be available to use through the Hawthorn release (end of Q1/September 2017), but support will end thereafter.  

Does this work with PyCharm?

Test execution and debugging of platform tests via pycharm does work: https://github.com/edx/devstack/blob/master/docs/pycharm_integration.rst#pycharm-integration

What is Missing?

- A few JS and Acceptance are failing around variable bitrates with the video player. (99% of tests do run and pass.)

- Bokchoy debugging in pycharm 

- Javascript debugging in pycharm

- Forums does not yet run on Docker Devstack.

(if you would like to help documenting or getting these things running, please stop by the #Docker slack chat channel)

Thank you

A lot of people have worked really hard on this for the past few months. Thank you to Clinton, Rob, Jeremy, Joel, George, Renzo and everyone that has contributed to the project.

If you run in to difficulty please do not hesitate to reach out for help. The team will be monitoring the channels above and will be able to assist with any setup troubles. And please let me know if you have any questions.

Thanks,

Docker Devstack Team

Brian Beggs

edX | Principal Software Engineer

141 Portland Street (9th Floor) 

Cambridge, MA 02139

[Ned Batchelder]

I am really excited to see the Docker Devstack work progressing.  I'll just clarify: Docker Devstack is meant for developers working against the latest code on master of edx-platform and other repos.  Ginkgo will not have Docker support: it will ship with a Vagrant-based devstack just as the previous releases have.

Our plan is that Docker Devstack will be supported in Hawthorn. We'll evaluate whether to support both Docker and Vagrant for devstack in that release.  Also unknown is whether there will be a Docker fullstack, and what will happen to the Vagrant-based fullstack.

Thanks for listening, and happy hacking,

--Ned.

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/5BF51543-A21E-4162-B2F8-07A169AA4F83%40edx.org.

[dan...@opencraft.com]

Hi,

I am a new Open edX developer (I recently joined OpenCraft), and I tried edx with docker (https://github.com/edx/devstack), and I'm surprised at how easy it was to get it running. Only bad thing: the setup is slow (although automatic).

Instructions:
1. First you need to install docker 17.06. I use Devuan GNU/Linux (Debian without systemd) and it was easier than expected: uninstall old docker, download a .deb from https://download.docker.com/linux/debian/dists/stretch/pool/stable/amd64/, dpkg -i, and it worked.
2. Install the command docker-compose, by following https://docs.docker.com/compose/install/#alternative-install-options (e.g. by pip)
3. Clone from https://github.com/edx/devstack
4. Follow the instructions there (make dev.clone, make dev.provision). The provisioning command takes a long time (hours), but it's all automatic. This process is very lean on memory
5. Start the services (make dev.up). Automatical
6. Access URLs, e.g. studio is at http://localhost:18010/, e-commerce at http://localhost:18130/dashboard/, etc. Use user described in admin

Good from docker-edx:
- everything straight-forward, all instructions work, you just need to wait
- very lean on memory (RAM) all the time, even when it's building lots of stuff. Runs very fast and is usable with 4 Gb, with only occasional hangs
- scalable configuration, very easy to read and understand. The repository https://github.com/edx/devstack itself is made of small and simple configuration files (starting with docker-compose.yml). You always understand what docker is doing and why. You get the power of docker: separate container, container inheritance, …
- simple to debug, e.g. to get a log for LMS: docker-compose logs lms
- you get everything included (also e-commerce) and already running from the start, each service in a URL. And a default user for each service
- super-fast start and stop of all containers („make dev.up“, „make down“), more than vagrant

Bad from docker-edx:
- the first step („make provision“) is very slow, much more than with vagrant, it took around 1'5 hours in my case.  It's because it runs every migration for every project, compiles assets, … but also because it includes more components (e-commerce, …) It seems that with Vagrant, you download everything wholly baked, but with docker you build it. Luckily it's just one time, and it's worth the wait. There are also workaround described in the README (using DB dumps instead of migrating)
- I interrupted the provisioning step and had to start it from the beginning.
- I saw some strange error messages, like „No space on disk“ when I actually had enough, but they didn't prevent services from running. And some of the common errors are covered in the README

So from what I have tested: the provisioning configuration is very comfortable to read, write, and understand; you can keep the configuration of all services in version control… It was very instructive and much faster and complete than vagrant. And things work. Only bad thing is the waiting time the first time.


I attach an example of what you get when you start all services:

: dc; /w/devstack ; make dev.up
docker-compose -f docker-compose.yml -f docker-compose-host.yml up -d
Creating network "devstack_default" with the default driver
Creating edx.devstack.chrome ...
Creating edx.devstack.memcached ...
Creating edx.devstack.firefox ...
Creating edx.devstack.mongo ...
Creating edx.devstack.memcached
Creating edx.devstack.mysql ...
Creating edx.devstack.elasticsearch ...
Creating edx.devstack.chrome
Creating edx.devstack.mongo
Creating edx.devstack.firefox
Creating edx.devstack.mysql
Creating edx.devstack.mongo ... done
Creating edx.devstack.studio ...
Creating edx.devstack.credentials ...
Creating edx.devstack.lms ...
Creating edx.devstack.ecommerce ...
Creating edx.devstack.elasticsearch ... done
Creating edx.devstack.discovery ...
Creating edx.devstack.ecommerce
Creating edx.devstack.discovery
Creating edx.devstack.lms
Creating edx.devstack.studio ... done


Note that it builds what it finds in ../edx-platform, so if you put there your own fork with modifications, it will build it in the same way.
I still didn’t try deploying themes, etc.

So with this I recommend more people to use Vagrant for development. It was simple enough for an openedx beginner, very instructive, and also the only comfortable way to run several openedx components in a laptop with just 4 Gb of RAM; at the same time it simplifies configuration. We need more people using it.

Greetings,

Daniel Clemente Laboreo
@OpenCraft