Inquiries regarding the installation of edx separation

[DoWon Kim]

Hello, I am Do Won Kim who lives in Korea.

I am writing because I have a question about open edx. I want to install lms and cms of edx on different servers.

So I searched the web countless times, but I couldn't find the answer.

Is there a manual that can be installed separately for lms and cms?

If there is a solution, please reply. Or, I'd appreciate it if you could send me a reference document related to this.

Thank you, and have a nice day.

[JITHIN SHA]

Hello Bro,

Myself Jithin, have tried openedx as part of a client requirement. Am sharing what i know about the setup.

Basically the openedx is provided with all in one installation, either in native installation or devstack. If you want to separate lms and cms, its possible but you have to do it yourself. I have separated openedx as app server and db servers. 

App includes cms and lms - it also has some other services like nginx, rabbitmq, elasticsearch , supervisor and so on.

DB has just the DBs ( mongodb and mysql )

All the lms and cms configurations are done on the lms.env.json, lms.auth.json and similarly cms as well.

Also there is few configurations inside the etc dir of edx. 

All you can get from the official document is how to play with the all in one setup. Based on that you have to assume and sort out the rest based on your requirement.

Hope you got a fair idea of the stuffs.

NB: Request to openedx experts : Please correct me, if am wrong in anything.

Thanks

J

--
You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/7bbbc3b3-2095-4821-97c0-a928db7d2d9b%40googlegroups.com.

--

JITHINSHA
ജിതിന്‍ഷാ കെ എസ്

[DoWon Kim]

JITHIN SHA, thank you for your reply..!

How did you separate the edx service and db(mysql, mongoDB)? 
Did you separate the edx from the installation?

Or did you work to separate the DBs after installation?

Can you tell me how you separated them?

Please help me with my troubles.

2019년 3월 19일 화요일 오후 11시 21분 36초 UTC+9, JITHIN SHA 님의 말:

[JITHIN SHA]

Hello Bro,

1. Installed one VM with all in one setup. 

2. Dumped the dbs (mongo and mysql). 

3. Stopped the db services and made this VM as App server. 

NB: Note the version of MongoDB server and MySQL server.

4. Setup another VM for db, install the versions of DB's found from the last step. 

5. Restore the DB's here. 

6. Configure the DB's so that users can access from the remote App server.

Now its time to make modifications in the configuration files of App server, so that edxapp can communicate.

If you check the mysql database, you can see some users there. Actually these users are used for App and DB communication.

7. Modify the host, passwords field in the lms.auth.json file. As i said earlier, there are few file in etc dir which are also required to modified like xqueue and so on.

Once the modifications are made you can restart supervisor services. You can watch the logs to find if there requires more modifications /edx/var/log/lms/edx.log.

This is how I made it. May be you can install without the all in one option. Am not so sure about that.

Tricks : During the installation of openedx all in one installation, there is a step where you can create encrypted passwords for DB users. I created a script to automatically create the users with encrypted passwords file. So before dumping the DB, i ran the script to create the users. After that i dumped the dbs and restored it. The only thing left was to modify the config files in edxapp. I have also made a small script for that too. Actually creating the script made my job very easy. 

It took like more than a month for me to sort the things. But openedx is clearly an open source code. You just needs to figure it out. 

Hope you have got a fair idea to how to deal with it.

Let me know if you need any more info.

Thanks,

J

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/973517e3-0163-49e8-a88b-b8cfcf98ccf4%40googlegroups.com.

[DoWon Kim]

Thank you very much for your specific response. You have given me hope. I'll try it.

If I have any questions, I'd like to ask you by e-mail. Could you tell me your e-mail?

2019년 3월 20일 수요일 오후 2시 43분 48초 UTC+9, JITHIN SHA 님의 말:

[Régis Behmo]

@Do Won Kim I'm curious, what tool(s) do you use for deploying on multiple servers? Ansible, Chef, Puppet, Terraform, Docker swarm, Kubernetes? Carefully crafted bash scripts? Something else?

As the principal maintainer of Tutor (https://github.com/regisb/tutor), I'm currently considering my options for deployment of Tutor/Open edX on multiple servers. There are many different ways to do this and I'm trying to figure out the best approach for the community. Btw, I'm also interested in answers from other people in the community :)

[Nate Aune]

At Appsembler, we use Terraform to provision the server resources (VMs, storage, etc) and Ansible to do the actual deployment of the Open edX software and all its dependencies.

The nice thing about Terraform is that it works on all major cloud providers (AWS, Google Cloud, Azure, Digital Ocean, etc) so if you create the scripts once you can reuse a lot across different providers.

There’s also a Terraform SaaS service available now: 

https://app.terraform.io/signup

--

You received this message because you are subscribed to the Google Groups "General Open edX discussion" group.

To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/2655bd37-1d63-407f-aa51-254bfe1f56c1%40googlegroups.com.

--

Website: http://www.appsembler.com

Blog: http://www.appsembler.com/blog

Twitter: http://twitter.com/natea

LinkedIn: http://linkedin.com/in/natea

Phone: +1 (617) 702-4331

[stan.v...@exlinc.com]

@Régis Behmo we built a one-click deployment POC last year on Kubernetes (OpenShift) using Ansible. I see that you coded a nice k8s.py script – considering that Ansible is written in Python, it’s all essentially the same, of course. Some advantage of using Ansible is that it enables you covering all technologies in scope (e.g., MySQL, MongoDB) from a single tool, and it has a deployment/workflow management system Tower (the open source version of it). Somewhat unfortunate, a couple of years ago, Ansible zigzagged into a patch of complete disarray, versions and styles incompatibility, which resulted in many people criticizing the tool, and rightfully so. However, I think that their late-2018 software is well-designed and stable. Working with OpenShift, we saw how the product team re-wrote the Ansible books completely, dropped the old-style ones and produced a simple, clean and fast installation suite for quite a complex product base. Just wish EDX can do that one day… So, I would give Ansible a chance. Although, as we know, the era of installers and installable software fades away.

In the POC, we had a complete system rolling out from scratch in under 5 min. Of course, we used a pre-migrated MySQL database. With all due respect to the migration concept of Django, the incremental one is not sustainable when doing multiple rollouts, and completely not necessary.

One area of concern was the location of the auxiliary services such as Memcached and RabbitMQ inside the k8s cluster – to avoid the latency of traffic when placing components across disparate hardware units and data centers. It probably makes sense co-locating Memcached in the same pods with the Django servers. RabbitMQ is a relic from the past. If we keep the LMS and CMS close to each other and put RabbitMQ near-by as well – we end up with huge pods or tight placement rules that defeat the purpose of doing k8s. So, probably, just bite the bullet and place RabbitMQ anywhere and absorb the latency and cross-zone traffic cost.

Lastly, image build can also be hugely optimized to cut down on the deployment effort for minor code fixes, although that involves image updates, which some folks think are anti-docker. Bottom line, adopting VM-centered designs to modern techs is not easy, but possible.