Using Docker for deploying Arches

[Rita Sinha]

Hi,

 I have been recently lurking around open source projects when I came across  The Arches Project and developed keen interest in contributing to the org.

After going through the project, I'm eager to improve the existing installation of Arches.

After going through the current installation guide, I would like to suggest using Docker to deploy Arches. 

For those not familiar with docker, can take the reference from here-

http://en.wikipedia.org/wiki/Docker_(software)

Regards,
Rita Sinha

[Van Daele, Koen]

Hi Rita,

welcome to the Arches project.

I'm aware about what Docker is. It does seems like it might be the way of the future for easier installations. But I've never used it myself. Do you have a lot of experience with it? Suggestions for the team if they want to move in that direction? Or do you want to do contribute in making this happen?

Farallon are currently finishing up Arches 3, so it would probably be a thing for that release, not for the older ones.

Cheers,
Koen
________________________________________
Van: arches...@googlegroups.com [arches...@googlegroups.com] namens Rita Sinha [rita.s...@gmail.com]
Verzonden: donderdag 8 januari 2015 14:53
Aan: arches...@googlegroups.com
Onderwerp: [Arches] Using Docker for deploying Arches

--
-- To post, send email to arches...@googlegroups.com. To unsubscribe, send email to archesprojec...@googlegroups.com. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en
---
You received this message because you are subscribed to the Google Groups "Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archesprojec...@googlegroups.com<mailto:archesprojec...@googlegroups.com>.
For more options, visit https://groups.google.com/d/optout.

[Rita Sinha]

HI Van,

I have used docker as a build system for building/testing firmware
and drivers over different Linux distors.

Docker is a very recent technology and I can't say I have a lot of
experience over it but yes, I would want to contribute to it to make
it happen for Arches if the community stands in support.

I was intending to take it as GSOC-2015 project which would be
starting in a month or so. By that time we can do a POC and get
community review over the deliverable solution for Arches.

Regards,
Rita SInha

[Alex Proca]

Hello,

I started dockerizing the arches project but abandoned project due to lack of time. If you want to continue from where I left or if you want a starting point have a look at https://bitbucket.org/alexproca/arches

[Rita Sinha]

Hi Alex,

Thanks for the much needed start for dockerizing the arches
project. I have gone through your work and prefer starting from where
you left.

I am too having some issues with dedicating a lot of time on this
project presently but plan to actively work on it in summers and
just wanted to familiarize with the Arches developer community and the
project as a whole.

Regards,
Rita sinha

> --
> -- To post, send email to arches...@googlegroups.com. To unsubscribe,
> send email to archesprojec...@googlegroups.com. For more
> information, visit https://groups.google.com/d/forum/archesproject?hl=en
> ---

> You received this message because you are subscribed to a topic in the

> Google Groups "Arches Project" group.

> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/archesproject/LwYW_TTLpcc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> archesprojec...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Joel Aldor]

Has anyone continued the dockerization of Arches 3?

[Vincent Meijer]

Yes, we at USF's CVAST are in a far stadium of dockerizing Arches. After we get everything ready and stable, I will share. Stay tuned!

[Joel Aldor]

Thank you very much Vincent! We're looking forward! 

[Jae Hun Lee]

Hello, Vincent

I'd like to share my work regarding this subject. I hope you can improve with my Docker file: http://goo.gl/pcsXug
My Arches docker image is called oixhwotl/arches3-v1:15.10
(15.10 is a tag and it means it was built based on Ubuntu 15.10)
(v1 means that everything is in one container. I planned that v2 contains minimal files and works with other containers such as ElasticSearch, PostgreSQL and nginx.)

HOW TO USE IT

1. Install docker application

2. Download the image
$ docker pull oixhwotl/arches3-v1:15.10

3.  Copy out my_Hip_app to local storage if you don't have your own app
(If you have your own files, then this step is not necessary. <local storage path> represent the path to your files)
$ docker run -it oixhwotl/arches3-v1:15.10
# exit
$ docker ps -a
(get the image contianer id)
$ docker cp <container id>:/roiot/my_hip_app <local storage path>

4. Run server mounting your app
$ docker run -it -p 8000:8000 -v <local storage path>:/root/my_hip_app --name arches oixhwotl/arches3-v1:15.10
# source .profile
# start_services.sh
(this will start PostgreSQL and ElasticSearch)
# cd my_hip_app
# python manage.py packages -o install
# python manage.py runserver 0.0.0.0:8000

Before exiting the container, save your data by following the instructions in this post: https://groups.google.com/forum/#!searchin/archesproject/script%7Csort:relevance/archesproject/sXu6qpMuoAs/qDKQlQstIwAJ

5. To run again
$ docker start arches && docker attach arches
# source .profile
... and you know what to do


The data directory of PostgreSQL is also exportable, and you can create persistent data container for it.
https://docs.docker.com/engine/tutorials/dockervolumes/

For Windows, use following command to check and forward the connection
> netsh interface portproxy show all
> netsh interface portproxy add v4tov4 listenport=80 connectport=8000 connectaddress=<docker ip>
> netsh interface portproxy del v4tov4 listenport=80


Good luck Vincent.

[Joel Aldor]

Hi Jae, 

Thank you very much for this one. Looking into your Dockerfile code, I suppose we still have to look forward to your v2. This is a really great development, though! :)

Regards,

Joel

On Tuesday, August 23, 2016 at 1:21:17 PM UTC+8, Jae Hun Lee wrote:

Hello, Vincent

I'd like to share my work regarding this subject. I hope you can improve with my Docker file: http://goo.gl/pcsXug
My Arches docker image is called oixhwotl/arches3-v1:15.10
(15.10 is a tag and it means it was built based on Ubuntu 15.10)
(v1 means that everything is in one container. I planned that v2 contains minimal files and works with other containers such as ElasticSearch, PostgreSQL and nginx.)

HOW TO USE IT

1. Install docker application

2. Download the image
$ docker pull oixhwotl/arches3-v1:15.10

3.  Copy out my_Hip_app to local storage if you don't have your own app

$ docker run -it oixhwotl/arches3-v1:15.10
# exit
$ docker ps -a
(get the image contianer id)
$ docker cp <container id>:/roiot/my_hip_app <local storage path>

4. Run server mounting your app
$ docker run -it -p 8000:8000 -v <local storage path>:/root/my_hip_app --name arches oixhwotl/arches3-v1:15.10
# source .profile
# start_services.sh
(this will start PostgreSQL and ElasticSearch)
# cd my_hip_app
# python manage.py packages -o install
# python manage.py runserver 0.0.0.0:8000

Before exiting the container, save your data by following the instructions in this post: https://groups.google.com/forum/#!searchin/archesproject/script%7Csort:relevance/archesproject/sXu6qpMuoAs/qDKQlQstIwAJ

The data directory of PostgreSQL is also exportable, and you can create persistent data container for it.
https://docs.docker.com/engine/tutorials/dockervolumes/

For Windows, use following command to check and forward the connection
> netsh interface portproxy show all
> netsh interface portproxy add v4tov4 listenport=80 connectport=8000 connectaddress=<docker ip>
> netsh interface portproxy del v4tov4 listenport=80

Good lock Vincent.

[Jae Hun Lee]

Hi Joel,

I have to confess that I have been holding back my dockenization work because of the recruitment process. I regret doing so now because none of candidate could install Arches on their computers, and I am devastated by them.
Anyway, I am sure that Vincent will continue where I left off and provide the improved docker image to ease installation and operation of Arches. I would recommend to use docker-compose.

Hi Vincent,

I hope my Dockerfile helps your project. Good luck :)

Best,
Jae

Good luck Vincent.

[Vincent Meijer]

Hi all,

@Jae, thanks for sending your Dockerfile. I think we are pretty much on the same track. Nice work!

What I did to end up with separate Docker containers for Django, Postgres and Elasticsearch is create one Dockerfile with the complete installation procedure and them split them up. 

I placed all my code in a public repository: https://bitbucket.org/cvast/cvast-arches-public/overview 

Please read the Readme, as there are a few things you need to do before you can run it. 

I kind of rushed getting this public repo in place and making it as generic as possible. If anything goes wrong, let me know!

I hope this contribution to the Arches community will help making Arches more accessible and easier to install. 

To get started with docker: https://www.docker.com/ 

Vincent

Center for Virtualization and Applied Spatial Technologies (CVAST)

University of South Florida

[Alexei Peters]

This is awesome work guys!  The Arches community is definitely better off with your involvement.  

I have yet to install the dockerized version but anything that can ease the installation of Arches is much appreciated.

Just so you know, we're aware of the difficulties with installing Arches and for v4 have spent some time improving this process.

Cheers,

Alexei

Director of Web Development - Farallon Geographics, Inc. - 971.227.3173

--
-- To post, send email to arches...@googlegroups.com. To unsubscribe, send email to archesproject+unsubscribe@googlegroups.com. For more information, visit https://groups.google.com/d/forum/archesproject?hl=en
---
You received this message because you are subscribed to the Google Groups "Arches Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to archesproject+unsubscribe@googlegroups.com.

[Joel Aldor]

We are definitely excited for version 4 Alexei! When is this finally coming out for our testing?

You received this message because you are subscribed to the Google Groups "Arches Project" group.

To unsubscribe from this group and stop receiving emails from it, send an email to archesprojec...@googlegroups.com.

[Vincent Meijer]

Hi everyone, 

I was wondering if anyone ever had any success with using Jae's or my Docker-ready version of Arches.

FYI: We moved our repositories to Github. Our Arches implementation can now be found here: https://github.com/cvast/cvast-arches 

(At the moment I only use static pages, as we are still working on the database part.)

For other useful tools, see: https://github.com/cvast/

Vincent

[lymankj]

Hi! Is this version still in development? Could not get it to run successfully by running 'docker-compose up'. Do you have any other instructions? Also, does it support data volumes for persistent data?

[Vincent Meijer]

Hey lymankj,

Check out this fork of Arches, which will be included in the official Arches repo soon: https://github.com/cvast/arches/tree/1650_docker_support 

The Readme is here: https://github.com/cvast/arches/tree/1650_docker_support/docker 

Please note that this is Arches v4 and it's still in beta (both the Docker image and Arches itself). 

If you need v3, let me know and I can send you a Dockerfile that does more or less what you need.

Vincent

[Vincent Meijer]

Posting Kepa's reply:

Hi Vincent, thanks for that link! No problems pulling and running Arches, works great!

Really appreciate an easy to use version of Arches to test it out now while writing a proposal to deploy it more permanently. Especially as you're in between major version numbers...

Can you recommend how to best add some existing sample datasets into the docker instance, like the one at https://github.com/archesproject/disco_data ? Do you know of any others more archaeological-site specific?

Thanks!

Kepa

[Vincent Meijer]

Hey Kepa, 

Sorry for the late reply. I'm currently on fieldwork. 

Check out the Readme to see how you can set up your own custom Arches app: https://github.com/cvast/arches/tree/1650_docker_support/docker

Here is how I implemented my own custom Arches app: https://github.com/cvast/cvast-arches

Also have a look at my entrypoint to see how I load my custom database graphs and concepts: https://github.com/cvast/cvast-arches/blob/master/docker/entrypoint/cvast-entrypoint.sh 

Make sure you place your own entrypoint file into the `/docker/entrypoint` folder in the Docker container:

E.g. like I do in my Dockerfile: `COPY ./docker/entrypoint /docker/entrypoint`

Let me know how it goes!

Vincent