[GSoC'19] Dockerize of OpenWISP

168 views
Skip to first unread message

Ajay Tripathi

unread,
May 19, 2019, 10:30:36 AM5/19/19
to OpenWISP
** Sorry for reposting, just starting a new thread just for record in the format used by fellow GSoC students. **

5.12 - 5.18:
In the past days, i've managed to complete the SSL work[1] and added correct headers on nginx.
I have also made minor code improvements and optimised docker images build time in this pull request.
I have move the code to docker-openwisp[2] and I would be working on the official repository now.

Currently:
I am making basic travis-ci tests[3].


Ajay

---
Ref:

Federico Capoano

unread,
May 19, 2019, 2:14:15 PM5/19/19
to OpenWISP
Great work Ajay, what are your next steps?

Is your kanban board ready with all the issues yet?

Federico

Ajay Tripathi

unread,
May 21, 2019, 7:24:16 AM5/21/19
to OpenWISP
Hello,


On Sunday, May 19, 2019 at 11:44:15 PM UTC+5:30, Federico Capoano wrote:
Great work Ajay, what are your next steps?

There are 3 things I am looking into right now:
- Basic best practices to reduce image size.
- Solving the nginx header issues - (issue #1).
- Creating the websocket image.

Other than that, I am also looking into travis-ci testing script.
I will come back to travis-ci in next week's report. :-)

Is your kanban board ready with all the issues yet?

Yes, I have put all the issues in the kanban board now. 
It's general but I am going to make it more and more specific as I get a better idea.


Ajay 

Federico Capoano

unread,
May 21, 2019, 9:47:28 AM5/21/19
to OpenWISP
On Tue, May 21, 2019 at 7:24 AM Ajay Tripathi <ajay...@gmail.com> wrote:
Hello,

On Sunday, May 19, 2019 at 11:44:15 PM UTC+5:30, Federico Capoano wrote:
Great work Ajay, what are your next steps?

There are 3 things I am looking into right now:
- Basic best practices to reduce image size.
- Solving the nginx header issues - (issue #1).
- Creating the websocket image.

Other than that, I am also looking into travis-ci testing script.
I will come back to travis-ci in next week's report. :-)

Great!

Let me know when it is deployed on Andrew's or Marco's systems so I can test it.

What do you intend to test on travis specifically? Eg: the building of the image, any other thing?
 
Is your kanban board ready with all the issues yet?

Yes, I have put all the issues in the kanban board now. 
It's general but I am going to make it more and more specific as I get a better idea.

It's ok, let get started and improve day by day. 

Marco Giuntini

unread,
May 21, 2019, 10:17:48 AM5/21/19
to open...@googlegroups.com
I'm trying to deploy in GKE but I'm stuk on nfs-server.
I'm able to deploy and configure nfs-server pod and service, claim the NGS volume but for some reason the mount in the container fails.

--
You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6X-HRf2UAGTiaDhZsz3epf%3DgLgAKdaZ%2BSdKZCMaS8G_uw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Ajay Tripathi

unread,
May 22, 2019, 10:08:11 AM5/22/19
to OpenWISP
Hi,


On Tuesday, May 21, 2019 at 7:47:48 PM UTC+5:30, Marco Giuntini wrote:
I'm trying to deploy in GKE but I'm stuk on nfs-server.
I'm able to deploy and configure nfs-server pod and service, claim the NGS volume but for some reason the mount in the container fails.

I am not sure why that is happening, can you please share any relevant information about the issue so that I may keep an eye on the issue and cross-check if it's a problem with the images on my end as well. :-)


Best,
Ajay T.

Marco Giuntini

unread,
May 22, 2019, 10:59:05 AM5/22/19
to open...@googlegroups.com
This is the error that I get on the container level in GKE:
MountVolume.SetUp failed for volume "nfs-volume" : mount failed: exit status 1 Mounting command: systemd-run Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/cc9377c9-7ca1-11e9-a8e7-42010aa40039/volumes/kubernetes.io~nfs/nfs-volume --scope -- /home/kubernetes/containerized_mounter/mounter mount -t nfs nfs-server.default.svc.cluster.local:/ /var/lib/kubelet/pods/cc9377c9-7ca1-11e9-a8e7-42010aa40039/volumes/kubernetes.io~nfs/nfs-volume Output: Running scope as unit: run-r3d2b5b1e168c404fa0122cfe0a9827a2.scope Mount failed: mount failed: exit status 32 Mounting command: chroot Mounting arguments: [/home/kubernetes/containerized_mounter/rootfs mount -t nfs nfs-server.default.svc.cluster.local:/ /var/lib/kubelet/pods/cc9377c9-7ca1-11e9-a8e7-42010aa40039/volumes/kubernetes.io~nfs/nfs-volume] Output: mount.nfs: Failed to resolve server nfs-server.default.svc.cluster.local: Name or service not known 

 
--
Marco Giuntini
Skype: hispanico70420
Twitter: @Hispanico81
PGP Key ID: BD774009


--
You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/5e1a3260-e24a-4690-9cbd-42e5d0927b3f%40googlegroups.com.

Marco Giuntini

unread,
May 22, 2019, 11:05:41 AM5/22/19
to open...@googlegroups.com
I've found my mistake.

I'm not using the default namespace, so this is the correct configuration:
[..]
    spec:
      volumes:
        - name: nfs-volume
          nfs:
            server: nfs-server.k8s-openwisp-8664319.svc.cluster.local
            path: "/"
            readOnly: false




--
Marco Giuntini
Skype: hispanico70420
Twitter: @Hispanico81
PGP Key ID: BD774009

Ajay Tripathi

unread,
May 24, 2019, 11:31:12 AM5/24/19
to OpenWISP
Hi,

Weekly Update (5.18 - 5.24):

Task 1 > 
PR #23
I worked on the adding travis-ci tests for the repository[3].
Currently, I am waiting for @nemesisdesign to have a look at github comment[1].
and @hispanico to take a look at the github comment[2].

Task 2 > 
Pull #27
I worked on fixing the SSL certificate situation[4].
Now, openwisp-nginx container is taking care of the certificates. We can get letsencrypt (http01) certs
for production or create self-signed certs for developement or turn off the certs simply by changing the
value of `SSL_CERT_MODE` variable.

Task 3 >
Found issues with k8s deployment. Fixing them in branch kubernetes[5].
PR will be available after the above 2 are merged and I have accomodated for the changes from these[3][4] PRs.

Next:
Now, I have started the work for the websocket image in a different branch and I will be completing that.
After the websocket image, I plan to get started with the postfix image.


Best,
Ajay Tripathi

---
Ref:

Federico Capoano

unread,
May 24, 2019, 7:02:19 PM5/24/19
to OpenWISP
Thanks Ajay, I left some comments on your pull requests.

Fed

--
You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/7372bbc9-ace4-4909-b3b6-2524fb898add%40googlegroups.com.

Ajay Tripathi

unread,
May 31, 2019, 3:53:34 AM5/31/19
to OpenWISP
Hi,

Weekly Updates:

Review:

Review 1> Pull request #27[1] is waiting for review and there are more pull requests to be added in queue after this one.

Question(s):

Question 1> From my understanding at this point regarding websockets. I don't think daphane needs to be scaled up, only the django worker needs to be scaled. So, I think it makes sense to put daphane in it's own container and runworker in a seperate container. Please let me know if that sounds good. :-)

Question 2> I am getting 502 on adding new user in fresh installation (Ubuntu) after following the instructions in ansible-openwisp2. Error is not logged in the `openwisp2/log/` folder. Trying to debug it but I am not able to find postfix logs. Can you please give me pointers on where can I find it?

Major tasks:



Task 3> Created documentation for environment variables: https://github.com/atb00ker/docker-openwisp/blob/docs/docs/ENV.md

Task 4> Worked on urlpatterns to fix Error 500(s) in current deployment: https://github.com/atb00ker/docker-openwisp/commit/7f8498793d48bbd7da0576d824e4cfbb35e8ebc1

Next:

Next Task 1> Fix the remaing urlpatterns.
Next Task 2> Looking into uniform method for Logging across all containers to stdout.
Next Task 3> Creating celery image.


Thanks,
Ajay T.

---
Ref:


Federico Capoano

unread,
Jun 3, 2019, 9:28:11 PM6/3/19
to OpenWISP
On Fri, May 31, 2019 at 3:53 AM Ajay Tripathi <ajay...@gmail.com> wrote:
Hi,

Weekly Updates:

Review:

Review 1> Pull request #27[1] is waiting for review and there are more pull requests to be added in queue after this one.

Question(s):

Question 1> From my understanding at this point regarding websockets. I don't think daphane needs to be scaled up, only the django worker needs to be scaled. So, I think it makes sense to put daphane in it's own container and runworker in a seperate container. Please let me know if that sounds good. :-)

sounds good, go ahead :-)
 
Question 2> I am getting 502 on adding new user in fresh installation (Ubuntu) after following the instructions in ansible-openwisp2. Error is not logged in the `openwisp2/log/` folder. Trying to debug it but I am not able to find postfix logs. Can you please give me pointers on where can I find it?

should be in /var/log/mail.error or /var/log/mail.log, something along those lines.  
 
Major tasks:



Task 3> Created documentation for environment variables: https://github.com/atb00ker/docker-openwisp/blob/docs/docs/ENV.md

Task 4> Worked on urlpatterns to fix Error 500(s) in current deployment: https://github.com/atb00ker/docker-openwisp/commit/7f8498793d48bbd7da0576d824e4cfbb35e8ebc1

Next:

Next Task 1> Fix the remaing urlpatterns.
Next Task 2> Looking into uniform method for Logging across all containers to stdout.
Next Task 3> Creating celery image.


Great, thanks! 

Ajay Tripathi

unread,
Jun 7, 2019, 12:17:46 PM6/7/19
to OpenWISP
Hello,

Weekly update:

This week I worked on a couple of pull requests[1].
A summary of these pull requests:
#37 adds a postfix image
#36 fixes the url pattern problems with openwisp dashbboard and now we are not getting Error 500 
on certain tasks like adding a device.
#35 Begins moving logging from files to console
#34 Fixes issues with kubernetes deployment sample files.
#33 adds minor improvements to the docker images.

Notes:
Websocket: channels2 and channels have different deployment methods. openwisp-controller needs to be updated to django-loci's newer version before I can use channels2 for deployment. Hence, currently I have implemented channels and will upgrade as soon as possible.

Next week:
- Configurability of django settings.
- implementing default management VPN.


Thanks,
Ajay Tripathi

---
Ref:

Federico Capoano

unread,
Jun 7, 2019, 1:06:13 PM6/7/19
to OpenWISP
Hey Ajay,

great progress!

Is not clear from the email if there are pending PRs to review.
I'm a bit overwhelmed and losing track of what's going on right now, so I need to be reminded of these things.

Thanks
Federico

--
You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/b1bd1bf0-7ba3-4b19-a221-d2e6d8855760%40googlegroups.com.

Ajay Tripathi

unread,
Jun 7, 2019, 3:25:47 PM6/7/19
to OpenWISP

Federico Capoano

unread,
Jun 12, 2019, 11:24:14 AM6/12/19
to OpenWISP
I see the PRs have been merged.
Thanks to Marco Giuntini for reviewing.

Is there any remaining blocker?

Federico

--
You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/ff2b1c0e-eacd-41ec-a3f5-fa53412d8c16%40googlegroups.com.

Ajay Tripathi

unread,
Jun 14, 2019, 9:02:45 PM6/14/19
to OpenWISP
Hello,

Weekly update:

2. Updated docs for environment variables. (Will make the PR after #39 is merged.)

3. docker issue: the current version of dashboard is not working on @2stacks's system but it's working in my system and travis-ci. We have tried a couple of things but still haven't tracked the cause or made the issue reproducable on other systems. Error:
dashboard_1 | ImportError: cannot import name 'hex_regex' from 'django.contrib.gis.geometry' (/usr/local/lib/python3.7/site-packages/django/contrib/gis/geometry/init.py)

4. Postfix issue: When creating a new user, even if sending the mail to user fails(say, because postfix is not configured correctly or down), the new user is created, is this the expected behaviour? I see no errors anywhere on the site & it doesn't feel like the expected behaviour so I want to confirm on this.

5. Connection issue: When trying to setup the openwisp_controller.connection, I am facing an issue that when I go to change any of the devices configuration and apply, I get a timeout because request takes too long.
After hours of debugging, It looks like the issue is because channel_layers:
The line that hangs further execution(/usr/local/lib/python3.7/site-packages/channels/worker.py):
finally:
    # Send consumer finished so DB conns close etc.
    consumer_finished.send(sender=self.__class__)

Note:
  - Celery is connected and working, I am able to do conn.update_config() from shell.
  - redis is connected and working, I can ping and it works during other tasks as well.
  - websockets is working Location page are working as expected as well.

To my best understanding right now, either it's a bug in redis or related upstream dependencies or some configuration / django signal that I can't seem to find, any pointers would be helpful. The last commit here[1] are my configurations / changes for openwisp_controller.connection


Thanks,
Ajay Tripathi

Ref:

Ajay Tripathi

unread,
Jun 21, 2019, 3:15:49 AM6/21/19
to OpenWISP
Hello,

Weekly update:

Most of my week was about fighting the issue #40: https://github.com/openwisp/docker-openwisp/issues/40 (Update included in the issue itself.)

Next Week:

In 2 days, I will be leaving for the GCI trip. I will return from the trip on Friday. (28th June) and I will update about 
the next issue I am going to start in the next week's mail.



Thanks,
Ajay Tripathi

Federico Capoano

unread,
Jun 23, 2019, 10:04:28 AM6/23/19
to OpenWISP
On Fri, Jun 21, 2019 at 3:15 AM Ajay Tripathi <ajay...@gmail.com> wrote:
Hello,

Weekly update:

Most of my week was about fighting the issue #40: https://github.com/openwisp/docker-openwisp/issues/40 (Update included in the issue itself.)

Let's schedule a call to discuss this issue so you can explain me in detail.
Merged.

Next Week:

In 2 days, I will be leaving for the GCI trip. I will return from the trip on Friday. (28th June) and I will update about 
the next issue I am going to start in the next week's mail.

That's great! Have fun and share some photos if you can! If you do, tag or mention openwisp on social media :-)

Federico

Ajay Tripathi

unread,
Jul 5, 2019, 2:24:46 PM7/5/19
to OpenWISP
Hello,


Weekly update:

1, Completed on #41, which adds variable documentation: https://github.com/openwisp/docker-openwisp/pull/41.
2. Opened pull request #42, improvements for the Makefile: https://github.com/openwisp/docker-openwisp/pull/42

On Sunday, June 23, 2019 at 7:34:28 PM UTC+5:30, Federico Capoano wrote:

Let's schedule a call to discuss this issue so you can explain me in detail.
 
Sure. I will come back to this as soon as possible.

Next Week:

1. Completing freeradius image.
2. Working on the OpenVPN image.

Action required: 



Best, 
Ajay Tripathi
 

Ajay Tripathi

unread,
Jul 12, 2019, 2:47:54 PM7/12/19
to OpenWISP
Hi,

Weekly update:

1. Worked on freeradius image and a draft pull request is available here: https://github.com/openwisp/docker-openwisp/pull/44
2. Worked on OpenVPN image and branch is available here: https://github.com/atb00ker/docker-openwisp/tree/openvpn

Next Week:

1. Completing OpenVPN and making the pull request.
2. Working on the Celery image (again).

Action required:

1. Please let me know about the expected default configurations for the OpenVPN. (Currently working on dummy configurations.)
2. #42 is open for review: https://github.com/openwisp/docker-openwisp/pull/42


Best, 
Ajay Tripathi

Federico Capoano

unread,
Jul 12, 2019, 6:17:37 PM7/12/19
to OpenWISP
On Fri, Jul 12, 2019 at 2:48 PM Ajay Tripathi <ajay...@gmail.com> wrote:
Hi,

Weekly update:

1. Worked on freeradius image and a draft pull request is available here: https://github.com/openwisp/docker-openwisp/pull/44
2. Worked on OpenVPN image and branch is available here: https://github.com/atb00ker/docker-openwisp/tree/openvpn

Next Week:

1. Completing OpenVPN and making the pull request.
2. Working on the Celery image (again).

Action required:

1. Please let me know about the expected default configurations for the OpenVPN. (Currently working on dummy configurations.)

A management VPN using tun mode, no encryption, TCP mode, server mode, TLS, here's an example config:

auth SHA1
ca ca.pem
cert cert.pem
cipher none
comp-lzo no
dev tun0
dev-type tun
dh dh.pem
fast-io
group openvpn
keepalive 10 120
key key.pem
log /var/log/openvpn/management.log
mode server
mssfix 1450
mtu-disc no
persist-key
persist-tun
port 1194
proto udp
script-security 1
server 10.8.0.0 255.255.255.0
status /var/log/openvpn/management.status
status-version 1
tls-server
topology p2p
user nobody
verb 3
Added some comments.

Ajay Tripathi

unread,
Jul 20, 2019, 3:16:16 PM7/20/19
to OpenWISP
Hello,


Weekly update:

1. Finally fixed the celery image.
2. Worked on OpenVPN image.

Next Week:

1. Working on these pull requests to get them to merged: https://github.com/openwisp/docker-openwisp/pulls
i.e testing them on kubernetes and testing.
2. Working on moving some of the urls to admin namespace.

Ajay Tripathi

unread,
Jul 27, 2019, 1:14:06 PM7/27/19
to OpenWISP
Hello,

Weekly update:

1. Worked on freeradius image
1. Working on running the periodic tasks of the radius and network-topology containers.

Federico Capoano

unread,
Jul 27, 2019, 1:43:12 PM7/27/19
to OpenWISP
On Sat, Jul 27, 2019 at 1:14 PM Ajay Tripathi <ajay...@gmail.com> wrote:
Hello,

Weekly update:

1. Worked on freeradius image
2. Worked OpenVPN pull request.

Waiting for review on:


First round of review done.
 
Next Week:

1. Working on running the periodic tasks of the radius and network-topology containers.

Great! 

Thanks
Federico

Ajay Tripathi

unread,
Aug 2, 2019, 4:04:48 PM8/2/19
to OpenWISP
Hello,

Since only 2.5 weeks are left for GSoC, I will be updating https://github.com/openwisp/docker-openwisp/issues/56 to keep track of issues to be done before GSoC gets over and make all the complete work available here for submission as per the GSoC requirements before 26 August.

Weekly update:

1. Worked on celery-beat for scheduling the periodic tasks of radius and network-topology: https://github.com/atb00ker/docker-openwisp/tree/celery-beat
2. Made pull requests for moving urls to admin-namespace:
    2.1. openwisp/django-netjsonconfig#127
    2.2. openwisp/django-x509#62
    2.3. openwisp/openwisp-controller#116

Next week:



Best,
Ajay Tripathi

Federico Capoano

unread,
Aug 2, 2019, 4:20:39 PM8/2/19
to OpenWISP
Great, remember the periodic tasks, without those tasks some modules won't work as they should.

Fed

--
You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/2a5effde-0afd-421e-85ee-1182abed569b%40googlegroups.com.

Ajay Tripathi

unread,
Aug 9, 2019, 1:30:40 AM8/9/19
to OpenWISP
Hi,



On Saturday, August 3, 2019 at 1:50:39 AM UTC+5:30, Federico Capoano wrote:
Great, remember the periodic tasks, without those tasks some modules won't work as they should.

Yes, work for periodic tasks is ready and will follow after celery pull is merged. :-)

 Following are up for review:
Issues that need attention:

How should we finally approach the 
3. What alll CORS settings need to be configurable? https://github.com/openwisp/docker-openwisp/issues/28



Best, 
Ajay Tripathi

Ajay Tripathi

unread,
Aug 17, 2019, 8:23:22 AM8/17/19
to OpenWISP
Hello,

The following are ready for review:


Next:

1. Celery-beat (for periodic tasks for radius and network-topology), will make the pull after #48 is merged.
2. Selenium: Moving tests to selenium because shell script is unreliable.



Best, 
Ajay Tripathi

Federico Capoano

unread,
Aug 18, 2019, 9:28:25 PM8/18/19
to OpenWISP
On Sat, Aug 17, 2019 at 8:23 AM Ajay Tripathi <ajay...@gmail.com> wrote:

Approved!
 
Next:

1. Celery-beat (for periodic tasks for radius and network-topology), will make the pull after #48 is merged.
2. Selenium: Moving tests to selenium because shell script is unreliable.

Great.

Ajay Tripathi

unread,
Aug 19, 2019, 8:36:07 AM8/19/19
to OpenWISP
Great, thanks.


The following are ready for review:

Federico Capoano

unread,
Aug 20, 2019, 9:05:41 PM8/20/19
to OpenWISP
Merged / reviewed. 
Reply all
Reply to author
Forward
0 new messages