[openwisp] [GSoC'19] OpenWisp Template share feature and Template Library

[Noumbissi Valere]

5.12 - 5.18

Sorry for sending for this late. i thought since i don't have any PR yet, a report won't be necessary.

I created the issues for the different sections of the projects which will enable me and mentors track my progress. 

The issues are still to be reviewed and modifications made if needed. 

Currently:

Federico demanded for a discussion on the project before i go into details. thus i am still waiting to have that discussion.

While waiting for this discussion, i am working on issue #112 

[Federico Capoano]

Update: I reviewed the current work Noumbissi has done, things are going well.

I talked with Noumbissi to define the current priority: make sure the sharing and import of templates works, taking in consideration the validation of the template contents and the interaction with variables, which is the hardest part (to understand more this point read this issue: https://github.com/openwisp/django-netjsonconfig/issues/93)

Federico

[Noumbissi Valere]

5.19 - 5.24

I worked on the PR 115 in which i modified the Template model to enable:

-switching between the various  types of templates (private, public, import and secret_key)

- added additional fields which will be needed for this functionality

- ensure variable validation with the backend

Going to work on:

- improve PR 115 if mentors request for changes

- implement the feature for template importing. this means i will need to work on issue 96 and issue 95

--
You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/426fb520-3712-4db8-b1ae-d8e7e3a64fb5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Federico Capoano]

PR reviewed, ping me when last changes are done.

Next step: I want us to focus on this and the issues mentioned in there: https://github.com/openwisp/django-netjsonconfig/issues/93

Without solving those, we won't be able to make this feature work as we want.

Keep in mind the validation can fail also in the javascript validation, we'll have to look for a solution to that as well.

The strategy of filling the sample values may work as well in the javascript validation.

Federico

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAOwnGe4e51WUbXk4b1LeK9r4aBQ8iDg9hugDebu6f6pqj8ei8g%40mail.gmail.com.

[Noumbissi Valere]

Weekly update,

During the week, i was working on solving issue 93 .

Work done:

with respect to the last commit on PR 115 , It is now possible to declare variables in the web UI both at template and device creation.

Currently working on.

I am currently working on how to make variable declaration in the advance mode pass the Schema validation using JavaScript.

  

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6XCeN594yDGvWuneD1PUwTY1M%2BBJ0Fw4ZdJ0zyc%3Db%3DDbw%40mail.gmail.com.

[Federico Capoano]

Reviewed your PR in detail.

The JS part needs some improvement on the UX side: I see the JS validation fails when using variables (although it's still possible to save the template) even though the result is valid.

I left instrucstions and asked you to create 2 new issues which I want us to work soon.

Let me know if everything is clear.

Federico

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAOwnGe65sFWAfa1znb0A9ajdJhOAMKco-oHUrnXZpK-4bawyNg%40mail.gmail.com.

[Noumbissi Valere]

Good day mentors,

Please I wish to know the various template fields which should be serialized and exported during the importation of a an external template.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6Whz1UkfeODV%2B_8hUhVRtNWCX82otpeC9G7TCuzo3NrZQ%40mail.gmail.com.

[Federico Capoano]

Hi Noumbissi,

I guess all the fields except internal notes and organization, do you have additional suggestions?

Federico

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAOwnGe4gHig-S6RwfeVGZ4mcYtb%2BvuY_BLaKmme1ao3Tgn3VuA%40mail.gmail.com.

[Noumbissi Valere]

Weekly update:

This week, i pushed the following PRs:

- Optimized template model : performed changes on the template model.

- JavaScript Validation : Ensure variable-default_values JavaScript validation passes.(need review)

-API for serialization : Added the API which will be used for serialization of templates during imports (needs review).

-API for search : Added API to enable list of public templates and the search of public templates by name and description (needs review).

Am waiting for response to the last comment on Add logic for template import .

Going to work on:

-Next week, i intend to work on the backend features to make importation possible. So admins and mentors can start importing and sharing templates while waiting for the front end.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6XkFapd47T_Lrep5fs8XUySLjocFcjP%2B954YvXEmmCpsQ%40mail.gmail.com.

[Noumbissi Valere]

Hello,

After creating the API to search for templates by name and description in the django_netjsonconfig and tested, (API to search by name and description), 

I created another API in openwisp_controller which search for templates by organization (API to search by organization) using the one created in django_netjsonconfig as base API. upon trying to run *pipenv run ./tests/manage.py runserver* on openwisp_controller to test the API, i get the error message
*RuntimeError: Model class django_netjsonconfig.models.Config doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.*

Ajay and I have tried to debug and solve the issue to no avail. please can someone help me with this? I need it to continue. Thanks.
here is a picture of the traceback error.

[Federico Capoano]

Since we added the base fields for importing and exporting templates in django-netjsonconfig, then we should do this work there too and then work on openwisp-controller only to add support for multi-tenancy.

Try that and let me know.

Let's also try to discuss via voice call when possible.

Federico

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAOwnGe5S4Dof22tTwrWLms1FOPFu-YgthExGX9fHhcw%3D%3DUXO7w%40mail.gmail.com.

[Noumbissi Valere]

I am very sorry for this very late.

Weekly progress

I worked on the following PRs this week,

- updated PR 117

- Moved API for serialization from openwisp_controller to django_netjsonconfig at PR 119 

work to do:

- perform the required changes on PR 117 and PR 119 

- work on logic to import templates from external source

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6U6jUh2DBMudejr9f9%2BghqcOMry372tnd_1_rRCp3_GNQ%40mail.gmail.com.

[Federico Capoano]

Hey Noumbissi,

On Fri, Jun 14, 2019 at 11:54 PM Noumbissi Valere <noumbis...@gmail.com> wrote:

I am very sorry for this very late.

Weekly progress

I worked on the following PRs this week,

- updated PR 117

- Moved API for serialization from openwisp_controller to django_netjsonconfig at PR 119 

work to do:

- perform the required changes on PR 117 and PR 119 

- work on logic to import templates from external source

What is next after that?

Fed 

[Noumbissi Valere]

After this, I will love to add the TemplateSubscription model and the Celery tasks to handle notifications

--

You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6VSrR_ff7Ojx696hXfySwFx-VE1rwFO87D-f6NhZiL-YA%40mail.gmail.com.

[Noumbissi Valere]

Weekly Report 

During this week, i performed changes and optimized the following PRs as requested. And right now they are waiting for review.

-PR 119 

-PR 123 

-PR 117 

After these, I decided to make these APIs available to the openwisp_controller repo. But i still got the following error

RuntimeError: Model class django_x509.models.Ca doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

I tried diverse debugging methods but i didn't find the solution. I will be glad to receive an idea on what to do. 

here is the PR of the work i have done so far on this issue. PR 101 

Also, the openwisp_controller and openwisp-user does not have the gsoc2019 branch against which am suppose to send my PRs, Please i will love that this branch be created so that i can send my PRs to it rather than master. Thanks.

Currently working on:

I have created the TemplateSubscription model and I am presently writing the various APIs for Celery. I will be sending PRs for this work in the next couple of days.

Thanks.

[Federico Capoano]

On Fri, Jun 21, 2019 at 2:41 PM Noumbissi Valere <noumbis...@gmail.com> wrote:

Weekly Report 

During this week, i performed changes and optimized the following PRs as requested. And right now they are waiting for review.

-PR 119

PR approved, you can go ahead.

 

-PR 123 

PR approved, you can go ahead.

 

-PR 117 

 

 

After these, I decided to make these APIs available to the openwisp_controller repo. But i still got the following error

RuntimeError: Model class django_x509.models.Ca doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

I tried diverse debugging methods but i didn't find the solution. I will be glad to receive an idea on what to do. 

here is the PR of the work i have done so far on this issue. PR 101 

The problem is caused by this file: https://github.com/openwisp/django-netjsonconfig/blob/gsoc2019/django_netjsonconfig/api/views.py

That view is not programmed to be reusable by third party apps, because it imports a concrete model, which will fail when you try to import that view in a django project which does not list django_netjsonconfig in INSTALLED_APPS (like we do in openwisp-controller, because we use django-netjsonconfig as a base to extend).

It looks like that view is being loaded automatically by django for some reason. Check why it is being imported and avoid it, if you do that, the problem will be fixed.

 

Also, the openwisp_controller and openwisp-user does not have the gsoc2019 branch against which am suppose to send my PRs, Please i will love that this branch be created so that i can send my PRs to it rather than master. Thanks.

Created for both. 

 

Currently working on:

I have created the TemplateSubscription model and I am presently writing the various APIs for Celery. I will be sending PRs for this work in the next couple of days.

Great.

Thanks

Fed 

[Noumbissi Valere]

Weekly Update

This week, I worked on the notification model and the celery APIs for notifications.

 what can be done now

As of this PR, it is possible for admins to 

-Receive notifications when a user imports their templates

-Receive notifications when user delete their templates

-Users who import templates get the contents of their templates periodically synchronized and a notification send to the designer to indicate last synchronized date. 

here is the PR:

Notification model

What to do next

- Perform changes which may be requested by mentors.

- work solve the ModuleNotFoundErrror on openwisp_controller in order to make the APIs on django_netjsonconfig available to openwisp_controller. 

--

You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6XRL7hBZt6tLoYgzg99C%3DZpqjr67jc-hqMH1fMKpgR2Ew%40mail.gmail.com.

[Noumbissi Valere]

Good day mentors;

weekly report

This is week, i worked on the following issues. 

- PR 119. serialization API (waiting for review)

- PR 123. import logic (waiting for review)

       I enhanced the API for serialization and import logic so that other apps like openwisp-controller can use it without issues.

- PR 113. This PR  added the serialization API and import logic from django_netjsonconfig to openwisp-controller (waiting for review)

- notification model. This PR handles subscription notifications to template designers 

- PR 114. Adds the notification features from django_netjsonconfig to openwisp-controller

- API for external template creation. This adds the base API for external template creation which will be used at the template library backend (waiting for review)

Difficulties:

- I am facing some issues with celery. i have done the setup as stated in the documentation. but yet for me to work on the notification model PR, i had to open extra terminals on which i ran the worker and the beat process. without these, the celery will not behave as it opt to. I don't think this is what will be done in production. 

Secondly when i made this functionality present in openwisp-controller in PR 114, the celery tasks didn't work as expected. 

I am unable to figure out what the problem is. 

Please can someone look at the PRs and tell me what am doing wrong? thanks.

Next week:

I plan to work on openwisp-users repo to add APis for CRUD of organization and user

[Noumbissi Valere]

Good day mentors.

Weekly Report

I worked on the following PRs.

PR 8 backend. in which i created the directory structure of library backend, implemented socail and local login and enable APIs for CRUD of organizations from openwisp_users, external_template creation, get detail template and functionality to list and search for templates from openwisp_controller. With these, I think i have exhausted the objectives of this backend as per the the project idea page. (waiting for review).

PR 81 CRUD org. Implemented API to list, create, update, delete and get details for a particular organization. (waiting for review)

PR 115 create template. Moved API to create external templates from django_netjsonconfig to openwisp_controller. (waiting for review)

Next work.

Next week, i intend to 

• perform changes requested by mentors if any
• begin with the creation of the library frontend

  

[Federico Capoano]

On Fri, Jul 12, 2019 at 4:51 PM Noumbissi Valere <noumbis...@gmail.com> wrote:

Good day mentors.

Weekly Report

I worked on the following PRs.

PR 8 backend. in which i created the directory structure of library backend, implemented socail and local login and enable APIs for CRUD of organizations from openwisp_users, external_template creation, get detail template and functionality to list and search for templates from openwisp_controller. With these, I think i have exhausted the objectives of this backend as per the the project idea page. (waiting for review).

I've added some comments.

 

PR 81 CRUD org. Implemented API to list, create, update, delete and get details for a particular organization. (waiting for review)

Did we say we would do this in openwisp-users? I think you can do that in the template library backend for the moment.

 

PR 115 create template. Moved API to create external templates from django_netjsonconfig to openwisp_controller. (waiting for review)

I've added some comments and requested changes.

 

Next work.

Next week, i intend to 

• perform changes requested by mentors if any
• begin with the creation of the library frontend

Great.

Thanks

Federico

[Noumbissi Valere]

Good day mentors.

Weekly report.

-I focused on performing changes requested by mentors for existing PRs. in the course of this, some PRs were merged and others required further changes which am currently working on. This was to ensure that all the work i have done so far are approved and merged.

  

permission:

The time table for my final year examination is out and I am suppose to begin writing next week 24th July till the 2nd of August. 

I wish to take some days off in order to focus on this examination and will be back from the 2nd of August. 

During my days off, i will not be able to writes codes as before but will do my best to write some code and push when i have the time.

--

You received this message because you are subscribed to the Google Groups "OpenWISP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openwisp+u...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6U8qSKxryB3wCuMW0ra_Yghhz%3DDddShba0XRj876vS32g%40mail.gmail.com.

[Noumbissi Valere]

Good day mentors.

Sorry for the late reply,

I am currently writing my final year exams so i don't code as much as before reason why I am slow. I will be finishing with the exams on the 2nd of August. 

But I am currently working on addressing the issues which were raised during the last review of my work when i have the chance to work. I will be sending a PR for this work soon. 

[Noumbissi Valere]

Good day mentors:

I ended with my Exams today. I know I am way behind schedule and have limited time.

Will used the remaining week to progress and hopefully finish up. 

This week I worked on

https://github.com/openwisp/django-netjsonconfig/pull/124

Next Week.
I Intend to work on the Frontend library next week

[Noumbissi Valere]

Good day mentors.

Weekly report

This week, i worked the following PRs:

https://github.com/openwisp/openwisp-template-library-backend/pull/8

https://github.com/openwisp/openwisp-controller/pull/115

And the frontend library. Its PR is not yet ready but as of now, it is possible to list templates found on the backend server, search for templates by name, organization and description, view template detail and view organization details of the template.

Currently working on:

I am currently working on social login and password management.

to do next:

-add view feature to create templates on the frontend

-write tests

-Send this PR

-Perform changes if need be

-Document all my work for the repos i have worked on for the work done.

Question:

One of the features on the project idea page is to enable admins to add, update and delete organizations on the frontend. My question is why are we doing this, each openwisp instance have this functionality already. I am proposing that we let admins handle organizations for the templates as they have been doing and then on the library they create just templates. on the library, the organization of the template can be gotten from the templates during its creation since each template is serialized with its organization field. updates will equally be done when celery launch the synchronization task. 

[Federico Capoano]

I'm reviewing the PRs now, I should have replied to your question via IM.

Fed

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAOwnGe4_FAs0AnffiEJYvk0VfZLzYxrA5VHmjXqBn1n61hNX%3DQ%40mail.gmail.com.

[Noumbissi Valere]

Good day mentors.

weekly report

This week i worked on

Frontend library . I created the structure, wrote the code for the final logics which were discussed, added the tests for the various components implemented and add some notes to the README.

Backend Library. I added some notes in the readme file of this library

Openwisp-controller. Increase coverage tests and updated the readme file. Concerning the last comment on this PR, I need to pass the Vpn to the Asbtract Template class. this is used to create Vpn during imports. If i move the Vpn class back down, i will get the error *which says am using Vpn before defining it.* I have search on a way round it but found none. I love to correct this too but don't know what else to do because that Vpn needs to down to the abstract class. Please can you help me on a way round this? thanks.

django_netjsonconfig. Increase coverage test and updated the README file.

To do next:

Work hand in gloves with mentors to test this work to make all the necessary and cited features were implemented and equally perform changes if any is requested. 

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6WrdwAKpNa7hEi2owSpqsVg8SLNO1GxZjv%2Bk5OPndvfXw%40mail.gmail.com.

[Federico Capoano]

Reviewed and merged, think about a way (eg: prepare some instructions or deploy an instance somewhere online) or to help us testing your work please.

Federico

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAOwnGe6dEr4-wUPUK8hPhbFA7d%3D_qeXvLqGQSrwoW3FEGtRmKA%40mail.gmail.com.

[Noumbissi Valere]

Good day mentors

I prepared the instructions which could be followed to get the work up and running on a machine.

after this PR is merged to the backend, these instructions can be followed.

 

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6WwqL65J-0js1z3n97eMGo4cXG2LUpRW7MCj5HE_kmaPA%40mail.gmail.com.

[Federico Capoano]

Hey Noumbissi, can you make sure those instructions are in the README of the frontend please?

Also add a link to those instructions to the backend README.

F.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAOwnGe6%3DQvWqbuAu0xFq0d5J%3DyN%3DRjLQg8Wk1P43NAGFPgu_sw%40mail.gmail.com.

[Noumbissi Valere]

Hi Federico,

Ok. I will do that. 

In the meantime am readily available to perform corrections on the repos if errors were found during testing. 

To view this discussion on the web, visit https://groups.google.com/d/msgid/openwisp/CAERYH6UcnVABeCb2FetXjfsEZ7-iLz7kNCdTpRv56Dd2MOyx7A%40mail.gmail.com.

[Noumbissi Valere]

Hey Federico,

The README files have been fixed and updated

https://github.com/openwisp/openwisp-template-library-frontend/pull/5

https://github.com/openwisp/openwisp-template-library-backend/pull/11

[Noumbissi Valere]

Good Mentors

Here is the link to my github gist containing the work i did this summer which i submitted to google

https://gist.github.com/NoumbissiValere/04b0c1edca61cae6cc10687464b4ecc5