GSoC Draft project proposal
vedant agarwala
Name: Vedant Agarwala
Email Address: vedant....@kdemail.net
IRC Nick: ‘vedu’ on #calamares, #kde-soc, #gsoc
Location: Kolkata, India GMT+5.30
Proposal Title: Python interface refactor + Ruby modules interface
Motivation for Proposal and Overview:
Calamares’ main job, simply put, is to be a job executor. Hence it must be made such that its really easy for people to write executable jobs- this includes the ability to write job modules in different languages.
Currently it possible to write job modules in python as well as C++/Qt and write view modules in Qt. I want to improve the python support by adding more features to it and making it easy to use. Features like creating a UI, present in the C++ modules (think classes like `AbstractPage` and `ViewStep`), need to be added. We can also extend this support to ruby- modules can be written in ruby as well.
We are using boost.python to bind python to our C++ code. I researched and discussed a few more such libraries (like pythonQt, pyste). I feel SWIG is better suited to our needs and a possible replacement for boost.python; mainly becauses it supports other languages as well. SWIG has been around for a long time, is well documented and used by a number of large size projects [1]. Key difference between the two is: SWIG is a code generator; Boost.Python is an interface library. Code generators require less maintenance, as changes in the C++ code should be automatically reflected in the Python wrappers [2].
Implementation Details:
There will 2 major parts to this project: porting boost.python to SWIG, and extending its support for ruby (using SWIG).
Since I will be using SWIG, the 2 sub-projects will overlap and I will have to keep in mind that my SWIG code will have to support both python and ruby (and perhaps, more languages in the future).
I will start with adding SWIG using cmake. I will be sure to use the argument `-c++` to initialize SWIG as we are only dealing with C++ classes. In order to compile the wrapper code for ruby, the compiler needs the ruby.h header file [3]. I have already tested using the `swig` CLI manually (details ahead).
Then, I start porting Boost.python to SWIG. Boost.python is used the following classes:
GlobalStorage
PythonHelper
PythonJob
PythonJobApi
Qt signal/slot feature is not supported by SWIG so we will have to do some workarounds for it- like wrapping the signal `progress`, as it is done in PythonJob.h. This hack is essentially same as the one used for boost.python. Also, in all of the mentioned classes, we just have to change the boost::python objects to plain old C++ objects and SWIG will handle it. We don’t need to use boost.python data types (like boost::python::object, boost::python::dict, etc.). However, in cases where the default is not enough, we can use %directives. From my initial research it seems that we won’t need to use other directives, the defaults will suffice. However, this may change in the future as we add more features to our python/ruby APIs. In any case, we will only have to add directives in the .i files (think of them as header files) and not in the .cpp files- a really nice way to modularize code.
In the PythonJob.cpp we have a declaration: `BOOST_PYTHON_MODULE( libcalamares )`. This is the main focus of this project and will be replaced with a plain old C++ definition with some SWIG %module directives. In a new libcalamares.i file, I will declare the Libcalamares module and its related methods. Here, I may need to use some other SWIG % directives. The best part about a libcalamares.i file is that by using a single file we can generate both python and ruby interfaces; in fact, create any number of language interfaces that SWIG supports!
This did sound too good to be true, so I created an example.i [4] and ran both `swig -ruby example.i` and `swig -python example.i`. I got 2 files- example_wrap. for ruby and, example.py for python. This means we can replace the the Python* classes (PythonHelper, PythonJob, PythonJobApi, etc.) with generic Script* classes (ScriptHelper, ScriptJob, ScriptJobApi, and so on). ScriptJob.h and ScriptJob.cpp will host the declaration and definitions of the module components. This follows that in the enum Calamares::Module::Interface, the value `Python` can be replaced by `Script`. But this is still debatable and I may end up needing separate `Python` and `Ruby` values.
For scripting jobs I will follow the same API-
a main.py (or a main.rb)
a run() method as entry point
a .conf YAML file.
We need to provide the above features (atleast) to both our python and ruby modules.
We have a test for python modules- I will run that same test module to check whether we have ported without any errors and then I will build it fully so that all modules (including python ones) use the newly ported API and not the boost.python implementation.
All the changes to these 4 classes will be invisible to the rest of the code. Once the port is complete, all the python modules, tests, etc. should run as good as before. Plus we will have ruby support as well. I will create a dummymodule for ruby (just like there is one for python), as well as a test for ruby modules.
Tentative Timeline:
May-
<--- GSoC commences--->
week 4: Include SWIG in the build process
June-
week 1: Fix cmake errors.
week 2: Create libcalamares.i
week 3: Port the rest of the boost.python code.
week 4: Run existing tests (especially those for python modules)
<--- Mid term submissions --->
July-
week 1: Start work on the ruby module
week 2: Add tests for the ruby API.
week 3: Make sure ruby wrappers are generated as expected.
week 4: Make sure unit test suite runs completely.
August-
week 1: Add documentation.
<--- suggested “pens down” --->
week 2: Some more documentation and code cleaning.
<--- firm “pens down” --->
week 3: Prepare and submit reports.
Other Obligations: None. I can easily spent about 50 hours a week coding; since my college gets over in April end. I will be free the entire duration of my summer vacation.
About Me: I am currently in my final undergraduate year in National Institute of Technology, Durgapur, India, studying Computer Science and Engineering. I have submitted patches to KDE (bug-fixes for Rekonq, Amarok, Akonadi[5]) and Mixxx DJ software[6]. I took part in and ‘passed’ Season of KDE 2013, completed GSoC 2014[7] and mentored in SoK 2014.
I love coding for open source.
After GSoC I will continue to be a developer for open-source and look to mentor students from next years’ programs like GSoC, SoK, Google Code-in etc.
External Links:
[1] https://github.com/cython/cython/wiki/WrappingCorCpp#swig
[2] https://dev.lsstcorp.org/trac/wiki/SwigVsBoostPython
[3] http://www.swig.org/Doc3.0/Ruby.html#Ruby
[4] https://gist.github.com/vedant1811/26f34f41e4d794941a75
[5] https://git.reviewboard.kde.org/users/vedanta/
[6] https://bugs.launchpad.net/~vedu
[7] https://www.google-melange.com/gsoc/project/details/google/gsoc2014/vedant/5639274879778816
vedant agarwala
On Friday, March 11, 2016 4:23:52 PM CET you wrote:
> Hello,
>
> I am taking a shot at GSoC 2016 with Calamares.
>
Can you please share this with me as a Google document? It'd be much easier to
review and comment that way.
Cheers,
--
Teo Mrnjavac
http://teom.org | t...@kde.org
vedant agarwala
Vedant.
Teo Mrnjavac
> Hello,
>
> I've updated the document.
>
> Requesting you to review it again.
>