GSOC 2017 Proposal - Qubes Manager
John Casey
Google Summer of Code 2017
Qubes Proposal - GUI Improvements
0 Abstract
This is a project intended to improve the UX of the Qubes Manager (QM). Specifically it will encompass the direction and discussion from the Qubes mailing list and GitHub.
This project will involve becoming familiar with the Qubes paradigm and code base. This will include reading existing bug reports and feature requests. Work will proceed in several short iterations, with commits and reports as deliverables. It will require working primarily in the QM portion of the Qubes codebase, but depending on the nature of the bug, may include work in other subsystems.
Disclaimer
No one is more familiar with the Qubes codebase than the development team themselves. As a result of my inexperience, I may have overestimated my ability, or conversely, underestimated the complexity of the underlying codebase.
If you feel that my proposed timeline or deliverables are inappropriate please let me know and we can work on improving both. I would really appreciate this, as I would like to maximise my contribution to this project.
1 Project Goals
The Qubes development team have decided that the Qubes Manager is in need of a radical overhaul (#1870 [1], #2132 [2]). Specifically they intend to split the QM into two separate interfaces. One which displays current information about the system and allows control of same: currently running VMs, resource usage, starting and stopping VMs, etc. The second is concerned with the persistent attributes of the system. This covers creation and management of AppVMs, as well as Qube specific actions.
It is the latter of these that is the focus of my proposal. Discussing this with Marek and Jean on the mailing list has revealed to me that the Qubes team have not completely settled on a graphical framework for this project. As a result, further discussion will be required to determine whether to proceed with the Qt framework or to port the project to GTK. However, I feel the technology used is relatively independent of the design of the manager itself, and so I proceed.
The development of this can be broken up into three distinct phases.
Analysis of the existing wireframes written by bnvk [3]
If necessary, port this code to GTK.
Expand upon this codebase, so as to deliver a QM manager that allows users access to persistent configurations as in the current manager.
Add features as enumerated in #1870 [1]
2 Implementation
In a project like this, communication is crucial. I believe that the only way to avoid confusion when collaborating is to have “everyone nodding, around the same diagram”. To this end, I anticipate regular (ideally daily) contact between myself and my mentor as well as weekly formal reports on progress.
As earlier noted, the basis for this project is the work already started by bnvk. Last year they submitted concepts and wireframes for the new QM. This to me is the best starting point as their suggestions were well received by the development team.
I will then expand on this, taking functionality present in the current QM and bringing it into appropriate panels within the new QM. Naturally, this will be thoroughly documented and tested. Particular attention will be paid to ensure that existing functionality is not lost, such as accessibility for keyboard-only power users.
Once complete, I will bring a list of suggested features and enhancements to the development team, and perhaps to the community at large if this is deemed appropriate by my mentor. We can then discuss which of these features are suitable for me to undertake.
These new features can then be implemented in the new QM in preparation for the Qubes 4.0 release. After GSOC, I will be able to tackle more complex features and bug-fixes in relation to the Qubes Manager.
3 Road Map
Qubes will be my focus this summer, with continued contributions afterwards. I really enjoy learning new systems and skills in my free time, and have no issue allocating time as necessary. Due to my relative inexperience, I expect significant catch-up time will be necessary to be able to properly contribute to Qubes. Considering this, I expect to work no less than full time - hopefully upwards of 40 hours per week.
My only other commitments are relatively trivial. I work occasionally (approximately once per week), as evening support staff for a summer school in my college. It is primarily a pastoral role, helping with meal times, first aid and facilities management.
At the end of the summer, provisionally the last two weeks in August, I may be attending training for the role of RA with my college for the next academic year.
Because it is the Summer of Code I do intend to take a couple weekends hiking and camping in the mountains local to me, but I do not anticipate that this will conflict with my Qubes schedule.
Below I have detailed a possible plan for delivering on the promises I have made above:
Timeline
3.0 - Installation and Familiarisation with Qubes, Setup qubes-builder
April 3rd - May 30th
As I am not currently intimately familiar with Qubes, I will reinstall it and transition to it as my day-to-day OS. Ideally I will build the OS from source, but I believe using Qubes will be the most effective resource when designing UX improvements.
3.1 - Reading Documentation, Identifying Skills, Trivial Contribution
May 30th - June 6th
Qubes has a plethora of user and developer feedback, and future plans. My first step to contributing to Qubes will be to immerse myself in this documentation.
A beautiful interface is useless if it is ineffective. Understanding and using the various system calls and configurations used by the QM will be crucial to improving it.
In order to ensure that my environment is setup correctly and that I am adhering to existing style practices I should submit a trivial contribution to Qubes. An example of this would be the revising of the “DispVM” name across Qubes as documented in #2671 [4].
3.2 - Learning GTK/Qt, Contribute a Bug Fix to Qubes, Compile Report on QM Requirements
June 6th - 20th
At this point, with guidance from Jean and Marek, the GTK/Qt decision will have been made and I will have begun implementing trivial UI components in Qubes. Using this knowledge, I will be able to contribute to one or more of the issues that have been pain points for users (e.g #1117) [5].
Next, I will analyse the current QM and report on what parts belong in the new QM, and what should instead be included with the “current state” display widget. These findings will be taken to the team for feedback and will form the basis for the design of the new QM.
3.3 - Expand on QM Design by bnvk, Draft Design, Discuss with Development Team and Amend Proposal.
June 20th - 27th
Based on this report, I will draft UI mockups and design the specific tabs and menus of the new interface. This can be taken to the team for feature suggestions and improvements.
I will compile a memo on my planned approach to the development of a new QM and discuss this with my mentor. As I receive feedback, the planned approach will be amended and improved.
3.4 - Implement Proposed QM Core Functionality
June 27th - July 11th
Taking the memo and groundwork laid by bnvk, I will begin creating the new QM interface and porting features. Deliverables will take the form of a commit of a basic interface for the new QM, with the basic functionality implemented.
3.5 - Complete Core QM
July 11th - 25th
Complete the QM, including all features related to the persistent attributes of the system. Deliverable will take the form of a commit that can be installed and tested by the community at large.
3.6 - Additional Features
July 25th - August 29th
I will take the features selected by my mentor and I, ideally with a timeframe of one week each, but of course this will depend on the nature and complexity of the enhancement. The deliverables will be commits with the full functionality as agreed.
3.7 - Compile Final Report on GSOC
August 29th
Produce and publish a final report on my experience with GSOC and Qubes - hopefully with input from the development team. This should prove useful for others interested in contributing to an OSS project such as Qubes or in the Google Summer of Code.
4 Experience
I am a competent C programmer, with more than four years of experience. I have less experience with Python, only a couple of years, though it quickly became my preferred language primarily due to the community support and its widespread adoption. I tend to use a fairly object-oriented style, with flexible data structures and consistently named functions and variables.
I am partial to the OTBS, but I am no evangelist and will conform to existing style standards.
I have used C for high-level scripts and a small amount of embedded development, specifically with Arduino.
Through my college course I have completed several C courses and one in OOP, taught through C++. As a result I have an excellent basis in imperative programming.
Most of my experience in development has been on MacOS because that was what I grew up with. I stayed with MacOS due to its similarity to Linux and ease of use. In recent years, I have been transitioning to Linux and am currently using Manjaro for my day-to-day needs. I used Qubes briefly several months ago, but required a computer for class and couldn’t reconcile the setup time when I had coursework due. I am switching my computer to Qubes to familiarise myself with the system.
Almost all my computer coursework during the past three years has been on Unix, specifically MacOS. I'm very comfortable using the shell and other standard Unix facilities. As part of this coursework, I took a class in Unix programming. This was one of my most beneficial classes and included the structure and operation of the Unix OS, Bash scripting and the GNU build system.
5 Contact
John Casey
IRC - jmcasey
Twitter - @JackBobCasey
I am attending University College Dublin, Ireland. I am currently completing the third year of my bachelors in Electronic Engineering, with the intention of pursuing a Masters in Electronic and Computer Engineering.
I strongly support software freedom. I use GNU/Linux daily in my development work.
I have not contributed code to a public open software project, but have helped provide code for the robotic hand built my college’s Robotics Club. I am certain that it could have been better, but I know that now it is publicly available, others can improve and augment my codebase.
I briefly experimented with Qubes in the summer of 2016, but moved away from it. I have recently moved back to Qubes to get a feel for its compartmentalization features before installing it on my main laptop.
I have some experience with distributed development. I'm familiar with mailing lists, revision control, etc. I know how to use Git, and I am more than willing to learn other systems; after all that is the goal of GSOC
To conclude, I learn quickly, and can code efficiently and communicate professionally. Given the size of the Qubes team and community, working within a group and towards a common goal is essential, and I excel at this. I am very interested in contributing to Qubes as I believe that it has a positive effect on its users and on the security community at large.
6 References
[1] https://github.com/QubesOS/qubes-issues/issues/1870
[2] https://github.com/QubesOS/qubes-issues/issues/1117
[3] https://github.com/bnvk/qubes-manager-new
[4] https://github.com/QubesOS/qubes-issues/issues/2671
[5] https://github.com/QubesOS/qubes-issues/issues/1117
Jean-Philippe Ouellet
On Sun, Apr 2, 2017 at 10:23 PM, John Casey <john.c...@ucdconnect.ie> wrote:
> The Qubes development team have decided that the Qubes Manager is in need of
> a radical overhaul (#1870 [1], #2132 [2]). Specifically they intend to split
> the QM into two separate interfaces. One which displays current information
> about the system and allows control of same: currently running VMs, resource
> usage, starting and stopping VMs, etc. The second is concerned with the
> persistent attributes of the system. This covers creation and management of
> AppVMs, as well as Qube specific actions.
>
>
> It is the latter of these that is the focus of my proposal. Discussing this
> with Marek and Jean on the mailing list has revealed to me that the Qubes
> team have not completely settled on a graphical framework for this project.
> As a result, further discussion will be required to determine whether to
> proceed with the Qt framework or to port the project to GTK. However, I feel
> the technology used is relatively independent of the design of the manager
> itself, and so I proceed.
> The development of this can be broken up into three distinct phases.
>
>
> Analysis of the existing wireframes written by bnvk [3]
>
> If necessary, port this code to GTK.
whole purpose is to already plug in easily with GTK.
[1]: https://glade.gnome.org/
> Expand upon this codebase, so as to deliver a QM manager that allows users
> access to persistent configurations as in the current manager.
moving target and ongoing area of work. Woju (CC'd) and Marmarek can
give you a better idea of the state of this code. It's possible that
the API has not stabilized and is not ready to be used yet. I have not
personally been following this work very closely.
[2]: https://qubes-core-admin.readthedocs.io/en/latest/
In addition to bvnk's work which I see you've already found, there is
also Kalkin's work on the "domain state" component of the split
manager. Screenshots here [3], code here [4]. This is also already in
GTK.
[3]: https://github.com/QubesOS/qubes-issues/issues/2132#issuecomment-255106656
[4]: https://github.com/kalkin/manager-new
> Particular attention will be paid
> to ensure that existing functionality is not lost, such as accessibility for
> keyboard-only power users.
> Once complete, I will bring a list of suggested features and enhancements to
> the development team, and perhaps to the community at large if this is
> deemed appropriate by my mentor. We can then discuss which of these features
> are suitable for me to undertake.
proposal, but... it is what it is. IMO you should have a clear idea of
what to do and how to do it by the start of the coding period (May
30th).
> 3 Road Map
>
> Qubes will be my focus this summer, with continued contributions afterwards.
allowed to consider things being promised during the summer. Planned
future contributions are of course encouraged, but do not increase the
chances of your proposal being accepted.
> Because it is the Summer of Code I do intend to take a couple weekends
> hiking and camping in the mountains local to me, but I do not anticipate
> that this will conflict with my Qubes schedule.
> Below I have detailed a possible plan for delivering on the promises I have
> made above:
>
> Timeline
beginning things too late.
You should be reading documentation and bootstrapping your familiarity
with our workflows for contributing code as soon as possible. Keep in
mind that June 26th is the first evaluation deadline, at which point
you are expected to have made reasonable progress towards the stated
goals of your actual proposal, not only (allegedly) read docs. The
general GSoC expectation is that the familiarizing happens before the
coding period begins, and during the coding period you really write
code.
[5]: https://developers.google.com/open-source/gsoc/timeline
> 3.0 - Installation and Familiarisation with Qubes, Setup qubes-builder
>
> April 3rd - May 30th
>
> As I am not currently intimately familiar with Qubes, I will reinstall it
> and transition to it as my day-to-day OS. Ideally I will build the OS from
> source, but I believe using Qubes will be the most effective resource when
> designing UX improvements.
working on it :)
I'd strongly recommend not taking a month just to install Qubes if you
can avoid it. The best way to learn is to just jump right in and force
yourself to use it for everything you do. I think you'll find the
qubes-users mailing list a welcoming and helpful place if you run into
any issues in doing so. I'd say switch entirely to Qubes ASAP.
One trick I've used for several friends to ease the migration is to
throw your existing system into an HVM so you can continue to use it
as-is while getting used to the Qubes way of doing things. This
requires you have another (larger) disk on hand though which you can
swap with your existing one. Not sure if this would help in your case,
but it's worth a thought.
> 3.1 - Reading Documentation, Identifying Skills, Trivial Contribution
>
> May 30th - June 6th
>
> Qubes has a plethora of user and developer feedback, and future plans. My
> first step to contributing to Qubes will be to immerse myself in this
> documentation.
be your first timeline section. Definitely don't wait until May 30th
(when Google wants you to be writing code) to begin reading docs!
Also, do not expect this to only take a week! Reading docs is very
much an ongoing process.
> A beautiful interface is useless if it is ineffective. Understanding and
> using the various system calls and configurations used by the QM will be
> crucial to improving it.
need be concerned with. Not quite sure what you mean by configurations
either.
Anyway, I'm not sure you need to be too concerned with the
implementation details of the existing qubes-manager. Much is
changing, and for good reason. Much of the current internals (PyQt,
old-core-API) are not at all applicable to the intended future
version.
> In order to ensure that my environment is setup correctly and that I am
> adhering to existing style practices I should submit a trivial contribution
> to Qubes. An example of this would be the revising of the “DispVM” name
> across Qubes as documented in #2671 [4].
> 3.2 - Learning GTK/Qt, Contribute a Bug Fix to Qubes, Compile Report on QM
> Requirements
>
> June 6th - 20th
>
> At this point, with guidance from Jean and Marek, the GTK/Qt decision will
> have been made
otherwise, and Joanna convinced me that GTK is the way to go.
> and I will have begun implementing trivial UI components in
> Qubes. Using this knowledge, I will be able to contribute to one or more of
> the issues that have been pain points for users (e.g #1117) [5].
>
>
> Next, I will analyse the current QM and report on what parts belong in the
> new QM, and what should instead be included with the “current state” display
> widget. These findings will be taken to the team for feedback and will form
> the basis for the design of the new QM.
> 3.3 - Expand on QM Design by bnvk, Draft Design, Discuss with Development
> Team and Amend Proposal.
>
> June 20th - 27th
>
> Based on this report, I will draft UI mockups and design the specific tabs
> and menus of the new interface. This can be taken to the team for feature
> suggestions and improvements.
>
>
> I will compile a memo on my planned approach to the development of a new QM
> and discuss this with my mentor. As I receive feedback, the planned approach
> will be amended and improved.
planning and proposing which has already been done by others) and not
enough implementing.
I'd suggest moving all this earlier and getting started much sooner.
> 3.4 - Implement Proposed QM Core Functionality
>
> June 27th - July 11th
>
> Taking the memo and groundwork laid by bnvk, I will begin creating the new
> QM interface and porting features. Deliverables will take the form of a
> commit of a basic interface for the new QM, with the basic functionality
> implemented.
[6]: https://github.com/bnvk/qubes-manager-new
> 3.5 - Complete Core QM
>
> July 11th - 25th
>
> Complete the QM, including all features related to the persistent attributes
> of the system. Deliverable will take the form of a commit that can be
> installed and tested by the community at large.
I guarantee this will take longer than you expect. This is the whole
essence of the project and will have many sub-tasks and unforeseen
surprises (perhaps such as "hmm... it seems we actually need X in the
core management API to implement this") which could each set you back
a week on their own.
John Casey
On Monday, 3 April 2017 06:56:39 UTC+1, Jean-Philippe Ouellet wrote:
First, note that we prefer plain-text emails over HTML on the mailing lists.
On Sun, Apr 2, 2017 at 10:23 PM, John Casey <john.c...@ucdconnect.ie> wrote:
> The Qubes development team have decided that the Qubes Manager is in need of
> a radical overhaul (#1870 [1], #2132 [2]). Specifically they intend to split
> the QM into two separate interfaces. One which displays current information
> about the system and allows control of same: currently running VMs, resource
> usage, starting and stopping VMs, etc. The second is concerned with the
> persistent attributes of the system. This covers creation and management of
> AppVMs, as well as Qube specific actions.
>
>
> It is the latter of these that is the focus of my proposal. Discussing this
> with Marek and Jean on the mailing list has revealed to me that the Qubes
> team have not completely settled on a graphical framework for this project.
> As a result, further discussion will be required to determine whether to
> proceed with the Qt framework or to port the project to GTK. However, I feel
> the technology used is relatively independent of the design of the manager
> itself, and so I proceed.
It's resolved. GTK is preferred.
> The development of this can be broken up into three distinct phases.
>
>
> Analysis of the existing wireframes written by bnvk [3]
>
> If necessary, port this code to GTK.
He did not create wireframes, he designed the GUI in Glade [1] whose
whole purpose is to already plug in easily with GTK.
[1]: https://glade.gnome.org/
> Expand upon this codebase, so as to deliver a QM manager that allows users
> access to persistent configurations as in the current manager.
This will require using the new "core3" API [2], which is still a
moving target and ongoing area of work. Woju (CC'd) and Marmarek can
give you a better idea of the state of this code. It's possible that
the API has not stabilized and is not ready to be used yet. I have not
personally been following this work very closely.
[2]: https://qubes-core-admin.readthedocs.io/en/latest/
In addition to bvnk's work which I see you've already found, there is
also Kalkin's work on the "domain state" component of the split
manager. Screenshots here [3], code here [4]. This is also already in
GTK.
[3]: https://github.com/QubesOS/qubes-issues/issues/2132#issuecomment-255106656
[4]: https://github.com/kalkin/manager-new
> Particular attention will be paid
> to ensure that existing functionality is not lost, such as accessibility for
> keyboard-only power users.
+1 :)
> Once complete, I will bring a list of suggested features and enhancements to
> the development team, and perhaps to the community at large if this is
> deemed appropriate by my mentor. We can then discuss which of these features
> are suitable for me to undertake.
Ideally this would already have been done and included in your
proposal, but... it is what it is. IMO you should have a clear idea of
what to do and how to do it by the start of the coding period (May
30th).
> 3 Road Map
>
> Qubes will be my focus this summer, with continued contributions afterwards.
Note that for the purposes of evaluating your proposal, we are only
allowed to consider things being promised during the summer. Planned
future contributions are of course encouraged, but do not increase the
chances of your proposal being accepted.
> Because it is the Summer of Code I do intend to take a couple weekends
> hiking and camping in the mountains local to me, but I do not anticipate
> that this will conflict with my Qubes schedule.
Time for design reflection! ;)
> Below I have detailed a possible plan for delivering on the promises I have
> made above:
>
> Timeline
Compare yours to the actual GSoC timeline. [5] I think you are
beginning things too late.
You should be reading documentation and bootstrapping your familiarity
with our workflows for contributing code as soon as possible. Keep in
mind that June 26th is the first evaluation deadline, at which point
you are expected to have made reasonable progress towards the stated
goals of your actual proposal, not only (allegedly) read docs. The
general GSoC expectation is that the familiarizing happens before the
coding period begins, and during the coding period you really write
code.
[5]: https://developers.google.com/open-source/gsoc/timeline
> 3.0 - Installation and Familiarisation with Qubes, Setup qubes-builder
>
> April 3rd - May 30th
>
> As I am not currently intimately familiar with Qubes, I will reinstall it
> and transition to it as my day-to-day OS. Ideally I will build the OS from
> source, but I believe using Qubes will be the most effective resource when
> designing UX improvements.
Yes. Experiencing the Qubes UX yourself is a hard requirement for
working on it :)
I'd strongly recommend not taking a month just to install Qubes if you
can avoid it. The best way to learn is to just jump right in and force
yourself to use it for everything you do. I think you'll find the
qubes-users mailing list a welcoming and helpful place if you run into
any issues in doing so. I'd say switch entirely to Qubes ASAP.
One trick I've used for several friends to ease the migration is to
throw your existing system into an HVM so you can continue to use it
as-is while getting used to the Qubes way of doing things. This
requires you have another (larger) disk on hand though which you can
swap with your existing one. Not sure if this would help in your case,
but it's worth a thought.
> 3.1 - Reading Documentation, Identifying Skills, Trivial Contribution
>
> May 30th - June 6th
>
> Qubes has a plethora of user and developer feedback, and future plans. My
> first step to contributing to Qubes will be to immerse myself in this
> documentation.
IMO you should be doing this as soon as possible. Perhaps this should
be your first timeline section. Definitely don't wait until May 30th
(when Google wants you to be writing code) to begin reading docs!
Also, do not expect this to only take a week! Reading docs is very
much an ongoing process.
> A beautiful interface is useless if it is ineffective. Understanding and
> using the various system calls and configurations used by the QM will be
> crucial to improving it.
No system calls here! Well... there are of course, but not that you
need be concerned with. Not quite sure what you mean by configurations
either.
Anyway, I'm not sure you need to be too concerned with the
implementation details of the existing qubes-manager. Much is
changing, and for good reason. Much of the current internals (PyQt,
old-core-API) are not at all applicable to the intended future
version.
> In order to ensure that my environment is setup correctly and that I am
> adhering to existing style practices I should submit a trivial contribution
> to Qubes. An example of this would be the revising of the “DispVM” name
> across Qubes as documented in #2671 [4].
Definitely don't wait until May/June to do this.
> 3.2 - Learning GTK/Qt, Contribute a Bug Fix to Qubes, Compile Report on QM
> Requirements
>
> June 6th - 20th
>
> At this point, with guidance from Jean and Marek, the GTK/Qt decision will
> have been made
Already been made. It's GTK. I was the only person considering
otherwise, and Joanna convinced me that GTK is the way to go.
> and I will have begun implementing trivial UI components in
> Qubes. Using this knowledge, I will be able to contribute to one or more of
> the issues that have been pain points for users (e.g #1117) [5].
>
>
> Next, I will analyse the current QM and report on what parts belong in the
> new QM, and what should instead be included with the “current state” display
> widget. These findings will be taken to the team for feedback and will form
> the basis for the design of the new QM.
It is my understanding that this has for the most part already been done.
> 3.3 - Expand on QM Design by bnvk, Draft Design, Discuss with Development
> Team and Amend Proposal.
>
> June 20th - 27th
>
> Based on this report, I will draft UI mockups and design the specific tabs
> and menus of the new interface. This can be taken to the team for feature
> suggestions and improvements.
>
>
> I will compile a memo on my planned approach to the development of a new QM
> and discuss this with my mentor. As I receive feedback, the planned approach
> will be amended and improved.
IMO you have way too much planning and proposing (which duplicates the
planning and proposing which has already been done by others) and not
enough implementing.
I'd suggest moving all this earlier and getting started much sooner.
> 3.4 - Implement Proposed QM Core Functionality
>
> June 27th - July 11th
>
> Taking the memo and groundwork laid by bnvk, I will begin creating the new
> QM interface and porting features. Deliverables will take the form of a
> commit of a basic interface for the new QM, with the basic functionality
> implemented.
Can you elaborate on how this is different from the current state of [6]?
[6]: https://github.com/bnvk/qubes-manager-new
> 3.5 - Complete Core QM
>
> July 11th - 25th
>
> Complete the QM, including all features related to the persistent attributes
> of the system. Deliverable will take the form of a commit that can be
> installed and tested by the community at large.
https://www.reddit.com/r/restofthefuckingowl/
I guarantee this will take longer than you expect. This is the whole
essence of the project and will have many sub-tasks and unforeseen
surprises (perhaps such as "hmm... it seems we actually need X in the
core management API to implement this") which could each set you back
a week on their own.
John Casey
On Monday, 3 April 2017 06:56:39 UTC+1, Jean-Philippe Ouellet wrote: