GSoC Introduction

[Muhammad Usman]

Hi,

My name is Muhammad Usman and I am a third year PhD student (Software Engineering) at the University of Texas at Austin, USA. 

 

I am doing research in Automated Software Testing. I am interested in contributing to the Android Development in Qubes project. 

Please guide me about the next steps.

Let me know if you have any questions. 

Thanks,

Muhammad Usman

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, Mar 17, 2020 at 05:32:03PM -0700, Muhammad Usman wrote:
> Hi,

Hello Muhammad,

> My name is Muhammad Usman and I am a third year PhD student (Software
> Engineering) at the University of Texas at Austin, USA.
>
> I am doing research in Automated Software Testing. I am interested in
> contributing to the Android Development in Qubes project.
>
> Please guide me about the next steps.

Have you tried Qubes before? If not, I would recommend trying to install
it somewhere first. For testing purposes it is ok to install on external
disk, or even sufficiently large USB stick (although it will be quite
slow).

Once you have that, read this issue:
https://github.com/QubesOS/qubes-issues/issues/2233
There are various ideas how this could be achieved (not necessarily the
final design), you can also try some of them on just installed system.
This should give you ideas for the gsoc application.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl58FHwACgkQ24/THMrX
1yxdHwf/QD/URd+LBnNARmuBxL0bIt5npkj5H8oeyApIoWdKjDkK5qky+0KZXbN3
1LW/rpwbl8AMyg+vB4v/jLsgpHvE3bjLg3rLBgf0PftGAuq9B+Hp0UHOg5NI78Oy
mJyqeNpMMEs/dYMoeqX/bjnBIPxF4/sMBpVfGGqYFeU52vmORvAOOCi0YMDYBYqm
nmMf1uENcE7JMTgJyZJUxiJIWPZQu3wXKdwo22OP/fjkfcN0fLUmXQHftgg0Tlwk
h7chJjaF2H3BbrXOdbX2Z9o181l0vwZkGhNfh3ZdsmDQkkyWCv9ItePY2Rv/Jdb3
DGVSaNHQa7U4d/0VVQGVK7z/BW28KA==
=IwcB
-----END PGP SIGNATURE-----

[黃于軒]

Hi.

I am William Huang, an undergrad studying at National Taiwan University. I have had experience with Qubes OS and (albeit limited) interactions on the mailing lists (under the handle WillyPillow) since the v3.x days. (In fact, this project is probably my main reason to participate in GSoC 2020 :) )

The projects that interest me the most are the following.

- Mechanism for maintaining in-VM configuration
- Template manager, new template distribution mechanism

Both are features I have wanted as a user.

In addition, "Whonix IPv6 and nftables support" is also a project that I am quite interested in, having been messing around with iptables, pfSense, and stuff in my home network setup (and passing the IPv6 certification on HE.net :) ) recently.

Regarding the template distribution project, I would like some clarification. While <https://github.com/QubesOS/qubes-issues/issues/2063> seemed to be looking for an alternative for RPM, <https://github.com/QubesOS/qubes-issues/issues/2534> indicated that sticking with RPM (but not installing it directly) might be a good idea. Is it correct to say that the dev team is leaning towards implementing the mechanism on top of RPM?

Also, are there any extra resources to read regarding the three projects aside from the links on the GSoC page?


Thanks!

--William Huang / WillyPillow

&gt; https://blog.nerde.pw/
&gt;
&gt; PGP fingerprint = 6CCF 3FC7 32AC 9D83 D154 217F 1C16 C70E E7C3 1C84
&gt;
&gt; Protonmail PGP =&nbsp;D02D CEFF ACE5 5A7B FF5D 871E 4004 1CB1 F52B 127E

[donoban]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 2020-03-28 18:49, 黃于軒 wrote:
> Hi.
>
> I am William Huang, an undergrad studying at National Taiwan
> University. I have had experience with Qubes OS and (albeit
> limited) interactions on the mailing lists (under the handle
> WillyPillow) since the v3.x days. (In fact, this project is
> probably my main reason to participate in GSoC 2020 :) )
>
> The projects that interest me the most are the following.
>
> - Mechanism for maintaining in-VM configuration - Template manager,
> new template distribution mechanism
>

Hi,

What you mean with Template manager? If you refer to the "Template
Manager" integrated with Qubes Manager, I am working on a rewrite of
Qube Manager using QT model/view which simplifies things like multiple
row selection [1].

Currently it supports multiple VM's selection and
start/stop/remove/etc... options. I think that It should be easy to
integrate a Template change option in context menu and make the
current Template Manager obsolete. This way you could select some
halted VM's directly in Qube Manager, right click and choose "Change
Template", see submenu of available templates and select the desired one
.

I hope to finish the pull request soon (it delayed too long :/ ).

[1] https://github.com/QubesOS/qubes-manager/pull/195
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEznLCgPSfWTT+LPrmFBMQ2OPtCKUFAl6AfioACgkQFBMQ2OPt
CKUOQA//QZZ5T4jE/BPX5Hfen3C0whiyjadX/baWDl9n6Rva+ZkXfLmeaSLphkGN
B2OmZ4VNxpqIvFdTi+w9Mm0vYzlxZ0S9NLrveMCL5CV3LeNPU7VbrlD6RkvSzwTq
o+gV9f/Ta7soxCyYrS4Tx/m9vxPW9qJCInQpPf56VaEhWaO36TtXQfIp5cUgquxf
dbfyWCD8EdIibG6qQ17suey7FC4EDw25NWLDa8nh/rzfxyAA62aRmf4OfnDRfFKV
Ki2AvQc4m3mfp0nSeaxiD1TG3QIYoppN9hwtmhl+X6K4/SlIf7JZ7smC/UIEzq+t
7pYFWwuZ9Q6NPWcgVbtJ3gbX0mOEbkh8rq/hfizWpnul6nXKFr//XzBIqv5Rp/YN
8nhoiQ/cDjN66YxFhHrcnb9t78vugRhZ6eZsSVdrrO4riEM9EBAOeURglVfS7/Sk
CKC4ihLiI03zn7LEZ4wJ3WbHJCSr6Axz3PlAoRgTFZLWxP7xGwSnxf2A3bp79fxR
vuc6GEkcu0hTocQjueCMtenr9jy2img0USWhaWuw7XJ6HuC2tkiwgxvZD5JXkpkK
/qqG4S+6apRfjiBuDw8DZqxWZ2knjsF4LpOc0gHV5Y+UuFJy9ZGN3DQ7t8UnivrS
rIe+rUdmSMjL98RtHg3xKogBqqf/WT5Z90TdWfrumSRh2c6KkXg=
=HDvN
-----END PGP SIGNATURE-----

[unman]

On Sun, Mar 29, 2020 at 12:53:33PM +0200, donoban wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>

> On 2020-03-28 18:49, ????????? wrote:
> > Hi.
> >
> > I am William Huang, an undergrad studying at National Taiwan
> > University. I have had experience with Qubes OS and (albeit
> > limited) interactions on the mailing lists (under the handle
> > WillyPillow) since the v3.x days. (In fact, this project is
> > probably my main reason to participate in GSoC 2020 :) )
> >
> > The projects that interest me the most are the following.
> >
> > - Mechanism for maintaining in-VM configuration - Template manager,
> > new template distribution mechanism
> >
>
> Hi,
>
> What you mean with Template manager? If you refer to the "Template
> Manager" integrated with Qubes Manager, I am working on a rewrite of
> Qube Manager using QT model/view which simplifies things like multiple
> row selection [1].
>
> Currently it supports multiple VM's selection and
> start/stop/remove/etc... options. I think that It should be easy to
> integrate a Template change option in context menu and make the
> current Template Manager obsolete. This way you could select some
> halted VM's directly in Qube Manager, right click and choose "Change
> Template", see submenu of available templates and select the desired one
> .
>
> I hope to finish the pull request soon (it delayed too long :/ ).
>
> [1] https://github.com/QubesOS/qubes-manager/pull/195

I think this is more about maintaining, updating, and distributing the
*available* templates, than applying those to qubes, which is what you
are talking about.
It's a difficult topic but worth pursuing.

[donoban]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 2020-03-29 14:55, unman wrote:
>> What you mean with Template manager? If you refer to the
>> "Template Manager" integrated with Qubes Manager, I am working on
>> a rewrite of Qube Manager using QT model/view which simplifies
>> things like multiple row selection [1].
>>
>> Currently it supports multiple VM's selection and
>> start/stop/remove/etc... options. I think that It should be easy
>> to integrate a Template change option in context menu and make
>> the current Template Manager obsolete. This way you could select
>> some halted VM's directly in Qube Manager, right click and choose
>> "Change Template", see submenu of available templates and select
>> the desired one .
>>
>> I hope to finish the pull request soon (it delayed too long :/
>> ).
>>
>> [1] https://github.com/QubesOS/qubes-manager/pull/195
>
> I think this is more about maintaining, updating, and distributing
> the *available* templates, than applying those to qubes, which is
> what you are talking about. It's a difficult topic but worth
> pursuing.
>

Oh I see. Definitively a separated window where add, remove and
upgrade templates will improve Qubes usability.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEznLCgPSfWTT+LPrmFBMQ2OPtCKUFAl6AnXwACgkQFBMQ2OPt
CKX6bRAAqiUUJK+Huk8SwQQXfpiwSAepXJ0cQtvE8o3UdcaFWntH2ckbcJC1zA92
xloD46plu5jagFTnWDwzt4Hos4eEdy7Ne6qTRhh9BfjO5MQa1Ddoa3ELYCiRCDzo
TQB8cthml0Eu+GQFWSBflhYHrj+4bnhexI5FTReQ93NJOqAd8dfDaeDo5BmDKeIx
fnJW1ZQ0rJxEbVLbXUKFts53cG/A1Gswaf1dh9HIJkW54RaIkicMZoXZtR5oI4id
MWWRO9w4FfWE7o58VpRCuu5m0m6HtPGyJXgEfee43pK9cUJrvWGaxGrV4CsQl1SZ
oJB7qgLF3krr9aHHSRm20ugX7/5jOzWlh5FzPQdXg9cQ8xbMUTY8YkAxheVaA3xh
QPKcvavh3h5XS5uYYXcBi50q9RQMdfvDk2vVLOu0rSxb6uenaLPA65G8wrbI2qUz
/EDfQ2jGIBXnVemU8ENoBQPAi4zTh7DisoggzSQwm2LHsiNp0jDsYrlsnuhSoy5f
8FpsbUlOzyllBhDFptZMlpNFI0r0um86KPRkoDDMRSmCgG/nRDh0N3/TQaBR55Zr
EjveiBN3tW75R8iSq0vtfsh2xBgbqJ5u6wio+Y6LK7DO6r5aTvGmNcA1/YI5gqyB
lXT0ZbuSfDR2kFmhHXqNYf2/bhzhplx/N7NGkgOy2admgvhLlRU=
=6Z84
-----END PGP SIGNATURE-----

[WillyPillow]

On Sunday, March 29, 2020 9:07 PM, donoban <don...@riseup.net> wrote:

> On 2020-03-29 14:55, unman wrote:
>

> > > What you mean with Template manager? If you refer to the
> > > "Template Manager" integrated with Qubes Manager, I am working on
> > > a rewrite of Qube Manager using QT model/view which simplifies
> > > things like multiple row selection [1].
> > > Currently it supports multiple VM's selection and
> > > start/stop/remove/etc... options. I think that It should be easy
> > > to integrate a Template change option in context menu and make
> > > the current Template Manager obsolete. This way you could select
> > > some halted VM's directly in Qube Manager, right click and choose
> > > "Change Template", see submenu of available templates and select
> > > the desired one .
> > > I hope to finish the pull request soon (it delayed too long :/
> > > ).
> > > [1] https://github.com/QubesOS/qubes-manager/pull/195
> >

> > I think this is more about maintaining, updating, and distributing

> > the available templates, than applying those to qubes, which is

> > what you are talking about. It's a difficult topic but worth
> > pursuing.
>

> Oh I see. Definitively a separated window where add, remove and
> upgrade templates will improve Qubes usability.

Hi.

Indeed, I meant what unman was talking about. Thanks for helping me clarify that :)

--WillyPillow

> https://blog.nerde.pw/

>
> PGP fingerprint = 6CCF 3FC7 32AC 9D83 D154 217F 1C16 C70E E7C3 1C84
>

> Protonmail PGP = D02D CEFF ACE5 5A7B FF5D 871E 4004 1CB1 F52B 127E

[WillyPillow]

> --

Hi.

The following is a draft for my proposal to GSoC 2020 regarding templates. Feedback and critique are greatly appreciated.

(I apologize terribly for sending this so late. TBH I did not realize that Qubes was in GSoC until very recently :/)

# Introduction

This is a project intended to improve template handling in Qubes OS. Currently, images of template VMs are distributed by RPM packages and managed by `yum`/`dnf`. However, tracking inherently dynamic VM images with a package manager that is suited for tracking static files creates some challenges. For example, users may accidentally update the images, overriding local changes (#996, #1647). (Or in the case of #2061, want to specifically override the changes.) Other operations that work well on non-RPM template VMs are also somewhat inconsistent on RPM-managed templates, such as renaming (#839), removing (#5509) and backup/restore (#1385, #1453, [1], [2]), creating inconvenience and confusion for users (#1403, #4518).

In addition to the distribution mechanism, users may also wish to have an integrated template management application for better UX (#2062, #2064, #2534, #3040), as opposed to the current situation where multiple programs are required for different purposes, e.g., `qubes-dom0-update`, `dnf`, `qvm-remove`, `qubes-manager`.

To tackle these issues, I propose i) designing a better mechanism for handling template installation, and ii) creating a user-facing application to deal with the aforementioned mechanism and other template-related configuration, consolidating the management of templates.

[1]: https://groups.google.com/forum/#!topic/qubes-devel/rwc2_miCNNE/discussion

[2]: https://groups.google.com/forum/#!topic/qubes-users/uQEUpv4THsY/discussion

# Project goals

* Design a template distribution/handling mechanism
* Extracting and handling `root.img` from RPMs
* Template management application
* CLI/GUI
* Features:
* List available templates
* Download / install / reinstall / update / remove templates
* Switch VMs to certain templates
* Possibly other features mentioned in issues such as #2062, #2064, #2534, and #3040
* Ability to run outside of dom0 (UI for #1705)

# Implementation

The consensus among the developers seems to be that sticking with RPM but not installing the package directly is a better idea [3][3.1], which I agree with, in part because handling package integrity is a bit non-trivial and may lead to security issues (c.f. QSB-028 [3.2]).

Installed template versions can be kept either in a separate database or the metadata of the template VM. Alternatively, it is possible to keep the version number in the template name, with the benefit that multiple versions of the same template can be installed at the same time.

The extraction and verification of template packages can be done in DispVMs if necessary.

After extracting the `root.img`, it remains to install it via a process similar to the post-processing script in `linux-template-builder/templates.spec`, which mainly consists of calling `qvm-template-postprocess`.

The application can be written in Python to take advantage of APIs such as `qubes-core-admin` and `qubes-core-admin-client`. Also, the application can be consolidated with the existing Template Manager in `qubes-manager`.

Using the Admin API, it should be possible to use the tool outside of dom0, making the management VM scenario mentioned in the Admin API post [4] even easier.

[3]: https://github.com/QubesOS/qubes-issues/issues/2534#issuecomment-453749075

[3.1]: https://github.com/QubesOS/qubes-issues/issues/2064#issue-159825365

[3.2]: https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-028-2016.txt

[4]: https://www.qubes-os.org/news/2017/06/27/qubes-admin-api/

# Timeline

This project will very likely be my main focus for the summer. There may be some ICPC-training-related events at the end of August, but I do not expect it to interfere with my schedule much.

I am familiar with the mailing lists and am willing to report back my progress via email regularly.

* Now ~ June 2 (Week [-oo, 0]):
* Installation of Qubes on daily driver laptop
* Previous experiences with Qubes were on my desktop, which I now rarely use
* Familiarize myself with the related codebase and the Admin API
* Familiarize myself with PyQt / PyGTK
* Attempt trivial contributions and bug fixes to Qubes
* Week [1, 2]:
* Proposal amendments & draft designs & initial version of the design document
* Week 3:
* Initial CLI program with support for already-downloaded RPMs
* Week [4, 5]:
* Initial implementation of other features in the *Goals* section above
* Week [6, 7]:
* Initial GUI implementation
* Week 8:
* Buffer & clean up
* Week [9, 10]:
* Documentation
* Week [11, 12]:
* Additional features

# About me

I am an undergrad (sophomore) studying Computer Science at National Taiwan University. I have been a proud user of Qubes OS since around 2015~2016 (also played with `qubes-builder` to build kernels with NIC-related patches at that time), and have experience both on the mailing lists and the `qubes-issues` tracker. Moreover, I have written some (albeit simple) blog posts about Qubes OS, such as <https://blog.nerde.pw/2017/02/06/freenet-on-qubes.html>. While I do not have direct code contributions to Qubes (yet), my contributions to other OSS projects can be found under my Github/Gitlab profile @WillyPillow [5][6].

[5]: https://github.com/WillyPillow/
[6]: https://gitlab.com/WillyPillow/

I have more than 8 years of programming and Linux experience. Language-wise, I am familiar with C++ and Python (among other languages). Besides, being a bit of a data-hoarder, I am somewhat familiar with storage-related topics such as LVM. I am also familiar with mailing lists and tools like Git. I consider myself a quick learner and can pick up stuff pretty well as I go.

Timezone-wise, being someone who takes part in online competitive programming contests, I am fairly okay with adjusting my schedule to accommodate for events in other time zones.

While my native language is Mandarin Chinese, I have lived in the US for some while, and have experience with English-speaking online communities in general, so communication should not be an issue.

Since Qubes is the sole reason I am applying to GSoC this year, I do not plan to submit proposals to other organizations.

Qubes OS is a project that I have always been quite interested in, and I hope I can have the opportunity to work with the team on this project.

# Contact

<w...@nerde.pw>

Thanks.

--William Huang / WillyPillow