Packaging 3rd-party software

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

Currently most of Qubes OS repositories are about Qubes-specific code as
almost all other components we pull from upstream distribution(s). This
works well with our current repository naming scheme described here:
http://blog.invisiblethings.org/2013/03/21/introducing-qubes-odyssey-framework.html

As explained there, few of those are really 3rd-party software, but with
some Qubes-specific modifications. For example Xen (vmm-xen), Linux
kernel (linux-kernel), Libvirt (core-libvirt). In ideal world we'd use
upstream versions directly...

But recently there is more and more need to package additional software.
This is because either:
- package is Qubes-specific and unlikely to be included in upstream
distribution (for example screenshooting tool[1] made by Eva Star, or
qubes-vpn[5] by Manuel Amador)
- upstream package is outdated and very unlikely to updated (this is
mostly about dom0 packages, as we rarely update dom0 distribution);
for example tboot[2]
- package needs some Qubes-specific modifications (already mentioned
kernel, libvirt)
- package is missing in upstream distribution; for example pvgrub2 (Grub2
compiled with Xen support)[3], scrypt[4]

Again, in ideal world, none of those would apply, but well...

In the current scheme we'd use "linux-" prefix for such "generic"
packages (like linux-kernel, linux-pvgrub2), and "qubes-app-" for
Qubes-specifc extensions (all of them are maintained by Qubes team, so
not exactly apply to this case). But I'm not sure what is the best
approach here.

1. Stick with the current fuzzy rules: put the package in related
component if applicable - so tboot would go into antievilmaid
repository, otherwise create linux-* or qubes-app-* repository.

2. Always create linux-* repository.

3. Create one repository for all of them and use subdirectories (it's
mostly about .spec files and sometimes patches...)

4. Create new organization on github and separate repositories there,
possibly with different naming scheme - like "keep upstream name". If
this one, we need a name for this organization... "qubesos-packages"?

5. Something else?

Personally, I like the 4th option. Such approach is used by some projects
already, for example https://github.com/salt-formulas for salt stack
modules ("formulas").

Any thoughts?

[1] https://github.com/evadogstar/qvm-screenshot-tool
[2]
https://github.com/QubesOS/qubes-issues/issues/2155#issuecomment-250929993
[3] https://github.com/QubesOS/qubes-linux-pvgrub2
[4]
https://github.com/QubesOS/qubes-issues/issues/971#issuecomment-225843964
[5] https://github.com/Rudd-O/qubes-vpn

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJX/rY5AAoJENuP0xzK19csndYH/3n/MWwAEpglNYXH7YWQbJBf
9EfUcRay2qoSaiwDEPFDsCBs0d61VHuowJ947D4e2ON9TexoUa9FnSqW4ny+zcrz
wrI+ViVa5MBggSipcs87gjEbM+vqNl0Vmrr3nawrpJhcSbi7GpLz9HrfgTPx5O9Q
18f0tDO3J4CazMo/eY8K0dI/dpVyJ60BUVnao5UDifmjesimVWY3mPfIrX1wsBc4
kUqR7JFZ1xKha5fULW1j07K/heIxn1HXZXd6O8NUc1kt07E/CGJ3IbC7yddemNVJ
nwVpMLIAlqn06Nai3kG6wuDO5jA+Q227/DtW+DVZ0byGncHM6LuKWNcPScRlwdU=
=VVbK
-----END PGP SIGNATURE-----

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Option 4 sounds fine to me.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJX/xpnAAoJENtN07w5UDAwmSsP/3T+ci7CBoZEWQ92+EXAQ7de
te7kd1K0KZv+lJxa73QQRg74TaGYEdX28iy+Ufi0E3p/GLGGhDLa+e0RWJTZSIfo
yK7g2s/6AjLtHQKKCt/Tnw5+WR72rATpkpAL2ygqzWkL4LGwFkev76mAd1bahDJ2
+HRyEzigLbsduPOIEikkil5UEzv166zEC1XFVEEQa30s633HvF1nMkQ1plmU0+R8
h2iVUquityAwnrnN+fCp6TybkJoEPcDM/9YD4PC4pYkdhi2nK1UWkJWKy+FRWKOY
avyUddhBaycCx2Kl+/vlydOToxK775CPntMDQFtJzSaazRQmcXbxjlO4HlYruyEL
OY6M4W9EqvP0BGUpTwSakC4jcp0FzQmMmdrnV/65ApydnWqQS7YlFW1hbWdRgHp6
XT+rbjudtHQ+FQl7jcS0jtgE0Rj9BZpM7LvQGmNUvnmy0F4k4W7TxmZ6C84k+D/Z
20+t++WnNcFWZb4LUOuDTunDapJNOinPdITuhNdf2IW37rkkG6MvOgpwYFudDWxl
6GBcrrfnBzW908ASdTHbLzfVFakMFWIKV0H3+hArY6czl3/Kxny4kVpdI+10SS0P
pROM5lf3u5ECvKwVRLjoAqk4R0AAGdqLbys7MuWE3a2FyTPpLtw3XKQB36mBtTqC
ZEylWrj0Gl7V9jCHTvDr
=MfnX
-----END PGP SIGNATURE-----

[Wojtek Porczyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, Oct 12, 2016 at 10:23:55PM -0700, Andrew David Wong wrote:
> On 2016-10-12 15:16, Marek Marczykowski-Górecki wrote:
> > Hi,
> >
> > Currently most of Qubes OS repositories are about Qubes-specific code as
> > almost all other components we pull from upstream distribution(s). This
> > works well with our current repository naming scheme described here:
> > http://blog.invisiblethings.org/2013/03/21/introducing-qubes-odyssey-framework.html

> > 4. Create new organization on github and separate repositories there,
> > possibly with different naming scheme - like "keep upstream name". If
> > this one, we need a name for this organization... "qubesos-packages"?

> > Personally, I like the 4th option. Such approach is used by some projects
> > already, for example https://github.com/salt-formulas for salt stack
> > modules ("formulas").

> Option 4 sounds fine to me.

Second that.


- --
pozdrawiam / best regards _.-._
Wojtek Porczyk .-^' '^-.
Invisible Things Lab |'-.-^-.-'|
| | | |
I do not fear computers, | '-.-' |
I fear lack of them. '-._ : ,-'
-- Isaac Asimov `^-^-_>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJX/3ExAAoJEL9r2TIQOiNRkqAP/j1ZQNqRpT2ylUSvPN9eziWn
9qYVdAeEL2tBRLPbTo3QZ+/QMqQoVcz93fuTwqdyKUOiE3fLzMpVC9+ewiOy/PZA
uX+QRcKFf+cdKQzOblZUDViufqVbn2j7SfsWWVV8jyb2Iq7bq56Na90UbeKJM/Pz
BMiGW8briJAyrqZHCHeEC13YUUC9aWXqiYt22pTXxfVAAX81beRCKpP1WwAxo1eH
Xzcc4nKT8BY0KwZVMafYdkMf5EBSpsM6cm+3/rtuBZ60f2dyvIEhKpo02ThqL01y
McnDOobZEN1iOhYaIshEDjYBrMcnaRoxZ3Tvd4XiTV4goI/Ww1QP9QCHJlZrA+S1
qaYj1e++teyq0w4Q8WYpD3YKw8Lp8LVhew/XqdwwlLcQnUuzR/K3p9ddhgS+MP1o
hc9VOzGf+LhOaggqXiyGtjAsD2dq3wvPNqcwBCDggSZoA2o6KqsREu9ODCpwr9s9
3ioA8/FZ4oaRQiaO1iahoiVUSCb3o6dhufEyYG46X7mG6c896uWXdYn8rh7e7pTK
aQb8hHn0fPHBfrQYStb2bj+i582ddbn/ozvlk2UyQP1sB1uEQ5FDZbdxzoawVCOY
jQ6aoMEXJA+pC4KoWb1q8bR4OayM/HHOuA1VTVlq26KVXEJAtFVm6Z6976wUA4tj
QnM/alDXaJAHYDCUk+k1
=vwhV
-----END PGP SIGNATURE-----

[Manuel Amador (Rudd-O)]

On 10/12/2016 10:16 PM, Marek Marczykowski-Górecki wrote:
> Hi,
>
> [...packagin 3rd party software...]
> Any thoughts?

I think it depends on whether the 3rd party software is meant to be
upstreamed into Qubes OS.

For example, in the case of my tools, I would like them to be
upstreamed, therefore the ideal thing to do would be to incorporate them
into the QubesOS Github org, and then add them as extra sources in the
builder. That requires, of course, that Qubes OS the project provide a
proper process for vetting for upstreaming, and upstreaming vetted
software. Ultimately the Qubes OS devs end up controlling that
software, and the future contribution process is simply based on pull
requests.

For software that isn't meant to be upstreamed, but should be
installable, it would be great if there was a separate qubes-contrib
repo where people with cool Qubes extensions can upload packages —
again, following a review process, ideally based on diffs of source
RPMs, which only accepts SRPMS that were signed by their original
submitters — prior to them getting signed by the folks in charge of the
qubes-contrib repo. This would provide a distribution channel for those
packages. This is tricky though, because we are betting on the people
in charge of the repo (and building / signing the packages) to catch
potential malicious software being submitted. Effectively we need a
queue submitter -> new, signed SRPM up for review -> reviewer looks at
diff between last published SRPM and this incoming SRPM -> reviewer
approves -> builder builds -> built SRPM and RPMs -> reviewer does Q/A
on built products -> reviewer signs -> reviewer pushes to
qubes-contrib. This of course needs to be 100% fully automated.

And, of course, both options are labor-intensive.

--
Rudd-O
http://rudd-o.com/

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, Oct 13, 2016 at 02:21:24PM +0000, Manuel Amador (Rudd-O) wrote:
> On 10/12/2016 10:16 PM, Marek Marczykowski-Górecki wrote:
> > Hi,
> >
> > [...packagin 3rd party software...]
> > Any thoughts?
>
> I think it depends on whether the 3rd party software is meant to be
> upstreamed into Qubes OS.
>
> For example, in the case of my tools, I would like them to be
> upstreamed, therefore the ideal thing to do would be to incorporate them
> into the QubesOS Github org, and then add them as extra sources in the
> builder. That requires, of course, that Qubes OS the project provide a
> proper process for vetting for upstreaming, and upstreaming vetted
> software. Ultimately the Qubes OS devs end up controlling that
> software, and the future contribution process is simply based on pull
> requests.

This is generally a good idea, but I'm afraid some social effect: this
may look like taking the software away from the original author, taking
the credit for it. But on the other hand, the repository still will have
commit history, and "forked from ..." reference.

Andrew, any though on this aspect?

> For software that isn't meant to be upstreamed, but should be
> installable, it would be great if there was a separate qubes-contrib
> repo where people with cool Qubes extensions can upload packages —
> again, following a review process, ideally based on diffs of source
> RPMs, which only accepts SRPMS that were signed by their original
> submitters — prior to them getting signed by the folks in charge of the
> qubes-contrib repo. This would provide a distribution channel for those
> packages. This is tricky though, because we are betting on the people
> in charge of the repo (and building / signing the packages) to catch
> potential malicious software being submitted. Effectively we need a
> queue submitter -> new, signed SRPM up for review -> reviewer looks at
> diff between last published SRPM and this incoming SRPM -> reviewer
> approves -> builder builds -> built SRPM and RPMs -> reviewer does Q/A
> on built products -> reviewer signs -> reviewer pushes to
> qubes-contrib. This of course needs to be 100% fully automated.

This is nice idea, but for much larger scale. Writing/adopting such tool
(do you know anything like this?) alone require a lot of work. I think
it worth doing this, when just reviewing the submissions will become
full time job. On the other hand, I'd very like have such active Qubes
OS community ;)

> And, of course, both options are labor-intensive.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJX/8IHAAoJENuP0xzK19cs0LUH/0hnk1r6vEHkwps446KvPrW9
qlQI/SbSwRXAf8JuqrcXT9XAKyxaYe4vdsRh2vsfR5paKS4nFgWxChr1BU5SYLGq
iXyJcjDcXBObm9CXLkEWIb04K4KuupfCOl6xh4BnRmkulw5OwkRrZjNU4q7hUnXE
R2ik1sosUNPCKRo6k0r1gAMXDpaxttMNgziPz+ylQiHmPtCPU99duRzDy7cRPXcb
XVcL2lDqbDEagV3DZf66DqjCWQUDbo7lKVZmLheFRGSIL43MeLuuC0zyXW/0zksk
ReAczARvn3V3X94p+ajxrOmDn7Q1TBpBqbTwI97ljyZwYGtwL/ExCuZH+tOY7jE=
=TLko
-----END PGP SIGNATURE-----

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 2016-10-13 10:19, Marek Marczykowski-Górecki wrote:
> On Thu, Oct 13, 2016 at 02:21:24PM +0000, Manuel Amador (Rudd-O) wrote:
>> On 10/12/2016 10:16 PM, Marek Marczykowski-Górecki wrote:
>>> Hi,
>>>
>>> [...packagin 3rd party software...]
>>> Any thoughts?
>
>> I think it depends on whether the 3rd party software is meant to be
>> upstreamed into Qubes OS.
>
>> For example, in the case of my tools, I would like them to be
>> upstreamed, therefore the ideal thing to do would be to incorporate them
>> into the QubesOS Github org, and then add them as extra sources in the
>> builder. That requires, of course, that Qubes OS the project provide a
>> proper process for vetting for upstreaming, and upstreaming vetted
>> software. Ultimately the Qubes OS devs end up controlling that
>> software, and the future contribution process is simply based on pull
>> requests.
>
> This is generally a good idea, but I'm afraid some social effect: this
> may look like taking the software away from the original author, taking
> the credit for it. But on the other hand, the repository still will have
> commit history, and "forked from ..." reference.
>
> Andrew, any though on this aspect?
>

We certainly should aim for a solution that allows authors to retain
ownership of and control over their own software, as well as
receive credit and recognition for it, if only because this is likely
to promote more contributions from individuals who care (quite
reasonably, IMHO) about those things.

In options 1-4, the software ends up in a Qubes-owned repo no matter
what, right? It's just a matter of whether it's the official QubesOS
repo or a new repo we create specifically for that purpose.

If we want to allow authors to retain control (on GitHub) of their
own software (and not any other contributor's software), then it seems
like the only option is to allow each author to host their own
software under their own account (or an account they control).

What if we just fork those repos (either into the official QubesOS
account or into a separate one created for this purpose), then
update the forks based on changes to the author-owned original
repos (only accepting author-signed commits and/or tags, of course,
and perhaps after a review process)? Isn't that essentially what
we're already doing with i3[1], awesome[2], and yubikey[3]?
(Awesome yubikey are forked from repos owned by woju
and you, so maybe those don't really count.)

[1] https://github.com/QubesOS/qubes-desktop-linux-i3
]2] https://github.com/QubesOS/qubes-desktop-linux-awesome
[3] https://github.com/QubesOS/qubes-app-yubikey

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJX/9LIAAoJENtN07w5UDAwP6MP/jDvpkSu5EA+oIpBa2uiiVxC
0QD1EDbYT+YXsRc6aGFQJCfvEX5l6dmU4cB8bLyvd3CFUf0pk8J2JJ6fAWifhRgX
ivwFhDmqy32BD6efci7wkEZFzqaeoQ+njeK/u9Rcy0z2DU/fi1EXEytbPe8e3BAj
7WRxFnMSmoxSWnvDUIopJjvzz5D+Z6aas9Ci0mQuLS6X5dHgZiutf+J3akwIAFQr
wKQd/0SSs8Drg6CZrQN/EvcqcQc0T6wIogSdD/JV4L3eq8xpRTM6q/MrwmW3u3xr
fLWBlMDBzNuwdKy+suyTok1cKD2IvNM1RHhhfMdDUY+P67EsU1UVKnmn4ogz0ydQ
83L+O3X17T+bCeT23GyIWAXtp1FzhhXc/XjHx9g4Sn5khHkQbqUVOnzdG1mkw7DV
9NlWAgddu3DJW9Vs2yutXVO8UnaOtR5WTEattUKZS/0Oa1V/KN0jW7HZPyuhfQem
+l6qUKHH2M0ywZszcY/PHshCWgNGZ/ozg8BdkobJsB2sab2xi6B9bZfVa70se0pz
vvo2SaW1J/XTiD/bAdXTjs3y4sW/689nLfE+ETkflG+SocO0g1uaA4RhRjR9hrDB
KR5Rzw673+8UtH3VST09e7nf0wR+IqO7eVsLwL1TbNoLVtAIpbvtURIVoCo5nlFo
rzNjf1pM96g4qnNnPdmZ
=6fEE
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Not necessary - option 3 is about keeping only build scripts (.spec,
etc) in our repository, but pull the actual source from upstream.
I'm somehow reluctant for creating a single multi-package repository,
mostly because qubes-builder (currently) does not support building a
single package from such repo. So updating a single package means
rebuilding all of them. But should be easy to fix - for example keep
packages in separate directories (which is good idea anyway) and only
point qubes-builder at those subdirectories.

> If we want to allow authors to retain control (on GitHub) of their
> own software (and not any other contributor's software), then it seems
> like the only option is to allow each author to host their own
> software under their own account (or an account they control).
>
> What if we just fork those repos (either into the official QubesOS
> account or into a separate one created for this purpose), then
> update the forks based on changes to the author-owned original
> repos (only accepting author-signed commits and/or tags, of course,
> and perhaps after a review process)? Isn't that essentially what
> we're already doing with i3[1], awesome[2], and yubikey[3]?
> (Awesome yubikey are forked from repos owned by woju
> and you, so maybe those don't really count.)

Yes, this is the other option. The question here is: where should those
forks be (current github org or a new one) and how should be named
(original name, or some forced naming scheme like qubes-app-*)?

> [1] https://github.com/QubesOS/qubes-desktop-linux-i3
> ]2] https://github.com/QubesOS/qubes-desktop-linux-awesome
> [3] https://github.com/QubesOS/qubes-app-yubikey
>

- --

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJX/+0WAAoJENuP0xzK19csiOMH/RgGHaqCMFgbkkIARDDIzln8
f/+CmDswfN9lDSpHxLCOALPY7YWrmJqyMb60Vf/h9cYk1ctD1ecCC6Qmmf8kFSiG
QLXkmS6XeRPz4J2OhfQOHmsKYfqMbklnYnpuhGgbdaCibf5PUl/sY9dlP96dStep
H0TfppWwmoD3lCObup5LnBuySJk27YXhwLM23eMh7wI/np6qGNTbQSPgdLEi/lWK
lEWss5Tdj/231jtbgYFfgu3JDo5XOP9kMlZSO3anCQ47i0EmA69ngnciaeYxFSEQ
1HG0qUyXzxmT6bMBPxcQxb91Qj8SxR4IinwdysSC0CUGM7Z1dv+exdZiD9J9llI=
=nGEI
-----END PGP SIGNATURE-----

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Ok, I see.

> I'm somehow reluctant for creating a single multi-package repository,
> mostly because qubes-builder (currently) does not support building a
> single package from such repo. So updating a single package means
> rebuilding all of them. But should be easy to fix - for example keep
> packages in separate directories (which is good idea anyway) and only
> point qubes-builder at those subdirectories.
>

Yes, this seems like unnecessary overhead (compared to option 4).

>> If we want to allow authors to retain control (on GitHub) of their
>> own software (and not any other contributor's software), then it seems
>> like the only option is to allow each author to host their own
>> software under their own account (or an account they control).
>
>> What if we just fork those repos (either into the official QubesOS
>> account or into a separate one created for this purpose), then
>> update the forks based on changes to the author-owned original
>> repos (only accepting author-signed commits and/or tags, of course,
>> and perhaps after a review process)? Isn't that essentially what
>> we're already doing with i3[1], awesome[2], and yubikey[3]?
>> (Awesome yubikey are forked from repos owned by woju
>> and you, so maybe those don't really count.)
>
> Yes, this is the other option. The question here is: where should those
> forks be (current github org or a new one) and how should be named
> (original name, or some forced naming scheme like qubes-app-*)?
>

I think it's fine to choose option 4 here, in line with your preference.
Since it would be a dedicated GitHub organization, we could simply
preserve the upstream names. Or, if there are organizational benefits
to prefixing (e.g., to sort them into groups or categories), then
adopt some prefixing scheme. It probably wouldn't be beneficial to
prefix them all the same way (e.g., qubes-app-*). (Maybe the GitHub
organization itself should be called Qubes-apps or QubesOS-apps?)

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJX//pjAAoJENtN07w5UDAwlD0QAK8FgXEuUWHe/WQ+keD9ORjh
gyRBNf3G5Vf96tRAKCFWvFf1z4DLB2Rhmq8e6RtZBB9venvt0cirdzxKgOyCqGqH
4nvVTln96KJd774zhbfNQYMDLrhpT04kWvLy96FXUFj2auPcwuYpoV5riFH6DJxa
aEsP9mujYL4ERH/YVazoGONWxRph0oXGVLkzx9vwDksd1zTNWPaoYVMtIMKlRr23
/BXIh0F6eD6fDI1vj7JKgxYjpEwJku8U9x0Bf5d5dBepUIFo8tT2PB59KoZidGBi
cCXDCToXRGtmtPnFt8MjxvJJohL8qQM5V8zch4FsmWYSe2QydWD38MGsXdtk52xr
qkjLQxgevt3+LojgedhS+Hgku3aH9Cbpqh3iNSwCo7T+lOw7GNWjyCdRh/HDVAsw
eMtnSFQUGo+PJ/7h4L2rYT0KNOrWU9x2LfFwjQXFbdW44LmhHlGmEZaAhH94FR/7
uVEjRBm0NVOT+3DRw/GqtiFM95VMhPo4cVRviTFlb87rd7l7+NCKVykBpE411VLo
XFvBCD/7M7QFzqdIZfZQ+Txx/3oCY7ui5mzxbtfUrkwECowL1rdOLRL/5K7iTFF2
MxwsX7t9HjUYW9xBSftwJEW9FtbjB3yxOyiPhbwmEqnUiZ/EnOs0aslcIqEC4PCF
ugil9BorDiU8PXJBz3Ma
=sxMC
-----END PGP SIGNATURE-----

[Wojtek Porczyk]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Just a thought: can't we invite those original authors to this new
organisation? They could upload the code directly, but we'd still have to tag
the code for builder to verify. They'll have a nice logo on their GitHub
profile as a token of recognition.

- --
pozdrawiam / best regards _.-._
Wojtek Porczyk .-^' '^-.
Invisible Things Lab |'-.-^-.-'|
| | | |
I do not fear computers, | '-.-' |
I fear lack of them. '-._ : ,-'
-- Isaac Asimov `^-^-_>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJYAOOrAAoJEL9r2TIQOiNRnTwP/AxFOLphyb6QwhtFcWi29z2O
ehsvXVsP1sVD19HwLvohZk9Fxsd/IMG7lCgCDqPql/9fuOS2wzFXU8vARWXhJ6Dn
zS4WOsUT/a6P91UrDxyPiLAl2iKYQ1CZ03J3AGjyYpAMTYj8wuwWnu27KwCVj4Gu
5UlupIuWEQNOJUnT8cX+zGvmDKfwj9Fb13P+rZpo4aEdnxLRmsT/J+d4HMtMxe8j
dOOr91143cshZNHU15hF29WXC7er/+lGqacukYFuk/tS41BQb00uEnYo3m/r40+q
3PTKncwsv3EflNzCOteqFEBtklsKcQL/NVz1EWEOg6yhLkl116T06qwtloXcYZEN
+KYFkM9wlTdnaCV4MNAW6xy+v9E2CS+2BeC++Qawtffd10WKV1ziMIox3nmwrpLx
nu11bYkSYY0ZazPZB5UwYKXjSr8erjrnNukaIjnE4T3tEDO/zf4eVbUCrSO7y0oy
TFMUbmvZdVesDB10bMnEaGlMALAgFdu67pODdjRQBb7oObaIFwrxSudSxe28qfCU
AxUcr5sh4+9cBk8L1Qi5f8aLsR9F0/7r16zw5B7x/t3tamMfJA8i67tsXFPzH7sx
FAWEeNelmt6zubEBlQCrfUxpCLs8bIOtNpSY3t29Zx1RkY4BUepKM1DyNj+7Inij
/RJgFwDYG3q0naS7zhsR
=yxry
-----END PGP SIGNATURE-----

[Manuel Amador (Rudd-O)]

On 10/13/2016 09:19 PM, Andrew David Wong wrote:
>
> I think it's fine to choose option 4 here, in line with your preference.
> Since it would be a dedicated GitHub organization, we could simply
> preserve the upstream names. Or, if there are organizational benefits
> to prefixing (e.g., to sort them into groups or categories), then
> adopt some prefixing scheme. It probably wouldn't be beneficial to
> prefix them all the same way (e.g., qubes-app-*). (Maybe the GitHub
> organization itself should be called Qubes-apps or QubesOS-apps?)

QubesApps, to complement QubesOS.



--
Rudd-O
http://rudd-o.com/

[Manuel Amador (Rudd-O)]

On 10/14/2016 01:54 PM, Wojtek Porczyk wrote:
>
> Just a thought: can't we invite those original authors to this new
> organisation? They could upload the code directly, but we'd still have
> to tag
> the code for builder to verify. They'll have a nice logo on their GitHub
> profile as a token of recognition.

I like this idea — so long as the branches / tags to build are signed by
the right folks (releng@qubes-os), then the builder can kick in.

But this does need to have automation to upload signed packages and
such. So the pipeline thing I mentioned is still necessary in some form.

--
Rudd-O
http://rudd-o.com/

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Yes, good idea. Membership in the GitHub organization should be
available as an incentive for contributors. However,I don't think
uploading code directly to this organization (as opposed to the
contributor's own GitHub account) should be a requirement.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYAZWVAAoJENtN07w5UDAwKfEP/3rEWXVoC6w6sItOQsP1XMhP
ECmUE25aYPXtfhBUJoheuUTmVEv0IG6kQWx6X3RowfgVJ/azHp03tHsHoq8pOvIv
ClYC3PMf9KIrpe/99ca2S2/iriVhq1F+CsutqULqE0SpunJqIOL5BNvxY1Vtr5NV
nILYyDa/Lbm/bouUHtHaIVWo7rfgSswF4xp0vIrgw3uk2P8E9T9XsOZeRUSPIJqR
wWHVNA++0zLAJiXKkhDghqShNFF7Fl3wHdwDxNkxwUp05OkwT5Cv0mTTYuG/NYxO
0tQled1OM7tr4LULpfr80mtyY3jazPvu0tSOirFsE0dCMpJlRvtFT2q9jOJ/3xca
9TkMyNKIqxdfvjtxrIlkBvPjqEYcVss7f6TDhOCwmPsBwvrjlGx2wAeefaMiDSce
YBhZ1IOpH+2iyWISYqlM2FRUzPJb6WDUCMBx/9TWV0Epavp+FzWS0g3WhTXTA8yU
vwzqSKI8RlGzx7A7wQZng8JX9sWfD22ByVibXIGw/Eklm1FT486pMknRUpapI8Vq
LDRjmAAVNvxBawyn4cm09ikIyQ/W6+PpJTU3V3ARVwycOLyGiAWN9S+g/ZjFGyVp
M7RYpMCQaoXiPVywtnzwNDvwJ6OrhhI7t2g4AjfIgMat0yy9iMrpKVV01ceB1Ii+
C+lpWVWtb3J7xAT0TQSg
=ov9x
-----END PGP SIGNATURE-----

[Holger Levsen]

On Thu, Oct 13, 2016 at 01:34:13PM +0200, Wojtek Porczyk wrote:
> > Option 4 sounds fine to me.
> Second that.

same here, even though I'm late to this party. :)

Regarding the name, maybe something more general, like
QubesOS-3rdparty? I fear that QubesOs-Apps could become too wrong /
misnomed too soon…


--
cheers,
Holger

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Coming back to this idea.

I think handling contributions there can be easily automated with
https://github.com/QubesOS/qubes-issues/issues/1818
So, the only actual work needed, will be reviewing changes, then adding
a signed tag. Everything else will be handled automatically.

Things to decide:

1. Should those packages be put into the same repository as main qubes
packages? And consequently - signed by the same or a different key?

For having it together:
- easier for users
- we'd like to have some of those packages installed by default (for
example screenshoting tool) - so, repository would be probably
enabled by default anyway
- after enabling repository, its key is imported into rpm database, and
then can be used to verify any package (even downloaded from a
different repository) - so at this level it gives very little
separation between repositories; BTW the same apply also to apt
(debian)

For having it separate:
- even though tricky (see above), it is still possible to trust one but
not the other
- separate build environments (which will be there anyway) are more
meaningful (ability to check where the package was built)

I'm slightly leaning towards having it separate, but enabled by default.
But I'd like to hear other opinions.

Extra thought: maybe builder-github should be extended to push
notifications somewhere, even if no issue was referenced? If so - where?

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYS2GYAAoJENuP0xzK19csPFkH/RszeynRDXgXklYNYzhuFsFt
GuBDTDnXXdauem5U2ZgfIth7lHjkVg6Tc3pFMmXW/Ls7+x2bFdyvIp9ZmWnlHXYj
w+uwqjHkigIkgKGTvru/QxMw9YUOktcjhdD6gjPlWO2SZhOPjwgVaxTKbodIoHei
LlbOpz+xa3Zyu55c4wcDWsR9ecsQzz7NMz9rLn7h4YOsQW++aWPJ+Yq5WHH9NHr/
nPWZ6hYa710atwpRhy6QZYpelYMbB53Vyeu9Ht+rmPE75LWdufaAFrEZJeBBt+xl
/OCfeGo5ivlrZqvcQJKM8kk6JPlmiAbRPcp3MEvoR+m5NvnrPAk3oufhWuSRM4A=
=VKzH
-----END PGP SIGNATURE-----

[Jean-Philippe Ouellet]

On Fri, Dec 9, 2016 at 8:59 PM, Marek Marczykowski-Górecki wrote:
> I think handling contributions there can be easily automated with
> https://github.com/QubesOS/qubes-issues/issues/1818
> So, the only actual work needed, will be reviewing changes, then adding
> a signed tag. Everything else will be handled automatically.

I think an important open question is: How should code-reviews be tracked?

I think qubes-builder's model of not having untrusted code in the
place that trusted code may be expected [1] is a good model. The same
can be applied here with respect to pushing un-reviewed vs.
peer/qubes-core reviewed code to "official" repos.

I imagine this:
1) A community member comes up with some cool thing they want to add
to qubes, which is clearly additional separate functionality that does
not belong in a core repo. (Thinking qubes-app-linux-* here.)
2) The community member writes said code in their own repo.
3) When they consider it v1.0, they propose for it to be added to the
set of core-approved community-contributed things.
4) The code is reviewed by core qubes people (and hopefully other
community members) and a the reviewed copy is added to a newly created
github.com/qubesos-contrib/qubes-app-foo-bar repo.
5) The community member makes some additional changes (features, bug
fixes, etc.) and pushes it to their own repo, and opens a pull request
to the qubesos-contrib/... repo.
6) The updates are reviewed, and if OK, merged.

This maintains the invariant that qubesos-contrib/... repos (all heads
of all branches) are always in a reviewed (and hopefully trustworthy)
state, which I argue is a nice property.

Some issues with this:
1) There is no longer any reason for the community member (and author
of the contributed component) to be a member of the 2nd github
organization (no nice badge on their github profile, which for some
may serve as incentive to contribute)
2) If everything is to be reviewed before merging, then there is less
of a reason to have a separate qubesos-contrib org in the first place.
3) How should patches to these contributed components from people
other than the original author be handled? Should we require they be
merged (or at least OK'd, perhaps with some timeout) by the original
author?

[1]: https://www.qubes-os.org/news/2016/05/30/build-security/#the-sources-verification-bug

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

As we don't trust the external infrastructure, everything downloaded
from the network have signatures checked - this include checking git
tags on git fetch/pull. And I think exactly this mechanism can be used
to mark reviewed code. So, a community member could push a code to
qubesos-contrib/... repo, then wait for it being reviewed - and when got
reviewed, will get tagged by qubes core team. This will allow (and
actually will trigger) build process.
Depending only on github repo permissions is against the rule of not
trusting the infrastructure.

Also, I think it will be less work for qubes team to just stick tags
after reviewing the code, than merging pull requests (and handling
possible conflicts etc) - this should be a role of tool maintainer.

> Some issues with this:
> 1) There is no longer any reason for the community member (and author
> of the contributed component) to be a member of the 2nd github
> organization (no nice badge on their github profile, which for some
> may serve as incentive to contribute)
> 2) If everything is to be reviewed before merging, then there is less
> of a reason to have a separate qubesos-contrib org in the first place.
> 3) How should patches to these contributed components from people
> other than the original author be handled? Should we require they be
> merged (or at least OK'd, perhaps with some timeout) by the original
> author?
>
> [1]: https://www.qubes-os.org/news/2016/05/30/build-security/#the-sources-verification-bug

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYS389AAoJENuP0xzK19cs8rEH/RNc9/yy2wGNxbQNXllRcigU
VhJ/NASRA5nafn+JahSySJy7zEmH6YatMg/hmWId+hS9MNsW0DKn41XVTSwdbD6L
L2eJHDDP4c1SGEBBeps/D5gtjEc0Vabc58L1IfqGcDSM8tQgkQ2xz93DtQrJsYTp
CkCepxBSvtDQdE+eJK+KMd+LFPnEBfPsSIUdlgLOpD58ejM8beYo18HDSJcIRXka
qsuTHl5vKCTKDamVGXXIftavLMOdBs+vhgSoBp0z7HD+FP6JJLwmCS4i/45GJGpr
LmM8deH3XDvecNjNs7e3flbk8rKm3pKdUglLhBGUEADNOJpE7jaxain/I8nlznc=
=qJcT
-----END PGP SIGNATURE-----

[Jean-Philippe Ouellet]

(Re-ordered the quoted email for more readable reply)

On Fri, Dec 9, 2016 at 11:06 PM, Marek Marczykowski-Górecki
<marm...@invisiblethingslab.com> wrote:
> Depending only on github repo permissions is against the rule of not
> trusting the infrastructure.

I definitely agree, and on the "here's what we actually download and
compile" there is no substitute for consistent signature verification.
However, for the purpose of tracking what work is to be done, we
already trust github very much.

> As we don't trust the external infrastructure, everything downloaded
> from the network have signatures checked - this include checking git
> tags on git fetch/pull. And I think exactly this mechanism can be used
> to mark reviewed code. So, a community member could push a code to
> qubesos-contrib/... repo, then wait for it being reviewed - and when got
> reviewed, will get tagged by qubes core team. This will allow (and
> actually will trigger) build process.

The purpose of the github PRs here would be to improve the workflow of
a maintainer telling the world "Hey, here's a set of complete and
locally-tested changes, please review." with two goals:
1) Help ensure things get reviewed in a timely fashion and don't get
forgotten about.
2) Provide a clear default way to provide feedback in the case of
unacceptable changes.

This is a much more familiar workflow to people than a push of a
signed tag meaning "please review this, and push another signed tag".
What if these changes are unacceptable when reviewed? Do we open an
issue? AFAICT PRs are just issues with additional user interface to
refer to specific changes in exactly the manner we would need to
poorly emulate with issues.

The actual reviewing should happen on a developers local machine (not
github UI), to:
1) verify that the changes being reviewed are indeed what the
maintainer proposes (by verifying the maintainer's tag)
2) verify that all commits since the last review have indeed been
reviewed (as determined by the reviewer's own last signed tag)

Then it's a simple fast-forward of master pushing to close the PR. No
additional trust in github besides essentially for to-do list
management (which we already do).

IMO something like this should already be the workflow for reviewing
PRs to core (non -contrib) code. (Perhaps it is, idk.)

> Also, I think it will be less work for qubes team to just stick tags
> after reviewing the code, than merging pull requests (and handling
> possible conflicts etc) - this should be a role of tool maintainer.

I guess that answers:

>> 3) How should patches to these contributed components from people
>> other than the original author be handled? Should we require they be
>> merged (or at least OK'd, perhaps with some timeout) by the original
>> author?

Slightly off-topic:

This discussion reminds me that I should really get around to standing
my PGP infrastructure back up and signing select mails again :) I
suppose I'm a bit of a hypocrite in this regard!

I stopped because the security of my key was depended on by others for
code-signing, vulnerability reporting, and web-of-trust verification
(I'm rather well cross-signed), and I deemed having my private key
accessible to my entire daily-use machine to be an unacceptable risk.
Of course, the entire reason for Qubes' existence is to decrease this
category of risk, so perhaps it's time to reconsider. :)

[Jean-Philippe Ouellet]

I guess more succinctly:

I think signed tags triggering builds makes a lot of sense.
I think signed tags triggering code reviews does not.

I believe that a workflow involving PRs would be preferable because:
- Random comments on commits are IMO not an effective way to keep
track of issues.
- PRs notify reviewers to review and provide an expected location for
feedback in the event of unacceptable proposed changes.

[Jean-Philippe Ouellet]

On Sat, Dec 10, 2016 at 12:05 AM, Jean-Philippe Ouellet <j...@vt.edu> wrote:
> IMO something like this should already be the workflow for reviewing
> PRs to core (non -contrib) code. (Perhaps it is, idk.)

I am actually quite curious what your reviewing workflow currently
does look like.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sat, Dec 10, 2016 at 12:05:07AM -0500, Jean-Philippe Ouellet wrote:
> (Re-ordered the quoted email for more readable reply)
>
> On Fri, Dec 9, 2016 at 11:06 PM, Marek Marczykowski-Górecki
> <marm...@invisiblethingslab.com> wrote:
> > Depending only on github repo permissions is against the rule of not
> > trusting the infrastructure.
>
> I definitely agree, and on the "here's what we actually download and
> compile" there is no substitute for consistent signature verification.
> However, for the purpose of tracking what work is to be done, we
> already trust github very much.

Yes, that's right...

> > As we don't trust the external infrastructure, everything downloaded
> > from the network have signatures checked - this include checking git
> > tags on git fetch/pull. And I think exactly this mechanism can be used
> > to mark reviewed code. So, a community member could push a code to
> > qubesos-contrib/... repo, then wait for it being reviewed - and when got
> > reviewed, will get tagged by qubes core team. This will allow (and
> > actually will trigger) build process.
>
> The purpose of the github PRs here would be to improve the workflow of
> a maintainer telling the world "Hey, here's a set of complete and
> locally-tested changes, please review." with two goals:
> 1) Help ensure things get reviewed in a timely fashion and don't get
> forgotten about.
> 2) Provide a clear default way to provide feedback in the case of
> unacceptable changes.

Indeed, especially the second point makes very much sense. I think we
should require such PR to be fast-forward, to avoid conflict resolution
(or unintended interactions of changes, even without conflicts).

> This is a much more familiar workflow to people than a push of a
> signed tag meaning "please review this, and push another signed tag".
> What if these changes are unacceptable when reviewed? Do we open an
> issue? AFAICT PRs are just issues with additional user interface to
> refer to specific changes in exactly the manner we would need to
> poorly emulate with issues.
>
> The actual reviewing should happen on a developers local machine (not
> github UI), to:
> 1) verify that the changes being reviewed are indeed what the
> maintainer proposes (by verifying the maintainer's tag)
> 2) verify that all commits since the last review have indeed been
> reviewed (as determined by the reviewer's own last signed tag)
>
> Then it's a simple fast-forward of master pushing to close the PR. No
> additional trust in github besides essentially for to-do list
> management (which we already do).

Ok, I agree. But I still think another organisation makes sense. Mostly
for two reasons:
- badge for the author of such tool
- different (less strict) rules for repository naming scheme

> IMO something like this should already be the workflow for reviewing
> PRs to core (non -contrib) code. (Perhaps it is, idk.)

It is actually the case for non-trivial changes, take a look for example
here (and also merged ones):
https://github.com/QubesOS/qubes-core-admin/pulls
Github probably show "merged by marmarek" in all the cases, but for
core3-devel branch (upcoming Qubes 4.0) the actual workflow is:
1. Pull request against qubesos/qubes-core-admin:core3-devel
2. Wojtek review the changes and push to
woju/qubes-core-admin:core3-devel-staging (together with signed tag of
course)
3. Travis CI run automated tests and if succeed, update his core3-devel
to what is in core3-devel-staging (this is fast-forward, so signed tag
still matches the code)
4. Another script monitor what is in woju/qubes-core-admin:core3-devel
and update qubesos/qubes-core-admin:core3-devel accordingly (only if the
change is fast-forward and properly signed of course).

This is the workflow we started in this repository and probably will
implement in others too (especially using PR, then pushing to -staging to
always go through automated tests). Another practice used by many
open-source projects it to route all the patches through the mailing
list. But I find github more convenient, plus it preserve signed tags +
commits...

Previous workflow was to push to personal repository first (in many
cases directly to master branch) and push to qubesos org after changes
being reviewed. This is why historically code in my github account was
fresher than in qubesos org.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYTA0xAAoJENuP0xzK19csYxgH/imqsBItDjr67MCC1VWwpCTc
Q25dw/Vd24L1MmxI9fCsDpTMg8AfHFnXYFRoDiHrCcEKXQwzv2JzH4BfWqKGqZwH
Dm94LKcMSQlUAUkP5oQf7E6um5HvL7BIdtKI015IXrsaDZjhIxqx25lHTkOuFBYH
cUY6YcjC4BopaVX6pNWdCKpdw/rGWzdRD70kQHl/+mDR5I7jiehiKQ6qkTCD+7TY
lDKZlZNzxkG82Is8GYVJShSzzvTkN1wiqgEGuGFhr9FHVQ8GjjaEhM/4xQrGUFgc
8lR4gzRBrIOAYEoaiJ7J/bASCwOCEtdKOlq80tPNjH7mvWeTyvInyDTM2h5LMG8=
=K0P+
-----END PGP SIGNATURE-----

[Jean-Philippe Ouellet]

On Sat, Dec 10, 2016 at 9:12 AM, Marek Marczykowski-Górecki
<marm...@invisiblethingslab.com> wrote:
>> The purpose of the github PRs here would be to improve the workflow of
>> a maintainer telling the world "Hey, here's a set of complete and
>> locally-tested changes, please review." with two goals:
>> 1) Help ensure things get reviewed in a timely fashion and don't get
>> forgotten about.
>> 2) Provide a clear default way to provide feedback in the case of
>> unacceptable changes.
>
> Indeed, especially the second point makes very much sense. I think we
> should require such PR to be fast-forward, to avoid conflict resolution
> (or unintended interactions of changes, even without conflicts).

Perhaps it goes without saying, but for me a main reason to require
fast-forwards is it enforces your already-reviewed signed-tag to be an
ancestor commit of the proposed changes, which means incremental
reviewing is meaningful. Otherwise you would need to perform a larger
audit each time, which wastes much time and effort.

> Ok, I agree. But I still think another organisation makes sense. Mostly
> for two reasons:
> - badge for the author of such tool
> - different (less strict) rules for repository naming scheme

Makes sense.

However, I still wonder about whether the maintainer having direct
(not via PR) push access is a desired property. I can see no reason
for it other than a "sense of ownership" over ones code, which does
not really mean much since everything must be reviewed before packages
are built/signed/uploaded anyway, so the maintainer does not have the
final say regardless.

> This is the workflow we started in this repository and probably will
> implement in others too (especially using PR, then pushing to -staging to
> always go through automated tests). Another practice used by many
> open-source projects it to route all the patches through the mailing
> list. But I find github more convenient, plus it preserve signed tags +
> commits...

I like this idea very much. Sometimes I find myself prematurely
opening pull-requests just to trigger CI, then force-pushing several
would-be-v1, -v2 patchsets to the comparing branch shortly after. This
is non-ideal IMO.

> Previous workflow was to push to personal repository first (in many
> cases directly to master branch) and push to qubesos org after changes
> being reviewed. This is why historically code in my github account was
> fresher than in qubesos org.

Works too, but minus CI.

Local testing is great, but I'm not gonna kick off a full ISO build on
my laptop for every trivial patch, and occasionally things have
unforeseen impacts elsewhere in the tree.

[Jean-Philippe Ouellet]

On Sat, Dec 10, 2016 at 9:35 AM, Jean-Philippe Ouellet <j...@vt.edu> wrote:
> On Sat, Dec 10, 2016 at 9:12 AM, Marek Marczykowski-Górecki
> <marm...@invisiblethingslab.com> wrote:
>> This is the workflow we started in this repository and probably will
>> implement in others too (especially using PR, then pushing to -staging to
>> always go through automated tests). Another practice used by many
>> open-source projects it to route all the patches through the mailing
>> list. But I find github more convenient, plus it preserve signed tags +
>> commits...
>
> I like this idea very much. Sometimes I find myself prematurely
> opening pull-requests just to trigger CI, then force-pushing several
> would-be-v1, -v2 patchsets to the comparing branch shortly after. This
> is non-ideal IMO.

To be clear, I mean I like -staging, not necessarily LKML-style code-review.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sat, Dec 10, 2016 at 09:35:33AM -0500, Jean-Philippe Ouellet wrote:
> On Sat, Dec 10, 2016 at 9:12 AM, Marek Marczykowski-Górecki
> <marm...@invisiblethingslab.com> wrote:
> >> The purpose of the github PRs here would be to improve the workflow of
> >> a maintainer telling the world "Hey, here's a set of complete and
> >> locally-tested changes, please review." with two goals:
> >> 1) Help ensure things get reviewed in a timely fashion and don't get
> >> forgotten about.
> >> 2) Provide a clear default way to provide feedback in the case of
> >> unacceptable changes.
> >
> > Indeed, especially the second point makes very much sense. I think we
> > should require such PR to be fast-forward, to avoid conflict resolution
> > (or unintended interactions of changes, even without conflicts).
>
> Perhaps it goes without saying, but for me a main reason to require
> fast-forwards is it enforces your already-reviewed signed-tag to be an
> ancestor commit of the proposed changes, which means incremental
> reviewing is meaningful. Otherwise you would need to perform a larger
> audit each time, which wastes much time and effort.

A `git diff` could be used to avoid this, but still it makes much sense
to have it always fast-forward.

> > Ok, I agree. But I still think another organisation makes sense. Mostly
> > for two reasons:
> > - badge for the author of such tool
> > - different (less strict) rules for repository naming scheme
>
> Makes sense.
>
> However, I still wonder about whether the maintainer having direct
> (not via PR) push access is a desired property. I can see no reason
> for it other than a "sense of ownership" over ones code, which does
> not really mean much since everything must be reviewed before packages
> are built/signed/uploaded anyway, so the maintainer does not have the
> final say regardless.

Being member of the organization does not imply write access to the
repository. I agree this somehow makes the separate organization
a bit weird, but those two points above still holds.

> > This is the workflow we started in this repository and probably will
> > implement in others too (especially using PR, then pushing to -staging to
> > always go through automated tests). Another practice used by many
> > open-source projects it to route all the patches through the mailing
> > list. But I find github more convenient, plus it preserve signed tags +
> > commits...
>
> I like this idea very much. Sometimes I find myself prematurely
> opening pull-requests just to trigger CI, then force-pushing several
> would-be-v1, -v2 patchsets to the comparing branch shortly after. This
> is non-ideal IMO.

You know, you can push a branch, wait for Travis and only then open PR
;)
This require just signing up to Travis yourself (and enabling it for
particular repository).

> > Previous workflow was to push to personal repository first (in many
> > cases directly to master branch) and push to qubesos org after changes
> > being reviewed. This is why historically code in my github account was
> > fresher than in qubesos org.
>
> Works too, but minus CI.
>
> Local testing is great, but I'm not gonna kick off a full ISO build on
> my laptop for every trivial patch, and occasionally things have
> unforeseen impacts elsewhere in the tree.

Actually not all tests can be run in Travis. There is no running Xen
instance, so it does not replace local testing. In practice we call full
test run from time to time (for example just before pushing packages to
current-testing repository, or just after it). The full test run takes
several hours, often more than 24...

We have a partial infrastructure to plug the full test run into -staging
workflow automatically, it is isn't finished yet. So for now, it is
triggered manually.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYTBhIAAoJENuP0xzK19csjUYIAJfS1dvZDbtRFBGoCxNOiJUN
EyHfGIDf64Dq7CVAYdl86USi/y4eEZL81/4WKlxjz6GEbQA1m8VhrBh+4lQI35ZK
GDwSvT038V6N/7Nz4yJPn9X3He8XSxI4jvd92gFfZaQBTnQx3hQFl+Dfkt9eAT4N
3IHUFoqqQbcOndKa17WvfVDK0JdqxV/8v9JG6y/U4qZ8zR4hPKH+OsMOuakW91Jp
JsU7w8iaP8X/QrY7UcMBykQbFzw/U+ZfU+AEdTMKDCP0ZXnr/pHRlkN0B7fdZOg5
S4dg4Ar3ApqCh9XDNEqXVuC6Xo/9mQixL0v00bQarl6YHKE/28wbSQRiaA4myuE=
=DS6x
-----END PGP SIGNATURE-----

[Jean-Philippe Ouellet]

On Sat, Dec 10, 2016 at 9:59 AM, Marek Marczykowski-Górecki
<marm...@invisiblethingslab.com> wrote:
> A `git diff` could be used to avoid this, but still it makes much sense
> to have it always fast-forward.

This does not catch someone rewriting old commit messages to hide
things. AFAIK there is no better way than manually checking each
commit since the common ancestor.

> Being member of the organization does not imply write access to the
> repository. I agree this somehow makes the separate organization
> a bit weird, but those two points above still holds.

Ah, I did not understand that. In that case all seems good :)

> You know, you can push a branch, wait for Travis and only then open PR
> ;)
> This require just signing up to Travis yourself (and enabling it for
> particular repository).

I tried to do that, but (at least at the time) the authorization
scheme used gave them a token enabling write access to the contents of
(all!) your repositories, and I did not want to allow that. Perhaps
this has changed since, idk.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sat, Dec 10, 2016 at 10:47:02AM -0500, Jean-Philippe Ouellet wrote:
> On Sat, Dec 10, 2016 at 9:59 AM, Marek Marczykowski-Górecki
> <marm...@invisiblethingslab.com> wrote:
> > A `git diff` could be used to avoid this, but still it makes much sense
> > to have it always fast-forward.
>
> This does not catch someone rewriting old commit messages to hide
> things. AFAIK there is no better way than manually checking each
> commit since the common ancestor.

You can still use notation like `git log -p ..FETCH_HEAD`. But this does
not give you full picture about code differences (because of changes in
your local branch). Anyway we all agree that only fast-forward PRs
should be allowed here.

> > You know, you can push a branch, wait for Travis and only then open PR
> > ;)
> > This require just signing up to Travis yourself (and enabling it for
> > particular repository).
>
> I tried to do that, but (at least at the time) the authorization
> scheme used gave them a token enabling write access to the contents of
> (all!) your repositories, and I did not want to allow that. Perhaps
> this has changed since, idk.

Hmm, currently it looks to have write access only to:
- commit statuses
- deployment statuses
- hooks
For all your repositories.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYTDbgAAoJENuP0xzK19csXLwH/1eUzJD51o/Jf08B7Fn8dt6i
p3eDgxux0pvbNLwOkzQzq6Hd5wk1530m/hMrr99FeDIIpRLXIxYsGqqtl39y2XGD
qp7mZTNUw8wNxsoR9QpDo72KecTRyfwgScKM9ZdjogarKeZYtNuHci1yt9k/Vgjq
BV1U5oF8UtFjQI2ruuCKGYAd+9wKKNRArFH/qNOOke3yjVrzNs4clB9fsz2EDm8T
z0o9ofjNz3HVH1xHRlZdVNi0qQAJdlTFgVagj3pwxWs+avRRTNgk6DILSGl0Ah4E
wcGJED7qt5gothoPSBd4Mr5xK+nsb74uK8UGZ9pno2+fuI3gO+ucEgBQ3PuERoE=
=cJdd
-----END PGP SIGNATURE-----

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 2016-12-09 17:59, Marek Marczykowski-Górecki wrote:
> [...] I'm slightly leaning towards having it separate, but enabled

> by default. But I'd like to hear other opinions.
>

IMO, separate sounds fine.

> Extra thought: maybe builder-github should be extended to push
> notifications somewhere, even if no issue was referenced? If so -
> where?
>

Can we just create an issue for the contribution if one
doesn't already exist, then push notifications there?

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJYTRK6AAoJENtN07w5UDAwEM0P/RQng/1b4Covo2McwJsRG4jM
UPqzPnYLamSTMQcA3LmsgfUsrO4usxpTZbTypFprh9qw+jDm79i83+l4bgEW1hk/
u4XrYjJ8WgKz6pNArrFyeX0EeMuUxIZn94yB2Vo3JWtSLyWaNg8mhS7Ie3Wl3qnO
ziyKqidTPKQIKuPpRBxdRQ+yO4MpIUI0B95oGoXYkkoyW5lgh0HWKxNgepD78zL7
+7qgUlO6c27hkSoSz5I4idDyiIywv+OnNNCWQT0yGzIEzIYixZnx6/v3tukDwSR9
UXq3AV+CKm9fsUqaYj4Y0oHJyv75I8ATCQOguWRtyKVLyYG+F9v59LYu2QAx4j88
IwMddZIj9x+S0tw+MBnxGDwX3oSIjEMbJgW7y4Jh19EoC7nNaGrnWbeIX2Nv6ch0
IxzVS96z4G5OkRzAI95ze4PIEDLmdxEp8iTIaQyxOBuptPvcNgjb6D9C2beTBUa+
p57d4WbrGoV0lckrWfRz9vj0OnbzsAR1bmaEMC4lkEME2Tk6wxmwLJrbVONit4Q0
NytF/lE/ARsmq0ude/YDFjVZZNUaQUF3EOOrYXlcW+Uki1kcblz0VlCZ/UPOOq6z
p7SLk0OypbbuLcgeF31QKkn38IxPuoYAAD6EdSvy5eH7eeJh1S302hj2KyFuVIWr
RLEyXJ/yu0q3qNMTY/AB
=Fej2
-----END PGP SIGNATURE-----