[GSoC] Qubes-MIME-Handlers Weekly Progress Report #3

[Andrew Morgan]

Hey everybody, another progress report for ya.

As always, you can find the report with screenshots here:
https://blog.amorgan.xyz/gsoc-weekly-progress-report-3.html

Otherwise the text-only version is reproduced below:

---

Good news everyone! The code base has been completely refactored from
multiple scripts with duplicated code into one lean python cli tool. All
the other scripts simply make use of this now.

Additionally, 90% of the bash in the project has now been replaced with
python, which should allow more portability and more maintainable code.

# New CLI Tool

This new utility is called qvm-trust, and can be used in the following
manner:

$ qvm-trust -h
usage: qvm-trust [-h] [-c] [-t] [-u] [-q] path [path ...]

Set or check file/folder trust levels.

positional arguments:
path a folder or file path

optional arguments:
-h, --help show this help message and exit
-c, --check Check whether a file or folder is trusted
-t, --trusted Set files or folders as trusted
-u, --untrusted Set files or folders as untrusted
-q, --quiet Do not print to stdout

Some example usage includes: setting a file as untrusted:

$ qvm-trust --untrusted ./leaked-documents.pdf

Checking the trusted state of a file:

$ qvm-trust --check ./leaked-documents.pdf
File is untrusted

Setting multiple files and folders as untrusted:

$ qvm-trust --untrusted ~/Downloads ./my-sketchy-file.docx ~/QubesIncoming

Checking trusted state through the return code (useful for scripting):

$ qvm-trust --check ~/Downloads --quiet
$ echo $?
2

A reference for the return codes is as follows. Additionally I plan to
write up a man page with all usage information soon:

Error codes:
Unable to read extended attributes: -2
Unable to read an input file: -1
No errors: 0
File/Folder is untrusted: 1
File/Folder is trusted: 2

# Nautilus and Dolphin integration

The Nautilus extension and Dolphin service files now make use of this
script. This cut down the file size of the Nautilus extension
considerably, as well as removed the need for the large bash script that
was dedicated specifically to checking trust in Nautilus, as well as the
two extensions for Nautilus. Now there is simply one extension that
handles both files and folders.

xattr is also no longer a dependency as we now do all the extended file
attribute checking and setting through python.

Additionally the ability to mark file-types as untrusted has been
removed, as there really wasn't a considerable use case for it and it
made implementation more complex than necessary. In the future it could
be added, but at the moment there's no real need for it.

However one benefit to its removal is no more complicated check-box UI
to navigate through! It's all simply there in the right-click menu:

[Screenshot] Nautilus context menu with right click for folders Nautilus
context menu with right click for files

There is now a single context menu item for both files and folders.

On the Dolphin side, the check-boxes have similarly been deprecated,
however I was unable to find a way to add the sleek icon next to the
service menu item as done in Nautilus. Unfortunately there doesn't seem
to be a way to change a service's icon based on a script's output, so
for now I've settled on a sub-menu which the user can choose whether to
mark a file/folder as trusted or untrusted:

[Screenshot] Dolphin with right-click sub-menu

Separate menus since Dolphin can't switch icons on the fly.

Also if you didn't already notice, 'Open in DisposableVM' now shows up
with a little Qubes logo in the right-click menu of untrusted files!

[Screenshot] Qubes logo now shows up for untrusted files

This was achieved with a .desktop file that is called upon when a
certain MIME-type is clicked. This was originally the plan for
recognizing untrusted files in the file-manager, however it was not
considered a viable option as an untrusted file could have any file-type
(i.e PDF, TXT, DOCX, etc). It wouldn't make sense to register all of
these types to be opened in DisposableVMs, so it was dropped.

That was until we started 'locking' untrusted files to disallow reading
by local applications. This had the unintended but incredibly useful
side-effect of having file managers set the MIME-type to
application/octet-stream. At that point we simply had to register that
MIME-type for our custom .desktop file and we were good!
Small bugs/issues

There are still some small roadblocks that still need to be cleared up:

While the functionality in Nautilus is all there, Nautilus doesn't
always seem to re-run the file-checking code in our extension. This can
cause the wrong icon to appear when right-clicking a file, which is
certainly confusing. The correct icon is shown if you click some empty
space, then right-click the file again. I feel this is more a Nautilus
bug than an issue with our file-checking code. I'll hit up the devs on
#nautilus soon and see if I can get some answers.

KDE's Dolphin handles MIME-type checking a bit differently than
Nautilus does. For instance, instead of solely checking for MIME-types
by reading the contents of the file, it gives the greatest priority to
the file extension.

[Screenshot] Dolphin thinks a PDF is a text file, due to its extension

This is a PDF that has been renamed with the '.txt' extension.
Dolphin thinks it is a text file.

[Screenshot] KWrite being suggested instead of qvm-trust

KWrite is suggested to edit it.

Dolphin doesn't mark untrusted files as a specific file-type,
instead it simply says 'unknown':

[Screenshot] Dolphin is confused

What could it be?!

Clicking on 'Create New File Type' does allow us to make new ones
called application/x-kdeuser, however even after doing so the file-type
is still unknown.

**Actually while writing this I just thought... we could combine 2
and 3 together and just append '.untrusted' to untrusted file types,
then register the '.untrusted' file extension with our desktop handler.
Hm... will try that right after I post this :)**

Dolphin does not allow unreadable files to be opened. It cancels the
application launch and presents an 'Access Denied' message. Not entirely
sure if there is a way around this outside of a patch. It would be great
to find a workaround.

[Screenshot] Dolphin won't let us open unreadable files

This file is unreadable, dummy!

**Update: this may not be an issue with the '.untrusted' workaround
above. Apparently if the file-type is recognizable it will send it to
the application that's registered**

The location of the untrusted-folders file. This file keeps track of
the folder paths that are untrusted, and is used by the daemon. Even
after reading the Qubes Contribution Guidelines I'm still not sure where
to put this. /usr/share/qubes seems to be the apt location but there's
not much in there already. Marek?

# Going Forward

We're currently heading into week 2 of 4 of what was supposed to be
patching the file managers, however now that it seems that's no longer
necessary, we're pretty much ahead of schedule! After Dolphin is fixed
up, I'll create a daemon to monitor untrusted folders and their
contents. If a new file is placed or created inside them, they'll
instantly be marked as untrusted.

After that is unit tests and documentation cleanup. Pretty standard stuff.

Anyways, that's about it for this week. Til next time!

[Joanna Rutkowska]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, Jun 21, 2017 at 11:15:30PM -0700, Andrew Morgan wrote:
> Hey everybody, another progress report for ya.
>
> As always, you can find the report with screenshots here:
> https://blog.amorgan.xyz/gsoc-weekly-progress-report-3.html
>
> Otherwise the text-only version is reproduced below:
>

Hello, a few quick questions and comments:

> ---
>
> Good news everyone! The code base has been completely refactored from
> multiple scripts with duplicated code into one lean python cli tool. All
> the other scripts simply make use of this now.
>
> Additionally, 90% of the bash in the project has now been replaced with
> python, which should allow more portability and more maintainable code.
>
> # New CLI Tool
>
> This new utility is called qvm-trust, and can be used in the following
> manner:
>

I think it'd be better to name the tool qvm-file-trust.

> $ qvm-trust -h
> usage: qvm-trust [-h] [-c] [-t] [-u] [-q] path [path ...]
>
> Set or check file/folder trust levels.
>
> positional arguments:
> path a folder or file path
>
> optional arguments:
> -h, --help show this help message and exit
> -c, --check Check whether a file or folder is trusted
> -t, --trusted Set files or folders as trusted
> -u, --untrusted Set files or folders as untrusted
> -q, --quiet Do not print to stdout
>
> Some example usage includes: setting a file as untrusted:
>
> $ qvm-trust --untrusted ./leaked-documents.pdf
>

So, where is this information kept for individual files? If somebody sent me a
file (e.g. via email attachment or I download it from the Web), can they also
make the file "trusted"? In your previous post you mentioned the use of extended
attributes -- how do you make sure that if I get e.g. a tgz-ed or cpio-ed
archive, then unpack, that there will be no attacker-controlled attributes
there, which would trick me into opening these (untrusted) files locally,
instead of via DispVM?

> **Actually while writing this I just thought... we could combine 2
> and 3 together and just append '.untrusted' to untrusted file types,
> then register the '.untrusted' file extension with our desktop handler.
> Hm... will try that right after I post this :)**
>

This gives control into the hands of attacker, e.g. to force opening the crafted
file in a DispVM. While this seems innocent (esp. compared to the inverse
scenario if we wanted to use '.trusted' exception), still this feels somehow
wrong.

> The location of the untrusted-folders file. This file keeps track of
> the folder paths that are untrusted, and is used by the daemon. Even
> after reading the Qubes Contribution Guidelines I'm still not sure where
> to put this. /usr/share/qubes seems to be the apt location but there's
> not much in there already. Marek?
>

Ideally we want to keep it somehow hidden from the user and protected against
accidental modifications. E.g. should be very hard for the user to (get
convinced to) copy/move some files there.

> # Going Forward
>
> We're currently heading into week 2 of 4 of what was supposed to be
> patching the file managers, however now that it seems that's no longer
> necessary, we're pretty much ahead of schedule! After Dolphin is fixed
> up, I'll create a daemon to monitor untrusted folders and their
> contents. If a new file is placed or created inside them, they'll
> instantly be marked as untrusted.
>

Hm, that's exposing additional attack surface on the VM, so don't like. Also,
again, how are files going to be marked?

joanna.
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCAAGBQJZS3lEAAoJEDOT2L8N3GcY9+QP/1sQxJ55WwVynZ1ad+JeUafY
+E7IB3GAZmx/N//RQ6ILURu0h5Yg6jM5vc0iS6NUCCByFJqs9R4wu5vwAlX2NxlI
m0z3QcQ+fBEiK1pEmf/5Vm6Lh9s38ow+qdeBuDd7xFgjD8iX2SCvkwkOjEUG72Re
kEYMlnUW5l5dBWIjLRC/7Bdn8Sisp7uT2Uh1H0U6FwmL+VwerXYgtp+Zg3lmJQlo
kU2B/PLauAtirIvRApiSOmJxzKBFRENXZwnysEK84k3W5znkek/OV08igjoilD7r
zklaFUdQffWcq10Bfz90OP4pIs7jZfyeNiLJi1muYQ9f5R0Z9gJlmy4XLgwDb0Mf
eosjKwR+A9xnI6PDxybwR00JLjYzPoCH/Ov4xDHtsCFElCNy6jV8KJ+8WSkdI3ug
jSTlRKF3UNF9vWYhFigQvxJ8XCNG/b3s6qnQ/dk7MrMo7Qjb95CJYXzjuFZKyVLO
cME4y9iWp2SNt0SFrVCfQ+/IMRjQohdz8J7XYItTt7K6BkBRv7a+qaWo8MvQReEz
Kg3thXvc5KixxCJ4EXkZL8xboId2S9Ir8wTlZ13fuaxqEdeKDuXVPaJWsJcp1yJx
CssR7EqXpYFa7PnSNA1LkxcolGsHb1S0dZX3MxSinUH5Xvnz4ehANDxRHJROgHRZ
fj7/08okahp28DYQ96xI
=MJiX
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, Jun 21, 2017 at 11:15:30PM -0700, Andrew Morgan wrote:

IMO --check should return 0 for trusted files. This way it would be
possible to use `if qvm-trust --check ...; then` (or negated with `!`).

Also consider making --check default action.

As for the code:
- use main() function called `if __name__ == '__main__'`, instead of
putting the code directly there - it will ease writing unit tests
later, and possibly reusing the code by simply importing this script
- use try ... finally syntax to restore file permissions, even if xattr
read fails; and more important (also achievable by try/finally): keep
restoring file permissions in the same function that change it in the
first place

(try/finally works even for exit() call, as in python it just raise
SystemExit exception)


An idea: mark folder untrusted also using xattr. Then it will be immune
to folder rename. And checking file inside would require "just" checking
directories up in the tree.
This does not replace the need for some daemon to mark individual files
inside as untrusted (when appears), but may (or may not) make things
easier.

> # Nautilus and Dolphin integration
>
> The Nautilus extension and Dolphin service files now make use of this
> script. This cut down the file size of the Nautilus extension
> considerably, as well as removed the need for the large bash script that
> was dedicated specifically to checking trust in Nautilus, as well as the
> two extensions for Nautilus. Now there is simply one extension that
> handles both files and folders.
>
> xattr is also no longer a dependency as we now do all the extended file
> attribute checking and setting through python.
>
> Additionally the ability to mark file-types as untrusted has been
> removed, as there really wasn't a considerable use case for it and it
> made implementation more complex than necessary. In the future it could
> be added, but at the moment there's no real need for it.
>
> However one benefit to its removal is no more complicated check-box UI
> to navigate through! It's all simply there in the right-click menu:
>
> [Screenshot] Nautilus context menu with right click for folders Nautilus
> context menu with right click for files

Nice!

Renaming the file might not be that good idea. What if there is already
a (different) file with .untrusted suffix in the same directory?

So, this may require some patching... Just make sure first there is no
another plugin API in dolphin for things like this.

> Dolphin does not allow unreadable files to be opened. It cancels the
> application launch and presents an 'Access Denied' message. Not entirely
> sure if there is a way around this outside of a patch. It would be great
> to find a workaround.
>
> [Screenshot] Dolphin won't let us open unreadable files
>
> This file is unreadable, dummy!
>
> **Update: this may not be an issue with the '.untrusted' workaround
> above. Apparently if the file-type is recognizable it will send it to
> the application that's registered**
>
> The location of the untrusted-folders file. This file keeps track of
> the folder paths that are untrusted, and is used by the daemon. Even
> after reading the Qubes Contribution Guidelines I'm still not sure where
> to put this. /usr/share/qubes seems to be the apt location but there's
> not much in there already. Marek?

/usr/share should not be modified at runtime. As we're talking about
configuration, it's better to put it in /etc/qubes/ (defaults for all
VMs based on given template) and ~/.config/qubes/ (VM local list) -
merge lists from those two places. And qvm-trust tool should modify the
later.
Merging two lists means you need some way to exclude directory (for
example generally ~/Downloads configured as untrusted, but you want to
exclude it in "untrusted" VM - to open downloaded files directly in the
VM there).

Clarification: currently there is no way to mark file as "trusted" if
the directory is "untrusted" (assuming file marking daemon is working),
right? I think this is a good thing.

> # Going Forward
>
> We're currently heading into week 2 of 4 of what was supposed to be
> patching the file managers, however now that it seems that's no longer
> necessary, we're pretty much ahead of schedule! After Dolphin is fixed
> up, I'll create a daemon to monitor untrusted folders and their
> contents. If a new file is placed or created inside them, they'll
> instantly be marked as untrusted.
>
> After that is unit tests and documentation cleanup. Pretty standard stuff.
>
> Anyways, that's about it for this week. Til next time!

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZS3u+AAoJENuP0xzK19cskOMH/A2tsQtHisafw6L/3dlc6vVH
CSXaVz09r4kGmSF0FKwwi4IouhdezqItkHj9FzUwsA9ZBov612KEjdYDhLM4NkHf
sYV5Jdhp0ZgNrWIv2SC+ypeJ01Qbl1tuoK1uDT0wBNQxc1B58BSHsuH/de61XMbN
88v1d4ZfHuc2GBf+eqRWxU299YIgCgctLDnL4VHCjvFQM5cy87KMm5gZN++gaz+V
RHWgYE1q8T7aphWHu4g49qEcsTTlI0Mqzcsu+Ko8SS2oZhoj5VL6r2NaMuXVRSQQ
g5P7PdMEgzzZ2bfp7+pobw5+1PLSwglYIst5zpGAUktchyIjQKEk8joBpNXQFAQ=
=h5Fo
-----END PGP SIGNATURE-----

[Andrew Morgan]

On 06/22/2017 01:01 AM, Joanna Rutkowska wrote:
>
> On Wed, Jun 21, 2017 at 11:15:30PM -0700, Andrew Morgan wrote:
>> Hey everybody, another progress report for ya.
>
>> As always, you can find the report with screenshots here:
>> https://blog.amorgan.xyz/gsoc-weekly-progress-report-3.html
>
>> Otherwise the text-only version is reproduced below:
>
>
> Hello, a few quick questions and comments:
>
>> ---
>
>> Good news everyone! The code base has been completely refactored from
>> multiple scripts with duplicated code into one lean python cli tool. All
>> the other scripts simply make use of this now.
>
>> Additionally, 90% of the bash in the project has now been replaced with
>> python, which should allow more portability and more maintainable code.
>
>> # New CLI Tool
>
>> This new utility is called qvm-trust, and can be used in the following
>> manner:
>
>
> I think it'd be better to name the tool qvm-file-trust.

Can do, though the tool can also handle folders. I presume that still works?

>
>> $ qvm-trust -h
>> usage: qvm-trust [-h] [-c] [-t] [-u] [-q] path [path ...]
>
>> Set or check file/folder trust levels.
>
>> positional arguments:
>> path a folder or file path
>
>> optional arguments:
>> -h, --help show this help message and exit
>> -c, --check Check whether a file or folder is trusted
>> -t, --trusted Set files or folders as trusted
>> -u, --untrusted Set files or folders as untrusted
>> -q, --quiet Do not print to stdout
>
>> Some example usage includes: setting a file as untrusted:
>
>> $ qvm-trust --untrusted ./leaked-documents.pdf
>
>
> So, where is this information kept for individual files? If somebody sent me a
> file (e.g. via email attachment or I download it from the Web), can they also
> make the file "trusted"? In your previous post you mentioned the use of extended
> attributes -- how do you make sure that if I get e.g. a tgz-ed or cpio-ed
> archive, then unpack, that there will be no attacker-controlled attributes
> there, which would trick me into opening these (untrusted) files locally,
> instead of via DispVM?

For individual files an xattr [1][2] with key 'user.qubes.untrusted' and
value 'true' is added to the file.

For both tar and cpio, once a file is added and extracted from an
archive it no longer carries the xattr and thus is set back to the
default of trusted. This is also true when uploading/downloading from
the web as attributes are part of the local filesystem and not the file
itself.

>
>> **Actually while writing this I just thought... we could combine 2
>> and 3 together and just append '.untrusted' to untrusted file types,
>> then register the '.untrusted' file extension with our desktop handler.
>> Hm... will try that right after I post this :)**
>
>
> This gives control into the hands of attacker, e.g. to force opening the crafted
> file in a DispVM. While this seems innocent (esp. compared to the inverse
> scenario if we wanted to use '.trusted' exception), still this feels somehow
> wrong.

Yes... I agree and likely got a bit too ahead of myself with that solution.

I'd also like to clarify on the above, Dolphin seems to only determine
the file-type through reading the file if it is not given a known
extension such as .txt or .pdf. If it is given a known extension, then
it's recognized as such, no matter what.

Interestingly, even after setting up a handler for an '.untrusted'
extension, it doesn't treat this as a recognized extension and instead
reverts back to checking the file contents for their MIME-type.

>
>> The location of the untrusted-folders file. This file keeps track of
>> the folder paths that are untrusted, and is used by the daemon. Even
>> after reading the Qubes Contribution Guidelines I'm still not sure where
>> to put this. /usr/share/qubes seems to be the apt location but there's
>> not much in there already. Marek?
>
>
> Ideally we want to keep it somehow hidden from the user and protected against
> accidental modifications.

I've also just realized it will need to be stored in $HOME or /rw/ if
it's to remain persistent across reboots.

> E.g. should be very hard for the user to (get convinced to) copy/move some files there.

Right, but still needs to be editable by the local user. Not sure if
/rw/ is sufficient. We can also make it a hidden file.

Or even better, `chmod 0` it as well.

>
>> # Going Forward
>
>> We're currently heading into week 2 of 4 of what was supposed to be
>> patching the file managers, however now that it seems that's no longer
>> necessary, we're pretty much ahead of schedule! After Dolphin is fixed
>> up, I'll create a daemon to monitor untrusted folders and their
>> contents. If a new file is placed or created inside them, they'll
>> instantly be marked as untrusted.
>
>
> Hm, that's exposing additional attack surface on the VM, so don't like. Also,
> again, how are files going to be marked?

We could set all files within a folder to untrusted when a folder is
marked as such, but I'm not sure of any other way to keep it
consistently updated without a daemon, unless there is an existing Qubes
component that could be modified to take care of this?

All the program would need to do is:

Place inotify listeners on each folder listed in the tracking file.
On an inotify NEW event, mark the new file as untrusted.

The only real attack I could see is again changing the contents of the
tracking file, but then again a malicious actor could edit many files in
$HOME and have a negative effect on the VM.

Andrew Morgan

[1]: https://en.wikipedia.org/wiki/Extended_file_attributes
[2]: http://www.freedesktop.org/wiki/CommonExtendedAttributes

[Andrew Morgan]

That is a good point, and would eliminate the need to hardcode a return
code.

>
> Also consider making --check default action.

Was thinking that myself :)

>
> As for the code:
> - use main() function called `if __name__ == '__main__'`, instead of
> putting the code directly there - it will ease writing unit tests
> later, and possibly reusing the code by simply importing this script
> - use try ... finally syntax to restore file permissions, even if xattr
> read fails; and more important (also achievable by try/finally): keep
> restoring file permissions in the same function that change it in the
> first place
>
> (try/finally works even for exit() call, as in python it just raise
> SystemExit exception)
>

All good points, will fix those in the next update.

>
> An idea: mark folder untrusted also using xattr. Then it will be immune
> to folder rename. And checking file inside would require "just" checking
> directories up in the tree.

This is true, however would require a Nautilus patch as the way we
currently tell Nautilus that a file is untrusted is mostly due to the
fact that it is unreadable (and thus has a special MIME-type).

In theory this would be quite easy to add into the check ability of
qvm-trust and thus if Nautilus and Dolphin could both reference it
before opening files then everything should work out.

Just means they need a patch to intercept the open call :)

> This does not replace the need for some daemon to mark individual files
> inside as untrusted (when appears), but may (or may not) make things
> easier.
>

True, you would still want a downloaded PDF to remain untrusted even
after moving it from the ~/Downloads folder.

Yes, and Joanna mentioned that it would be quite trivial for an attacker
to add this extension to a file, so I don't think we will pursue it.

>
>> Dolphin does not allow unreadable files to be opened. It cancels the
>> application launch and presents an 'Access Denied' message. Not entirely
>> sure if there is a way around this outside of a patch. It would be great
>> to find a workaround.
>
>> [Screenshot] Dolphin won't let us open unreadable files
>
>> This file is unreadable, dummy!
>
>> **Update: this may not be an issue with the '.untrusted' workaround
>> above. Apparently if the file-type is recognizable it will send it to
>> the application that's registered**
>
>> The location of the untrusted-folders file. This file keeps track of
>> the folder paths that are untrusted, and is used by the daemon. Even
>> after reading the Qubes Contribution Guidelines I'm still not sure where
>> to put this. /usr/share/qubes seems to be the apt location but there's
>> not much in there already. Marek?
>
> /usr/share should not be modified at runtime. As we're talking about
> configuration, it's better to put it in /etc/qubes/ (defaults for all
> VMs based on given template) and ~/.config/qubes/ (VM local list) -
> merge lists from those two places. And qvm-trust tool should modify the
> later.

Good idea, makes management quite simple.

> Merging two lists means you need some way to exclude directory (for
> example generally ~/Downloads configured as untrusted, but you want to
> exclude it in "untrusted" VM - to open downloaded files directly in the
> VM there).

Yeah, you'd need to clarify "trusted" as something other than the
default, which is fine. There could be 3 states: trusted, untrusted and
undefined. However this may make things more confusing for the user.

>
> Clarification: currently there is no way to mark file as "trusted" if
> the directory is "untrusted" (assuming file marking daemon is working),
> right? I think this is a good thing.
>

A file is "trusted" if it doesn't have the 'user.qubes.untrusted' xattr,
however if we do implement checking based on 'walking up' the file path
then this would be true.

In the current design, you can place a file into an untrusted folder,
it'll be marked untrusted by the daemon, then you can mark the file as
trusted again and it will remain that way inside the untrusted folder.

That may be useful to users who want to keep certain files in their
~/Downloads folder without having to open them in a DispVM every time.

[Joanna Rutkowska]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, Jun 22, 2017 at 01:32:32AM -0700, Andrew Morgan wrote:
> On 06/22/2017 01:01 AM, Joanna Rutkowska wrote:
> >
> > On Wed, Jun 21, 2017 at 11:15:30PM -0700, Andrew Morgan wrote:
> >> Hey everybody, another progress report for ya.
> >
> >> As always, you can find the report with screenshots here:
> >> https://blog.amorgan.xyz/gsoc-weekly-progress-report-3.html
> >
> >> Otherwise the text-only version is reproduced below:
> >
> >
> > Hello, a few quick questions and comments:
> >
> >> ---
> >
> >> Good news everyone! The code base has been completely refactored from
> >> multiple scripts with duplicated code into one lean python cli tool. All
> >> the other scripts simply make use of this now.
> >
> >> Additionally, 90% of the bash in the project has now been replaced with
> >> python, which should allow more portability and more maintainable code.
> >
> >> # New CLI Tool
> >
> >> This new utility is called qvm-trust, and can be used in the following
> >> manner:
> >
> >
> > I think it'd be better to name the tool qvm-file-trust.
>
> Can do, though the tool can also handle folders. I presume that still works?
>

But, at the end of the day, it's all about determining trustworthiness of files,
right? So, I think using 'file' is justified.

Ideally, I think the evaluation of trustworthiness should be like this:

1. All files are considered _untrusted_ unless we check all the rules below, and
none of them identifies the file as _untrusted_,

2. If the file is under a path which contains pre-defined untrusted string (such
as... "untrusted", ALWAYS case _insensitive_ match), the file is untrusted, no
matter what xattrs might be saying. E.g. the file might be from some FAT
filesystem attached to the VM via qvm-block or qvm-usb and we still want to
catch it.

3. If the file has xattr saying it's untrusted.

(4. If none of the above, then consider it trusted.)

I somehow don't like the solution which offloads everything to xattrs, because:
1. Some filesytems might not support it (e.g. FAT-based stick?), or have bugs,
2. Whatever daemon we might rely on marking the files as untrusted with per-file
attributes (e.g. whenever they land in ~/Downloads) might accidentally (or less
accidentally) crash (or be made to crash).

joanna.
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCAAGBQJZS4cpAAoJEDOT2L8N3GcY/fwQAJKrYzy/n4HeGsqwqT102e2R
Wt9qbUWJbzDH+60O1beclAt7EtfnoWfLlDSVtJxDSMki6zNUK2H3p+OJRfK9MANG
9Lq8v/rRu4FQA9n/wI9gCOj2JgnlRFKwUMlTTjlAZ7CWcPVN9TmlFybg0yhFdw0/
9YeIgFdwoSvfpmnN+4IDK47BWy7zctpF2GF+cPaMG8xeh53nX2wHASUal/qCF6Sz
nxcppGiijlfnfuXdwC1nDNVqpS4UN+RJG8E4uRFC5lkEgejx45p3wwMPduVh9ogD
0ZbOr7vzNda0g6DoFw+RQLyk71tbRPAuvCDq/+1AZXqjJX8V2ccqW6Gwj2r4fGxm
ro/nzZ7zHiPIxxs564Sgt7SIinUdiuBUjVdr143pXP+1yPLY+off+PxAcX1toLMz
DmuxQljK0HHPHuoOi2MnATnxBNqpYR/GiDB5ac+vvPIR5C42IonZiMPIsZbDioZr
5efavJ7RSX3R4dJfkKyiP/ZguBZ8hpqLQoDXM8+907Xv3iT8uK6mWabqpODbiqUb
UcNgIjJaQ++x/ShaQAAv2jVs1m4a/WzkLztiIcdRldkWTqxGBJforHdY8sQYScMi
AnauIvYgN2cE416MFnNUtMPrGxOKFN0j0I2D4KAVh8goM8/Qz1ahgnadsP8nEDHu
iuz8iUM3a2XUtAyR+OyU
=Vuq1
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, Jun 22, 2017 at 01:50:56AM -0700, Andrew Morgan wrote:
> On 06/22/2017 01:11 AM, Marek Marczykowski-Górecki wrote:
> > An idea: mark folder untrusted also using xattr. Then it will be immune
> > to folder rename. And checking file inside would require "just" checking
> > directories up in the tree.
>
> This is true, however would require a Nautilus patch as the way we
> currently tell Nautilus that a file is untrusted is mostly due to the
> fact that it is unreadable (and thus has a special MIME-type).

No no, I was talking about marking directory _in addition to_ marking
files - i.e. what directories should be handled by inotify-handling
daemon.

But as I'm thinking more about it, it would be less convenient, as you
can't easily get list of directories with given xattr, without looking
up all all the user home. And the same about noticing new directories.

So, a file with path list looks like a better idea.

> >> The location of the untrusted-folders file. This file keeps track of
> >> the folder paths that are untrusted, and is used by the daemon. Even
> >> after reading the Qubes Contribution Guidelines I'm still not sure where
> >> to put this. /usr/share/qubes seems to be the apt location but there's
> >> not much in there already. Marek?
> >
> > /usr/share should not be modified at runtime. As we're talking about
> > configuration, it's better to put it in /etc/qubes/ (defaults for all
> > VMs based on given template) and ~/.config/qubes/ (VM local list) -
> > merge lists from those two places. And qvm-trust tool should modify the
> > later.
>
> Good idea, makes management quite simple.
>
> > Merging two lists means you need some way to exclude directory (for
> > example generally ~/Downloads configured as untrusted, but you want to
> > exclude it in "untrusted" VM - to open downloaded files directly in the
> > VM there).
>
> Yeah, you'd need to clarify "trusted" as something other than the
> default, which is fine. There could be 3 states: trusted, untrusted and
> undefined. However this may make things more confusing for the user.

I was thinking about excluding whole directories, not subset of them.
Let me explain on example:

/etc/qubes/always-open-in-dispvm.list:

/home/user/Downloads
/home/user/QubesIncoming

untrusted:/home/user/.config/qubes/always-open-in-dispvm.list:

-/home/user/Downloads

So, files in ~/Downloads would be opened in DispVM everywhere besides
"untrusted" VM. But this excluding works only for entries earlier
included in the list, specifically "-/home/user/xyz" or
"-/home/user/Downloads/some-subdir" does nothing because none of them
were included before.
Yes - we don't want "trusted" subdirectory inside "untrusted" one.

> > Clarification: currently there is no way to mark file as "trusted" if
> > the directory is "untrusted" (assuming file marking daemon is working),
> > right? I think this is a good thing.
> >
>
> A file is "trusted" if it doesn't have the 'user.qubes.untrusted' xattr,
> however if we do implement checking based on 'walking up' the file path
> then this would be true.

Lets move this to the other thread (Joanna's response).

> In the current design, you can place a file into an untrusted folder,
> it'll be marked untrusted by the daemon, then you can mark the file as
> trusted again and it will remain that way inside the untrusted folder.
>
> That may be useful to users who want to keep certain files in their
> ~/Downloads folder without having to open them in a DispVM every time.

IMO the user should move the file out of Downloads first to have this
effect.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZS4j1AAoJENuP0xzK19cs0igH/jkOHbO0V++L80QRRCF1pyAw
M6Viwx1S+83O4/Q8EkqdduzdzNYI2PBXCBdgEQMWAU/qOxGswI9YSVVOERvSSmK7
HWblktv4GBJ4gtZkt/Z/il/j//vdhs1Vbfz1aW2fnllk22CTa8BiOFd6wiWUulaa
vPm+YLLilV0CXNOoJ9+iVW9ACpi+08huVvUrNTkIQoLmiZsU0Uh6di8z8JdHpiaX
0IKDIVTEdGIh8MTZYg9Y5H3+BoHvLcQxXr7B15rWXne9K/docVElW3V7Db+H1K5s
Gc+a1ZPvTjkxMW/mzjhCzvbeJXeV2t7hGRWt26eRi8WnVSQxYiS70lViKoEgVT8=
=6pDU
-----END PGP SIGNATURE-----

[Andrew Morgan]

On 06/22/2017 02:00 AM, Joanna Rutkowska wrote:
> On Thu, Jun 22, 2017 at 01:32:32AM -0700, Andrew Morgan wrote:
>> On 06/22/2017 01:01 AM, Joanna Rutkowska wrote:
>>>
>>> On Wed, Jun 21, 2017 at 11:15:30PM -0700, Andrew Morgan wrote:
>>>> Hey everybody, another progress report for ya.
>>>
>>>> As always, you can find the report with screenshots here:
>>>> https://blog.amorgan.xyz/gsoc-weekly-progress-report-3.html
>>>
>>>> Otherwise the text-only version is reproduced below:
>>>
>>>
>>> Hello, a few quick questions and comments:
>>>
>>>> ---
>>>
>>>> Good news everyone! The code base has been completely refactored from
>>>> multiple scripts with duplicated code into one lean python cli tool. All
>>>> the other scripts simply make use of this now.
>>>
>>>> Additionally, 90% of the bash in the project has now been replaced with
>>>> python, which should allow more portability and more maintainable code.
>>>
>>>> # New CLI Tool
>>>
>>>> This new utility is called qvm-trust, and can be used in the following
>>>> manner:
>>>
>>>
>>> I think it'd be better to name the tool qvm-file-trust.
>
>> Can do, though the tool can also handle folders. I presume that still works?
>
>
> But, at the end of the day, it's all about determining trustworthiness of files,
> right? So, I think using 'file' is justified.

No problem, changed :)

I think you mean _trusted_ on the first line, right?

>
> 2. If the file is under a path which contains pre-defined untrusted string (such
> as... "untrusted", ALWAYS case _insensitive_ match), the file is untrusted, no
> matter what xattrs might be saying. E.g. the file might be from some FAT
> filesystem attached to the VM via qvm-block or qvm-usb and we still want to
> catch it.
>
> 3. If the file has xattr saying it's untrusted.
>
> (4. If none of the above, then consider it trusted>

> I somehow don't like the solution which offloads everything to xattrs, because:
> 1. Some filesytems might not support it (e.g. FAT-based stick?), or have bugs,
> 2. Whatever daemon we might rely on marking the files as untrusted with per-file
> attributes (e.g. whenever they land in ~/Downloads) might accidentally (or less
> accidentally) crash (or be made to crash).
>
> joanna.
>

I agree, I like the idea of a fall-back for both systems in case one fails.

[Andrew Morgan]

On 06/22/2017 02:08 AM, Marek Marczykowski-Górecki wrote:
> On Thu, Jun 22, 2017 at 01:50:56AM -0700, Andrew Morgan wrote:
>> On 06/22/2017 01:11 AM, Marek Marczykowski-Górecki wrote:
>>> An idea: mark folder untrusted also using xattr. Then it will be immune
>>> to folder rename. And checking file inside would require "just" checking
>>> directories up in the tree.
>
>> This is true, however would require a Nautilus patch as the way we
>> currently tell Nautilus that a file is untrusted is mostly due to the
>> fact that it is unreadable (and thus has a special MIME-type).
>
> No no, I was talking about marking directory _in addition to_ marking
> files - i.e. what directories should be handled by inotify-handling
> daemon.
>
> But as I'm thinking more about it, it would be less convenient, as you
> can't easily get list of directories with given xattr, without looking
> up all all the user home. And the same about noticing new directories.
>
> So, a file with path list looks like a better idea.

Yeah that was the original idea for it, you definitely want something
efficient if it'll be running 24/7.

I had the same thought, this seems quite elegant.

> So, files in ~/Downloads would be opened in DispVM everywhere besides
> "untrusted" VM. But this excluding works only for entries earlier
> included in the list, specifically "-/home/user/xyz" or
> "-/home/user/Downloads/some-subdir" does nothing because none of them
> were included before.
> Yes - we don't want "trusted" subdirectory inside "untrusted" one.

Right, a 'walking up' scheme would negate this anyways.

>
>>> Clarification: currently there is no way to mark file as "trusted" if
>>> the directory is "untrusted" (assuming file marking daemon is working),
>>> right? I think this is a good thing.
>>>
>
>> A file is "trusted" if it doesn't have the 'user.qubes.untrusted' xattr,
>> however if we do implement checking based on 'walking up' the file path
>> then this would be true.
>
> Lets move this to the other thread (Joanna's response).
>
>> In the current design, you can place a file into an untrusted folder,
>> it'll be marked untrusted by the daemon, then you can mark the file as
>> trusted again and it will remain that way inside the untrusted folder.
>
>> That may be useful to users who want to keep certain files in their
>> ~/Downloads folder without having to open them in a DispVM every time.
>
> IMO the user should move the file out of Downloads first to have this
> effect.
>

Fair enough, I suppose we could simply prevent a user from marking a
file as trusted if it's in a untrusted directory? What if the user
manually removed the xattrs? Our daemon may have to do more work to make
sure all files in untrusted folders are always untrusted...

>

Andrew Morgan

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

IMO daemon should restore untrusted xattr in that case (it's just one
another INOTIFY flag). But see the other thread.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZTBjFAAoJENuP0xzK19csLHEH/iCmOcZwROXSAN/OD34xem++
1fHwWAb9jgk69mtCzx6Un/nPzB22tI2BgRlZGVVwKPq7EkW+zkuFAag985I4XAAT
V2ytUAnhF0da+Nihc5GmaBYz9MHuQEfbSL4vt2LENw5VlIGfuna6YItnZEopgsvk
ffE65OOW/6xAgRaS+/4THIE8IrByfBdybkPnhm0f+ITVcJ0RnddF1Z2YkJLH6H+0
QZtq3e1jQhd46IsyATXRfZUASMI3twb4gY506jTsD3EKgY/x9jSfCBc0E6wh08jo
atKw1pJaV8/gl+VpSxg+mvukUUu8taEBFTgcHGkc5IPrZC6vTC5JA64BM+1R7FA=
=cKM1
-----END PGP SIGNATURE-----

[Andrew Morgan]

On 06/22/2017 12:21 PM, Marek Marczykowski-Górecki wrote:
> On Thu, Jun 22, 2017 at 02:49:28AM -0700, Andrew Morgan wrote:
>> On 06/22/2017 02:08 AM, Marek Marczykowski-Górecki wrote:
>>> On Thu, Jun 22, 2017 at 01:50:56AM -0700, Andrew Morgan wrote:
>>>> That may be useful to users who want to keep certain files in their
>>>> ~/Downloads folder without having to open them in a DispVM every time.
>>>
>>> IMO the user should move the file out of Downloads first to have this
>>> effect.
>>>
>
>> Fair enough, I suppose we could simply prevent a user from marking a
>> file as trusted if it's in a untrusted directory? What if the user
>> manually removed the xattrs? Our daemon may have to do more work to make
>> sure all files in untrusted folders are always untrusted...
>
> IMO daemon should restore untrusted xattr in that case (it's just one
> another INOTIFY flag). But see the other thread.
>
>

Hey Marek,

I'm starting to write some unit tests for qvm-file-trust. I found a page
in the documentation on unit tests
(https://www.qubes-os.org/doc/automated-tests/), and from what I can see
all the tests are in qubes-core-admin.

The tests in the repo seem pretty high level, more at the VM level. For
testing the cli tool, should I create a new file or integrate into one
of the existing ones?

Also I noticed that most of the files seem to be gone in the master
branch as opposed to release3.2. Is it a good idea to base on release3.2
or will those files be permanently gone in R4?

Thanks, and apologies for the slight delay in the latest weekly update,
had some major sunburn from the beach that slowed down my productivity a
bit :) Should be out tomorrow.

Andrew Morgan

[Andrew Morgan]

On 06/22/2017 12:21 PM, Marek Marczykowski-Górecki wrote:
> On Thu, Jun 22, 2017 at 02:49:28AM -0700, Andrew Morgan wrote:
>> On 06/22/2017 02:08 AM, Marek Marczykowski-Górecki wrote:
>>> On Thu, Jun 22, 2017 at 01:50:56AM -0700, Andrew Morgan wrote:
>>>> That may be useful to users who want to keep certain files in their
>>>> ~/Downloads folder without having to open them in a DispVM every time.
>>>
>>> IMO the user should move the file out of Downloads first to have this
>>> effect.
>>>
>
>> Fair enough, I suppose we could simply prevent a user from marking a
>> file as trusted if it's in a untrusted directory? What if the user
>> manually removed the xattrs? Our daemon may have to do more work to make
>> sure all files in untrusted folders are always untrusted...
>
> IMO daemon should restore untrusted xattr in that case (it's just one
> another INOTIFY flag). But see the other thread.
>
>

Additionally, qvm-file-trust is written in python3, while the unittest
files are written in python2. Do you forsee this as a problem? Should I
convert to py2?

Thanks,
Andrew Morgan

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Wed, Jun 28, 2017 at 08:00:35PM -0700, Andrew Morgan wrote:
> I'm starting to write some unit tests for qvm-file-trust. I found a page
> in the documentation on unit tests
> (https://www.qubes-os.org/doc/automated-tests/), and from what I can see
> all the tests are in qubes-core-admin.

Not all, but most ;)
There is also "extra tests loader":
https://github.com/QubesOS/qubes-core-admin/blob/master/qubes/tests/extra.py
It's purpose is to load tests from other repositories and run them as
part of one test run in dom0.

See split-gpg for example:
https://github.com/QubesOS/qubes-app-linux-split-gpg/tree/master/tests

This is mostly about integration tests, which require running Qubes
system. Above API allow running tests either directly from specific
component, or as part of full test run.

Specifically:
- test case class for integration tests should inherit from
qubes.tests.extra.ExtraTestCase, unit tests can inherit from standard
unittest.TestCase; put them in separate files, so it's possible to
load unittests outside of dom0
- use self.create_vms to create required VMs, do not modify existing
VMs
- you can use standard methods on returned objects (.start(), .run()
etc)
- define appropriate entry_point in setup.py to register tests (see
example above)

For unit tests, you can use standard unittest.TestCase and ignore most
of above things. See for example qubes-firewall unit tests:
https://github.com/QubesOS/qubes-core-agent-linux/blob/master/qubesagent/test_firewall.py
https://github.com/QubesOS/qubes-core-agent-linux/blob/master/run-tests

And then it's integrated into travis:
https://github.com/QubesOS/qubes-core-agent-linux/blob/master/.travis.yml

In short, for unit tests, create separate file, inherit from
unittest.TestCase and write them the way to not require running in Qubes
VM or dom0. unittest.mock module can be useful for this.
For integration tests (if any), put them in separate file and inherit
from qubes.tests.extra.ExtraTestCase. Those tests can check if things
really open DispVM etc. That class should orchestrate
thins from dom0 - but can simply run a test script in the VM - again,
see split-gpg (thunderbird tests).

As for unittest.mock, a simple example:

class TC_00_Example(unittest.TestCase):
@unittest.mock.patch('subprocess.check_call')
def test_xyz(self, mock_subprocess):
# do some things here
mock_subprocess.assert_called_one_with(['qvm-open-in-dvm',
'some-path')

More in module documentation. And in test_firewall.py linked above.

> The tests in the repo seem pretty high level, more at the VM level.

In Qubes 3.x most of our tests are integration tests, then in Qubes 4.0
we have a lot of unit tests in core-admin (and some in other
repositories).

> For
> testing the cli tool, should I create a new file or integrate into one
> of the existing ones?

New file, preferably in the same repository as the tool.

> Also I noticed that most of the files seem to be gone in the master
> branch as opposed to release3.2. Is it a good idea to base on release3.2
> or will those files be permanently gone in R4?

Those files are moved to qubes/tests. But see above.

> Additionally, qvm-file-trust is written in python3, while the unittest
> files are written in python2. Do you forsee this as a problem? Should I
> convert to py2?

Keep it python3. First - in Qubes 4.0 we also migrate to python3, second
- - unit tests do not need to use test loader in core-admin.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZVK9kAAoJENuP0xzK19csk2wIAJcW4hbCKTfKY8MEK7y9DLgn
KMDq2cGvz2kdpbcApDOpjn0kYw5JkLXgvyBsZpEjUjzUWq9Bn88kDTowfld4HDTE
NWbJhrjleI1csTLUZnXOa//xQkf0AjUSrqkEHjYe6LILJMtiEC0ouHHqMm4Sian5
84Uw6bDqZi4SUmHQJkuo+OInypB5ypmTH5hVY4Ehixz21D5aMUhi7BJ5UWweJ6zS
qGz8SM+5DBNIDo1eugpu3YnrNhutICYdAH0wHLSQfhBNErYSYsQMPmwqRnePEAic
L1HWUEA9UdCVoB49LZFIVy0pgGOMbRsNI0V0AIYhLzknJwdGqa3038sCF871XbU=
=kWPa
-----END PGP SIGNATURE-----

[Noor Christensen]

On Wed, Jun 28, 2017 at 08:00:35PM -0700, Andrew Morgan wrote:

> Thanks, and apologies for the slight delay in the latest weekly update,
> had some major sunburn from the beach that slowed down my productivity a
> bit :) Should be out tomorrow.

Hey, it's _Summer_ of Code right?

Sorry... Thanks for your hard work :-)

-- noor

|_|O|_|
|_|_|O| Noor Christensen
|O|O|O| no...@fripost.org ~ 0x401DA1E0

[Jean-Philippe Ouellet]

On Thu, Jun 22, 2017 at 4:11 AM, Marek Marczykowski-Górecki
<marm...@invisiblethingslab.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On Wed, Jun 21, 2017 at 11:15:30PM -0700, Andrew Morgan wrote:
>> $ qvm-trust --check ~/Downloads --quiet
>> $ echo $?
>> 2
>>
>> A reference for the return codes is as follows. Additionally I plan to
>> write up a man page with all usage information soon:
>>
>> Error codes:
>> Unable to read extended attributes: -2
>> Unable to read an input file: -1
>> No errors: 0
>> File/Folder is untrusted: 1
>> File/Folder is trusted: 2

Note that exit statuses are usually treated as 8-bit unsigned values,
so -1 and -2 would actually be 255 and 254 respectively.

Standard semantic values for exit codes have existed for several
decades, and can be found in sysexits.h (also documented in man pages
such as [1]).

Canonical usage might look something like the following:
- EXIT_SUCCESS (0) for no errors and file being trusted
- EXIT_FAILURE (1) for no errors but file being untrusted
- EX_USAGE (64) for invalid command usage
- EX_DATAERR (65) for xattr errors (such as value other than "true"
for user.qubes.untrusted)
- EX_NOINPUT (66) for specified file to check not existing
- EX_OSFILE (72) for problems with the "Qubes untrusted folders" file(s)
- EX_NOPERM (77) for e.g. chmod failing due to file ownership during
--{un,}trust operation
etc.

The fact that only no errors & file being trusted results in a zero
exit status (truthy value for shells) means that any other errors
result in the trust system failing-closed (treating as untrusted),
which IMO is what we want.

EXIT_* can be found defined in POSIX [2], and EX_* are from sysexits.

[1]: https://man.openbsd.org/sysexits
[2]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/exit.html

> IMO --check should return 0 for trusted files. This way it would be
> possible to use `if qvm-trust --check ...; then` (or negated with `!`).

+1

> Also consider making --check default action.

+1

Also, what would you think of writing nothing to stdout by default
(essentially --quiet by default) and instead taking --verbose/-v to
print "File is {un,}trusted"?

http://www.catb.org/esr/writings/taoup/html/ch01s06.html#id2878450
http://www.catb.org/esr/writings/taoup/html/ch11s09.html

[Andrew Morgan]

This is true, I should have read the fine print :-P

Andrew Morgan

[Andrew Morgan]

On 06/29/2017 09:21 AM, Jean-Philippe Ouellet wrote:
> On Thu, Jun 22, 2017 at 4:11 AM, Marek Marczykowski-Górecki
> <marm...@invisiblethingslab.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA256
>>
>> On Wed, Jun 21, 2017 at 11:15:30PM -0700, Andrew Morgan wrote:
>>> $ qvm-trust --check ~/Downloads --quiet
>>> $ echo $?
>>> 2
>>>
>>> A reference for the return codes is as follows. Additionally I plan to
>>> write up a man page with all usage information soon:
>>>
>>> Error codes:
>>> Unable to read extended attributes: -2
>>> Unable to read an input file: -1
>>> No errors: 0
>>> File/Folder is untrusted: 1
>>> File/Folder is trusted: 2
>
> Note that exit statuses are usually treated as 8-bit unsigned values,
> so -1 and -2 would actually be 255 and 254 respectively.
>

Yes I wondered about this and whether it may have compatibility issues.
At the moment they're just arbitrary numbers and changing them is no hassle.

> Standard semantic values for exit codes have existed for several
> decades, and can be found in sysexits.h (also documented in man pages
> such as [1]).
>
> Canonical usage might look something like the following:
> - EXIT_SUCCESS (0) for no errors and file being trusted
> - EXIT_FAILURE (1) for no errors but file being untrusted
> - EX_USAGE (64) for invalid command usage
> - EX_DATAERR (65) for xattr errors (such as value other than "true"
> for user.qubes.untrusted)
> - EX_NOINPUT (66) for specified file to check not existing
> - EX_OSFILE (72) for problems with the "Qubes untrusted folders" file(s)
> - EX_NOPERM (77) for e.g. chmod failing due to file ownership during
> --{un,}trust operation
> etc.

Yes after a quick search it seems that POSIX's error codes are
recommended for python as well, so no reason not to go with the
standards. Will patch these in...

>
> The fact that only no errors & file being trusted results in a zero
> exit status (truthy value for shells) means that any other errors
> result in the trust system failing-closed (treating as untrusted),
> which IMO is what we want.>
> EXIT_* can be found defined in POSIX [2], and EX_* are from sysexits.
>
> [1]: https://man.openbsd.org/sysexits
> [2]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/exit.html
>
>> IMO --check should return 0 for trusted files. This way it would be
>> possible to use `if qvm-trust --check ...; then` (or negated with `!`).
>
> +1

Default in the latest update.

>
>> Also consider making --check default action.
>
> +1

ditto

>
> Also, what would you think of writing nothing to stdout by default
> (essentially --quiet by default) and instead taking --verbose/-v to
> print "File is {un,}trusted"?
>
> http://www.catb.org/esr/writings/taoup/html/ch01s06.html#id2878450
> http://www.catb.org/esr/writings/taoup/html/ch11s09.html
>

The -q flag was modeled after grep which uses the -q flag to silence all
output, which is useful for scripts, while users in the cli would
primarily want output.

While qvm-file-trust is used by the file managers, there's no reason it
can't be used on the command line like any other qvm-* program. As far
as I know quiet output isn't the default for any other Qubes program, or
maybe I'm missing something?

Very grateful for the feedback :-)

Andrew Morgan

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, Jun 30, 2017 at 01:27:36AM -0700, Andrew Morgan wrote:
> The -q flag was modeled after grep which uses the -q flag to silence all
> output, which is useful for scripts, while users in the cli would
> primarily want output.
>
> While qvm-file-trust is used by the file managers, there's no reason it
> can't be used on the command line like any other qvm-* program. As far
> as I know quiet output isn't the default for any other Qubes program, or
> maybe I'm missing something?

IMO it's fine to leave non-quiet option by default, and have -q for
scripts. After all, command arguments should be optimized for
interactive usage (if that is one intended use case), because this is
where command length matter.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJZVhlSAAoJENuP0xzK19csI4kH/A0UPL5MRZ4OUfDvrMtKrf8S
mX09OumDHIxqPwIkYoQghI7GThLxPMMJDEThdlt8EpaQVWvt74SBzssTfn//d6P3
sz5N1GY1t2PS7ArNsouwv8HkVzK7pduQoGQOqbj+MN/oCB81VaW6PiXs2EXkbD8Y
Ij3e/SRXlsfa5dclMpCd1ziJa75tZLYYVbdmrRNWJExWYO5+ebr8XTOiZ8Kq4QLn
YK+WSMFmVNC3mh4q5tqCb8LqsNG8cvl3ADgrC7jIxdO97q/+ulwMpcTXivKP+DiJ
5gASIB7GG1h4+Q0hDYOm7jYnQ3hWQzOR46bECxM5hrgaKPGgRqCSGjZmiJKoL8g=
=xDoQ
-----END PGP SIGNATURE-----