Qubes as Server OS?

1895 views
Skip to first unread message

stevenwi...@gmail.com

unread,
Dec 22, 2016, 3:41:25 PM12/22/16
to qubes-users
I thought about the fact if its possible to use Qubes OS as a Server OS for example for shared hosting or for application servers,etc.

You could basically use Template VMs and start AppVMs running the needed softwares for example on a shared hosting system.

Would something in this direction even be possible and would any other use cases be possible too?

I guess its possible to use it as VM Host too?

Are you using Qubes OS internally in some way like for the web server or at the moment not? :D

Marek Marczykowski-Górecki

unread,
Dec 22, 2016, 10:39:28 PM12/22/16
to stevenwi...@gmail.com, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, Dec 22, 2016 at 12:41:25PM -0800, stevenwi...@gmail.com wrote:
> I thought about the fact if its possible to use Qubes OS as a Server OS for example for shared hosting or for application servers,etc.
>
> You could basically use Template VMs and start AppVMs running the needed softwares for example on a shared hosting system.
>
> Would something in this direction even be possible and would any other use cases be possible too?
>
> I guess its possible to use it as VM Host too?

Most Qubes OS features are really for client systems. Like all the GUI
integration stuff, requiring various confirmations specifically from
local user, lack of network in dom0 etc.
If you take that out, mostly standard Xen installation will left there.
Just like any hosting provider have...

> Are you using Qubes OS internally in some way like for the web server or at the moment not? :D

We're under assumption that server infrastructure should not be trusted.
So, everything downloaded from the net - either ours servers, or others
- - should be verified. This is why(*) we use github instead of our own git
server, github pages + cloudflare for website, various mirrors for
binaries distribution and so on. And why we have reproducible builds on
the roadmap - to do the same with build machines - to not trust any
single machine building packages, but distribute that trust, verify
where possible.
The only thing we want from the infrastructure is reliability - and
using 3rd-party services makes it easier.

That said, technically you can setup some network services. For example
run a web server in netvm, and call some qrexec service in other
VM. Or redirect traffic to various VMs.
It still require local administration, so it's not suited to be running
in some data center. In theory you could hook up some IP KVM for that,
but then the whole setup would be only as secure as that IP KVM...

Some examples/documentation:
https://www.qubes-os.org/doc/development-workflow/#sending-packages-to-different-vm
https://www.qubes-os.org/doc/firewall/#port-forwarding-to-a-qube-from-the-outside-world
https://github.com/Rudd-O/qubes-network-server
https://github.com/marmarek/signature-checker/#github-webhook-integration

(*) And the other reason is limiting costs. But still, not running own
infrastructure make it easier to keep it that way - think twice when you
send/receive something from 3rd-party service, put some script etc.
Running own infrastructure would make it tempting have some trust in it.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYXJxqAAoJENuP0xzK19csXgYH/1qZU2Hqe1fNz4kzbq9yVxOd
jZCzAK+d/UFtZRFlSE8hNLkx1GKLd/eQEhP9LZ+5fs6gSJtPPVF9ElCTYnTQlZ9/
HJEjpS3IJlhk0cYBM7KM5mSNCFXLXQYWf9M73N15nVP5j9V1Ewkyl5V0DHxPpVWI
r5j2pslzgXsNql38gA1hAeA9JPB8W3+Dwm7zug2ln0PfinCER9q7oA39JWjPN2Wd
zgGtVkyQU42T22p+vSZvEebUNO8As8uy7SVvkNHI77IpUA7G7kdVviRCqJ/nLpXV
3h7VgFEdtDoXfFZ+7uhKXL0Y4mT5O7+w4WQSsU0ZtQUBrD5flexXTyF6xj87Bno=
=GNuw
-----END PGP SIGNATURE-----

raah...@gmail.com

unread,
Dec 22, 2016, 10:41:13 PM12/22/16
to qubes-users, stevenwi...@gmail.com

alot of overhead man just use a barebones system.

Jean-Philippe Ouellet

unread,
Dec 23, 2016, 5:19:25 PM12/23/16
to Marek Marczykowski-Górecki, Steven Winderlich, qubes-users
On Thu, Dec 22, 2016 at 10:39 PM, Marek Marczykowski-Górecki
<marm...@invisiblethingslab.com> wrote:
> On Thu, Dec 22, 2016 at 12:41:25PM -0800, stevenwi...@gmail.com wrote:
>> I thought about the fact if its possible to use Qubes OS as a Server OS for example for shared hosting or for application servers,etc.
>
> Most Qubes OS features are really for client systems. Like all the GUI
> integration stuff, requiring various confirmations specifically from
> local user, lack of network in dom0 etc.

These points are very true. However, IMO Qubes as a server is not
entirely as silly a choice for the capable and adventurous as it might
first appear.

> If you take that out, mostly standard Xen installation will left there.
> Just like any hosting provider have...

... except with decent dom0 disaggregation working out of the box, and
I'm personally making good use of qrexec in a server context as well.

Securely accessing dom0 remotely is left as an exercise for the reader. ;)

Nicklaus McClendon

unread,
Dec 23, 2016, 6:05:13 PM12/23/16
to Jean-Philippe Ouellet, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 12/23/2016 05:18 PM, Jean-Philippe Ouellet wrote:
> ... except with decent dom0 disaggregation working out of the box,
> and I'm personally making good use of qrexec in a server context
> as well.
>
> Securely accessing dom0 remotely is left as an exercise for the
> reader. ;)
>

I'm intrigued. How is qrexec utilized? qrexec is better than networked
access in the case of Qubes because it is verified through dom0, which
is part of the TCB. If you can't access dom0, qrexec is default allowed,
which removes the added security of it. If you're remotely accessing
dom0, you're adding the networking stack to the TCB, and once again have
a basic Xen installation with extra unnecessary overhead. qrexec with a
networked dom0 doesn't seem anymore secure than using SSH to run remote
scripts between networked VMs.
- --
kulinacs <nick...@kulinacs.com>
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEPL+ie5e8l/3OecVUuXLc0JPgMlYFAlhdrX4ACgkQuXLc0JPg
MlalzRAAgKmZPLtbbkuEyRr3fyJVDjxLwrV20c+WdtH/t9Snx//RPdLPmhEkqMTM
D4fscVTjjJzjXFg5m5JGZQpPKiZgENEwYxgnuyqIxWg7gcySr83lXCGRdL64u1n8
r5ydg42R7gqaeS8fh+Fkhxext+4PmOGieinh9FZRPYc+eT+VWsbZvMK7Yhso1bZO
U64JroC8O/JwUzJOl9VhqHChRjcbcxQszbyQadFT0QpEZ5HUoVcuW5nSj5w7jttJ
lCV8CkIOMwGDuzaZJU3b2dRIxMqe2C4wQRtlsXHTO9JANN4S22z+OBlfkb/KhxGB
d6caVxqgj0wgg0xWX7Fz5LBpNQtrL5xqBDVDMil3KSRsHEjJb23Ky6opJyJNaMWW
jtvShsp6fFZD18262ZwUwwqRMYV6sE4a3nITAo3yoIRaT3LHBKnUBHXuRqxKYJPf
AmkQAbyovsDKJ/9GHhHnOPLNunJqx/2pjuO4SaT1/7/+vb/ARXLmsq10jXdYKDsF
o2XeC3DisRlStU5/vXXOx4NW4cbj21a9hyaQ9pQxEv9QOi/0kDSdVzXyaw6qR3U+
cXMBYplL2ZnrjXCPpmZY4wI92STATMMgw/Vb8ZSbeSRKiaFYHrQv2SpFvMRW6VK1
4tPTkLF242cTUzkOFWtvD8kMwvHy1aGx/hfm1r11TlvPQHXgTzg=
=2+BL
-----END PGP SIGNATURE-----

raah...@gmail.com

unread,
Dec 23, 2016, 6:10:24 PM12/23/16
to qubes-users, marm...@invisiblethingslab.com, stevenwi...@gmail.com
)

but if its sole purpose is just being a server then who even cares if dom0 is compromised or not? might as well just use xenserver. if you talking about hosting something on your home desktop i guess thats a different story. I would use a cloud instead imo lol.

Jean-Philippe Ouellet

unread,
Dec 23, 2016, 6:41:27 PM12/23/16
to cooloutac, qubes-users, Marek Marczykowski-Górecki, Steven Winderlich
On Fri, Dec 23, 2016 at 6:10 PM, <raah...@gmail.com> wrote:
> but if its sole purpose is just being a server then who even cares if dom0 is compromised or not?

I strongly disagree.
1) If your server performs more than one purpose, having strong trust
boundaries as (attempted to be) provided by Xen is still very much
desired.
2) A non-compromised dom0 still adds another layer of defense against
an adversary who has fully compromised a guest from being able to
persist via access to low-level devices, such as possibly reflashing
your bios. This of course has to be weighed against the risk of
additional complexity introduced by running a hypervisor in the first
place. Xen does have bugs...

> might as well just use xenserver.

Sure. For a lot of use cases I agree that indeed makes more sense. I'm
only claiming that, while Qubes is not designed with servers as a
priority at all, there are still features that one may find beneficial
in that context.

> I would use a cloud instead imo lol.

Different strokes for different folks. :)

Jean-Philippe Ouellet

unread,
Dec 23, 2016, 7:10:25 PM12/23/16
to Nicklaus McClendon, qubes-users
On Fri, Dec 23, 2016 at 6:04 PM, Nicklaus McClendon
<nick...@kulinacs.com> wrote:
> I'm intrigued. How is qrexec utilized?

Something which I have not set up yet, but intend to soon, is a split
email server model, where the MTA and MDA are in separate VMs, and
incoming mail is delivered over qrexec. This would have the property
that an exploited SMTP server would not have any access to
already-delivered mails (assuming residual message contents are not
left hanging around in memory too long). Furthermore, I could have
multiple separate mail-sink VMs (with local mail storage and IMAP
servers), which are connected to separately via different instances of
my mail client on my Qubes laptop.

This means I could heuristically filter my mail based on assumed
purpose (for example, mail to a qubes mailing list go to one VM, mail
from my bank goes to another, etc.). This means a compromised mail
client on my Qubes laptop would be restricted to only the mail that
domain is expected to have, and pivoting from that compromised client
to compromising my IMAP server would not be sufficient to obtain the
mail from a different domain.

Additionally, the usual mail pipeline suspects like spamassasin,
clamav, etc. could be run in a DispVM per message (a super heavyweight
solution to be sure, but should not matter for personal mail workloads
on my own server).

> If you can't access dom0, qrexec is default allowed,

Uhh.... What? Can you elaborate?

As far as I understand and can verify, this is not the case:
[user@dom0 ~]$ /usr/lib/qubes/qrexec-policy --just-evaluate 56 disp8
disp9 qubes.Filecopy foo
[user@dom0 ~]$ echo $?
0
[user@dom0 ~]$ DISPLAY= /usr/lib/qubes/qrexec-policy --just-evaluate
56 disp8 disp9 qubes.Filecopy foo
qrexec-policy: cannot connect to X server
[user@dom0 ~]$ echo $?
1

> which removes the added security of it.

Definitely not entirely.

> If you're remotely accessing dom0, you're adding the networking stack to the TCB,

Not necessarily. At least your NIC need not be trusted, potentially more.

> and once again have a basic Xen installation with extra unnecessary overhead.

... and if overhead is your primary concern, why even bother with Xen
at all? Why not use containers or such.

> qrexec with a networked dom0 doesn't seem anymore secure than using SSH to run remote scripts between networked VMs.

Assuming you mean SSH between the VMs, I'm not sure I agree.
Networking has a much larger attack surface, and you are relying on
crypto with keys that can be stolen rather than vchan where the
authenticity of source and dest and integrity of msg contents are
already trusted. OTOH, OpenSSH has a pretty stellar track record and
has likely received more scrutiny than vchan (which I have not audited
myself yet either), so I can certainly understand the contrary
argument too.

raah...@gmail.com

unread,
Dec 23, 2016, 7:13:36 PM12/23/16
to qubes-users, raah...@gmail.com, marm...@invisiblethingslab.com, stevenwi...@gmail.com

ya but my point is if it infects the guest then who cares about dom0. but you say your server would be used for more then one purpose so I guess its diff. But even still. unless you plan to have all your diff "server vms" integrate/interact with each other in some way it doesn't make sense to me to use qubes instead of xen.

Nicklaus McClendon

unread,
Dec 23, 2016, 7:35:46 PM12/23/16
to Jean-Philippe Ouellet, qubes...@googlegroups.com
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 12/23/2016 07:09 PM, Jean-Philippe Ouellet wrote:
>> If you can't access dom0, qrexec is default allowed,
>
> Uhh.... What? Can you elaborate?

qrexec usage is normally defined by an RPC. This RPC has a policy,
either allow, deny, or ask. My understanding is that if you don't have
access to dom0 to respond to a prompt, you must allow the running RPC
by default if you want to use it. This argument, of course, hinges on
my skepticism of secure remote dom0 access.

>> which removes the added security of it.
>
> Definitely not entirely.

I'm not sure what security is added by having a default allow Qubes
RPC policy, but once again, this could be mitigated with secure dom0
access.

>
>> If you're remotely accessing dom0, you're adding the networking
>> stack to the TCB,
>
> Not necessarily. At least your NIC need not be trusted, potentially
> more.

I think the NIC still must be trusted in some form or fashion,
however, as a rogue attacker in the network vm could just shut off
access to whatever secure management VM being utilized. I'm not sure
how to classify this though. An attack on the NIC could stop remote
management, but I don't think this would harm security. In any case,
there is a lot to be added to the TCB by allowing network access, like
a management VM, that makes me question its usefulness.

>> and once again have a basic Xen installation with extra
>> unnecessary overhead.
>
> ... and if overhead is your primary concern, why even bother with
> Xen at all? Why not use containers or such.

Overhead relative to functionality. A type 1 hypervisor offers better
isolation than containers, and I was suggesting that Qubes features
may not provide more security in a server environment.

- --
kulinacs <nick...@kulinacs.com>
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEPL+ie5e8l/3OecVUuXLc0JPgMlYFAlhdwsoACgkQuXLc0JPg
MlaxFxAAl8ElXNawe9okc1RkzE6jbZUyEXK4r3n2SHsdZh1L2VIgmDBhcZankrm5
FoTMCpcqpoUIFde1PyaA5ZiIC9wCgDaiLMRHRkblCV6CYZ+7mufb0hUUX98jngLS
Qkgee6lE0qB4iZWbtMKyOjlZbkODe1y8L65MjYB6qEy28IaXlHD64rptMi86qnvS
4Wag811+4Uox9reWCY5cmqriyMmx5sQN2cauWOausYIB8GRk/yYRXAk4Ex7bSuYh
DXHrS2RIyZ9w5M7By1hcKaD+n29AJ42xVmJPlnK7kcTV67yO/gW7PDjwlLQj5W7F
b2Ql4IBv56eTjP16cEeiq95K24enfx6drHBJuyRbopGVleAsGP4ZXroxZbufPxpf
vrV6hdcxPoDZ+vigfdQTP5pxtZYk4msut0rAYi/36fhhvCkSmYYPDEZXUzlFg0us
wSCw/9Tf/mir5aG7vNa4UzF2NKJRZHoudG+Td5cAJULpCgnocaQ+K27fOpA3UrK2
LzEuZi1iG5KbXyRueeOi+PYYtDuf86vFBxS1z79fooEN1mu8ksv4IsJcG5lbs6p2
C3398XeoyuE5oy+fzSXgU+Yem2w5AQ4PNIfzU+5XDXG9tBHXMXbjL/EhHAIxldgq
C0fce2Sfdw+pPj6nyE5qDe0gsvCotQgkjC1/NZmx/i/7FqaOUL8=
=SWEu
-----END PGP SIGNATURE-----

Jean-Philippe Ouellet

unread,
Dec 23, 2016, 9:42:16 PM12/23/16
to Nicklaus McClendon, qubes-users
On Fri, Dec 23, 2016 at 7:35 PM, Nicklaus McClendon
<nick...@kulinacs.com> wrote:
> On 12/23/2016 07:09 PM, Jean-Philippe Ouellet wrote:
>>> If you can't access dom0, qrexec is default allowed,
>>
>> Uhh.... What? Can you elaborate?
>
> qrexec usage is normally defined by an RPC. This RPC has a policy,
> either allow, deny, or ask. My understanding is that if you don't have
> access to dom0 to respond to a prompt, you must allow the running RPC
> by default if you want to use it. This argument, of course, hinges on
> my skepticism of secure remote dom0 access.
>
> I'm not sure what security is added by having a default allow Qubes
> RPC policy, but once again, this could be mitigated with secure dom0
> access.

I don't think manually allowing each time something must be used makes
much sense at all in a server environment (where everything is
expected to be automated, rather than as a direct result of user
actions). The qrexec policy framework makes sense as a mechanism to
clearly define allowed inter-vm data flow.

Given that actions on a server are not likely to be directly
user-triggered anyway, I don't think a user would even have much
meaningful information with which to even detect when an "allow" is
unexpected, and therefore I don't think is more useful than "allow
always".

> I think the NIC still must be trusted in some form or fashion,
> however, as a rogue attacker in the network vm could just shut off
> access to whatever secure management VM being utilized. I'm not sure
> how to classify this though.

Simple Denial of Service. And sure, but there are tons of other ways
to do that which do not involve exploiting flaws in my NIC, and (more
importantly) doing so doesn't lead an attacker to any more useful
ability than they had already with respect to the things I'm actually
trying to protect.

> I was suggesting that Qubes features may not provide more security in a server environment.

The case I am making is for their utility (hopefully) without
significantly reduced security. I am definitely not claiming increased
security compared to Xen alone.


Anyway, I didn't mean to start a discussion arguing in favor of Qubes
over other things as a server. It is clearly not designed for that. I
only meant to point out that those who see fit to use it as a server
are not necessarily unjustified in doing so.
Reply all
Reply to author
Forward
0 new messages