Qubes 4.0 on high(er) end workstations?
267 views
Skip to first unread message
Steve Coleman
Oct 15, 2018, 12:14:26 PM10/15/18
to qubes...@googlegroups.com
I had attempted to upgrade my HP machine at home to R4.0 a while back
and ran into a VT-d related message about reassignable interrupts not
being found, yet I do have the VT-d enabled in bios. I never had any
indication while running R3.2, or before, that there was any issue with
the VT-d functionality. No bios upgrades are available from the
manufacture and I can't really afford to be without a functional machine
should I need to spend time trying work out why, or to force an upgrade.
Since support for R3.2 will at some point be deprecated, I thought I
should start doing some investigation for some new hardware while I have
a chance and before I am pressured to move forward. If I stand up a new
machine I will be better able to investigate any issues on the older
machine later.
The selection of laptops looks good on the HCL, and there has been quite
a bit of discussion on various options. But it would appear that there
are very few Desktop machines on the Qubes 4.0 HCL list have been fully
tested and are green all the way across. In fact the one machine that is
green all the way across for R4.0 just happens to be my own HCL report,
for my work desktop system. Even then its difficult to compare the
relative computational power that each entry has without searching for
each machines specs, one by one. The CPU identifier, if specified, might
give a relative ranking, thought the number or cores, ram, Ghz, and
disks are notably absent thus it hard to rank them.
Since my old and outdated Dell Optiplex 990 seems to be the only game in
town, I'm therefore stuck looking at the Dell Optiplex 7050, but then I
don't have any particular loyalty to Dell. I don't mind building a
system from scratch using a good motherboard, if I had to, but it seems
the motherboards listed on the HCL are even less well tested for R4.0
than the desktop systems are. Not a single board on that list is even
running R4.0!
So, I figured I should just ask here, What high end R4.0 systems work
for you? What Desktop systems are fairly high end (Cores, GB's DRAM,
ample disk storage bays, multiple monitors) that are working well under
R4.0?
Are there *any* systems with a tested TPM setup capable of the
Anti-Evil-Maid configuration that have not yet made it onto the HCL? Or
is it only laptops that are doing this? I could force a laptop work if
it is both dockable and can come with enough Dram/Disk space, but then I
would never undock it, and thus I would be paying big $$$ for something
I'm not even planning to use it for.
Oh, if there is something running good out there, and it passes all the
tests under R4.0, please consider helping to update the HCL with R4.0
machines that actually work! Its always nice to know which ones to
avoid, but knowing what works is a much better way to go.
Thank you for your consideration.
Steve.
and ran into a VT-d related message about reassignable interrupts not
being found, yet I do have the VT-d enabled in bios. I never had any
indication while running R3.2, or before, that there was any issue with
the VT-d functionality. No bios upgrades are available from the
manufacture and I can't really afford to be without a functional machine
should I need to spend time trying work out why, or to force an upgrade.
Since support for R3.2 will at some point be deprecated, I thought I
should start doing some investigation for some new hardware while I have
a chance and before I am pressured to move forward. If I stand up a new
machine I will be better able to investigate any issues on the older
machine later.
The selection of laptops looks good on the HCL, and there has been quite
a bit of discussion on various options. But it would appear that there
are very few Desktop machines on the Qubes 4.0 HCL list have been fully
tested and are green all the way across. In fact the one machine that is
green all the way across for R4.0 just happens to be my own HCL report,
for my work desktop system. Even then its difficult to compare the
relative computational power that each entry has without searching for
each machines specs, one by one. The CPU identifier, if specified, might
give a relative ranking, thought the number or cores, ram, Ghz, and
disks are notably absent thus it hard to rank them.
Since my old and outdated Dell Optiplex 990 seems to be the only game in
town, I'm therefore stuck looking at the Dell Optiplex 7050, but then I
don't have any particular loyalty to Dell. I don't mind building a
system from scratch using a good motherboard, if I had to, but it seems
the motherboards listed on the HCL are even less well tested for R4.0
than the desktop systems are. Not a single board on that list is even
running R4.0!
So, I figured I should just ask here, What high end R4.0 systems work
for you? What Desktop systems are fairly high end (Cores, GB's DRAM,
ample disk storage bays, multiple monitors) that are working well under
R4.0?
Are there *any* systems with a tested TPM setup capable of the
Anti-Evil-Maid configuration that have not yet made it onto the HCL? Or
is it only laptops that are doing this? I could force a laptop work if
it is both dockable and can come with enough Dram/Disk space, but then I
would never undock it, and thus I would be paying big $$$ for something
I'm not even planning to use it for.
Oh, if there is something running good out there, and it passes all the
tests under R4.0, please consider helping to update the HCL with R4.0
machines that actually work! Its always nice to know which ones to
avoid, but knowing what works is a much better way to go.
Thank you for your consideration.
Steve.
Yethal
Oct 15, 2018, 2:09:51 PM10/15/18
to qubes-users
MB: Asrock x99 itx/ac
CPU: i7 6800K
RAM: Corsair Vengeance LPX 3000mhz 32GB
SSD: Samsung Pro 950 NVME 256GB
GPU: Nvidia GTX 750Ti (do not buy)
VT-x, VT-d, Interrupt Remapping works. Mobo has a TPM header. It also has a PS/2 port (extremely important in Qubes and often overlooked)
Tai...@gmx.com
Oct 15, 2018, 7:15:54 PM10/15/18
to qubes...@googlegroups.com, Steve....@jhuapl.edu
I have many posts on this but since you have an .edu and made a long
post yourself here are two great options.
You wanna assemble stuff yourself which is pretty easy - I did my first
at age 12 and it worked on the first power on.
Libre motherboards that work with qubes 4:
* KCMA-D8 (90 used on fleabay from china) and one or two 8 core socket
C32 4386 opteron CPU's plus ECC RDIMM RAM in 8GB sticks (for 64 total)
or 16gb (for 128 total)
* KGPE-D16 ($130 on fleabay brand new) and one or two 16 core 6386 CPU's
or 8 core 6328 CPU's (60 on fleabay brand new) which supports up to
192GB RAM.
Since they support libre firmware it doesn't matter that you are getting
used hardware although I believe newegg still has the KGPE-D16 if you
must have new hardware.
Both support Crossfire xDMA and IOMMU-GFX for gaming or cad in a VM, all
the devices have their own IOMMU groups and it supports ACS.
The D8 and D16 are the last and best owner controlled x86 motherboards
and they support coreboot-libre or libreboot, and also OpenBMC for
secure libre remote access with the ASMB4 or ASMB5 chip - it comes with
the new in box KGPE-D16 but they also crop up time to time on fleabay
for a few bucks.
I would say that TPM's/AEM is a not needed if you implement
kernel/initramfs code signing in grub as a coreboot payload, set the
write lock bit on the flash chip and then put a lock on your case but if
you still want a TPM it has a header for a v1.2 device make sure to buy
a supported model.
Other options are the Raptor Computing Systems Libre Firmware OpenPOWER
systems such as the TALOS 2 and the more affordable Blackbird which are
the future of owner controlled computing[1] although currently qubes/xen
doesn't have a POWER port so you would have to use POWER-KVM which
arguably is better security wise than xen+black boxed x86 junk and again
is the future not a dead platform.
I am an expert on this topic, let me know if you need any help and if
you think my advice is patron-grade.
[1]x86 is dead freedomwise, both AMD and intel have a variety of
anti-features that make you just a licensee not an owner - OpenPOWER is
the only owner controlled performance CPU arch luckily it is now more
affordable than equivilant x86 performance enterprise hardware and you
get more features+freedom :D
It is impossible to disable ME/PSP or make libre firmware for a new gen
x86 system.
post yourself here are two great options.
You wanna assemble stuff yourself which is pretty easy - I did my first
at age 12 and it worked on the first power on.
Libre motherboards that work with qubes 4:
* KCMA-D8 (90 used on fleabay from china) and one or two 8 core socket
C32 4386 opteron CPU's plus ECC RDIMM RAM in 8GB sticks (for 64 total)
or 16gb (for 128 total)
* KGPE-D16 ($130 on fleabay brand new) and one or two 16 core 6386 CPU's
or 8 core 6328 CPU's (60 on fleabay brand new) which supports up to
192GB RAM.
Since they support libre firmware it doesn't matter that you are getting
used hardware although I believe newegg still has the KGPE-D16 if you
must have new hardware.
Both support Crossfire xDMA and IOMMU-GFX for gaming or cad in a VM, all
the devices have their own IOMMU groups and it supports ACS.
The D8 and D16 are the last and best owner controlled x86 motherboards
and they support coreboot-libre or libreboot, and also OpenBMC for
secure libre remote access with the ASMB4 or ASMB5 chip - it comes with
the new in box KGPE-D16 but they also crop up time to time on fleabay
for a few bucks.
I would say that TPM's/AEM is a not needed if you implement
kernel/initramfs code signing in grub as a coreboot payload, set the
write lock bit on the flash chip and then put a lock on your case but if
you still want a TPM it has a header for a v1.2 device make sure to buy
a supported model.
Other options are the Raptor Computing Systems Libre Firmware OpenPOWER
systems such as the TALOS 2 and the more affordable Blackbird which are
the future of owner controlled computing[1] although currently qubes/xen
doesn't have a POWER port so you would have to use POWER-KVM which
arguably is better security wise than xen+black boxed x86 junk and again
is the future not a dead platform.
I am an expert on this topic, let me know if you need any help and if
you think my advice is patron-grade.
[1]x86 is dead freedomwise, both AMD and intel have a variety of
anti-features that make you just a licensee not an owner - OpenPOWER is
the only owner controlled performance CPU arch luckily it is now more
affordable than equivilant x86 performance enterprise hardware and you
get more features+freedom :D
It is impossible to disable ME/PSP or make libre firmware for a new gen
x86 system.
Tai...@gmx.com
Oct 15, 2018, 7:22:58 PM10/15/18
to qubes...@googlegroups.com
On 10/15/2018 02:09 PM, Yethal wrote:> It also has a PS/2 port
You instea dwant more than one USB controller on a system so you can
have both trusted for keyboard/mice and untrusted for random stuff (all
my recs in my other reply have this, the D16/D8's have a second
controller via a few onboard usb headers)
PS/2 is not secure at all - your keystrokes are outputted on the ground
wire.
I suggest purchasing a usb keyboard that doesn't have firmware such as
the excellent us made unicomp model m mechanical keyboard, to prevent
use of a keyboard virus.
Definitely agreed with not buying nvidia junk though, they artificially
hamper virt with their geforce stuff and they also hate linux drivers
and FOSS.
(extremely important in Qubes and often overlooked)
Misinformation.
You instea dwant more than one USB controller on a system so you can
have both trusted for keyboard/mice and untrusted for random stuff (all
my recs in my other reply have this, the D16/D8's have a second
controller via a few onboard usb headers)
PS/2 is not secure at all - your keystrokes are outputted on the ground
wire.
I suggest purchasing a usb keyboard that doesn't have firmware such as
the excellent us made unicomp model m mechanical keyboard, to prevent
use of a keyboard virus.
Definitely agreed with not buying nvidia junk though, they artificially
hamper virt with their geforce stuff and they also hate linux drivers
and FOSS.
unman
Oct 16, 2018, 10:21:02 AM10/16/18
to qubes...@googlegroups.com
On Mon, Oct 15, 2018 at 07:25:12PM -0400, Tai...@gmx.com wrote:
> On 10/15/2018 02:09 PM, Yethal wrote:> It also has a PS/2 port
> (extremely important in Qubes and often overlooked)
> Misinformation.
>
> You instea dwant more than one USB controller on a system so you can
> have both trusted for keyboard/mice and untrusted for random stuff (all
> my recs in my other reply have this, the D16/D8's have a second
> controller via a few onboard usb headers)
>
> PS/2 is not secure at all - your keystrokes are outputted on the ground
> wire.
You really should be more specific on this issue. SOME PS/2 keyboards
> On 10/15/2018 02:09 PM, Yethal wrote:> It also has a PS/2 port
> (extremely important in Qubes and often overlooked)
> Misinformation.
>
> You instea dwant more than one USB controller on a system so you can
> have both trusted for keyboard/mice and untrusted for random stuff (all
> my recs in my other reply have this, the D16/D8's have a second
> controller via a few onboard usb headers)
>
> PS/2 is not secure at all - your keystrokes are outputted on the ground
> wire.
allow keystrokes to be read from ground. It's possible to mitigate this
in various ways or clean signal from the earth wire.
Almost all keyboards are open to side channel attacks. It's possible to
reduce the risk of those attacks in various ways depending on your
risk profile.
You are absolutely right that having multiple USB controllers is a
benefit in Qubes, but for many people, using a PS/2 keyboard will
address the main risk factors accompanying use of USB devices, and
imo shouldn't be so quickly dismissed.
Yethal
Oct 16, 2018, 12:21:59 PM10/16/18
to qubes-users
Also, if an attacker is capable of tapping into the ground wire of your keyboard to listen to the keystrokes then they are more than capable of simply plugging a usb keylogger and/or usb hub and a flashdrive. IMHO a usb controller in dom0 poses much bigger security risk due to reduced attack complexity.
Tai...@gmx.com
Oct 16, 2018, 8:19:32 PM10/16/18
to qubes...@googlegroups.com
per controller so for example one trusted for inputs and one not trusted
for random stuff.
Ground wires where I live go far away from where I am sitting as they do
in any large office complex so that is not so good. Any secure facility
has ground wire isolation for that reason.
Yethal
Oct 18, 2018, 10:10:57 AM10/18/18
to qubes-users
shizo
Oct 19, 2018, 3:35:55 AM10/19/18
to qubes-users
Hi!:) Thanks for the information. Do you have irс? Because of stupid mails, you have to write here with thousands of different accounts, I think it would be more convenient to communicate there.
I have been using Qubes on laptops for three years and apart from creating firmware, nothing can be done about it. Decided to collect a workstation on your advice (x220, coreboot/heads)
I want to use virtualization at full capacity
Advise the video card?
Is it RX580?
I just bought :
2x CPU AMD-OPTERON-16-Core-6276 ~80 bucks :))
ASUS KGPE-D16 - 170 bucks
will she make noise at home?
I have a passive cooling of the house, radiators - is it good idea?
Is it hard to attach a video card to hvm?
I am installing it myself at home, I work in a data center, and for example I was surprised when I found out that the motherboard Supermicro has the same spi chipsets (winbond, micron, macronix)
how much memory is needed for the first time? 32-64gb?
Fucking IBM (Power9) is too expensive.
last hope is ASUS KGPE-D16.
Thanks :))
I have been using Qubes on laptops for three years and apart from creating firmware, nothing can be done about it. Decided to collect a workstation on your advice (x220, coreboot/heads)
I want to use virtualization at full capacity
Advise the video card?
Is it RX580?
I just bought :
2x CPU AMD-OPTERON-16-Core-6276 ~80 bucks :))
ASUS KGPE-D16 - 170 bucks
will she make noise at home?
I have a passive cooling of the house, radiators - is it good idea?
Is it hard to attach a video card to hvm?
I am installing it myself at home, I work in a data center, and for example I was surprised when I found out that the motherboard Supermicro has the same spi chipsets (winbond, micron, macronix)
how much memory is needed for the first time? 32-64gb?
Fucking IBM (Power9) is too expensive.
last hope is ASUS KGPE-D16.
Thanks :))
shizo
Oct 19, 2018, 3:55:39 AM10/19/18
to qubes-users
https://store.vikings.net/vikings-d16-workstation
https://minifree.org/product-category/desktop-pcs/
https://tehnoetic.com/desktops/tet-d16ws
https://minifree.org/product-category/desktop-pcs/
https://tehnoetic.com/desktops/tet-d16ws
you can still see it, but they have crazy prices
and for some reason, the video card is nvidia, not amd
Tai...@gmx.com
Oct 21, 2018, 11:47:45 PM10/21/18
to qubes...@googlegroups.com
You can just buy a new one for $150 off fleabay atm no reason to pay
MSRP for anything most of the time, get used cpu ram too unless new is
not too much more such as here.
6276 trash for gaming, get one or two 6328 ($60/ea brand new on fleabay)
Search my many posts going years back for information and if you have
any non answered questions email me...but as I currently make minimum
wage in real life I charge bitcoin for answering the same questions
repeatedly (all the info you need to do this I and others have already
posted many many times) or corresponding with people who use gmail (as
gmail violates many of my beliefs...and wanting a special security
workstation while still using gmail is silly)
I game on my D16 but you need a decent CPU like 6328 (best), 6287SE
6386SE - MUST INSTALL MICROCODE UPDATES BTW or either nothing will work
or it will be very insecure. In coreboot check binary only repo+generate
microcode from tree.
If you buy a new D16 you get a ASMB4/5 modules which you can install
OpenBMC FOSS remote access on in addition to coreboot-libre, it also
controls your fans otherwise use fancontrol/pwmconfig to slow them down
from max speed.
Nvidia anything is junk as they hate linux - AMD RX580 works fine with
D16 gaming in a VM both linux and windows guests on a linux host even
Crossfire xDMA also works in a VM.
IF you properly configure everything and do nothing else on those CPU
cores (dedicated and pinned cores) your performance will be only 1% less
than bare metal, if you want to do other stuff on the device you need to
buy more than one CPU probably so dual 6328 instead of just one...but
they are cheap so is the G34 140W tower 3U/4U coolers right now.
shizo
Oct 22, 2018, 4:13:39 AM10/22/18
to qubes-users
понедельник, 22 октября 2018 г., 6:47:45 UTC+3 пользователь Tai...@gmx.com написал:
Thats impossible, my english sucks, but you are wizard. Double thanks to you.
I understood about processors and motherboard and video and flash settings. Now next step.
I would not miss the RAM, I have a limited money, is it correct if I purchase it according to the table here?
https://www.coreboot.org/Board:asus/kgpe-d16
crucial ("crucial by Micron") CT16G3ERSLD4160B (MT36KSF2G72PZ-1G6P1NE) 192GB 16GB DDR3-1600 Registered Yes Leave H1, H2, G1, G2 empty (see page 2-16 in the ASUS manual), LVDDR3_SEL1 can be set to "Force 1.35V" Opteron 6278/6282SE/6284SE/6287SE 1.03G, 1.04 coreboot d6735b0
Only this RAM will work?
Offtop. And second question.
My friend has a laptop that supports optimus, I installed qubes there in UEFI mode, but there is no bootloader, i want try to gpu passthrouth 950m to hvm, for example to ubuntu desktop template. Is it possible?
I read this https://paste.debian.net/1043341
How to change xen options ( i need disable iommu-gfx and passthrouth second gpu, because hybrid graphics) in UEFI mode? there is no grub/refind/
xl dmesg | grep iommu
(XEN) Command line: loglvl=all dom0_mem=min:1024M dom_mem=max:4096M iommu=no-igfx ucode=scan smt=off
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [VT-D] Passed iommu=no-igfx option. Disabling IGD VT-d engine.
qvm-pci
dom0:01_00.0 3D controller: NVIDIA Corporation GM107M [GeForce GTX 950M]
Maybe someone else help me.
I understood about processors and motherboard and video and flash settings. Now next step.
I would not miss the RAM, I have a limited money, is it correct if I purchase it according to the table here?
https://www.coreboot.org/Board:asus/kgpe-d16
crucial ("crucial by Micron") CT16G3ERSLD4160B (MT36KSF2G72PZ-1G6P1NE) 192GB 16GB DDR3-1600 Registered Yes Leave H1, H2, G1, G2 empty (see page 2-16 in the ASUS manual), LVDDR3_SEL1 can be set to "Force 1.35V" Opteron 6278/6282SE/6284SE/6287SE 1.03G, 1.04 coreboot d6735b0
Only this RAM will work?
Offtop. And second question.
My friend has a laptop that supports optimus, I installed qubes there in UEFI mode, but there is no bootloader, i want try to gpu passthrouth 950m to hvm, for example to ubuntu desktop template. Is it possible?
I read this https://paste.debian.net/1043341
How to change xen options ( i need disable iommu-gfx and passthrouth second gpu, because hybrid graphics) in UEFI mode? there is no grub/refind/
xl dmesg | grep iommu
(XEN) Command line: loglvl=all dom0_mem=min:1024M dom_mem=max:4096M iommu=no-igfx ucode=scan smt=off
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [VT-D] Passed iommu=no-igfx option. Disabling IGD VT-d engine.
qvm-pci
dom0:01_00.0 3D controller: NVIDIA Corporation GM107M [GeForce GTX 950M]
Maybe someone else help me.
travorfi...@gmail.com
Oct 23, 2018, 4:10:30 AM10/23/18
to qubes-users
>coreboot Qubes and AMD eGPU + x220 in theory ?
Anybody tried this? I have no experience with IOMMU and eGPU use Qubes.
Maybe i'am stupid. How does this relate to security? do you run Google Chrome in virtual machines or systemd? explain to me please if you have 2 minutes for me :)
Also https://www.qubes-os.org/doc/multimedia/
if I understand correctly in isolated VM (Qubes OS design) you can run anything of software. SEcure/unsecure it doesn't matter. Dom0 is fully isolated from internet and VM's.
If you wan't full libre PC, you need deblob kernel, flash libreboot, dissaassemble propietary parts of your PC and run cli software (not bloated, clean code) from libre repositories supported by FSF.
> Will it support advance Expresscard support.
By this I suppose you mean the PCIe interface. The ExpressCard should
look like any other PCIe and if there is a card plugged in then
coreboot should enumerate it, and FILO should be able to see a
controller. Hotplugging is not supported by coreboot and FILO. That
means that a PCIe card must be plugged in before coreboot enumerates
the PCIe. (Usually a few 100 ms after power on.) I don't think we
have seen coreboot run on a system with ExpressCard yet however, so
this is just the theory of how it should work. If you can send debug
output from a system with coreboot and ExpressCard that would be
interesting.
https://www.coreboot.org/Board:lenovo/x220
Tested (and works):
Expresscard slot (including hotplugging)
...
https://egpu.io/forums/expresscard-mpcie-m-2-adapters/any-concern-with-coreboot-and-egpu/
https://www.reddit.com/r/eGPU/comments/89xw5k/egpu_with_coreboot/
https://www.reddit.com/r/eGPU/comments/6n3epq/egpu_x230_with_coreboot_does_it_work/
https://github.com/QubesOS/qubes-issues/issues/2841
>iommu=force
from Tai...@gmx.com and Yuraeitha
https://groups.google.com/forum/#!topic/qubes-users/MEMdWdsht5k
Well the day a proper secure, user owned laptop hardware, which is something not looking like it came from the last decade, has proper thunderbolt and similar tech only available on modern laptops (which I need, in all seriousness), I'll immediately buy and never look back.
Considering that the TALOS 2, KGPE-D16, KCMA-D8 and the G505S's
firmwares are open source and every component such as pci-e addon cards
that aren't are restricted by the IOMMU - again you give dangerous
advice and suggest that people focus on some vague theoretical backdoor
rather than what is a proven fact (that intel machines are owned by
intel, not you) and thus tell them they shouldn't even bother with security.
Are there any detailed instructions for using IOMMU and passthrouth GPU into HVM? or it really bad IDEA? (insecure stuff)
sorry, just don't hit :)
Reply all
Reply to author
Forward
0 new messages