sys-net & sys-firewall fail to start, new install

[scott.lewi...@gmail.com]

Hello, I apologize in advance if this has already been addressed but searching for "sys-net" & "libxl" give many confusing results, I don't see any that match both not working, and I'm completely new to Qubes.
My problem is that after 2 different installs of the same version, 4.01, I keep getting the same problems: When I get into Qubes I cannot update dom0, and sys-net & sys-firewall both fail to start even though my NIC is correctly listed in devices for -net.

Basic info:
Dell Optiplex 780, bios= A15 as recommended in HCL.
Qubes 4.01, sigs checked good.
Install 1 done by USB, created in windows, had same issues as described below.
Re-installed by DVD thinking I'd muffed something (think I read on reddit that USB installs made from windows had issues) and the behavior is exactly the same.

Behavior observed:
(anything in [xxx] is me telling you the variation, as I get this for 2 VM's, 2 error codes, etc.)
* Dom0 starts and runs, but -net & -firewall will not.
* When I attempt to manually start either, I get this error message:
"ERROR: Start failed: internal error: libxenlight failed to create new domain 'sys-net' [or sys-firewall], see /var/log/libvirt/libxl/libxl-driver.log for details".

When I less the log, I get these errors dozens of time with timestamps ranging from date of install to today:
1: "[date:time]: libxl: libxl_create.c:610:libxl_domain_make:domain creation fail: Operation not supported"
2: "[date:time]: libxl: libxl_create.c:982:initiate_domain_create:cannot make domain: -3"
Those errors and nothing else.

I've tried to update dom0 by command as found in docs, as well as through GUI without success. I found a page elsewhere that described temporarily putting NIC in firewall or another VM to see if it worked, but I must be doing it wrong as I always get loopback errors when trying to save.
It always tells me failed, regardless. No wonder if network isn't working.

Under sys-net, I see the correct name and model of my NIC, but I can't seem to figure out how to see if the NIC is operational since sys-net isn't working (new to qubes sorry!), however I verified my router is not showing an IP assigned, though link LED is on.

If anyone can help me try to figure this out, I'd really appreciate it. I'm still a bit green on linux but I learn fast, I recently gave up on Windows (I hate 10 so much I could scream) so I'm trying to dive in with qubes (deep end of the pool I think) as it best suits my needs with the VM implementation.

I'd post the log itself, but I can't seem to figure out how to make USB copy work yet since I've been beating my head on the networking issue.

Thanks, and sorry in advance if this duplicated another thread.

[Mike Keehan]

Similar issues in the past were raised in this bug report -

https://github.com/QubesOS/qubes-issues/issues/3125

It might give you some pointers.

(I found it by searching for "Qubes libxl_domain_make:domain creation
fail: Operation not supported" in google.)

Mike.

[Scott Lewis]

Thanks but unfortunately I don't know that issue helps, as those who were able to resolve it did so by updating which I cannot do since sys-net fails to start.

This issue, https://github.com/QubesOS/qubes-issues/issues/3349 , seems to cover that problem but nothing's mentioned about net & firewall failing to start.

--
You received this message because you are subscribed to a topic in the Google Groups "qubes-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qubes-users/mGvt8QW2mBQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qubes-users...@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/20190118122156.4973a340.mike%40keehan.net.
For more options, visit https://groups.google.com/d/optout.

[unman]

Test whether issue is with NIC by removing the card from sys-net and confirming that
sys-firewall starts (after sys-net).
Shutdown both.
Re-attach NIC to sys-net.
Report back.


You can extract logs from dom0 by creating a qube with NO netvm, and
using qvm-copy-to-vm to transfer the log to that qube. Attach USB
device to the qube (using devices widget) and copy log off system.

[scott.lewi...@gmail.com]

On Friday, January 18, 2019 at 9:16:59 AM UTC-5, unman wrote:

Thanks Mike, here's the results:
* Trying to start the USB VM results in error: ERROR: Start failed: internal error: libxenlight failed to create new domain 'USBlogs', see /var/log/libvirt/libxl/libxl-driver.log for details (at least I know why I couldn't get it to work the 1 time I tried before OP).
* Trying to start -net resulted in the same error, with the NIC removed.
* Trying to start -firewall resulted in the error saying -net couldn't start, pointing to libxenlight's log also.

Other things I observed again but not sure I detailed enough before:
* Sys-net won't save with any setting but none for networking, but firewall gives me the error that networking is disabled and must be enabled, isn't that normal though since that's the network VM? The other 4 options give me the same "loops in network are unsupported" error for any of 'default (sys-firewall), sys-firewall, sys-net, sys-whonix'.
* Sys-firewall gives me config error that it's connected to sys-net which doesn't support firewall (I assume due to above).

[scott.lewi...@gmail.com]

Sorry, thanks for answering Mike & unman, wasn't paying enough attention.

[Mike Keehan]

Actually, it was unman who suggested looking at the logs:)

If you set sys-firewall's networking to none, can you start it
successfully?
Do you have enough memory? Use xentop to see how much is free.
Does your bios have vtx and iommu enabled OK?
What VMs startup when you boot apart from dom0?

Mike.

[Bryce]

Weirdly, I replied but it said msg was deleted, perhaps because I posted a link to images of xentop and errors; must not be allowed.
* Yes, xentop shows the correct amount of physical memory I think; 15623796k (I have 16g (4x4) installed to this desktop).
* With -firewall @ none, it fails to start giving me the libxenlight error for failed to start.
* Vt-x & -d are enabled in bios. During install one of the 2 times (by date I think the 1st time via USB install), the installer complained about IOMMU remapping missing, but it installed and boots fine now, not sure if that's cuz I reinstalled via DVD or not.
* No VM's other than dom0 will run unfortunately :(

[Mike Keehan]

Ah, afraid I've run out of ideas now.

I was hoping it was just the iommu not being enabled, but not starting
any vm at all is not something I've seen before, as long as there is
enough memory. 16Gb is fine.

Sorry, out of inspiration now,

Mike.

[Bryce]

Thanks anyways Mike. I did manage to attach the pics of memory and qubes manager at least, in case they're useful for anyone.

[Mike Keehan]

That's odd. You say you are installing 4.01, yet the qubes manager
shows that fedora is only release 26. I thought that Qubes 4.01 had
Fedora 29 in it.

Mike.

[Bryce]

> That's odd. You say you are installing 4.01, yet the qubes manager
> shows that fedora is only release 26. I thought that Qubes 4.01 had
> Fedora 29 in it.
>
> Mike.

Well Mike that could be it; you made me check when you said that, the image I used was "Qubes-R4.0-x86_64.iso" from 11/27/18. I checked the about in qubes itself is 4.0 (R4.0). Looks like I screwed up on which version I created the image from, I wrote down 4.01 but that's not what I used- too much spiked eggnog maybe.
Since it doesn't seem to be possible for me to update with the VM's not starting for USB or Net, I will get the latest image and and reinstall to see if there's any issue.
Thanks for pointing out my goof, I'll try that today and see if the results are any different!

[Bryce]

I Reinstalled w/ 4.01 (downloaded today) and sadly I seem to have the same issue.
Here's what I did:
* During install I realized what I had forgot before, that previous occasions I had to turn off "Intel VT for Direct I/O" in bios in order to get the installer to even work. If I don't turn that off, the screen goes black and nothing happens no matter which option I pick from the setup list- normal install, test & install, and basic graphics install all do the same.
So I turn it off, and the install works fine until the language selections screen then I get an error that (I think) IOMMU or it's remapping support are missing, only advanced or devs etc should proceed. Thought I grabbed a pic but it didn't turn out.
I go through the install until it needs to reboot, then immediately go back into bios and enable VT-d, and let it boot as normal. On the previous installations I don't think I did it before completing install, only after. It boots up fine, completes install with one error- during the templates setup I get the attached error about sys-firewall not starting up again due to libxenlight.
I login, get to desktop and open qubes manager- all without more errors.
Only dom0 is running again.
Trying to start sys-net or sys-firewall gives me the same errors as before about libxenlight failing to start the qubes, even though VT-d is turned on.
I tried unman's earlier suggestion of removing and re-adding the NIC, no luck.

I hope someone has another idea, is there a way to force vt-d on in qubes?
Some screenshots attached.

[Mike Keehan]

Ah, what a shame. It does seem as if your iommu is a problem.

Without any other ideas, the only thing I can suggest is that you try
the latest Qubes 3.2 just to allow you to try out Qubes. It doesn't
require the vt-d stuff, so might work on your box.

The other thing you can do is try googling for iommu and your cpu
together to see if others have had problems (not necessarily in
Qubes, just in general use).

Mike.

[Bryce]

On Saturday, January 19, 2019 at 6:37:47 AM UTC-5, Mike Keehan wrote:
>
> Ah, what a shame. It does seem as if your iommu is a problem.
>
> Without any other ideas, the only thing I can suggest is that you try
> the latest Qubes 3.2 just to allow you to try out Qubes. It doesn't
> require the vt-d stuff, so might work on your box.
>
> The other thing you can do is try googling for iommu and your cpu
> together to see if others have had problems (not necessarily in
> Qubes, just in general use).
>
> Mike.

Well, just to put this out there, it seems as if my CPU doesn't support SLAT and maybe that's why (I thought from intel's page it did). Installing the latest 3 stable and I don't get any errors about IOMMU like I did with 4, but just like with 4 I do have to disable vt-d in bios in order to boot to the installer.

Unfortunately as soon as I turn vt-d back on, the system goes thru post, I see qubes start to boot and then after a minute (about when I think I'd see the disk password prompt) I get a message from my monitor that the input timing is wrong, which I've never seen on several other operating systems!
Wierldy, I was able to reproduce this behavior by turning off vt-d, booting up fully and then was able to update without issues unlike in 4. Rebooted twice without issues, then turned vt-d back on and got the exact same behavior.

So I guess I just try out using 3.2 without vt-d.

I'll take a look around the users' group, but I don't see any real details on the qubes site for what the key differences are between 3 & 4 are, can anyone point me to something that will tell me how worth it (secure) that it is to use 3.2 instead of replacing hardware to run 4?

[Mike Keehan]

Hi Bryce,

Glad you got something working for you :)

This is a good description of the changes in Qubes 4.0 :-
https://www.qubes-os.org/doc/releases/4.0/release-notes/

Mike.

[Bryce]

On Wednesday, January 23, 2019 at 6:29:04 AM UTC-5, Mike Keehan wrote:
>
> Hi Bryce,
>
> Glad you got something working for you :)
>
> This is a good description of the changes in Qubes 4.0 :-
> https://www.qubes-os.org/doc/releases/4.0/release-notes/
>
> Mike.

Thanks Mike