How do I attach two virtualized NICs to one HVM?

[Charlie]

Hi.

I'm wanting to run OPNsense on Qubes along with some other VMs. I have a working OPNsense HVM, but to actually be useful OPNsense needs two NIC's (a LAN and WAN). I can't figure out a comfortable way to do such on Qubes though.

Extra info:
OPNsense is, itself, a firewall, so I don't think I'll need the Qubes provided firewall VM. I would, however, like to have the netcode isolation that a net VM provides. If that proves impossible, then I'd like to know how to attach the two NICs to the OPNsense HVM without making them inaccessible from the rest of Qubes OS. I'm mostly trying to do this for the captive portal feature set OPNsense provides, so if anyone has an easier way to do that on Qubes, I would accept that as an answer too.

Thanks
-Charlie

[Unman]

Hi Charlie,

I was going to say that I didn't understand your problem, but I think I
do - you want to use the HVM as a netVM, and attach qubes downstream to
it.

You obviously know how to attach NICs to the HVM, and you've discovered
that you cant set up an HVM as a netVM.
What you could try is to set the networking by hand (or script it) using
xen tools.
To attach qube1, set the netVM to none, and then try using xl:
something like
"xl network-attach qube1 script=/etc/xen/scripts/vif-route-qubes
backend=HVM" should do it.
You'll have to configure the interfaces by hand, but that shouldn't be
an issue.

unman

[charles.l...@gmail.com]

On Tuesday, January 31, 2017 at 3:11:33 PM UTC-8, Unman wrote:
> Hi Charlie,
>

Hello hello.

> I was going to say that I didn't understand your problem, but I think I
> do - you want to use the HVM as a netVM, and attach qubes downstream to
> it.

Not quite sure what you mean by this. Are you thinking I want to use OPNsense as a replacement for Qubes' own firewall VM? If so, that's not what I was thinking, though I'm not opposed to doing it that way. Actually, that almost sounds better.

No, close though. I'm actually trying to run the HVM parallel to the other VM's. That is, have my isolated OPNsense HVM act as NAT/firewall between two NIC's, say eth0(WAN) and eth1(LAN), but also give any other VMs concurrent access to eth0(WAN). With this config, OPNsense would only be intended to serve an isolated subnet my captive portal for a WiFi hotspot.

I'm *TOTALLY NOT* fixing to sell WiFi to my neighbors, BTW, as that would likely violate my ISP's ToS... (¬‿¬)

hummm ...I should make a diagram for what I'm doing or something, as the above isn't much better of an explanation. Or maybe I can explain it better if I come at it from a different direction.

...

OK, let's assume I had three NIC's and two PC's.

The first PC runs Qubes OS, has some AP VMs, and only has one NIC for its WAN. For simplification, we are going to ignore virtualized machines and treat this as one OS.

The second PC is running OPNsense OS, and has two NIC's. The first NIC is connected to the WAN, the second connected to an internal LAN.

Now imagine I merged these two PCs together into one.

I now have two OS's running side by side in one PC with three NICs. Two of those NICs would really be doing the same thing, connecting to the WAN. So I could also merge the two WAN NICs.

Now I have two OS's in one PC with just two NICs. The Qubes OS with one NIC as its WAN, and the OPNsense OS with the same WAN NIC, and another as a LAN NIC.

This is about what I'm trying to do on Qubes. Does that make sense?

>
> You obviously know how to attach NICs to the HVM, and you've discovered
> that you cant set up an HVM as a netVM.

umm, maybe, maybe not. (^_^)

I can attach NIC's to the HVM by PCI passthrough, then install it on the HVM's OS. If I do that though, the NIC is no longer accessible to the rest of the Qubes system. That would be fine for the LAN NIC, I suppose, but that's not going to work for the WAN NIC since Qubes still needs WAN access.

Just typing this out is starting to give me new ideas though.

I'm thinking maybe I could pass the LAN NIC up to the HVM with PCI passthrough, but then use the Qubes firewall vif for the WAN NIC.

> What you could try is to set the networking by hand (or script it) using
> xen tools.
> To attach qube1, set the netVM to none, and then try using xl:
> something like
> "xl network-attach qube1 script=/etc/xen/scripts/vif-route-qubes
> backend=HVM" should do it.
> You'll have to configure the interfaces by hand, but that shouldn't be
> an issue.
>
> unman

Cool, thanks. I'm checking out the xl man page now. There's a lot there, looks like just the info I need if nothing else works though. Particularly the virtual interface stuff, as you alluded to. I'll post back whatever I figure out.

[charles.l...@gmail.com]

Ok, so, after much messing around with it I still couldn't seem to get anything to work. It was baffling, I was pretty sure I was doing everything right but no dice.

... turns out that both of my add in NICs are broken, both in completely unrelated ways :( One just doesn't install correctly, it's seen as an unknown PCI device. The other installs right, but has some layer 1 problem. (can't detect a carrier, as if no cable was plugged in)

So, until I get a new NIC, I'm probably going to have to put this off for later.

Anyway, thanks for your help. I'll probably start a new thread if I'm still having problems with a new NIC.