Re: VPN Disconnects when Qubes goes to sleep (and does not reconnect when coming out of sleep)?

[Michael Siepmann]

On Jun 12, 2017, Andrew Morgan wrote:

Did you follow the "Set up a ProxyVM as a VPN gateway using iptables and
CLI scripts" section of the Qubes VPN docs
(https://www.qubes-os.org/doc/vpn/ )?

If so you should be good just to execute the `/rw/config/rc.local` file
on your VPN VM after every suspend either manually, through a keyboard
shortcut (which I do personally with the following command):

qvm-run -i root sys-vpn "/rw/config/rc.local"

I followed the "Set up a ProxyVM as a VPN gateway using iptables and CLI scripts" instructions but for me executing "/rw/config/rc.local" doesn't make it work again.

I've also tried commenting out or deleting "persist tun" from my OpenVPN config file, as Chris Laprise as suggested in the thread "is vpn made manually, not supposed to restart after suspend?" on May 21 but that isn't helping either.

My current workaround is a script I wrote in dom0 that first does "qvm-prefs VMname -s netvm none" for all the VMs I normally have running that use sys-vpn (my ProxyVM VPN gateway), then shuts sys-vpn down, waits 10 seconds, starts sys-vpn, then does "qvm-prefs VMname -s netvm sys-vpn" for all those VMs.

Any ideas what could be going on so that neither executing /rw/config/rc.local nor commenting out "persist tun" works in my case?

Thanks!

Michael Siepmann

--

Michael Siepmann, Ph.D.
The Tech Design Psychologist™
Shaping technology to help people flourish™
303-835-0501   TechDesignPsych.com   OpenPGP: 6D65A4F7
 

[Chris Laprise]

On 11/30/2017 11:44 PM, Michael Siepmann wrote:
>
> On Jun 12, 2017, Andrew Morgan wrote:
>
>> Did you follow the "Set up a ProxyVM as a VPN gateway using iptables and
>> CLI scripts" section of the Qubes VPN docs
>> (https://www.qubes-os.org/doc/vpn/
>> <https://www.qubes-os.org/doc/vpn/> )?
>>
>> If so you should be good just to execute the `/rw/config/rc.local` file
>> on your VPN VM after every suspend either manually, through a keyboard
>> shortcut (which I do personally with the following command):
>>
>> qvm-run -i root sys-vpn "/rw/config/rc.local"
>
> I followed the "Set up a ProxyVM as a VPN gateway using iptables and
> CLI scripts" instructions but for me executing "/rw/config/rc.local"
> doesn't make it work again.
>
> I've also tried commenting out or deleting "persist tun" from my
> OpenVPN config file, as Chris Laprise as suggested in the thread "is
> vpn made manually, not supposed to restart after suspend?" on May 21
> but that isn't helping either.
>
> My current workaround is a script I wrote in dom0 that first does
> "qvm-prefs VMname -s netvm none" for all the VMs I normally have
> running that use sys-vpn (my ProxyVM VPN gateway), then shuts sys-vpn
> down, waits 10 seconds, starts sys-vpn, then does "qvm-prefs VMname -s
> netvm sys-vpn" for all those VMs.
>
> Any ideas what could be going on so that neither executing
> /rw/config/rc.local nor commenting out "persist tun" works in my case?
>

I have a couple ideas as to workarounds. Instead of re-starting sys-vpn,
you could:

  qvm-run -u root sys-vpn 'pkill openvpn'
  qvm-run -u root sys-vpn 'sh /rw/config/rc.local'

...before you re-enable the netvm prefs.

Also, one thing that changing the netvm prefs does is to trigger
qubes-firewall-user-script to run again. You might compare the state of
iptables before and after your workaround to see if something went
missing after waking from sleep. If that's the case, you could just
trigger the script as a third command added to the above.

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[Michael Siepmann]

Thanks! I tried those commands and they don't get it working again. I still have to shut it down and restart it. I also checked the iptables and that does not seem not to be the problem. However, I've found out that after a brief suspend the VPN may continue working, but in cases when it stops working, the process ends and can't be restarted. In the following the first ps command was just after resuming, and the second a few seconds later, after I'd seen the "VPN is down" notification:

[user@sys-vpn ~]$ sudo sg qvpn -c "ps -leaf | grep openvpn | grep -v grep"
5 S root      1093     1  0  80   0 - 16371 poll_s 14:33 ?        00:00:42 openvpn --cd /rw/config/vpn/ --config openvpn-client.ovpn --daemon
[user@sys-vpn ~]$ sudo sg qvpn -c "ps -leaf | grep openvpn | grep -v grep"
[user@sys-vpn ~]$ 
[user@sys-vpn ~]$ sudo sh /rw/config/rc.local
[user@sys-vpn ~]$ sudo sg qvpn -c "ps -leaf | grep openvpn | grep -v grep"
[user@sys-vpn ~]$ 

I also tried "pkill openvpn" when it is working, and I can't restart it after that either:

[user@sys-vpn ~]$ sudo sg qvpn -c "ps -leaf | grep openvpn | grep -v grep"
5 S root      1134     1  0  80   0 - 16371 poll_s 21:26 ?        00:00:00 openvpn --cd /rw/config/vpn/ --config openvpn-client.ovpn --daemon
[user@sys-vpn ~]$ sudo sg qvpn -c "pkill openvpn"
[user@sys-vpn ~]$ sudo sg qvpn -c "ps -leaf | grep openvpn | grep -v grep"
[user@sys-vpn ~]$ sudo sh /rw/config/rc.local
[user@sys-vpn ~]$ sudo sg qvpn -c "ps -leaf | grep openvpn | grep -v grep"
[user@sys-vpn ~]$ 

Any ideas why this might be happening?

[Chris Laprise]

Looking at openvpn entries in 'journalctl' can give you a better idea.

I've seen instances where openvpn versions starting with 2.4 have this
bad reaction to disconnection (which is what sleep/wake is in this
case); with openvpn 2.3 you could count on it to keep
going/re-connecting. But there may also be an issue with the way
Qubes/Xen are handling the virtual interfaces between VMs; the symptoms
remind me of basic networking problems many of us have experienced with
prior Qubes releases, where only VM restarts would re-build inter-VM
links correctly.

But if there isn't a basic networking problem, moving to a service-based
config will be more robust and should keep openvpn running. One way to
do this is to have your rc.local script start openvpn with systemd-run
(and the right options), but I've already created a project that uses a
full systemd config to manage VPN processes...

https://github.com/tasket/Qubes-vpn-support

Setup is much easier than the vpn doc, though it currently only works
with Qubes 3.2 which I'm guessing you're using. The usual 'systemctl
start/stop/status' commands give you control over the
qubes-vpn-handler.service that manages openvpn.

[Michael Siepmann]

Thank you! So far this seems to be working fine, automatically
reconnecting after resume. Any chance of getting this approach mentioned
on https://www.qubes-os.org/doc/vpn ?

[Chris Laprise]

Great! I think it could be linked in the revised doc once the Qubes R4.0
issues are worked out.

[Michael Siepmann]

Recently the "LINK IS UP" notification has been staying visible until I
manually close it.  At first it wasn't doing that. Are you seeing that
too? Could a recent update have caused this?

[Chris Laprise]

On 12/17/2017 09:45 AM, Michael Siepmann wrote:

> Recently the "LINK IS UP" notification has been staying visible until I
> manually close it.  At first it wasn't doing that. Are you seeing that
> too? Could a recent update have caused this?

Yes, I'm seeing it again on fedora-26 but not debian-9. This has been an
off-and-on issue with notify-send over the years.

[Michael Siepmann]

 On 12/17/2017 08:33 AM, Chris Laprise wrote:
> On 12/17/2017 09:45 AM, Michael Siepmann wrote:
>
>> Recently the "LINK IS UP" notification has been staying visible until I
>> manually close it.  At first it wasn't doing that. Are you seeing that
>> too? Could a recent update have caused this?
>
> Yes, I'm seeing it again on fedora-26 but not debian-9. This has been
> an off-and-on issue with notify-send over the years.
>

I tried changing the template for my sys-vpn to debian-9 but the VPN
service didn't work. However, when I switched it back to fedora-25 (I'm
not using 26 yet) the persistent notification issue was fixed! (When the
issue was happening, btw, it was only with the "up" notifications, not
the "down" ones.)