suspend to ram, r8169, networking in sys-net not working after resume

[arga...@tutanota.com]

Hey guys, sorry about sending confidential email earlier, didn't realize tutanota would encrypt it like that.

My issue:

On qubes version r4.0 after resuming from suspend networking isn't working. On qubes r3.2 this wasn't an issue.


After resuming from suspend to ram, networking on sys-net isn't working:
$ ip addr
6: ens5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000
    link/ether 4c:cc:6a:30:f5:90 brd ff:ff:ff:ff:ff:ff

Usually it would be:
$ ip addr
2: ens5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 4c:cc:6a:30:f5:90 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.11/24 brd 192.168.5.255 scope global dynamic ens5
       valid_lft 86362sec preferred_lft 86362sec
    inet6 fe80::e5b7:4276:7bc8:f799/64 scope link
       valid_lft forever preferred_lft forever

$ lspci
00:05.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)

---------------------------------------------------------------------------
This is the output of dmesg after resuming from suspend:

[  182.294472] audit: type=1106 audit(1524300197.335:110): pid=2337 uid=0 auid=1000 ses=1 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[  182.294519] audit: type=1104 audit(1524300197.335:111): pid=2337 uid=0 auid=1000 ses=1 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[  250.187171] audit: type=1100 audit(1524300265.223:112): pid=2361 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:authentication grantors=pam_rootok acct="root" exe="/usr/lib/qubes/qrexec-agent" hostname=? addr=? terminal=? res=success'
[  250.187533] audit: type=1103 audit(1524300265.223:113): pid=2361 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:setcred grantors=pam_rootok acct="root" exe="/usr/lib/qubes/qrexec-agent" hostname=? addr=? terminal=? res=success'
[  250.209409] audit: type=1101 audit(1524300265.245:114): pid=2362 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:accounting grantors=pam_unix acct="root" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  250.209448] audit: type=1006 audit(1524300265.245:115): pid=2362 uid=0 old-auid=4294967295 auid=0 tty=(none) old-ses=4294967295 ses=2 res=1
[  250.209470] audit: type=1105 audit(1524300265.245:116): pid=2362 uid=0 auid=0 ses=2 msg='op=PAM:session_open grantors=pam_selinux,pam_selinux,pam_loginuid,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  250.251619] audit: type=1130 audit(1524300265.287:117): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=user@0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  250.254009] audit: type=1105 audit(1524300265.289:118): pid=2361 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_systemd,pam_unix,pam_lastlog acct="root" exe="/usr/lib/qubes/qrexec-agent" hostname=? addr=? terminal=? res=success'
[  250.276190] audit: type=1106 audit(1524300265.311:119): pid=2361 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_systemd,pam_unix,pam_lastlog acct="root" exe="/usr/lib/qubes/qrexec-agent" hostname=? addr=? terminal=? res=success'
[  250.276237] audit: type=1104 audit(1524300265.312:120): pid=2361 uid=0 auid=4294967295 ses=4294967295 msg='op=PAM:setcred grantors=pam_rootok acct="root" exe="/usr/lib/qubes/qrexec-agent" hostname=? addr=? terminal=? res=success'
[  250.297413] audit: type=1131 audit(1524300265.332:121): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=user@0 comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  250.829214] Freezing user space processes ... (elapsed 0.000 seconds) done.
[  250.830138] OOM killer disabled.
[  250.830145] Freezing remaining freezable tasks ... (elapsed 0.000 seconds) done.
[  250.833225] suspending xenstore...
[  256.412047] xen:events: Xen HVM callback vector for event delivery is enabled
[  256.412836] Xen Platform PCI: I/O protocol version 1
[  256.413015] xen:grant_table: Grant tables using version 1 layout
[  256.458068] OOM killer enabled.
[  256.458078] Restarting tasks ... done.
[  256.567633] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[  256.567649] Warning! ehci_hcd should always be loaded before uhci_hcd and ohci_hcd, not after
[  256.569187] ehci-pci: EHCI PCI platform driver
<b>[  256.574784] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[  256.583753] r8169 0000:00:05.0 eth0: RTL8169 at 0xffffb2b0802a5000, 4c:cc:6a:30:f5:90, XID 14100800 IRQ 77
[  256.583777] r8169 0000:00:05.0 eth0: jumbo features [frames: 7152 bytes, tx checksumming: ok]
[  256.640588] r8169 0000:00:05.0 ens5: renamed from eth0
[  256.652242] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready
[  257.003311] r8169 0000:00:05.0 ens5: rtl_phy_reset_cond == 1 (loop: 100, delay: 1).
[  257.003442] r8169: ens5: unknown chipset (mac_version = 0).
[  257.003602] r8169 0000:00:05.0 ens5: link down
[  257.003714] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready</b>
[  266.612625] kauditd_printk_skb: 31 callbacks suppressed
[  266.612628] audit: type=1131 audit(1524300294.032:153): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=NetworkManager-dispatcher comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  274.723620] audit: type=1130 audit(1524300302.142:154): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=packagekit comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
[  277.568548] audit: type=1123 audit(1524300304.987:155): pid=2697 uid=1000 auid=1000 ses=1 msg='cwd="/home/user" cmd="dmesg" terminal=pts/0 res=success'
[  277.568655] audit: type=1110 audit(1524300304.987:156): pid=2697 uid=0 auid=1000 ses=1 msg='op=PAM:setcred grantors=pam_env,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'
[  277.572833] audit: type=1105 audit(1524300304.990:157): pid=2697 uid=0 auid=1000 ses=1 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'

As you can see, I get:
[  257.003602] r8169 0000:00:05.0 ens5: link down
[  257.003714] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready

---------------------------------------------------------------------------
After resuming I've tried:
[user@sys-net ~]$ sudo rmmod r8169
[user@sys-net ~]$ sudo rmmod mii
[user@sys-net ~]$ sudo modprobe mii
[user@sys-net ~]$ sudo modprobe r8169

but dmesg gives the same error:
[user@sys-net ~]$ sudo dmesg
[  521.651333] r8169 0000:00:05.0 ens5: renamed from eth0
[  521.662074] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready
[  521.966877] r8169 0000:00:05.0 ens5: rtl_phy_reset_cond == 1 (loop: 100, delay: 1).
[  521.967049] r8169: ens5: unknown chipset (mac_version = 0).
[  521.967201] r8169 0000:00:05.0 ens5: link down
[  521.967344] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready


---------------------------------------------------------------------------
Essentially because of this issue I cannot use suspend because after resuming I don't have networking. To fix it I have to reboot sys-net which also means rebooting all of the VMs that use networking or reboot the whole PC.

There was no issue with suspend when I was using r3.2, this is after I've upgraded to r4.0

[Ivan Mitev]

Hi,

did you try to `rmmod` *before* suspending ?

> but dmesg gives the same error:
> [user@sys-net ~]$ sudo dmesg
> [  521.651333] r8169 0000:00:05.0 ens5: renamed from eth0
> [  521.662074] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready
> [  521.966877] r8169 0000:00:05.0 ens5: rtl_phy_reset_cond == 1 (loop: 100, delay: 1).
> [  521.967049] r8169: ens5: unknown chipset (mac_version = 0).
> [  521.967201] r8169 0000:00:05.0 ens5: link down
> [  521.967344] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready

maybe it takes a bit of time to re-initialize your network card. Do you
get the same error when you manually "up" the device ? eg.:

ip l set ens5 up

>
>
> ---------------------------------------------------------------------------
> Essentially because of this issue I cannot use suspend because after resuming I don't have networking. To fix it I have to reboot sys-net which also means rebooting all of the VMs that use networking or reboot the whole PC.

In 4.0 you don't need to reboot (or reattach) all the VMs anymore; if
you kill sys-net and boot it, sys-firewall *should* automatically
reconnect to sys-net (emphasize on "should": I found out it wasn't
always the case, but that was during rcX time, maybe it works reliably now).

[arga...@tutanota.com]

--
Securely sent with Tutanota. Claim your encrypted mailbox today!
https://tutanota.com

21. Apr 2018 13:16 by iv...@maa.bz:

Tried this now, still the same situation.

 

but dmesg gives the same error:
[user@sys-net ~]$ sudo dmesg
[  521.651333] r8169 0000:00:05.0 ens5: renamed from eth0
[  521.662074] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready
[  521.966877] r8169 0000:00:05.0 ens5: rtl_phy_reset_cond == 1 (loop: 100, delay: 1).
[  521.967049] r8169: ens5: unknown chipset (mac_version = 0).
[  521.967201] r8169 0000:00:05.0 ens5: link down
[  521.967344] IPv6: ADDRCONF(NETDEV_UP): ens5: link is not ready

maybe it takes a bit of time to re-initialize your network card. Do you
get the same error when you manually "up" the device ? eg.:

ip l set ens5 up

runing "up" didn't change anything, output of `ip addr` is still the same. It says "NO-CARRIER" which would suggest the cable is disconnected, but that's not it.

 

---------------------------------------------------------------------------
Essentially because of this issue I cannot use suspend because after resuming I don't have networking. To fix it I have to reboot sys-net which also means rebooting all of the VMs that use networking or reboot the whole PC.

In 4.0 you don't need to reboot (or reattach) all the VMs anymore; if
you kill sys-net and boot it, sys-firewall *should* automatically
reconnect to sys-net (emphasize on "should": I found out it wasn't
always the case, but that was during rcX time, maybe it works reliably now).

I've tried as you suggested but when I kill and boot sys-net, sys-firewall gets unresponsive. I tried running `qvm-run -p sys-firewall 'gnome-terminal'` from dom0, but I get "vchan connection timeout" after a few seconds.

 

There was no issue with suspend when I was using r3.2, this is after I've upgraded to r4.0

--
You received this message because you are subscribed to the Google Groups "qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/5ce7d214-537f-c18d-73b0-c62820f58b45%40maa.bz.
For more options, visit https://groups.google.com/d/optout.

[Mike Keehan]

On Sat, 21 Apr 2018 14:20:02 +0200 (CEST)
<arga...@tutanota.com> wrote:

> >> My issue:
> >>
> >> On qubes version r4.0 after resuming from suspend networking isn't
> >> working. On qubes r3.2 this wasn't an issue.
> >>
> >>
> >> After resuming from suspend to ram, networking on sys-net isn't
> >> working: $ ip addr
> >> 6: ens5: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc
> >> fq_codel state DOWN group default qlen 1000 link/ether
> >> 4c:cc:6a:30:f5:90 brd ff:ff:ff:ff:ff:ff
> >>

...snip...

> >> ---------------------------------------------------------------------------
> >> After resuming I've tried:
> >> [user@sys-net ~]$ sudo rmmod r8169
> >> [user@sys-net ~]$ sudo rmmod mii
> >> [user@sys-net ~]$ sudo modprobe mii
> >> [user@sys-net ~]$ sudo modprobe r8169
> >
> > did you try to `rmmod` *before* suspending ?
>

Try adding the module names to the sys-net file
/etc/qubes-suspend-module-blacklist, and Qubes should handle them
for you.

Mike.

[cooloutac]

I tried that for my ethernet module but it didn't help.

[gdr...@gmail.com]

Hi,

In 4.0, I have the same issue.

Everything worked well. I added internal memory. After rebooting my computer, the network connection has been disabled. Cable is properly connected.

$ lspci | grep 8139
00:05.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI EXpress Gigabit Ethernet Controller (rev 06)

It isn't a hardware problem because Ethernet card (same product) has been changed (same issue two weeks ago).

$ ifconfig -a

ens5: flags=4098<BROADCAST,MULTICAST> mtu 1500
ether 00:00:00:00:00:00 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1000 (Local Loopback)
RX packets 3608 bytes 292280 (285.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 3608 bytes 292280 (285.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

$ sudo dmesg | grep 8139
[ 1.614444] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
[ 1.629483] r8169 0000:00:05.0 eth0: RTL8168e/8111e at 0xffff9becc019d000, 00:00:00:00:00:00, XID 0c200000 IRQ 71
[ 1.629507] r8169 0000:00:05.0 eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko]
[ 1.720634] r8169 0000:00:05.0 ens5: renamed from eth0

Help !! how can I fix it ?

Thank you so much.

Best regards.

[Alex]

On 9/10/18 10:34 PM, gdr...@gmail.com wrote:
>>>>>> My issue:
>>>>>>
>>>>>> On qubes version r4.0 after resuming from suspend networking isn't
>>>>>> working. On qubes r3.2 this wasn't an issue.
>>>>>>

>>>>>>[...]

>
> Hi,
>
> In 4.0, I have the same issue.

> [...]

>
> Help !! how can I fix it ?
>

This also happens to me; I'm using realtek ethernet adapter too. With
R3.2 this happened rarely, but with R4.0 this happens nearly every time
the computer resumes from sleep.

As of now, I just issue a direct "shutdown now" to the sys-net, and then
just restart the VM. As soon as network is available again, all the
other Qubes reach it too - in R3.2 I had to shut them ALL down and then
bring them up in sequence, with R4.0 I can just powercycle sys-net and
it somehow works.

Still, it's a bit of a problem...

--
Alex

[GDRUB]

Hi,

I tried that but it didn't help.

I replaced my network card TP-Link TG-3468 with a StarTech.com (port PCI) card. Now, it seems to work well.