cjdns

[justus...@gmail.com]

I have a strange networking issue when trying to use cjdns in a Fedora app vm.

The cjdns daemon connects to peers and creates an ipv6 overlay network which is accessible via a tun device (tun0). I know this is working because I'm able to use ping6 to contact a host elsewhere in the overlay network... almost.

By using tcpdump on tun0, I can verify that pings are going out and responses are being successfully received:

13:48:21.225299 IP6 (flowlabel 0x3bd7f, hlim 64, next-header ICMPv6 (58) payload length: 64) work > fc10:a664:d429:edec:b105:8fcc:a742:b713: [icmp6 sum ok] ICMP6, echo request, seq 15
13:48:21.288088 IP6 (hlim 42, next-header ICMPv6 (58) payload length: 64) fc10:a664:d429:edec:b105:8fcc:a742:b713 > work: [icmp6 sum ok] ICMP6, echo reply, seq 15
13:48:22.249223 IP6 (flowlabel 0x3bd7f, hlim 64, next-header ICMPv6 (58) payload length: 64) work > fc10:a664:d429:edec:b105:8fcc:a742:b713: [icmp6 sum ok] ICMP6, echo request, seq 16
13:48:22.305391 IP6 (hlim 42, next-header ICMPv6 (58) payload length: 64) fc10:a664:d429:edec:b105:8fcc:a742:b713 > work: [icmp6 sum ok] ICMP6, echo reply, seq 16
13:48:23.273320 IP6 (flowlabel 0x3bd7f, hlim 64, next-header ICMPv6 (58) payload length: 64) work > fc10:a664:d429:edec:b105:8fcc:a742:b713: [icmp6 sum ok] ICMP6, echo request, seq 17
13:48:23.351557 IP6 (hlim 42, next-header ICMPv6 (58) payload length: 64) fc10:a664:d429:edec:b105:8fcc:a742:b713 > work: [icmp6 sum ok] ICMP6, echo reply, seq 17

"work" is the appVM and fc10:a664:d429:edec:b105:8fcc:a742:b713 is the remote host.

I wouldn't be receiving those echo reply packets unless cjdns was working, so there's nothing wrong there.

However, the ping6 command never receives the replies:

PING seed.cjdns.stashcrypto.net(fc10:a664:d429:edec:b105:8fcc:a742:b713 (fc10:a664:d429:edec:b105:8fcc:a742:b713)) 56 data bytes
^C
--- seed.cjdns.stashcrypto.net ping statistics ---
17 packets transmitted, 0 received, 100% packet loss, time 16386ms

What would cause those reply packets to get lost between the tun0 device and the ping6 process?

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Check ip6tables - by default all IPv6 input is blocked on Qubes.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYSIzcAAoJENuP0xzK19csK9QH/icOVRhiUpKBwx2Agh/68wKx
xwmZv2E7j9ZGYdiGIGg5CdbRLzxi4dxjRzmLmYLWr5O2uWODBkegoFcK+rYLgVv/
N68FofRkrLSQQZ2P+GgL+Fb5lCA7tY5b2RQJS7Y0eMQe0/igXrRlwlclPYyBSy18
fqIYZQCY3esryy892sfeD0S7/UsYbBNuaFl3nrdAnf3gPNHUBUPpinTKMf8w0Svd
v1+quGUqrsNlYN/Oi4K8aAX1tUif4pnYdT9XvRERS72BHBP/crqjnpFelaSK9JY6
l0uGI6jTEil8o3UEoBoHQT7g8bkEPJm1TFLJOB8cGdDz1kHzDzTI7L6wS/ySFhg=
=EHQB
-----END PGP SIGNATURE-----

[justus...@gmail.com]

> Check ip6tables - by default all IPv6 input is blocked on Qubes.

Thanks.

I ended up solving the problem with an ExecStartPost line in cjdns.service.

For the benefit of anyone who searches this thread, this is the cjdns.service I use to make sure you can have a persistent config:

[Unit]
Description=cjdns: routing engine designed for security, scalability, speed and ease of use
Wants=network.target
After=network.target cjdns-loadmodules.service
Requires=cjdns-loadmodules.service

[Service]
ProtectHome=true
ProtectSystem=true
SyslogIdentifier=cjdroute
CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_RAW CAP_SETUID CAP_SETGID CAP_SYS_CHROOT CAP_AUDIT_CONTROL
ExecStartPre=/bin/sh -ec "if ! test -s /rw/config/cjdroute.conf; \
then umask 077; \
/usr/sbin/cjdroute --genconf | cat > /rw/config/cjdroute.conf; \
echo 'WARNING: A new /rw/config/cjdroute.conf file has been generated.'; \
fi"
ExecStart=/bin/sh -c "exec /usr/sbin/cjdroute --nobg < /rw/config/cjdroute.conf"
ExecStartPost=/usr/sbin/ip6tables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
Restart=always

[Install]
WantedBy=multi-user.target
Also=cjdns-resume.service

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I think this can be improved (and simplified):

1. Use bind-dirs[1] (works also for files) to make config persistent;
this require to have _some_ config file in the template (can be empty).

2. Use systemd drop-in to add ExecStartPost (simply create
/etc/systemd/system/cjdns.service.d/50_user.conf with just [Service] and
ExecStartPost=... there).

2a. Or alternatively - place ip6tables command in /rw/config/rc.local




[1] https://www.qubes-os.org/doc/bind-dirs/

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJYSJmfAAoJENuP0xzK19csm2gH/0OWxru9rF6BgHo/iXm26GBo
cplVD/t9OtpJiJAjJqzVrgniwurF7Oz5AnXtPhTtabFenekEBnirFDdkBycg+nZE
yGlFAnoTiaIHuRIGhDHUNJIbi26qgYOhuWzJQm0MiUblQrnv+sjmavXTaoBouxBr
xTPkxX3e6g8K18AYJWX1yNGf8QcFOYhr5ZzXTnQ3aroKOud5xqtZk3JIzzLExRTU
FWn7H7/Ah9Zd6BB/X3z9PS99B54WmGyoUj12YiQx2aTQhQ8dLGQK061n/TURNc2E
ZepnaXpyEUaKWOKAM+hS2csyYPGSc9hbjVXPRg6jhj/8Tu1HwpbQuuH2QUU982w=
=DMXf
-----END PGP SIGNATURE-----

[justus...@gmail.com]

On Wednesday, December 7, 2016 at 5:22:11 PM UTC-6, Marek Marczykowski-Górecki wrote:
> I think this can be improved (and simplified):
>
> 1. Use bind-dirs[1] (works also for files) to make config persistent;
> this require to have _some_ config file in the template (can be empty).

I didn't know about bind-dirs. I'm not sure if this will work because cjdns needs to generate its keys and put them in the config file, and it only does that if there's no existing file.

> 2. Use systemd drop-in to add ExecStartPost (simply create
> /etc/systemd/system/cjdns.service.d/50_user.conf with just [Service] and
> ExecStartPost=... there).

I didn't go with a drop in because I had to change the provided ExecStart and ExecStartPre lines to change the config file location, and if the changes were going to be that extensive anyway it seemed cleaner to just replace the entire file.