Jump to content
TorGuard
  • 0

Ubuntu 20.04 on amd64 and arm64 - routing problem

Rate this question

chuck

Asked by chuck,

 Share Followers 2

Question

chuck

chuck 0

Posted
Posted

I've been using Wireguard on windows to connect to Torguard using the default Wireguard app for some time, everything works, so I know the problem is not with the config file or server setup.

 

I'm using the exact same config (while the windows client is down, of course) to connect from Ubuntu 20.04 and the routing doesn't seem to work. I'm out of ideas, maybe someone can help me. Same thing happens on AMD64 (VM under Hyper-V) and on ARM64 (raspberry-pi/4 with 8 gigs or RAM).

 

Here's the config: 

[Interface]
PrivateKey = ********
ListenPort = 51820
Address = 10.29.0.***/24
DNS = 1.1.1.1

[Peer]
PublicKey = ******************
AllowedIPs = 0.0.0.0/1, 128.0.0.0/1
Endpoint = 178.62.238.***:443
PersistentKeepalive = 25

I do 'wq-quick up wg0' and get this output: 

[email protected]:~/mystack$ wg-quick up wg0
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.29.0.***/24 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] resolvconf -a wg0 -m 0 -x
[#] ip -4 route add 128.0.0.0/1 dev wg0
[#] ip -4 route add 0.0.0.0/1 dev wg0
[email protected]:~/mystack$

And indeed an interface is created, and routing is set up:
(192.168.86.0/24 is google-wifi's network, 10.1.*,10.10.*, and 172.* are docker's networks) 

[email protected]:~/mystack$ ip link show wg0
194: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none
    
[email protected]:~/mystack$ ip r
0.0.0.0/1 dev wg0 scope link
default via 192.168.86.1 dev eth0 proto dhcp src 192.168.86.122 metric 100
10.1.1.0/24 dev br-4f3e279976ca proto kernel scope link src 10.1.1.1
10.10.10.0/24 dev br-9c24576b7aa1 proto kernel scope link src 10.10.10.1
10.29.0.0/24 dev wg0 proto kernel scope link src 10.29.0.***
128.0.0.0/1 dev wg0 scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.86.0/24 dev eth0 proto kernel scope link src 192.168.86.122
192.168.86.1 dev eth0 proto dhcp scope link src 192.168.86.122 metric 100

[email protected]:~/mystack$ ifconfig wg0
wg0: flags=209<UP,POINTOPOINT,RUNNING,NOARP>  mtu 1420
        inet 10.29.0.***  netmask 255.255.255.0  destination 10.29.0.***
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 1  bytes 92 (92.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 180 (180.0 B)
        TX errors 0  dropped 1378 overruns 0  carrier 0  collisions 0

[email protected]:~/mystack$

However, the routing doesn't work, ping doesn't work 

[email protected]:~/mystack$ traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max
  1   *  *  *
  2   *  *  *
^C
[email protected]:~/mystack$ ping 10.29.0.1
PING 10.29.0.1 (10.29.0.1) 56(84) bytes of data.
^C
--- 10.29.0.1 ping statistics ---
23 packets transmitted, 0 received, 100% packet loss, time 22535ms

[email protected]:~/mystack$

On a hunch I added the 'table=off' directive, to prevent Wireguard from setting up routing. And suddenly ping works!
  

[email protected]:~/mystack$ wg-quick down wg0
[#] ip link delete dev wg0
[#] resolvconf -d wg0 -f

[email protected]:~/mystack$ sudo -E vi /etc/wireguard/wg0.conf
[email protected]:~/mystack$ wg-quick up wg0
[#] ip link add wg0 type wireguard
[#] wg setconf wg0 /dev/fd/63
[#] ip -4 address add 10.29.0.***/24 dev wg0
[#] ip link set mtu 1420 up dev wg0
[#] resolvconf -a wg0 -m 0 -x

[email protected]:~/mystack$ ip link show wg0
197: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none
    
[email protected]:~/mystack$ ip r
default via 192.168.86.1 dev eth0 proto dhcp src 192.168.86.122 metric 100
10.1.1.0/24 dev br-4f3e279976ca proto kernel scope link src 10.1.1.1
10.10.10.0/24 dev br-9c24576b7aa1 proto kernel scope link src 10.10.10.1
10.29.0.0/24 dev wg0 proto kernel scope link src 10.29.0.***
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.86.0/24 dev eth0 proto kernel scope link src 192.168.86.122
192.168.86.1 dev eth0 proto dhcp scope link src 192.168.86.122 metric 100

[email protected]:~/mystack$ ifconfig wg0
wg0: flags=209<UP,POINTOPOINT,RUNNING,NOARP>  mtu 1420
        inet 10.29.0.***  netmask 255.255.255.0  destination 10.29.0.***
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 1  bytes 92 (92.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2  bytes 180 (180.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[email protected]:~/mystack$ ping 10.29.0.1
PING 10.29.0.1 (10.29.0.1) 56(84) bytes of data.
64 bytes from 10.29.0.1: icmp_seq=1 ttl=64 time=45.9 ms
64 bytes from 10.29.0.1: icmp_seq=2 ttl=64 time=45.0 ms
64 bytes from 10.29.0.1: icmp_seq=3 ttl=64 time=46.3 ms
64 bytes from 10.29.0.1: icmp_seq=4 ttl=64 time=46.6 ms
64 bytes from 10.29.0.1: icmp_seq=5 ttl=64 time=46.2 ms
64 bytes from 10.29.0.1: icmp_seq=6 ttl=64 time=47.7 ms
^C
--- 10.29.0.1 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5007ms
rtt min/avg/max/mdev = 45.018/46.280/47.720/0.808 ms
[email protected]:~/mystack$

Now I tried adding the routing of the lower half manually and got traceroute working: 

[email protected]:~/mystack$ sudo ip r add 0.0.0.0/1 dev wg0
[email protected]:~/mystack$ ip r
0.0.0.0/1 dev wg0 scope link
...................
[email protected]:~/mystack$ traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max
  1   10.29.0.1  46.497ms  45.750ms  44.777ms
  2   128.199.32.254  45.407ms  46.283ms  54.180ms
  3   138.197.250.122  46.796ms  46.541ms  45.867ms
  4   138.197.250.94  45.288ms  45.667ms  45.754ms
  5   80.249.211.140  57.569ms  59.616ms  66.315ms
  6   1.1.1.1  46.149ms  47.032ms  47.249ms
[email protected]:~/mystack$

However if I add the upper half everything breaks. 

[email protected]:~/mystack$ sudo ip r add 128.0.0.0/1 dev wg0

[email protected]:~/mystack$ ip r
0.0.0.0/1 dev wg0 scope link
...
128.0.0.0/1 dev wg0 scope link
...
[email protected]:~/mystack$ traceroute cnn.com
traceroute to cnn.com (151.101.193.67), 64 hops max
  1   *  *  *
  2   *  *  *
  3   *  *  *
  4   *  *  *
  5   *  *  *
  6   *  *


Any ideas?

 

Link to post
Share on other sites

12 answers to this question

Recommended Posts

  • 0
19807409

19807409 22

Posted
Posted

Try please just with 

AllowedIPs = 0.0.0.0/0

And report if it works. 

$ traceroute 1.1.1.1
traceroute to 1.1.1.1 (1.1.1.1), 64 hops max
  1   10.*.*.*  22,434ms  24,792ms  26,354ms 
  2   37.*.*.*  24,577ms  24,908ms  24,932ms 
  3   37.*.*.*  23,883ms  24,990ms  34,910ms 
  4   193.*.*.*  26,028ms  24,869ms  24,947ms 
  5   1.1.1.1  23,907ms  25,962ms  23,877ms

 

Link to post
Share on other sites
  • 0
chuck

chuck 0

Posted
  • Author
Posted
Quote 
AllowedIPs = 0.0.0.0/0

Of course, this was the first thing I did. Doesn't work.

 

Also, with the help of my devops friend we discovered that the problem lies in the 178.0.0.0/8 range. All the other ranges seem to work just fine. TCPdump shows no packets from the other end.

Link to post
Share on other sites
  • 0
19807409

19807409 22

Posted
Posted

This is strange, did you try other server too?

Did you also try to add 178.0.0.0/8 to AllowedIp's too?

 

Link to post
Share on other sites
  • 0
chuck

chuck 0

Posted
  • Author
Posted
Quote

Did you also try to add 178.0.0.0/8 to AllowedIp's too?

We tried 'halving' the ranges, doing 'ir r add x.x.x.x/y dev wg0' and that's how we discovered that 178.../8 doesn't work. So if anything this range needs to be excluded, not included ;) But then what's the point of VPN that doesn't cover the whole of internet?..

Link to post
Share on other sites
  • 0
19807409

19807409 22

Posted
Posted
2 hours ago, chuck said:

We tried 'halving' the ranges, doing 'ir r add x.x.x.x/y dev wg0' and that's how we discovered that 178.../8 doesn't work. So if anything this range needs to be excluded, not included ;) But then what's the point of VPN that doesn't cover the whole of internet?.

That would be right if I could reproduce your issue, however, I can't reproduce your issue on server I connect to and 

AllowedIPs = 0.0.0.0/0, ::/0

 should work but you said it does not for the server you connect to.

What does 

iptables -t nat -nvL

show you? As well, what did support reply to you?

EDIT: connected to NL server 178.62.238.* and I experience none of issues described above, here is the config: 

# TorGuard NL - WireGuard Config
[Interface]
PrivateKey = PRIVATEKEY
ListenPort = 51820
DNS = 1.1.1.1
Address = 10.*.*.*/24

[Peer]
PublicKey = PUBLICKEY
AllowedIPs = 0.0.0.0/0
Endpoint = 178.62.238.*:443
PersistentKeepalive = 25

 

ip link show wg3

16: wg3: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none

ip r

default via *.*.*.* dev wlp2s0 proto dhcp metric 600
10.*.*.*/24 dev wg3 proto kernel scope link src 10.*.*.*
169.*.*.*/16 dev wlp2s0 scope link metric 1000
172.*.*.0/16 dev docker0 proto kernel scope link src 172.*.*.1 linkdown
*.*.*.*/24 dev wlp2s0 proto kernel scope link src *.*.*.* metric 600

ifconfig wg3

wg3: flags=209<UP,POINTOPOINT,RUNNING,NOARP>  mtu 1420
        inet 10.*.*.*  netmask 255.255.255.0  destination 10.*.*.*
        unspec 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  txqueuelen 1000  (UNSPEC)
        RX packets 68868  bytes 97970252 (97.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 35578  bytes 3641904 (3.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

traceroute 1.1.1.1

traceroute to 1.1.1.1 (1.1.1.1), 64 hops max
  1   10.*.*.1  40,425ms  40,055ms  39,890ms
  2   *.*.*.*  44,988ms  39,942ms  40,185ms
  3   *.*.*.*  40,923ms  39,977ms  39,911ms
  4   *.*.*.*  38,770ms  39,971ms  50,949ms
  5   *.*.*.*  62,123ms  56,762ms  55,193ms
  6   1.1.1.1  43,194ms  39,574ms  39,480ms

ping 10.29.0.1

PING 10.29.0.1 (10.29.0.1) 56(84) Bytes Daten.
64 bytes from 10.29.0.1: icmp_seq=1 ttl=64 Zeit=48.0 ms
64 bytes from 10.29.0.1: icmp_seq=2 ttl=64 Zeit=43.4 ms
64 bytes from 10.29.0.1: icmp_seq=3 ttl=64 Zeit=64.6 ms
64 bytes from 10.29.0.1: icmp_seq=4 ttl=64 Zeit=62.9 ms

By that, it clearly seems to be issue on your network/config.

Link to post
Share on other sites
  • 0
chuck

chuck 0

Posted
  • Author
Posted
Quote

What does 

iptables -t nat -nvL

show you? As well, what did support reply to you?

 

Here's the output. Didn't ask support yet, I presume they'll say 'stick to windows if it works' :))) 

[email protected]:~# ip r
0.0.0.0/1 dev wg0 scope link
default via 192.168.86.1 dev eth0 proto dhcp src 192.168.86.122 metric 100
10.1.1.0/24 dev br-f91d4df405f5 proto kernel scope link src 10.1.1.1
10.10.10.0/24 dev br-094ce994e233 proto kernel scope link src 10.10.10.1
10.29.0.0/24 dev wg0 proto kernel scope link src 10.29.0.***
128.0.0.0/1 dev wg0 scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown
192.168.86.0/24 dev eth0 proto kernel scope link src 192.168.86.122
192.168.86.1 dev eth0 proto dhcp scope link src 192.168.86.122 metric 100

[email protected]:~# iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 30543 packets, 3432K bytes)
 pkts bytes target     prot opt in     out     source               destination        
 1027 63332 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL

Chain INPUT (policy ACCEPT 8368 packets, 2041K bytes)
 pkts bytes target     prot opt in     out     source               destination        

Chain OUTPUT (policy ACCEPT 13023 packets, 1111K bytes)
 pkts bytes target     prot opt in     out     source               destination        
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL

Chain POSTROUTING (policy ACCEPT 30348 packets, 2209K bytes)
 pkts bytes target     prot opt in     out     source               destination        
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0       
 5423  325K MASQUERADE  all  --  *      !br-f91d4df405f5  10.1.1.0/24          0.0.0.0/0
    0     0 MASQUERADE  all  --  *      !br-094ce994e233  10.10.10.0/24        0.0.0.0/0
33415 2415K ts-postrouting  all  --  *      *       0.0.0.0/0            0.0.0.0/0     
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.2             10.1.1.2             tcp dpt:80
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.2             10.1.1.2             tcp dpt:9300
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.3             10.1.1.3             tcp dpt:9000
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.4             10.1.1.4             tcp dpt:8081
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.5             10.1.1.5             tcp dpt:8083
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.8             10.1.1.8             tcp dpt:443
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.8             10.1.1.8             tcp dpt:80
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.7             10.1.1.7             tcp dpt:8096
    0     0 MASQUERADE  tcp  --  *      *       10.1.1.9             10.1.1.9             tcp dpt:80

Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination        
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0         
   67  4020 RETURN     all  --  br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0 
    0     0 RETURN     all  --  br-094ce994e233 *       0.0.0.0/0            0.0.0.0/0 
    0     0 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9400 to:10.1.1.2:80
  206 10712 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9300 to:10.1.1.2:9300
    6   312 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9000 to:10.1.1.3:9000
    6   312 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9100 to:10.1.1.4:8081
   15   780 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9500 to:10.1.1.5:8083
    0     0 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 to:10.1.1.8:443
   13   676 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:80 to:10.1.1.8:80
    0     0 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9200 to:10.1.1.7:8096
   28  1456 DNAT       tcp  --  !br-f91d4df405f5 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:9600 to:10.1.1.9:80

Chain ts-postrouting (1 references)
 pkts bytes target     prot opt in     out     source               destination        
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x40000
[email protected]:~#

 

Link to post
Share on other sites
  • 0
19807409

19807409 22

Posted
Posted
13 minutes ago, chuck said:

Here's the output. Didn't ask support yet, I presume they'll say 'stick to windows if it works' :)))

 

Thx, will take a look after the dinner. I doubt it support would be answering such nonsense, they will probably first try to find out or to point you to issue which might cause it, if, then it should be known to them. TorGuard support is very good, try it at least.

If you look up my edited post, you will see that I connect to probably same server now and traceroute as well as ping works.

I hope they do not advice to stick to windows, that is bad advice. Stick to linux if you want things to work without a need to touch them further.

Post above was tested and done with 

Linux legion 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

EDIT:

iptables -t nat -nvL

Chain PREROUTING (policy ACCEPT 1713 packets, 363K bytes)
 pkts bytes target     prot opt in     out     source               destination         
  555  149K DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
	Chain INPUT (policy ACCEPT 1699 packets, 358K bytes)
 pkts bytes target     prot opt in     out     source               destination         
	Chain OUTPUT (policy ACCEPT 62645 packets, 5832K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL
	Chain POSTROUTING (policy ACCEPT 62644 packets, 5832K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    1   168 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
	Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0

 

Link to post
Share on other sites
  • 0
chuck

chuck 0

Posted
  • Author
Posted

Support was very helpful. Turns out, for some reason Wireguard client for Ubuntu does not add explicit routing for the endpoint. 

Endpoint = 178.62.238.***:443


After I added the route manually - everything worked.

In hindsight I should've guessed where does 178.*** in my tests come from 🤦‍♂️

Link to post
Share on other sites
  • 0
Gecko74

Gecko74 0

Posted
Posted
11 hours ago, chuck said:

Support was very helpful. Turns out, for some reason Wireguard client for Ubuntu does not add explicit routing for the endpoint. 

Endpoint = 178.62.238.***:443


After I added the route manually - everything worked.

In hindsight I should've guessed where does 178.*** in my tests come from 🤦‍♂️

Could you elaborate on that? 
I seem to be having the exact same issue. It was running smoothly until sometime yesterday when all traffic just stopped.

Link to post
Share on other sites
  • 0
chuck

chuck 0

Posted
  • Author
Posted
7 minutes ago, Gecko74 said:

Could you elaborate on that? 


It's actually in the docs, if you know what to look for, under 'known issues':

https://wiki.archlinux.org/index.php/WireGuard#Known_issues

 

What happens is that the 128.0.0.0/1 mask includes the endpoint, so it can't reach TorGuard serverrs. For some reason wireguard/ubuntu does not realize this and does not make an exception. So the easiest way is to add 'PostUp = ip r a <endpoint> via <your normal ip>' etc.

Link to post
Share on other sites
  • 0
Gecko74

Gecko74 0

Posted
Posted

Thank you!

Just added the endpoint as a PostUp route in wg0.conf, and it works flawlessly.

Can't help but wonder why it's needed all of a sudden.

Link to post
Share on other sites
  • 0
Gecko74

Gecko74 0

Posted
Posted

Odd... Did the change, and everything was working fine.

Did apt update && apt -y dist-upgrade, and noticed a wireguard update was installed. Removed the PostUp setting and rebooted. It still works.

It looks to me like something may have been updated on the Torguard side of things and I just needed to update my WG client.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
 Share
Followers 2
Go to question listing
×
×
  • Create New...