Ask
Reply
Solution
18-06-2020 05:06 PM - edited 19-06-2020 12:50 PM
Anyone else seeing a routing blackhole like this:
$ tracepath twitter.com 1?: [LOCALHOST] pmtu 1500 1: RT-AC86U-60B0.local 6.848ms 1: RT-AC86U-60B0.local 2.058ms 2: RT-AC86U-60B0.local 2.252ms pmtu 1492 2: no reply 3: 63.130.127.213 8.214ms asymm 4 4: 10.200.18.45 12.856ms 5: 63.130.104.198 11.914ms asymm 6 6: 63.130.127.213 10.504ms asymm 4 7: 10.200.18.45 10.930ms asymm 4 8: 63.130.104.198 11.497ms asymm 6 9: 63.130.127.213 12.669ms asymm 4 10: 10.200.18.45 12.699ms asymm 4 11: 63.130.104.198 15.672ms asymm 6 12: 63.130.127.213 14.249ms asymm 4 13: 10.200.18.45 14.565ms asymm 4 14: 63.130.104.198 15.385ms asymm 6 15: 63.130.127.213 16.622ms asymm 4 16: 10.200.18.45 14.558ms asymm 4 17: 63.130.104.198 17.474ms asymm 6 18: 63.130.127.213 18.652ms asymm 4 19: 10.200.18.45 17.761ms asymm 4 20: 63.130.104.198 18.485ms asymm 6 21: 63.130.127.213 17.823ms asymm 4 22: 10.200.18.45 19.614ms asymm 4 23: 63.130.104.198 20.963ms asymm 6 24: 63.130.127.213 21.040ms asymm 4 25: 10.200.18.45 20.610ms asymm 4 26: 63.130.104.198 22.114ms asymm 6 27: 63.130.127.213 21.395ms asymm 4 28: 10.200.18.45 22.728ms asymm 4 29: 63.130.104.198 23.665ms asymm 6 30: 63.130.127.213 25.953ms asymm 4 Too many hops: pmtu 1492 Resume: pmtu 1492
18-06-2020 07:26 PM - edited 18-06-2020 07:31 PM
Nope, currently here I get:
traceroute twitter.com
traceroute to twitter.com (104.244.42.1), 30 hops max, 60 byte packets
1 rtac86u.broadband (192.168.1.1) 0.163 ms 0.120 ms 0.096 ms
2 host-212-158-250-39.dslgb.com (212.158.250.39) 9.303 ms 9.447 ms 9.464 ms
3 63.130.105.110 (63.130.105.110) 9.366 ms 9.713 ms 9.700 ms
4 10.200.18.53 (10.200.18.53) 10.304 ms 10.266 ms 10.365 ms
5 63.130.105.134 (63.130.105.134) 15.803 ms 16.647 ms 16.611 ms
6 10.200.18.37 (10.200.18.37) 17.012 ms 15.894 ms 15.795 ms
7 xe-1-1-0.cr1-lon1.twttr.com (195.66.225.142) 16.783 ms 16.851 ms 16.508 ms
8 104.244.42.1 (104.244.42.1) 17.130 ms 17.092 ms 17.056 ms
Cant explain why you are seeing what you are seeing, but you shouldn't be hitting the router 3 times before it even leaves your own network! Is there anything odd going on in the routers log files when you do this?
18-06-2020 09:13 PM - edited 18-06-2020 09:15 PM
Thanks for checking.
[Edit: router logs look normal]
I think the three internal hops is an artifact of using tracepath rather than traceroute, here's the same route examined with traceroute:
$ sudo traceroute 104.244.42.1 traceroute to 104.244.42.1 (104.244.42.1), 30 hops max, 60 byte packets 1 RT-AC86U-60B0.local (192.168.1.1) 2.220 ms 2.204 ms 2.248 ms 2 * * * 3 63.130.127.213 (63.130.127.213) 9.649 ms 9.993 ms 9.991 ms 4 10.200.18.45 (10.200.18.45) 9.617 ms 9.971 ms 10.242 ms 5 63.130.104.198 (63.130.104.198) 11.026 ms 11.331 ms 11.336 ms 6 63.130.127.213 (63.130.127.213) 11.076 ms 9.314 ms 9.244 ms 7 10.200.18.45 (10.200.18.45) 9.404 ms 11.744 ms 11.132 ms 8 63.130.104.198 (63.130.104.198) 12.704 ms 12.032 ms 12.692 ms 9 63.130.127.213 (63.130.127.213) 11.767 ms 12.044 ms 12.182 ms 10 10.200.18.45 (10.200.18.45) 11.782 ms 11.935 ms 11.096 ms 11 63.130.104.198 (63.130.104.198) 12.712 ms 12.566 ms 14.793 ms 12 63.130.127.213 (63.130.127.213) 14.262 ms 13.678 ms 13.998 ms 13 10.200.18.45 (10.200.18.45) 14.014 ms 13.347 ms 13.568 ms 14 63.130.104.198 (63.130.104.198) 14.114 ms 13.828 ms 13.822 ms 15 63.130.127.213 (63.130.127.213) 14.639 ms 15.617 ms 14.723 ms 16 10.200.18.45 (10.200.18.45) 17.400 ms 16.603 ms 17.393 ms 17 63.130.104.198 (63.130.104.198) 18.005 ms 17.335 ms 17.330 ms 18 63.130.127.213 (63.130.127.213) 16.888 ms 16.315 ms 16.725 ms 19 10.200.18.45 (10.200.18.45) 16.773 ms 16.787 ms 15.955 ms
etc. etc. round and round util it gives up at 30 hops.
I can see from your IP that you're on a different subnet, and thus your routing obviously differs.
For info, I'm gigafast, and the VF dhcpd has give my router 84.65.0.1 as the default gateway.
18-06-2020 09:35 PM
I'm hitting the same problem, pings to twitter.com fails with 'Time to live exceeded', and a traceroute shows:
traceroute to www.twitter.com (104.244.42.65), 60 hops max, 60 byte packets
1 vodafone.lan (192.168.1.1) 0.798 ms 1.152 ms 1.537 ms
2 host-212-158-250-40.dslgb.com (212.158.250.40) 9.638 ms 9.741 ms 9.834 ms
3 63.130.127.221 (63.130.127.221) 11.330 ms 11.443 ms 13.053 ms
4 10.200.18.45 (10.200.18.45) 13.191 ms 15.380 ms 13.215 ms
5 63.130.104.198 (63.130.104.198) 15.389 ms 15.487 ms 15.848 ms
6 63.130.127.213 (63.130.127.213) 15.588 ms 14.400 ms 14.462 ms
7 10.200.18.45 (10.200.18.45) 14.445 ms 10.187 ms 10.120 ms
8 63.130.104.198 (63.130.104.198) 12.783 ms 12.380 ms 12.514 ms
9 63.130.127.213 (63.130.127.213) 12.104 ms 11.499 ms 11.153 ms
10 10.200.18.45 (10.200.18.45) 11.451 ms 11.841 ms 11.778 ms
11 63.130.104.198 (63.130.104.198) 13.472 ms 13.708 ms 13.654 ms
12 63.130.127.213 (63.130.127.213) 13.021 ms 13.382 ms 13.317 ms
13 10.200.18.45 (10.200.18.45) 14.001 ms 13.655 ms 13.153 ms
14 63.130.104.198 (63.130.104.198) 15.573 ms 15.976 ms 15.804 ms
15 63.130.127.213 (63.130.127.213) 14.215 ms 14.662 ms 14.089 ms
... (the last three IPs loop continually)
Do Vodafone have any network status page, or a place to report problems? The only status page I could find just shows their phone coverage.
18-06-2020 09:41 PM
Right, I'll log it with support and point them here.
Suggest others do the same.
18-06-2020 10:32 PM - edited 18-06-2020 10:34 PM
Yes, for twitter.com and 1.1.1.1 (cloudflare DNS)
Appears to be an Vodafone internal l3 routing loop given I'm not seeing the packets leave Vodafone/c&w nor am I seeing the same routing from other ISPs.
tac2@MacBook-Pro ~ sudo /usr/local/sbin/mtr twitter.com --report --report-wide
Password:
Start: 2020-06-18T22:30:55+0100
HOST: MacBook-Pro.3ke.uk Loss% Snt Last Avg Best Wrst StDev
1.|-- router.3ke.uk 0.0% 10 1.2 1.3 1.1 1.5 0.1
2.|-- host-212-158-250-38.dslgb.com 0.0% 10 7.1 14.0 7.0 45.7 11.8
3.|-- 63.130.127.213 0.0% 10 7.7 10.3 7.2 35.3 8.8
4.|-- 10.200.18.45 0.0% 10 9.2 8.0 7.3 9.2 0.6
5.|-- 63.130.104.198 0.0% 10 9.7 10.4 8.5 21.0 3.7
6.|-- 63.130.127.213 0.0% 10 9.6 14.2 8.8 39.5 10.8
7.|-- 10.200.18.45 0.0% 10 10.0 10.7 9.2 21.4 3.8
8.|-- 63.130.104.198 0.0% 10 11.5 16.4 10.3 43.4 11.3
9.|-- 63.130.127.213 0.0% 10 10.6 10.8 10.3 12.2 0.5
10.|-- 10.200.18.45 0.0% 10 11.9 11.7 10.8 14.9 1.2
11.|-- 63.130.104.198 0.0% 10 13.1 12.6 12.1 14.1 0.6
12.|-- 63.130.127.213 0.0% 10 13.5 14.1 12.2 23.1 3.5
13.|-- 10.200.18.45 0.0% 10 13.4 13.3 12.3 15.1 0.8
14.|-- 63.130.104.198 0.0% 10 14.5 17.7 13.6 42.3 8.8
15.|-- 63.130.127.213 0.0% 10 14.8 15.0 13.1 19.0 2.0
16.|-- 10.200.18.45 0.0% 10 14.3 14.8 13.8 19.7 1.7
17.|-- 63.130.104.198 0.0% 10 15.7 17.3 15.1 32.5 5.3
18.|-- 63.130.127.213 0.0% 10 15.2 16.0 15.2 17.4 0.8
19.|-- 10.200.18.45 0.0% 10 15.6 15.9 15.3 16.4 0.3
20.|-- 63.130.104.198 0.0% 10 17.3 20.2 16.6 47.1 9.5
21.|-- 63.130.127.213 0.0% 10 16.6 17.3 16.6 18.4 0.5
22.|-- 10.200.18.45 0.0% 10 17.4 18.4 17.0 26.7 2.9
23.|-- 63.130.104.198 0.0% 10 18.4 21.8 18.4 44.4 8.0
24.|-- 63.130.127.213 0.0% 10 18.5 18.9 18.4 19.8 0.4
25.|-- 10.200.18.45 0.0% 10 18.8 18.9 18.4 19.7 0.4
26.|-- 63.130.104.198 0.0% 10 22.4 23.3 19.5 43.2 7.1
27.|-- 63.130.127.213 0.0% 10 20.8 20.2 19.5 20.8 0.3
28.|-- 10.200.18.45 0.0% 10 20.3 20.7 19.9 22.1 0.6
29.|-- 63.130.104.198 0.0% 10 27.6 24.7 21.4 42.5 6.5
30.|-- 63.130.127.213 0.0% 10 23.3 22.1 21.0 23.4 0.7
tac2@MacBook-Pro ~ sudo /usr/local/sbin/mtr 1.1.1.1 --report --report-wide
Start: 2020-06-18T21:58:53+0100
HOST: MacBook-Pro.3ke.uk Loss% Snt Last Avg Best Wrst StDev
1.|-- router.3ke.uk 0.0% 10 1.2 1.6 1.1 2.7 0.5
2.|-- host-212-158-250-38.dslgb.com 0.0% 10 7.3 8.8 7.0 16.1 2.9
3.|-- 63.130.127.213 0.0% 10 7.2 10.3 7.2 34.3 8.4
4.|-- 10.200.18.45 0.0% 10 8.8 12.4 7.6 49.1 12.9
5.|-- 63.130.104.198 0.0% 10 8.7 9.9 8.7 15.8 2.1
6.|-- 63.130.127.213 0.0% 10 9.0 11.6 8.8 30.1 6.6
7.|-- 10.200.18.45 0.0% 10 9.2 9.6 8.9 10.1 0.4
8.|-- 63.130.104.198 0.0% 10 10.8 11.1 10.4 12.9 0.7
9.|-- 63.130.127.213 0.0% 10 10.9 13.4 10.3 38.5 8.8
10.|-- 10.200.18.45 0.0% 10 11.9 11.8 10.7 14.5 1.3
11.|-- 63.130.104.198 0.0% 10 12.3 12.3 11.7 13.2 0.4
12.|-- 63.130.127.213 0.0% 10 12.1 12.4 11.8 13.1 0.4
13.|-- 10.200.18.45 0.0% 10 12.5 14.3 12.1 27.8 4.8
14.|-- 63.130.104.198 0.0% 10 14.3 14.3 13.8 16.0 0.6
15.|-- 63.130.127.213 0.0% 10 14.4 16.3 13.6 36.8 7.2
16.|-- 10.200.18.45 0.0% 10 14.3 14.4 14.0 15.1 0.4
17.|-- 63.130.104.198 0.0% 10 16.5 15.9 15.3 16.8 0.5
18.|-- 63.130.127.213 0.0% 10 15.5 18.1 15.1 40.8 8.0
19.|-- 10.200.18.45 0.0% 10 15.9 16.3 15.5 19.2 1.1
20.|-- 63.130.104.198 0.0% 10 17.5 17.3 16.7 18.1 0.4
21.|-- 63.130.127.213 0.0% 10 17.6 19.0 16.5 31.2 4.5
22.|-- 10.200.18.45 0.0% 10 17.5 17.4 16.8 17.7 0.2
23.|-- 63.130.104.198 0.0% 10 18.5 21.2 18.5 43.0 7.7
24.|-- 63.130.127.213 0.0% 10 19.5 19.3 17.9 24.7 2.0
25.|-- 10.200.18.45 0.0% 10 18.7 21.3 18.7 40.6 6.8
26.|-- 63.130.104.198 0.0% 10 21.0 21.7 20.0 33.5 4.1
27.|-- 63.130.127.213 0.0% 10 20.6 22.5 20.0 43.1 7.2
28.|-- 10.200.18.45 0.0% 10 20.4 22.5 20.4 37.6 5.3
29.|-- 63.130.104.198 0.0% 10 30.5 25.9 21.5 49.0 8.6
30.|-- 63.130.127.213 0.0% 10 22.1 22.1 21.2 23.5 0.7
18-06-2020 10:44 PM
The test would be to see where the routing problem is.
Probably want to start looking for that by checking if:
NsLookup
Server: 35.178.60.174
Address 1: 35.178.60.174 ec2-35-178-60-174.eu-west-2.compute.amazonaws.com
Name: twitter.com
Address 1: 104.244.42.129
Address 2: 104.244.42.193
Just check that we are all in the same ballpark regards where we are looking.
*I don't have any linux machines here that use the specific "tracepath! instruction.
18-06-2020 11:07 PM - edited 18-06-2020 11:12 PM
It seems to be a fairly straightforward upstream router configuration cockup.
I've just spent 30 mins on chat with a fantastically nice, but totally out of his depth, first line support guy.
I was told to call the Gigafast support no. (08080-034-515) in the morning.
19-06-2020 12:28 AM
I'm having the same problem with cloudflare (1.1.1.1), traffic looping between the above IP's and not leaving Vodafone.
19-06-2020 10:22 AM
Escalated to third line support just now.