Packet loss - every night, peak time

[jdvega]

Hi there,

Every night - during the week 6pm -> 10PM, or 4PM -> 10PM on weekends - I see really bad packet loss.  At least 10%, if not 30+%.

Is this common on Vodafone?  I do see quite a few similar reports.

ThinkBroadband graph below, this is every night at peak time:

https://www.thinkbroadband.com/broadband/monitoring/quality/share/60522871a7682f8b6c0c7a9584bd2d8c8fb70fb0-16-11-2023

I had to dig out the Vodafone router - as they refused to support me with my router - we've now been through 2 factory resets and firmware updates (neither of which I had any faith would resolve the problem) but I played along anyway.

Was due a call back, 48 hours after the last reset - as the router needed to 'settle', apparently - no call was received.  The problem persists, as I assumed it would.

The CE Vodafone router itself seems to have a crazy ICMP limit - certainly since they updated the firmware, it is now many times worse than it was, although I only used it briefly initially - i.e you can run an MTR over 10 hops, you get about 30 passes before the packet loss starts from the Vodafone router itself (192.168.1.1) at any time of the day.  Stop the trace, wait a bit, you can then it again for a while.

I'm also reasonably confident that there is rather aggressive ICMP limiting/filtering on the core network as well.  Which is quite frankly ridiculous - backbone routers are known to deprioritise ICMP, but the scale of which Vodafone seem to do so is just wrong.

Obligatory trace to BBC (ICMP), at 08:00, first 10 or so passes are fine:

Run it a bit longer, still at 08:00 in the morning, the Vodafone router starts dropping/limiting ICMP packets:

Running a TCP MTR instead - mtr -T -P 80 www.bbc.co.uk - no packet loss over a longer run, albeit this is 08:00 in the morning.  

So the Vodafone router itself, is aggressively limiting ICMP - I'm pretty sure this didn't happen, until they updated the firmware, but as I mentioned I only used the Vodafone router briefly.

Right now, I have to run the Vodafone router with my firewall in the DMZ.

I'm not sure I have the strength to go through the support team again and/or more router resets...

[jdvega]

Searching the problem, I see another person saw problems on the same gateway as me - many, many months ago:

90.247.192.1

descr: sdnn13-bng-c1

... they got to leave charge free after providing months worth of data, I suspect - given my experience of support thus far - that is going to have to be my end goal.

[jdvega]

Phoned up again this morning:

Apparently Tier 1 cannot escalate to Tier 2, unless there is a total loss of service <face palm>

As I didn't get the call back from the initial Engineer on Wednesday, phoning up again after they failed to return my call, seems to mean we have to start at the beginning again!

Supposedly, if you speak to another agent, the initial agent is no longer allowed to phone you back!  This is absolutely bonkers.

Right now, the current agent is suggesting we have to go through the pin hole reset on the ONT, Vodafone CE router again... then I have to wait 48 hours for her to phone me back again, before she can escalate to Tier 2.

... but if I speak to another agent, then she's no longer allowed to phone me back!

[jdvega]

Since the 28th, all packet loss - including ICMP - appears to have been resolved.  Not sure whether this was a fault that was resolved, ICMP rate limits increased, etc.

However, since the 28th November, I'm no longer seeing (primarily) peak time congestion/contention on my circuit/gateway - fingers crossed...

[Jayach]

Good news.

Are you still on the same gateway 90.247.192.1 ?

[jdvega]

Yes - never been on anything else.