pfsense behind Vodafone wifi-hub (ADSL) / double NAT / 50% plus packet loss all the time

[cmu2808]

This is a really strange one... so just moved into a new property and had to settle for a basic ADSL connection, Vodafone recently provided the connection to my property together with their standard wifi-hub... upon initial inspection simply running of the wifi-hub everything seemed ok, however now that I've put my pfsense (running version 2.6.0) behind the wifi-hub I'm seeing a lot of packet loss.  

So the setup is exactly the same as it was in my previous property (where i had no issues). I basically have the pfsense WAN interface connected to one of the Vodafone wifi-hub ethernet ports and wifi-hub sits in the same network range as the pfsense WAN interface IP - Vodafone wifi-hub is .1 and the pfsense WAN interface IP is .254.  I've placed the .254 address in the "DMZ"  and I disabled all functions on the Vodafone wifi-hub, turning it essentially into a ADSL modem.  But still have the annoying double NAT config!  

Doing a continues ping (towards 1.1.1.1) from my desktop, connecting to the internet via my AP (which connects into a switch, which connects into pfsense) - I'm getting 50% plus packet loss. 

The second hop below is the Vodafone wifi-hub, locally going from pfsense to wifi-hub it seems to be registering 65% loss!

Thinking this might be a wifi issue, I run a cable from my switch to my desktop - same result!

Thinking it might be a bad cable between the pfsense WAN interface and Vodafone router, I swapped cables multiple times - same result!  I've tried the different ethernet ports on the Vodafone router always the same result - packet loss!  Even played with the MTU and negotiation settings on the pfsense WAN interface - no luck!

Thinking this is pfsense related, I've done the same continues ping test connecting my desktop directly over the Vodafone wifi-hub, over both wifi and cabled connections registering 5-10% loss - which I guess is acceptable. 

Although at some point yesterday I was registering, 25% loss connecting directly over the Vodafone wifi-hub  - which led me to think there was something wrong with the line or the router - Vodafone engineer on the way to check line and router, who I doubt will find anything wrong.

Currently running quality monitor via thinkbroadband.com to make sure the issue is definitely not related with the connection itself.  

Now despite all of this packet loss I'm registering, when connecting to internet via my pfsense, don't seem to be having any connectivity issues.  No drops or buffering during streaming, no drops over VoIP, VPN connectivity seems solid,  So I'm really confused???

My next move I think is to order a simple ADSL modem (Draytek Vigor 130) and remove the Vodafone wifi-hub out of the equation, got the login details from Vodafone, but looking for alternative suggestions before I go ahead with that plan and spend the money.  

[cmu2808]

Right, line is solid now, no more errors on the line according to Open Reach engineer - he had to go and tweak things in the street cabinet.  Connecting over VF router (over WIFI) pinging 90.255.255.90 seems to be ok - only few drops on one of the hops.

And pinging 1.1.1.1 over VF router (over WIFI) I'm getting the usual 8% loss.

Pinging 1.1.1.1 over VF router (over WIFI) using CMD PING - I'm registering drops.

Pinging 90.255.255.90 over VF router (over WIFI) using CMD PING - I'm registering drops. 

Pinging 1.1.1.1 over VF router (over CABLE) - I'm registering drops. 

Overall VF router functioning right I guess, but there are still drops connecting directly over VF hub, now that line issues are out, not sure what else I can do at this point... really getting sick of this tbh, and behind pfsense still massive packet loss, which now I'm certain is related with the double NAT - simply VF router is not handling the double NAT very well (knowing never had this issue with Virgin router).  I will go an buy myself a Vigor 130 ADSL modem and see what results that produces.  However despite the packet loss overall internet quality behind pfsense is fine for now - no drops in streams, VPN or VOIP!

Thanks for all the input from everyone.  I'll update once I get the Vigor 130 into production

[cmu2808]

Hey all,

Good news today my Vigor 130 arrived.  I've configured it so that the PPPoE authentication is done by pfsense.  And results are extremely good!  All packet loss registered by pfsense monitors has gone away, green lights end to end

No packet loss pinging 1.1.1.1 over wifi from my desktop either.

So issue was with the VF router unfortunately, just to be clear, taking pfsense out of equation and simply using VF router over wifi/cable was producing packet loss before, so not really sure if it's a faulty bit of equipment.  I'm down by £80 but worth seeing no packet loss on my firewall   I'll monitor for a week or so, will let you know if issues.  

For those who are interested in a similar setup - the Vigor 130 / pfsense config is detailed below:

Internet Access > General Setup > VDSL2 / Service columns - Enable, Tag value 101 Priority 0

PPPoE/PPPoA > PPPoE/PPPoA Client - Disable

MPoA / Static or dynamic IP > MPoA (RFC1483/2684 - Enable / MTU - 1492

Then on pfsense change your WAN interface IPv4 Configuration Type to PPPoE, and enter PPPoE creds - Username/Password you get from Vodafone

Change your default g/w under routing to point at the new PPPoE g/w

Also make sure you under General Setup your DNS servers are pointing at the PPPoE g/w

[CrimsonLiar]

Be a little cautious about using MPoA rather than PPPoE/PPPoA.  MPoA which is unsupported used to work just fine, but changes in issuing the WAN IP means that you may not get the WAN IP lease refreshed.

[cmu2808]

@CrimsonLiar The WAN interface of pfsense can be reset on scheduled basis to perhaps get around WAN IP lease refresh issues (not ideal but it's something).  I need to see if that's going to be a problem first though.    

This is just curiosity, one thing I'm confused about is there is no place in UI to enter VDSL creds on the Vigor 130, which makes me think how can it logon to the network, if like with Vodafone you need to provide creds (in the documentation it simply says the modem is pre-configured for UK ISPs, assuming some ISPs don't require creds?)  Any ideas?  And if I'm not in full bridge mode not sure how I would provide creds to logon to network.

[Jayach]

The Vigor 130 is a modem, it's job is to negotiate with the Openreach cabinet, and in the UK it will always be an Openreach cabinet so the VLAN etc are fixed. I assume that is what it means by pre-configured for UK ISPs.

The internet credentials will need to be supplied by the router, as you are doing with pfsense.

At least I think that that's correct. @CrimsonLiar can probably go deeper.

[CrimsonLiar]

Nope you pretty much got it on the VLANs - though you can edit them if you need to.  And yes the PPPoE username and settings should be set on the pfsense box and applied to the (WAN) port you are connecting to the modem.

The basic manual for the Vigor 130 is pretty short since the usable options are pretty minimal.  There is a massively larger manual that lists every setting that is available, almost all of which will have either no effect or will mess things up rather than make anything better.

The common terminal commands you may need are:

vdsl status    - pretty much self explanatory

vdsl snr?   - used to read and set the snr margin (cannot override the cabinet and grossly misunderstood/misused).

[cmu2808]

Ok so how would it work if I'm not using MPoA Bridge Mode? and use PPPoE as you suggested (due to the WAN IP lease refresh issue).  Assuming by PPPoE you meant PPPoE pass-through Mode.  How is that configured then?  How is it any different?  

[CrimsonLiar]

With the mode of the Vigor 130 (must the the UK version) set up as PPPoE/PPPoA it's still going to function as a bridge/pass-through device - as you can test for yourself once it's set up!

All the same the settings on the Pfsense box will be the same whether the Vigor 130 is set up as PPPoE/PPPoA or MPoA.  So when you set up the WAN connection on the Pfsense box, it should be as simple as setting the "IPv4 Configuration Type" to PPPoE and then entering your broadband username and password into the credentials.  That is as much as you need to do to get it to work.  Of course, once it's up and working you can play with it, and dropping the connection between the Pfsense and the modem will generate a new lease without needing to reboot the modem!

*I'd be interested to see if you get the full lease details reported to the Pfsense box.

[Cynric]

Here's my stats from behind a Double-NAT and Pi-Hole for comparison.

|------------------------------------------------------------------------------------------|
|                                      WinMTR statistics                                   |
|                       Host              -   %  | Sent | Recv | Best | Avrg | Wrst | Last |
|------------------------------------------------|------|------|------|------|------|------|
|                             192.168.0.1 -    0 | 4136 | 4136 |    1 |    3 |   37 |    1 |
|                             192.168.1.1 -    0 | 4136 | 4136 |    1 |    3 |   90 |    2 |
|                            172.xx.xx.xx -    0 | 4136 | 4136 |   13 |   15 |  251 |   14 |
|      132.hiper04.sheff.dial.plus.net.uk -    0 | 4136 | 4136 |   16 |   19 |  253 |   19 |
|  peer7-et-3-0-4.telehouse.ukcore.bt.net -    1 | 4133 | 4132 |   15 |   21 |  255 |   18 |
|                          109.159.253.95 -    0 | 4136 | 4136 |   16 |   22 |  253 |   19 |
|                           141.101.71.47 -    0 | 4137 | 4137 |   16 |   20 |  250 |   18 |
|                         one.one.one.one -    0 | 4136 | 4136 |   16 |   19 |  253 |   17 |
|________________________________________________|______|______|______|______|______|______|
   WinMTR v0.92 GPL V2 by Appnor MSP - Fully Managed Hosting & Cloud Provider