This Post will be in english, but if there is interest we can re-publish it in german
So, I’ve seen a post on twitter coming by recently
So what do we have here:
– Our Friend here wants to load a website on Yahoo
– he forces it to be loaded over IPv6
– He asks for others to test, so obviously he has an issue.
Now I have an Init7 Line – Historically I used to be a DSL customer from Init7, and I have managed to talk them into giving us a block of 30 IPs, which I will hold onto as they are in rare supply nowadays – it was I think 2007 when I got the block, but I used dialup in the early 2000s and then DSL already, but only with one IP. Anyway. We have an init7 Line, even if we are an ISP ourselves.
So I test it, and I have no issue either.
What do I know about this user?
– He has a router I supplied him once. It’s a Mikrotik HEX s (one of our favourite routers, by price/power)
– He is logged in via PPPOE; so not native but via Swisscom BBCS. What’s the effect here? Mikrotik does lower MTU on PPPOE by default. 1480 to be precise.
And with that insider Information, I knew already what his issue was before I even had to test (the website loaded for me, it wasn’t a routing issue)
The Technical Background
IPv4 (the old Internet Protcol) has specified that if a packet is not going to fit through a line, it has to be fragmented (Broken up into pieces). Imagine your post office agent cutting up your packet, and then you have to glue it back together when your letterbox is too small. also you have to wait for two deliveries. very inefficient.
IPv6 has come up with the idea that this shall be coordinated ahead of time to save on inefficiencies (and lower CPU usage on internet routers who have to do the cutting/glueing)
Now, this is called PMTUD – Path MTU discovery.
The ancient fear
I don’t remember which year it was. 1995? 1998? People used dial up. Directly on their computers who then were directly exposed to the internet and got a public IP. Personal Firewalls sometimes got installed, but it was before XP SP2 which brought one in Windows. So if you reinstalled your PC and had to download all the Programs first, you were naked. Zonewall? Remember it? You’re old my friend – me too…
Now, some crazy madlad had the invention of sending oversized ping packets and that caused, at one point, Windows to crash to a bluescreen.
so Network Admins blocked Ping packets, but they mostly blocked the entire Protocol ping relies on – ICMP.
Cut forward to 2021, and there are still network Admins who are afraid 25 Years later, and prefer to block an essential Protocol, in this case ICMPv6, which in turns breaks Standards.
Known places
in Switzerland? www.sbb.ch used to be annoying if you had lower MTU and didn’t adjust on your end, same with www.bluewin.ch (ask our friend at https://www.sunny.ch/en/about-sunny/profile/ – we looked at it once together)
Apparently, yahoo is the same. And as far as I remember, it’s all F5 Firepass Loadbalancers.
So what can you do?
Now, why is this reduced MTU like it is?
– PPPOE is PPP Protocol over ethernet, which means there is another encapsulation. Imagine your letter sent in a bubble wrap. it makes it thicker immead, and that’s encapsulation. You can fit less into the letter before you can’t push it through the letterbox flap anymore.
Same applies for the internet.
Regular Fiber over BBCS (Swisscom Wholesale) would run a VLAN (4 Bytes) and then subtract 8 bytes for PPPOE if you have no special settings or a stupid mediaconvert. if you increase your fiber Port to 1512 Bytes, you will be fine. But you need to set this correctly in your router and on many places to apply properly.
The other (quick and dirty way) is to avoid sending ever too big packets. The problem is, you might send a small packet, but you need to tell the other side that you can only receive packets which are a bit smaller.
So what is the right size?
Mikrotik uses an MTU of 1480 by default. we are not going to touch this, even tho 1492 would be a bit better.
We have then to subtract the IPv6 header – which is 40 bytes. If we would have the same issue on IPv4 we could subtract 20 bytes for IPv4, but we will disregard this here.
1480-40 bytes leaves us 1440 Bytes. Now that’s the IPv6 Payload. What are we running on top of IPv6? it’s HTTPS/HTTP which is in this case running on TCP; so lets cut that down and have another 20 Bytes lost.
So the Payload of the TCP Connection will be 1420.
So we tell our mikrotik to inform the other end we can receive only 1420 byte Payloads at once.
So we login to our router, open the shell (e.g. via webshell, Winbox, or via telnet:)
The solution is: mangle (and you will see that our tweet has a mistake as I started with the value of 1492 instead of 1480 and only corrected it into one place)
Command to enter:
/ipv6 firewall mangle add chain=forward protocol=tcp action=change-mss new-mss=1420 tcp-mss=1421-65535 tcp-flags=syn out-interface=pppoe-out
(since mtik uses 1480 mtu by default)
So what does that all mean
we enter into the context of the ipv6, Firewall, Mangle
into the forward chain (packets which gets forwarded by the Firewall)
for TCP Protocol
that we want to change the MSS (Maximum Segment Size)
to the value of 1420
for all packets which are set to be 1421-65535 (so too big)
and where the flag is syn (so a new connection)
and if it goes out of the PPPOE Interface
And your website will finally load