2020111962000979 (2020-11-19 11:59:15): kernel Panic -Mtik router baselPriority:
Description: Kernel Panic in Basel
We have seen our router in Basel showing a kernel panic.
The router has automatically rebooted, causing OSPF Flaps everywhere.
This problem has been extended in severity due to the current state of
migration of the network. We are switching into our 4th Generation of
routers, and some of the older mikrotik gear does not want to play nice
with our extreme gear we are currently putting into several locations.
This is an issue that has not yet cleared out what is the trigger.
We have upgraded the router in basel to a newer release (6.47.1 to
6.47.7) during the next reboot that was necesary, and are looking if
this helps. clearly seen is that the extreme router cannot form
adjacencies with it.
Reason for this migration is to go to complete hardware routers due to a
line of issues we encountered with our routers so far - the bigger a
network gets, the more flaws a solution shows.
MK1: Mikrotik: Broken iBGP for Ipv6, and slow convergence. easy with 2
routers, unbearable with 13
MK2: Ubiquiti routers: They have route leak issues with quagga and
cannot handle a larger configuration as we need
MK3: Vyos: works generally stable, ospfv3 can crash on our table, so we
need to remain on FRR7.2 which is a bit an older release
MK4: Extreme Hardware: works mostly fine, but does not want to come up
with mikrotik unless you restart interfaces on the mtiks, and restart
the process in the brocade. Mikrotik routers show full adajacency,
brocades stay in "EXCH"
2020111862000515 (2020-11-18 03:24:13): OSPF crash // CZHPriority:
Description: Our brocade router in CZH has apparently a hanging process on OSPF; which resulted in a watchdog-restart of one of the old borderrouters that still has that enabled.
Seems some ospf process died. We are in the progress of replacing our core routers as previously communicated and there seems to be some friction between the old and the new routers.
A reload in CZH has brought back stability. We are going to switch the CZH with another router as since the install, the 10G Card had been malfunctioning and a replacement of that specific card didn't fix the issue either (Re: packet loss on sunday)
2020111362000373 (2020-11-13 11:18:05): Swap defective linecardPriority:
Description: We are replacing the 10G Linecard in our core router in Zurich ColoZuri at midnight. we expect some minor route flaps and a few momnets of packet loss as we probably just swap it out due to the late time.
2400 (Midnight) is start of the CZH Maint.
Updates
2020-11-14 13:06:51
we have swapped the defective linecard, but it seems there is now 95% paket
loss for IPv6 and other issues. We will be rolling back to the scenario we had
before and replace the entire chassis next.
