VLANs on the Mikrotik hAP ac³

▼ For my BGP lab/training setups I have a bunch of routers, real or virtual, that each need several IP addresses to talk to other routers. VLANs are the perfect way to keep all of this manageable: with VLANs, it's possible to have separate IP interfaces, but still just use a single Ethernet port to hook everything up. Simple unmanaged switches simply forward the packets to the right port without looking at the VLAN header, so no need to configure the network.

Until I got my Mikrotik hAP ac³ this week...

In its default configuration it will do what unmanaged switches do: ignore the VLAN header. In addition, you can add VLAN interfaces to the Mikrotik and give those an IP address. I even got IPv6 prefix delegation to work. For this, set up the DHCPv6 client to request a prefix. Then fill in the name of a pool. But don't create that pool yourself—that won't work. The pool will be created automatically. Then configured an IPv6 on a VLAN interface that I created and specified that the pool should be used for that. "Advertise" is enabled by default, which means that the Mikrotik will send out router advertisements and machines that have the VLAN configured will configure an IPv6 address for themselves on that VLAN interface.

Eventually I want to set up a guest network with its own VLAN and IPv4 NAT. For now I just set up IP addresses on the VLAN interfaces so the Mikrotik will be able to talk to my virtual BGP routers on these VLANs. That initially went well.

However, I also wanted have these BGP VMs talk to an old Cisco 2503 router, which doesn't support VLANs. So I wanted to set up an access port. The idea was that the Cisco would be part of VLAN 203 without the need for it to insert a VLAN TAG itself. On the other ports, VLAN 203 (and a few more) get a VLAN header. However, not exactly like a trunk port where all packets have a VLAN header: regular traffic to/from the internet goes out on these same ports untagged.

Mixed tagged/untagged ports turned out to be unsupported by the Atheros 8327 chip that powers the switch ports. What you can do is have one or more access ports and one or more trunk ports. This is already somewhat complex. The access ports need to be set up with a default VLAN. (Under "switch" and then "ports".) Makes sense. In the documentation, it says to use add-if-missing will add a VLAN header if there isn't one. Makes sense. However, it turns out that you need to do this on the outgoing port, i.e., not the access port itself, but the trunk port(s) that are part of the same VLAN. However, then (with a slight delay?) untagged traffic from the default VLAN won't work anymore.

The good news is that even though the hardware can't have mixed ports, the software can. So remove those settings above, and go to the bridge section of the configuration instead. There, select your bridge instance and turn on VLAN filtering. After that, you can go to the VLAN tab an add one or more VLANs there, where you can specify which ports should be tagged and untagged for that VLAN. So now we can have access ports and mixed ports. I do believe performance will be lower because VLAN traffic will need to be handled in software. Also not the PVID setting, which is the default VLAN untagged traffic. (1 out of the box.)

Unfortunately, the Mikrotik's IP addresses on the VLAN interface I tested with were now unreachable, and I couldn't figure out any settings to fix that. So after I was done with the Cisco 2503, I unchecked the "VLAN filtering" in the bridge settings... and the Mikrotek's IP addresses on VLAN interfaces came back to life.

Permalink - posted 2021-04-04

No joke: running BGP on a $100 home router / Wi-Fi access point

▼ For some time, I've been hearing about Mikrotek routers, which couple being quite capable with being affordable. But I never got my hands on one. I'm now in the process of upgrading my home network, and learned about the Mikrotik hAP ac³. The ac³ defies easy classification, but I think it's mostly a home router and/or Wi-Fi access point. I paid € 95 and shipping, and I believe it's available in the US for about $100.

I was somewhat disappointed to learn that "5 gigabit ports" doesn't mean ports that are capable of 5 gigabit, but 5 ports that just ordinary 1 Gbps Ethernet. Initially it seemed the box didn't support IPv6, but it turns you have to enable that under "packages" and then reboot. (Not shutdown.)

However, I wasn't disappointed to learn that the ac³ supports RIP, OSPF and BGP, both for IPv4 and IPv6. So I made a configuration on the ac³ that fits with the lab network that I use for my training courses. Initially it didn't seem to work, but then after a reboot of the ac³ the BGP sessions came up. Pretty cool!

The hardest part was setting up VLAN IP interfaces. For some reason I needed to use the command line to create the VLANs:

/interface vlan add interface=bridge name=vlan1401 vlan-id=1401
/interface ethernet switch vlan add ports=ether3,switch1-cpu independent-learning=no switch=switch1 vlan-id=1401


After that, I could use the web interface to add IP addresses. I also set up BGP through the web interface, but the command line export of that looks like this:

> /routing bgp export
# apr/01/2021 20:50:37 by RouterOS 6.48.1
# software id = C3HH-K6NW
#
# model = RBD53iG-5HacD2HnD
# serial number = E7290D96999E
/routing bgp instance
set default disabled=yes
add as=65082 client-to-client-reflection=no name=Router82 router-id=203.0.113.82
/routing bgp network
add network=10.0.0.0/8 synchronize=no
/routing bgp peer
add address-families=ip,ipv6 hold-time=1m30s instance=Router82 name=ISP30 out-filter=out-as-filter
remote-address=192.0.2.21 remote-as=65030
add instance=Router82 name=ISP40 remote-address=192.0.2.41 remote-as=65040


Initially it seems the Mikrotik won't show you any BGP info, but with a bit more digging you'll get there:

Way back in 1996 the global BGP table was 30,000 prefixes and that just about fit in the 16 MB that was the maximum a Cisco 2501 router would take. The ac³ has 128 MB storage (using less than half of that) and 256 MB RAM, which is probably enough for 200k - 500k prefixes but not a full table. And of course the ac³ is not an obvious choice for such a task in the first place.

Permalink - posted 2021-04-01

Ethernet cables in the post-gigabit era

▼ As I'm preparing for the arrival of more > 1000 Mbps Ethernet equipment, I did an inventory of the Ethernet cables I've collected over the years. Turns out there's more variety than expected: I have cat 5 UTP, cat 5e UTP, cat 5e STP and cat 6 STP. (The good news: crossover cables are no longer a thing. Even the red cable is a regular straight cable.)

Aside:

UTP is unshielded twisted pair. The cat 5 and cat 5e specifications don't require shielding, and I believe the same is true for cat 6 and cat 6a. Just the twisting already makes the cable pairs much less susceptible to interference than an untwisted cable. However, it's not uncommon to see twisted pair with shielding, which we may be tempted to call "STP". However, things are a bit more complex than that: there's two types of shielding and two places where it can be applied. There's foil shielding and braided shielding, and this can go around all four pairs together, or each pair can be shielded individually. Some examples:

• UTP: no shielding
• F/UTP: foil on the outside
• U/FTP: no outside shielding, foil around each pair
• S/UTP: braiding on the outside
• S/FTP: braiding on the outside, foil around each pair
• And so on

However, just "FTP" or "STP" are also used.

The most common types of twisted pair used for Ethernet are cat 5, cat 5e, cat 6 and cat 6a. New categories add more stringent crosstalk limits and typically have a higher MHz bandwidth rating. Note that each higher category is compatible with the lower categories. So if you want to use a cat 6a cable to hook up your phone, that'll work just fine.

Without going over all the specs, this table shows the compatibility between cable categories and Ethernet speeds. (As 10GE is barely showing up now in homes and offices, fifteen years after being standardized, I'm going to ignore possible speeds faster than 10GE.)

	GE	2.5GE	5GE	10GE
Cat 5	😀	⛔	⛔	⛔
Cat 5e	😀	😀	😀	⛔
Cat 6	😀	😀	😀	max 55 m
Cat 6a	😀	😀	😀	😀

(With 😀 meaning works with a maximum distance of 100 meters and ⛔ doesn't work.)

Update: to make sure 5GE works at the full 100 meter distance, it needs cat 6, as 5GE requires a bandwidth of 200 MHz. Cat 5e supports 100 MHz, cat 6 250 MHz and cat 6a 500 MHz. However, with NBASE-T downshift (with the N in NBASE-T meaning 2.5/5GE) the devices on both ends try the highest speed they have in common and "downshift" if that doesn't work reliably. So in practice you'll often get better speeds than expected for the cable type, especially for shorter cables and if there's no other cables running together for significant stretches. NBASE-T downshift can also be applied to 10GE, GE and 100 Mbps Ethernet.

So from the table above, it's pretty obvious that you'll want to avoid cat 5. Turns out that a good number of the (relatively short) Ethernet cables I have lying around are cat 5. I'll have to be careful to only use those in places where I won't be using GE+ speeds for the foreseeable future.

But... is it worth the trouble to use cat 6 or better?

Well, you definitely don't need cat 6a in a regular home, as you're not going to exceed cat 6's 55 meter limit for 10GE. So it's between cat 5e and cat 6.

A couple of years ago, I installed cat 6 between the living room where the cable modem resides and my home office. This was F/UTP. In addition to the shielding, there's a plastic spline in the cable that keeps the four twisted pairs apart. All of this makes the cable a lot thicker and harder to bend than regular cat 5e UTP. Connecting the shielding to shielded connectors at both ends also provides the opportunity for ground loops and the associated electrical shocks.

So I'd say that cheaper and easier to work with (and easier to get in the desired color!) cat 5e is probably a better choice in many cases. Although when laying down cabling that would be very hard to replace, it might be a good idea to install shielded cat 6 or cat 6a—with some extra length to make sure you can replace connectors that prove inadequate when you upgrade to 10GE.

Permalink - posted 2021-03-30

When the BGP table hits 1 million prefixes, will history repeat itself?

▼ On the APNIC blog, Danny Pinto asks What will happen when the routing table hits 1024k? Back in 2014, the IPv4 BGP table reached 512k, a common limit in many routers at the time, and some bad things happened. See my post BGP table hitting 512k limit in older routers. And pretty much the same thing happened in 2008, when the BGP table hit 256k.

Interestingly, reaching the 1024k limit was predicted for 2019, but now in March 2019, we're still well below 900k: BGP table growth has leveled off from an average 10% per year in the 2010s to 5% in 2020. My prediction is that we won't reach 1024k IPv4 prefixes in the global BGP table until 2024.

However, asking when we'll reach 1024k IPv4 prefixes is probably asking the wrong question. (But, do read the APNIC blog article for a few relevant pointers.) Back in 2014, there were less than 20k IPv6 prefixes in BGP, and many networks didn't run IPv6 yet. Well, actually, there are 71k networks (autonomous systems) advertising one or more IPv4 prefixes, but only 22k networks announcing one or more IPv6 prefixes.

Still, many networks run IPv4 and IPv6 side-by-side, which means 874k IPv4 prefixes + 114k IPv6 prefixes = 988k total. Add to that internal prefixes, which can add up to many thousands in larger networks, and I'm pretty sure many networks are already over the 1024k limit today, or will be soon.

So I'm sure a few networks will be caught unprepared and run into trouble when the IPv4 table goes over 1024k, but to a much lesser degree than in 2008 and 2014.

In any event, when buying a new router, check if it can handle 2 million total prefixes in the main routing table (RIB) and the forwarding information base (FIB). Also read the fine print, sometimes IPv6 prefixes can count double, and it's relatively common that the 2M limit applies to a combination of firewall rules as well as IPv4 and IPv6 prefixes.

There is usually a separate limit for the BGP RIB, which needs to be much larger, because unlike the main RIB and the FIB, the BGP RIB keeps multiple copies of every prefix: one for each BGP neighbor. So for the BGP RIB you'll want something like 8 million. However, the BGP RIB is usually only limited by available RAM.

Permalink - posted 2021-03-23

I did it again... I made another keyboard tester

▼ The other day I noticed how dirty my VT420 terminal's keyboard was, so I decided to take off the keycaps to clean them.

Then of course it's always good to put the keycaps back where they belong. That's a task made a lot easier with a keyboard tester app such as the online one I made a while ago. However, I don't think the VT420 supports the latest Javascript features...

So I whipped up a command line keyboard tester: kbdtest.php. Save the page, rename to kbdtest.php, then run with "php kbdtest.php". Works on the Mac, haven't tested other systems.

The script draws an image of the main part of a keyboard, with the keys that should work displayed in red. Then as you type them, they're made the default color and bright or bold. Typing the same key multiple times adds and removes an underline. The first number is a count of how many different keys you've typed, the second one only counts printable characters and tab, backspace and enter. (Upper vs lower case is ignored.) When you've typed all 71 of those, the program ends. You can of course also use control C. When there's 8 characters left, the remaining ones start to blink.

Note the box drawing characters. These are actually produced by issuing an ANSI escape code to switch to the DEC special graphics character set. In that mode, the lower case characters become line/box drawing characters. Turns out: this also works in the Terminal on the Mac!

I couldn't find a workable way to get the actual keyboard scan codes or something similar on the command line, so I can't do anything with keymaps, like the web keyboard tester does. However, turns out you can still read special keys such as the function keys and arrow keys. These produce ANSI escape codes. And again, that works in the Terminal exactly like it does on the VT420 terminal.

Permalink - posted 2021-03-16