The effectiveness of AS path prepending

▼ In a recent blog post The Effectiveness of AS Path Prepending (1) Russ White asks:

Just about everyone prepends AS’ to shift inbound traffic from one provider to another—but does this really work?

(AS path prepending means making the network path as BGP sees it longer to make a path less attractive so traffic will flow over another, shorter path.)

That's an interesting question, as I've been telling people for a long time that it often works too well. A single prepend can flip more than half your traffic from one link to another. In part because of that, Rolf Winter and I worked on a new Origin Preference Attribute that would provide a finer-grained tool for BGP traffic engineering. (Slides, old Internet-Draft, SAC'12 publication.)

But the answer that Russ cites from research into AS path prepending makes sense:

We observe that the effectiveness of prepending can strongly depend on the location (for around 20% of cases, ASPP has moved no targets, while for another 20% , it moved almost all targets).

The the fact that different networks get such different results when performing AS path prepending can be explained by looking at the type of network and the type of connectivity they have. Suppose a small network X has connections to big ISPs A and B. A and B have direct peering relations with most of the other large ISPs. So to the rest of the world, the two paths to X are the same length:

A X = 2 hops
B X is 2 hops

When X now prepends towards A, it's A X X = 3 hops vs B X = 2 hops so most traffic will now flow through the B X path. Prepending was extremely effective.

But what if instead of X doing prepending, A prepends. Now for traffic to X, it's still:

A A X = 3 hops
B X = 2 hops

But other networks that directly connect to A and have traffic for A itself, it doesn't matter if they see A = 1 hop or A A = 2 hops: the traffic needs to go to A, so the path length isn't considered. Even when network C is a customer of B and C sees a direct path and a path over B:

A A A = 3 hops
B A = 2 hops

C will use the direct path, even though it's longer, because the rule is: paths to customers are prioritized over paths to peers and paths to peers over paths to transit providers.

So: will AS path prepending be effective? As with most things in life, it depends.

Permalink - posted 2021-05-13

Hunting down the stuck BGP routes

▼ Ben Cox (Benjojo) has an interesting post about stuck BGP routes and a flaw in many BGP implementations where they hang when their neighbor stops accepting data over TCP: Hunting down the stuck BGP routes

A stuck BGP route means that a prefix was advertised at some point, and then it's withdrawn but the withdrawal somehow gets lost somewhere, so part of the internet still sees the withdrawn route.

There doesn't seem to be an obvious way to get these prefixes unstuck, although I would try advertising them again and withdrawing them again. The AS in the middle where the prefixes are stuck should probably clear (reset) BGP sessions until the problem is fixed, although I would be tempted to just reboot the whole router just to be safe.

The hanging issue happens when a router gets behind on processing incoming BGP updates. At some point its TCP receive buffer fills up, so it sets its TCP window size to 0. Then the buffer on the sending BGP neighbor starts to fill up with updates and keepalives. When that buffer is full, the sending BGP process can't write data to the TCP socket anymore... and bad things start to happen.

Ben feels that the BGP spec needs to be updated to address this issue. However, this is not a matter of interoperation that needs to be standardized, but simply an implementation issue that each vendor can fix on their own, in my opinion.

Permalink - posted 2021-04-22

Getting the "Music" miniplayer back to actual mini

▼ I just had a moderate-size scare: I typed something into iTunes Music, and the miniplayer now showed playback controls:

That's not what I want. I want it to show the album artwork. That's how I've used iTunes and now Music for many years. If I need to interact with the controls, those will then come up when I place my mouse pointer over the miniplayer window. In the view menu, there is no option to get back to my usual setup.

But it's pretty easy to do this if you know how: just drag the bottom corner of the miniplayer window up until the window is roughly rectangular, and after releasing the mouse button:

This is actually quite a good way to change the layout of the miniplayer. (You can also make the window even smaller or even larger to get different results.) However, it's also extremely un-discoverable.

A free tip, from me to you.

Permalink - posted 2021-04-13

VLANs on the Mikrotik hAP ac³

▼ For my BGP lab/training setups I have a bunch of routers, real or virtual, that each need several IP addresses to talk to other routers. VLANs are the perfect way to keep all of this manageable: with VLANs, it's possible to have separate IP interfaces, but still just use a single Ethernet port to hook everything up. Simple unmanaged switches simply forward the packets to the right port without looking at the VLAN header, so no need to configure the network.

Until I got my Mikrotik hAP ac³ this week...

In its default configuration it will do what unmanaged switches do: ignore the VLAN header. In addition, you can add VLAN interfaces to the Mikrotik and give those an IP address. I even got IPv6 prefix delegation to work. For this, set up the DHCPv6 client to request a prefix. Then fill in the name of a pool. But don't create that pool yourself—that won't work. The pool will be created automatically. Then configured an IPv6 on a VLAN interface that I created and specified that the pool should be used for that. "Advertise" is enabled by default, which means that the Mikrotik will send out router advertisements and machines that have the VLAN configured will configure an IPv6 address for themselves on that VLAN interface.

Eventually I want to set up a guest network with its own VLAN and IPv4 NAT. For now I just set up IP addresses on the VLAN interfaces so the Mikrotik will be able to talk to my virtual BGP routers on these VLANs. That initially went well.

However, I also wanted have these BGP VMs talk to an old Cisco 2503 router, which doesn't support VLANs. So I wanted to set up an access port. The idea was that the Cisco would be part of VLAN 203 without the need for it to insert a VLAN TAG itself. On the other ports, VLAN 203 (and a few more) get a VLAN header. However, not exactly like a trunk port where all packets have a VLAN header: regular traffic to/from the internet goes out on these same ports untagged.

Mixed tagged/untagged ports turned out to be unsupported by the Atheros 8327 chip that powers the switch ports. What you can do is have one or more access ports and one or more trunk ports. This is already somewhat complex. The access ports need to be set up with a default VLAN. (Under "switch" and then "ports".) Makes sense. In the documentation, it says to use add-if-missing will add a VLAN header if there isn't one. Makes sense. However, it turns out that you need to do this on the outgoing port, i.e., not the access port itself, but the trunk port(s) that are part of the same VLAN. However, then (with a slight delay?) untagged traffic from the default VLAN won't work anymore.

The good news is that even though the hardware can't have mixed ports, the software can. So remove those settings above, and go to the bridge section of the configuration instead. There, select your bridge instance and turn on VLAN filtering. After that, you can go to the VLAN tab an add one or more VLANs there, where you can specify which ports should be tagged and untagged for that VLAN. So now we can have access ports and mixed ports. I do believe performance will be lower because VLAN traffic will need to be handled in software. Also not the PVID setting, which is the default VLAN untagged traffic. (1 out of the box.)

Unfortunately, the Mikrotik's IP addresses on the VLAN interface I tested with were now unreachable, and I couldn't figure out any settings to fix that. So after I was done with the Cisco 2503, I unchecked the "VLAN filtering" in the bridge settings... and the Mikrotek's IP addresses on VLAN interfaces came back to life.

Permalink - posted 2021-04-04