iljitsch.com

topics: all · BGP / IPv6 / more · settings · b&w · my business: inet⁶ consult · Twitter · Mastodon · LinkedIn · email · 🇺🇸 🇳🇱

Hi, I'm Iljitsch van Beijnum. This page has all posts about all subjects.

Packet reordering

During the second week of April there was some discussion on reordering of packets on parallel links at Internet Exchanges. Equipment vendors try very hard to make sure this doesn't happen, but this has the risk that balancing traffic over parallel links doesn't work as good as it should. It is generally accepted that reordering leeds to inefficiency or even slowdowns in TCP implementations, but it seems unlikely reordering will happen much hosts are connected at the speed of the parallel links (ie, Gigabit Ethernet) or there is significant congestion.

Permalink - posted 2002-06-29

Telephony best practices for IP

In March, the NANOG list focussed some of its attention on efforts of the telephony world to com up with "best practices" about packet networks/the internet. (Network Reliability and Interoperability Council publications) In meeting minutes, 100% of the carriers reported to abide by these best practices, which include firewalling routers and DNS servers.

There were discussions about the merits of filtering/firewalling at OC-192 (10 Gbps) speeds (which, depending on your definition of "firewall" may be impossible to do) and about out-of-band versus in-band management. The former is always used in the telephony world, the latter often in IP. The main problem with out of band management is that the management network may be unavailable when the network is in critical need of being "managed". Also, many vendors do not support a clear separation between production and for-management interfaces.

Permalink - posted 2002-06-28

April fools day RFCs

April fools day is coming up again! Don't let it catch you by surprise. Over the years, a number of RFCs have been published on April first, such as...

Full article / permalink - posted 2002-04-01

TCP high performance and maximum usable bandwidth

The second half of Februari saw two main topics on the NANOG list: DS3 performance and satellite latency. The long round trip times for satellite connections wreak havoc on TCP performance. In order to be able to utilize the available bandwidth, TCP needs to keep sending data without waiting for an acknowledgment for at least a full round trip time. Or in other words: TCP performance is limited to the window size multiplied by the round trip time. The TCP window (amount of data TCP will send before stopping and waiting for an acknowledgment) is limited by two factors: the send buffer on the sending system and the 16 bit window size field in the TCP header. So on a 600 ms RTT satellite link the maximum TCP performance is limited to 107 kilobytes per second (850 kbps) by the size of the header field, and if a sender uses a 16 kilobyte buffer (a fairly common size) this drops to as little as 27 kilobytes per second (215 kbps). Because of the TCP slow start mechanism, it takes several seconds to reach this speed as well. Fortunately, RFC 1323, TCP Extensions for High Performance introduces a "window scale" option to increase the TCP window to a maximum of 1 GB, if both ends of the connection allocate enough buffer space.

The other subject that received a lot of attention, the maximum usable bandwidth of a DS3/T3 line, is also related to TCP performance. When the line gets close to being fully utilized, short data bursts (which are very common in IP) will fill up the send queue. When the queue is full, additional incoming packets are discarded. This is called a "tail drop". If the TCP session which loses a packet doesn't support "fast retransmit", or if several packets from the same session are dropped, this TCP session will go into "slow start" and slow down a lot. This often happens to several TCP sessions at the same time, so those now all perform slow start at the same time. So they all reach the point where the line can't handle the traffic load at the same time, and another small burst will trigger another round of tail drops.

A possible solution is to use Random Early Detect (RED) queuing rather than First In, First Out (FIFO). RED will start dropping more and more packets as the queue fills up, to trigger TCP congestion avoidance and slow down the TCP sessions more gently. But this only works if there aren't (m)any tail drops, which is unlikely if there is only limited buffer space. Unfortunately, Cisco uses a default queue size of 40 packets. Queuing theory tells us this queue will be filled entirely (on average) at 97% line utilization. So at 97%, even a one packet burst will result in a tail drop. The solution is to increase the queue size, in addition to enabling RED. On a Cisco:

interface ATM0
 random-detect
 hold-queue 500 out

This gives RED the opportunity to start dropping individual packets long before the queue fills up entirely and tail drops occur. The price is a somewhat longer queuing delay. At 99% utilization, there will be an average of 98 packets in the queue, but at 45 Mbps this will only introduce a delay of 9 ms.

Permalink - posted 2002-03-31

BGP and geography

CAIDA, the Cooperative Association for Internet Data Analysis, has an interesting page with geographical, demographical and many BGP-related statistics per continent and country. As expected, South America, Africa and Asia hardly show up, but there are some surprises. For instance, the US uses more than three times as many address as the whole of Europe.

Permalink - posted 2002-03-31

Radware Peerdirector

It seems more and more products that try to optimize BGP are reaching the market. One is the Radware Peerdirector. There is a whitepaper (PDF) about how the Peerdirector works on the site. This product monitors bandwidth use and reconfigures routers to change prefix advertisements to different peers to optimize how the available bandwidth is used.

Of course, such a product doesn't do anything a network manager can't do manually. However, periodically changing information in BGP introduces instabilities into the global routing system, which is undesirable at a minimum, and may even pose real harm under certain conditions.

Permalink - posted 2002-03-30

older posts - newer posts

Search for:
RSS feed

Archives: 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023, 2024, 2025