BGP Graceful Shutdown

In this article

BGP sessions can drop for many reasons.  Dropping a session causes BGP to withdraw the routes learned through that neighbor. If it’s unexpected, this can cause congestion or other issues.  The graceful shutdown option in BGP can help mitigate this.

BGP graceful shutdown lets you move traffic before taking the session down. Rather than waiting for the neighbor to disappear, you signal that the routes should no longer be preferred. The goal is to shift traffic to another path before maintenance begins so you don’t get any unexpected surprises.

Graceful Shutdown Changes Routing on Your Schedule

A normal BGP shutdown removes the routes learned through that session. If those routes were being used to forward traffic, the router needs to find another path. Depending on the network and its BGP policies, that change can cause packet loss while routing converges.

With graceful shutdown, the routes remain available while their preference is reduced. Routers that have another usable path can move traffic to it before the BGP session goes down. You still need another path. Graceful shutdown cannot maintain reachability if the session being shut down is the only route to the destination.

The GRACEFUL_SHUTDOWN Community

BGP graceful shutdown uses a well-known community defined in RFC 8326. The community is:

65535:0

Its name is GRACEFUL_SHUTDOWN.

When a router advertises a route with this community, it signals that the path is being prepared for removal. A neighboring network that supports the feature can match the community and lower the route’s local preference. RFC 8326 recommends setting local preference to 0 when processing the community. That allows other available routes to win the BGP decision process. The route carrying the community remains available if no better path exists. RFC Editor The receiving network must have policy that handles the community. Simply attaching 65535:0 does not guarantee that the neighbor will change its routing decisions.

Moving Traffic Away From a Peering Session

Suppose an ISP has two connections to the same upstream provider. One is in Indianapolis, and the other is in Chicago. Both sessions advertise 203.0.113.0/24.

The ISP needs to perform maintenance on the Indianapolis router. Rather than immediately shutting down BGP, it applies the graceful shutdown community to routes advertised over the Indianapolis session.

If the upstream knows about the community, it lowers the local preference of those routes. The Chicago path can become preferred while Indianapolis remains in the table. Traffic can move to Chicago before the Indianapolis session is disabled. This gives you, as the operator, more control over when traffic changes.  

There is another direction to consider. The ISP also receives routes from the upstream through Indianapolis. To move its outbound traffic, the ISP can lower the local preference of routes learned through that session.

Both sides need to move traffic away from the connection. Changing the preference in only one direction can leave traffic using Indianapolis until the session finally drops.

Using Graceful Shutdown at an Internet Exchange

An ISP connected to FD-IX may have several paths to the same destination. Some routes come from bilateral peers, while others arrive through the route servers. Transit may provide another path to those destinations.

If you need to take an FD-IX connection offline, graceful shutdown can help move traffic away from that connection before maintenance begins. For routes you learn from the exchange, you can lower local preference on the affected session. If another route exists through a different connection, BGP will select it through the magic of BGP. For inbound traffic, it depends on the neighboring network’s policy. Remember we can control our outbound, but can only influence the inbound.

With bilateral peering, the other network must recognize the graceful shutdown community. With route-server peering, you also need to know whether the route server preserves and distributes the community and whether receiving members honor it.

Graceful Shutdown Is Not Graceful Restart

BGP graceful shutdown and graceful restart solve different problems.

Graceful shutdown prepares the network for a session going down. Routes receive lower preference so traffic can move elsewhere before the session is removed. Shutdown moves traffic with the end goal of the connection shutting down.

With graceful restart, the BGP session can drop without the neighbor immediately removing all the routes it learned from that router. Instead, the neighbor holds those routes while waiting for the session to come back up. This assumes the restarting router is still forwarding traffic. If the circuit is down or the router has stopped forwarding packets, traffic could end up going nowhere. The neighbor starts a restart timer when the session drops. If BGP comes back up before that timer expires, the neighbor accepts the new updates and removes stale routes once the routing table has been refreshed. If the session never comes back, the timer expires and the neighbor removes the routes it was holding.

j2networks family of sites
https://j2sw.com
https://startawisp.info
https://indycolo.net
#packetsdownrange #routethelight