Wednesday, 4 April 2012

BGP Route Refresh Capability

In this blog post we're going to talk about BGP route refresh capability and how this is affected by using the neighbor soft-reconfiguration feature on Cisco platforms.

BGP Route Refresh Capability
This capability allows a BGP instance to request a BGP peer to re-advertise all its routes towards the requesting BGP speaker.  This can be done without any disruption to the forwarding plane.

This can be used for a few purposes.  Firstly it can be used when there has been an update to the ingress routing policy, secondly it can be used to refresh routes when there has been some corruption of the BGP routing table.

Details of this BGP capability can be found in RFC 2918

Cisco Implementation
On Cisco platforms then BGP capabilities are negotiated when the neighbours come up.  To find out if route refresh capability is supported locally and by the BGP peer then use the following command

R1#sh ip bgp neighbors | s Neighbor capabilities:
  Neighbor capabilities:
    Route refresh: advertised and received(old & new)

To initiate a BGP route refresh then use one of the following commands

# clear ip bgp soft
# clear ip bgp in

At a packet level then the requesting BGP peer will send a ROUTE-REFRESH message (as shown below) and the responding peer will simply send a standard UPDATE message.

So the first part of this blog is a fairly straight forward explanation of this BGP capability, however there is a gottcha on Cisco platforms.  When we enable soft reconfiguration then BGP route refresh capability is no longer used, we will discuss this further near the end.

Soft Reconfiguration
To configure soft reconfiguration we do the following

(config) # router bgp 179
(config-rtr)# neighbor soft-reconfiguration inbound

Or we can also configure the router to configure BGP sessions with soft reconfiguration only when BGP route refresh capability is not supported by our BGP peer.

(config-rtr)# bgp soft-reconfig-backup

When we have soft reconfiguration configured then the Cisco router starts to keep a copy of adj-RIB-in, whereas normally it discards this information after the routes are passed to loc-RIB.  Full details about BGP can be found in RFC4271, however essentially

·         When BGP receives routes from a peer it places them into adj-RIB-in unaltered
·         Routes in adj-RIB-in are then passed through inbound routing policy and go into loc-RIB.  Loc-RIB is essentially the best paths of the BGP RIB.
·         Adj-RIB-out is a table of routes after they have passed through the outbound routing policy.  Therefore there is a adj-RIB-out for each peer-group.  On Cisco this is actually just pointer records back to the loc-RIB.

To view the contents of the adj-RIB-in then we use the following command

# sh ip bgp neighbors received-routes

Incidentally the following two commands will show the routes in the loc-RIB that come from a specific peer (ie post ingress routing policy) and also the contents of the adj-RIB-out for a specific peer respectively

# sh ip bgp neighbors routes
# sh ip bgp neighbors advertised-routes

And to refresh routes from the adj-RIB-in to the loc-RIB we can use the following commands

# clear ip bgp soft
# clear ip bgp in

So now that we have soft-reconfiguration turned on the above commands do NOT initiate a route refresh message as shown in the previous section.  Instead the commands refresh the routes from the adj-RIB-in.  Seeing as the adj-RIB-in and loc-RIB are both stored in memory on the same router, this mechanism is susceptible to memory corruption issues.

Route Refresh Capability vs Soft Reconfiguration

To summarise then I will try to note the pros and cons below.

When soft reconfiguration is enabled and there is some memory corruption on the adj-RIB-in then the only way to refresh the routes is to do a hard reset of the BGP peering (NOTE: actually if we have control of the remote peer we can still do a clear ip bgp out).  With BGP route refresh capability, if routes in the loc-RIB get corrupted we can refresh the routes dynamically without any disruption to service.  This may be a moot point, but consider the fact that BGP peers can stay up for years at a time and have full Internet routing tables.  For Tier 1 ISP networks this is a real issue.  For IGPs we continually refresh the routes across the network (ie every 30 mins for OSPF), this is not the case for BGP.  In fact if you have ever encountered strange routing issues which were solved by bouncing the BGP peer, this is probably the exact issue you saw.

Soft reconfiguration inbound will use more memory than route refresh capability.  However conversely, route refresh capability will use network bandwidth when we refresh routes, which can be significant on smaller links.

Soft reconfiguration inbound is useful to see the BGP routes learned before policy is applied.  Therefore it can be used for inbound policy troubleshooting.  I always use this feature when I bring up a new BGP peer so I can check the correct routes are being learnt before we allow the routes through our policy.

Lastly we should consider Cisco’s recommendation on this matter.  If we refer to the command reference here we can see the following statement “using the neighbor soft-reconfiguration command has a negative effect on network operations and should only be used as a last resort”.  Soft reconfiguration inbound predates BGP route refresh capability and we should use the new feature in production configurations.  However soft reconfiguration can still be useful for service turn up and troubleshooting.  Hopefully reading this blog will enable you to understand when to use each feature.

Want to learn how to automate your network with Python?

Please check my series of blog posts on how to easily script and automate various network equipment.  Easy step by step guides for Cisco, Juniper, Palo Alto and more ....

Python for Network Engineers