When redundant Layer 3 paths exist between two eBGP neighbors, the use of interface IP addresses for the underlying TCP connection can result in an outage when only one of the two links fails.
BGP neighborships fail when the underlying TCP connection fails.
TCP uses a concept called a socket, which consists of a local TCP port number and an IP address.
That IP address must be associated with a working interface (an interface whose state is line status up, line protocol up, per the show interfaces command).
If the interface whose IP address is used by BGP were to fail, then the TCP socket would fail, closing the TCP connection. As a result, the BGP neighborship can only be up when the associated interfaces also happens to be up.
One option would be to configure two neighbor commands on each router, one for each of the neighbor’s interface IP addresses - big bandwidth and memory consumption.
The preferred option, which uses loopback interfaces as the TCP connection endpoints, solves the availability problem while avoiding the extra overhead.
The two routers each configure a loopback interface and IP address, and use those loopback IP addresses as the source of their single BGP TCP connection.
If one of the multiple links fails, the loopback interface does not fail. As long as the two routers have working routes to reach each other’s loopback IP addresses, the TCP connection does not fail.
Configuring eBGP peers to use a loopback interface IP address with BGP requires several steps, as follows:
Step 1. Configure an IP address on a loopback interface on each router.
Step 2. Tell BGP on each router to use the loopback IP address as the source IP address using the neighbor... update-source ip-address command.
Step 3. Configure the BGP neighbor command on each router to refer to the other router’s loopback IP address at the neighbor IP address in the neighbor neighbor-ip remote-as command.
Step 4. Make sure each router has IP routes so that they can forward packets to the loopback interface IP address of the other router.
Step 5. Configure eBGP multihop using the neighbor... ebgp-multihop hops command. This command defines the TTL that the router will use when creating the BGP packets.
The fifth configuration step for using loopback IP addresses with eBGP peers refers to a feature called eBGP multihop. By default, when building packets to send to an eBGP peer, IOS sets the IP Time-To-Live (TTL) field in the IP header to a value of 1.
With this default action, the eBGP neighborship fails to complete when using loopback interface IP addresses.
Let's consider routers R1 and R101 and configure all the above options (steps):
R1(config)#interface loopback 1
*Mar 1 00:02:23.023: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
R1(config-if)#ip address 1.1.1.1 255.255.255.255
R1(config)#ip route 101.101.101.101 255.255.255.255 201.1.1.2
R101(config)#interface loopback 101
*Mar 1 00:02:49.035: %LINEPROTO-5-UPDOWN: Line protocol on Interface Loopback0, changed state to up
R101(config-if)#ip address 101.101.101.101 255.255.255.255
R101(config)#ip route 1.1.1.1 255.255.255.255 201.1.1.1
R1#ping 101.101.101.101
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 101.101.101.101, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/20/44 ms
R101#ping 1.1.1.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 1.1.1.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/18/52 ms
Now the BGP configuration (+ MD5 authentication):
R1(config)#router bgp 100
R1(config-router)#neighbor 101.101.101.101 remote-as 101
R1(config-router)#neighbor 101.101.101.101 update-source loopback 1
R1(config-router)#neighbor 101.101.101.101 ebgp-multihop 2
R1(config-router)#neighbor 101.101.101.101 password ccnp
R1(config-router)#
*Mar 1 00:06:19.835: %TCP-6-BADAUTH: No MD5 digest from 101.101.101.101(179) to 1.1.1.1(55651) (RST)
R101(config)#router bgp 101
R101(config-router)#neighbor 1.1.1.1 remote-as 100
R101(config-router)#neighbor 1.1.1.1 update-source loopback 101
R101(config-router)#neighbor 1.1.1.1 ebgp-multihop 2
R101(config-router)#neighbor 1.1.1.1 password ccnp
R1(config-router)#
*Mar 1 00:08:11.431: %BGP-5-ADJCHANGE: neighbor 101.101.101.101 Up
R101(config-router)#
*Mar 1 00:08:12.883: %BGP-5-ADJCHANGE: neighbor 1.1.1.1 Up
R1#show ip bgp summary
BGP router identifier 1.1.1.1, local AS number 100
BGP table version is 1, main routing table version 1
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
101.101.101.101 4 101 4 4 1 0 0 00:01:07 0
R101#show ip bgp summary
BGP router identifier 101.101.101.101, local AS number 101
BGP table version is 1, main routing table version 1
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
1.1.1.1 4 100 3 3 1 0 0 00:00:55 0
Note: The Neighbor column lists the IP address as defined on the local router’s neighbor command and not the neighbor’s BGP RID.
Note: The State/PfxRcd column either lists the BGP neighbor state or the number of prefixes received (PfxRcd) from that neighbor.
A numeric value under this heading implies a neighbor state of established, because the peers must be in established state before Updates can be sent. If the peer is not in an established state, the value in this heading lists the text name of the current BGP state.
BGP Neighbor States
Idle - The BGP process is either administratively down or awaiting the next retry attempt.
Connect - The BGP process is waiting for the TCP connection to be completed. You cannot determine from this state information whether the TCP connection can complete.
Active - The TCP connection has been completed, but no BGP messages have been sent to the peer yet.
Opensent - The TCP connection exists and a BGP Open message has been sent to the peer, but the matching Open message has not yet been received from the other router.
Openconfirm - An Open message has been both sent to and received from the other router. The next step is to receive a BGP Keepalive message (to confirm all neighbor-related parameters matched) or BGP Notification message (to learn there is some mismatch in neighbor parameters).
Established - All neighbor parameters match, the neighbor relationship works, and the peers can now exchange Update messages.
More information about a neighbor can be seen using:
R1#show ip bgp neighbors 101.101.101.101
BGP neighbor is 101.101.101.101, remote AS 101, external link
BGP version 4, remote router ID 101.101.101.101
BGP state = Established, up for 00:10:11
Last read 00:00:10, last write 00:00:10, hold time is 180, keepalive interval is 60 seconds
Neighbor capabilities:
Route refresh: advertised and received(old & new)
Address family IPv4 Unicast: advertised and received
Message statistics:
InQ depth is 0
OutQ depth is 0
Sent Rcvd
Opens: 1 1
Notifications: 0 0
Updates: 0 0
Keepalives: 12 12
Route Refresh: 0 0
Total: 13 13
Default minimum time between advertisement runs is 30 seconds
For address family: IPv4 Unicast
BGP table version 1, neighbor version 1/0
Output queue size : 0
Index 1, Offset 0, Mask 0x2
1 update-group member
Sent Rcvd
Prefix activity: ---- ----
Prefixes Current: 0 0
Prefixes Total: 0 0
Implicit Withdraw: 0 0
Explicit Withdraw: 0 0
Used as bestpath: n/a 0
Used as multipath: n/a 0
Outbound Inbound
Local Policy Denied Prefixes: -------- -------
Total: 0 0
Number of NLRIs in the update sent: max 0, min 0
Connections established 1; dropped 0
Last reset never
External BGP neighbor may be up to 2 hops away.
Connection state is ESTAB, I/O status: 1, unread input bytes: 0
Connection is ECN Disabled, Mininum incoming TTL 0, Outgoing TTL 2
Local host: 1.1.1.1, Local port: 179
Foreign host: 101.101.101.101, Foreign port: 13302
Enqueued packets for retransmit: 0, input: 0 mis-ordered: 0 (0 bytes)
Event Timers (current time is 0x10E064):
Timer Starts Wakeups Next
Retrans 13 0 0x0
TimeWait 0 0 0x0
AckHold 13 12 0x0
SendWnd 0 0 0x0
KeepAlive 0 0 0x0
GiveUp 0 0 0x0
PmtuAger 0 0 0x0
DeadWait 0 0 0x0
iss: 517553984 snduna: 517554258 sndnxt: 517554258 sndwnd: 16111
irs: 3069838838 rcvnxt: 3069839112 rcvwnd: 16111 delrcvwnd: 273
SRTT: 247 ms, RTTO: 663 ms, RTV: 416 ms, KRTT: 0 ms
minRTT: 4 ms, maxRTT: 300 ms, ACK hold: 200 ms
Flags: passive open, nagle, gen tcbs, md5
IP Precedence value : 6
Datagrams (max data segment is 536 bytes):
Rcvd: 15 (out of order: 0), with data: 13, total data bytes: 273
Sent: 25 (retransmit: 0, fastretransmit: 0, partialack: 0, Second Congestion: 0), with data: 12, total data bytes: 273
We can further verify TCP socket state and details using:
R1#show tcp brief
TCB Local Address Foreign Address (state)
65CD8A58 1.1.1.1.179 101.101.101.101.13302 ESTAB
Cisco IOS provides a means to administratively disable any BGP neighbor. Using the debug ip bgp command, BGP messages and states can be viewed:
R1#debug ip bgp
R1(config)#router bgp 100
R1(config-router)#neighbor 101.101.101.101 shutdown
*Mar 1 00:33:37.235: BGPNSF state: 101.101.101.101 went from nsf_not_active to nsf_not_active
*Mar 1 00:33:37.239: BGP: 101.101.101.101 went from Established to Idle
*Mar 1 00:33:37.239: %BGP-5-ADJCHANGE: neighbor 101.101.101.101 Down Admin. shutdown
*Mar 1 00:33:37.239: BGP: 101.101.101.101 closing
R1#show ip bgp summary
BGP router identifier 1.1.1.1, local AS number 100
BGP table version is 1, main routing table version 1
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
101.101.101.101 4 101 28 28 0 0 0 00:02:07 Idle (Admin)
R1(config)#router bgp 100
R1(config-router)#no neighbor 101.101.101.101 shutdown
*Mar 1 00:36:08.247: BGP: 101.101.101.101 went from Idle to Active
*Mar 1 00:36:08.263: BGP: 101.101.101.101 open active, local address 1.1.1.1
*Mar 1 00:36:08.335: BGP: 101.101.101.101 went from Active to OpenSent
*Mar 1 00:36:08.335: BGP: 101.101.101.101 sending OPEN, version 4, my as: 100, holdtime 180 seconds
*Mar 1 00:36:08.347: BGP: 101.101.101.101 send message type 1, length (incl. header) 45
*Mar 1 00:36:08.391: BGP: 101.101.101.101 rcv message type 1, length (excl. header) 26
*Mar 1 00:36:08.395: BGP: 101.101.101.101 rcv OPEN, version 4, holdtime 180 seconds
*Mar 1 00:36:08.395: BGP: 101.101.101.101 rcv OPEN w/ OPTION parameter len: 16
*Mar 1 00:36:08.399: BGP: 101.101.101.101 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
*Mar 1 00:36:08.399: BGP: 101.101.101.101 OPEN has CAPABILITY code: 1, length 4
*Mar 1 00:36:08.403: BGP: 101.101.101.101 OPEN has MP_EXT CAP for afi/safi: 1/1
*Mar 1 00:36:08.403: BGP: 101.101.101.101 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
*Mar 1 00:36:08.407: BGP: 101.101.101.101 OPEN has CAPABILITY code: 128, length 0
*Mar 1 00:36:08.407: BGP: 101.101.101.101 OPEN has ROUTE-REFRESH capability(old) for all address-families
*Mar 1 00:36:08.411: BGP: 101.101.101.101 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
*Mar 1 00:36:08.411: BGP: 101.101.101.101 OPEN has CAPABILITY code: 2, length 0
*Mar 1 00:36:08.415: BGP: 101.101.101.101 OPEN has ROUTE-REFRESH capability(new) for all address-families
BGP: 101.101.101.101 rcvd OPEN w/ remote AS 101
*Mar 1 00:36:08.419: BGP: 101.101.101.101 went from OpenSent to OpenConfirm
*Mar 1 00:36:08.419: BGP: 101.101.101.101 went from OpenConfirm to Established
*Mar 1 00:36:08.419: %BGP-5-ADJCHANGE: neighbor 101.101.101.101 Up
BGP Message Types
Open - Used to establish a neighbor relationship and exchange basic parameters, including ASN and MD5 authentication values.
Keepalive - Sent on a periodic basis to maintain the neighbor relationship. The lack of receipt of a Keepalive message within the negotiated Hold timer causes BGP to bring down the neighbor connection.
Update - Used to exchange PAs and the associated prefix/length (NLRI) that use those attributes.
Notification - Used to signal a BGP error; typically results in a reset to the neighbor relationship.
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment