Developing an Optimum Design for Layer 3
To achieve high availability and fast convergence in the Cisco enterprise campus network, the designer needs to manage multiple objectives, including the following:
This section reviews design models and recommended practices for high availability and fast convergence in Layer 3 of the Cisco enterprise campus network.
Managing Oversubscription and Bandwidth
Typical campus networks are designed with oversubscription, as illustrated in Figure 2-10. The rule-of-thumb recommendation for data oversubscription is 20:1 for access ports on the access-to-distribution uplink. The recommendation is 4:1 for the distribution-to-core links. When you use these oversubscription ratios, congestion may occur infrequently on the uplinks. QoS is needed for these occasions. If congestion is frequently occurring, the design does not have sufficient uplink bandwidth.
As access layer bandwidth capacity increases to 1 Gb/s, multiples of 1 Gb/s, and even 10 Gb/s, the bandwidth aggregation on the distribution-to-core uplinks might be supported on many Gigabit Ethernet EtherChannels, on 10 Gigabit Ethernet links, and on 10 Gigabit EtherChannels.
Bandwidth Management with EtherChannel
As bandwidth from the distribution layer to the core increases, oversubscription to the access layer must be managed, and some design decisions must be made.
Just adding more uplinks between the distribution and core layers leads to more peer relationships, with an increase in associated overhead.
EtherChannels can reduce the number of peers by creating single logical interface. However, you must consider some issues about how routing protocols will react to single link failure:
-
OSPF running on a Cisco IOS Software-based switch will notice a failed link, and will increase the link cost. Traffic is rerouted, and this design leads to a convergence event.
-
OSPF running on a Cisco Hybrid-based switch will not change link cost. Because it will continue to use the EtherChannel, this may lead to an overload in the remaining links in the bundle as OSPF continues to divide traffic equally across channels with different bandwidths.
-
EIGRP might not change link cost, because the protocol looks at the end-to-end cost. This design might also overload remaining links.
The EtherChannel Min-Links feature is supported on LACP EtherChannels. This feature allows you to configure the minimum number of member ports that must be in the link-up state and bundled in the EtherChannel for the port channel interface to transition to the link-up state. You can use the EtherChannel Min-Links feature to prevent low-bandwidth LACP EtherChannels from becoming active.
Bandwidth Management with 10 Gigabit Interfaces
Upgrading the uplinks between the distribution and core layers to 10 Gigabit Ethernet links is an alternative design for managing bandwidth. The 10 Gigabit Ethernet links can also support the increased bandwidth requirements.
This is a recommended design:
-
Unlike the multiple link solution, 10 Gigabit Ethernet links do not increase routing complexity. The number of routing peers is not increased.
-
Unlike the EtherChannel solution, the routing protocols will have the ability to deterministically select the best path between the distribution and core layer.
Link Load Balancing
In Figure 2-11, many equal-cost, redundant paths are provided in the recommended network topology from one access switch to the other across the distribution and core switches. From the perspective of the access layer, there are at least three sets of equal-cost, redundant links to cross to reach another building block, such as the data center.
Cisco Express Forwarding (CEF) is a deterministic algorithm. As shown in the figure, when packets traverse the network that all use the same input value to the CEF hash, a “go to the right” or “go to the left” decision is made for each redundant path. When this results in some redundant links that are ignored or underutilized, the network is said to be experiencing CEF polarization.
To avoid CEF polarization, you can tune the input into the CEF algorithm across the layers in the network. The default input hash value is Layer 3 for source and destination. If you change this input value to Layer 3 plus Layer 4, the output hash value also changes.
As a recommendation, use alternating hashes in the core and distribution layer switches:
-
In the core layer, continue to use the default, which is based on only Layer 3 information.
-
In the distribution layer, use the Layer 3 plus Layer 4 information as input into the CEF hashing algorithm with the command Dist2-6500 (config)#mls ip cef load-sharing full.
This alternating approach helps eliminate the always-right or always-left biased decisions and helps balance the traffic over equal-cost, redundant links in the network.
Link Load Balancing
EtherChannel allows load sharing of traffic among the links in the channel and redundancy in the event that one or more links in the channel fail.
You can tune the hashing algorithm used to select the specific EtherChannel link on which a packet is transmitted. You can use the default Layer 3 source and destination information, or you can add a level of load balancing to the process by adding the Layer 4 TCP/IP port information as an input to the algorithm.
Figure 2-12 illustrates some results from experiments at Cisco in a test environment using a typical IP addressing scheme of one subnet per VLAN, two VLANs per access switch, and the RFC 1918 private address space. The default Layer 3 hash algorithm provided about one-third to two-thirds utilization. When the algorithm was changed to include Layer 4 information, nearly full utilization was achieved with the same topology and traffic pattern.
The recommended practice is to use Layer 3 plus Layer 4 load balancing to provide as much information as possible for input to the EtherChannel algorithm to achieve the best or most uniform utilization of EtherChannel members. The command port-channel load-balance is used to present the more unique values to the hashing algorithm. This can be achieved using the command dist1-6500(config)#port-channel load-balance src-dst-port.
To achieve the best load balancing, use two, four, or eight ports in the port channel.
Routing Protocol Design
This section reviews design recommendations for routing protocols in the enterprise campus.
Routing protocols are typically deployed across the distribution-to-core and core-to-core interconnections.
Layer 3 routing design can be used in the access layer, too, but this design is currently not as common.
Layer 3 routing protocols are used to quickly reroute around failed nodes or links while providing load balancing over redundant paths.
Build Redundant Triangles
For optimum distribution-to-core layer convergence, build redundant triangles, not squares, to take advantage of equal-cost, redundant paths for the best deterministic convergence.
The topology connecting the distribution and core switches should be built using triangles, with equal-cost paths to all redundant nodes. The triangle design is shown in Figure 2-13 Model A, and uses dual equal-cost paths to avoid timer-based, nondeterministic convergence. Instead of indirect neighbor or route-loss detection using hellos and dead timers, the triangle design failover is hardware based and relies on physical link loss to mark a path as unusable and reroute all traffic to the alternate equal-cost path. There is no need for OSPF or EIGRP to recalculate a new path.
In contrast, the square topology shown in Figure 2-14 Model B requires routing protocol convergence to fail over to an alternate path in the event of a link or node failure. It is possible to build a topology that does not rely on equal-cost, redundant paths to compensate for limited physical fiber connectivity or to reduce cost. However, with this design, it is not possible to achieve the same deterministic convergence in the event of a link or node failure, and for this reason the design will not be optimized for high availability.
Peer Only on Transit Links
Another recommended practice is to limit unnecessary peering across the access layer by peering only on transit links.
By default, the distribution layer switches send routing updates and attempt to peer across the uplinks from the access switches to the remote distribution switches on every VLAN. This is unnecessary and wastes CPU processing time.
Figure 2-14 shows an example network where with 4 VLANs per access switch and 3 access switches, 12 unnecessary adjacencies are formed. Only the adjacency on the link between the distribution switches is needed. This redundant Layer 3 peering has no benefit from a high-availability perspective, and only adds load in terms of memory, routing protocol update overhead, and complexity. In addition, in the event of a link failure, it is possible for traffic to transit through a neighboring access layer switch, which is not desirable.
As a recommended practice, limit unnecessary routing peer adjacencies by configuring the ports toward Layer 2 access switches as passive, which will suppress the advertising of routing updates. If a distribution switch does not receive routing updates from a potential peer on a specific interface, it will not need to process these updates, and it will not form a neighbor adjacency with the potential peer across that interface.
There are two approaches to configuring passive interfaces for the access switches:
-
Use the passive-interface default command, and selectively use the no passive-interface command to enable a neighboring relationship where peering is desired.
-
Use the passive-interface command to selectively make specific interfaces passive.
Passive interface configuration example for OSPF:
Passive interface configuration example for EIGRP:
You should use whichever technique requires the fewest lines of configuration or is the easiest for you to manage.
Summarize at the Distribution Layer
A hierarchical routing design reduces routing update traffic and avoids unnecessary routing computations. Such a hierarchy is achieved through allocating IP networks in contiguous blocks that can be easily summarized by a dynamic routing protocol.
It is a recommended practice to configure route summarization at the distribution layer to advertise a single summary route to represent multiple IP networks within the building (switch block). As a result, fewer routes will be advertised through the core layer and subsequently to the distribution layer switches in other buildings (switch blocks). If the routing information is not summarized toward the core, EIGRP and OSPF require interaction with a potentially large number of peers to converge around a failed node.
Summarization at the distribution layer optimizes the rerouting process. If a link to an access layer device goes down, return traffic at the distribution layer to that device is dropped until the IGP converges. When summarization is used and the distribution nodes send summarized information toward the core, an individual distribution node does not advertise loss of connectivity to a single VLAN or subnet. This means that the core does not know that it cannot send traffic to the distribution switch where the access link has failed. Summaries limit the number of peers that an EIGRP router must query or the number of link-state advertisements (LSA) that OSPF must process, and thereby speeds the rerouting process.
Summarization should be performed at the boundary where the distribution layer of each building connects to the core. The method for configuring route summarization varies, depending on the IGP being used. Route summarization is covered in detail in Chapter 3, “Developing an Optimum Design for Layer 3.” These designs require a Layer 3 link between the distribution switches, as shown in Figure 2-15, to allow the distribution node that loses connectivity to a given VLAN or subnet the ability to reroute traffic across the distribution-to-distribution link. To be effective, the address space selected for the distribution-to-distribution link must be within the address space being summarized.
Summarization relies on a solid network addressing design.
0 comments
Post a Comment