Configuring Link Aggregation with EtherChannel
In a network where resources might be located far from where users might need them, some links between switches or between switches and servers become heavily solicited. These links’ speed can be increased to a certain point but not ad infinitum. EtherChannel is a technology that enables circumventing this issue by creating logical links made of several physical links.
This section examines the benefits of EtherChannel and the various technologies available to implement it and also the types of EtherChannel protocol. In addition to that, it explains how to configure Layer 2 EtherChannels and how to load balance traffic between physical links inside a given EtherChannel bundle. EtherChannels can also operate in a Layer 3 mode. The following topics are discussed in detail in the next section:
-
Understand the benefits of EtherChannel.
-
Compare the Port Aggregation Protocol (PAgP) and the Link Aggregation Protocol (LACP).
-
Given an enterprise VLAN network design that contains Layer 2 EtherChannel links, configure and verify EtherChannels.
-
EtherChannel load balancing options.
Describe EtherChannel
The increasing deployment of higher-speed switched Ethernet to the desktop attributes to the proliferation of bandwidth-intensive intranet applications. Any-to-any communications of new intranet applications, such as video to the desktop, interactive messaging, VoIP, and collaborative white-boarding, are increasing the need for scalable bandwidth within the core and at the edge of campus networks. At the same time, mission-critical applications call for resilient network designs. With the wide deployment of faster switched Ethernet links in the campus, users need to either aggregate their existing resources or upgrade the speed in their uplinks and core to scale performance across the network backbone.
Traffic coming from several VLANs at 100 Mbps aggregate on the access switches at the bottom and need to be sent to distribution switches in the middle. Obviously, bandwidth larger than 100 Mbps must be available on the link between two switches to accommodate the traffic load coming from all the VLANs. A first solution is to use a faster port speed, such as 1 Gbps or 10 Gbps. As the speed increases on the VLANs links, this solution finds its limitation where the fastest possible port is no longer fast enough to aggregate the traffic coming from all VLANs. A second solution is to multiply the numbers of physical links between both switches to increase the overall speed of the switch-to-switch communication. A downside of this method is that there must be a strict consistency in each physical link configuration. A second issue is that spanning tree may block one of the links, as shown in Figure 2-20. In Figure 2-20, the second links are in spanning-tree blocking mode between access and distribution.
EtherChannel is a technology that was originally developed by Cisco as a LAN switch-to-switch technique of grouping several Fast or Gigabit Ethernet ports into one logical channel, as shown in Figure 2-21, where now all the links between access and distribution switches are bundled into EtherChannel and in forwarding mode. This technology has many benefits:
-
It relies on the existing switch ports: There is no need to upgrade the switch-to-switch link to a faster and more expensive connection.
-
Most of the configuration tasks can be done on the EtherChannel interface instead of each individual port, thus ensuring configuration consistency throughout the switch-to-switch links.
-
EtherChannel provides redundancy: As the overall link is seen as one logical connection, the loss of one physical link does not create a change in the topology. Spanning-tree recalculation does not need to take place. As long as at least one physical link is present, the EtherChannel is functional, even if its overall throughput decreases.
-
Load balancing is possible between the links part of the same EtherChannel. Depending on the hardware platform, you can implement one or several methods, such as source-MAC to destination-MAC or source-IP to destination-IP load balancing across the physical links.
Up to eight physical links can be bundled together into a single logical EtherChannel link. Keep in mind that the logic of EtherChannel is to increase the bandwidth between switches. EtherChannel cannot only be supported in switches but also is very popular in nonswitch devices such as servers. In any case, EtherChannel creates a logical point-to-point link. EtherChannel links can be between two switches or between an EtherChannel-enabled server and a switch but cannot send traffic to two different switches through the same EtherChannel link. One EtherChannel link always connects two devices only, and these devices’ individual port configurations must be consistent. If the physical ports of one side are configured as trunks, the physical ports of the other side must also be configured as trunks. Each EtherChannel has a logical port channel interface. A configuration applied to the port channel interface affects all physical interfaces assigned to that interface. (Such commands can be STP commands or commands to configure a Layer 2 EtherChannel as a trunk or an access port.)
The EtherChannel technology can be used to bundle ports of the same type. On a Layer 2 switch, EtherChannel is used to aggregate access ports or trunks. Keep in mind that EtherChannel creates an aggregation that is seen as one logical link. When several EtherChannel bundles exist between two switches, spanning tree might block one of the bundles to prevent redundant links. When spanning tree blocks one of the redundant links, it blocks one EtherChannel, thus blocking all the ports belonging to this EtherChannel link. Where there is only one EtherChannel link, all physical links in the EtherChannel are active because spanning tree sees only one (logical) link.
On Layer 3 switches, convert the ports to routed ports. EtherChannel links can also be created on Layer 3 links. This functionality is highlighted in Chapter 4, “Implementing Inter-VLAN Routing.”
PAgP and LACP Protocols
EtherChannel uses one of the two management protocols to create Port-Channel. These protocols ensure that when EtherChannel is created, all ports have the same type of configuration. In EtherChannel, it is mandatory that all ports have the same speed, duplex setting, and VLAN information. Any port modification after the creation of the channel also changes all the other channel ports. The following are the two known protocols to create EtherChannel:
-
The Port Aggregation Protocol (PAgP) is a Cisco Proprietary protocol that aids in the automatic creation of Fast EtherChannel links. When an EtherChannel link is configured using PAgP, PAgP packets are sent between EtherChannel–capable ports to negotiate the forming of a channel. When PAgP identifies matched Ethernet links, it groups the links into an EtherChannel. Spanning tree adds the EtherChannel as a single bridge port.
-
LACP is part of an IEEE specification (802.3ad) that also enables several physical ports to be bundled together to form an EtherChannel. LACP enables a switch to negotiate an automatic bundle by sending LACP packets to the peer. It performs a similar function as PAgP with Cisco EtherChannel. Because LACP is an IEEE standard, you can use it to facilitate EtherChannels in mixed-switch environments. In a Cisco environment, both protocols are supported.
The following subsections discuss each protocol in more detail.
PAgP Modes
PAgP packets are sent every 30 seconds. PAgP checks for configuration consistency and manages link additions and failures between two switches.
The PAgP helps create the EtherChannel link by detecting each side’s configuration and making sure they are compatible. Table 2-7 shows the different settings for PAgP.
Table 2-7: Different Setting Options for PAgP
Open table as spreadsheet | Mode | Purpose |
| Auto | Places an interface in a passive negotiating state in which the interface responds to the PAgP packets that it receives but does not initiate PAgP negotiation (default). |
| Desirable | Places an interface in an active negotiating state in which the interface initiates negotiations with other interfaces by sending PAgP packets. |
| On | Forces the interface to channel without PAgP. Interfaces configured in the “on” mode do not exchange PAgP packets. |
| Non-silent | If a switch is connected to a partner that is PAgP-capable, configure the switch interface for non-silent operation. The non-silent keyword is always used with the auto or desirable mode. If you do not specify non-silent with the auto or desirable mode, silent is assumed. The silent setting is for connections to file servers or packet analyzers; this setting enables PAgP to operate, to attach the interface to a channel group, and to use the interface for transmission. |
| Note | Table 2-7 depicts the mode for PagP. The names of the modes are similar to the DTP protocol, but they have different functions. |
As shown in Figure 2-22, the modes need to be compatible on each side. Figure 2-22 illustrates the modes option setting on both sides and its results. If one configures a side to be in Auto mode, it will be placed in a passive state, waiting for the other side to initiate the EtherChannel negotiation. If the other side is also set to Auto, the negotiation never starts and the EtherChannel does not form. If one deconfigures all modes by using the no command or if no mode is configured, the interface is placed in an Off mode and EtherChannel is disabled.
Notice the “On” form. This command manually places the interface in EtherChannel without any negotiation. It works only if the other side is also set to On. If the other side is set to negotiate parameters through PAgP, no EtherChannel will form because the On side will not negotiate.
LACP Modes
LACP provides the same negotiation benefits as PAgP. LACP helps create the EtherChannel link by detecting each side’s configuration and making sure they are compatible so that the EtherChannel link can be enabled when needed. Table 2-8 shows the different settings for LACP.
Table 2-8: LACP Modes Options
Open table as spreadsheet | Mode | Purpose |
| Passive | Places a port in a passive negotiating state. In this state, the port responds to the LACP packets that it receives but does not initiate LACP packet negotiation (default). |
| Active | Places a port in an active negotiating state. In this state, the port initiates negotiations with other ports by sending LACP packets. |
| On | Forces the interface to the channel without PAgP or LACP. |
Just like with PAgP, modes must be compatible on both sides for the EtherChannel link to form, as shown in Figure 2-23. Figure 2-23 displays the different options setting on both sides of the links and its results if EtherChannel is created. The On form is mentioned here again because it creates an EtherChannel configuration unconditionally, or LACP dynamic negotiation.
The following are some additional parameters that you can use when configuring LACP:
-
System priority: Each switch running LACP must have a system priority. The system priority can be specified automatically or through the command-line interface (CLI). The switch uses the MAC address and the system priority to form the system ID.
-
Port priority: Each port in the switch must have a port priority. The port priority can be specified automatically or through the CLI. The port priority and the port number form the port identifier. The switch uses the port priority to decide which ports to put in standby mode when a hardware limitation prevents all compatible ports from aggregating.
-
Administrative key: Each port in the switch must have an administrative key value, which can be specified automatically or through the CLI. The administrative key defines the capability of a port to aggregate with other ports, determined by these factors:
-
The port physical characteristics, such as data rate, duplex capability, and point-to-point or shared medium
-
The configuration constraints that you establish
| Note | All the preceding options of LACP are optional to configure. Usually, defaults are the best to use. To configure any of these options, refer to your configuration guide. |
When enabled, LACP attempts to configure the maximum number of compatible ports in a channel. In some instances, LACP cannot aggregate all the compatible ports. For example, the remote system might have more restrictive hardware limitations. When this occurs, all the ports that cannot be actively included in the channel are put in a hot standby state and used only if one of the channeled ports fails.
Configure Port Channels Using EtherChannel
Before implementing EtherChannel in a network, a network engineer should plan the steps necessary to make it successful. Prior planning can help reduce any problems during the installation by logically organizing the steps necessary and providing checkpoints and verification, as necessary. The following guideline is helpful for planning the EtherChannel:
-
The first step is to identify the ports for the EtherChannel on both switches. This helps identify any issues with previous configurations on the ports and ensures that the proper connections are available.
-
The network designer should already have decided whether this is going to be a Layer 3 or a Layer 2 connection. If it is a Layer 2 connection, each interface should have the appropriate protocol identified (PAgP or LACP), have a channel group number to associate all the given interfaces to a port group, and know whether negotiations should occur.
-
If this is a Layer 3 connection, a new virtual interface is created. This port-channel interface is then given an IP address. Each of the physical interfaces is then made into an EtherChannel by specifying the same channel group number as the port-channel interface number.
-
When the connections are established, a couple of commands can ensure that both sides of the EtherChannel have formed and are providing aggregated bandwidth.
Guidelines for Configuring EtherChannel
This subsection describes the recommendations and guidelines for configuring EtherChannel. Follow these guidelines and restrictions when configuring EtherChannel interfaces:
-
EtherChannel support: All Ethernet interfaces on all modules support EtherChannel (maximum of eight interfaces) with no requirement that interfaces be physically contiguous or on the same module.
-
Speed and duplex: Configure all interfaces in an EtherChannel to operate at the same speed and in the same duplex mode. Also, if one interface in the bundle is shut down, it is treated as a link failure, and traffic traverses other links in the bundle.
-
Switched port analyzer (SPAN) and EtherChannel: An EtherChannel does not form if one of the interfaces is a SPAN destination port.
-
Layer 3 EtherChannels: Assign Layer 3 addresses to the port-channel logical interface, not to the physical interfaces in the channel.
-
VLAN match: All interfaces in the EtherChannel bundle must be assigned to the same VLAN or be configured as a trunk. In addition, Native VLANs should be matched across all the links on both switches.
-
Range of VLANs: An EtherChannel supports the same allowed range of VLANs on all the interfaces in a trunking Layer 2 EtherChannel. If the allowed range of VLANs is not the same, the interfaces do not form an EtherChannel, even when set to auto or desirable mode. For Layer 2 EtherChannels, either assign all interfaces in the EtherChannel to the same VLAN or configure them as trunks.
-
STP path cost: Interfaces with different STP port path costs can form an EtherChannel as long as they are otherwise compatibly configured. Setting a different STP port path costs does not, by itself, make interfaces incompatible for the formation of an EtherChannel.
-
Port channel versus interface configuration: After you configure an EtherChannel, any configuration that you apply to the port-channel interface affects the EtherChannel. Any configuration that you apply to the physical interfaces affects only the specific interface you configured.
Layer 2 EtherChannel Configuration Steps
Configuring a Layer2 EtherChannel requires the following three or four steps:
Example 2-20 shows a Layer 2 EtherChannel bundle between two switches using LACP. As shown in Figure 2-24, each switch shows two ports that are left to their default configuration. The switch on the left created EtherChannel 2, and the switch on the right created EtherChannel 5. These numbers are locally significant and do not need to match the partner’s configuration. The actual configuration of the link is then conducted on the EtherChannel interface. The link is configured to be unconditionally in 802.1Q trunk mode, and some VLANs are pruned.
Example 2-20: EtherChannel Configuration
Switch(config)# interface fastethernet 0/23
Switch(config-if)# channel-group 2 mode active
Switch(config)# interface fastethernet 0/24
Switch(config-if)# channel-group 2 mode active
Switch(config)# interface port-channel 2
Switch(config-if)# switchport mode trunk
Switch(config-if)# switchport trunk native VLAN 99
Switch(config-if)# switchport trunk allowed VLAN 2,3,99
!!!Remote Switch configuration
RSwitch(config)# interface fastethernet 0/23
RSwitch(config-if)# channel-group 5 mode on
RSwitch(config)# interface fastethernet 0/24
RSwitch(config-if)# channel-group 5 mode on
RSwitch(config)# interface port-channel 5
RSwitch(config-if)# switchport mode trunk
RSwitch(config-if)# switchport trunk native VLAN 99
RSwitch(config-if)# switchport trunk allowed VLAN 2,3,99
Example 2-20 follows these guidelines from the previous guidelines:
-
Speed and duplex: Configure all interfaces in an EtherChannel to operate at the same speed and in the same duplex mode.
-
VLAN match: All interfaces in the EtherChannel bundle must be assigned to the same VLAN or be configured as a trunk. Also make sure that all the interfaces are part of the same native VLANs on both switches.
-
Range of VLANs: An EtherChannel supports the same allowed range of VLANs on all the interfaces in a trunking Layer 2 EtherChannel.
Keep in mind that the EtherChannel interface configuration must be compatible with the underlying physical ports’ configuration. In the preceding example, initially there is no specific configuration on each individual port for trunking, which implies that the default dynamic auto mode is applied. In this mode, ports detect whether the other side is a trunk and dynamically changes to trunk mode if needed. This mode is compatible with the trunk mode configured on the EtherChannel interface. The physical ports inherit the EtherChannel configuration and change to trunk mode.
Verifying EtherChannel
This subsection discuss how to verify EtherChannel is correctly established and working. You can use several commands to verify an EtherChannel configuration. On any physical interface member of an EtherChannel bundle, the show interfaces type port/mod EtherChannel can provide information on the role of the interface in the EtherChannel, as shown in Example 2-21.
In Example 2-21, the interface fastethernet 0/24 is part of the EtherChannel bundle 1. The protocol for this EtherChannel is LACP.
Example 2-21: EtherChannel Verification Command for Interface
Switch# show interfaces fa0/24 etherchannel
Port state = Up Sngl-port-Bndl Mstr Not-in-Bndl
Channel group = 1 Mode = Active Gcchange = -
Port-channel = null GC = - Pseudo port-channel = Po1
Port index = 0 Load = 0x00 Protocol = LACP
Also, the show etherchannel number port-channel can be used to display information about a specific port-channel.
In Example 2-22, Port-channel 1 consists of two physical ports, fa0/23 and fa0/24. It uses LACP in active mode. It is properly connected to another switch with a compatible configuration. This is why the port-channel is said to be in use.
Example 2-22: EtherChannel Verification Port-Channel Command
Switch#show etherchannel 1 port-channel
Port-channels in the group:
---------------------------
Port-channel: Po7 (Primary Aggregator)
Age of the Port-channel = 195d:03h:10m:44s
Logical slot/port = 0/1 Number of ports = 2
Port state = Port-channel Ag-Inuse
Protocol = LACP
Ports in the Port-channel:
Index Load Port EC state No of bits
------+------+------+------------------+-----------
0 55 fa0/23 Active 4
1 45 fa0/24 Active 4
When several port-channel interfaces are configured on the same device, the show etherchannel summary command is useful for simply displaying one-line information per port-channel, as shown in Example 2-23.
In Example 2-23, the switch has three EtherChannels configured, group 2 and group 7 use LACP, and group 9 uses PAgP. Each EtherChannel has the member interfaces listed as well. It also indicates that all three groups are Layer 2 EtherChannels and that they are all in use. (This is shown by the letters SU next to the Port-channel number.)
Example 2-23: show etherchannel summary Command Output
Switch# show etherchannel summary
Flags: D - down P - bundled in port-channel
I - stand-alone s - suspended
H - Hot-standby (LACP only)
R - Layer3 S - Layer2
U - in use f - failed to allocate aggregator
M - not in use, minimum links not met
u - unsuitable for bundling
w - waiting to be aggregated
d - default port
Number of channel-groups in use: 2
Number of aggregators: 2
Group Port-channel Protocol Ports
------+-------------+-----------+--------------------------------------------
2 Po2(SU) LACP g0/49(P) g0/50(P) g0/51(P) g0/52(P)
7 Po7(SU) LACP g0/47(P) g0/48(P)
9 Po9(SU) PAgP g0/8(P) g0/9(P)
The show-running command also displays a section of your configuration relevant to EtherChannel groups. The argument interface is useful for displaying information about a physical interface part of an EtherChannel or to display information about the port-channel interface itself.
In Example 2-24, interface gigabitethernet 0/48 is part of EtherChannel bundle 7. It is set to trunk mode to ensure that its configuration will be compatible with the configuration of the Port-channel interface.
Example 2-24: show running-config interface Command Output
Switch# show running-config interface g0/48
Building configuration...
Current configuration : 154 bytes
interface GigabitEthernet0/48
switchport access vlan 41
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 7 mode active
end
In Example 2-25, the Port-channel interface is a Layer 2 EtherChannel set to trunk mode.
Example 2-25: show running-config interface port-channel Command Output
Switch# show running-config interface port-channel 7
Building configuration...
Current configuration : 92 bytes
interface Port-channel7
switchport trunk encapsulation dot1q
switchport mode trunk
end
EtherChannel Load Balancing Options
When the EtherChannel becomes one entity, the traffic is load balanced across multiple links. This subsection describes the configuration of load balancing among the ports included in an EtherChannel.
Load balancing traffic across port members of the same EtherChannel is a key element in an EtherChannel configuration. As shown in Figure 2-25, two workstations communicate with a router through a switch. The link between the router and the switch is an EtherChannel link relying on two physical ports. Suppose that the majority of the traffic is going from the PCs toward the router. How should the traffic be load balanced across the two physical links? If load balancing is based on the destination IP address, most traffic will go through the same physical link. Because the router has only one single IP address for its bundled interface, this IP address will be associated to a first physical port, and all traffic destined to it will go through a single physical link. This renders the load-balancing mechanism inefficient. On the other hand, if load balancing is based on the source MAC address, the traffic might be far better balanced across both physical links because each PC has its MAC address seen on the switch.
As a rule, use the load-balancing option that provides the greatest variety in your configuration. For example, if the traffic on a channel is going to only a single MAC address, using the destination-MAC address always chooses the same link in the channel; using source addresses might result in better load balancing.
| Note | EtherChannel balances traffic load across the links in a channel. The default and the load balancing methods available can vary among the Cisco switch families. The 2960, 3560, 3750 default to src-mac, whereas the 4550 and 6500 families default to src-dst-ip. This is the recommended option for Layer 3 switches. |
Load balancing is applied globally for all EtherChannel bundles in the switch. To configure EtherChannel load balancing, use the port-channel load-balance command. Load balancing can be based on these variables. The load-balancing keywords are as follows:
-
src-mac: Source MAC addresses
-
dst-mac: Destination MAC addresses
-
src-dst-mac: Source and destination MAC addresses
-
src-ip: Source IP addresses
-
dst-ip: Destination IP addresses
-
src-dst-ip: Source and destination IP addresses (default)
-
src-port: Source TCP/User Datagram Protocol (UDP) port
-
dst-port: Destination TCP/UDP port
-
src-dst-port: Source and destination TCP/UDP port
Each mode is self-explanatory. For example, with source-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the source -MAC address of the incoming packet. Therefore, to provide load balancing, packets from different hosts use different ports in the channel, but packets from the same host use the same port in the channel. (And the MAC address learned by the switch does not change.)
With destination-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the destination-MAC address of the frame. Therefore, packets to the same destination are forwarded over the same port, and packets to a different destination are sent on a different port in the channel.
In Example 2-26, the EtherChannel load-balancing mechanism was configured to use source and destination IP addresses pairs. This rule is applied to IPv4 and IPv6 traffic, whereas the non-IP load-balancing mechanism uses source and destination MAC address pairs. It has been noticed that using source-destination IP load balancing, the balancing ends up more like 70-30 on the links.
Example 2-26: EtherChannel Load Balancing
Switch(config)# port-channel load-balance src-dst-ip
Switch(config)# exit
Switch# show etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-dst-ip
EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source XOR Destination MAC address
IPv4: Source XOR Destination IP address
IPv6: Source XOR Destination IP address
Summary
In review, a VLAN is a logical grouping of switch ports that connects nodes of virtually any type, regardless of physical location. VLAN segmentation is based on traffic flow patterns. A VLAN is usually defined as end to end or local. An end-to-end VLAN spans the entire switched network, whereas a local VLAN is limited to the switches in the Building Access and Building Distribution submodules. The creation of a VLAN implementation plan depends on the business and technical requirements.
Furthermore, a trunk is a Layer 2 point-to-point link between networking devices that can carry the traffic of multiple VLANs. ISL and 802.1Q are the two trunking protocols that connect two switches. The 802.1Q protocol is an open-standard protocol also used for VLAN trunking.
VTP is used to distribute and synchronize information about VLANs configured throughout a switched network. VTP pruning helps to stop flooding of unnecessary traffic on trunk links. VTP configuration sometimes needs to be added to small network deployments, whereas VTP transparent mode is usually privileged for larger networks. When configuring VLANs over several switches, ensure that the configuration is compatible throughout switches in the same domain.
In addition, device communication within the same VLAN can be fine-tuned using Private VLANs. A Private VLAN is associated to a primary VLAN, and then mapped to one or several ports. A primary VLAN can map to one isolated and several community VLANs.
Private VLANs can span across several switches using regular 802.1Q trunks or Private VLAN trunks.
To increase bandwidth and provide redundancy, use EtherChannel by aggregating individual, similar links between switches. EtherChannel can be dynamically configured between switches using either the Cisco proprietary PAgP or the IEEE 802.3ad LACP. EtherChannel is configured by assigning interfaces to the EtherChannel bundle and configuring the resulting port-channel interface. EtherChannel load balances traffic over all the links in the bundle. The method that is chosen directly impacts the efficiency of this load-balancing mechanism.
0 comments
Post a Comment