EtherChannels & Load-Balance

An EtherChannel is comprised of physical, individual FastEthernet, GigabitEthernet, or Ten-GigabitEthernet (10Gbps) links that are bundled together into a single logical link.

Each EtherChannel can consist of up to eight (8) ports. Physical links in an EtherChannel must share similar characteristics, such as be defined in the same VLAN or have the same speed and duplex settings and STP configurations. On a Layer 2 switch, EtherChannel is used to aggregate access ports or trunks. When several EtherChannel bundles exist between two
switches, spanning tree might block one of the bundles to prevent redundant links. When spanning tree blocks one of the redundant links, it blocks one EtherChannel, thus blocking all the ports belonging to this EtherChannel link. Where there is only one EtherChannel link, all physical links in the EtherChannel are active because spanning tree sees only one (logical) link.

There are two link aggregation protocol options that can be used to automate the creation of an EtherChannel group: Port Aggregation Protocol (PAgP) and Link Aggregation Control Protocol (LACP).  PAgP is a Cisco proprietary protocol while LACP is part of the IEEE 802.3ad specification for creating a logical link from multiple physical links.

PAgP (Port Aggregation Protocol)

PAgP is a Cisco proprietary protocol that aids in the automatic creation of a EtherChannel. When an EtherChannel link is configured using PAgP, PAgP packets are sent between EtherChannel–capable ports to negotiate the forming of a channel. These packets are sent to the destination Multicast MAC address 01-00-0C-CC-CC-CC, which is also the same Multicast address that is used by CDP, UDLD, VTP, and DTP. PAgP packets are sent every 30 seconds. PAgP checks for configuration consistency and manages link additions and failures between two switches.

PAgP supports different port modes that determine whether an EtherChannel will be formed between two PAgP-capable switches. PAgP working in the following modes;

  • Auto – Auto mode is a PAgP mode that will negotiate with another PAgP port only if the port receives a PAgP packet. When this mode is enabled, the port(s) will never initiate PAgP communications but instead will listen passively for any received PAgP packets before creating an EtherChannel with the neighboring switch. This is the default mode.
  • Desirable – Places an interface in an active negotiating state in which the interface
    initiates negotiations with other interfaces by sending PAgP packets.
  • On – Forces the interface to channel without PAgP. Interfaces configured in the
    “on” mode do not exchange PAgP packets.

Switches running PAgP are classified as either physical learners or aggregate learners.

PAgP physical learners are switches that learn MAC addresses using the physical ports within the EtherChannel instead of via the logical EtherChannel link. Physical learners forward traffic to addresses based on the physical port via which the address was learned. The switch will send packets to the neighboring switch using the same port in the EtherChannel from which it learned the source address.

Aggregate learner learns addresses based on the aggregate or logical EtherChannel port. This allows the switch to transmit packets to the source by using any of the interfaces in the EtherChannel. Aggregate learning is the default.

By default, PAgP is not able to detect whether a neighboring switch is a physical learner. Therefore, when configuring PAgP EtherChannels on switches that support only physical learning, the learning method must be manually set to physical learning. It is important when running in this mode, to set the load-distribution method to source-based distribution so that any given source MAC address is always sent on the same physical port.

While PAgP allows for all links within the EtherChannel to be used to forward and receive user traffic, there are some restrictions, DTP and CDP send and receive packets over all the physical interfaces in the EtherChannel, while PAgP sends and receives PAgP Protocol Data Units only from interfaces that are up and have PAgP enabled for auto or desirable modes. When an EtherChannel bundle is configured as a trunk port, the trunk sends and receives PAgP frames on the lowest numbered VLAN. STP always chooses the first operational port in an EtherChannel bundle.

The following command can be used to validate the port that will be used by STP to send packets and receive packets;

Switch#show pagp neighbor
Flags:  S – Device is sending Slow hello.  C – Device is in Consistent state.
A – Device is in Auto mode.        P – Device learns on physical port.

Channel group 4 neighbors
Partner              Partner          Partner         Partner Group
Port        Name                 Device ID              Port       Age  Flags   Cap.
Gi1/1/3   Switch.1            00c5.a003.0080   Gi0/1        4s SC      10001
Gi1/1/4   Switch.1            00c5.a003.0080   Gi0/2        3s SC      10001

STP will send packets only out of port Gi1/1/3 because it is the first operational interface. If that port fails, STP will send packets out of Gi1/1/4. The default port used by PAgP can be viewed with the following command;

Switch#show etherchannel summary
Flags:  D – down        P – bundled in port-channel
I – stand-alone s – suspended
H – Hot-standby (LACP only)
R – Layer3      S – Layer2
U – in use      N – not in use, no aggregation
f – failed to allocate aggregator

M – not in use, no aggregation due to minimum links not met
m – not in use, port not aggregated due to minimum links not met
u – unsuitable for bundling
d – default port

w – waiting to be aggregated
Number of channel-groups in use: 13
Number of aggregators:           13

Group  Port-channel  Protocol    Ports

4      Po4(SU)         PAgP      Gi1/1/3(P)     Gi1/1/4(P)

When configuring additional STP features such as Loop Guard on an EtherChannel, it is important to remember that if Loop Guard blocks the first port, no BPDUs will be sent over the channel, even if other ports in the channel bundle are operational. This is because PAgP will enforce uniform Loop Guard configuration on all of the ports that are part of the EtherChannel group.

LACP (Link Aggregation Control Protocol)

Link Aggregation Control Protocol (LACP) is part of the IEEE 802.3ad specification for creating a logical link from multiple physical links. Because LACP and PAgP are incompatible, both ends of the link need to run LACP in order to automate the formation of EtherChannel groups. As is the case with PAgP, when configuring LACP EtherChannels, all LAN ports must be the same speed and must all be configured as either Layer 2 or Layer 3 LAN ports. If a link within a port channel fails, traffic previously carried over the failed link is switched over to the remaining links within the port channel.

LACP supports the automatic creation of port channels by exchanging LACP packets between ports. It learns the capabilities of port groups dynamically and informs the other ports. Unlike PAgP, where ports are required to have the same speed and duplex settings, LACP mandates that ports be only full-duplex, as half-duplex is not supported. Half-duplex ports in an LACP EtherChannel are placed into the suspended state.

By default, all inbound Broadcast and Multicast packets on one link in a port channel are blocked from returning on any other link of the port channel. LACP packets are sent to the IEEE 802.3 Slow Protocols Multicast group address 01-80-C2-00-00-02. LACP frames are encoded with the EtherType value 0x8809.

LACP Architecture

The LACP application is a client to the MAC Sub-Layer. With LACP, link aggregation applies to the MAC Sub-Layer of the Data Link Layer. The Link Aggregation Sub-Layer binds multiple physical ports and presents them to upper Layers of the stack as a single logical port.

LACP Architecture core components;

  • Collector – The Collector accepts frames from the physical link and passes them to upper layers. The Collector parses special marker packets from the incoming stream. These packets are then passed to the LACP agent. The LACP agent can also instruct the Distributor to generate marker response packets.
  • Distributor – The Distributor transmits outgoing frames from the MAC client to links in the aggregation group. This block also implements the distribution algorithm, which is used to redistribute traffic among links. To do so, the higher layer sends a special marker packet at the point where it wishes to re-distribute traffic and waits for the marker response packet. The marker response packet is generated by the receiving agent only after all frames have been passed to the higher layer applications.
  • Marker and Marker Response – These are used to maintain frame ordering when moving frames from one link in the group to another, such as when a physical link fails.

The LACP defines frame collection and distribution along with an LACP agent. The LACP defines two modes to re-distribute traffic among links. First is with the use of special packets called markers. The frame collector at either end of the link parses special marker packets from the incoming stream. These packets are then passed to the LACP agent. In addition to this, the LACP agent can also instruct the distributor to generate marker response packets.

LACP Modes

LACP helps create the EtherChannel link by detecting each side’s configuration and making sure they are compatible so that the EtherChannel link can be enabled when needed.

  • Passive – Places a port in a passive negotiating state. In this state, the port responds to the LACP packets that it receives but does not initiate LACP packet negotiation (default).
  • Active – Places a port in an active negotiating state. In this state, the port initiates negotiations with other ports by sending LACP packets.
  • On – Forces the interface to the channel without PAgP or LACP.

LACP Parameters

There are several LACP parameters that are contained in the LACP PDUs that are exchanged between switches. LACP uses the following parameters;

  • LACP System Priority – A system priority needs to be configured on both devices running LACP. This configuration can be done by the CLI or automatically. LACP uses the system priority with the device MAC address to form the system ID as well as during negotiation with other systems.
  • LACP Port Priority – Each port in the switch must have a port priority. The port priority can be specified automatically or through the CLI. The port priority and the port number form the port identifier. LACP uses the port priority to decide which ports should be put into standby mode when there is a hardware limitation that prevents all compatible ports from aggregating.
  • LACP Administrative Key – LACP automatically configures an administrative key value on each port configured to use LACP. The administrative key defines the ability of a port to aggregate with other ports. Only ports that have the same administrative key are allowed to be aggregated into the same port channel group. A port’s ability to aggregate with other ports is determined by physical characteristics, such as data rate, duplex capability, and point-to-point or shared medium, or by administrator-defined configuration restrictions or constraints.

LACP Hot-Standby Ports

By default, when LACP is configured on ports, it tries to configure the maximum number of compatible ports in a port channel, up to the maximum allowed by the hardware, which is typically eight ports. However, if LACP is unable to aggregate all the ports that are compatible into an EtherChannel,then all the ports that cannot be actively included in the channel are put in hot-standby state and are used only if one of the active ports in the EtherChannel fails.

Under the interface command you can run the following command that will give you control the number ports allowed to be bundled into the etherchannel;

Switch(config-if)#lacp max-bundle 4

Under the etherchannel interface you can run the following command which will give you the same results as the physical interface command, specifies the minimum number of member ports that must be in the link-up state and bundled in the EtherChannel for the port channel interface to transition to the link-up state;

Switch(config-if)port-channel min-link 4

Layer 2 EtherChannel Configuration

Specify the interfaces that will compose the EtherChannel group;

Switch(config)# interface range f0/1 – 4

Specify the channeling protocol to be used;

Switch(config-if-range)# channel-protocol {pagp | lacp}

Create the port-channel interface;

Switch(config-if-range)# channel-group number mode {active | on | {auto [non-silent]} | {desirable [non-silent]} | passive

Specify the port-channel interface. When in the interface configuration mode, you can configure
additional parameters. The physical interfaces will inherit these parameters;

Switch(config)# interface port-channel number

EtherChannel Load-Blancing

When the EtherChannel becomes one entity, the traffic is load balanced across multiple links. Load balancing traffic across port members of the same EtherChannel is a key element in an
EtherChannel configuration. For both PAgP and LACP EtherChannels, Cisco switches use a polymorphic algorithm that utilizes key fields from the header of the packet to generate a hash, which is then matched to a physical link in the EtherChannel group.

This operation can be performed on MAC addresses or IP addresses and can be based solely on source or destination addresses, or even both source and destination addresses. The load-distribution method is configured under the global configuration mode with the following command, only a single method can be used at any given time;

Switch(config)#port-channel load-balance ?
dst-ip       Dst IP Addr
dst-mac      Dst Mac Addr
src-dst-ip   Src XOR Dst IP Addr
src-dst-mac  Src XOR Dst Mac Addr
src-ip       Src IP Addr
src-mac      Src Mac Addr

The following is a verification command of what load balaning is been used on the switch;

Switch#show etherchannel load-balance
EtherChannel Load-Balancing Configuration:

EtherChannel Load-Balancing Addresses Used Per-Protocol:
Non-IP: Source MAC address
IPv4: Source MAC address
IPv6: Source MAC address