Chapter 13: Deploying High Availability Features (Part01)

..... update soon .....

----------------------------------------------------------------------------------------------------------------

Chapter 8: Increasing Firewall Availability with Failover

Overview

Refer to the following sections for information about these topics:

• 8-1: Firewall Failover Overview— Provides a concise reference of information about how Cisco firewall failover works.

• 8-2: Configuring Firewall Failover— Covers the steps needed to configure and use firewalls as a failover pair.

• 8-3: Firewall Failover Configuration Examples— Presents several complete examples of different types of failover configurations.

• 8-4: Managing Firewall Failover— Explains the commands you can use to verify failover operation and to manually intervene in the failover process.

• 8-5: Upgrading Firewalls in Failover Mode— Discusses a strategy for upgrading the operating system image on a failover pair of firewalls.

The previous chapters in this book explain how to configure a single Cisco firewall to inspect traffic and provide the necessary security policies in a network. As long as that firewall continues to run properly, has a continuous source of power, and has consistent network connectivity, it can offer reliable service. What happens when those conditions are less than perfect?

Cisco firewalls can be configured as failover pairs, allowing two physical firewall platforms to operate in tandem. The result is greater reliability, because one or both firewalls are always available for use. Firewall failover is possible in two forms:

• Active-standby— One firewall takes on the active role, handling all the normal security functions. The other firewall stays in standby mode, ready to take over the active role in the event of a failure.

• Active-active— Both firewalls can operate with one or more separate security contexts. For each context, one firewall can take on the active role, handling all the normal firewall functions for that context. The other firewall can take on the standby role for the context, waiting to take over the active role from its peer.

The active and standby roles can be arbitrarily assigned across the whole set of security contexts. In this way, one firewall is active for one group of contexts, and the other firewall is active for another group.

8-1: Firewall Failover Overview

When a single firewall is used in a network, the security it provides generally has the following attributes:

• Lower cost— Only one hardware platform and a software license are needed.

• Single point of failure— If the firewall hardware or software fails, no traffic can be forwarded from one side to the other.

• Performance is limited— The total throughput of the stateful inspection process is limited to the firewall’s maximum performance.

If one firewall is potentially a single point of failure, it is logical to think that two firewalls would be better. Cisco firewalls can be made more available when they are configured to work as a failover pair. Firewall failover can operate in two different fashions: active-standby and active-active. The characteristics of each can be described as follows:

• Two firewalls can act as an active-standby failover pair, having the following characteristics:

  • Total cost is increased, because two firewalls must be used.

  • The firewall pair can be physically separated, allowing no single point of failure.

  • Performance is the same as that of a single firewall, because only one of the pair can actively inspect traffic at any time.

  • If the active firewall fails, the standby firewall can take over traffic inspection.

  • Active-standby failover is available on all Adaptive Security Appliance (ASA), PIX, and Firewall Services Module (FWSM) platforms and software releases, as long as a single security context is configured.

• Two firewalls can act as an active-active failover pair, which requires two firewalls configured for multiple-security context mode. The characteristics of this functionality are as follows:

  • Cost is doubled over that of a single firewall, because two fully functional firewalls must be used.

  • The firewall pair can be physically separated.

  • For each security context, one firewall takes on an active role, and the other is in standby mode.

  • Performance can be doubled over that of a single firewall. The failover roles can be alternated across multiple contexts, allowing both firewalls to actively inspect traffic for different contexts simultaneously.

  • If the active firewall for a context fails, the standby firewall for that context can take over traffic inspection.

  • Active-active failover is available on ASA platforms running release 7.0(1) or greater, PIX (515E, 525, and 535), and on FWSM platforms running release 3.1(1) or greater.

Tip

Beginning with ASA 8.0(1), firewall interfaces can be configured into a logical group for redundancy. Redundant interfaces are not a replacement for failover operation; rather they can offer some additional benefits to a failover pair. If an interface within a redundant group fails, another interface on the same firewall unit can instantly take over—without losing connectivity and without triggering a firewall failover. To recover from other failures outside of a redundant interface group, firewall failover comes into play.

How Failover Works

Firewall failover is currently available on the ASA platforms, PIX 515E, 525, and 535 models, and on the Catalyst 6500 FWSM.

Failover can be configured only if the firewall licensing enables it.

For active-standby failover, one firewall must have an “unrestricted” license, and the other has an “unrestricted” or “failover-only” license. The FWSM has active-standby failover enabled by default.

For active-active failover, both firewalls must have an “unrestricted” license. This is because both can actively inspect traffic at the same time.

Two identical firewall units can coexist as a failover or redundant pair by having their roles coordinated. In active-standby failover, one unit functions as the active unit and the other as the standby unit for all traffic inspection at any given time. One of the two firewalls always sits idle, waiting to take over the active role. Figure 8-1 illustrates this arrangement. The firewall on the left is active, and the one on the right is in standby mode.

Figure 8-1: Active-Standby Firewall Failover Concept

The two firewalls are in regular communication with each other over either a serial failover cable or a LAN-based connection. The firewalls can be configured for stateful failover so that the active unit keeps the standby unit synchronized with information about connections that are built or torn down.

Each interface of the active unit must connect to the respective interface of the standby unit so that each firewall can monitor the health of the interfaces.

If a failure is detected on the active unit, the two firewalls effectively swap roles. Figure 8-2 shows this concept. The firewall on the right has moved from the standby role into the active role.

Figure 8-2: Active-Standby Failover After a Failure

In active-active failover, the firewalls still alternate their roles so that one unit is active and one is in standby. The difference is that the active-standby combination is carried out on a per-context basis, with each firewall running multiple security contexts. If the active-standby roles are alternated across different security contexts, both units can actively inspect traffic at the same time—hence the term active-active failover, where neither unit is required to sit idle.

Figure 8-3 illustrates the active-active concept, in which each firewall is configured to run two separate security contexts, Context A and Context B. Now each context in one firewall can take on either the active or standby role, and the corresponding context in the other firewall takes on the alternate role. In the figure, the top firewall has the active role for Context A, and the bottom firewall is active for Context B.

Figure 8-3: Active-Active Firewall Failover Concept

If the active roles are divided appropriately across the firewalls, it becomes possible for both firewalls to be active on at least one context at any time. In other words, one whole firewall is not required to sit idle.

During a failure in active-active failover, the two firewalls effectively swap roles, but only for contexts in which a failure is detected. In Figure 8-4, the entire top firewall has failed, rendering both of its contexts useless. The bottom firewall then takes on the active role for Context A and Context B, although it was already active for Context B.

Figure 8-4: Active-Active Failover After a Failure

Firewall Failover Roles

A failover pair of firewalls can be located together if needed. A pair of Catalyst 6500 FWSMs can even be located in a single switch chassis. However, if the firewalls are geographically separated, they are less vulnerable to power or network outages or other disasters. Cisco firewalls can be separated and still function as a failover pair. Two FWSMs can also be split across a pair of switches.

The active unit performs all the firewall functions, whereas the standby only waits for the active unit to fail. At that time, the two units exchange roles until the next failure. In an active-active pair, the two exchange roles within each security context during a failure.

Configuration changes should always be made on the active unit. The firewall configurations are always coordinated between the two failover units using any of the following methods:

• The active unit automatically updates the running configuration of the standby unit as commands are entered, so the two are always synchronized.

• The copy running-config startup-config and write mem commands save the running configuration to Flash memory on the active unit and then to the Flash on the standby unit.

• The write standby command can be used to force the running configuration to be replicated from the active unit to the standby unit.

Note

Only the running configuration is kept automatically synchronized between failover peers. The startup configuration is not affected until you manually synchronize the two units by using the write memory or copy running-config startup-config command on the active unit. Also, each firewall maintains its own Flash file system. Files are not replicated across Flash file systems as a part of failover. This means that each firewall must maintain its own operating system and Cisco Adaptive Security Device Manager (ASDM) images. To upgrade a software image, you must upgrade each of the failover peers independently.

From a physical standpoint, one firewall is configured to be the primary unit, and the other becomes the secondary unit. These roles are used only to determine the IP and MAC addresses of the active and standby units, not the active and standby roles. The following actions are taken based on the primary and secondary designations:

• The active unit takes on the MAC and IP addresses of the primary physical firewall on each interface.

• The standby unit takes on the MAC and IP addresses of the secondary physical firewall on each interface.

• The units toggle or swap these addresses after a failover occurs so that the addresses of the active unit interfaces are always consistent and predictable.

The primary and secondary roles can be determined by one of the following configurations:

• LAN-based failover— The two units communicate across a LAN connection. The firewalls can be separated up to the distance limitation of the LAN media. Primary and secondary roles are manually configured. Configuration changes are replicated across the LAN at a high speed.

• Failover cable— A 6-foot serial cable connects the two firewalls. The “primary” end of the cable connects to the primary firewall (the firewall with an “unrestricted” license) and the “secondary” end to the secondary firewall. Configuration changes are replicated over the cable at 115.2 kbps. (The failover cable is unavailable on the ASA or FWSM platforms.)

Note

The failover pair of firewalls must be exactly the same model and have at least the minimum amount of RAM, the same amount of Flash memory, identical operating system releases, and compatible failover licensing. Beginning with FWSM release 2.2 and ASA release 7.0(1), each of the firewalls can run different operating system maintenance releases during an image upgrade. The “hitless upgrade” or “zero downtime upgrade” feature allows failover operation to continue as long as the pair of firewalls is running the same software major and minor release. For example, failover can continue to work even if one unit runs PIX 8.0(1) while the other runs 8.0(3). However, the two firewalls cannot run in failover mode if one runs 8.0(1) and the other runs 7.2(1).

Detecting a Firewall Failure

Each interface of one firewall must connect to the same network as the corresponding interface of the other firewall. Each firewall can then monitor every active interface of its failover peer.

The active and standby firewalls determine a failure by sending hello messages to each other at regular intervals (every 15 seconds by default). These messages are sent over the failover cable (if present) or the LAN-based failover interface to detect failures of an entire firewall. The hellos are also sent on all interfaces configured for failover so that the firewall peer can determine the health of each interface. These messages are sent as short packets using IP protocol 105.

If a hello message is not received on the failover LAN or the failover cable for three polling intervals (the default), the firewall declares the other unit “failed” and attempts to become the active unit.

The firewalls can also use a configurable hold timer that must expire before declaring the other unit failed. You can shorten the hello and hold timers so that a failure is detected sooner, if desired.

With a failover cable, a power failure or a reload on one firewall unit can be sensed on the other unit. Firewalls linked by a LAN-based failover connection can sense a peer’s health only via the regular hello messages. If one firewall is powered off, its peer can detect the failure only by noticing the absence of several consecutive hellos.

Sometimes, a firewall interface (or the network providing its connectivity) might fail while the firewall stays operational. Failover peers can detect interface failures according to the following conditions:

• If the two firewall units have changed failover roles or one of them has just powered up, the switch ports connected to the interfaces might move through the Spanning Tree Protocol states before forwarding traffic again. While a switch port is in the Listening and Learning states, regular data packets are not forwarded. This can cause failover hello messages to be dropped, causing the firewalls to begin testing their interfaces.

  To prevent this from happening, a firewall interface enters the Waiting state for two hello message periods. If more hello messages are missed after that, the interface is tested. Otherwise, failover just monitors the interface normally. With the default hello interval (15 seconds), interface testing does not begin until 30 seconds after the interface changes state. This coordinates well with the default Spanning Tree Protocol timers, which can block traffic for two periods of the Forward Delay timer (15 seconds).

• If a failover message is not seen on an interface within three polling intervals (the default), that interface is moved into a “testing” mode to determine if a failover is necessary. The other firewall is notified of the test via the failover cable or the LAN-based interface.

• Interfaces in the “testing” mode are moved through the following sequence of tests:

  • Interface status— The interface is failed if the link status is down.

  • Network activity— If no packets are received over a 5-second interval, testing continues; otherwise, the interface can still be used.

  • ARP— The interface stimulates received traffic by sending Address Resolution Protocol (ARP) requests for the ten newest entries in the firewall’s ARP table. If no traffic is received in 5 seconds, testing continues.

  • Ping— Traffic is stimulated by sending an Internet Control Message Protocol (ICMP) echo request to the broadcast address on the interface. If no replies are received over a 5-second interval, both the interface and the testing firewall unit are marked in a “failed” state.

• At the conclusion of the tests, the two firewalls attempt to compare their status. If the standby firewall has more operational interfaces than the active unit, a failover occurs. However, if both units have similar failures, no failover occurs.

Failover Communication

Firewall pairs can support several different types of failover, depending on how they are configured. Each type allows the firewalls to communicate with each other in a slightly different manner:

• Stateless failover— The state of UDP and TCP connections is not kept when the standby firewall becomes active. All active connections are dropped and must be reestablished.

• Stateful failover— The state of UDP and TCP connections, as well as address translations (xlates), H.323, Serial Interface Protocol (SIP), and Multiple Gateway Control Point (MGCP) connections, are sent to the standby firewall over the stateful LAN interface. This stateful data is updated in real time as a stream of packets using IP protocol 8.

• LAN-based failover— Failover communication between the firewalls is carried over a LAN rather than the serial failover cable. Only failover hello messages and configuration replication updates are carried over the LAN-based connection.

  LAN-based failover requires one physical interface to be set aside for failover traffic. If stateful failover is being used, too, it should have its own interface. However, it can be configured to share the same interface with LAN-based failover. The LAN-based failover interface cannot be a subinterface or a virtual LAN (VLAN) on a trunk interface.

Figure 8-5 illustrates the basic connections for a failover pair. One of the firewalls is always active and takes on the active IP addresses for all its interfaces. The other firewall is standby and takes on the standby addresses.

Figure 8-5: Basic Firewall Failover Pair Connections

Each firewall interface must connect to the same network IP subnet as the corresponding interface on the failover peer. For example, if the active unit’s outside interface uses 192.168.177.1, the standby unit’s outside interface must use an address in that same subnet. Obviously, each pair of peer interfaces must be connected on a common VLAN or Layer 2 network. In other words, the two firewalls must be able to send hello messages on an interface and reach the peer’s corresponding interface without using a router or default gateway.

The failover PIX pair can have a serial failover cable connecting them if they are in close proximity and do not need a high-speed link for configuration updates. The failover pair can also connect by a LAN-based failover link that uses a physical LAN interface on each firewall. LAN-based failover is available on PIX, ASA, and FWSM platforms and allows the failover pair to be separated by long distances, if necessary.

Stateful failover also requires a LAN-based link between the firewalls. This link can share the LAN-based failover interface, or it can be an interface set aside for stateful updates. A higher-speed interface is preferable, to support a high rate of connection state updates. In fact, you should choose the fastest interface that is present on the firewall platform so that state information can be replicated as fast as connections are formed.

Active-Active Failover Requirements

In active-active failover, the two firewalls are assigned the customary primary and secondary roles. You can give the primary or secondary unit priority for becoming the active unit on a per-context basis. This applies to the admin and any user contexts.

Because only two firewalls are permitted in a failover pair, there can be only two combinations of primary and secondary:

• A—primary, B—secondary

• A—secondary, B—primary

Each of these combinations is called a failover group. Therefore, the contexts are assigned membership in one of the two failover groups.

Figure 8-6 shows the basic arrangement of the failover pair of firewalls, along with security contexts, failover groups, and firewall states (active or standby).

Figure 8-6: Active-Active Failover with Multiple Security Contexts

Within a failover group, either the primary or secondary firewall is given preference for becoming the active unit. One firewall can even preempt or usurp the active role if it does not already have it. This means that on a given firewall unit, all contexts in a failover group take on the same active or standby state. The same contexts on the other firewall take on the alternate state.

Because you configure each context’s membership in a failover group, you can control the distribution of active contexts between the two physical firewalls. Some contexts might be heavily loaded, while others are not. Therefore, simply dividing the contexts evenly between the firewalls does not always result in an even distribution of firewall CPU, memory, and performance. You might have to experiment with context membership to maximize the use of each firewall.

8-2: Configuring Firewall Failover

To configure failover on a pair of Cisco firewalls, you can use the configuration steps listed in this section. Before failover is configured and enabled, you need to enter the configuration commands on each firewall. After failover is enabled, all configuration commands should be entered only on the active firewall. This is because the active unit replicates the configuration commands to the standby unit automatically. The only exception is any command related to failover itself.

For active-active failover, all failover configuration commands must be entered on the system execution space of the firewall that is currently active for failover group 1. This is because the failover for the system space is always handled by failover group 1. The failover IP addresses and interface monitoring must be configured in the individual security contexts.

1. Identify the primary and secondary firewall units.

   Failover communication depends on each firewall having a distinct role. The primary unit must have an “Unrestricted” license, and the secondary unit can have an “Unrestricted,” “Failover Only,” or “Active-Active Failover Only” license.

   If both units have an “Unrestricted” license, the roles can be chosen arbitrarily.

   You need to assign the primary and secondary roles in one of two ways:

   • By connecting labeled ends of a failover cable (see Step 2)

   • By configuring the roles in LAN-based failover (see Step 3a)

2. (Optional) Connect the firewalls with the serial failover cable.

   By default, the serial failover cable is expected to connect two PIX firewalls before failover can be used. If you intend to use this method of failover communication, connect the cable connector labeled “Primary” to the nine-pin failover connector on the primary unit. Then connect the “Secondary” end to the secondary unit.

   From this point on, the two units communicate failover “hello,” configuration changes, and stateful update messages over the serial cable.

   Tip

   To bring up failover mode, you must use either the serial failover cable described here or LAN-based failover, configured in the next step.

3. (Optional) Connect the firewalls over a LAN for LAN-based failover.

   A LAN connection can be used to carry failover communication much more efficiently than the serial failover cable. It can also be used if the two firewalls must be geographically separated.

   You should use a Fast Ethernet or Gigabit Ethernet connection that is dedicated to failover traffic. The connection between firewalls should be on an isolated virtual LAN (VLAN), configured for full duplex and fast convergence so that the connection is highly available.

   Tip

   Do not use a crossover Ethernet cable or a fiber-optic patch cable to directly connect the two failover LAN interfaces if the firewalls are located close to each other. Instead, each interface should connect to a switch port so that the link status is always up to one firewall interface if the other firewall interface fails. Otherwise, both units sense a link-down condition and assume that their own interfaces have a failure. You should also prepare the switch ports where the LAN-based failover interfaces connect so that failover communication can begin almost immediately. You should enable Spanning Tree Protocol PortFast and disable trunking and EtherChannel negotiation. You can use the following IOS Software commands to configure the switch ports: Switch# configure terminal Switch(config)# interface type mod/num ! Enable PortFast for immediate traffic forwarding Switch(config-if)# spanning-tree portfast ! Disable trunking by making it an access switch port Switch(config-if)# switchport mode access ! Disable EtherChannel negotiation Switch(config-if)# no channel-group

   Configuration Steps 3a through 3e should be used to configure the primary unit. Be sure to use the failover lan unit primary command described in Step 3a.

   Then, connect to the secondary unit and repeat the same commands to configure LAN-based failover on it. The commands should be identical, except for the failover lan unit secondary command described in Step 3a. Otherwise, do not try to exchange the IP addresses between primary and secondary units in the other commands. The failover pair sorts out the IP addresses according to their roles.

   1. Identify the primary and secondary units:

      FWSM	Firewall(config)# failover lan unit {primary | secondary}
      PIX 6.3	Firewall(config)# failover lan unit {primary | secondary}
      ASA	Firewall(config)# failover lan unit {primary | secondary}

      Open table as spreadsheet

      Each unit must be configured with its own failover identity, because no physical failover cable connection exists to differentiate them.

      Note

      Normally, you make configuration changes to only one firewall unit (the active one), and the changes are replicated automatically. In this step, each firewall must have a different keyword (primary or secondary) in its configuration to differentiate its firewall identity. Therefore, you must add this command to the primary and secondary firewalls independently.

   2. (FWSM only) Allow the primary FWSM to preempt the secondary:

      Firewall(config)# failover preempt [delay]

      Normally, the firewall that has the active role keeps it until it fails or until you manually intervene. Beginning with FWSM 3.2(1), you can configure the primary FWSM to preempt its peer for control over the active role. With the failover preempt command, as soon as the primary FWSM boots up, it can preempt the secondary unit and take over the active role. By default, the primary unit preempts immediately. You can set a delay (1 to 1200 seconds) to force the primary unit to wait before assuming the active role.

   3. Configure the LAN interface to be used:

      FWSM	Firewall(config)# failover interface ip if_name ip_address mask standby ip_address
      PIX 6.3	Firewall(config)# interface phy_if phy_speed Firewall(config)# nameif phy_if if_name security level Firewall(config)# ip address if_name ip_address netmask Firewall(config)# failover ip address if_name ip_address
      ASA	Firewall(config)# interface phy_if Firewall(config-if)# speed speed Firewall(config-if)# duplex duplex Firewall(config-if)# no shutdown Firewall(config-if)# exit Firewall(config)# failover interface ip if_name ip_address mask standby ip_address

      Open table as spreadsheet

      You need to configure the LAN interface for its name, speed, duplex, and security level on PIX 6.3 and ASA platforms. On any platform, you need to provide an IP address for the active unit and the standby unit.

      On an ASA, the physical interface phy_if (ethernetn or gigabitethernetn) can be configured for only speed and duplex, which are both optional. The failover interface ip command assigns an IP address for the active and standby units according to an arbitrary interface name if_name. At this point, the interface name is not bound to a physical interface for LAN-based failover.

      Note

      Failover interfaces on FWSM and ASA platforms are not assigned IP addresses as part of interface configuration mode. This is because the active-active failover commands must be configured in the system execution space, which does not participate in IP communication.

      In PIX 6.3, interface phy_if is of the form ethernetn (10/100) or gb-ethernetn (Gigabit Ethernet). The hardware_speed can be one of the following: 10baset (10 Mbps half duplex), 10full (10 Mbps full duplex), 100basetx (100 Mbps half duplex), 100full (100 Mbps full duplex), 1000auto (Gigabit autonegotiation), 1000full (Gigabit autonegotiation to use full duplex only), or 1000full nonegotiate (Gigabit full duplex). Assign an arbitrary interface named if_name (“lan-fo” or “failover,” for example) and a security level as securitylevel (0 to 100).

   4. Identify the LAN interface used for failover communication:

      FWSM	Firewall(config)# failover lan interface if_name vlan vlan
      PIX 6.3	Firewall(config)# failover lan interface if_name
      ASA	Firewall(config)# failover lan interface if_name phy_if

      Open table as spreadsheet

      All failover communication is sent and received over the interface named if_name. On an ASA, you must also bind the failover LAN interface name to a physical interface phy_if (ethernetn or gigabitethernetn). On an FWSM, you must bind the failover LAN interface name to a VLAN number vlan.

      Tip

      With two FWSMs, the failover LAN interface is also a specific VLAN. If the two FWSMs are located in a single Catalyst 6500 chassis, the VLAN is used only internally within the chassis. If the two FWSMs are located in two separate switches, you must define this VLAN on and pass it between both switches. You can do this with a single-VLAN link or over a trunk link. Before you can use the failover LAN interface VLAN, you must define it on the switch supervisor and then make it available to the FWSM by including it in the firewall vlan-group group-name vlan-list and firewall module module vlan-group group-name commands.

   5. (Optional) Encrypt failover messages:

      FWSM	Firewall(config)# failover key {key-string | hex key}
      PIX 6.3	Firewall(config)# failover lan key key-string
      ASA	Firewall(config)# failover key {key-string | hex key}

      Open table as spreadsheet

      Because other stations might intercept or overhear traffic on the failover LAN, you can define a preshared key to make the failover traffic more secure. The key is used to authenticate the failover pair of firewalls, as well as to encrypt the failover information. You can define the key as key-string (an arbitrary text string up to 63 characters) or as a hexadecimal key (an arbitrary string of exactly 32 hex digits).

      The key-string is not displayed in the firewall configuration after it is configured.

      Obviously, the same key-string must be configured on both primary and secondary firewalls so that the failover traffic can be encrypted and unencrypted correctly between them. If the keys are not identical, you will see the following message on the firewall console:

      WARNING: Failover message decryption failure. Please make sure both units have the same failover shared key and crypto license

   6. Enable LAN-based failover:

      FWSM	—
      PIX 6.3	Firewall(config)# failover lan enable
      ASA	—

      Open table as spreadsheet

      By default, a PIX failover pair expects to use the serial failover cable. You must start LAN-based failover explicitly with this command. From that point on, a connected serial cable is no longer used and can be removed.

      On an ASA or FWSM, LAN-based failover is the only method for failover communication. Therefore, it is enabled by default.

4. (Active-active only) Define failover groups.

   Failover groups must be configured from the system execution space on the primary firewall only.

   1. Choose a failover group:

      Firewall(config)# failover group {1 | 2}

      Only two failover groups are supported. Because the failover mechanism in each group is independent, each group has its own active and standby roles. Contexts are assigned membership in one of the two groups listed in Step 9.

   2. Prefer a firewall unit to have the active role:

      Firewall(config-fover-group)# {primary | secondary}

      In an initial condition, where the firewalls have booted up or failover has just been enabled, and both firewalls are functioning properly, one of them must be “elected” to take on the active failover role. By default, the primary unit has a higher priority to become active in each failover group.

      You can designate a higher priority for the primary or secondary firewall unit with this command for the failover group being configured. Naturally, if that unit fails, the other unit still can take over the active role.

   3. (Optional) Allow the higher-priority unit to assume immediate control:

      Firewall(config-fover-group)# preempt
      

      Normally, if an active unit fails, the standby unit assumes the active role indefinitely. The firewall units do not automatically revert to their original roles after a failure is resolved. Instead, they keep their roles until another failure occurs or there is manual intervention.

      You can use the preempt command to allow the higher-priority unit in a failover group to always preempt the other unit for active control.

      For example, suppose you have configured failover group 1 to give higher priority for the active role to the secondary unit. Normally, if the secondary unit fails, the primary unit assumes the active role and keeps it even after the secondary unit is restored. With preempt, the secondary unit can take over the active role as soon as it is restored to service. You would use the following commands to accomplish this:

      Firewall(config)# failover group 1
      Firewall(config-fover-group)# secondary
      Firewall(config-fover-group)# preempt
      

5. (Optional) Use virtual MAC addresses for an interface:

   FWSM	—
   PIX 6.3	Firewall(config)# failover mac address if_name active_mac standby_mac
   ASA single context	Firewall(config)# failover mac address phy_if active_mac standby_mac
   ASA multiple context	Firewall(config)# failover group {1 | 2} Firewall(config-fover-group)# mac address phy_if active_mac standby_mac

   Open table as spreadsheet

   Normally, the active and standby units exchange information about their MAC addresses as a part of the regular failover messages. If the active unit goes down, the standby can replace the MAC addresses on all its interfaces with the previous active unit’s addresses. In the rare case where both units fail and the standby unit is rebooted alone, the standby unit has no knowledge of what the active MAC addresses should be. This is because the MAC address information was not exchanged between the units because of the failure.

   This command allows both units to have stable information about what the active (active_mac) and standby (standby_mac) MAC addresses should be on an interface. In PIX 6.3, the interface name interface-name (outside, for example) is used, whereas an ASA uses the physical interface name (gigabit0, for example). Both addresses are given in dotted-triplet format, such as 0006.5b02.a841.

   In ASA, the MAC addresses are defined as global values in single-context mode, where only active-standby failover applies. If the firewalls are operating in multiple-context mode, where active-active failover is used, the MAC addresses are configured within the failover group on the system execution space because two different failover groups of contexts are maintained.

   Tip

   To use the failover mac address command, you must be able to give unique MAC addresses to both the active and standby unit interfaces. Finding unique values is not always straightforward. An easy method is to display the burned-in addresses (BIAs) of all interfaces on the primary and secondary firewall units with the show interface command. The addresses of the primary unit can always be assigned to the active firewall, and those of the secondary unit can be assigned to the standby firewall. After all, that is how the IP addresses are handled. Then, for each interface, use the command failover mac address interface primary_mac secondary_mac. At that point, it is usually a good idea to save the configurations and reboot both firewall units to make sure that the new MAC addresses are being used correctly. If you are using LAN-based failover, the MAC addresses of that interface cannot be changed or defined using this command. Instead, the default BIAs of that interface are used. This is because even though the active and standby roles change during a failover, the primary and secondary roles do not. Therefore, the primary and secondary units must have consistent identities and interface addresses.

6. (Optional) Define a health monitoring policy.

   1. Evaluate the health of the failover peer:

      FWSM	Firewall(config)# failover polltime [unit] [msec] time [holdtime holdtime]
      PIX 6.3	Firewall(config)# failover poll time
      ASA	Firewall(config)# failover polltime [unit] [msec] time [holdtime [msec] holdtime]

      Open table as spreadsheet

      Each failover unit sends periodic hello messages to its peer over the serial failover cable or the LAN-based failover interface. This is offered as evidence that the unit is still alive.

      In FWSM or PIX 6.3, you can adjust the hello message interval to time (3 to 15 seconds; the default is 15). On an ASA, you can use time (1 to 15 seconds; the default is 1) to give the interval in whole-second increments. You can also use msec time (500 to 999 milliseconds; the default is 500) to set the interval more granularly. Beginning with ASA 7.2(1), the minimum poll time is reduced to 200 milliseconds, making sub-second failover times possible.

      The unit keyword can be used to denote hello timing between units or failover peers, rather than between individual interfaces. However, if unit is not given, it is assumed anyway; the keyword exists only to make the command easier for administrators to interpret.

      One unit expects to receive hellos from its peer at regular intervals, although it does not know what that interval should be. If no hello messages are received before a holdtime timer expires, the other peer is considered to have failed. In ASA and FWSM, you can set that timer by adding holdtime holdtime (3 to 45 seconds; FWSM default is 45, ASA default is 15). PIX 6.3 uses a fixed holdtime that is always 3 times the hello interval. (The default is 3 times 15, or 45 seconds.) Beginning with ASA 7.2(1), you can add the msec keyword to set the holdtime in milliseconds (800 to 999 milliseconds).

      The holdtime timer must always be set to a minimum of 3 times the unit hello interval. This is because the firewalls always check for the loss of at least three consecutive hellos from a peer before taking action. If the holdtime keyword is not given, the firewall adjusts the holdtime automatically. The only exception to this behavior is if the unit hello interval is set to 5 seconds or less, where the default holdtime is automatically set to 15 seconds. This is done to prevent very aggressive failure detection with very short hello intervals.

      Tip

      In ASA, the most aggressive peer monitoring policy has a unit interval of 200 milliseconds and a minimum holdtime of 800 milliseconds. This allows a standby unit to detect a failure with the active unit and take over its role within 800 milliseconds. In comparison, PIX 6.3 allows a minimum hello interval of 3 seconds, but with a minimum holdtime of 15 seconds. Be careful if you decide to tighten up the unit and holdtime intervals for a more aggressive failure detection policy. Delayed or lost hellos on a congested LAN-based failover interface could be misinterpreted as a failure. If your LAN-based failover traffic is carried over switches that separate the two firewall units, make sure the switches are configured to use the most efficient spanning-tree and link-negotiation features possible. Otherwise, a Layer 2 topology change (a link or switch failure) could block the failover messages for up to 50 seconds!

   2. Evaluate the health of interfaces:

      PIX 6.3	N/A
      ASA, FWSM single context	Firewall(config)# failover polltime interface [msec] time [holdtime holdtime]
      ASA, FWSM multiple context	Firewall(config)# failover group {1 | 2} Firewall(config-fover-group)# polltime interface [msec] time [holdtime holdtime]

      Open table as spreadsheet

      Failover hello messages are sent out firewall interfaces at time (3 to 15 seconds; FWSM default is 15, ASA default is 5) intervals. A corresponding holdtime timer exists. On an ASA, the holdtime can range from 5 to 75 seconds (default 5), while a FWSM uses 3 to 15 seconds (default 15) for interface monitoring.

      In ASA, the interface polltime is a global value in single-context mode, where only active-standby failover applies. If the firewalls are operating in multiple-context mode, where active-active failover is used, polltime is configured within the failover group on the system execution space because two different failover groups of contexts are maintained.

      With PIX 6.3, hello messages are sent out all connected interfaces at the same interval as failover unit hello messages. This interval is set with the failover poll command.

   3. Define an interface failure policy:

      PIX 6.3	N/A
      ASA, FWSM single context	Firewall(config)# failover interface-policy num[%]
      ASA, FWSM multiple context	Firewall/context(config)# failover interface-policy num[%]

      Open table as spreadsheet

      By default, if a firewall tests and finds that at least one of its monitored interfaces has failed, it declares itself failed. In that case, if the firewall was in active mode, the other unit takes over the active role.

      To set the self-declared failure threshold, you can specify the number of failed interfaces as num (1 to the maximum number of interfaces; the default is 1) or a percentage of failed interfaces as num% (1 percent to 100 percent).

      Notice that the interface failure policy is set on a per-context basis. On a single-context firewall, this is configured in global configuration mode. If multiple-context mode is running, you must connect to the appropriate security context and enter the command there.

7. (Optional) Use stateful failover for maximum availability.

   Stateful failover can be used to synchronize the standby failover unit with connection information from the active unit. In this way, as connections are built or torn down, the standby unit can always keep its inspection tables up to date. If a failover occurs, the standby unit can take over the active role and maintain all the open connections without interruption.

   Stateful failover requires a LAN connection between firewalls. This is to support the high bandwidth needed to carry updates about connections that are being inspected. Be aware that no stateful information is carried over the serial failover cable if one is being used to connect the firewalls.

   1. Configure an interface to use for stateful update traffic:

      FWSM	Firewall(config)# failover interface ip if_name ip_address mask standby ip_address
      PIX 6.3	Firewall(config)# interface phy_if phy_speed Firewall(config)# nameif phy_if if_name securitylevel Firewall(config)# ip address if_name ip_address netmask Firewall(config)# failover ip address if_name ip_address
      ASA	Firewall(config)# interface phy_if Firewall(config-if)# speed speed Firewall(config-if)# duplex duplex Firewall(config-if)# no shutdown Firewall(config-if)# exit Firewall(config)# failover interface ip if_name ip_address mask standby ip_address

      Open table as spreadsheet

      The stateful interface needs to be configured for its name, speed, duplex, and security level. It also needs an IP address for the active unit and the standby unit.

      In ASA, the physical interface phy_if (ethernetn or gigabitethernetn) can be configured only for speed and duplex, which are both optional. The failover interface ip command assigns an IP address for the active and standby units according to an arbitrary interface name if_name. At this point, the interface name is not bound to a physical interface for stateful failover. This is done in Step 7c.

      In PIX 6.3, interface phy_if is of the form ethernetn (10/100) or gb-ethernetn (Gigabit Ethernet). The hardware_speed can be one of the following: 10baset (10 Mbps half duplex), 10full (10 Mbps full duplex), 100basetx (100 Mbps half duplex), 100full (100 Mbps full duplex), 1000auto (Gigabit autonegotiation), 1000full (Gigabit autonegotiation to use full duplex only), or 1000full nonegotiate (Gigabit full duplex). Assign an arbitrary interface name if_name (“stateful,” for example) and a security level as securitylevel (0 to 100).

      The two firewalls should have a dedicated link for this purpose, because stateful updates occur in real time as connections form or go away.

      Tip

      You can use one dedicated LAN interface (10/100 or Gigabit Ethernet) to carry both LAN-based failover and stateful failover information. The interface bandwidth must be large enough to carry the aggregate failover load. However, it is always best to keep the LAN-based failover and stateful failover data streams on separate interfaces. The stateful failover data stream is usually much larger than the LAN-based failover because of the usually large number of connections that come and go. Therefore, you should set aside the fastest firewall interface that is available for stateful failover. In addition, LAN-based failover messages must be able to travel between the two units without being lost or delayed. Otherwise, the loss of LAN-based failover messages indicates that one or both units have failed. You can link the two stateful failover interfaces directly with a fiber-optic or crossover patch cord without connecting them to intermediate switches. However, neither firewall unit can determine which unit has had an interface failure, because the link status is lost on both units simultaneously. The best-practice recommendations stress the need for an active device such as a switch to connect the stateful failover interfaces. If one unit loses an interface, a switch would keep the link status up for the other firewall unit. In the case of FWSMs, they each have a 6-Gbps internal trunk link to the switch backplane. With their high performance, stateful failover information can easily burst up to the link bandwidth. Therefore, if two FWSMs are located in separate chassis, you should provide a stateful failover VLAN link of at least 6 Gbps. You can do this by aggregating Gigabit Ethernet links into a Gigabit EtherChannel.

   2. Identify the interface used for stateful failover communication:

      FWSM	Firewall(config)# failover link if_name [vlan vlan]
      PIX 6.3	Firewall(config)# failover link if_name
      ASA	Firewall(config)# failover link if_name [phy_if]

      Open table as spreadsheet

      All stateful failover updates are sent and received over the interface named if_name (stateful, for example). Stateful failover can share the same interface as LAN-based failover if needed. However, you should always try to keep stateful and LAN-based failover isolated on two separate interfaces set aside for these purposes.

      In ASA, you must also bind the interface name if_name (stateful, for example) to the physical interface name phy_if (gigabit0, for example). On a FWSM, you must bind the interface name if_name to a VLAN number vlan. If LAN-based and stateful failover share the same interface, the LAN-based failover lan interface command already configures this binding. In that case, the physical interface phy_if or VLAN vlan can be omitted from this command.

   3. Keep stateful information about HTTP sessions:

      PIX 6.3	Firewall(config)# failover replicate http
      ASA, FWSM single context	Firewall(config)# failover replication http
      ASA, FWSM multiple context	Firewall(config)# failover group {1 | 2} Firewall(config-fover-group)# replication http

      Open table as spreadsheet

      By default, connection state information is replicated to the standby unit for all TCP protocols except HTTP. The HTTP connections are unique because they are short-lived, usually lasting only as long as it takes to load a web page. If a firewall failover occurs, chances are that any active HTTP requests will be retried or new ones will be generated without any connection state information. However, if it is important that all HTTP connections be preserved across an actual firewall failover, use this command.

      In ASA and FWSM, HTTP state replication is a global value in single-context mode, where only active-standby failover applies. If the firewalls are operating in multiple-context mode, where active-active failover is used, HTTP state replication is configured within the failover group on the system execution space because two different failover groups of contexts are maintained.

      Tip

      Beginning in ASA 8.0(1), session initiation protocol (SIP) signaling is also replicated between firewalls operating as a failover pair. FWSM 3.2 adds authentication, authorization, and accounting (AAA) state replication between a failover pair. These state replications are always enabled and cannot be configured.

8. Enable the failover process:

   FWSM	Firewall(config)# failover
   PIX 6.3	Firewall(config)# failover
   ASA	Firewall(config)# failover

   Open table as spreadsheet

   By default, failover is disabled even though you can configure the failover features. You must use the failover command to enable failover on the primary unit. Then, connect to the secondary unit and enter the command there too. After both units have failover enabled, they should discover each other and begin cooperating as a failover pair. At that time, the primary unit should begin replicating its configuration to the secondary unit.

   As well, each of the configuration commands entered from this point on is automatically replicated from the active unit to the standby unit.

9. (Active-active only) Assign contexts to failover groups.

   By default, all configured contexts belong to failover group 1. To assign the admin or a user context to a failover group, use the following commands in the system execution space:

   PIX 6.3	—
   ASA, FWSM single context	—
   ASA,FWSM multiple context	Firewall(config)# context name Firewall(config-ctx)# join-failover-group {1 | 2}

   Open table as spreadsheet

   A context can be a member of only one failover group. You can repeat these commands to assign other contexts to a failover group.

10. Give each firewall interface an active and a standby IP address:

    FWSM single context	Firewall(config)# nameif if_device if_name security_level Firewall(config)# ip address if_name ip_address [mask] [standby ip_address]
    FWSM multiple context	Firewall/context(config)# nameif if_device if_name security_level Firewall/context(config)# ip address if_name ip_address [mask] [standby ip_address]
    PIX 6.3	Firewall(config)# nameif if_device if_name security_level Firewall(config)# ip address if_name ip_address [mask] Firewall(config)# failover ip address if_name ip_address
    ASA single context	Firewall(config)# interface type[mod/]num Firewall(config-if)# ip address ip_address [mask] [standby ip_address]
    ASA multiple context	Firewall/context(config)# interface type[mod/]num Firewall/context(config-if)# ip address ip_address [mask] [standby ip_address]

    Open table as spreadsheet

    On the interface if_name, the active unit uses an IP address given by the ip address command. The standby unit uses a different address given by the failover ip address command or the standby keyword. After a failover occurs, the two units swap IP addresses so that the active unit always uses a consistent address. Although the standby interface is not active, it can respond to pings from other hosts to show that the unit is alive.

    In ASA or FWSM multiple-context mode, most of the failover configuration must be done in the system execution space. However, to assign IP addresses to the various context interfaces, you need to connect to each context and configure them there. This applies to the admin context as well as any configured user contexts.

    Note

    Identical interfaces on the active and standby firewalls or contexts must have IP addresses that belong to the same network subnet. For example, if interface gigabit0 on the active unit is given 192.168.1.1 255.255.255.0 as its address, the standby unit’s gigabit0 interface must also belong to the 192.168.1.0/24 subnet.

11. (Optional) Identify interfaces to be monitored:

    PIX 6.3	N/A
    ASA, FWSM single context	Firewall(config)# monitor-interface if_name
    ASA, FWSM multiple context	Firewall/context(config)# monitor-interface if_name

    Open table as spreadsheet

    Before a firewall can measure a threshold of its own failed interfaces, you must identify each interface to be monitored. By default, all physical interfaces are monitored, but no logical interfaces (VLANs) are monitored. A firewall can monitor up to 250 interfaces.

    You can enable monitoring on an interface by giving its name as if_name (outside, for example). If you want to disable monitoring on an interface, begin this command with the no keyword. This command can be repeated to identify more than one interface.

    For active-active failover, interfaces are marked for monitoring in each of the configured admin and user contexts.

Tip

You can display a list of interfaces and their monitoring status with the show failover command. For example, the following output shows that the outside interface of the admin context and the inside and outside interfaces of the CustomerA context are being monitored. The interfaces of the CustomerB context are not: Firewall# show failover Failover On Cable status: N/A - LAN-based failover enabled Failover unit Primary Failover LAN Interface: Failover Ethernet0 (up) Unit Poll frequency 3 seconds, holdtime 9 seconds Interface Poll frequency 15 seconds Interface Policy 1 Monitored Interfaces 4 of 250 maximum Group 1 last failover at: 15:33:52 EST Dec 1 2004 Group 2 last failover at: 12:33:40 EST Nov 30 2004 This host: Primary Group 1 State: Standby Ready Active time: 233703 (sec) Group 2 State: Active Active time: 168885 (sec) admin Interface outside (192.168.93.141): Normal CustomerA Interface outside (192.168.93.142): Normal CustomerA Interface inside (192.168.200.11): Normal CustomerB Interface inside (192.168.220.10): Normal (Not-Monitored) CustomerB Interface outside (192.168.200.12): Normal (Not-Monitored) Other host: Secondary Group 1 State: Active Active time: 71814 (sec) Group 2 State: Standby Ready Active time: 136665 (sec) admin Interface outside (192.168.93.138): Normal CustomerA Interface outside (192.168.93.139): Normal CustomerA Interface inside (192.168.200.10): Normal CustomerB Interface inside (192.168.220.11): Normal (Not-Monitored) CustomerB Interface outside (192.168.200.13): Normal (Not-Monitored)