Chapter 4: Advanced WAN Services Design Considerations (Part01)

Overview

Upon completing this chapter you will be able to:

• Choose an appropriate advanced WAN technology based upon customer requirements

• Describe the optical technologies used in support of advanced WAN technologies

• Describe Metro Ethernet, VPLS, and MPLS VPN technologies

• Discuss customer requirements and SLAs as part of a WAN design

This chapter reviews the key concepts associated with the design and selection of WAN services. The optical technologies Synchronous Optical Network (SONET), Synchronous Digital Hierarchy (SDH), coarse wavelength-division multiplexing (CWDM), dense wavelength-division multiplexing (DWDM), and Resilient Packet Ring (RPR) are considered along with their companion, Metro Ethernet services.

Advanced WAN Service Layers

Service providers are interested in providing advanced WAN services that can be supported with low impact on their existing fiber infrastructure. Managed services such as storage, content switching, web hosting, instant messaging, and security built on Ethernet allow the service provider to deliver advanced WAN services to customers that are using Ethernet user-network interfaces (UNI). Figure 4-1 shows the relationship between the different WAN services layers and customer applications.

Figure 4-1: Advanced WAN Services Layers

Customers have multiple reasons for wanting advanced WAN services based on Ethernet:

• Familiar equipment is used, so customers can use their existing devices.

• A familiar protocol is implemented.

• Higher bandwidth is possible than with traditional WAN links.

• Lower bits-per-second costs can be supported.

• The underlying optical technologies allow the service provider to provide these advanced WAN services on their existing fiber infrastructure.

Enterprise Optical Interconnections

Several common optical interconnection technologies are used to connect enterprise locations:

• SDH: SONET is a North American high-speed baseband digital transport standard specifying incrementally increasing data stream rates for movement across digital optical links. SDH is the European standard for digital optical links.

• DWDM and CWDM: DWDM and CWDM are technologies that increase the information-carrying capacity of existing fiber-optic infrastructure by transmitting and receiving data on different light wavelengths on a single strand of fiber.

• Dynamic Packet Transport (DPT) and Resilient Packet Ring (RPR): DPT is an RPR technology designed for service providers to deliver scalable Internet service, reliable IP-aware optical transport, and simplified network operations, principally for metropolitan-area applications. DPT is based on Spatial Reuse Protocol (SRP), a Cisco-developed MAC-layer protocol for ring-based packet internetworking.

These technologies can be used directly over leased dark fiber, or used by a service provider as the transport mechanism underlying an Ethernet or other offering.

Overview of SONET and SDH

SONET is a time-division multiplexing (TDM) technique for framing voice and data onto a single wavelength on fiber. It typically uses fixed time division to allocate bandwidth between entry points on a ring. Many long-haul fiber connections are SONET, in part because SONET repeater technology is used in many service provider networks to boost signals that are carried across long distances. SONET was historically used to prevent dropped calls in a TDM environment. SONET can provide reliable transport with TDM bandwidth guarantees for TDM voice and public-safety voice and radio traffic.

The maximum distance for single-mode installations is determined by the amount of light loss in the fiber path. Good-quality single-mode fiber with very few splices can carry an OC-12c/STM-4 signal 50 miles (80 km) or more without a repeater. Good-quality multimode fiber can carry the signal up to 1640 feet (500 m).

SONET typically uses fiber rings. When the ring fails, traffic wraps the other way around the ring. One benefit of SONET is that some network equipment may not notice the 50-ms failure leading to a ring wrap, particularly if SONET access gear keeps Ethernet services on a link in an up state. One drawback to a SONET design is that it requires provisioning double the protected bandwidth. Bandwidth along SONET is committed as circuits between two points on the ring.

Not all SONET topologies are ring based. Sometimes single or double pairs of fiber are run in linear fashion from a SONET network. Physical constraints such as river crossings can narrow the two sides of the ring into more of a figure 8, which is potentially a single point of failure. Although the high reliability of SONET is often mentioned in sales presentations, it is wise to verify that the entire SONET path being used is a true ring.

SONET can be used with SONET access equipment that may statistically multiplex Ethernet (10 Mb/s, Fast Ethernet, or Gigabit Ethernet) onto a SONET circuit. This allows some degree of oversubscription of bandwidth. The actual oversubscription amount is typically not disclosed by the provider.

Optical Carrier (OC) rates are the digital hierarchies of the SONET and SDH standards. Table 4-1 shows the commonly deployed SONET and SDH rates.

Table 4-1: Common SONET and SDH Rates
Open table as spreadsheet

Hierarchy	OC-x	Speed	SONET	SDH
—	OC-1	51.85 Mb/s	STS-1	STM-0
Level Zero	OC-3	155.52 Mb/s	STS-3	STM-1
Level One	OC-12	622.08 Mb/s	STS-12	STM-4
Level Two	OC-48	2.488 Gb/s	STS-48	STM-16
Level Three	OC-192	9.953 Gb/s	STS-192	STM-64
Level Four	OC-768	39.813 Gb/s	STS-768	STM-256

Note

SONET is an ANSI specification. SDH is the SONET-equivalent specification proposed by the ITU. European carriers use SDH widely; Asian and Pacific Rim carriers use SONET more frequently. The primary differences between the SONET and SDH specifications are the basic transmission rate and some header information.

In advanced WAN designs, the designer has to balance the current and future uses of the transport and other network components, customer requirements, customer perceptions, and the costs of the various network components. Whether SONET (or SDH) is the best solution depends on the situation.

Enterprise View of SONET

From the enterprise customer perspective, SONET is the transport underlying some other form of connection. The connection might be TDM-based, such as T1 or T3, or it may be one of the various types of Ethernet services offered by a service provider. SONET may be included as part of the Ethernet service due to its robustness, and because of the service-provider-installed base in SONET infrastructure. Traditional TDM circuits may also be aggregated and then transported over SONET.

You should ask several key questions of a service provider that is offering SONET for your network transport:

• Is the service based on connecting across end-to-end SONET rings, or are there segments that are linear or otherwise not geographically diverse? You need to consider whether there are single points of failure in the transport.

• What path does your service follow? If you are buying services from two providers for redundancy, it might be useful to determine whether the provider’s SONET follows different paths. Sometimes different providers lease fiber from the same supplier, or along the same rights of way such as gas pipelines, train tracks, and high-voltage electrical wire paths.

• Is there oversubscription and sharing, or is bandwidth dedicated for your use? Although you might not get the oversubscription details, you should know what is being allocated for your use. If the service provider is managing the customer edge (CE) device, you should also be interested in the quality of service (QoS) procedures used to manage the rate transition from your site to the SONET network.

WDM Overview

A wavelength-division multiplexing (WDM) system uses a multiplexer (mux) at the transmitter to place multiple optical signals on a fiber, and a demultiplexer (demux) at the receiver to split them off of the fiber. The signals use different wavelengths. Before being multiplexed, source signals might be converted from electrical to optical format, or from optical format to electrical format and back to optical format.

CWDM Technical Overview

CWDM is an optical technology for transmitting up to 16 channels, each in a separate wavelength or color, over the same fiber strand. The CWDM solutions help enable enterprises and service providers to increase the bandwidth of an existing Gigabit Ethernet optical infrastructure without adding new fiber strands.

Unlike DWDM, which can transmit up to 160 channels on the same fiber by tightly packing them, CWDM technology relies on wider spacing between channels. This design makes CWDM a relatively inexpensive technology for transmitting multiple Gb/s signals on a single fiber strand as compared with DWDM because it can support less-sophisticated, and therefore cheaper, transceiver designs. In the point-to-point configuration shown in Figure 4-2, two endpoints are directly connected through a fiber link. The ITU has standardized a 20-nm channel-spacing grid for use with CWDM, using the wavelengths between 1310 nm and 1610 nm. Most CWDM systems support 8 channels in the 1470-to-1610 nm range. The Cisco CWDM Gigabit Interface Converter (GBIC)/small form-factor pluggable (SFP) solution allows organizations to add or drop as many as eight channels (Gigabit Ethernet or Fibre Channel) into a pair of single-mode (SM) fiber strands. As a result, the need for additional fiber is minimized. You can create redundant point-to-point links by adding or dropping redundant channels into a second pair of SM fiber strands.

Figure 4-2: CWDM Technical Overview

CWDM multiplexing is achieved through special passive (nonpowered) glass devices known as filters. The filters act as prisms; directing lights from many incoming and outgoing fibers (client ports) to a common transmit and receive trunk port. Optical multiplexing in a ring with CWDM networks is supported with optical add/drop multiplexers (OADM). OADMs can drop off one or more CWDM wavelengths at a specific location and replace that signal with one or more different outbound signals.

The Cisco CWDM GBIC/SFP solution has two main components—a set of eight different pluggable transceivers (Cisco CWDM GBICs and Cisco CWDM SFPs), and a set of different Cisco CWDM passive multiplexers/demultiplexers or OADMs. Both the transceivers and the passive multiplexers are compliant with the CWDM grid defined in the ITU-T G.694.2 standards.

CWDM can be used by enterprises on leased dark fiber to increase capacity, for example from 1 to 8 or 16 Gb/s, over metro-area distances. One problem with CWDM is that the wavelengths are not compatible with erbium-doped fiber amplifier (EDFA) technology, which amplifies all light signals within their frequency range.

Note

EDFA technology is beginning to make repeaters obsolete. EDFA is a form of fiber-optical amplification that transmits a light signal through a section of erbium-doped fiber and amplifies the signal with a laser pump diode. EDFA is used in transmitter booster amplifiers, inline repeating amplifiers, and in receiver preamplifiers.

CWDM supports up to a 30-dB power budget on an SM fiber. This restricts the distances over which CWDM may be used. CWDM supports distances of about 60 miles (100 km) in a point-to-point topology, and about 25 miles (40 km) in a ring topology.

In some areas, service providers use CWDM to provide lambda or wavelength services. A lambda service is where a provider manages equipment and multiplexes customer traffic onto one or more wavelengths for a high-speed connection, typically between two or more points.

DWDM Technical Overview

DWDM is a core technology in an optical transport network. The concepts of DWDM are similar to those for CWDM. However, DWDM spaces the wavelengths more tightly, yielding up to 160 channels.

The tighter channel spacing in DWDM requires more sophisticated, precise, and therefore more expensive transceiver designs. In a service provider’s backbone network, the majority of embedded fiber is standard SM fiber (G.652) with high dispersion in the 1550-nm window. DWDM supports 32 or more channels in the narrow band around 1550 nm at 100-GHz spacing or about 0.8 nm, as illustrated in Figure 4-3.

Figure 4-3: DWDM Technical Overview

Note

Current Cisco DWDM cards can support 32 wavelengths.

Because of the EDFA compatibility of the wavelengths used, DWDM is also available over much longer distances than CDWM, and supports metropolitan-area network (MAN) and WAN applications. In practice, signals can travel for up to 75 miles (120 km) between amplifiers if fiber with EDFA is used. At distances of 375 miles (600) to 600 miles (1000 km), the signal must be regenerated.

DWDM can be used as a high-speed enterprise WAN connectivity service. Typical DWDM uses include connectivity between sites and data centers that is 1-, 2-, or 4-Gb/s fiber channel; IBM fiber connectivity (FICON) and Enterprise System Connection (ESCON); and Gigabit and 10 Gigabit Ethernet.

Protection options include client-side protection using rerouting, an optical splitter that allows the signal to go both ways around a ring, or line-card-based protection that detects loss of signal and wraps.

DWDM Systems

DWDM typically uses a transponder, mux/demux, and an amplifier:

• Transponder: Receives the input optical signal (from a client-layer SONET/SDH or other signal), converts that signal into the electrical domain, and retransmits the signal using a 1550-nm band laser.

• Multiplexer: Takes the various 1550-nm band signals and places them onto an SM fiber. The terminal multiplexer may or may not also support a local EDFA. An OADM extracts a channel of signal, and inserts (replaces it with) an outgoing signal from a site.

• Amplifier: Provides power amplification of the multiwavelength optical signal.

The diagram in Figure 4-4 shows how DWDM can be used with the reconfigurable OADM (ROADM). A ROADM allows reconfiguration on-the-fly so that commands select the wavelengths to be dropped and added. Other forms of OADM are tied to specific wavelengths. Reconfiguration with older OADMs meant swapping cards to select different frequencies (wavelengths). This might require interrupting the entire set of channels.

Figure 4-4: DWDM Systems

The primary challenge with mux/demux is to minimize crosstalk and maximize channel separation so that the system can distinguish each wavelength.

RPR Overview

RPR is a Layer 2 transport architecture providing packet-based transmission based on a dual counter-rotating ring topology. The June 2004 IEEE 802.17 standard defines RPR as a MAN technology supporting a ring structure using unidirectional, counter-rotating ringlets. Each ringlet is made up of links with data flow in the same direction. The use of dual fiber-optic rings provides a high level of packet survivability. If a station fails or fiber is cut, data is transmitted over the alternate ring.

RPR is similar to the older Cisco Spatial Reuse Protocol (SRP). SRP is implemented in the Cisco Dynamic Packet Transport (DPT) products. Newer Cisco DPT interfaces have been designed to include support for the 802.17 RPR protocol.

Whereas DPT and SRP use SONET/SDH as the physical medium, IEEE 802.17 RPR has been defined to use either SONET/SDH or the physical layer of Gigabit and 10 Gigabit Ethernet. DTP, SRP, and RPR can all support metro and long distance use.

RPR in the Enterprise

Figure 4-5 illustrates how the customer views RPR—as a transport ring that supports connections between their locations. RPR overcomes some limitations of SONET/SDH. Because SONET/SDH is designed to support the characteristics of voice traffic, SONET and SDH are limited in their ability to efficiently carry bursty data traffic. Voice traffic typically has consistent, well-characterized usage patterns, but data traffic bursts as large files are transferred.

Figure 4-5: RPR Customer View

Unlike point-to-point voice traffic, data traffic is characterized by the predominance of point-to-multipoint and multipoint-to-multipoint transmission and bursty traffic. RPR efficiently supports data traffic on service provider networks because RPR can take advantage of the multiple QoS and class of service (CoS) features of data traffic. RPR can also offer network efficiency by sharing or oversubscribing core bandwidth.

From the customer’s perspective, SONET typically provides TDM bandwidth guarantees, although they do not match up precisely with typical Ethernet speeds. The guarantee holds unless the provider performs edge oversubscription, with more access bandwidth than what is available across the SONET ring.

RPR is based on a statistical multiplexing approach that behaves more like Ethernet, and does not provide TDM-style bandwidth guarantees. RPR can use QoS to protect and prioritize important traffic, but bandwidth guarantees are harder to provide. Service providers need to be prepared to use a different approach with RPR to meet service-level agreements (SLA).