This chapter consists of three sections. The first section focuses on network issues that affect the wireless LAN components and their operation. It is important to distinguish that troubleshooting actual wireless LAN components is not intended here. Those topics are covered in wireless-specific books and courses. In the second section, troubleshooting network issues that affect unified communications (VoIP, IP telephony, and so on) are the target. Finally, in the third section, troubleshooting issues that affect video applications are discussed. Again, note that these sections do not provide any technical coverage of unified communications or video applications. (You can find books and courses especially designed to discuss topics such as troubleshooting unified communications and so on.) Throughout this chapter, several troubleshooting examples are presented so that you become familiar with how certain misconfigurations manifest themselves as network problems and how you can go about isolating those issues and solve them with systematic troubleshooting skills.
Troubleshooting Converged Networks to Support Wireless Operations
This section’s focus is on the readiness of the wired network to support wireless deployments. This subject includes network services such as Power over Ethernet (PoE), Dynamic Host Configuration Protocol (DHCP), quality of service (QoS), and security. The impact of wireless traffic and services on the rest of the network shall also be discussed.
| Note | This section does not cover troubleshooting wireless technologies. |
The Cisco Unified Wireless Network is composed of five interconnected elements that work together to deliver a unified enterprise-class wireless solution, as follows:
Common Wireless Integration Issues
Designing (and troubleshooting) a wireless network that integrates into a campus network, requires several considerations, including the following matters:
-
Is the wireless network based on the autonomous model or will it be based on its counterpart, the split MAC model (using lightweight access points and wireless controllers)?
-
What are the switch capabilities and requirements in terms of PoE, trunking, wireless local-area network (WLAN)-to-VLAN mapping, security, and QoS?
-
How will the Lightweight Access Point Protocol (LWAPP) be handled?
-
What type of roaming will the network support?
In a standalone or autonomous solution, autonomous access points (APs) provide all the wireless services. Deployment is based on those APs functioning as critical wireless devices, with the rest of the network providing services such as PoE, security, and QoS. Network servers, such as the Cisco Secure Access Control Server (ACS), are used for security and implement protocols such as RADIUS and TACACS+.
The controller-based architecture splits the processing of the IEEE 802.11 protocol between two devices—the AP and a centralized Cisco wireless LAN controller (WLC). The processing of the 802.11 data and management protocols and the AP functionality is also divided between the two devices. This approach is called split MAC or lightweight. Communications between the devices (lightweight APs and the WLCs) are implemented through LWAPP tunnels, as illustrated in Figure 8-1.
Understanding the autonomous and split MAC models is important because the model used defines where and how to troubleshoot potential problems when you integrate the wireless infrastructure into a campus LAN. For example, in both models, the role of the switch and other wired network elements is very important. The location of power source equipment, the configuration of trunks, and the mapping between WLANs and VLANs become important items in gathering information for your troubleshooting process. The security surrounding the wireless solution plays an important role in the proper transport of protocols such as LWAPP across the wired network. Firewalls and access control lists (ACLs) must allow the protocol between APs and Cisco WLCs. Some of the common wireless integration issues are as follows:
-
Even if there is radio frequency (RF) connectivity between the AP and the client, there can still be a problem at the side where traffic flows from the client, through the AP, to the rest of the network. In a controller-based solution, the boundary between the wireless and the wired network is the Cisco WLC because traffic is tunneled between the AP and the WLC. The WLC is an important point of troubleshooting.
-
If any filters are configured on either the Ethernet side or the radio side of the AP, disable them temporarily, until you resolve connectivity issues. Disabling the filters helps you to determine whether the filters are contributing to the problem. If the filters are long or complex, reintroduce them in phases so that you can isolate what is causing the problem. The filters might be blocking critical traffic, such as those related to LWAPP tunnels, or perhaps related to wireless security (IEEE 802.1x, Extensible Authentication Protocol [EAP], or RADIUS).
-
IP addressing typically needs to be investigated, especially in roaming scenarios. You might find that static IP addressing or reachability of the DHCP server can become problematic when integrating wireless services into the network.
-
Maintaining QoS markings consistently across wireless-to-wired boundaries is a challenge. The necessity of this is especially true in Voice over Wireless LAN (VoWLAN) deployments. Proper configuration of trust boundaries in access switches requires attention, too.
-
Other potential issues related to the network services typically provided by the switches that are connected to APs include availability of PoE, amount of PoE, whether the access ports are functioning in PortFast mode, and whether proper VLANs are allowed and operational on the trunks.
Applying the adequate troubleshooting approach to the different scenarios and situations is critical. During information gathering, you should use your knowledge of switching because several issues are related to trunking, VLANs, and switch port configuration. For example, the show interfaces switchport command provides helpful information in knowing the status of those features on specific switch ports that connect APs to the rest of the network. This command displays the administrative status and operational status of the switching components. Trunking, VLAN pruning status, allowed VLANs, and other important information are also displayed by this command. Furthermore, you can use a design tool such as the Cisco Power Calculator to isolate power issues, verify capacity planning, and determine whether the power budget on the switch is enough to power the APs. Other useful IOS troubleshooting commands include show vlan, show interfaces status, show interfaces trunk, and show access-lists.
WLAN Connectivity Troubleshooting Example: Misconfigured Trunk
This troubleshooting example is based on the network diagram/topology shown in Figure 8-2. The physical topology is shown on the left, with a lightweight AP, a WLC, and a wireless client, and the logical diagram is shown on the right side. Wireless services have suddenly stopped; clients are not able to associate to the AP. Even from the wired PCs that are used for troubleshooting, it is not possible to connect to the AP or the WLC, using either Secure Shell (SSH) or HTTP-Secure (HTTPS).
Using a bottom-up approach, you start with the access switch and look at the interfaces, looking for clues at the physical and data link layers. You can start with the show cdp neighbors command to try to identify on the switch which ports are connected to the controller and which are connected to the access point. Based on the results shown in Example 8-1, WLC connects to interface Gi 0/36 and the AP connects to interface Gi 0/37.
Example 8-1: The Output Displays the Interfaces to Which the AP and WLC Are Connected
SW1# show cdp neighbors
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone
Device ID Local Intrfce Holdtme Capability Platform Port ID
ap Gig 0/37 128 T I AIR-LAP125Gig 0
521-8 Gig 0/39 135 AIR-LAP521Fas 0
521-7 Gig 0/34 122 AIR-LAP521Fas 0
Cisco_9a:8c:e0 Gig 0/36 175 H AIR-WLC210Unit - 0 Slot - 0
Port - 1
SW1#
Next, examine the status of the interfaces with the show interface status command, as demonstrated in Example 8-2. The Gi 0/37 interface connected to the AP is associated to VLAN 10, and the Gi 0/36 interface connected to the WLC is configured as trunk.
Example 8-2: show interface status Output Confirms That the Gi 0/36 Interface Is Set Up as a Trunk
SW# show interface status
Port Name Status vlan Duplex Speed Type
Gi0/1 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/2 notconnect 1 auto auto 10/100/1000BaseTX
! output omitted for brevity
Gi0/34 connected 1 a-full a-100 10/100/1000BaseTX
Gi0/35 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/36 connected trunk a-full a-100 10/100/1000BaseTX
Gi0/37 connected 10 a-full a-1000 10/100/1000BaseTX
Gi0/38 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/39 connected 1 a-full a-100 10/100/1000BaseTX
Gi0/40 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/41 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/42 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/43 connected 1 a-full a-100 10/100/1000BaseTX
Gi0/44 connected 1 a-full a-1000 10/100/1000BaseTX
Gi0/45 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/46 connected 1 a-full a-1000 10/100/1000BaseTX
Gi0/47 notconnect 1 auto auto 10/100/1000BaseTX
--More--
You need to find out which VLANs are used for AP to WLC communication, which VLAN is used for client traffic, and whether the access point is operational and registering to the WLC using LWAPP or Control and Provisioning of Wireless Access Points (CAPWAP). The wireless administrator informs us that the AP has a static IP address and that the WLC and the AP should be on the same VLAN, but the WLC is not seeing registration requests from the AP. The static IP address on the AP enables you to rule out DHCP preventing the AP from initiating a LWAPP request. The Layer 1 and Layer 2 status of the interfaces are operational, and the wireless team tells you that it looks the same on their side, for both the AP and the WLC. You should therefore pursue the AP to WLC registration process; if the AP cannot register with the WLC, it will not be able to service client requests. Using the debug ip packet command on the switch, you can observe AP’s registration process. Because debug might produce too much output, you would filter it to see only the traffic related to LWAPP using access list 100. LWAPP uses UDP port 12223 for control messages. Example 8-3 shows this output. The AP’s request originating from interface Gi 0/37, which is associated to VLAN 10, is present (see rcvd at the end of the debug output lines shown in Example 8-3), but this traffic is not forwarded to the trunk on the Gi 0/36 interface. That is probably why there is no response from the WLC.
Example 8-3: Filtered debug Output Shows Requests from the AP, but No Response from the WLC
SW1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)# access-list 100 permit udp any any eq 12223
SW1(config)# end
SW1# debug ip packet 100
IP packet debugging is on for access list 100
SW1#
5d13h: %SYS-5-CONFIG_I: Configured from console by console
SW1#
5d13h: IP: s=10.80.1.30 (vlan10), d=255.255.255.255, len 123, rcvd 1
5d13h: IP: s=10.10.10.104 (vlan10), d=255.255.255.255, len 123, rcvd 1
5d13h: IP: s=10.10.10.104 (vlan10), d=255.255.255.255, len 123, rcvd 1
To verify that VLAN 10 is allowed on the trunk interface (Gi 0/36), you use the show interfaces switchport command. The output shown in Example 8-4 reveals that only VLAN1 is enabled (allowed) on the trunk; in other words, other VLANs such as VLAN 10 are not allowed on the trunk. This is definitely wrong.
Example 8-4: The show interface switchport Command Reveals That VLAN 10 Is Not Allowed on the Trunk Between the Switch and the WLC
SW1# show interfaces switchport | begin 0/36
Name: Gi0/36
Switchport: Enabled
Administrative Mode: trunk
Operational Mode: trunk
Administrative Trunking Encapsulation: dot1q
Operational Trunking Encapsulation: dot1q
Negotiation of Trunking: On
Access Mode VLAN: 1 (default)
Trunking Native Mode VLAN: 99 (Inactive)
Administrative Native VLAN tagging: enabled
Voice VLAN: none
Administrative private-vlan host-association: none
Administrative private-vlan mapping: none
Administrative private-vlan native VLAN: none
Administrative private-vlan Native VLAN tagging: enabled
Administrative private-vlan trunk encapsulation: dot1q
Administrative private-vlan trunk normal VLANs: none
Administrative private-vlan trunk private VLANs: none
Operational private-vlan: none
Trunking VLANs Enabled: 1
Pruning VLANs Enabled: 2-1001
Capture Mode Disabled
Capture VLANs Allowed: All
--More--
You should add the appropriate VLANs to the list of allowed VLANs on the trunk interface. The wireless team tells you that the client VLAN is 10, and that the management VLAN is 20. As demonstrated in Example 8-5, use the switchport trunk allowed vlan add 10,20 command so that VLANs 10 and 20 are allowed on the trunk interface Gi 0/36.
Example 8-5: VLANs 10 and 20 Are Added as Allowed VLANs on the Trunk Interface
SW1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)# int g0/36
SW1(config-if)# switchport trunk allowed vlan add 10,20
SW1(config-if)# end
SW1#
Based on your responsibilities and authorities, you cannot test wireless connectivity or LWAPP operations, but after a few minutes the wireless team tells you that the problem is solved. The AP is registering again to the WLC, and wireless connectivity has been restored.
WLAN Connectivity Troubleshooting Example: Duplex and Trust Issues
This wireless troubleshooting example is based on the network topology shown in Figure 8-3. The wireless operations team complains about the reliability and performance of wireless traffic. The symptom they observe is that the AP interface pointing to the wired network goes up and down intermittently, and when the port is operational, there is a substantial slowdown on Voice over WLAN.
The first thing to do is display the log and look for any clues about the interface (Gi 0/34) that apparently goes up and down intermittently. Example 8-6 shows output from the show logging | include 0/34 command, which provides the interface identifier for the port connecting the AP.
Example 8-6: The Log Shows a Duplex Mismatch Problem
SW1# show logging | include 0/34
00:12:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
00:13:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
00:14:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
00:15:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
00:16:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
00:17:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
00:18:19: %SYS-5-CONFIG_I: Configured from console by console
00:19:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
00:20:00: %CDP-4-DUPLEX_MISMATCH: duplex mismatch discovered on
GigabitEthernet0/34 (not half duplex), with 521-7 FastEthernet0 (half duplex)
SW1#
There is a duplex mismatch, but you should see the duplex mismatch messages on the console, too. A plain show logging command tells you that the console logging is disabled, which makes a lot of sense in a production switch. If you enable it, you will see the duplex mismatch messages. Example 8-7 demonstrates how to fix the duplex problem by configuring the interface for full-duplex 100 Mbps. Note that it is a good practice to find out why the duplex was set to half to begin with.
Example 8-7: Console Logging Is Disabled, but the Duplex Problem Can Be Fixed
SW1# show logging
Syslog logging: enabled (0 messages dropped, 1 messages rate-limited, 0 flushes,
0 overruns, xml disabled, filtering disabled
Console logging: disabled
Monitor logging: level debugging, 0 messages logged, xml disabled,
filtering disabled
Buffer logging: level debugging, 48 messages logged, xml disabled,
filtering disabled
Exception Logging: size (4096 bytes)
Count and timestamp logging messages: disabled
File logging: disabled
Trap logging: level informational, 51 message lines logged
--More--
SW1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)# int g0/34
SW1(config-if)# duplex full
SW1(config-if)# speed 100
SW1(config-if)# end
SW1#
Next, you call the wireless team, and they inform you that the AP comes up and does not go down again, but they are still experiencing performance issues, especially for VoIP traffic coming from the wireless network. High CPU utilization could be an issue, so you need to investigate that using the show processes cpu command, the output of which shows a relatively low level of utilization at this point, as demonstrated in Example 8-8. When you look at the available baseline for this device, provided by the team as additional documentation, you realize that these values are not too far from the baseline.
Example 8-8: The CPU Utilization Seems Normal
SW# show processes CPU
CPU utilization for five seconds: 4%/0%; one minute: 6%, five minutes: 5%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
1 0 5 0 0.00% 0.00% 0.00% 0 Chunk Manager
2 0 275 0 0.00% 0.00% 0.00% 0 Load Meter
3 0 33 0 0.00% 0.00% 0.00% 0 SpanTree Helper
4 1019 149 6838 0.00% 0.07% 0.05% 0 Check heaps
5 0 1 0 0.00% 0.00% 0.00% 0 Pool Manager
6 0 2 0 0.00% 0.00% 0.00% 0 Timers
7 118 845 139 0.00% 0.00% 0.00% 0 ARP Input
8 0 1 0 0.00% 0.00% 0.00% 0 AAA_SERVER_DEADT
9 0 2 0 0.00% 0.00% 0.00% 0 AAA high-capacit
10 0 1 0 0.00% 0.00% 0.00% 0 Policy Manager
11 26 3 8666 0.00% 0.00% 0.00% 0 Entity MIB API
12 0 1 0 0.00% 0.00% 0.00% 0 IFS Agent Manage
13 0 25 0 0.00% 0.00% 0.00% 0 IPC Dynamic Cach
14 0 1 0 0.00% 0.00% 0.00% 0 IPC Zone Manager
15 0 1412 0 0.00% 0.00% 0.00% 0 IPC Periodic Tim
16 0 1412 0 0.00% 0.00% 0.00% 0 IPC Deferred Por
17 0 94 0 0.00% 0.00% 0.00% 0 IPC Seat Manager
18 0 345 0 0.00% 0.00% 0.00% 0 HC Counter Timer
19 0 1 0 0.00% 0.00% 0.00% 0 Net Input
20 9 1412 6 0.00% 0.00% 0.00% 0 Dynamic ARP Insp
21 0 1 0 0.00% 0.00% 0.00% 0 ARP Snoop
22 9 2 4500 0.00% 0.00% 0.00% 0 XML Proxy Client
23 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
--More--
Next, you consider possible QoS configuration errors. This sounds likely as a possibility; perhaps wireless voice traffic is not being properly prioritized when entering the network. In other words, the voice traffic may not be tagged with proper QoS priorities. In the case of a LWAPP deployment, if the AP is tagging packets with values, it is the differentiated services code point (DSCP) field that gets used. You should check to see whether the switch port is honoring that. At the switch, use the show mls qos int gi0/34 command to display the trust boundary settings. A trust boundary is the point within the network where QoS markings such as DSCP are first accepted. By default, switch ports will reset DSCP values unless you explicitly tell the port to trust those values. The output of show mls qos (shown in Example 8-9) indicates that the switch does not trust anything coming from the AP. This could be a real issue; voice traffic is being prioritized on the wireless network, but it is losing its priority when crossing over to the wired network.
Example 8-9: The Interface is Shown as Configured Not to Trust the QoS (DSCP) Setting
SW1# show mls qos int g0/34
GigabitEthernet0/34
trust state: not trusted
trust mode: not trusted
trust enabled flag: ena
COS override: dis
default COS: 0
DSCP Mutation Map: Default DSCP Mutation Map
Trust device: None
qos mode: port-based
SW1#
You must set the switch port to trust DSCP values (following best practices and guidelines) using the mls qos trust dscp command, which is entered at the interface configuration mode. After entering the command, you inspect the configuration with the show mls qos command. The output indicates that the switch is now trusting DSCP values. Example 8-10 shows the necessary configuration and verification. After a while, the wireless network support staff confirm that performance issues are alleviated for VoWLAN traffic. The problem is solved.
Example 8-10: The Interface Is Configured to Trust the QoS (DSCP) Setting
SW1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)# int g0/34
SW1(config-if)# mls qos trust dscp
SW1(config-if)# end
SW1# end
SW1#
SW1# show mls qos int g0/34
GigabitEthernet0/34
trust state: trust dscp
trust mode: trust dscp
trust enabled flag: ena
COS override: dis
default COS: 0
DSCP Mutation Map: Default DSCP Mutation Map
Trust device: None
qos mode: port-based
SW1#
WLAN Connectivity Troubleshooting Example: LWAPP Denied by New Security Implementations
The third wireless troubleshooting example is based on the topology shown in Figure 8-4. The wireless team tells you that wireless operations have stopped and that none of the APs are able to register to the WLC. This problem has been expected, because a security auditor recently performed a security assessment and recommended a few improvements to the network policy. In taking all the necessary precautions, all configurations have been reverted to their pre-audit state, except for the LAN switch shown in Figure 8-4.
After investigating the recent change in security policy, you find that Cisco IOS firewall services were installed in some switches that are critical to the network. The auditor recommended hardening network devices at locations subject to higher levels of risk. Having studied the security implications and considerations on a wireless network, you know that the wireless-to-wired edge is a likely candidate to have a firewall deployed. This line of thinking allows you to focus on the Cisco IOS firewall, without discarding the possibility of other issues. Therefore, instead of focusing on a bottom-up or top-down approach, you start at the firewall level and analyze the implications of it in the wireless infrastructure. The reported symptom, wireless APs not being able to register to the WLC, provides another hint as to what to look for: LWAPP traffic may be denied by the firewall. This is a valid hypothesis with a good likelihood of being accurate, and you need to verify it. While gathering information about the Cisco IOS firewall, you must remember that Cisco IOS Software allows the firewall to be configured using one of two methods:
-
The classical Cisco IOS firewall, which uses ACLs exclusively on interfaces
-
The zone-based firewall, more widely used and more flexible for a comprehensive deployment of firewall rules
You check the zone-based policy first, but after entering the show zone-pair security command you receive an error message, effectively informing you that no zone-based policies are configured on this router. The show zone-pair security command is normally used to display the policy attached to zone-pairs. Next, you consider interface ACLs on the switch using the show ip interface command for the interface connected to the access point. The result, shown in Example 8-11, reveals that there is an ACL called FIREWALL applied inbound to the Gi 0/34 interface.
Example 8-11: The show interface Command Reveals an Inbound ACL on Gi 0/34
R1# show ip interface g0/34
GigabitEthernet0/34 is up, line protocol is up
Inbound access list is FIREWALL
R1#
You display the access list (see Example 8-12) and notice that it allows routing protocols and management protocols such as SSH. If you think in the context of a controller-based wireless architecture, you’ll notice one important thing is missing: permission for the LWAPP ports. Both control (the traffic between AP and WLC) and user traffic traverse through the LWAPP tunnel; however, the firewall is blocking those ports. This proves how important it is that the designers of the security policy be aware of the services and applications running on the network.
Example 8-12: The Access List Does Not Permit LWAPP Traffic
R1# sh access-list
Extended IP access list 100
10 permit udp 10.10.10.0 0.0.0.255 any eq 12223
20 permit udp any any eq 12223
Extended IP access list FIREWALL
10 permit icmp any any echo-reply
20 permit tcp any any eq www
30 permit tcp any any eq ftp
40 permit tcp any any eq ftp-data
50 permit tcp any any eq telnet
60 permit tcp any anyeq smtp
70 permit tcp any any eq pop3
80 permit eigrp any any
90 permit udp any any eq rip
R1#
It looks like you have found the problem, and you can verify it by changing the access list. Following best practices, you add a line to the ACL, and a remark indicating why this line was added. You need to permit UDP 12222 for user data traffic, and UDP 12223 for AP-to-WLC control messages. This work is shown in Example 8-13. The wireless team reports that this fix seems to have solved the problem. You changed the security policy by changing the firewall rules. You should carefully monitor the accuracy of the change and the potential implications it might have. A simple show access-lists command displays the number of packets matching each ACL line. You can closely monitor the ACL matches under different traffic loads and profiles to determine the implications of the recent change.
Example 8-13: The ACL Is Modified to Permit LWAPP Traffic
R1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)# ip access-list extended FIREWALL
R1(config-ext-nacl)# remark ---allowing LWAPP control and data ports---
R1(config-ext-nacl)# permit udp any any range 12222 12223
R1(config-ext-nacl)# end
R1#
WLAN Connectivity Troubleshooting Example: DHCP Issues
The fourth and final wireless troubleshooting example is based on the network topology shown in Figure 8-5. In this case, the AP and the WLC are in different VLANs, and the R1 router is performing inter-VLAN routing and also acts as the DHCP server for the APs. You have received a call from the wireless team stating that none of the APs can register to the WLC. All APs are DHCP clients but are not able to obtain their IP address from the DHCP server (which is R1 at address 10.50.50.100). The wireless group is blaming this problem on the wired network, so it is your job to find the problem and fix it.
Several things could be wrong here. One of them is the process that the APs go through to obtain an IP address lease from the DHCP server. After the APs have obtained an IP address, they register with the WLC. You will look at both processes to isolate the problem. So, you will start with the DHCP server. You switch to the console of the router acting as the DHCP server and enter the show ip dhcp server statistics command, the output from which is shown in Example 8-14.
Example 8-14: DHCP Server Statistics Do Not Reveal Any Conclusions
R1# show ip dhcp server statistics
Memory usage 5317
Address pools 1
Database agents 0
Automatic bindings 2
Manual bindings 0
Expired bindings 0
Malformed messages 0
Message Received
BOOTREQUEST 0
DHCPDISCOVER 9
DHCPREQUEST 6
DHCPDECLINE 0
DHCPRELEASE 12
DHCPINFORM 0
Message Sent
BOOTREPLY 0
DHCPOFFER 9
DHCPPACK 6
DHCPNAK 0
R1#
To isolate the DHCP server as a problem, you need to clear these statistics and start monitoring from there. This step is a very important step in troubleshooting. You need to analyze statistics in the context of a controlled scenario, where you have control over when test traffic is in the network, and how your test affects the different statistics. Therefore, you use the clear ip dhcp server statistics command, and start from there. As you can see in Example 8-15, the show ip dhcp server statistics shows no activity this time.
Example 8-15: The DHCP Server Shows No Activity
R1# clear ip dhcp server statistics
R1#
R1#
R1# show ip dhcp server statistics
Memory usage 5317
Address pools 1
Database agents 0
Automatic bindings 2
Manual bindings 0
Expired bindings 0
Malformed messages 0
Message Received
BOOTREQUEST 0
DHCPDISCOVER 0
DHCPREQUEST 0
DHCPDECLINE 0
DHCPRELEASE 0
DHCPINFORM 0
Message Sent
BOOTREPLY 0
DHCPOFFER 0
DHCPPACK 0
DHCPNAK 0
R1#
Next, you will use the debug ip udp command to monitor for any DHCP client activity such as DHCP discover messages, but see no reference to UDP port 67 (DHCP client). Because the DHCP clients are in a subnet different than that of the DHCP server, you might be missing the DHCP relay agent. This will have to be configured in the switch connecting the AP to the rest of the network. You use the show running interface command, selecting the gi0/34 port that points to the APs, but notice that the command ip-helper address is missing. This is a switchport interface associated to VLAN 10, so you must inspect interface VLAN 10 instead. See the results in Example 8-16. There is no ip-helper address configured there either.
Example 8-16: Neither the Interface Nor the Access VLAN Show a Helper Address Configuration
SW1# show running-config interface g0/34
Building configuration...
Current configuration : 108 bytes
!
interface GigabitEthernet0/34
switchport access vlan 10
switchport mode access
mls qos trust dscp
end
SW1#
SW1# show running-config interface vlan 10
Building configuration...
Current configuration : 61 bytes
!
interface vlan10
ip address 10.10.10.1 255.255.255.0
end
SW1#
As Example 8-17 shows, the show running | include helper command reveals that one IP helper address is configured on the switch, but it is pointing to an old address of the DHCP server, and it is obviously not on the right interface. So, you need to fix that issue. You go into the interface VLAN 10 and enter the correct IP helper address (after proper investigation, you might have to delete the old/erroneous command).
Example 8-17: We Correcting the Helper Address Configuration
SW1# show running-config | include helper
ip helper-address 10.100.100.100
SW1#
SW1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
SW1(config)# int vlan 10
SW1(config-if)# ip helper-address 10.50.50.100
SW1(config-if)# end
SW1#
Back at the DHCP server, now the debug results show UDP packets finally arriving at the DHCP server, as shown in Example 8-18.
Example 8-18: Client DHCP Requests Show Up at the Server (Router)
02:13:57: UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=584
02:13:58: DHCPD: assigned IP address 10.10.10.115 to client 0100.1bd5.1324.42.
02:13:58: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:13:58: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:13:58: UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=584
02:14:00: UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=584
02:14:01: UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=584
02:14:03: UDP: rcvd src=0.0.0.0(68), dst=255.255.255.255(67), length=584
02:14:05: DHCPD: assigned IP address 10.10.10.116 to client 0100.2290.Obc1.02.
02:14:05: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:14:05: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:14:05: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:14:05: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:14:05: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:14:05: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:14:05: UDP: sent src=0.0.0.0(67), dst=255.255.255.255(68), length=308
02:14:09: UDP: rcvd src=10.10.10.105(61019), dst=255.255.255.255(12223),
length=103
02:14:09: UDP: rcvd src=10.10.10.115(68), dst=10.10.10.1(67), length=584
02:14:09: UDP: sent src=10.10.10.1(67), dst=10.10.10.115(68), length=308
02:14:09: UDP: rcvd src=10.10.10.104(61121), dst=255.255.255.255(12223),
length=103
A few minutes later, you speak to the wireless support team and they verify the successful IP address assignment; however, there is still no registration into the WLC. The wireless operations team tells you to check the configuration of option 43 on the DHCP server. On the DHCP server, you display the details of the address pool using the show ip dhcp pool command and see nothing there. The show running | section ip dhcp pool displays no option 43 either (see Example 8-19).
Example 8-19: The DHCP Pool Shows No Option 43 Configuration
R1# show running-config | section ip dhcp pool
ip dhcp pool vlan10
network 10.10.10.0 255.255.255.0
default-router 10.10.10.1
!
Option 43 is used to notify the DHCP client the AP-management IP address of the WLC. Therefore, you need to go into the DHCP pool configuration mode using the ip dhcp pool command, and enter the AP-management IP address of the WLC as part of option 43. For that, you will use the command option 43, followed by the right IP address in hexadecimal format, as shown in Example 8-20. The hex string is assembled by concatenating Type, Length, and Value. Type is always f1 (hex), Length is the number of controller management IP addresses times 4 in hex, and Value is the IP address of the controller listed sequentially in hex. If there is only one WLC management address, the Length is 04 (hex), and in this case the WLC management IP address is 10.10.10.10, which is 0a0a0a0a (hex).
Example 8-20: Adding the Option 43 Configuration to the DHCP Configuration
R1# conf t
Enter configuration commands, one per line. End with CNTL/Z.
R1(config)# ip dhcp pool vlan10
R1(dhcp-config)# option 43 hex f1040a0a0a0a
R1(dhcp-config)# end
R1#
The wireless operations team notifies you that the problem is now fully solved.
0 comments
Post a Comment