Check the path – A lesson in PCoIP troubleshooting

I’ll start with the moral of the story – when you’re having network issues, check the entire path.

I had an ongoing issue on a client’s network involving VMware View 5 and PCoIP. All internal traffic worked fine. Any PCoIP traffic that passed through the Cisco ASA firewall timed out. I could not get PCoIP traffic to work externally through the DMZ. It was the same behavior you get when the firewall has blocked ports. However, the firewall rules all seemed to be correct. When I switched the View client to use RDP, it connected to the desktop.

I opened a ticket with VMware support. They performed a bunch of troubleshooting, including running custom utilities to validate that all firewall ports were open. Everything checked out, and they blamed the Cisco ASA. I found a View-related bugfix on the Cisco website and tried flashing the ASA, but the problem persisted. So I opened up a ticket with Cisco TAC. They did their troubleshooting including packet captures and found nothing wrong. Their analysis was that the internal View connection broker was sending a FIN message to terminate the PCoIP session.

Back I went to VMware support, and they again failed to come up with a solution. So I was sitting there one day and my eyes floated over to the network diagram hanging on the wall. BAM. The cause of the problem was staring me right in the face. The customer has a Cymphonix web filter device, it functions as a bridge between the Cisco ASA and the internal network core. I tried digging through the logs on the Cymphonix but found no evidence of any packets being dropped. I put in a call to Cymphonix support anyway. I discovered that the Cymphonix was definitely dropping the PCoIP packets.

I had run afoul of a feature called Anonymous Proxy Guard. It’s designed to prevent a user from bypassing the Cymphonix by using another proxy server. The feature detects the PCoIP protocol as an anonymous proxy and drops the packets. Cymphonix support couldn’t give me a satisfactory explanation for why the device didn’t/couldn’t log the drops. I reconfigured the Cymphonix to stop using Anonymous Proxy Guard for all traffic inbound and outbound from the View Security servers. Problem solved.

The next time you’re having a problem like this, trace the entire network path!

Leave a Reply

Your email address will not be published. Required fields are marked *