TLDR: Disable OCS
Problem symptoms
Yesterday I experienced very choppy Airplay performance from my MacBook Pro. The video would periodically freeze and the audio drop out. Every few seconds there would be a noticeable gap in the audio and the motion would also stutter. This was very annoying as I was screensharing Jackbox games to the TV with visitors playing from their phones.
Setup
AppleTV 4th-gen 4k connected via ethernet to my wifi VLAN. Pro-tip: this reduces bandwidth use on your wifi & you can also do this with your Chromecast – buy a Ugreen usb ethernet adaptor for it on Ebay.
Wifi is provided by a Cambium cnPilot E400, about 4M away from my laptop. It is ceiling mounted, but the other side of an internal (GIB) wall. Laptop has 2010 802.11n draft wifi module and is connected on the 5GHz band.
Sanity checks
- Plugging the laptop into an ethernet connection on the wifi VLAN resolved the issue, strongly pointing to a wifi problem.
- Speed tests (albeit over the internet!) showed the wifi was able to deliver over 100Mbps of real bandwidth
- Ping tests (again internet based) showed fairly low jitter <3ms.
Off Channel Scan
Having exhausted these basic tests, I did some more detailed work. I reviewed all options in the cnPilot config, and came across one in particular that sounded relevant – off channel scan (OCS). This feature briefly tunes the radio to a different channel to compare the noise and interference levels. It can inform the automatic selection of channels adaptively to avoid nearby access points. Disabling this feature cured the choppy Airplay performance.
Digging deeper
To better understand the impact that OCS was having, I initiated an Airplay session and fired up Wireshark to observe traffic exiting my laptop. Filtering just the IP of the Apple TV, I logged around 30 seconds of traffic and quickly noticed clusters of TCP retransmissions & duplicate acknowledgements occurring. Plotting the cumulative number of packets sent against their timestamps revealed pauses during which data was not even able to transmit from my laptop to the AP, which also coincided with the TCP retransmissions. To magnify the effect, I cranked the OCS dwell time right up to its maximum of 300ms rather than the 50ms minimum that it was previously set to (and was enough to stutter my Airplay).
It was also possible to see pauses in the egress traffic from the Apple TV’s switch port when mirroring it to a monitoring PC. After some pauses, a pronounced catchup can be seen as the retransmissions arrive.
KISS
Is there a simpler way to test for this? To some extent, one can also see the impact of OCS while pinging the Apple TV. Note that the 1 second repetition rate of pings forms a bad race condition with the OCS function so it could take a couple of minutes to see a bad ping. Once again, this graph is with OCS maxed out to 300ms dwell time. It would pay to test for a few minutes, especially with smaller dwell times.
Conclusion
OCS on the cnPilot E400 adversely affects Airplay performance. Only 1 neighbour is using 5GHz, so there is no need to automatically monitor for interference. I read that some high end APs have a dedicated radio for this task to avoid the performance hit that I observed.
I’m documenting this interaction between Airplay and Cambium OCS here publicly as I did not have much success myself at finding this info when searching for it.