Sanity Check on my Unreliable WiFi: Do I Need More Access Points?

  • 1
  • Question
  • Updated 5 months ago
  • Answered
TLDR: ZD1200 with 9 r710 APs covering 21K sq ft macOS-dominant biz, constant connection issues. Would more access points actually fix the problem?

Long form: I took over an existing enterprise network, consisting of an Untangle, 1Gb switches of various makes and models, and Ruckus for the wireless. The Ruckus consists of a ZoneDirector 1200 and 9 r710 Access Points, on a dedicated switch connect to the Untangle by 1Gb ethernet. The office is a rectangular 21,000 square foot space in a dense mixed-use urban area, with over 100 different wifi networks in range (I'm counting 2.4ghz and 5ghz separately here). We are a macOS-only shop with an average 100-150 devices on our primary wifi, and the same on our siloed guest wifi (total wireless device count rarely tops 300 devices). 

I get daily complaints about wireless not working. The most common issues are failing to connect after waking a laptop from sleep, AirPlay not working, and wifi appearing as connected but not transferring data. None of these are reliably fixed by toggling the antenna or restarting devices. 

Here are some of the more significant changes I've made in an attempt to reduce connection issues:
  • Reduced channelization from 80MHz to 40MHz
  • Reduced channelization from 40MHz to 20MHz
  • Capped the maximum devices on any AP to 50
  • Physically moved a couple of the APs to improve coverage
  • Disabled the 2.4GHz spectrum on all APs except one for specific devices (did it by disabling WLAN service on the 802.11b/g/n radio)
  • Modified every Mac's default connection preference to be the AP with the strongest signal, not the last AP accessed (which is the default)
  • I added a script to the most afflicted computers, which keeps the antenna turned off for 5 seconds after waking from sleep
  • Note: TX Power is configured to Auto, but all radios always run at Full power anyway
There are some other specific tweaks done for increasing compatibility with Apple products, but I've been working on this for over a year and I can't recall every detail. And as it's been over a year, the boss had demanded a permanent fix regardless of cost. That opens up my options in terms of adding access points or even switching to a different vendor. 

BUT: I have enough hardware on paper to completely control my airspace, and I don't think more access points will "solve" anything. If I succeed in fixing my network just by overpowering my neighbors, they'll be forced to do the same. I do not want a wifi arms race. 

Any suggestions?

Photo of Big` Spaceship

Big` Spaceship

  • 5 Posts
  • 0 Reply Likes
  • frustrated

Posted 5 months ago

  • 1
Photo of Michael Brado

Michael Brado, Official Rep

  • 2955 Posts
  • 414 Reply Likes
You don't want to compete around the edges, just provide good service to your clients. 
What version of code on your ZD?  We just posted our first 10.2.1 maintenance release.On your Monitor/Access Points page, how many clients do you see (what range)?There might be client sleep mode wake issues, so we always recommend latest client drivers/OS too.
Photo of Big` Spaceship

Big` Spaceship

  • 5 Posts
  • 0 Reply Likes
ZoneDirector is currently 10.1.1.0 build 35
As of right now, there are a total of 169 clients on the two wlans
Photo of Andrew Giancola

Andrew Giancola

  • 87 Posts
  • 24 Reply Likes
I feel like you could use IPERF3 here to better understand your Space, and how 'Sticky' your clients are before roaming to 'better' neighbor APs. I'm excited for you- this looks like a real challenge!
Are your clients all of a single type? I feel my best success come from understanding my WORST performing clients (I'm looking at you Apple iOS devices!) then seeking to 'Tune' my radios.
Best of Luck!
Photo of Big` Spaceship

Big` Spaceship

  • 5 Posts
  • 0 Reply Likes
I haven't tried IPERF3. I've relied largely on Homedale to understand radio traffic and congestion. It has been quite a challenge, but it's been fun

All the laptops are Macs, except for some on-location employees from a client that use Windows 10 (I don't have any issues with them). They range in version as far back as 2012 models running El Capitan and newer. I'm working on updating everyone to Mojave. We're pretty evenly split between Android and iOS for phones. 
Photo of Victor Cenac

Victor Cenac

  • 45 Posts
  • 15 Reply Likes
After having similar issues with MacOS (not Dell/Windows), I fixed them with these settings:
In WLAN Settings (enabled settings):
Enable Proxy ARP
Support for 802.11d (only applies to radios configured to operate in 2.4 GHz band)
Enable Client Fingerprinting
Enable 802.11k Neighbor-list Report

On the AP group:
Channel Settings:
2.4 channels 1 6 11
5 GHz 36 40 44 48 149 153 157 161 (indor)

I hope this helps somebody. I lost lots of hair fighting with the macs and also envied my sysadmin friends with networks with different names on the Aps, that never heard of such issues.
Photo of Darrel Rhodes

Darrel Rhodes, Employee

  • 133 Posts
  • 71 Reply Likes
Thanks for the detailed description, especially of the remedial measures you've taken to try and solve the issue.

I'm a dedicated Mac OSX user with a MacBook Pro and iMac 27", both running Mojave and previously Sierra and High Sierra. I use both machines almost every day with Ruckus Wi-Fi and often need to wake them from sleep, with no issues.

Knowing the version of code your ZD is running will be important. However this sounds like a system-wide problem and not limited to certain clients, or APs or even a frequency, so my gut-feel is that something else in the network is causing the issue.

Do any of your Mac clients go home with employees and suffer the same problems? Do non-Mac OS devices have issues, such as phones or have you tried a Windows device?

I don't think throwing more APs at the issue will be the cure, especially if clients close to existing APs are having the same problem.  In fact it's likely to make matters worse (probably for a different reason) as well as a waste of money.

Do you know about holding down alt/option and clicking the Wi-Fi icon on the Mac's menu bar? This will give you some good insights in to signal, noise, SNR, MCS data-rate, etc and will help with diagnostics.

My recommndation would be to look at hiring a Wi-Fi survey expert to use Ekahau or Airmagnet to live survey your building and check for RSSI, acute levels of interference and other Wi-Fi anomalies.  A spectrum analysis to check for non-Wi-Fi sources of interference would also be a good idea. Also the expert should be able to run a Wireshark packet trace to see what is happening when the client is experiencing issues.

Hope that helps,
Darrel.
Photo of pmonardo

pmonardo, Employee

  • 33 Posts
  • 19 Reply Likes
A few things about macs:
1. they don't like to roam
2. they do not support OKC Caching (if using an 802.1x network - more on that later)

A few things:
1. Is the Ruckus equipment covered under a support contract? If yes, open a TAC case in parallel while you troubleshoot issues. They sometimes can go under the "hood" to help diagnose what is going on and provide feedback.
2. Has a RF site survey been done? You said you inherited this network. I would get an RF site survey done so you have clear picture of the RF airspace. 
  • if all APs are running at full power, I would suggest to tune down the radios.
  • How is the Airtime utilizing look on the 5Ghz? High Airtime will point usually to too much CCI or high power.
3. What version of MACOS are you running on your clients? Have you opened a case with Apple support?
4. Is the preferred network at the top of the list? 
5. is the MAC allowed to turn off the wireless when it goes to sleep? Should disabled that. 
6. Airplay requires bonjour services to be running on ZD or at least configured (gateway and/or fencing) - this won't travel across L2 domains unless Bonjour services are running. Are you running a flat vlan architecture? 
7. What type of encryption are you using on your wireless network? 802.1x? WPA2? etc..
  • if 802.1x, MACs do you not support OKC Caching, only PMK Caching
  • is 802.11k enabled?
  • is 802.11r enabled?
8. How many clients per AP on average are connected? 
9. Have you looked at the wired network for saturated links? 
10. what is your backhaul like? Internet pipe? 
11. Have you look at potentially doing a spectrum analysis sweep to rule out non-802.11 wi-fi interference? You can convert one of the APs into a spectrum analyzer (won't serve clients) and take a look in a specific area.


Photo of David Black

David Black

  • 66 Posts
  • 36 Reply Likes
Sounds like you have more than enough APs. Here’s a few things to check:
- definitely do not enable 11r (ZD default = disabled)
- as previously mentioned, 11d should be enabled (ZD default = enabled)
-11k don’t care, that’s not your issue
- turn off client isolation
- if you’re using auto channel, make sure it’s background scanning (definitely don’t use channelfly)
- reduce 5GHz transmit power by 6dB to 9dB. Start at -6, then test, -7, test some more, etc.
- if you decide to re-enable 2.4, start at -9dB and potentially reduce to -10 or “min”
- verify that the wlan is set for ofdm only
- I’m not sure how many wlans you’ve deployed, but 3 or less would be a good range.
- verify that the wlans are not tunneled
- once things are stable, consider re-enabling all 5GHz channels except 120, 124, 128; and increasing channel width back to 40.
I hope some of this helps. Please let us know.
Photo of David Black

David Black

  • 66 Posts
  • 36 Reply Likes
PS - if everything is on the same L2 network, you don’t need to mess with Bonjour services. Airplay will work fine without it provided that isolation is off.
(Edited)
Photo of PARESH PATEL

PARESH PATEL

  • 16 Posts
  • 0 Reply Likes
Perfect SSID setup which can work in almost any scenario..
Photo of Bway NOC

Bway NOC

  • 37 Posts
  • 10 Reply Likes
A few quick questions...

- I see lots of recommendations to turn off ChannelFly. Is CF basically just a failed feature that Ruckus chooses to continue to support rather than fix or pull it? I've not yet seen a thread about problematic connectivity that does not include the recommendation to disable what's touted as a major feature by Ruckus...
- 802.11k is not supported on Macs (https://support.apple.com/en-us/HT206207), so I assume there's no harm in enabling it.
- This same article notes that the signal threshold for roaming is -75dB (https://support.apple.com/en-us/HT206207) - this is a good argument for reducing power.
- Why disable 802.11r? OS-X does not support it, but iOS does, so is there harm in leaving it enabled?

This is interesting, and it notes that both Cisco and Aruba directly address these Mac roaming issues:  https://framebyframewifi.net/2017/08/20/macos-wi-fi-roaming/

There's also a neat utility linked there that the OP might find helpful:

https://www.adriangranados.com/apps/wifisignal
Photo of Victor Cenac

Victor Cenac

  • 40 Posts
  • 14 Reply Likes
In my defense :) I spent time with Ruckus support trying different soups of settings until the problem went away. I just went through my controller and noted all the settings I have turned on and dumped them all in here. 
I think mostly, what helped was the channel selection.
Photo of Michael Brado

Michael Brado, Official Rep

  • 2947 Posts
  • 413 Reply Likes
Thanks Bway NOC, but you had the moderator looking hard at the links you shared, to be sure they weren't spam!
Thanks Victor, for letting us know your config that helped you specifically!
Photo of Bway NOC

Bway NOC

  • 37 Posts
  • 10 Reply Likes
If your moderator thinks any of that is spam, they have a screw loose. :)  This forum is about sharing info and tools.

Even if you're uneasy about mentioning competitors, whoever wrote the blog post was not trashing Ruckus, just noting that Cisco and Aruba have a specific mechanism for dealing with "sticky" clients. It's a real problem - we have coworking space clients who have to take time to tell customers to "disconnect and reconnect" their macs even when they're sitting 30' from an AP because their laptop is still stuck on some AP out in the lobby sitting at -74 db...

I have no interest in that wifi app, it's just literally probably the only thing out there that gives you not just an easier to read wifi status menubar, but OS-level notification popups that you're roaming. It seems like it would be an invaluable on-site tool for any place with a bunch of mac (like the OP).
Photo of Michael Brado

Michael Brado, Official Rep

  • 2947 Posts
  • 413 Reply Likes
We appreciate your sharing of info and tools, thanks.
Photo of DSE

DSE

  • 68 Posts
  • 3 Reply Likes
Interesting setup David Black, but why you say " (definitely don’t use channelfly) " ??
Photo of DSE

DSE

  • 68 Posts
  • 3 Reply Likes
Interesting setup David Black, but why you say " (definitely don’t use channelfly) " ??
Photo of David Black

David Black

  • 66 Posts
  • 36 Reply Likes
Channelfly is one of those things that looks a lot better on paper that it performs in practice.  The theory is that a channel could be so heavily congested that moving to an adjacent channel and tolerating a high level of ACI can be the lesser of the evils.  We tried it when it first came out and we still continue to try it on later releases to see if it's improved.  Our opinion remains that the cure is worse than the disease.  It's best to just let the protocol work the way it was designed to work.
(Edited)
Photo of Big` Spaceship

Big` Spaceship

  • 5 Posts
  • 0 Reply Likes
You guys are amazing! It took me all day to write my question, and I stayed late to post it. I am the entire IT department here, so it will probably take all day again to write a full reply to everyone's suggestions and questions. 
Photo of Andrew Giancola

Andrew Giancola

  • 87 Posts
  • 24 Reply Likes
we're on a conference call now regarding Sticky clients. there is no predicting which AP a client will associate to. we've moved all over the store, and each time we're surprised and shocked by which AP we've been handed off to. This site we're reviewing uses Meraki MR66's. Sticky happens.
Photo of Michael Brado

Michael Brado, Official Rep

  • 2955 Posts
  • 414 Reply Likes
The decision to roam is made by the client radio drivers and supplicant... not the APs or controller.

We can only "help" them be less sticky by reducing power and supported data rates, such that we hope they will release and roam sooner.
(Edited)
Photo of Bway NOC

Bway NOC

  • 38 Posts
  • 10 Reply Likes
I suspect some of the solutions send a deauth or something to bump the client off in hopes that it will reconnect to something closer...

Also kind of bummed that Apple is being so crappy on the OS-X side - iOS supports 802.11r, 802.11k and seems to behave better. I remember when Android devices were the ultimate "sticky" clients.

I also yearn for the days when you'd see some desire by manufacturers to work with major vendors to iron issues like this out for everyone's benefit. But I think Apple is just running the clock out on OS-X until they have an 'iPad Pro Jumbo Laptop Edition' running iOS. :(
(Edited)
Photo of Michael Brado

Michael Brado, Official Rep

  • 2955 Posts
  • 414 Reply Likes
Our Client Load Balancing solution, will not dissassociate anyone, but will ignore the first couple association requests if the AP is loaded and there are neighbor APs who have more capacity (when CLB is enabled).
Photo of David Black

David Black

  • 66 Posts
  • 36 Reply Likes
What about smartroam?  It does what Bway NOC speculates - it disassociates the client when the RSSI reaches a threshold determined by the "smartroam factor" ranging from 0 (default) to 10 (0=disabled, 1=least aggressive, 10=most, recommended factor=3-5). I prefer raising the BSS min rate to 24 or sometimes higher to achieve smooth roaming.  Smartroam is a break before make connection whereas normal roaming is the opposite.  That's why I prefer raising the bss minrate.
Photo of Big` Spaceship

Big` Spaceship

  • 5 Posts
  • 0 Reply Likes
The best solution I've found to improve roaming on Macs is pushing this shell command to them:
/System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Resources/airport prefs joinMode=Strongest
More info on the options available here:
https://apple.stackexchange.com/questions/66919/how-to-enable-faster-wifi-roaming-with-mac-os-x-airport-base-stations/144832#144832