Beamflex causing roaming problems with warehouse devices

  • 1
  • Question
  • Updated 8 months ago
  • Answered
  • (Edited)
We’re having a really tough time getting 25 R610’s working in a warehouse environment with Vocollect (Honeywell) T5 devices that are used by the warehouse personnel. After a lot of investigation, review of logs, and packet captures it appears that the issue is around the T5 devices misreading the signal strength of the AP’s and staying on ap’s with poor coverage. The T5 devices are deigned to be used by operators that are driving forklifts around a warehouse so they are extremely sensitive to changes in the RF environment, always looking for a better ap.
Unfortunately they are using the AP beacons to read signal strength, so even if they are connected to an AP with great coverage, when they see a beacon from that ap with a weak signal, so when they see a beacon from an AP that physically does have good coverage but shows as having a weak signal, they assume the operator has moved from that AP and they imediately scan and jump to another ap. This process seems to repeat over and over again, resulting in a terrible experience for the client devices. These T5 devices do use older 802.11 b/g radios, and the newer devices like ipads, iphones, androids and laptops work great. Unfortunately the T5 devices are mission critical so I need a way to get them working reliably. Before Ruckus was in place, the warehouse was using Ubiquity access points, which worked great for the t5’s, just not for anything else... So we’re kind of stuck now... overall it seems like both the Ruckus and the T5 devices are trying to independently solve the same problem but they’re conflicting with each other resulting in terrible device experience. We’ve tried just about every configuration change possible including adjusting signal strength up and down, changes to power management mode on the devices, forcing b, forcing g, bss minrate, smartroam, changing roam thresholds on the device, etc.... but nothing seems to get the devices to believe the rucks signal is as strong as it actually iis because the beacons show lower strength. Does anyone have any experience with client devices that are using beacons to identify signal strength, and have any strategies to prevent them from getting confused by beamflex? Vocollect/Honeywell has asked if we can turn off beamflex, but it seems like that’s built in to the hardware and not changable. I have opened a case with Ruckus tech support but haven’t been able to get past the “Just reboot It” stage... I don’t think they understand we have a fundamental problem with the interaction of the devices and beamflex. Thanks for any thoughts....
Photo of Jeff Roback

Jeff Roback

  • 34 Posts
  • 8 Reply Likes

Posted 12 months ago

  • 1
Photo of Andrew Somerville

Andrew Somerville

  • 11 Posts
  • 5 Reply Likes
Are you 100% sure it is Beamflex? The maximum possible Beamflex gain on an R610 is only 6dB. I'm surprised that is sufficient to cause dramatic roaming issues. In any case since I assume all the APs are set to the same power level then the beacons from the 'correct' AP should still be stronger than the beacons from all other APs so why does the client device roam to a weaker signal AP?
Photo of Michael Brado

Michael Brado, Official Rep

  • 2584 Posts
  • 353 Reply Likes
That's a very good question, why the T5 roams to a weaker signal AP...?
Does the scanner manufacturer have an answer/patch to this fundamental
problem?

My suggestion is to reduce the tx power on AP 2.4g radios.  Do not reduce
bss-minrate on 2.4g, which would cause the AP to ignore a T5 that was still
trying to connect/roam from too far away, but lowering the AP 2.4g power
would prevent the T5s from seeing them from too far away.

Beamflex is an advantage, sending client power only in the direction of each
client, and thus less interference to other clients on the same AP.
Photo of Tom Pisacic

Tom Pisacic

  • 6 Posts
  • 0 Reply Likes
Hello,

we have similar problem as mentioned above. we have a warehouse with metal high racking around 40ft high and Ruckus 7782 omni are installed at the ceiling.  Racking is full of frozen food on pallets.We have tried almost everything possible, from smart-roaming, min data rate, load/band balancing, manual channels, manually set a tx power but still are experiencing issues.
I've been using the Ekahau Site Survey with Side Kick and the signal does not look that bad, SNR does not go lower then 20.. In most areas, SNR is above 25. 
Clients are Citadel RF devices with rubber duck antenna 3dBi 2.4GHz but thinking to switch over to dual band antennas. The app what our operators use, is a web based app which does not refresh until been clicked within the app.

What else can you suggest me to try? Do we have proper APs installed in this kind of environment? Do we need to switch out our APs 7782 omni and go for newer type of AP with directional antennas? Is there anything what I can do to improve with the current setup?

thanks!
Photo of Tom Pisacic

Tom Pisacic

  • 6 Posts
  • 0 Reply Likes
What about the tx power on 7782 APs? I have been trying to find some useful documentation which would help me to configure my controller and APs specifically for this kind of environment but could find anything. 

Before I had all channels and TX power set to Auto and was experiencing more issues than now. Are there any options on the controller which I should turn off/on, tweak them?


I was told that this type of APs are not just an omni antenna but 'smart" omni antenna, and there is no need for sector deployment as these APs can focus their signal in the direction of the client. (told by sells person). 

thanks for your reply!
Photo of Michael Brado

Michael Brado, Official Rep

  • 2556 Posts
  • 350 Reply Likes
Leave TX power at Auto, which will be 100% of AP radio + antenna gain.  We use BeamFlex to direct energy back to connected clients, instead of radiating in all directions, which helps improve customer signal strength and reduce co-channel interference.

You need to Site Survey your current environment to really see what coverage you have, or where you might need more.
It is recommended to have some APs at ground level, at alternate ends of different rows, to augment your ceiling AP coverage.
Photo of Tom Pisacic

Tom Pisacic

  • 6 Posts
  • 0 Reply Likes
I have done site survey with Ekahau and Sidekick, and so far SNR (not lower then 20), signal strength looks to be in specs (not lower then -67dBm).

We had TX power set to auto, and had a lot of channel overlap in most of the warehouse. That's why I have been playing around with TX power. Channels were set to auto as well (channelization to 40 by default on both bands).
At that point, WiFi card on the Citadel would loose connection, RED X showed up on the WiFi icon and only fix was to reboot the unit.

After I have done changes and configure channels, channelization set to 20, tx power manually set, there is no issue with the RED X but during a day, operations would experience issue with 'page cannot be found' in the web tool.

Recently, I ran a script to see what happens with the signal from the cilent side, and noticed my cleints would roam to an AP with lower signal. For instance, it would be connected to the AP with signal of -55dBm and it would roam to an AP with signal of -77dBm.

why would a client do that? the Roam Aggressiveness on a client is set to Medium.

thanks,

Tom
Photo of Tom Pisacic

Tom Pisacic

  • 6 Posts
  • 0 Reply Likes
Forgot to mention, the data rate is set for both band to 12Mbits.
Photo of Michael Brado

Michael Brado, Official Rep

  • 2556 Posts
  • 350 Reply Likes
Unfortunately, the decision to roam is entirely up to the client radio and drivers.

I wouldn't recommend using Ruckus load balancing that would "encourage" clients to roam if an AP has many more clients than those around it.
Photo of Jeff Roback

Jeff Roback

  • 34 Posts
  • 8 Reply Likes
In our situation this was caused by some very unusual roaming behavior in the client devices.  The devices we are using Vocollect (Honeywell) T5 are very old devices and thus use an older 2.4 GHz radio.   Based on my experience I think beamflex was definitely causing problems for the devices. 

Because the handsets were designed for use while walking around a warehouse, and they were created in 802.11b days, they have some pretty sophisticated code going on for how they detect and roam between access points. My best reverse engineering seemed to indicate that they assume that a decrease in signal strength means that someone is walking away from an AP and that they should roam from it and not try that one again for a while.   And this seemed to work very will with most of AP vendors that don't have much intelligence going on at the AP.  But in a Ruckus environment where the AP's have a lot of RF intelligence going on and beamflex was constantly changing the signal strength for individual APs, the two devices trying to figure each other out was creating havoc.   
I'm not a RF engineer, so this may be complete gibberish, but here's what it looked like to me:  In watching the packet captures and debug logs, the beacons seemed to be causing the wireless devices great difficultly.   Since the beacons are sent out without any client specific beamflexing, the RSSI levels for those would vary greatly from what the client was already seeing.   So the intelligence in the device would use RSSI levels in the beacons as a 'picture' of what the environment looked like and then make very bad choices about when and where to roam.   (The developers did confirm that the driver code utilizes RSSI from beacons to learn about the environment.).   And that accounted for why the devices would roam even when they had great signal strength... because directed packets were coming in a different signal levels than beacons.   

Of course the Ruckus gear was working great for the other devices in the warehouse, so I didn't want to break the environment for everyone else.

Also, I should mention the folks at Honeywell were incredibly helpful!  They jumped right in with me and put their hardware and software people on the problem which was great.

After spending a ton of time with them looking at packet captures and debug logs, we did end up getting this working with the following settings.

Device side:   
  1. Set device for = 802.11b only (this was critical.  Even through the devices supported 802.11g, that didn't work reliably).  Setting OFDM only on the AP side didn't work reliably either. 
  2. Adjust roam triggers to make the devices wait for a pretty bad signal before roaming.
  3. Set devices to only scan channels 1,6,11

Ruckus side:
  1. Hide SSID ENABLED  (This was critical.  It seemed that by hiding the SSID in the beacons, the mobile devices didn't see the AP's at constantly changing RSSI levels, so they got less confused).
  2. 802.11r, 802.11k, 802.11w disabled
  3. DTIM 2 (Manufacturer specific)
  4. Directed MC/BC threshold: 1
  5. Mgmt TX Rate 1 mbps
  6. Reduce the AP output strength so that no more that 1 AP per channel is visible at  -75 RSSI.  

If anyone else is using the vocollect devices... .this is the vocollect device config that ended up working for us with the above ruckus config;

[HKEY_LOCAL_MACHINE\Software\Vocollect\NETWORKD\RadioSettings]
    ModulationMode=b
    PowerMode=dword:0
[HKEY_LOCAL_MACHINE\Comm\SDCCF10G1\Parms\configs\GlobalConfig]
    bLRS=dword:00000421
    RoamTrigger=dword:00000046
    RoamDelta=dword:0000000F
Photo of Michael Brado

Michael Brado, Official Rep

  • 2577 Posts
  • 351 Reply Likes
Excellent feedback Jeff, thanks.  Hm, all direction broadcast beacons signal strength affected T5 roaming...  Did hiding the SSID alone help the T5s?
Photo of Tom Pisacic

Tom Pisacic

  • 6 Posts
  • 0 Reply Likes
Thanks Jeff, this is the detailed report. 

Since you mentioned to mange 1 AP per channel to be visible at -75 or lower, what is the AP overlap in that case (in percentage)?