What are the issues reported by the customers in 9.5.0.0.156 release?

  • 4
  • Question
  • Updated 6 years ago
Archived and Closed

This conversation is no longer open for comments or replies and is no longer visible to community members. The community moderator provided the following reason for archiving: old news

Photo of Joyanta Majumder

Joyanta Majumder, Employee

  • 3 Posts
  • 0 Reply Likes

Posted 7 years ago

  • 4
Photo of Andrew McCartney

Andrew McCartney

  • 18 Posts
  • 0 Reply Likes
I experienced APs rebooting when getting 60-70 clients associated.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
Did you get that fixed?
Photo of Andrew McCartney

Andrew McCartney

  • 18 Posts
  • 0 Reply Likes
We were advised to downgrade to 9.4.3.12. No reboots now with 150+ associated to some APs
Photo of NorcalEDU

NorcalEDU

  • 16 Posts
  • 2 Reply Likes
Not sure if it helps but;

I didn't try 9.5.0.0.156 but 9.5.1.0.50 and had the same issue as Andrew.

We have since downgraded to the same on advice from support and just upgraded to 9.4.3.0.16 which is working fine.
Photo of Andrew McCartney

Andrew McCartney

  • 18 Posts
  • 0 Reply Likes
It does help. I was thinking of going to 9.5.1.0.50 I guess I'll have to wait.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
I'm checking to see if we can ID the specific issue being described so we can give you a target fix.
Photo of Andrew McCartney

Andrew McCartney

  • 18 Posts
  • 0 Reply Likes
Thanks Keith, that would be helpful.
Photo of the Event Engineers

the Event Engineers

  • 7 Posts
  • 1 Reply Like
We experience the same: ZF7363's are rebooting when the try to serve 50+ clients. We use version 9.5.1.0.5. In the logs it shows the last reboot reason: watchdog timeout or application reboot.

Hopefully 9.6 will solve the issues :)

Photo of NorcalEDU

NorcalEDU

  • 16 Posts
  • 2 Reply Likes
Follow up:

I upgraded 9.5.0.0.156 last week and had to downgrade back to 9.4 due to the issue. I grabbed the logs from the a couple of the AP's that rebooted and the controllers if Ruckus is interested. Let me know and I can email or open a case again on the issue.

What I observed: Upgraded in the evening (9:00 p.m.?) and everything was great. No reboots overnight. At about 8:15 we started getting AP lost contact messages. It was random from site to site and I would bet that we never had more than 30 clients associated to one AP. At 8:45 a.m. I made the call after 7 different AP's rebooted to downgrade back to 9.4.3.0.16. Since the downgrade, I haven't lost an AP yet and it has been 6 days.

Now, I'm a K-8 school district and at about 8:00 a.m. is when we begin to ramp up for the day. But none of my schools are in session at that time and I wouldn't expect to see more than 300 clients across the entire org. The only thing that I can think that might be an issue, is that we aren't routed at each site but at the district level. This means that the broadcast for a single VLAN could have 1000+ clients (wired/wireless) depending on what network they are connected to (we are changing this over the next 13 months).

Sorry for the length, but the wireless is usually set it and forget it for us. I really want this to work as I'd like to start testing our new 7372 AP as a replacement to our 7363's for future purchases.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
This is a great conversation that's separate from the main topic, so I created a new topic to continue the discussion. Please reference the new topic here: Issues in with high client density post 9.5.0
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
What we think is going on in these cases is a change to the way memory allocations in the AP were handled in 9.5. We have a workaround available (e.g. anyone needing to go to 9.5.1 for new AP model support as mentioned).

Please open a support case at https://support.ruckuswireless.com/ca... -- add any debug, logs, etc you might have so we can quickly ensure it's the same issue).

We're also working on a more transparent fix, but no ETA yet.

-K
Photo of NorcalEDU

NorcalEDU

  • 16 Posts
  • 2 Reply Likes
First off....Sorry for posting the Case ID. Didn't know that was a problem.

Is this planned on being fixed soon? If not, then I also updated the ticket asking to have our configuration tuned so that we can run it. But if its planned to be resolved in the next month or so then I'll just wait. Don't want a band-aid fix on our wireless network.

We have a 7372 that we would like to test and a 7982 that I would like to see the additional features. Can't do this till we upgrade or have to upgrade and test outside of production hours.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
It will likely be addressed in the 9.6.1 release which is still a few weeks away.
Photo of the Event Engineers

the Event Engineers

  • 7 Posts
  • 1 Reply Like
We also created a case (i can send the Case ID if you would like to receive it ;)). I don't like to downgrade all our +/- 8 ZoneDirectors, because all the configuration options get lost... And we can't use the 7372. But these 9.5 release are giving us to much downtime due to all the 7363's that keep rebooting.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
I have super powers and can locate your cases :) I will make sure its routed appropriately
Photo of the Event Engineers

the Event Engineers

  • 7 Posts
  • 1 Reply Like
I would like to know the workaround that is available.

Meanwhile we have an other open case (Case ID: 00064008). It could be related to this AP reboot problem, but I'm not sure. Currently it's not possible to retrieve the logs as the ZD is turned of :)
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
It involves some configuration adjustment. Beyond that it's best to work with your support engineer as I can't post specifics here. It's context-dependent so there's no general-case workaround to post.
Photo of Terry Ammons

Terry Ammons

  • 3 Posts
  • 0 Reply Likes
I would like to know if the issues in 9.5 are resolved in 9.6 or is the best option to downgrade to 9.4.3.0.16?
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
The issue described above is not fixed in 9.6.0.0.267. So the options are to downgrade to 9.4.3.0.16 or to open a case and work with the support engineer to tune the required parameters.

Other issues present in 9.5 are addressed in 9.6 - please refer to the 9.6 release notes for details.

-K
Photo of Satech

Satech

  • 8 Posts
  • 0 Reply Likes
I noticed meshed AP's disconnecting & reassociating with no connected clients (running v9.5)
Photo of Satech

Satech

  • 8 Posts
  • 0 Reply Likes
When "downgrading", do previously backed up files have to have been saved while running the same s/ware version for them to load properly??
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
This is a great conversation that's separate from the main topic, so I created a new topic to continue the discussion. Please reference the new topic here: Backup files for downgrades
Photo of Andrew McCartney

Andrew McCartney

  • 18 Posts
  • 0 Reply Likes
Is there a date yet when V9.5 or indeed V9.6 will be fixed for this problem?
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
We have a fix in test now. I can't commit to a specific release vehicle but we are doing everything possible to get the fix into the earliest possible MR for both releases.

Meanwhile the workaround mentioned previously is performing well.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
I have confirmation that the fixes are in both 9.6.1 and 9.5.2 MRs (maintenance releases).

Now you are going to ask me for dates :) Soon!..(days, not weeks)
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
9.5.2 for ZoneDirector has been posted to the support website. Standalone AP images will posted later today.

Note that for the ZD1100, the same special upgrade instructions we posted for 9.6.0 apply
Photo of NorcalEDU

NorcalEDU

  • 16 Posts
  • 2 Reply Likes
Keith, glad to hear that a fix has been released for this issue. I'll see if we can get it scheduled for release in the next couple of days and post back.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
That would be great!
Photo of Michael Brado

Michael Brado, Official Rep

  • 3058 Posts
  • 439 Reply Likes
Customers who experience AP reboots on 9.5.0.0.156 but not on 9.4.3.0.12 should
open a SF ticket, and ask that your AP support infos from each version, be added
to bug "ER-799: APs reboot on 9.5.0 but not on 9.4.3 code".
Photo of NorcalEDU

NorcalEDU

  • 16 Posts
  • 2 Reply Likes
Sorry for the late reply on this, but we upgraded to 9.5.2.0 build 15 about 2 weeks ago and we haven't had any issues since. All seems to be working for us now.

Thanks for the fix and keeping everyone up to date on this issue.
Photo of NorcalEDU

NorcalEDU

  • 16 Posts
  • 2 Reply Likes
Well, now that we are back in full production, it seems that we have the issue back again (9.5.2.0 Build 15). I've opened a case with support again to look into the issue.
Photo of Andrew McCartney

Andrew McCartney

  • 18 Posts
  • 0 Reply Likes
Hi Was this an issue with the firmware? I was goint to move to 9.5.2.0 Build 15 next week.

Andy
Photo of NorcalEDU

NorcalEDU

  • 16 Posts
  • 2 Reply Likes
Andrew, The issue, which is what I think Ruckus is attributing this to, is that we have a very large Collision Domain without the capability of reducing the broadcast traffic at each AP. I agree it's huge and we are slowly bringing that down as funds become available and newer equipment is installed.

Ruckus keeps pointing back to the log that the AP has lost contact from the controller and feels it is a network issue. We monitor the network down to each switch and have never seen any congestion or lost a poll from any of the connected switches. We do show that the AP's just stop responding to IP packets. To back this up, we rolled back to 9.4.3 Build 16 (Not 15 as earlier stated) and we haven't lost a single AP since. Not sure what else was change in versions past 9.4.3 but something is causing the AP's to puke and then wait for an internal timer to reboot the AP.

On a side note, a neighboring district has been running 9.5 and 9.6 code and has never seen a problem. They don't have the number wireless clients that we have but otherwise they have no issues.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
I think @norcaledu's situation might be somewhat specific. However..we've continued to make improvements to the algorithms for memory management on the APs in high-density situations. 9.5.2 has some and then 9.6.1 introduced some additional enhancements.
Photo of Andrew McCartney

Andrew McCartney

  • 18 Posts
  • 0 Reply Likes
Thanks Keith, would you recommend 9.5.2 or 9.6.1? We have a very busy show in November with 25,000 visitors a day so possibly 8000 plus wireless users.
Photo of Keith - Pack Leader

Keith - Pack Leader

  • 860 Posts
  • 52 Reply Likes
9.6.1 plus this app note - http://c541678.r78.cf2.rackcdn.com/ap... should give your best results.

This conversation is no longer open for comments or replies.