Outdoor Mesh Setup - Root-AP rebooting - Mesh-APs disapearing

  • 1
  • Question
  • Updated 9 months ago
  • Answered
Hi,

i have a Ruckus Unleashed Mesh Network running with 9x T300 APs.
Firmware is up-to-date.
There is a huge parking area with about 12 IP Network Cameras. To connect the Cameras to the cabled Network and to the Storage where the recordings are stored, we used this Mesh-Network.
There are 2x Root-APs and 7x Mesh-APs. 3x Mesh-APs connecting to one Root-AP. The other 4x APs connecting to the other.

Customer told me, the Systems seems to be unstable.
Sometimes one Root-AP reboots and some Mesh-APs disapearing.

I checked the Logs and did not get any helpful information for me.
What i have seen is, that my Master (which is also a Root-AP) rebooted today and two days before. Strange Thing is it happend to the exact same time (02:06 AM).
The Logs say "warm boot successfully, last reboot reasion [application reboot]".

Any ideas why the reboot is happening?
Any other helpful information why my Root-APs rebooting?

Thanks and kind regards
Marco
Photo of Marco Eichstetter

Marco Eichstetter

  • 128 Posts
  • 7 Reply Likes

Posted 9 months ago

  • 1
Photo of Michael Brado

Michael Brado, Official Rep

  • 1889 Posts
  • 269 Reply Likes
What controller platform, ZoneDirector or SmartZone, and version please?

Root (wired) APs don't typically reboot.  Could the network they're on be flooded by broadcasts, multicasts, or IPv6?

Does your company do any heavy network load events at around 2:00am?  (such that APs can't comm with controllers)

Wait, you used the term "Master", ie Unleashed?  Version 200.2.9.13.186 is about ready for GA, so please wait a week,
upgrade, and evaluate again.  (but consider the questions above)

AND, be aware that Mesh APs *cannot* act in Master or Standby Master roles in the Unleashed network. 

If you are losing Root APs and all Mesh APs on them, you should try to install another wired Root AP that can backup
the Root APs in service now.  (and hopefully determine why you have reboots today)
(Edited)
Photo of Marco Eichstetter

Marco Eichstetter

  • 128 Posts
  • 7 Reply Likes
Hi,

Unleashed Network.

Thanks + Kind Regards
Marco
Photo of Marco Eichstetter

Marco Eichstetter

  • 128 Posts
  • 7 Reply Likes
Version:
200.1.9.12.62
Photo of Marco Eichstetter

Marco Eichstetter

  • 128 Posts
  • 7 Reply Likes
Hi Michael,

Thanks for your reply.
Customer is planning to serve more parking areas with Ruckus. But because the first Parking Area is not working like he want he is waiting for his order.
So it is important to me so investigate this.

But this is really strange...

I checked the traffic flowing over the Switch Ethernet Ports. This is about 12 - 16Mbit/s. So not really much traffic.
We checked the power grid. Both PoE Switches have an Uptime of a multible weeks. So a power outage could also not be the reason.
Both Root-APs are connected to different PoE Switches (TP-Link and HP) and they have sufficient PoE Power.
Also i realized there are some radard signals and there are some Channel changes. Because we are in Germany we only have DFS Channels for Outdoor usage. I am not sure if Channel chaning may cause such kind of reboots. Maybe i am should activate Indoor Channels for my Outdoor APs. But i am not sure if this whould mean trouble with any government department.

My Thoughts:
Like you wrote they may be a Broadcast on the Network. Because the Logs on my Unleashed Master are talking of an warm application reboot, i thought there may be a SNMP Trap from anywhere who is restarting my Master-AP.

What do you think?

Thanks.
KR
Marco
(Edited)
Photo of Marco Eichstetter

Marco Eichstetter

  • 128 Posts
  • 7 Reply Likes
Additional Information:
I checked the AP logs as well.
I realized the Root-AP has a CPU Load between 40 and 92%. Don't know if this play any Role. I think it is by design that the Master-AP has more CPU Load than the other APs.
Also following Reboot reasion was given:
reboot reason       : application reboot
reboot details      : election: The mesh type of master is not ROOT AP.

I collected some more Information.
It seems following happends.
On one day at 02:06 AM the Master reboots and the other Root-AP with Standby-Master-Role will be the new Master. Next Day at 02:06 AM the (new) Master reboots again and the Master Role is switching again.
On both APs the same reboot reasion is listed.
Photo of Marco Eichstetter

Marco Eichstetter

  • 128 Posts
  • 7 Reply Likes
For me it seems the Master, who is the Root-AP at the same time, is changing his Status from Root-AP to Non-Root-AP. Because a Non-Root-AP is not able to be a Master an Election Process is starting.

Maybe the Firewall (Default Gateway) has a daily Reboot configured.
Because i don't have Access to the Firewall i just configured a periodic Ping with Timestamp to check if there is a reboot of the Default Gateway.

Maybe this is the Problem.
We will see tommorrow.
Photo of Marco Eichstetter

Marco Eichstetter

  • 128 Posts
  • 7 Reply Likes
So there is a Feedback for this.
There is a daily scheduled Reboot of the Firewall (Default Gateway). So my Master-AP seems to believe he is no Root-AP and is change his Role to Mesh-AP. Because a Mesh-AP cannot be the Master, the AP starts to reboot.
This is, what i think happends within my setup.

This also means, if the Firewall Admin do a Firmware Update of his Firewall and do a reboot, my Ruckus Unleashed Mesh Network will be disturbed, too. Ok, the reboot will take only some Minutes. But of the Firewall gets broken it will take maybe some days to recover the Device.

I think an AP checks if he can reach the Default Gateway over his cabled Connection to decide if he is a Root-AP or a Mesh-AP.
Maybe it is possible to add a second IP (a Server or other Device which is running 24/7) for checking. So if the Firewall is broken/down, my Mesh Network will not be down. Feature Request?

Thanks.
Photo of Michael Brado

Michael Brado, Official Rep

  • 1889 Posts
  • 269 Reply Likes
Good morning Marco,

     We've just released Unleashed 200.2 (GA) into production, and the cloud upgrade
server will upgrade your Unleashed network to version 200.2.9.13.186.  We have now
adapted the Master/Standby-Master functionality, with a simpler election model for one
Master AP and all others being Members.  This election model will consider MAP vs
RAP status of your APs.  Define a Management IP address that all APs know, to keep
web browser access in case your Master AP ever gets rebooted.

     Please see the What's New and Release Notes, with Quick Start Guides for each
Unleashed AP (added R710, T710, T710s), and you can find individual AP versions on
the Downloads page, if you don't have an Internet connection to upgrade, as Unleashed
200.2 now has Local as well as Online upgrade options.

https://support.ruckuswireless.com/products/82-unleashed

Unleashed 200.2 Online Help:
http://docs.ruckuswireless.com/unleashed/200.2/index.html
(Edited)