dawoon_lee's profile

6 Messages

 • 

148 Points

Mon, Dec 13, 2021 1:51 AM

[CVE-2021-44228] Apache Log4j2 RCE

Hello.

Our customer is running a Ruckus SmartZone (sz-100) controller.
The version of the controller is 5.1.1.0.598.


The customer asked if the SmartZone has the following this security vulnerabilities.

** Vulnerability: [CVE-2021-44228] Apache Log4j2 RCE

Thank you for your valuable answers to the above questions.

Official Solution

Brand User

Official Rep

 • 

76 Messages

 • 

1.5K Points

1 m ago

The KBA is now live in the Support Portal at https://support.ruckuswireless.com/articles/000012025 and it's linked in each of the KSP patches.

The KSP patches and the updated Security Bulletin v1.3 are now linked from the new Log4j - RUCKUS Technical Support Response Center at https://support.ruckuswireless.com/log4j-ruckus-technical-support-response-center

The delay you saw with the KSPs being published earlier and then pulled was because 1) the KBA with KSP instructions didn't synch properly, so we pulled everything down until it was available, and 2) the KSPs had to be regenerated due to technical issues within the compression process (they were downloading as the wrong file types in Chrome). 

Everything is now up-to-date and available.  As you have feedback, please continue to chime in here---TAC is monitoring the thread actively through @vineet_nejawala and other engineers.

Thank you for your patience and your feedback throughout this process, and happy patching!

Allan.

(edited)

12 Messages

 • 

284 Points

@allan_grohe I'm on macOS Monterey 12.1 and the native built-in archive utility seems to extract the zipped patch/ksp into 3 files (digital_sig.bin, signing_cert.pem and the *.ksp file) and the KSP would not upload (displayed invalid file). I unarchived it with Keka for macOS and it was just one file (the *.ksp) and it uploaded into the controller with no issues.  Just a possible FYI for others that may experience this using the macOS built-in archive utility to extract the file. 

(edited)

75 Messages

 • 

1.1K Points

@JTakaMT Same thing for me, thanks for dropping the tip!  Keka FTW!

Official Rep

 • 

170 Messages

 • 

2.2K Points

@JTakaMT Yes the patch that needs to be uploaded is only .ksp file. Thank you for sharing your inputs that would help others.

Best Regards

vineet

kristphr

38 Messages

 • 

716 Points

@JTakaMT thank you for this!

Brand User

Official Rep

 • 

76 Messages

 • 

1.5K Points

@JTakaMT:  thank you, I should have mentioned that as well, since our TAC director runs on a Mac and did see that too.

@vineet_nejawala :  can you or Sameer please update the KBA with the MAC-specific guidance re: the decompression process?

Allan.

Allan.

==

Allan T. Grohe Jr.

Knowledge Management Program Director
for RUCKUS Customer Services & Support

Official Solution

Official Rep

 • 

170 Messages

 • 

2.2K Points

1 m ago

@ludia_it @nick_nordberg @michiel_timmers @mark_pledl 
We have updated out KBA and for 6.0 users, we recommended customer to "reload" vSZ instead of "service restart" after KSP is applied which would work correctly. 

Best Regards

vineet

4 Messages

 • 

46 Points

@vineet_nejawala 

with a reload it works fine on 6.0

thank you.

Official Rep

 • 

170 Messages

 • 

2.2K Points

1 m ago

Hello @dawoon_lee 

This vulnerability is really new and our Engineering has been notified about this issue to check if this vulnerability is affecting us and how we can mitigate the effects. We do understood this is a critical situation and we will update you with the information from our internal team. Below is our link where we will soon add our response.

https://support.ruckuswireless.com/security

Best Regards

Vineet

6 Messages

 • 

148 Points

@vineet_nejawala 

I look forward to your sincere reply.
Thank you.

12 Messages

 • 

284 Points

Thanks for responding Vineet. We are looking forward to a Security Bulletin/Announcement update from the Ruckus team soon. 

Official Rep

 • 

170 Messages

 • 

2.2K Points

@michael_thompson_e3bsvnhy1spi9

Thank you. The bulletin would be updated soon, so far we have been confirmed that code 3.6.2 is safe from this vulnerability and further is being tested.

Best Regards

Vineet 

Official Rep

 • 

170 Messages

 • 

2.2K Points

14 Messages

 • 

250 Points

@vineet_nejawala @allan_grohe 

can you please confirm that log4j2 is now version 2.17 (or newer). I got that request from a customer as 2.16 has a CVSS score from 7.5 regarding DoS. Thanks in advance. ( I am sorry but I do not find details in the link and these customers want to know it exactly)

Thanks in advance.

(edited)

165 Messages

 • 

2.8K Points

1 m ago

Hi Vineet. I have an open case 01288986 asking engineering about the same question

Ai can't believe I was the first on to open such case, but they way the ticket is being dealt with seems as if there is no coordinated effort. We're past 72 hours after the initial discovery and I would have expected at an absolute minimum an announcement on the website or mailing list. 

Also, at least a basic set of responses such as product x is using /not using log4j version y with jdk / jre version Z. At an absolute minimum, some basic communication.

My very first tests dont seem to indicate that the system is exploitable... (Referring to SZ 5.2.1) but these were basic fuzzing tests. 

6 Messages

 • 

120 Points

1 m ago

Has Ruckus put out a public statement on this?  I can't seem to find anything on their website for it.  Can/Should we be shutting down our virtual VSze servers to protect systems?

Official Rep

 • 

170 Messages

 • 

2.2K Points

Hi @tom_lebel

We are expecting a response soon on this. Commscope is aware of the latest Vulnerability CVE-2021-44228. Our engineering team is currently performing the appropriate assessment on all our product lines . This is the highest priority for us and we will update our security bulletin as soon as more information is available on the same. Here is the link to our security bulletin which will be updated soon: https://support.ruckuswireless.com/security 

Best Regards

Vineet

(edited)

14 Messages

 • 

250 Points

@tom_lebel -  there is no public statement. I am no Ruckus employee.

Can you please make a ticket to Ruckus - they will update you with information. I don't want to hand out any details without their permission.

Hope you understand!

Br,

Mark.

Brand User

Official Rep

 • 

76 Messages

 • 

1.5K Points

1 m ago

The RUCKUS Security Bulletin addressing Log4j is now published at https://support.ruckuswireless.com/security_bulletins/313

Allan.

165 Messages

 • 

2.8K Points

@allan_grohe hi Allan 

The advisory seems to be only available in text format (and not pdf) and ALL text advisories return an error and do not load. Other vulns have a pdf version and we can access that but the text version just errors out

Also, given the criticality of the incident it would be good for the advisory to be available without a support account. 

Official Rep

 • 

170 Messages

 • 

2.2K Points

@diego_garcia_del_rio

The text format is working well and we are working on pdf part too, meanwhile please find below content from our official response :

What is the issue?
A vulnerability was found in the Apache Log4j logging library from version 2.0 to 2.14.1. Products utilizing this library are susceptible to remote code execution vulnerability, where a remote attacker can leverage this vulnerability to gain full control of the impacted device.
For more details about this vulnerability, please see https://nvd.nist.gov/vuln/detail/CVE-2021-44228.

What action should I take?
RUCKUS is releasing the fix for these vulnerability through a software update. Since it is a critical issue, all affected customers are strongly encouraged to apply the fix once available.

In case of any questions contact RUCKUS TAC through regular means as described at https://support.ruckuswireless.com/contact-us and refer to this document to validate this entitlement.

Are there any workarounds available? 
No

What is the impact on Ruckus products?


The following products are not vulnerable: All Access Points, ZoneDirector, Unleashed, ICX Switches, SPoT/vSPoT, and RUCKUS Cloud.

The following products are under assessment: Cloudpath, IoT, MobileApps, RUCKUS Analytics, and SCI.

The following table describes the vulnerable products, software versions, and the recommended actions.

(edited)

165 Messages

 • 

2.8K Points

Thanks for the text. Download is working now. Is there an ETA on the patch? 

Is it possible to exploit this from the customer side (i.e. though a radius request, wispr login in proxy mode or similar?) Is there any partial mitigation (for example blocking access to the API / port 8443?) 

Is there a way te check if a system was infected, and, if so, can it be cleaned in any way? Or is a clean reinstall the only option ?

It's quite worrying and the advisory is extremely thin on details when compared to other advisories / alerts from other vendors

Best regards

Official Rep

 • 

170 Messages

 • 

2.2K Points

@diego_garcia_del_rio

We are waiting for more details/patch ETA (which should be soon) and engineering is working on solution(ksp) as we speak.

The exploit uses JNDI (Java Naming and Directory Interface), this causes the logging library to create an outbound connection to an LDAP server.Until patch is available from engineering firewalling can be used as protection from this attack:

Restrict inbound network access to SmartZone Services (Including the admin interface)
Restrict outbound connections from the SmartZone to the internet and other networks
As this attack requires an outbound connection from the SZ, if the SZ can not get out to the attacker the attack is mitigated.Limit any access from any unauthorized endpoints.

Note : Customer's running on vSZ code 3.6.2 do not have this vulnerability impact and hence do not need any ksp or solutions.

Best Regards

Vineet

 

(edited)

5 Messages

 • 

102 Points

@diego_garcia_del_rio You should expect this to be triggered from everything which produces a log line. This could be simply be the username of a client trying to authenticate or a SSID of a Rogue AP.

165 Messages

 • 

2.8K Points

1 m ago

Hello everyone concerned,

We've seen some recommendations to block outgoing connections from smartzone as a possible mitigation / protection (since the blocked outgoing connection would prevent the malicious ldap download from occurring). 

Keep in mind that a port other than the standard LDAP could be included in the exploit URL so its not enough to just block outgoing LDAP traffic.

Is there any way that ruckus can provide a list of "expected" IPs that the smartzone would connect to in normal operation?

Of course, any proxied radius connections, ftp server for logging or backup, etc would be entirely up to me, the administrator, to add.

But I have now added a block-and-log rule and I'm seeing outgoing connections to akamai from smartzone, in particular to IPs 23.205.105.175 and 23.205.105.155

2 Messages

 • 

72 Points

1 m ago

I just want to add, that you can check if there were attacks by searching the "web-critical" log for "jndi:ldap".
Does anybody know if the control interface is vulnerable too?

Official Rep

 • 

170 Messages

 • 

2.2K Points

@bjarne_goldau

Management interface would be the most likely vulnerable since a user can input a specific string from UI/Public API to trigger this vulnerability but IMHO any way that an attacker can input a specific string into logger directly or indirectly via any interface will affect SZ .  On the fix (ksp patch) we are currently completing testing cycle to make sure there is no regression and once completed it would be out.

Best Regards

Vineet

2 Messages

 • 

72 Points

Hi Vineet, Management interface is vulnerable, i tried that myself. I am asking about the control interface where the accesspoint connect to. Do you have any information on that?

Official Rep

 • 

170 Messages

 • 

2.2K Points

@bjarne_goldau

No information on that part so far. I would update here as soon as I hear one.

Best Regards

Vineet 

14 Messages

 • 

250 Points

@bjarne_goldau 

Hi Bjarne,

we let ran security scan (CERT BSI and so on included - only TCP - takes several hours) over our AP control interfaces (vSZ-H) with 5.2.2.0.1161. It just complained about icmp uptime spoofable, weak ssh key support from DSA and SH1 support (I think has to do with Centos 7 below it and maybe support of older APs). No log4j complain from it - but Ruckus needs to confirm.

165 Messages

 • 

2.8K Points

no cert scan / port scan will find this vulnerability. You need to be monitoring both the log files as well as an upstream DNS server to see if the system is issuing queries. I doubt any of the scanning tools claim to find this and/or have been updated so quickly as to add the attack. It would require you to poing smartzone to a particular dns server and the attack would be via API fields which most tools do not scan in depth. Personal testing (and others here on this thread) have shown the system to be vulnerable, at the very least, to information disclosure attacks through jndi / log4j.

It's different for a WAF (web application firewall) that can look at the API calls being made to smartzone. The problem here is that smartzone potentially has multiple attack vectors outside of the API channel, where data sent FROM the AP could trigger this. For example, a maliciously named client uses the ${jndi string as its dhcp hostname. I have not verified this path of exploitation, but so many user-related data is sent to the controller and logged that the risk is extremely high.

Brand User

Official Rep

 • 

76 Messages

 • 

1.5K Points

1 m ago

The v1.2 update to the Security Bulletin is now live in the Support Portal at https://support.ruckuswireless.com/security_bulletins/313 and the files will be mirrored on the www page in the morning:  https://www.commscope.com/security-bulletins/

Allan.

5 Messages

 • 

102 Points

@allan_grohe Thanks for the update. But it is really unclear how we get this update. Will this be a general download or needs this to be request from customer support?

And will the fix include the lastest recommendation from Apache LOG4J?

Because it seems that the first mentioned mitigation strategies are not sufficient. See recent announcement:

https://logging.apache.org/log4j/2.x/security.html#History

Cite: "This page previously mentioned other mitigation measures, but we discovered that these measures only limit exposure while leaving some attack vectors open.
...
The safest thing to do is to upgrade Log4j to a safe version, or remove the JndiLookup class from the log4j-core jar."

Safe version would be the latest 2.16

Official Rep

 • 

170 Messages

 • 

2.2K Points

@torge_szczepanek

We will share the ksp details as soon it is available and Yes the fix would include latest recommendation as engineering is constantly checking it and updating on our side too.

Best Regards

Vineet

5 Messages

 • 

100 Points

@vineet_nejawala The last version of the advisory (1.2) says that SZ 6.0 KSP should have be released 12/15/2021 (yesterday).

I tried to find it everywhere but was not able to. It also says to contact Customer Support to install it when available. Will we be contacted by email when it's available or we have to refresh the Downloads section each day to know it's released ?

O.

14 Messages

 • 

250 Points

@ludia_it

I got information that it is not ready and still under testing, so you cannot find it. They expect it to be released by end of 17th. More detailed information you can get only from vineet, allan and colleagues.

Br,

Mark. 

(edited)

12 Messages

 • 

284 Points

Copied from the security bulletin below, I went ahead and put in a support ticket as well just to have it in. 

SmartZone and 					Contact Customer Support to install 	SZ 6.0 KSP
Virtual SmartZone   5.0 to 6.0			KSP when available.			12/15/2021

						Contact Customer Support to install 	SZ 5.2.2 P1 and 
						KSP when available.			5.2.2 KSPs
											12/16/2021

						Contact Customer Support to install 	SZ 5.1 and 
						KSP when available.			5.0 KSPs
											12/17/2021

4 Messages

 • 

46 Points

1 m ago

I also don't get any information about the patch via chat or support case.

We need a solution here very quickly.

Official Rep

 • 

170 Messages

 • 

2.2K Points

@sven_siersdorfer

Apologies for the inconvenience. The patch for all codes should be released by today "12/17/2021" EOD. The goal is :

We will have a KBA showing the process for loading the KSP on SZ.
The fixes /KSPs will be open for download for customers with/or without a support.
The aim is to ensure that our customers have the ability to self-help on existing versions as much as possible.

Atlast,Sorry for the delay, but this will be a better outcome for all our customers and us, once completed.

Best Regards

Vineet

 

(edited)

4 Messages

 • 

46 Points

Thank you Vineet

165 Messages

 • 

2.8K Points

1 m ago

To anyone wanting to mitigate the impact, the vulnerability has two parts:

1) Information disclosure. This one is hard to contain as it basically means the system is able to leak internal data via DNS requests. But this mechanism will not "infect" the system. Some information such as internal usernames, process permissions, environment variables can be leaked to external parties. But the vSZ will not be infected.

2) Full in "remote code execution". This part of the attack means an external party is able to force the system to DOWNLOAD code and execute it. This one is the most dangerous one.

In my case, I have been able to mitigate the RCE (#2) by blocking all outgoing communication from smartzone. You will have to add certain entries to the allow-list/whitelist such as any LDAP or RADIUS servers, syslog and email servers used for notifications and system logging, FTP/SFTP servers for backups, SPOT servers if you're using location based services,  but pretty much all other outgoing communication can be blocked.

This means that if the system TRIES to download the  malware, it should be blocked by the external firewall. 

In my case, smartzone is deployed in google cloud so I rely on google's cloud firewall to configure the blocks. I have only had communication to two IPs which I believe are either part of the licensing servers or something similar (it was akamai IPs). 

In my case, we have a lets-encrypt certificate on the server so im seeing certain connections to "ocsp.int-x3.letsencrytp.org" 

193 Messages

 • 

3K Points

1 m ago

I trust the Ruckus engineers are on this case, working diligently as they can. I will wait until the patch is thoroughly tested and released. Thanks for all you do for us!

Official Rep

 • 

170 Messages

 • 

2.2K Points

@eightohtwoeleven

Apologies for the inconvenience. The patch for all codes should be released by today "12/17/2021" EOD. The goal is :

We will have a KBA showing the process for loading the KSP on SZ.
The fixes /KSPs will be open for download for customers with/or without a support.
The aim is to ensure that our customers have the ability to self-help on existing versions as much as possible.

Atlast,Sorry for the delay, but this will be a better outcome for all our customers and us, once completed.

Best Regards

Vineet

 

(edited)

5 Messages

 • 

50 Points

1 m ago

We are +1 week in... and still no patch????

14 Messages

 • 

250 Points

@michiel_timmers 

patches are downloadable.

https://support.ruckuswireless.com/software

1 Message

 • 

64 Points

Saw the patches briefly but now don't seem to appear on the downloads page anymore. Did they get pulled?

14 Messages

 • 

250 Points

@muu 

Yes it looks like. Maybe something wrong with them.

14 Messages

 • 

250 Points

The are back now. But KBA #12025 is still missing.

12 Messages

 • 

284 Points

Missing for me as well. I have a ticket that was already opened and updated the support ticket asking about the KB article. 

3 Messages

 • 

110 Points

1 m ago

Anyone found the proper install instructions for vSZ 5.2.2?  When I follow the link from SZ and vSZ - Steps to Implement CVE-2021-44228 log4j2 Patch -- KBA # 000012025 and search for that number nothing comes up.  Why are Ruckus documentation and download processes so convaluted?

165 Messages

 • 

2.8K Points

@jtchi Same here. Searching for downloads for smartzone there is nothing recently released. All the posted links to KSPs lean no-where (error pages / 404 pages ). Searching the KB for 000012025  returns nothing. Has this been marked as "externally available"?

5 Messages

 • 

100 Points

1 m ago

Just finished patching. (vSZ 6) 

I tried to restart the services after the patch as documented (service restart) on the first node but after 1 hour it was still waiting on the same services to get up. 

I had to reboot the node (reload).

On the second one, I just used the (reload) command. 

1 Message

 • 

60 Points

@ludia_it 

I had the same issue here.  

The message that kept repeating was:

"Wait for (Cassandra,Communicator,Configurer,Core,Courier,ElasticSearch,Mosquitto,NginX,RabbitMQ,ScgUniversalExporter,Switchm,Web) up."

Mine is a 2 node vSZ-H on firmware 6.0.0.0.1213

I took your lead and logged in with another session and did a reload.  Came back up after that.

5 Messages

 • 

100 Points

@vineet_nejawala @allan_grohe 

I think you should review your documentation to just do a reboot (reload) after the patch is applied. 

Official Rep

 • 

170 Messages

 • 

2.2K Points

@nick_nordberg @ludia_it 

Strangely we haven't faced this issue in QA test and with customers that we have so far applied patch to. Thank you for sharing you input we will look into this further.

Best Regards

Vineet 

75 Messages

 • 

1.1K Points

I had no issues with the "service restart" on a pair of SZ-124 units in a cluster.  It took roughly 20mins.

5 Messages

 • 

100 Points

@vineet_nejawala Might be related only to vSZ version 6.

Important Announcement