Add second ICX switch

  • 1
  • Question
  • Updated 3 weeks ago
First of all, great product and great community! Thank you!

So today I went out and bought another ICX switch (7150-c12p) and would like to use it on another room. It is running the router image (layer 3)

My core router has two vlans: vlan 30 (users) and vlan 200 (phones).

How would I go if I want to configure this new router to support both vlans.

Here is my try (didn't work btw):

lag test static id 1
ports ethernet 1/2/1 to 1/2/2

vlan 30 name users
untagged ethernet 1/1/1 to 1/1/12
tagged lag 1

vlan 200 name phones
tagged ethernet 1/1/1 to 1/1/12 lag 1

ip route 0.0.0.0/0 192.168.1.1


192.168.1.1 is my gateway.

Am I over complicating things?

Thanks!
Photo of Fernando Flórez

Fernando Flórez

  • 22 Posts
  • 6 Reply Likes

Posted 4 weeks ago

  • 1
Photo of Roberto Hernandez Jr

Roberto Hernandez Jr, Employee

  • 19 Posts
  • 7 Reply Likes
Hi Fernando,

We need to know more information before we can provide the appropriate answer. I assume that your ICX is directly connected to the router? If yes, then in order for the ICX and router to pass traffic, the interfaces facing each other need to be defined as trunk (tagged) and have the same VLANs as membership. For example:

router[int1]----TAG----[int2]switch

This means that in router[int1] you need to define it as tagged interface and on the [int2]switch, you also need to define it as Tagged. Then you need to make VLAN30 and VLAN200 part of those TAGGED interfaces. 

Here is a video that talks bout this:

https://www.youtube.com/watch?v=ixqn-CuuTJM

Regards,
 _Roberto H
Photo of Fernando Flórez

Fernando Flórez

  • 22 Posts
  • 6 Reply Likes
I'm trying to add a second switch below the core which is connected directly to the router.

My current topology is as follows:

router -> core switch (icx 7150-24p)

I would like it to be:

router -> core switch (icx 7150-24p) -> switch 2 (icx 7150-12p)


core switch implements:

vlan 30 name users
untagged ethernet 1/1/1 to 1/1/24
router-interface ve 30

vlan 200 name voip
tagged ethernet 1/1/1 to 1/1/24
router-interface ve 200

lag main static id 1
ports ethernet 1/2/1 to 1/2/2

interface ve 30
ip address 10.0.3.0/24

interface ve 200
ip address 10.0.4.0/24

inteface lag 1
ip address 10.0.0.2/24

ip route 0.0.0.0/0 192.168.1.1


second switch should implement vlan 200 (shared between switched) and an extra vlan 40:

vlan 40
untagged ethernet 1/1/1 to 1/1/12
router-interface ve 40

interface ve 40
ip address 10.0.0.3/24


All vlans should be able to talk between them.

Is this a correct setup? Sorry if this is a stupid question but it's my second week trying to learn this :)

Thanks!
Photo of Roberto Hernandez Jr

Roberto Hernandez Jr, Employee

  • 19 Posts
  • 7 Reply Likes
If all the VLANS need to go across then your configuration is not correct. You need to match at every point. Not having your full topology definition makes it hard to tell you what to do but here is an example and hopefully that helps.

Corerouter[A]-----[B]coreswitch[C]------[D]accessswitch[E]-------PC

Where:
[A, B, C, D, E] are interfaces in the respective device mentioned.


The connection between PC and accessswitch[E] needs to be untagged (considered access).
The connection between [D]accesswitch and coreswitch[C] needs to be tagged on both sides
The connection between [B]coreswicth and corerouter[A] needs to be tagged

Meaning:

configuration on accessswitch:
vlan 30 
untagged ethernet [E]
tagged ethernet [D]

configuration on coreswitch:
vlan 30 
tagged ethernet [C]
tagged ethernet [B]

configuration on Corerouter (not sure what your core router is so I'm just illustrating):
vlan 30 
tagge ethernet[A]


With the above configuration, PC will be able to send packets to corerouter on VLAN 30. You would repeat the same process for other VLANS. Please watch the video I sent you. There are other ways to accomplish this as well but not knowing your ultimate goal, it would be hard for me to suggest anything.
Photo of NETWizz

NETWizz

  • 26 Posts
  • 8 Reply Likes
Good Morning, sir:

On both the 7150-24P and the 7150-C12-PD, I would run the Layer-2 firmware if the routing process is on the router.

Next, I would not create a LAG unless you are using LACP.  From the router I would enable TAGGED or TRUNKING (terminology varies by vendor) and at any rate set it to use 802.1q AKA dot1q for the trucking.  Some vendors implement this as a sub-interface.  That said, you generally need a static lag if going to Brocade/Foundry/Ruckus equipment, and even then I do not know if the router is going to support sub-interfaces over LACP etc. 


At any rate looking at your 7150-24P I see you have it setup with a routing process, and it is hosting several subnets 10.0.3.0/24, 10.0.4.0/24, and 10.0.0.0/24

Please keep in mind this for example won't work because this is a network address or subnet-id; specifically, it is not a usable IP within the range of the subnet - I do not even know that it would actually take this:

interface ve 30
ip address 10.0.3.0/24
!

In contrast this would work and be the most common implementation (most folks start at one):

interface ve 30
ip address 10.0.3.1/24
!

Either way, the switch would insert 10.0.3.0/24 into its routing table as a directly connected network available via SVI of ve 30.


**************

The next issue is this:

inteface lag 1
ip address 10.0.0.2/24

ip route 0.0.0.0/0 192.168.1.1


The Router and the 7150-24P must be physically connected with layer-3 interfaces in the same subnet - they must be within the same layer-2 for any shared link.  It would appear the router is probably within 192.168.1.0/24 (specifically 192.168.1.1) though I cannot see the mask on the router.  Regardless, nothing from 192.168.1.0/24 can communicate with 10.0.0.0/24 when connected on a shared link because there are no routing tables.

Specifically, from your Brocade you would not be able to ping 192.168.1.1 because the router would have to source the ping from 10.0.0.2, which is not in the same subnet.


*********

Next once you get the routing working, the router needs a route back.  Specifically, the router that has the 192.168.1.1/24 assigned to one of its interfaces needs to know that to get to 10.0.0.0/24, 10.0.3.0/24, and 10.0.2.0/24 that it needs to do it via some entry in its routing table to point it at that Ruckus 7150 via an IP in the same network on its physically connected link.  Otherwise that router is just going to drop the traffic.

***********

Next, I presume the router is setup to access the network given its IP configuration looks like a consumer-grade product... that it is unlikely to be an Enterprise WAN router.  Hence, the next question is with regard to NAT translation for Internet sharing.  Specifically, is it translating for other subnets than 192.168.1.0/24??


**********

On the next switch, the 12-port, you implement another VLAN #40.  I am not certain the purpose, but not only that you have a routing process for it there by creating an SVI by defining an IP/mask on that VE interface for it.


Now as far as that 7150-C12-PD is concerned, it is the source of truth and being directly connected 10.0.0.0/24 is available with a metric of 0 (higher priority than even a static route)...


... but you have this same subnet on the 7150-24P on the LAG interface for connectivity to the router.  Yes, you have a different IP


I think the solution here would be to dispose of that VLAN all together and trunk (tag) VLAN 30 from the 7150-24P to the 7150-C12-PD then make all the other ports access ports (untagged) in VLAN 30.


You would also TAG your phones VLAN 200 into the C12 unit the same way and deliver that to the 12 interfaces via TAGGED as well presuming the phones are configured to TAG traffic onto VLAN 200.  Typically, we call this a Voice VLAN in the field because that name stuck.  The untagged VLAN on an interface that is the Native VLAN.  Prior to 08.0.80, Brocade/Ruckus used the dual-mode key-word in configuration to configure native VLANS.  Now you can make any specific interface untagged in only one VLAN and tagged to as many VLANS as you like.


Regardless, these rules exist because any Layer-2 frame can either have an 802.1q field in the header (i.e. tagged) or not, and if it is tagged then that specific frame can belong to only one specific VLAN.  An Interface can read the tag and sort the frame into the proper VLAN for which it is marked, or if no tag is present (untagged) it needs to know what to do with it and can send it to only one VLAN (that is whichever one it is untagged within).  That would be dual-mode with older firmware Ruckus/Brocade/Foundry, and other vendors would refer to this as the Native VLAN.


***********


If these are within the same wiring closet, you may want to stack the units if the reason you have the 24p and the C12 units are to get 36 ports.  It would reduce the overhead of configuration and management.  Realistically, if the router hosts both the Voice and Data, you would probably be better off just TAGGING/TRUNKING all VLANS from the router to the Ruckus units (or logical unit if stacked)... You would likely be better served with the Layer-2 firmware on the Ruckus units leaving the entire routing process on the Router.


Lastly, you may want to look into lldp-med on the Ruckus.  It properly configures the switch to talk to the phones and announce the Voice VLAN and other configuration details via the network discovery protocol LLDP (Link-Layer-Discovery-Protocol).  It actually works with Cisco phones, too... The MED part of the acronym is Media-Endpoint-Detection - it is a supplement of LLDP.


Best of luck, sir


Justin

(Edited)
Photo of Fernando Flórez

Fernando Flórez

  • 22 Posts
  • 6 Reply Likes
Thank you all!

So I think I've made some mistakes during the writing of this post and also mistakes trying not to openly comment on my subnetting.

The ip address used on the ve interfaces is wrong. It's a typo. The correctly IP addresses for the ve interfaces end in a .1 indeed and no .0 (sorry bout that).

My router's ip is not 192.168.1.1 actually, it is 10.0.0.1/24

To clear up things (sorry bout that again) this is my current setup on the larger switch (layer 3):

it has 2 vlans: 30 (users) and 200 (voice). All routing is done on this switch and it has a static route to the router (ip route 0.0.0.0/0 10.0.0.1). Actually I would like to only hit the router for external (internet) requests.

Now what I would like to do is to integrate an extra switch (12 port) into this. I know I can stack them but as a learning escenario I would like not to as this would be my common case.

As a way to separate users (not voice) I would like to place them on a different subnet (10.0.4.0/24) on the smaller switch.

Do I need to turn the smaller switch to a layer 2 for this or can I keep it as layer 3?

The new vlan 40 (10.0.4.0/24) should be declared on the big switch, small switch or both?

I would like both switches to communicate through a lag just so I can have a backup connection in case the first one goes down.

With the implementation of both switches I would still like to keep the routing inside the switches and only hit the router for internet access.

BTW, large switch has a static ip of 10.0.0.11 and that how it communicates with the router back and forth.

Next step would be to learn about spanning tree and then about voice priority.

Thank you!
Photo of Scott Farrand

Scott Farrand

  • 9 Posts
  • 3 Reply Likes
Honestly, it sounds like this is a small network and probably doesn't need routing configured.

Having two devices with vlan 30 configured, but with different IP addressing will require routing, and you'll need to keep the traffic separated by a routing device (could be static routes).

Unless you have a reason to have a separate broadcast domain/routing environment (say this were at a remote site rather than in the same IDF/building), I'd probably keep all of your users on the 10.0.0.0/24 segment instead of introducing 10.0.4.0/24...

The 10.0.0.11 ip address on your larger switch is very likely only used for management purposes, not for routing.

What was said above about stacking being an easy solution for extending the configuration to another switch for ease of management is 100% correct.

If you're not stacking the switches, you need to identify how the traffic is passing back and forth between the switches and while untagged 30 and tagged 200 will work, it would be cleaner to have tagged 30 and tagged 200 for the "trunk" port.

You didn't mention brand of voip phones - chances are very good that configuring LLDP-MED will help with the voice vlan assignment for your phones.
Photo of Fernando Flórez

Fernando Flórez

  • 22 Posts
  • 6 Reply Likes
Trunk was the keyword I needed! Thank you!

Understood and also got it working properly.
Photo of NETWizz

NETWizz

  • 26 Posts
  • 8 Reply Likes
What he is saying that is key is that the reason you want to trunk is to not risk inconsistent VLANs.

Specifically, if you had untagged vlan 60 connected on another switch to untagged vlan 70, frames end up moved to the wrong VLAN and it creates hastles to troubleshoot.

****

What you never said is if your two switches are in the same network closet.  If yes, you would be better off stacking them from both a configuration and a management perspective.

You would make them both run the same firmware to stack...
config t
stack enable
hitless-failover enable
exit
stack secure-setup


Verify the topology and unit ID numbers....

Then you have one (1) logical switch.


It's just more clean that way.
Photo of Fernando Flórez

Fernando Flórez

  • 22 Posts
  • 6 Reply Likes
Currently both switches are sitting together but wanted to learn how to do it this way. Already ordered a couple of SFP cables that may arrive soon so I can learn how to stack.

Now onto learning how to give voice priority with lldp.

Thank you all! And sorry for not explaining myself better sometimes.
Photo of NETWizz

NETWizz

  • 26 Posts
  • 8 Reply Likes
Sounds good.

Now, LLDP is not a prioritization protocol but rather a network discovery protocol similar to FDP (Foundry Discovery Protocol) used by the Ruckus/Brocade switches.  It is similar to CDP, which is Cisco Discovery Protocol.

For prioritization, you usually use QoS.  Typically, this is done on the router itself as it hands-off to the provider and they prioritize the traffic back to you.  The prioritization queue expedites packets that match a classification reducing latency, and it also provides a bandwidth reservation/guarantee.


What you normally do within a network is flag or mark the traffic as close to the source as possible.  Most phones set the DSCP field to EF (46) for Expedited Forwarding.  This is a Lyer-3 field in the packet that other devices can read and classify the traffic into QoS queues.


Hope that helps.  What LLDP does is announce the VLAN the phones should use etc.

Photo of Scott Farrand

Scott Farrand

  • 9 Posts
  • 3 Reply Likes
If the phone's setting the dscp and/or COS, and if the traffic is tagged, I believe the DSCP and COS tagging are "trusted"... but you may want to specifically enable dscp trust on these ports.

This might be meaningful to take the time to read... http://www.netadmin.us/docs/IP_Phone_Port_Configuration.pdf

Also, I believe this is a good thing to configure as well.

qos mechanism mixed-sp-wrr
Photo of Fernando Flórez

Fernando Flórez

  • 22 Posts
  • 6 Reply Likes
After some testing I found out that this is actually not solved.

Could someone please give me some guidance on how to connect two switches (one as a core and the other one as an access switch)? Can both be on router mode?

Thank you!
Photo of NETWizz

NETWizz

  • 26 Posts
  • 8 Reply Likes
While you can run routing processes wherever you want and on as many devices as you would like, any given subnet originates from one device where it is a "directly connected" network.  A device that performs routing we refer to as a Layer-3 device (referencing the OSI model for the layers).  Another term I commonly see is multi-layer switching, and that term seems to be predominant with Cisco.

That said, any specific Layer-3 device knows only about its "directly-connected" networks.  That is to say if there were two (2) directly connected networks (subnets) and it saw a packet (the PDU for Layer-3 is "packet" - it has the To/From IP addresses), the layer-3 device recognizes the packet belongs to a different directly-connected network it has, and that device will route the packet.

However, there is a problem where any given device knows only about its own directly-connected networks.  It is unaware how to get to networks that are directly-connected on other devices.  For this, you build routing tables via simple static route entries or you use a dynamic routing protocol, typically an IGP to dynamically create and update these routing tables.  Predominantly, the most popular ones are OSPF (multi-vendor) and EIGRP (Cisco proprietary). 


When you connect Layer-3 devices, you MUST connect them on a shared subnet.

For example, if you have two (2) routers each with the following networks:

192.168.1.0/24

172.16.0.0/16

and the next one

172.16.0.0/16

10.0.0.0/8


The first layer-3 device has no idea how to get to 10.0.0.0/8 and the second layer-3 device has know idea how to get to 192.168.1.0/24.

You would connect them on the shared subnet they both know how to get to, the 172.16.0.0/16, and they would each get a unique IP in that network.

For example:  172.16.0.1 and 172.16.0.2 could be used.

Then if you were to enter a static route you could describe how to get to the network you don't have via a network you do have.

For example (172.16.0.1 would be the IP on R1, and 172.16.0.2 is the IP on R2):

R2:  ip route 192.168.1.0 255.255.255.0 172.16.0.1

R1:  ip route 10.0.0.0 255.0.0.0 172.16.0.2

*************
*************


Now you asked, "Could someone please give me some guidance on how to connect two switches (one as a core and the other one as an access switch)?"


Here is an example:



************

hostname Access


vlan 10 name Data by port
   untagged ethe 1/1/1 to 1/1/24
   tagged ethe 1/2/1
!

vlan 20 name Voice by port
   tagged ethe 1/1/1 to 1/1/24 ethe 1/2/1
!


interface ethernet 1/1/1
trust-dscp
inline-power
!

<truncated for brevity each of the 24 interfaces are configured the same>

lldp run
cdp run
fdp run

lldp med network-policy application voice tagged vlan 20 priority 5 dscp 46 ports ethe 1/1/1 to 1/1/24



****************

hostname Core


vlan 10 name Data by port
   tagged ethe 1/2/1
   router interface ve 10
!

vlan 20 name Voice by port
   tagged  ethe 1/2/1
   router interface ve 20
!

vlan 100 name WAN by port
  untagged ethe 1/2/2
  router ve 100
!

ip route 0.0.0.0/0 10.123.1.1

lldp run
cdp run
fdp run

interface ve 10
 port-name Data Network Default-Gateway
 ip address 10.0.0.1 255.255.0.0
 ip helper-address 1 10.1.2.3
!

interface ve 20
port-name Voice Network Default-Gateway
 ip address 192.168.1.1 255.255.255.0
 ip helper-address 1 10.1.2.3
!

interface ve 100
 port-name to WAN router or Internet Firewall
 ip add 10.123.1.2 255.255.255.252
!

****************

Explanation of the examples above:

The Access switch is a Layer-2 device or a Layer-3 device with no directly-connected networks configured.  At any rate, you just connect the shared VLANs as trunk links (tagged).


In the above example an access switch (Layer-2) is connected to a core switch (Layer-3).  There are two VL AN 10 for Data and 20 for Voice.  Both devices are connected via port 1/2/1, which is a trunk using 802.1q to carry both "tagged" VLANS through the one, shared cable.


The access switch does NOT do any routing whatsoever, but it trusts DSCP and has inline-power enabled even though that wouldn't be shown in 08.0.80x firmware, which is what you used above, so it is with respect to this I make the configuration..  For any older firmware, you would use "dual-mode 10" on each interface to configure the Native VLAN.


We have all the discovery protocols on for the phones, which we are configuring to pick-up VLAN 20 automatically via lldp-med.


The Core does ALL of the routing via some SVIs (software virtual interfaces) known as VE's on Brocade/Ruckus/Foundry.  These would be "interface vlan 123" if you ever come across Cisco.  At any rate, they are configured with the subnet, so we have a 10.0.0.0/16 subnet to carry data, and a 192.168.100.0/24 subnet to carry Voice.


The core is forwarding DHCP requests to your DHCP server 10.1.2.3, which is presumably somewhere else in your WAN or your router wherever.  On the Core, I added VLAN 100 for routing, and made it untagged 1/2/2 for its uplink to your router, an Internet firewall, a WAN, something...

At any rate... the IP route statement sets the route of last resort to point to the other side of that WAN link (or your router).  Pretty much for anything not directly connected it just forwards it along.  Presumably, this next device knows how to get to your DHCP server 10.1.2.3 because I don't have that off of the core though you certainly could do that.  ;-)



*************

You asked: "Can both be on router mode?" ???

Each subnet should be created on only one(1) device that does the routing.  While you could put one of the VE's on one device and the other subnet on the VE of the other device, it would just become a mess because now you would need a shared subnet between the two devices for routing and routing tables constructed somehow, so each device knows how to find each respective subnet.

In the real-world, this is usually done with an aggregation-layer (or distribution-layer) added between the edge and the core.  From the aggregation-layer switch is often located at a site off of a WAN circuit and you would generally trunk your VLANS to the edge switches in the wiring closets, which would then have access-ports to the computers or more likely it would carry both Data and Voice by presenting a Native VLAN (no 802.1q tag) and Voice VLAN available by placing an 802.1q tag on the ethernet frames (frames are the PDU for layer-2 - they have MAC addresses instead of IP addresses though they encapsulate the Layer-3 packets within).  The aggregation-switch would have the VEs for the subnets available at that site or area because these would be directly connected networks, and it would have a point-to-point link typically a /30 back to the core.  This is where you would probably run OSPF (or similar) such that the aggregation-layer switch would announce its directly-connected networks to the core.


In your small network, I strongly recommend ALL subnets are created as being directly-connected to your Core, which has its VLANs stretched directly to the access switches that are simple, layer-2 switches.  No need to overcomplicate it.  The example above should work great if you tweak the subnets to the ones you actually use, and you let your router know that the Data and Voice subnets are available via your Core, too.
(Edited)
Photo of Fernando Flórez

Fernando Flórez

  • 22 Posts
  • 6 Reply Likes
Thank you! Thank you!