I'm working on setting up a 4 unit stack of ICX 6450-48s. Firmware installed is 8.0.3n - Routing. I am getting a continuous stream of the following entries to the console port:
T=17h5m20.1: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-up, ,
T=17h5m57.7: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-up, ,
T=17h7m59.8: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-dw, ,
T=17h8m9.4: 1/2/1 goes down. delete direction 0, trigger election
T=17h11m20.3: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-up, ,
T=17h13m56.9: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-dw, ,
T=17h17m9.5: 1/2/1 goes down. delete direction 0, trigger election
T=17h17m38.5: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-dw, ,
In addition the input errors and CRC numbers are steadily increasing.
I'm using the standard 1m stacking cables.
Does anyone know what might be happening? Is there a different version of firmware (earlier/later) that I should be using instead?
Any help is appreciated.
Todd
T=17h5m20.1: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-up, ,
T=17h5m57.7: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-up, ,
T=17h7m59.8: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-dw, ,
T=17h8m9.4: 1/2/1 goes down. delete direction 0, trigger election
T=17h11m20.3: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-up, ,
T=17h13m56.9: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-dw, ,
T=17h17m9.5: 1/2/1 goes down. delete direction 0, trigger election
T=17h17m38.5: Election, was active, no change, ID=1, pri=128, 4U(1-4), A=u1, nbr#=3 3, reason: u2: port-dw, ,
In addition the input errors and CRC numbers are steadily increasing.
I'm using the standard 1m stacking cables.
Does anyone know what might be happening? Is there a different version of firmware (earlier/later) that I should be using instead?
Any help is appreciated.
Todd
- 7 Posts
- 1 Reply Like
Posted 10 months ago
Ben, Employee
- 56 Posts
- 9 Reply Likes
If a stack port flaps or loses communication, it will trigger an election. You need to clean up the stack ports if you are seeing errors. What is the media type? Have you tried other cables? Can you gather the following:
show stack
show stat e 1/2/1 e 1/2/3 e 2/2/1 e 2/2/3 (assuming these are the stack ports)
dm statistic
show media
show log
show stack
show stat e 1/2/1 e 1/2/3 e 2/2/1 e 2/2/3 (assuming these are the stack ports)
dm statistic
show media
show log
- 7 Posts
- 1 Reply Like
dm statistic shows the following info for the relevant stack ports:
123-0-0-0(1/2/1)
31-0-0-0(1/2/3)
3-0-0-0(2/2/1)
3-0-0-0(2/2/3)
5-0-0-0(3/2/1)
5-0-0-0(3/2/3)
3-0-0-0(4/2/1)
3-0-0-0(4/2/3)
Show Media:
show media | ex M-C
Port 1/2/1: Type : 1GE Twinax 1m (SFP)
Port 1/2/3: Type : 1GE Twinax 1m (SFP)
Port 2/2/1: Type : 1GE Twinax 1m (SFP)
Port 2/2/3: Type : 1GE Twinax 1m (SFP)
Port 3/2/1: Type : 1GE Twinax 1m (SFP)
Port 3/2/3: Type : 1GE Twinax 1m (SFP)
Port 4/2/1: Type : 1GE Twinax 1m (SFP)
Port 4/2/3: Type : 1GE Twinax 1m (SFP)
Show stack:
ID Type Role Mac Address Pri State Comment
1 S ICX6450-48 active 609c.9f92.5dc0 128 local Ready
2 S ICX6450-48 member 609c.9f92.6680 0 remote Ready
3 S ICX6450-48 member 609c.9f92.66c0 0 remote Ready, standby if reloaded
4 S ICX6450-48 member 609c.9f92.6200 0 remote Ready
active
+---+ +---+ +---+ +---+
-2/1| 1 |2/3--2/1| 4 |2/3--2/1| 3 |2/3--2/1| 2 |2/3-
| +---+ +---+ +---+ +---+ |
| |
|--------------------------------------------------|
Note: There is no standby. Reason: u2: unknown, u3: unknown, u4: unknown,
Current stack management MAC is 609c.9f92.5dc0
show stat
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.5df1
Port 1/2/1 Counters:
InOctets 669139368 OutOctets 23361464
InPkts 491349 OutPkts 158425
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 491349 OutMulticastPkts 158425
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 224304
InFragments 0
InDiscards 0 OutErrors 0
CRC 224304 Collisions 0
InErrors 30455 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 1587096 OutBitsPerSec 42312
InPktsPerSec 145 OutPktsPerSec 44
InUtilization 0.01% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.5df3
Port 1/2/3 Counters:
InOctets 137094532 OutOctets 12410120
InPkts 253079 OutPkts 39417
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 253079 OutMulticastPkts 39417
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 12203
InFragments 0
InDiscards 0 OutErrors 0
CRC 12203 Collisions 0
InErrors 1426 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 331328 OutBitsPerSec 26888
InPktsPerSec 76 OutPktsPerSec 11
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
2/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.66b1
Port 2/2/1 Counters:
InOctets 62319460 OutOctets 4065972
InPkts 118924 OutPkts 12774
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 118924 OutMulticastPkts 12774
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 2102
InFragments 0
InDiscards 0 OutErrors 0
CRC 2102 Collisions 0
InErrors 255 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 144760 OutBitsPerSec 9856
InPktsPerSec 35 OutPktsPerSec 3
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
2/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.66b3
Port 2/2/3 Counters:
InOctets 8487136 OutOctets 327235660
InPkts 39114 OutPkts 362168
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 39114 OutMulticastPkts 362168
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 6928
InFragments 3
InDiscards 0 OutErrors 0
CRC 6928 Collisions 0
InErrors 1982 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 17024 OutBitsPerSec 912552
InPktsPerSec 12 OutPktsPerSec 125
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
3/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.66f1
Port 3/2/1 Counters:
InOctets 4682052 OutOctets 3859780
InPkts 12553 OutPkts 11721
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 12553 OutMulticastPkts 11721
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 564
InFragments 0
InDiscards 0 OutErrors 0
CRC 564 Collisions 0
InErrors 85 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 8448 OutBitsPerSec 8560
InPktsPerSec 3 OutPktsPerSec 3
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
3/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.66f3
Port 3/2/3 Counters:
InOctets 4225704 OutOctets 66324944
InPkts 12550 OutPkts 127644
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 12550 OutMulticastPkts 127644
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 630
InFragments 0
InDiscards 0 OutErrors 0
CRC 630 Collisions 0
InErrors 84 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 10392 OutBitsPerSec 160280
InPktsPerSec 3 OutPktsPerSec 38
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
4/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.6231
Port 4/2/1 Counters:
InOctets 11070496 OutOctets 120893080
InPkts 34259 OutPkts 234044
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 34259 OutMulticastPkts 234044
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 367
InFragments 0
InDiscards 0 OutErrors 0
CRC 367 Collisions 0
InErrors 42 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 22976 OutBitsPerSec 284928
InPktsPerSec 9 OutPktsPerSec 69
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
4/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.6233
Port 4/2/3 Counters:
InOctets 7730792 OutOctets 8668060
InPkts 23077 OutPkts 26172
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 23077 OutMulticastPkts 26172
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 510
InFragments 0
InDiscards 0 OutErrors 0
CRC 510 Collisions 0
InErrors 60 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 17824 OutBitsPerSec 18800
InPktsPerSec 6 OutPktsPerSec 7
InUtilization 0.00% OutUtilization 0.00%
123-0-0-0(1/2/1)
31-0-0-0(1/2/3)
3-0-0-0(2/2/1)
3-0-0-0(2/2/3)
5-0-0-0(3/2/1)
5-0-0-0(3/2/3)
3-0-0-0(4/2/1)
3-0-0-0(4/2/3)
Show Media:
show media | ex M-C
Port 1/2/1: Type : 1GE Twinax 1m (SFP)
Port 1/2/3: Type : 1GE Twinax 1m (SFP)
Port 2/2/1: Type : 1GE Twinax 1m (SFP)
Port 2/2/3: Type : 1GE Twinax 1m (SFP)
Port 3/2/1: Type : 1GE Twinax 1m (SFP)
Port 3/2/3: Type : 1GE Twinax 1m (SFP)
Port 4/2/1: Type : 1GE Twinax 1m (SFP)
Port 4/2/3: Type : 1GE Twinax 1m (SFP)
Show stack:
ID Type Role Mac Address Pri State Comment
1 S ICX6450-48 active 609c.9f92.5dc0 128 local Ready
2 S ICX6450-48 member 609c.9f92.6680 0 remote Ready
3 S ICX6450-48 member 609c.9f92.66c0 0 remote Ready, standby if reloaded
4 S ICX6450-48 member 609c.9f92.6200 0 remote Ready
active
+---+ +---+ +---+ +---+
-2/1| 1 |2/3--2/1| 4 |2/3--2/1| 3 |2/3--2/1| 2 |2/3-
| +---+ +---+ +---+ +---+ |
| |
|--------------------------------------------------|
Note: There is no standby. Reason: u2: unknown, u3: unknown, u4: unknown,
Current stack management MAC is 609c.9f92.5dc0
show stat
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.5df1
Port 1/2/1 Counters:
InOctets 669139368 OutOctets 23361464
InPkts 491349 OutPkts 158425
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 491349 OutMulticastPkts 158425
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 224304
InFragments 0
InDiscards 0 OutErrors 0
CRC 224304 Collisions 0
InErrors 30455 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 1587096 OutBitsPerSec 42312
InPktsPerSec 145 OutPktsPerSec 44
InUtilization 0.01% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
1/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.5df3
Port 1/2/3 Counters:
InOctets 137094532 OutOctets 12410120
InPkts 253079 OutPkts 39417
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 253079 OutMulticastPkts 39417
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 12203
InFragments 0
InDiscards 0 OutErrors 0
CRC 12203 Collisions 0
InErrors 1426 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 331328 OutBitsPerSec 26888
InPktsPerSec 76 OutPktsPerSec 11
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
2/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.66b1
Port 2/2/1 Counters:
InOctets 62319460 OutOctets 4065972
InPkts 118924 OutPkts 12774
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 118924 OutMulticastPkts 12774
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 2102
InFragments 0
InDiscards 0 OutErrors 0
CRC 2102 Collisions 0
InErrors 255 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 144760 OutBitsPerSec 9856
InPktsPerSec 35 OutPktsPerSec 3
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
2/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.66b3
Port 2/2/3 Counters:
InOctets 8487136 OutOctets 327235660
InPkts 39114 OutPkts 362168
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 39114 OutMulticastPkts 362168
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 6928
InFragments 3
InDiscards 0 OutErrors 0
CRC 6928 Collisions 0
InErrors 1982 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 17024 OutBitsPerSec 912552
InPktsPerSec 12 OutPktsPerSec 125
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
3/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.66f1
Port 3/2/1 Counters:
InOctets 4682052 OutOctets 3859780
InPkts 12553 OutPkts 11721
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 12553 OutMulticastPkts 11721
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 564
InFragments 0
InDiscards 0 OutErrors 0
CRC 564 Collisions 0
InErrors 85 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 8448 OutBitsPerSec 8560
InPktsPerSec 3 OutPktsPerSec 3
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
3/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.66f3
Port 3/2/3 Counters:
InOctets 4225704 OutOctets 66324944
InPkts 12550 OutPkts 127644
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 12550 OutMulticastPkts 127644
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 630
InFragments 0
InDiscards 0 OutErrors 0
CRC 630 Collisions 0
InErrors 84 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 10392 OutBitsPerSec 160280
InPktsPerSec 3 OutPktsPerSec 38
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
4/2/1 Up Forward Full 10G None No N/A 0 609c.9f92.6231
Port 4/2/1 Counters:
InOctets 11070496 OutOctets 120893080
InPkts 34259 OutPkts 234044
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 34259 OutMulticastPkts 234044
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 367
InFragments 0
InDiscards 0 OutErrors 0
CRC 367 Collisions 0
InErrors 42 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 22976 OutBitsPerSec 284928
InPktsPerSec 9 OutPktsPerSec 69
InUtilization 0.00% OutUtilization 0.00%
Port Link State Dupl Speed Trunk Tag Pvid Pri MAC Name
4/2/3 Up Forward Full 10G None No N/A 0 609c.9f92.6233
Port 4/2/3 Counters:
InOctets 7730792 OutOctets 8668060
InPkts 23077 OutPkts 26172
InBroadcastPkts 0 OutBroadcastPkts 0
InMulticastPkts 23077 OutMulticastPkts 26172
InUnicastPkts 0 OutUnicastPkts 0
InBadPkts 510
InFragments 0
InDiscards 0 OutErrors 0
CRC 510 Collisions 0
InErrors 60 LateCollisions 0
InGiantPkts 0
InShortPkts 0
InJabber 0
InFlowCtrlPkts 0 OutFlowCtrlPkts 0
InBitsPerSec 17824 OutBitsPerSec 18800
InPktsPerSec 6 OutPktsPerSec 7
InUtilization 0.00% OutUtilization 0.00%
Ben, Employee
- 56 Posts
- 9 Reply Likes
Looks like layer 1 port flaps to me. Left most column is layer 1 (physical) transitions:
123-0-0-0(1/2/1)
31-0-0-0(1/2/3)
Have you tried other cables?
123-0-0-0(1/2/1)
31-0-0-0(1/2/3)
Have you tried other cables?
(Edited)
- 7 Posts
- 1 Reply Like
I only have the 4. One of my future troubleshooting steps was to try them one at a time. I'll try it sooner rather than later and report the results.
Ben, Employee
- 56 Posts
- 9 Reply Likes
Sounds good. It's interesting that all of the connections show CRC errors. Bad batch of cables maybe?
- 7 Posts
- 1 Reply Like
Well, I tried each of the 4 cables with different switch/port parings, and the CRC errors occur for each test. So it could be bad cables, bad stacking ports, or still possible bad firmware. I'lll have to try some earlier code to see if the CRC errors are still present.
Ben, Employee
- 56 Posts
- 9 Reply Likes
Pretty unlikely the code has anything to do with it. Maybe run a 'clear stat' and 'dset' (clears dm statistic) and monitor those prior commands. Maybe there is just one culprit. That would help narrow it down.
- 7 Posts
- 1 Reply Like
I have been running a clear stat command and monitoring those numbers after the stack renegotiates. On the current units connected, over the past hour+, one interface is showing 7826 CRC /1121 InErrors, and the matching interface is showing 18226 CRC / 2281 InErrors.
I want to rule out firmware prior to opening up a RMA case.
I just remembered that I have a pair of 6430-24s that I can try the cables on as well. If the errors follow, that would at least rule out unit hardware issues.
I want to rule out firmware prior to opening up a RMA case.
I just remembered that I have a pair of 6430-24s that I can try the cables on as well. If the errors follow, that would at least rule out unit hardware issues.
Ben, Employee
- 56 Posts
- 9 Reply Likes
- 7 Posts
- 1 Reply Like
Sorry for the delay. The cause was incorrect cables. I was inadvertently sent 1Gig stacking cables, not the required 10Gig. Surprising that the interfaces came up with the incorrect cables. Lots of errors, but still, I would not have expected the interfaces to still be active.
Related Categories
-
ICX Campus Switches and General Switching Questions
- 152 Conversations
- 109 Followers