ICX6610 10G Port Flapping Problems

  • 1
  • Question
  • Updated 2 years ago
Hi,
We are using two stacked ICX6610 switches and have several servers and storage connected to it. We have observed problems on our 10G ports - on server site (vmware esx hosts) we see constant link down/up events, and on storage ports we have CRC errors and port flapping condition.
No errors are found on 1Gb ports, no flapping, no CRC, no link state changes.
Sometimes we even get to APD event on vmware (we use 10G for NFS connection hosts<->storage).
FastIron version used is 08.0.10g.
Brief description (excluding 1G network connections):
3 ESX hosts named host1, host2, host3 are connected to 10G ports.
host1 - ports 1/3/3 and 2/3/3
host2 -  ports 1/3/4 and 2/3/4
host3 - ports 1/3/5 and 2/3/5
No teaming on hosts side - only virtual port based load balancing on virtual switches.
Backup server is connected to ports 1/3/6 and 2/3/6. Here we use NIC teaming at server side (static link aggregateion) and switch side - LAG.
Storage is connected to ports 1/3/1 & 2/3/1 and uses LAG.
From one stack member there is only one active uplink to rest of network - port 1/1/24.

I have copy-pasted our switch config below including port descriptions. Does anybody see some error in configuration?

ICX6610-S-1>show configuration
!
Startup-config data location is flash memory
!
Startup configuration:
!
ver 08.0.10gT7f1
!
stack unit 1
  module 1 icx6610-24-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 1/2/1 to 1/2/2
  stack-trunk 1/2/6 to 1/2/7
  stack-port 1/2/1 1/2/6
stack unit 2
  module 1 icx6610-24-port-management-module
  module 2 icx6610-qsfp-10-port-160g-module
  module 3 icx6610-8-port-10g-dual-mode-module
  stack-trunk 2/2/1 to 2/2/2
  stack-trunk 2/2/6 to 2/2/7
  stack-port 2/2/1 2/2/6
stack enable
stack mac 609c.9f47.ee38
!
!
!
lag "storage2_10g_ports" static id 6
 ports ethernet 1/3/2 ethernet 2/3/2
 primary-port 1/3/2
 deploy
!
lag "storage1_10g_ports" static id 2
 ports ethernet 1/3/1 ethernet 2/3/1
 primary-port 1/3/1
 deploy
!
lag "BackupServer_10g_ports" static id 1
 ports ethernet 1/3/6 ethernet 2/3/6
 primary-port 1/3/6
 deploy
!
!
vlan 1 name DEFAULT-VLAN by port
!
vlan 6 by port
 tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
!
vlan 7 by port
 tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
 untagged ethe 1/1/19 to 1/1/20 ethe 2/1/19 to 2/1/20
!
vlan 110 name ManagementNetwork by port
 tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
 untagged ethe 1/1/1 to 1/1/3 ethe 1/1/10 to 1/1/12 ethe 2/1/1 to 2/1/3 ethe 2/1/10
!
vlan 111 name ServersNetwork by port
 tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
 untagged ethe 1/1/13 ethe 2/1/11 to 2/1/13
!
vlan 112 name StorageNetwork by port
 tagged ethe 1/3/1 to 1/3/5 ethe 2/3/1 to 2/3/5
 untagged ethe 1/3/6 ethe 2/3/6
!
vlan 113 name vMotionNetwork by port
 tagged ethe 1/3/3 to 1/3/5 ethe 2/3/3 to 2/3/5
!
vlan 155 name Servers2network by port
 tagged ethe 1/1/4 to 1/1/9 ethe 1/1/24 ethe 2/1/4 to 2/1/9
!
!
!
!
aaa authentication web-server default local
aaa authentication enable default local
aaa authentication login default local
jumbo
enable super-user-password .....
enable aaa console
hostname ICX6610-S-1
ip address xxx.xxx.xxx.xxx 255.255.255.0
no ip dhcp-client enable
ip default-gateway xxx.xxx.xxx.xxx
!
logging host xxx.xxx.xxx.x
logging persistence
no telnet server
telnet server enable vlan 111
username admin password .....
cdp run
!
!
clock summer-time
clock timezone gmt GMT+01
!
!
ntp
 server x.x.x.x minpoll 5 burst
!
!
web-management https
hitless-failover enable
interface ethernet 1/1/4
 port-name host01 slot2 port0 vmnic6
!
interface ethernet 1/1/5
 port-name host01 slot3 port0 vmnic8
!
interface ethernet 1/1/6
 port-name host02 slot2 port0 vmnic6
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/7
 port-name host02 slot3 port0 vmnic8
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/8
 port-name host03 slot2 port0 vmnic6
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/9
 port-name host03 slot3 port0 vmnic8
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/11
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/1/22
 port-name "VLAN 110 uplink to cisco port X - disconnected cable"
!
interface ethernet 1/1/23
 port-name "VLAN 111 uplink to cisco port X - disconnected cable"
!
interface ethernet 1/1/24
 port-name "Uplink port to central Cisco"
!
interface ethernet 1/3/1
 speed-duplex 10G-full
!
interface ethernet 1/3/2
 speed-duplex 10G-full
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/3
 port-name saphost01 slot0 port0 vmnic2
 speed-duplex 10G-full
!
interface ethernet 1/3/4
 port-name host02 slot0 port0 vmnic2
 speed-duplex 10G-full
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/5
 port-name host03 slot0 port0 vmnic2
 speed-duplex 10G-full
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/6
 speed-duplex 10G-full
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 1/3/7
 speed-duplex 10G-full
!
interface ethernet 1/3/8
 speed-duplex 10G-full
!
interface ethernet 2/1/4
 port-name host01 slot2 port1 vmnic7
!
interface ethernet 2/1/5
 port-name host01 slot3 port1 vmnic9
!
interface ethernet 2/1/6
 port-name host02 slot2 port 1 vmnic7
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/1/7
 port-name host02 slot3 port1 vmnic9
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/1/8
 port-name host03 slot2 port1 vmnic7
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/1/9
 port-name host03 slot3 port1 vmnic9
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/3/3
 port-name host01 slot1 port1 vmnic5
 speed-duplex 10G-full
!
interface ethernet 2/3/4
 port-name host02 slot1 port1 vmnic5
 speed-duplex 10G-full
 spanning-tree 802-1w admin-edge-port
!
interface ethernet 2/3/5
 port-name host03 slot1 port1 vmnic5
 speed-duplex 10G-full
 spanning-tree 802-1w admin-edge-port
!
!
!
!
lldp run
!
!
!
!
end

ICX6610-S-1>
Photo of AbeX

AbeX

  • 2 Posts
  • 0 Reply Likes

Posted 2 years ago

  • 1
Photo of Ben

Ben, Employee

  • 98 Posts
  • 33 Reply Likes
What is the media used? Is it showing up as a "state up/down" event in the logs? That would be a physical issue. I would move to 8030 latest since 8010 is a dead code train. Code may have nothing to do with this, however.