If you are running a Virtual Chassis on EX, QFX or MX platform and end up in a situation where one of the vc-port show up on one unit, and down on the other, this lecture is for you!
Let’s do a quick recap on the behavior :
1. Interface set as vc-port (SFP+ or QSFP+) interface link is up on one member, down on the other)
2. VC-Port (one side or both) may or may not show CRC errors
3. The vc-port status show that vcp interface is down
4. Log is reporting the following errors :
fpc1 PFE: ifl-family(2) platform attach failed for ifd vcp-255/1/2 [Unknown error: -4]
fpc1 PFE: ifl-family(2) platform change failed for ifd vcp-255/1/2 [no such file or directory]
fpc1 IFRT: ‘IFL family add’ (opcode 35) failed
fpc1 ifl 64; pfe family attach error (2)
fpc1 [EX-BCM PIC] ex_bcm_linkscan_handler: Link 56 UP
Example of the vc-port status :
> show virtual-chassis vc-port fpc0: -------------------------------------------------------------------------- Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface PIC / Port 1/2 Configured -1 Down 40000 1/3 Configured -1 Up 40000 1 vcp-255/1/3 1/1 Configured Absent 1/0 Configured Absent fpc1: -------------------------------------------------------------------------- Interface Type Trunk Status Speed Neighbor or ID (mbps) ID Interface PIC / Port 1/2 Configured -1 Up 40000 1/3 Configured -1 Up 40000 0 vcp-255/1/3 1/1 Configured Absent 1/0 Configured Absent
Additionally, issuing “show virtual-chassis vc-port” command show “Up” status for a brief moment but then show as “Down”.
The following troubleshooting attempts give no positive results :
- Remove the interfaces from the vc-port (convert back the port to standard Ethernet/uplink interfaces)
- Add back the removed interfaces as vc-port (convert Ethernet/uplink to vc-port)
- Remove the SFP+/QSFP+ optics and re-insert them
- Reboot both involved members of the VC
The steps above were attempted in a lab for testing purposes, it was unlikely to give positive results since I was suspecting optics from the start.
Because of the link state behavior, it is more than likely to be faulty SFP+/QSFP+ modules. Try first replacing the side where the link show as down. However, that does not mean that the side where it show down is at fault, it may be just the opposite! (in my case, the one with the link up status side was the faulty one).
After replacing the optic, that vc-port link should be back up again.
NOTE : If this occurred on a live chassis deployed a while ago, you may have previously observed early sign of failure, as port flapping and an increased system load.