My favorite command in all of NX-OS – no feature vpc

Kidding! It’s actually not that bad once you’re aware of how it works, what are the caveats, and how to troubleshoot if you run into an issue. There are an incredible amount of excellent vPC guides out there that I highly recommend you read, such as the vPC Best Practices Design Guide on Cisco.com. Below are some of my notes on setting vPC up in preparation for my next lab study – OTV (hint hint). I’m assuming you have already read the configuration guides and have a general understanding of vPC.

The topology for this lab is detailed here (open in another tab for reference):

I’ll be following these steps to configure vPC:
1. Configure VLANs
2. Configure Spanning-tree
3. Enable LACP feature
4. Enable vPC feature
5. Configure vPC Peer-keepalive
6. Configure the Port-channel for vPC Peer-link
7. Configure vPC domain
8. Enable vPC peer-link
9. Configure a vPC member

Let’s begin!

Configure VLANs

In this lab, I’m configuring a couple of VLANs that I plan to ultimately OTV in the future.

vlan 10
 name VLAN10-SERVERS
vlan 11
 name VLAN11-SERVERS

Configure Spanning-tree

# Good to know - this is the default STP configuration:
spanning-tree mode rapid-pvst
spanning-tree pathcost method short
no spanning-tree port type edge default
no spanning-tree port type network default
spanning-tree bridge assurance
no spanning-tree loopguard default
spanning-tree fcoe 

# Recommended best practices is to configure bpdu protection defaults for edge ports. 
# Configure a port type as "edge" and bpdu protections will be enabled!
spanning-tree port type edge bpduguard default
spanning-tree port type edge bpdufilter default

# Spanning-tree priority must match exactly for vPC peer-switch to function, which we'll be enabling later.
spanning-tree vlan 10,11 priority 0

MST can be configured to add flexibility, or, for hybrid environments with STP-attached devices (non-vPC), you can configure spanning-tree pseduo-information for load-balancing. I won’t be doing this in the lab yet, but still good to know (especially for an exam). Here is an example from the vPC Best Practice Guide:

Enable LACP and vPC features (required for vPC)

feature lacp
feature vpc

Configure vPC Peer-keepalive

The vPC peer-keepalive is just that – a keepalive. It’s used to guarantee both peer devices are up and available before forming the vPC domain. It’s also used to detect split-brain scenarios when a peer-link fails. The last thing you want is both peers active with no real-time synchronization between them. When a vPC peer-link fails, the vPC peer-keepalive is leveraged to tell the secondary peer device to shut down it’s vPC member ports.

In a perfect production environment, it’s best to use a Layer-3 port-channel across two line cards. Not everyone has a perfect environment, maybe you don’t have any M1 ports left, maybe you have routing limitations (F1-F2), use your imagination – so I’m going to show a couple of other methods that you can use to get the vPC peer-keepalive functioning.

On Mars, I’m going to use the management interface for the vPC peer-keepalive. Do not directly connect this to the other Nexus 7K if you want Supervisor failover to function without destroying vPC. You can route across your L3 infrastructure as a last resort, but I strongly recommend you always connect this via a separate L2 switch.

MARS-CORE-7K-1:

interface mgmt0
 vrf member management
 ip address 10.20.0.14/24

MARS-CORE-7K-2:

interface mgmt0
 vrf member management
 ip address 10.20.0.15/24

Make sure you can reach the other side:

MARS-CORE-7K-1# ping 10.20.0.15 vrf management 
PING 10.20.0.15 (10.20.0.15): 56 data bytes
64 bytes from 10.20.0.15: icmp_seq=0 ttl=254 time=1.253 ms
64 bytes from 10.20.0.15: icmp_seq=1 ttl=254 time=1.437 ms
64 bytes from 10.20.0.15: icmp_seq=2 ttl=254 time=1.467 ms
64 bytes from 10.20.0.15: icmp_seq=3 ttl=254 time=1.46 ms
64 bytes from 10.20.0.15: icmp_seq=4 ttl=254 time=1.453 ms
--- 10.20.0.15 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 1.253/1.414/1.467 ms

On Earth, we have even more restrictions (wretched Government!).  We only have F2 ports and we’re not able to use the Management interface. Since we cannot assign IP addresses to F2 ports, we’ll be configuring an SVI. Peer-keepalives will be sent across the SVI, so we will need to ensure that this VLAN is pruned from the peer-link.  You will want to disable STP on this link due to ISSU check failures. Recommended to use a L2 port-channel with this method, but it’s not required, and honestly, I don’t have the extra modules, otherwise I would.

EARTH-CORE-7K-1:

# Create a dedicated VRF for the vPC peer-keepalive link (best practice)
vrf context PEER-KEEPALIVE

interface vlan 999
 desc vPC Peer-keepalive
 vrf member PEER-KEEPALIVE
 ip add 10.10.99.1 255.255.255.252 
 no shut

interface e4/48
 switchport mode trunk
 switchport trunk allowed vlan 999
 # Disable STP
 spanning-tree port type edge trunk 
 spanning-tree bpdufilter enable
 no shut

EARTH-CORE-7K-2:

vrf context PEER-KEEPALIVE

interface vlan 999
 desc vPC Peer-keepalive
 vrf member PEER-KEEPALIVE
 ip add 10.10.99.2 255.255.255.252 
 no shut

interface e4/48
 switchport mode trunk
 switchport trunk allowed vlan 999
 # Disable STP
 spanning-tree port type edge trunk 
 spanning-tree bpdufilter enable
 no shut

verify with:

ping 10.10.99.X vrf PEER-KEEPALIVE

Notice both 7K’s are STP root for VLAN 999

EARTH-CORE-7K-1(config-if)# show spann vlan 999
VLAN0999
 Spanning tree enabled protocol rstp
 Root ID Priority 33767
 Address e8ed.f339.4f43
 This bridge is the root
 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 33767 (priority 32768 sys-id-ext 999)
 Address e8ed.f339.4f43
 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth4/48 Desg FWD 4 128.560 Edge P2p
EARTH-CORE-7K-2(config)# show spanning-tree vlan 999
VLAN0999
 Spanning tree enabled protocol rstp
 Root ID Priority 33767
 Address e8ed.f339.4e43
 This bridge is the root
 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 33767 (priority 32768 sys-id-ext 999)
 Address e8ed.f339.4e43
 Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth4/48 Desg FWD 4 128.560 Edge P2p

Configure the vPC peer-link and domain

The vPC peer-link is used to synchronize the state (Using CFSoE) between vPC peer devices, carry control plane protocols and carry vPC VLANs across the L2 trunk.  Even though it carries the VLANs, it is not meant as a transport.  vPC assumes that traffic coming from the peer-link must not have come from a vPC-attached member, otherwise the traffic would have come in on the member link.  With that said – traffic traversing the peer-link, when destined to a member port, will be Stephen Hawking’d, aka black holed, aka, dropped.   This is a loop prevention mechanism, and something to understand when deploying and managing vPC in a production environment.  If vPC were to flood this traffic out of it’s member ports, you could end up with a seriously impairing loop.  Particularly with Layer 3 topologies, this caveat will force you to carefully design with caution.

A few quick reminders on vPC domain components:

Role

• Lower number wins the switch election
• Defines which switch processes BPDUs

System-priority

• Needs to match on each vPC peer.
• By lowering the system priority, you basically guarantee that your peer partner (i.e. access switch or other vPC pair) will not be able to make any decision in regards to aggregation capabilities or advertisements.

When building a vPC domain, there are some general guidelines to follow. Search Google for the vPC Quick Start guide on Cisco.com, and that’s where you’ll find some of these:

1. Must have Peer-keepalive link up first; ensure the peer-link member ports are 10Gig interfaces
2. Use a minimum of two 10Gig ports (M1 up to 8 member ports & F1/F2 up to 16 member ports)
3. Use at least two different line cards to increase high availability
4. Always hard-set the primary role on the Nexus 7K on the left, and secondary role to the Nexus 7K on the right. This is technically just a design etiquette.
5. Make the Nexus 7Ks control LACP establishment for all port-channels; (lowest) vpc domain id + system priority. In other words – don’t use lower vPC domain id’s for your southern environment (e.g. Nexus 5Ks).  This will ensure predictable LACP establishment control on your Core.
6. Enable peer-switch; when activated both vPC peer devices must have the same STP priority set for all vPC VLANs – making them appear as unique STP root bridge, which I mentioned earlier in this article.
7. Enable peer-gateway, auto-recovery, delay restore, and ip arp synchronize (best practice).
8. Always perform VLAN pruning on vPC peer-link with the allowed list of vPC VLANs

EARTH-CORE-7K-1:

# Configure the Port-channel that will be used as the vPC peer-link
interface e3/9-10
 channel-group 1 mode active
interface port-channel1
 desc vPC Peer-link
 switchport
 switchport mode trunk
 switchport trunk allow vlan 10,11
 no shut

# vPC Domains MUST be unique for all vPC pairs configured in an L2 domain
vpc domain 1
 role priority 1
 system-priority 1000
 peer-keepalive destination 10.10.99.2 source 10.10.99.1 vrf PEER-KEEPALIVE

Let’s see what vPC status looks like so far:

EARTH-CORE-7K-2(config-vpc-domain)# show vpc
Legend:
 (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1 
Peer status : peer link not configured 
vPC keep-alive status : peer is alive 
Configuration consistency status : failed 
Configuration inconsistency reason: vPC peer-link does not exist 
Per-vlan consistency status : failed 
Type-2 consistency status : failed 
Type-2 inconsistency reason : vPC peer-link does not exist 
vPC role : none established 
Number of vPCs configured : 0 
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)

As you can see, the keep-alive status is alive, but the peer link is not configured. Let’s configure the peer-link:

NOTE: Bridge Assurance is enabled by default when configuring vPC peer-link (spanning-tree port type network); Do NOT disable it on the vPC peer-link.  More on this later…

EARTH-CORE-7K-1(config)# interface port-channel1
EARTH-CORE-7K-1(config-if)# spanning-tree port type network
EARTH-CORE-7K-1(config-if)# vpc peer-link 
Warning: 
Ensure that VPC peer-link member ports on both peers use identical VDC types (limit-resource module-type VDC config command).
Please note that spanning tree port type is changed to "network" port type on vPC peer-link.
This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance
(which is enabled by default) is not disabled.
EARTH-CORE-7K-1(config-if)# 2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETHPORT-5-IF_DOWN_NONE: Interface port-channel1 is down (None)
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet3/9 to Ethernet3/10
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet3/10 to Ethernet3/9
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet3/10 is down
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet3/9 is down
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet3/9 to none
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETHPORT-5-IF_DOWN_NONE: Interface port-channel1 is down (None)
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 10000000 Kbit
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETHPORT-5-IF_DOWN_CFG_CHANGE: Interface Ethernet3/9 is down(Config change)
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 100000 Kbit
2014 Mar 1 17:36:40 EARTH-CORE-7K-1 %ETHPORT-5-IF_DOWN_CFG_CHANGE: Interface Ethernet3/10 is down(Config change)
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet3/9 is down (None)
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet3/10 is down (None)
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel1 is down (No operational members)
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-SPEED: Interface Ethernet3/9, operational speed changed to 10 Gbps
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet3/9, operational duplex mode changed to Full
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet3/9, operational Receive Flow Control state changed to off
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet3/9, operational Transmit Flow Control state changed to off
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-SPEED: Interface port-channel1, operational speed changed to 10 Gbps
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_DUPLEX: Interface port-channel1, operational duplex mode changed to Full
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel1, operational Receive Flow Control state changed to off
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel1, operational Transmit Flow Control state changed to off
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-SPEED: Interface Ethernet3/10, operational speed changed to 10 Gbps
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_DUPLEX: Interface Ethernet3/10, operational duplex mode changed to Full
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet3/10, operational Receive Flow Control state changed to off
2014 Mar 1 17:36:41 EARTH-CORE-7K-1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet3/10, operational Transmit Flow Control state changed to off
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel1: Ethernet3/9 is up
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from none to Ethernet3/9
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel1: Ethernet3/10 is up
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 10000000 Kbit
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETHPORT-5-IF_UP: Interface Ethernet3/9 is up in mode trunk
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 10-11 on Interface port-channel1 are being suspended. (Reason: vPC peer is not reachable over cfs)
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETHPORT-5-IF_UP: Interface port-channel1 is up in mode trunk
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 20000000 Kbit
2014 Mar 1 17:36:44 EARTH-CORE-7K-1 %ETHPORT-5-IF_UP: Interface Ethernet3/10 is up in mode trunk

Notice VLANs 10-11 are being suspended since the vPC peer is not reachable over Cisco Fabric Services. Let’s take a look at the vPC status:

EARTH-CORE-7K-1(config-if)# show vpc
Legend:
 (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1 
Peer status : peer link is down 
 (vPC peer is not reachable over 
 cfs) 
vPC keep-alive status : peer is alive 
Configuration consistency status : failed 
Configuration inconsistency reason: Consistency Check Not Performed
Per-vlan consistency status : success 
Type-2 inconsistency reason : Consistency Check Not Performed
vPC role : none established 
Number of vPCs configured : 0 
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans 
-- ---- ------ --------------------------------------------------
1 Po1 up -

We’re seeing Type-2 inconsistencies now, which means there is a configuration mismatch.  Let’s  move onto EARTH-CORE-7K-2 and complete the configuration:

interface e3/9-10
 channel-group 1 mode active
interface port-channel1
 desc vPC Peer-link
 switchport
 switchport mode trunk
 switchport trunk allow vlan 10,11
 no shut

vpc domain 1
 role priority 2
 system-priority 1000
 peer-keepalive destination 10.10.99.1 source 10.10.99.2 vrf PEER-KEEPALIVE

EARTH-CORE-7K-2(config)# int po1
EARTH-CORE-7K-2(config-if)# spanning-tree port type network 
EARTH-CORE-7K-2(config-if)# vpc peer-link 
Warning: 
Ensure that VPC peer-link member ports on both peers use identical VDC types (limit-resource module-type VDC config command).
Please note that spanning tree port type is changed to "network" port type on vPC peer-link.
This will enable spanning tree Bridge Assurance on vPC peer-link provided the STP Bridge Assurance
(which is enabled by default) is not disabled.
2014 Mar 1 17:37:38 EARTH-CORE-7K-2 %ETHPORT-5-IF_DOWN_NONE: Interface port-channel1 is down (None)
2014 Mar 1 17:37:38 EARTH-CORE-7K-2 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet3/9 to Ethernet3/10
2014 Mar 1 17:37:38 EARTH-CORE-7K-2 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet3/10 is down
2014 Mar 1 17:37:38 EARTH-CORE-7K-2 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel1: Ethernet3/9 is down
2014 Mar 1 17:37:38 EARTH-CORE-7K-2 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from Ethernet3/10 to none
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DOWN_NONE: Interface port-channel1 is down (None)
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 10000000 Kbit
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DOWN_CFG_CHANGE: Interface Ethernet3/9 is down(Config change)
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 100000 Kbit
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DOWN_CFG_CHANGE: Interface Ethernet3/10 is down(Config change)
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet3/9 is down (None)
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DOWN_NONE: Interface Ethernet3/10 is down (None)
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface port-channel1 is down (No operational members)
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-SPEED: Interface Ethernet3/9, operational speed changed to 10 Gbps
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DUPLEX: Interface Ethernet3/9, operational duplex mode changed to Full
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet3/9, operational Receive Flow Control state changed to off
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet3/9, operational Transmit Flow Control state changed to off
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-SPEED: Interface port-channel1, operational speed changed to 10 Gbps
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DUPLEX: Interface port-channel1, operational duplex mode changed to Full
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface port-channel1, operational Receive Flow Control state changed to off
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface port-channel1, operational Transmit Flow Control state changed to off
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-SPEED: Interface Ethernet3/10, operational speed changed to 10 Gbps
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_DUPLEX: Interface Ethernet3/10, operational duplex mode changed to Full
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet3/10, operational Receive Flow Control state changed to off
2014 Mar 1 17:37:39 EARTH-CORE-7K-2 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet3/10, operational Transmit Flow Control state changed to off
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel1: Ethernet3/9 is up
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel1: Ethernet3/10 is up
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETH_PORT_CHANNEL-5-FOP_CHANGED: port-channel1: first operational port changed from none to Ethernet3/9
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 10000000 Kbit
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETHPORT-5-IF_UP: Interface Ethernet3/9 is up in mode trunk
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 10-11 on Interface port-channel1 are being suspended. (Reason: vPC peer is not reachable over cfs)
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETHPORT-5-IF_UP: Interface port-channel1 is up in mode trunk
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface port-channel1,bandwidth changed to 20000000 Kbit
2014 Mar 1 17:37:42 EARTH-CORE-7K-2 %ETHPORT-5-IF_UP: Interface Ethernet3/10 is up in mode trunk
2014 Mar 1 17:37:45 EARTH-CORE-7K-2 %ETHPORT-3-IF_ERROR_VLANS_REMOVED: VLANs 10-11 on Interface port-channel1 are removed from suspended state.

Looks like the VLANs have been removed from the suspended state, sounds promising! Let’s take a look at vPC status:

EARTH-CORE-7K-2(config-if)# show vpc
Legend:
 (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1 
Peer status : peer adjacency formed ok 
vPC keep-alive status : peer is alive 
Configuration consistency status : success 
Per-vlan consistency status : success 
Type-2 consistency status : success 
vPC role : secondary 
Number of vPCs configured : 0 
Peer Gateway : Disabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans 
-- ---- ------ --------------------------------------------------
1 Po1 up 10-11

Great! vPC is up and operational! Let’s take a look at some helpful show commands:

EARTH-CORE-7K-1# show vpc role
vPC Role status
----------------------------------------------------
vPC role : primary <- 7K-1 is the Primary
vPC system-mac : 00:23:04:ee:be:01 <- system-mac auto-created based on domain id
vPC system-priority : 1000
vPC local system-mac : e8:ed:f3:39:4f:43 <- Local to this 7K
vPC local role-priority : 1 

EARTH-CORE-7K-1# sho vpc consistency-parameters global
Legend:
 Type 1 : vPC will be suspended in case of mismatch <- IMPORTANT 
Name Type Local Value Peer Value 
------------- ---- ---------------------- -----------------------
STP Mode 1 Rapid-PVST Rapid-PVST 
STP Disabled 1 None None 
STP MST Region Name 1 "" "" 
STP MST Region Revision 1 0 0 
STP MST Region Instance to 1 
 VLAN Mapping 
STP Loopguard 1 Disabled Disabled 
STP Bridge Assurance 1 Enabled Enabled 
STP Port Type, Edge 1 Normal, Enabled, Normal, Enabled, 
BPDUFilter, Edge BPDUGuard Enabled Enabled 
STP MST Simulate PVST 1 Enabled Enabled 
Interface-vlan admin up 2 
Interface-vlan routing 2 
capability 
Allowed VLANs - 10-11 10-11 
Local error VLANs - - -

EARTH-CORE-7K-1# show vpc peer-keepalive
vPC keep-alive status : peer is alive 
--Peer is alive for : (94105) seconds, (350) msec
--Send status : Success 
--Last send at : 2014.03.02 19:35:05 501 ms
--Sent on interface : Vlan999
--Receive status : Success
--Last receive at : 2014.03.02 19:35:05 501 ms
--Received on interface : Vlan999
--Last update from peer : (0) seconds, (844) msec
vPC Keep-alive parameters
--Destination : 10.10.99.2
--Keepalive interval : 1000 msec
--Keepalive timeout : 5 seconds
--Keepalive hold timeout : 3 seconds
--Keepalive vrf : PEER-KEEPALIVE
--Keepalive udp port : 3200
--Keepalive tos : 192

I’m going to configure one single-sided vPC member, just to show how it functions:
EARTH-CORE-7K-1:

EARTH-CORE-7K-1(config)# int e4/45
EARTH-CORE-7K-1(config-if)# channel-group 8 mode active
EARTH-CORE-7K-1(config-if)# int po8
EARTH-CORE-7K-1(config-if)# switchport 
EARTH-CORE-7K-1(config-if)# switchport mode trunk
EARTH-CORE-7K-1(config-if)# switchport trunk all vlan 10,11
EARTH-CORE-7K-1(config-if)# spanning-tree port type normal
EARTH-CORE-7K-1(config-if)# vpc 8
EARTH-CORE-7K-1(config-if)# no shut
EARTH-CORE-7K-1(config-if)# int e4/45
EARTH-CORE-7K-1(config-if)# no shut

EARTH-CORE-7K-2:

EARTH-CORE-7K-2(config)# int e4/45
EARTH-CORE-7K-2(config-if)# channel-group 8 mode active
EARTH-CORE-7K-2(config-if)# int po8
EARTH-CORE-7K-2(config-if)# switchport 
EARTH-CORE-7K-2(config-if)# switchport mode trunk
EARTH-CORE-7K-2(config-if)# switchport trunk all vlan 10,11
EARTH-CORE-7K-2(config-if)# spanning-tree port type normal
EARTH-CORE-7K-2(config-if)# vpc 8
EARTH-CORE-7K-2(config-if)# no shut
EARTH-CORE-7K-2(config-if)# int e4/45
EARTH-CORE-7K-2(config-if)# no shut

EARTH-OTV-7k-1

EARTH-OTV-7k-1(config)# int e4/37-38
EARTH-OTV-7k-1(config-if-range)# channel-group 8 mode active
EARTH-OTV-7k-1(config-if-range)# int po8
EARTH-OTV-7k-1(config-if)# switchport 
EARTH-OTV-7k-1(config-if)# switchport mode trunk
EARTH-OTV-7k-1(config-if)# switchport trunk all vlan 10,11
EARTH-OTV-7k-1(config)# int po8
EARTH-OTV-7k-1(config-if)# no shut
EARTH-OTV-7k-1(config-if)# int e4/37-38
EARTH-OTV-7k-1(config-if-range)# no shut

Verify on EARTH-CORE-7K-2:

EARTH-CORE-7K-2# sh vpc
Legend:
 (*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1 
Peer status : peer adjacency formed ok 
vPC keep-alive status : peer is alive 
Configuration consistency status : success 
Per-vlan consistency status : success 
Type-2 consistency status : success 
vPC role : secondary 
Number of vPCs configured : 1 
Peer Gateway : Enabled
Peer gateway excluded VLANs : -
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Enabled (timeout = 240 seconds)
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans 
-- ---- ------ --------------------------------------------------
1 Po1 up 10-11
vPC status
----------------------------------------------------------------------
id Port Status Consistency Reason Active vlans
-- ---- ------ ----------- ------ ------------
8 Po8 up success success 10-11

In the next blog, I’ll be configuring the strongly recommended, best practice vPC peer-link parameters, do not skip it!