FabricPath for CCIE DC

We’re all cut from the same cloth, or in other words, fabric. It only makes sense that we connect with each other in the most immediate way, with all lines of communication open and inviting. In this blog post I’ll be looking at FabricPath, it’s purpose and how it pertains to the CCIE Data Center lab exam. I’ll also run through a configuration, observing behaviors along the way. For those just looking for a sample config, a full config is provided at the bottom of this post.

This post assumes you already have a basic understanding of FabricPath.  For those looking for details on FabricPath, here are some great resources that helped me along the way.  

Nexus 7000 FabricPath
Cisco FabricPath Best Practices

Cisco Live:
BRKDCT-3313 – FabricPath Operation and Troubleshooting (2014)
BRKDCT-2081 – Cisco FabricPath Technology and Design (2014)

INE:
http://www.ine.com/

What is FabricPath and why use it?

FabricPath is Layer 2 routing, also known as MAC-in-MAC routing. This is achieved by running IS-IS protocol in the L2 control plane, where it is responsible for building the topology and Shortest Path Tree (SPT). Routing protocols for the win!

FabricPath was designed to overcome the limitation of Spanning-Tree Protocol (STP). What limitations? Some that come to mind are poor convergence, unnecessary flooding and maintenance of full CAM tables. Oh, and did I mention no ability for equal-cost multipathing (ECMP)?!

With FabricPath, we have the capability to actively forward on all links. If there is a failure on one of the links, traffic will be redistributed across all the others. When traffic comes into the FabricPath domain, a single lookup is performed to identify the switch closest to the destination, providing optimal flows.

Starting Topology

We will be using this topology for this walkthrough.

spanning-tree

The above topology is STP-only.  Before we dive into the configuration, let’s check out the current VLAN database and Spanning-Tree on N7K3. We’ll take a look at this again in a few minutes.

VLANs and Spanning-Tree

N7K3# show vlan

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16
40   VLAN0040                         active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16
50   VLAN0050                         active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16
60   VLAN0060                         active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16
70   VLAN0070                         active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16
80   VLAN0080                         active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16

VLAN Type         Vlan-mode
---- -----        ----------
1    enet         CE     
40   enet         CE     
50   enet         CE     
60   enet         CE     
70   enet         CE     
80   enet         CE     

Remote SPAN VLANs
-------------------------------------------------------------------------------

Primary  Secondary  Type             Ports
-------  ---------  ---------------  -------------------------------------------


N7K3# show spanning-tree vlan 40,50

VLAN0040
  Spanning tree enabled protocol rstp
  Root ID    Priority    24616
             Address     e8ed.f339.4f44
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    24616  (priority 24576 sys-id-ext 40)
             Address     e8ed.f339.4f44
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth4/13          Desg FWD 2         128.525  P2p 
Eth4/14          Desg FWD 2         128.526  P2p 
Eth4/15          Desg FWD 2         128.527  Network P2p 
Eth4/16          Desg FWD 2         128.528  Network P2p 


VLAN0050
  Spanning tree enabled protocol rstp
  Root ID    Priority    24626
             Address     e8ed.f339.4f44
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    24626  (priority 24576 sys-id-ext 50)
             Address     e8ed.f339.4f44
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Eth4/13          Desg FWD 2         128.525  P2p 
Eth4/14          Desg FWD 2         128.526  P2p 
Eth4/15          Desg FWD 2         128.527  Network P2p 
Eth4/16          Desg FWD 2         128.528  Network P2p 

 

 

FabricPath Topology

Our end-goal will look like this, with FabricPath running between the N7Ks, and Classical Ethernet running to the bottom two switches.

spine-leaf

FabricPath Configuration

Install the feature-set in the Admin VDC

ADMIN# conf t
ADMIN(config)# install feature-set fabricpath

Notice this automatically allows the feature-set in the VDC

vdc N7K3 id 4
  limit-resource module-type m2xl f2e 
  allow feature-set fabricpath

Enable the FabricPath feature-set on the switches/VDCs

N7K3# conf t
N7K3(config)# feature-set fabricpath 

N7K4# conf t
N7K4(config)# feature-set fabricpath 

N7K5# conf t
N7K5(config)# feature-set fabricpath 

N7K6# conf t
N7K6(config)# feature-set fabricpath 

Notice the default configuration of FabricPath

N7K3# show run fabricpath 

!Command: show running-config fabricpath
!Time: Sun Aug  3 20:01:52 2014

version 6.2(6)
feature-set fabricpath

fabricpath domain default

All we’ve done so far is enable the FabricPath feature-set. Notice below that we are already assigned a SID (switch-ID). This is a 12-bit address dynamically assigned via DRAP (Dynamic Resource Allocation Protocol), which is used for identifying the switch in the FabricPath domain.  

The system-id is the MAC of the switch or VDC (verify with show vdc internal mac_address_table on the Admin VDC)

N7K3# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 1
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
*   370         e8ed.f339.4f44    Primary     Confirmed No      No      


N7K4(config)# show fabricpath s
static      switch-id   system-id   
N7K4(config)# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 1
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
*   327         e8ed.f339.4e44    Primary     Confirmed No      No    

N7K5(config)# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 1
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
*   104         e8ed.f339.4f45    Primary     Confirmed No      No      


N7K6(config)# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 1
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
*   76          e8ed.f339.4e45    Primary     Confirmed Yes     No    

As you can see, these SIDs are a little all over the place. We can statically configure these so they’re easier to recognize in the FabricPath domain.

Configure Static FabricPath Switch-ID

N7K3(config)# fabricpath switch-id 73
N7K3(config)# show fabricpath switch-id local 
Switch-Id: 73
System-Id: e8ed.f339.4f44 

N7K4(config)# fabricpath switch-id 74
N7K4(config)# show fabricpath switch-id local
Switch-Id: 74
System-Id: e8ed.f339.4e44 

N7K5(config)# fabricpath switch-id 75
N7K5(config)# show fabricpath switch-id local 
Switch-Id: 75
System-Id: e8ed.f339.4f45 

N7K6(config)# fabricpath switch-id 76
N7K6(config)# show fabricpath switch-id local 
Switch-Id: 76
System-Id: e8ed.f339.4e45 

Notice we can already run this command to look at the IS-IS adjacencies, which will be used to build our MAC-in-MAC routing topology and shortest path tree.

N7K6(config)# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface

Nothing yet, so let’s bring up some interfaces and look again. First we’ll configure FabricPath on all the layer-2 interfaces on N7K3 and N7K4.

Configure FabricPath switchports

N7K3(config)# int e4/13-16
N7K3(config-if-range)# switchport mode fabricpath 

N7K4(config)# int e4/13-16
N7K4(config-if-range)# switchport mode fabricpath 

2014 Aug  3 20:09:13 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [30986]  P2P adj L1 e8ed.f339.4e44 over Ethernet4/13 - DOWN (New) on MT-0
2014 Aug  3 20:09:13 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [30986]  P2P adj L1 e8ed.f339.4e44 over Ethernet4/13 - UP on MT-0
2014 Aug  3 20:09:14 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [30986]  P2P adj L1 e8ed.f339.4e44 over Ethernet4/14 - DOWN (New) on MT-0
2014 Aug  3 20:09:14 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [30986]  P2P adj L1 e8ed.f339.4e44 over Ethernet4/14 - INIT on MT-0
2014 Aug  3 20:09:14 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [30986]  P2P adj L1 e8ed.f339.4e44 over Ethernet4/14 - UP on MT-0
2014 Aug  3 20:09:24 N7K3 %ETHPORT-5-IF_UP: Interface Ethernet4/14 is up in mode fabricpath
2014 Aug  3 20:09:24 N7K3 %ETHPORT-5-IF_UP: Interface Ethernet4/13 is up in mode fabricpath
2014 Aug  3 20:09:24 N7K3 %L3VM-5-FP_TPG_INTF_UP: Interface Ethernet4/14 up in fabricpath topology 0
2014 Aug  3 20:09:24 N7K3 %L3VM-5-FP_TPG_INTF_UP: Interface Ethernet4/13 up in fabricpath topology 0

Immediately in the logs we can see adjacencies form and topology build. Let’s look at our ISIS adjacencies again:

N7K3# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K4            N/A             1      UP     00:00:25   Ethernet4/13
N7K4            N/A             1      UP     00:00:29   Ethernet4/14

N7K4# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K3            N/A             1      UP     00:00:26   Ethernet4/13
N7K3            N/A             1      UP     00:00:24   Ethernet4/14

Awesome, we have an adjacency up on both links! Let’s take a look at the switch-id table:

N7K3# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 2
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
*   73          e8ed.f339.4f44    Primary     Confirmed Yes     No      
    74          e8ed.f339.4e44    Primary     Confirmed Yes     No   
	
N7K4# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 2
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
    73          e8ed.f339.4f44    Primary     Confirmed Yes     No      
*   74          e8ed.f339.4e44    Primary     Confirmed Yes     No   

Great, we see eachothers SIDs and System-IDs. Notice the “*” indicates the local switch.

Since all of our layer 2 interfaces are now running FabricPath, is there a need for Spanning-Tree?

fp-ports-1

N7K3# show spanning-tree 
No spanning tree instance exists.

N7K4# show spanning-tree 
No spanning tree instance exists.

The switches sure don’t think so! And they’re right, since all of our L2 ports are in switchport mode fabricpath, we ensured the switch that there will be no active CE (Classical Ethernet) VLANs on this switch. Check out the CAM table and you’ll already see some new fields for FabricPath:

N7K4# sh mac address-table 
Legend: 
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False
   VLAN     MAC Address      Type      age     Secure NTFY Ports/SWID.SSID.LID
---------+-----------------+--------+---------+------+----+------------------
* 40       0000.0c07.ac28    static       -       F    F  73.0.4325
* 50       0000.5e00.0132    static       -       F    F  73.0.4325
G     -    e8ed.f339.4e44    static       -       F    F  0.0.0(R)
G 40       e8ed.f339.4e44    static       -       F    F  sup-eth1(R)
G 50       e8ed.f339.4e44    static       -       F    F  sup-eth1(R)

We now see that traffic to 0000.0c07.ac28 will be FabricPath encapsulated with the frame directed towards Switch-ID 73, sub-Switch-ID 0 (used in vPC), and Local ID 4325 (FabricPath edge port the frame will be forwarded on).  Note: SID and SWID are used interchangeably to represent Switch-ID.

Next let’s bring up the FabricPath interfaces on N7K5 and N7K6 that are facing N7K3 and N7K4

N7K5(config)# int e4/17-18
N7K5(config-if-range)# switchport mode fabricpath 

N7K6(config)# int e4/17-18
N7K6(config-if-range)# switchport mode fabricpath

N7K5# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K3            N/A             1      UP     00:00:28   Ethernet4/17
N7K4            N/A             1      UP     00:00:27   Ethernet4/18

N7K6# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K4            N/A             1      UP     00:00:29   Ethernet4/17
N7K3            N/A             1      UP     00:00:31   Ethernet4/18

N7K3# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K4            N/A             1      UP     00:00:29   Ethernet4/13
N7K4            N/A             1      UP     00:00:30   Ethernet4/14
N7K5            N/A             1      UP     00:00:23   Ethernet4/15
N7K6            N/A             1      UP     00:00:30   Ethernet4/16

N7K4# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K3            N/A             1      UP     00:00:29   Ethernet4/13
N7K3            N/A             1      UP     00:00:26   Ethernet4/14
N7K6            N/A             1      UP     00:00:31   Ethernet4/15
N7K5            N/A             1      UP     00:00:28   Ethernet4/16

We have adjacencies! Next step is to actually configure VLANs to run in FabricPath mode.

Configure FabricPath VLANs on N7K3 and N7K4

N7K3(config)# vlan 40,50
N7K3(config-vlan)# mode fabricpath 

N7K4(config)# vlan 40,50
N7K4(config-vlan)# mode fabricpath 

N7K3# sh vlan

VLAN Name                             Status    Ports
---- -------------------------------- --------- -------------------------------
1    default                          active    
40   VLAN0040                         active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16
50   VLAN0050                         active    Eth4/13, Eth4/14, Eth4/15
                                                Eth4/16
60   VLAN0060                         active    
70   VLAN0070                         active    
80   VLAN0080                         active    

VLAN Type         Vlan-mode
---- -----        ----------
1    enet         CE     
40   enet         FABRICPATH  
50   enet         FABRICPATH  
60   enet         CE     
70   enet         CE     
80   enet         CE     

Notice that VLANs 40 and 50 now show FABRICPATH as the mode. The VLANs will now participate in the FabricPath domain and will run conversational MAC learning. This topic is covered well in the articles posted at the top of this blog. In short, with conversational MAC learning, the switch will only learn a MAC address if it already knows the destination MAC address, and only if it is a unicast packet. This saves on CAM resources and optimizes the control plane.

Configure FabricPath VLANs on N7K5 and N7K6

First, I want to show you what happens when we configure fabricpath VLANs in a vPC environment.

N7K5(config)# vlan 40,50
N7K5(config-vlan)# mode fabricpath 
N7K5(config-vlan)# end

N7K5# 2014 Aug  3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel22 are being suspended. (Reason: Vlan is not allowed on Peer-link)
2014 Aug  3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel21 are being suspended. (Reason: Vlan is not allowed on Peer-link)
2014 Aug  3 20:39:11 N7K5 %VSHD-5-VSHD_SYSLOG_CONFIG_I: Configured from vty by admin on 10.122.0.209@pts/8
2014 Aug  3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel20 are being suspended. (Reason: Vlan mode not allowed on vPC)
2014 Aug  3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel22 are being suspended. (Reason: Vlan is not allowed on Peer-link)
2014 Aug  3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel20 are being suspended. (Reason: Vlan mode not allowed on vPC)
2014 Aug  3 20:39:11 N7K5 %ETHPORT-3-IF_ERROR_VLANS_SUSPENDED: VLANs 40,50 on Interface port-channel21 are being suspended. (Reason: Vlan is not allowed on Peer-link)

Notice the impact when configuring the VLANs without enabling FabricPath on the vPC – the VLANs go suspended. Check vPC and you’ll see that VLANs 40 and 50 are now removed from the Peer-link

N7K5(config-vlan)# sh vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 20  
Peer status                       : peer adjacency formed ok      
vPC keep-alive status             : peer is alive                 
Configuration consistency status  : success 
Per-vlan consistency status       : success                       
Type-2 consistency status         : success 
vPC role                          : primary                       
Number of vPCs configured         : 2   
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled (timeout = 240 seconds)

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans    
--   ----   ------ --------------------------------------------------
1    Po20   up     60,70,80                                                  

vPC status
----------------------------------------------------------------------
id   Port      Status Consistency Reason                  Active vlans
--   ----      ------ ----------- ------                  ------------
21   Po21      up     success     success                    60,70,80        
22   Po22      up     success     success                    60,70,80        

Let’s go ahead and convert these back to CE

N7K5(config-vlan)# no mode fabricpath 

Configure vPC+ FabricPath VLANs on N7K5 and N7K6 vPC+

The first thing we want to do is configure the switch-id which will be used to identify the vPC “virtual switch” domain in the FabricPath domain. If you do not do this, vPC will throw you an error:

2014 Aug  3 20:41:09 N7K5 %VPC-2-VPC_CORE_PORT_FPATH_BUP_FAILED: Failed to bring up vPC+ peer link port port-channel1 in Fabric Path Port Mode - vPC+ Fabric Path switch ID not configured

Configure the vPC FabricPath switch-ID

N7K5(config)# vpc domain 20
N7K5(config-vpc-domain)# fabricpath switch-id 20
Configuring fabricpath switch id will flap vPCs. Continue (yes/no)? [no] yes
Note: 
 --------:: Re-init of peer-link and vPCs started  ::--------
 
N7K6(config)# vpc domain 20
N7K6(config-vpc-domain)# fabricpath switch-id 20


N7K5(config-vpc-domain)# show vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 20  
vPC+ switch id                    : 20
Peer status                       : peer link is down             
                                  (Peer-link is not in fabricpath            
                                  mode for vPC+)                             
vPC keep-alive status             : peer is alive                 
vPC fabricpath status             : peer is reachable through fabricpath
Configuration consistency status  : success 
Per-vlan consistency status       : success                       
Type-2 consistency status         : success 
vPC role                          : primary                       
Number of vPCs configured         : 2   
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled (timeout = 240 seconds)
Fabricpath load balancing         : Disabled
Port Channel Limit                : limit to 244

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans    
--   ----   ------ --------------------------------------------------
1    Po20   down   -                                                         

vPC status
-------------------------------------------------------------------------------
id   Port      Status Consistency Reason           Active vlans  vPC+ Attribute
--   ----      ------ ----------- ------           ------------  --------------
21   Po21      down   success     success          -             DF: No, FP  
                                                                 MAC:        
                                                                 20.1.65535  
22   Po22      down   success     success          -             DF: No, FP  
                                                                 MAC:        
                                                                 20.1.65535  

Our Peer-link is down, this is because we also need to configure the switchport mode on the vpc peer-link port-channel:

N7K5(config-vpc-domain)# int po20
N7K5(config-if)# switchport mode fabricpath 

N7K6(config-vpc-domain)# int po20
N7K6(config-if)# switchport mode fabricpath 

N7K5# show vpc
Legend:
                (*) - local vPC is down, forwarding via vPC peer-link

vPC domain id                     : 20  
vPC+ switch id                    : 20
Peer status                       : peer adjacency formed ok      
vPC keep-alive status             : peer is alive                 
vPC fabricpath status             : peer is reachable through fabricpath
Configuration consistency status  : success 
Per-vlan consistency status       : success                       
Type-2 consistency status         : success 
vPC role                          : primary                       
Number of vPCs configured         : 2   
Peer Gateway                      : Disabled
Dual-active excluded VLANs        : -
Graceful Consistency Check        : Enabled
Auto-recovery status              : Enabled (timeout = 240 seconds)
Fabricpath load balancing         : Disabled
Port Channel Limit                : limit to 244

vPC Peer-link status
---------------------------------------------------------------------
id   Port   Status Active vlans    
--   ----   ------ --------------------------------------------------
1    Po20   up     -                                                         

vPC status
-------------------------------------------------------------------------------
id   Port      Status Consistency Reason           Active vlans  vPC+ Attribute
--   ----      ------ ----------- ------           ------------  --------------
21   Po21      up     success     success          -             DF: No, FP  
                                                                 MAC:        
                                                                 20.11.65535 
22   Po22      up     success     success          -             DF: No, FP  
                                                                 MAC:        
                                                                 20.12.65535 

Great, our peer-link is back up! Notice above these two things:

1. We now have a Port Channel Limit of 244. This new limit is imposed due to the new sub-switch (sSID) ID used when running vPC+. This feid identifies the actual port-channel interfaces associated with a a particular vPC+ switch pair.

2. We now have vPC+ Attributes. 20.11.65535 is the SID.sSID.LID we talked about early. Notice we now have sSIDs.

Let’s take a look at the FabricPath Switch-ID Table.

N7K3# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 6
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
    20          e8ed.f339.4e45    Primary     Confirmed No      Yes     
    20          e8ed.f339.4f45    Primary     Confirmed No      Yes     
*   73          e8ed.f339.4f44    Primary     Confirmed Yes     No      
    74          e8ed.f339.4e44    Primary     Confirmed Yes     No      
    75          e8ed.f339.4f45    Primary     Confirmed Yes     No      
    76          e8ed.f339.4e45    Primary     Confirmed Yes     No   
	
N7K4# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 6
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
    20          e8ed.f339.4e45    Primary     Confirmed No      Yes     
    20          e8ed.f339.4f45    Primary     Confirmed No      Yes     
    73          e8ed.f339.4f44    Primary     Confirmed Yes     No      
*   74          e8ed.f339.4e44    Primary     Confirmed Yes     No      
    75          e8ed.f339.4f45    Primary     Confirmed Yes     No      
    76          e8ed.f339.4e45    Primary     Confirmed Yes     No      
	
N7K5# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 6
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
[E] 20          e8ed.f339.4f45    Primary     Confirmed No      Yes     
    20          e8ed.f339.4e45    Primary     Confirmed No      Yes     
    73          e8ed.f339.4f44    Primary     Confirmed Yes     No      
    74          e8ed.f339.4e44    Primary     Confirmed Yes     No      
*   75          e8ed.f339.4f45    Primary     Confirmed Yes     No      
    76          e8ed.f339.4e45    Primary     Confirmed Yes     No      
	
N7K6# show fabricpath switch-id 
                        FABRICPATH SWITCH-ID TABLE 
Legend: '*' - this system
        '[E]' - local Emulated Switch-id
        '[A]' - local Anycast Switch-id
Total Switch-ids: 6
=============================================================================
    SWITCH-ID      SYSTEM-ID       FLAGS         STATE    STATIC  EMULATED/  
                                                                  ANYCAST    
--------------+----------------+------------+-----------+--------------------
[E] 20          e8ed.f339.4e45    Primary     Confirmed No      Yes     
    20          e8ed.f339.4f45    Primary     Confirmed No      Yes     
    73          e8ed.f339.4f44    Primary     Confirmed Yes     No      
    74          e8ed.f339.4e44    Primary     Confirmed Yes     No      
    75          e8ed.f339.4f45    Primary     Confirmed Yes     No      
*   76          e8ed.f339.4e45    Primary     Confirmed Yes     No

We have Emulated Switch-IDs that identify the vPC+ switches. You’ll see a single emulated switch-id with two system-IDs that match the actual vPC peers.

Since we did not configure all interfaces on N7K5 and N7K6 as mode fabricpath, we must still run spanning-tree for the classical ethernet ports.

fp-ports-2

N7K5# sh spanning-tree vlan 40

VLAN0040
  Spanning tree enabled protocol rstp
  Root ID    Priority    32808
             Address     c84c.75fa.6000
             This bridge is the root
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32808  (priority 32768 sys-id-ext 40)
             Address     c84c.75fa.6000
             Hello Time  2  sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role Sts Cost      Prio.Nbr Type
---------------- ---- --- --------- -------- --------------------------------
Po21             Desg FWD 1         128.4116 (vPC) P2p 
Po22             Desg FWD 1         128.4117 (vPC) P2p 

Routing

Let’s take a look at the routing table:

N7K3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id


FabricPath Unicast Route Table for Topology-Default

0/73/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:45:07, local
1/20/0, number of next-hops: 2
        via Eth4/15, [115/40], 0 day/s 00:06:59, isis_fabricpath-default
        via Eth4/16, [115/40], 0 day/s 00:06:59, isis_fabricpath-default
1/74/0, number of next-hops: 2
        via Eth4/13, [115/40], 0 day/s 00:40:25, isis_fabricpath-default
        via Eth4/14, [115/40], 0 day/s 00:40:25, isis_fabricpath-default
1/75/0, number of next-hops: 1
        via Eth4/15, [115/40], 0 day/s 00:14:25, isis_fabricpath-default
1/76/0, number of next-hops: 1
        via Eth4/16, [115/40], 0 day/s 00:14:12, isis_fabricpath-default
		
N7K4# show fabricpath route
...

0/74/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:44:48, local
1/20/0, number of next-hops: 2
        via Eth4/15, [115/40], 0 day/s 00:06:55, isis_fabricpath-default
        via Eth4/16, [115/40], 0 day/s 00:06:55, isis_fabricpath-default
1/73/0, number of next-hops: 2
        via Eth4/13, [115/40], 0 day/s 00:40:21, isis_fabricpath-default
        via Eth4/14, [115/40], 0 day/s 00:40:21, isis_fabricpath-default
1/75/0, number of next-hops: 1
        via Eth4/16, [115/40], 0 day/s 00:14:21, isis_fabricpath-default
1/76/0, number of next-hops: 1
        via Eth4/15, [115/40], 0 day/s 00:14:08, isis_fabricpath-default
		
N7K5# show fabricpath route
...

0/20/1, number of next-hops: 0
0/20/11, number of next-hops: 1
        via Po21, [80/0], 0 day/s 00:06:50, vpcm
0/20/12, number of next-hops: 1
        via Po22, [80/0], 0 day/s 00:06:50, vpcm
0/75/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:44:30, local
1/20/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:06:50, local
1/73/0, number of next-hops: 1
        via Eth4/17, [115/40], 0 day/s 00:14:16, isis_fabricpath-default
1/74/0, number of next-hops: 1
        via Eth4/18, [115/40], 0 day/s 00:14:16, isis_fabricpath-default
1/76/0, number of next-hops: 1
        via Po20, [115/20], 0 day/s 00:06:50, isis_fabricpath-default
2/20/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:06:50, local
		
N7K6# show fabricpath route
...

0/20/1, number of next-hops: 0
0/20/11, number of next-hops: 1
        via Po21, [80/0], 0 day/s 00:06:45, vpcm
0/20/12, number of next-hops: 1
        via Po22, [80/0], 0 day/s 00:06:45, vpcm
0/76/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:44:10, local
1/20/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:06:45, local
1/73/0, number of next-hops: 1
        via Eth4/18, [115/40], 0 day/s 00:13:58, isis_fabricpath-default
1/74/0, number of next-hops: 1
        via Eth4/17, [115/40], 0 day/s 00:13:58, isis_fabricpath-default
1/75/0, number of next-hops: 1
        via Po20, [115/20], 0 day/s 00:06:45, isis_fabricpath-default
2/20/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:06:45, local

We can quickly see the ftag/switch-id/subswitch-id routes, and the paths based on the admin distance/metric. Notice on N7K3 that we have two equal-cost paths to N7K4 – 1/74/0 (FTAG 1 / SID 74 / sSID 0 because no vPC involved). FTAGs are used for multidestination traffic, which we will get to very soon.

1/74/0, number of next-hops: 2
        via Eth4/13, [115/40], 0 day/s 00:40:25, isis_fabricpath-default
        via Eth4/14, [115/40], 0 day/s 00:40:25, isis_fabricpath-default

Traffic Engineering

Remember, FabricPath is Layer-2 routing, and we can use our routing protocol IS-IS to engineer traffic. Let’s say we wanted N7K3 to prefer the path over Eth4/14 to get to N7K4. We could increase the metric on Eth4/13 to something higher than 40.

N7K3(config)# inte e4/13
N7K3(config-if)# fabricpath isis metric 100

N7K3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id


FabricPath Unicast Route Table for Topology-Default

0/73/0, number of next-hops: 0
        via ---- , [60/0], 0 day/s 00:46:15, local
1/20/0, number of next-hops: 2
        via Eth4/15, [115/40], 0 day/s 00:08:07, isis_fabricpath-default
        via Eth4/16, [115/40], 0 day/s 00:08:07, isis_fabricpath-default
1/74/0, number of next-hops: 1
        via Eth4/14, [115/40], 0 day/s 00:41:33, isis_fabricpath-default
1/75/0, number of next-hops: 1
        via Eth4/15, [115/40], 0 day/s 00:15:33, isis_fabricpath-default
1/76/0, number of next-hops: 1
        via Eth4/16, [115/40], 0 day/s 00:15:20, isis_fabricpath-default

We now only have 1 link listed in the route table to SID 74.

Multidestination Trees

We know that unicast traffic is L2 routed based on the SID and uses the IS-IS SPT to get to the destination SID. But what about multidestination traffic, such as multicast, broadcast and unknown unicasts? Well, that’s handled a little differently.

FabricPath automatically builds two separate logical trees for handling multidestination traffic. The first tree is used to handle broadcast and unknown unicasts, the second tree is used to handle multicast traffic. Each tree is assigned a network-wide identity, known as an FTAG.

Tree 1 = FTAG 1 = Broadcast and unknown unicast
Tree 2 = FTAG 2 = Multicast

Like Spanning-Tree, each tree has a root that is chosen automatically, based on this criteria:

1. Highest root priority – 8-bit value between 0-255 (Default is 64)
2. Highest System-ID – 48-bit VDC MAC address
3. Highest Switch-ID – 12-bit SID

Let’s take a look at the current multidestination topology.

N7K3# show fabricpath isis topology summ
FabricPath IS-IS Topology Summary
Fabricpath IS-IS domain: default
MT-0
  Configured interfaces:  Ethernet4/13  Ethernet4/14  Ethernet4/15  Ethernet4/16
 Max number of trees: 2  Number of trees supported: 2
    Tree id: 1, ftag: 1, root system: e8ed.f339.4f45, 75
    Tree id: 2, ftag: 2, root system: e8ed.f339.4f44, 73
Ftag Proxy Root: e8ed.f339.4f45

N7K4# show fabricpath isis topology summary 
FabricPath IS-IS Topology Summary
Fabricpath IS-IS domain: default
MT-0
  Configured interfaces:  Ethernet4/13  Ethernet4/14  Ethernet4/15  Ethernet4/16
 Max number of trees: 2  Number of trees supported: 2
    Tree id: 1, ftag: 1, root system: e8ed.f339.4f45, 75
    Tree id: 2, ftag: 2, root system: e8ed.f339.4f44, 73
Ftag Proxy Root: e8ed.f339.4f45

N7K5# sh fa i to s
FabricPath IS-IS Topology Summary
Fabricpath IS-IS domain: default
MT-0
  Configured interfaces:  Ethernet4/17  Ethernet4/18  port-channel20
 Max number of trees: 2  Number of trees supported: 2
    Tree id: 1, ftag: 1 [transit-traffic-only], root system: e8ed.f339.4f45, 75
    Tree id: 2, ftag: 2, root system: e8ed.f339.4f44, 73
Ftag Proxy Root: e8ed.f339.4f45

N7K6# show fabricpath isis topology summ
FabricPath IS-IS Topology Summary
Fabricpath IS-IS domain: default
MT-0
  Configured interfaces:  Ethernet4/17  Ethernet4/18  port-channel20
 Max number of trees: 2  Number of trees supported: 2
    Tree id: 1, ftag: 1, root system: e8ed.f339.4f45, 75
    Tree id: 2, ftag: 2 [transit-traffic-only], root system: e8ed.f339.4f44, 73
Ftag Proxy Root: e8ed.f339.4f45

From the output above, we can see that SID 75 has been chosen as the root for Tree 1, and SID 73 has been chosen as the root for Tree 2. Maybe we want to change this, and have N7K3 be the root of the tree for broadcast traffic, and N7K4 be the root for multicast traffic. Our new multidestination topology would look like this:

multidestination

To do so, we can change the root priority.

FabricPath Root Priority

First learning FabricPath, I thought this was an unusual place to configure root-priorty. To easily remember (in case you forget), just do a “show run fabricpath” and you will see the “fabricpath domain default” in the configuration at all times. This is a kindly reminder where you need to configure this parameter.

N7K3(config)# fabricpath domain default
N7K3(config-fabricpath-isis)# root-priority 255

Notice SID 73 is now the root of tree 1:

N7K3(config-fabricpath-isis)# show fabricpath isis top summ
FabricPath IS-IS Topology Summary
Fabricpath IS-IS domain: default
MT-0
  Configured interfaces:  Ethernet4/13  Ethernet4/14  Ethernet4/15  Ethernet4/16
 Max number of trees: 2  Number of trees supported: 2
    Tree id: 1, ftag: 1, root system: e8ed.f339.4f44, 73
    Tree id: 2, ftag: 2, root system: e8ed.f339.4f45, 75
Ftag Proxy Root: e8ed.f339.4f44

Configure N7K4 to be the root for multicast tree 2

N7K4(config)# fabricpath domain default 
N7K4(config-fabricpath-isis)# root-priority 254

N7K4(config-fabricpath-isis)# sh fabricpath isis topology summ
FabricPath IS-IS Topology Summary
Fabricpath IS-IS domain: default
MT-0
  Configured interfaces:  Ethernet4/13  Ethernet4/14  Ethernet4/15  Ethernet4/16
 Max number of trees: 2  Number of trees supported: 2
    Tree id: 1, ftag: 1, root system: e8ed.f339.4f44, 73
    Tree id: 2, ftag: 2, root system: e8ed.f339.4e44, 74
Ftag Proxy Root: e8ed.f339.4f44

Another way to look at trees is with the “show fabricpath isis trees” command. This will actually show you the metrics

N7K3# show fabricpath isis trees
Fabricpath IS-IS domain: default
Note: The metric mentioned for multidestination tree is from the root of that tree to that switch-id
*:directly connected neighbor or link
P:Physical switch-id, E:Emulated, A:Anycast

MT-0
Topology 0, Tree 1, Swid routing table
20, L1
 via Ethernet4/16, metric 40
74, L1
 via Ethernet4/14, metric 40
75, L1
 via Ethernet4/15, metric 40
76, L1
 via Ethernet4/16, metric 40

Topology 0, Tree 2, Swid routing table
20, L1
 via Ethernet4/14, metric 40
74, L1
 via Ethernet4/14, metric 0
75, L1
 via Ethernet4/14, metric 40
76, L1
 via Ethernet4/14, metric 40

ECMP

We can also verify the tree roots by looking at our mroute tables and observing the outgoing interfaces. Notice below that on N7K5 we are using E4/17 to get to Tree 1 (N7K3) and using E4/18 to get to Tree 2 (N7K4)

N7K5# sh fabricpath mroute ftag 1

(ftag/1, vlan/40, *, *), Flood, uptime: 00:44:49, isis 
 Outgoing interface list: (count: 3)
  Interface Ethernet4/17,   Switch-id 73, uptime: 00:55:38, isis 
  Interface Ethernet4/17,   Switch-id 74, uptime: 00:35:53, isis 
  Interface Ethernet4/17,   Switch-id 76, uptime: 00:35:53, isis 

truncated...

N7K5# sh fabricpath mroute ftag 2

(ftag/2, vlan/40, *, *), Flood, uptime: 00:44:51, isis 
 Outgoing interface list: (count: 3)
  Interface Ethernet4/18,   Switch-id 73, uptime: 00:35:36, isis 
  Interface Ethernet4/18,   Switch-id 74, uptime: 00:35:55, isis 
  Interface Ethernet4/18,   Switch-id 76, uptime: 00:35:36, isis 

truncated...

Another fun command is verifying ECMP load-balancing. The default (configurable) load-balancing is shown below. Notice if we change just a single parameter in our flow selector that a different interface is chosen for the ECMP.

N7K3# show fabricpath load-balance 
ECMP load-balancing configuration: 
L3/L4 Preference: Mixed
Hash Control: Symmetric
Rotate amount: 1 bytes
Use VLAN: TRUE


Ftag load-balancing configuration: 
Hash Control: Symmetric
Rotate amount: 1 bytes
Use VLAN: TRUE

N7K3(config)# interface Ethernet4/13
N7K3(config-if)# no fabricpath isis metric 100

N7K3# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 74 flow-type l3 src-ip 1.1.1.1 dst-ip 2.2.2.2 vlan 40 module 4
This flow selects interface Eth4/13

N7K3# show fabricpath load-balance unicast forwarding-path ftag 1 switchid 74 flow-type l3 src-ip 1.1.1.1 dst-ip 2.2.2.3 vlan 40 module 4
This flow selects interface Eth4/14

FabricPath Authentication

What would a routing protocol be without authentication? We have two forms of authentication with FabricPath. First we have interface authentication, which is the actual hello adjaceny authentication

N7K3(config)# key chain FPKEY
N7K3(config-keychain)# key 1
N7K3(config-keychain-key)# key-string FPKEY
N7K3(config-keychain-key)# exit
N7K3(config)# int e4/13-14 
N7K3(config-if-range)# fabricpath isis authentication-type md5
N7K3(config-if-range)# fabricpath isis authentication key-chain FPKEY

2014 Aug  3 23:04:48 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [30986]  P2P adj L1 N7K4 over Ethernet4/14 - DOWN (Hold timer expired) on MT-0
2014 Aug  3 23:05:47 N7K3 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [30986]  P2P adj L1 N7K4 over Ethernet4/13 - DOWN (Hold timer expired) on MT-0

N7K3# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K4            N/A             1      LOST   00:05:35   Ethernet4/13
N7K4            N/A             1      LOST   00:04:35   Ethernet4/14
N7K5            N/A             1      UP     00:00:29   Ethernet4/15
N7K6            N/A             1      UP     00:00:28   Ethernet4/16

Notice our adjacencies are lost. Let’s configure the other side.

N7K4(config)# key chain FPKEY
N7K4(config-keychain)# key 1
N7K4(config-keychain-key)# key-string FPKEY
N7K4(config-keychain-key)# exit
N7K4(config)# int e4/13-14 
N7K4(config-if-range)# fabricpath isis authentication-type md5
N7K4(config-if-range)# fabricpath isis authentication key-chain FPKEY

2014 Aug  3 23:07:20 N7K4 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [8149]  P2P adj L1 N7K3 over Ethernet4/14 - UP on MT-0
2014 Aug  3 23:07:26 N7K4 %ISIS_FABRICPATH-5-ADJCHANGE:  isis_fabricpath-default [8149]  P2P adj L1 N7K3 over Ethernet4/13 - UP on MT-0

And we’re back in business. Notice below that we can see authentication is enabled on the interface

N7K3# show fabricpath isis interf e4/13
Fabricpath IS-IS domain: default 
Interface: Ethernet4/13
  Status: protocol-up/link-up/admin-up
  Index: 0x0003, Local Circuit ID: 0x01, Circuit Type: L1
  Authentication type MD5
  Authentication keychain is FPKEY
  Authentication check specified
  Extended Local Circuit ID: 0x1A18C000, P2P Circuit ID: 0000.0000.0000.00
  Retx interval: 5, Retx throttle interval: 66 ms
  LSP interval: 33 ms, MTU: 1500
  P2P Adjs: 1, AdjsUp: 1, Priority 64
  Hello Interval: 10, Multi: 3, Next IIH: 00:00:02
  Level   Adjs   AdjsUp  Metric   CSNP  Next CSNP  Last LSP ID
  1          1        1      40     60  Inactive   ffff.ffff.ffff.ff-ff
  Topologies enabled:
    Level Topology Metric  MetricConfig Forwarding
    0     0        40      no           UP        
    1     0        40      no           UP     

The next form of authentication we have is FabricPath domain authentication which enforces authentication of the actual IS-IS LSPs. Authentication here will prevent routes from being learned, however, we can still form adjacencies even when the domain authentication is mismatched.

N7K3(config)# fabricpath domain default
N7K3(config-fabricpath-isis)# authentication-type md5 
N7K3(config-fabricpath-isis)# authentication key-chain FPKEY

Notice authentication is enabled

N7K3# show fabricpath isis

Fabricpath IS-IS domain : default
  System ID : e8ed.f339.4f44  IS-Type : L1 Fabric-Control SVI: Unknown
  SAP : 432  Queue Handle : 17
  Maximum LSP MTU: 1492
  Graceful Restart enabled. State: Inactive 
  Last graceful restart status : none
  Graceful Restart holding time:60 
  Metric-style : advertise(wide), accept(wide)
  Start-Mode: Complete [Start-type configuration]
  Area address(es) :
    00
  Process is up and running
  CIB ID: 1
  Interfaces supported by Fabricpath IS-IS :
    Ethernet4/13
    Ethernet4/14
    Ethernet4/15
    Ethernet4/16
  Level 1
  Authentication type: MD5
  Authentication keychain: FPKEY  Authentication check specified
  LSP Lifetime: 1200
  L1 LSP GEN interval- Max:8000 Initial:50      Second:50
  L1 SPF Interval- Max:8000     Initial:50      Second:50
  MT-0 Ref-Bw: 400000
        Max-Path: 16
  Address family Swid unicast :
    Number of interface : 4
    Distance : 115
  L1 Next SPF: Inactive

We have adjacencies, but we can no longer see Switch-IDs

N7K3# show fabricpath isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
e8ed.f339.4e44  N/A             1      UP     00:00:25   Ethernet4/13
e8ed.f339.4e44  N/A             1      UP     00:00:33   Ethernet4/14
e8ed.f339.4f45  N/A             1      UP     00:00:25   Ethernet4/15
e8ed.f339.4e45  N/A             1      UP     00:00:29   Ethernet4/16

We no longer have routes

N7K3# show fabricpath route
FabricPath Unicast Route Table
'a/b/c' denotes ftag/switch-id/subswitch-id
'[x/y]' denotes [admin distance/metric]
ftag 0 is local ftag
subswitch-id 0 is default subswitch-id


FabricPath Unicast Route Table for Topology-Default

0/73/0, number of next-hops: 0
        via ---- , [60/0], 1 day/s 02:15:27, local

Once we configure the other switches with domain authentication, our adjacency tables will populate the SIDs, and our route tables will build.

N7K3# sho fab isis adjacency 
Fabricpath IS-IS domain: default Fabricpath IS-IS adjacency database:
System ID       SNPA            Level  State  Hold Time  Interface
N7K4            N/A             1      UP     00:00:33   Ethernet4/13
N7K4            N/A             1      UP     00:00:24   Ethernet4/14
N7K5		N/A             1      UP     00:00:22   Ethernet4/15
N7K6		N/A             1      UP     00:00:26   Ethernet4/16

 
Full Sample config

As promised, here is a full sample config, N7K3 and N7K4 are configured for authentication.  N7K5 and N7K6 are running vPC+ FabricPath.

N7K3# sh run fabricpath

feature-set fabricpath

vlan 40,50
  mode fabricpath
fabricpath switch-id 73

interface Ethernet4/13
  fabricpath isis authentication-type md5
  fabricpath isis authentication key-chain FPKEY
  switchport mode fabricpath

interface Ethernet4/14
  fabricpath isis authentication-type md5
  fabricpath isis authentication key-chain FPKEY
  switchport mode fabricpath

interface Ethernet4/15
  switchport mode fabricpath

interface Ethernet4/16
  switchport mode fabricpath
fabricpath domain default
  authentication-type md5
  authentication key-chain FPKEY
  root-priority 255
  
N7K4# sh run fabricpath

feature-set fabricpath

vlan 40,50
  mode fabricpath
fabricpath switch-id 74

interface Ethernet4/13
  fabricpath isis authentication-type md5
  fabricpath isis authentication key-chain FPKEY
  switchport mode fabricpath

interface Ethernet4/14
  fabricpath isis authentication-type md5
  fabricpath isis authentication key-chain FPKEY
  switchport mode fabricpath

interface Ethernet4/15
  switchport mode fabricpath

interface Ethernet4/16
  switchport mode fabricpath
fabricpath domain default
  authentication-type md5
  authentication key-chain FPKEY
  root-priority 254
  
N7K5# sh run fabricpath

feature-set fabricpath

vlan 40,50
  mode fabricpath
fabricpath switch-id 75
vpc domain 20
  fabricpath switch-id 20

interface port-channel20
  switchport mode fabricpath

interface Ethernet4/17
  switchport mode fabricpath

interface Ethernet4/18
  switchport mode fabricpath

interface Ethernet4/19
  switchport mode fabricpath

interface Ethernet4/20
  switchport mode fabricpath
fabricpath domain default
  
N7K6# sh run fabricpath

feature-set fabricpath

vlan 40,50
  mode fabricpath
fabricpath switch-id 76
vpc domain 20
  fabricpath switch-id 20

interface port-channel20
  switchport mode fabricpath

interface Ethernet4/17
  switchport mode fabricpath

interface Ethernet4/18
  switchport mode fabricpath

interface Ethernet4/19
  switchport mode fabricpath

interface Ethernet4/20
  switchport mode fabricpath
fabricpath domain default

Helpful show commands

show fabricpath isis adjacency
show fabricpath switch-id
show fabricpath isis database [detail]
show fabricpath route
show fabricpath isis topology summary
show fabricpath isis trees [multidestination 1|2]
show fabricpath mroute
show key-chain
clear fabricpath isis adjacency *

16 comments

    1. You absolutely can, and I have done this during my studies. After doing so, I realized there was no point in it. What you gain in vPC, you lose in FabricPath. At the end, you’ll just have two emulated switches in the FabricPath domain, relying on vPC/LACP for your link hashing rather than FabricPath. Not ideal.

      Thanks!
      David

      1. Thanks. Say my current setup is a dual sided vpc.

        2xN7K
        2xN5K

        on N7Ks where all the default gateway (HSRP) of my downstream devices are all pointing to it.

        If I will only migrate the LEAF 2xN5K to be a vPC+ then upstream N7K is just totally fabricpath, no vpc just pure fb core ports, then those N7K will have a separate control plain and will probably have duplicates? Or I will migrate the N7Ks to vPC+ but the connection from SPINE to LEAF are FB core ports, will that be an ideal migration plan?

  1. Could you please send me the Nexus 7000 FabricPath pdf document you are linking on your site? Cisco has removed it from their site and I cannot find a copy on the internet.
    Thank you in advance!
    Tudor

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s