Brkaci 2008
Brkaci 2008
Brkaci 2008
#CiscoLive
Why do we Build Networks?
Doable..
but kind of
a science
project!
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 3
Agenda
• Introduction
• Management and Policy Plane
• A day in the life of a packet
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 4
Introduction
Phase 2 (LA)
Limited Availability
General information
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 6
Good News
SD-Access and ACI Fabric Similarities
SD-Access Fabric ACI Fabric
• Underlay • Underlay
• Overlay • Overlay
• SGT • EPG
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 7
Phase 2 (LA)
SDA-ACI Phase 2 Architecture
MGMT. Kafka enables a scalable and
Kafka Bus
Controller APIC
1 PLANE open Pub/Sub-based messaging
Peering bus for controllers’ federation
App Groups
POLICY Enables identity federation and
User Groups
APIC
Users
LISP Underlay Network COOP Data plane based learning of IP
DATA
1
VXLAN SGT (16 bits) VXLAN SGT (16 bits) iVXLAN EPG (16 bits)
4 PLANE
to group bindings for scale &
Header VNID (24 bits) Header VNID (24 bits) Header VNID (24 bits)
simplicity
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 8
Phase 2 (LA)
Software and Hardware Supported Versions
SDA-ACI Components Version
DNA-C 2.1.2.4
MDC 1.0.0.188
MDM 1.0.0.188
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 9
Phase 2 (LA)
Scalability Values
* To achieve the maximum scale for learned endpoints, you must configure the switch's Forwarding Scale Profile
policy, choosing the High IPv4 EP profile that is supported only on FX, FXP and GX ACI leaf models
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 10
Management
and Policy Plane
Phase 2 Management Domain - Kafka
• The phase 2 management plane
uses a Kafka messaging bus Kafka Messaging Bus (VN, SGT/EPG Group, Contract*, …) Exchange
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 12
MDC and MDM Apps
• MDM
• Responsible for domain registration and management
• Manages Kafka topics
• Certificate Management
• MDM app must be installed for domain registration and management. No configuration is
done from the MDM app
• MDC
• APIC configuration, config-sync/recovery
• Manages all multi-domain workflows
• Class-id management
• All multi-domain configuration and management is done from the MDC app
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 13
Kafka Connections
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 14
A day in the life
of a packet
Our setup for today
DNA-Center ISE
C
BGP/EVPN
OSPF OSPF
1/14
Doctor client PC VM
192.168.1.3/32 10.42.42.101
Consumer Provider
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 16
From SDA à
ACI
Doctor logs into PC C
SD-Access
B
Fabric Site B
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 18
Doctor logs into PC C
edge-2#
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 19
Doctor logs into PC C
SD-Access
B
Fabric Site B
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 20
Doctor logs into PC C
SD-Access
B
Fabric Site B
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 21
Doctor logs into PC
edge-2#show authentication sessions interface GigabitEthernet 1/0/14 details
Interface: GigabitEthernet1/0/14
IIF-ID: 0x1297C0CE C
MAC Address: a036.9f8f.7a72
IPv6 Address: Unknown
IPv4 Address: 192.168.1.3
User-Name: derek SD-Access
Device-type: Microsoft-Workstation B
Fabric Site B
Device-name: MSFT 5.0
Status: Authorized
Domain: DATA
Oper host mode: multi-auth
Oper control dir: both
Session timeout: N/A
Acct update timeout: 172800s (local), Remaining: 172770s
Common Session ID: 450210AC0000005BF320DD33
Acct Session ID: 0x00000007
Handle: 0x30000051
Current Policy: PMAP_DefaultWiredDot1xClosedAuth_1X_MAB
Local Policies:
Server Policies:
SGT Value: 16
edge-2#
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 22
Doctor logs into PC C
SD-Access
B
edge-2#show cts role-based sgt-map vrf Internal03 all Fabric Site B
%IPv6 protocol is not enabled in VRF Internal03
Active IPv4-SGT Bindings Information
edge-2#
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 23
Doctor logs into PC
C
edge-2#show cts environment-data 12-00:Development_Servers
CTS Environment Data 13-00:Test_Servers
==================== 14-00:PCI_Servers
Current state = COMPLETE 15-00:BYOD SD-Access
B
Last status = Successful 16-00:Doctors Fabric Site B
Local Device SGT: 17-00:Nurses
SGT tag = 2-00:TrustSec_Devices 255-00:Quarantined_Systems
Server List Info: 10001-
Installed list: CTSServerList1-0001, 1 server(s): 00:Ecommerce_Web13ACISDA56ad54a9EPG
*Server: 172.16.201.217, port 1812, A-ID Environment Data Lifetime = 86400 secs
FD8E99B10C8188CA4F373AAB06C76091 Last update time = 15:05:08 UTC Mon Mar 1 2021
Status = ALIVE Env-data expires in 0:01:34:05 (dd:hr:mm:sec)
auto-test = TRUE, keywrap-enable = FALSE, idle- Env-data refreshes in 0:01:34:05 (dd:hr:mm:sec)
time = 60 mins, deadtime = 20 secs Cache data applied = NONE
Security Group Name Table: State Machine is running
0-00:Unknown
2-00:TrustSec_Devices
3-00:Network_Services
4-00:Employees
5-00:Contractors
6-00:Guests
7-00:Production_Users
8-00:Developers
9-00:Auditors
10-00:Point_of_Sale_Systems
11-00:Production_Servers
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 24
Doctor logs into PC
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 25
Doctor sends traffic – SDA side C
SD-Access
B
Fabric Site B
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 26
Doctor sends traffic – SDA side C
SD-Access
B
Fabric Site B
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 27
Doctor sends traffic – SDA side C
SD-Access
B
Fabric Site B
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 28
Doctor sends traffic – SDA side C
edge-2#
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 29
Doctor sends traffic – SDA side
C
edge-2#show ip route vrf Internal03
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 30
Doctor sends traffic – SDA side - LISP
C
edge-2#show run int vlan 1021
Building configuration...
SD-Access
Current configuration : 322 bytes B
Fabric Site B
!
interface Vlan1021
description Configured from Cisco DNA-Center
mac-address 0000.0c9f.fb98
vrf forwarding Internal03
ip address 192.168.1.1 255.255.255.0
ip helper-address 172.16.201.201
no ip redirects
ip route-cache same-interface
no lisp mobility liveness test
Doctor client PC
lisp mobility 192_168_1_0-Internal03-IPV4
192.168.1.3/32
end
edge-2#show ip vrf
Name Default RD Interfaces
Internal03 <not set> Vl1021
LI0.4099
Mgmt-vrf <not set> Gi0/0
edge-2#
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 31
Doctor sends traffic – SDA side - LISP
C
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 32
Doctor sends traffic – SDA side - LISP
edge-2#show lisp instance-id 4099 ipv4 database 192.168.1.3/32
LISP ETR IPv4 Mapping Database for EID-table vrf Internal03 (IID 4099), LSBs: 0x1 C
Entries total 2, no-route 0, inactive 0
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 33
Doctor sends traffic – SDA side - LISP
C
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 34
Doctor sends traffic – SDA side – L2VPN
C
border01#show ip route vrf Internal03
No egress policy on
border
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 35
Doctor sends traffic – SDA side - L2VPN
C
border01#show ip route vrf Internal03 10.42.42.101
Pointing to default
route table
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 36
ACI / SDA connection - Underlay
Cp-TEP
10.92.92.228/32
Eth 1/41
OSPF
OSPF 10.82.82.6/30
BL
Gi0/1/0.901 Leaf 103
G 1/0/24 10.82.82.5/30
VPC Domain
10.82.82.1/30
B G 0/2/1
F
L3
Anycast-TEP
10.92.92.231/32
10.82.82.2/30 Gi0/2/0.901
Loopback1 / Tunnel0
10.82.83.4/32
10.82.82.9/30
Eth 1/41
BL
10.82.82.10/30
Leaf 104
Cp-TEP
10.92.92.227/32
SDA L3 ACI
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 37
Doctor sends traffic – SDA side - L2VPN
C
border01#show ip route
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 38
Doctor sends traffic – SDA side - L2VPN
C
Doctor client PC
192.168.1.3/32
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 39
Doctor sends traffic – SDA side - L2VPN
interface nve1
no ip address C
source-interface Loopback1
host-reachability protocol bgp
vxlan udp port 48879
group-based policy
SD-Access
B
member vni 4099 vrf Internal03 Fabric Site B
end
interface Loopback1
description Loopback for ACI-SDA
ip address 10.82.83.4 255.255.255.255
end
border01#
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 40
Doctor sends traffic – SDA side - L2VPN
C
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd Doctor client PC
10.92.92.227 4 101 720 794 7 0 0 11:55:22 1 192.168.1.3/32
10.92.92.228 4 101 720 800 7 0 0 11:55:17 1
border01#
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 41
Doctor sends traffic – SDA side - L2VPN
border01#show bgp l2vpn evpn
BGP table version is 7, local router ID is 172.16.1.254 C
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
x best-external, a additional-path, c RIB-compressed,
t secondary path, L long-lived-stale,
SD-Access
B
Origin codes: i - IGP, e - EGP, ? - incomplete Fabric Site B
RPKI validation codes: V valid, I invalid, N Not found
2
r
S
u te
3_o BL
rn al0
n te
NA
N T:I 4
_T E ed
T c
UL for
Border to
D EF A nen
U 3
spine in 4_ l03
outer p _5 064 na
2 er
md 39 Int
shadow VRF __ I: 2 T:
N
V
_T EN
AN
5
LT
1 DE
F AU
VRF: ACISDA:ACISDA VM
54_ 40 VNI: 2686976 10.42.42.101
_ 4
dp 1
_m 262
_ I:
VN
BL L
Leaf 103 Leaf 101
Leaf 104 Leaf 102
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 45
Doctor sends traffic – ACI side - Flow
bdsol-aci12-leaf3# show vrf
VRF-Name VRF-ID State Reason
__mdp_54_DEFAULT_TENANT:Internal 5 Up --
03
__mdp_54_DEFAULT_TENANT:Internal 6 Up --
03_outer
black-hole 3 Up --
management 2 Up --
overlay-1 4 Up --
BL
bdsol-aci12-leaf3# show ip route vrf __mdp_54_DEFAULT_TENANT:Internal03_outer
IP Route Table for VRF "__mdp_54_DEFAULT_TENANT:Internal03_outer"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 46
Doctor sends traffic – ACI side – Flow – Step 2
Spine remaps traffic
2
----------------------------------------------------------
Remote | Local
site Vrf PcTag | Vrf PcTag Rel-state
r
S ----------------------------------------------------------
1000 2392064 16 | 2621440 49153 [formed]
u te bdsol-aci12-spine1#
3_o BL
rn al0
n te
NA
N T:I 4
_T E ed
T c
UL for
D EF A nen
U 3
4_ l03
p _5 064 na
2 er
md 39 Int
__ I: 2 T:
N
V
_T EN
AN
5
LT
1 DE
F AU
VRF: ACISDA:ACISDA VM
54_ 40 VNI: 2686976 10.42.42.101
_ 4
dp 1
_m 262
_ I:
VN
BL L
Leaf 103 Leaf 101
Leaf 104 Leaf 102
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 47
Doctor sends traffic – ACI side - Flow
BL
----------------------------------------------------------
Remote | Local
site Vrf PcTag | Vrf PcTag Rel-state
----------------------------------------------------------
1000 2392064 16 | 2621440 49153 [formed]
VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 48
Doctor sends traffic – ACI side – Flow – Step 3
2
r
S
u te
3_o BL
rn al0
n te
NA
N T:I 4
_T E ed
T c
UL for
D EF A nen
U 3
4_ l03
p _5 064 na
2 er
md 39 t Traffic send
__ I: 2 T :In back to
N
V
T_
TE
N AN border in
inside
5
L
1 DE
F AU shadow VRF
VRF: ACISDA:ACISDA VM
54_ 40 VNI: 2686976 10.42.42.101
_ 4
dp 1
_m 262
_ I:
VN
BL L
Leaf 103 Leaf 101
Leaf 104 Leaf 102
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 49
Doctor sends traffic – ACI side - Flow
bdsol-aci12-spine1# show dcimgr repo sclass-maps
----------------------------------------------------------
Remote | Local
site Vrf PcTag | Vrf PcTag Rel-state
----------------------------------------------------------
1000 2392064 16 | 2621440 49153 [formed]
BL
module-1# show platform internal hal l3 routes | inc 2621440
|2621440| 0.0.0.0/ 0| UC| 5c| 19|
TCAM| 6| 0| 6|A| 75a9| 402e| NA| NA| NA| NA|
0| 0| 0| 0| 0| 1|
Hex/Dec
module-1# show platform internal hal l3 routes route-id 92 | inc "Next base Id"
Next base Id : 0x75a9
module-1# show platform internal hal l3 nexthops | grep 75a9
75a9 Y T F 0 1801000e 1801000e 0 1 1 0 0 402e 10 2 402e 0 20301e 0 0
0 0 0 1 0 0 0 00:0d:0d:0d:0d:0d 0 0 0 0 0 0 0 0 0 10.92.92.231
module-1# VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 50
Doctor sends traffic – ACI side – Flow – Step 4
Traffic send
from border
to access leaf
in dest. 2
VNI/pcTag
r
S
u te
3_o BL
rn al0
n te
NA
N T:I 4
_T E ed
T c
UL for
D EF A nen
U 3
4_ l03
p _5 064 na
2 er
md 39 Int
__ I: 2 T:
N
V
_T EN
AN
5
LT
1 DE
F AU
VRF: ACISDA:ACISDA VM
54_ 40 VNI: 2686976 10.42.42.101
_ 4
dp 1
_m 262
_ I:
VN
BL L
Leaf 103 Leaf 101
Leaf 104 Leaf 102
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 51
Doctor sends traffic – ACI side - Flow
bdsol-aci12-leaf3# show endpoint
Legend:
s - arp H - vtep V - vpc-attached p - peer-aged
R - peer-attached-rl B - bounce S - static M - span
D - bounce-to-proxy O - peer-attached a - local-aged m - svc-mgr
L - local E - shared-service
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info BL
+-----------------------------------+---------------+-----------------+--------------+-------------+
__mdp_54_DEFAULT_TENANT:Internal03 192.168.1.3 p tunnel15
…
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 52
Doctor sends traffic – ACI side - Flow
bdsol-aci12-spine1# show ip int brief | grep 32.67
lo9 10.0.32.67/32 protocol-up/link-up/admin-up
bdsol-aci12-spine2# show ip int brief | grep 10.0.32.67
lo9 10.0.32.67/32 protocol-up/link-up/admin-up
IP address : 10.42.42.101 BL
Vrf : 2686976
Flags : 0
EP bd vnid : 16121790
EP mac : 00:50:56:B6:3F:9D
Publisher Id : 10.0.8.64
Record timestamp : 03 03 2021 13:59:32 564569819
Publish timestamp : 03 03 2021 13:59:32 565091249
Seq No: 0
Remote publish timestamp: 01 01 1970 00:00:00 0
URIB Tunnel Info
Num tunnels : 1
Tunnel address : 10.0.32.64 VM
Tunnel ref count : 1
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 53
Doctor sends traffic – ACI side - Flow
bdsol-aci12-apic1# moquery -c fvAEPg -d "uni/tn-ACISDA/ap-E-commerce/epg-Web" | egrep "^dn|pcTag"
dn : uni/tn-ACISDA/ap-E-commerce/epg-Web
pcTag : 5474
Policy applied in
internal/campus
shadow VRF on
BL
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 54
Doctor sends traffic – ACI side - Flow
BL
VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 55
Doctor sends traffic – ACI side - Flow
BL
VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 56
Doctor sends traffic – ACI side - Flow
BL
VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 57
Doctor sends traffic – ACI side – Flow – Step 5
2
r
S
u te
3_o BL
rn al0
n te
NA
N T:I 4
_T E ed
T c
UL for
D EF A nen
U 3
4_ l03
p _5 064 na
Traffic leaked
2 er
md 39 t into destination
__ I: 2 :In VRF
N NT
V
_TE
NA 5
LT
1 DE
F AU
VRF: ACISDA:ACISDA VM
54_ 40 VNI: 2686976 10.42.42.101
_ 4
dp 1
_m 262
_ I:
VN
BL L
Reminder: ACI Route leaking
Leaf 103 Leaf 101
Configure the subnet on the
Leaf 104 Leaf 102 provider EPG
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 58
Doctor sends traffic – ACI side - Flow
bdsol-aci12-leaf1# show endpoint
Legend:
s - arp H - vtep V - vpc-attached p - peer-aged
R - peer-attached-rl B - bounce S - static M - span
D - bounce-to-proxy O - peer-attached a - local-aged m - svc-mgr
L - local E - shared-service
+-----------------------------------+---------------+-----------------+--------------+-------------+
VLAN/ Encap MAC Address MAC Info/ Interface
Domain VLAN IP Address IP Info BL
+-----------------------------------+---------------+-----------------+--------------+-------------+
13 vlan-1035 0050.56b6.3f9d LV po2
ACISDA:ACISDA vlan-1035 10.42.42.101 LV po2
13 vlan-1035 0050.5687.6567 LpV po2
ACISDA:ACISDA vlan-1035 10.42.42.102 LV po2
overlay-1 10.0.136.64 L lo0
overlay-1 10.0.32.64 L lo1
11/overlay-1 vxlan-16777209 40f0.7843.26b0 L eth1/2
11/overlay-1 vxlan-16777209 3c57.311a.4384 L eth1/1
11/overlay-1 vxlan-16777209 2cf8.9b29.2f1e L eth1/3
VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 59
Doctor sends traffic – ACI side – Complete flow
Spine remaps traffic
Traffic send
from border bdsol-aci12-spine1# show dcimgr repo sclass-maps
2
to access leaf ----------------------------------------------------------
in dest. Remote | Local
VNI/pcTag site Vrf PcTag | Vrf PcTag Rel-state
r
S ----------------------------------------------------------
1000 2392064 16 | 2621440 49153 [formed]
u te bdsol-aci12-spine1#
3_o BL
rn al0
n te
NA
N T:I 4
_T E ed
T c
UL for
Border to
D EF A nen
U 3
spine in 4_ l03
outer p _5 064 na
Traffic leaked
2 er
md 39 t Traffic send into destination
shadow VRF __ I: 2 :In back to VRF
N NT
V
_TE
NA border in
inside
5
LT
1 DE
F AU shadow VRF
VRF: ACISDA:ACISDA VM
54_ 40 VNI: 2686976 10.42.42.101
_ 4
dp 1
_m 262
_ I:
VN
BL L
Leaf 103 Leaf 101
Leaf 104 Leaf 102
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 60
From ACI à
SDA
Next slides will
cover the
difference
compared to
SDA à ACI
General flow
• Step 1: Access leaf sends traffic to border leaf
• Route leaking, towards internal VRF
• Step 2: Border leaf sends traffic in campus shadow VRF to spine
• Step 3: Spine changes VRF/pcTag
• Step 4: Spine sends traffic ACI Border leaf in outer VRF
• Step 5: SDA Border receives traffic
• Step 6: SDA Border forwards tragic to Edge
• Step 7: Edge delivers packet
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 62
Return traffic - ACI side – Access leaf
bdsol-aci12-leaf1# show ip route vrf ACISDA:ACISDA
IP Route Table for VRF "ACISDA:ACISDA"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
How do we receive
__mdp_54_DEFAULT_TENANT:Internal03_outer
__mdp_54_DEFAULT_TENANT:Internal03
2392064
2621440 these routes ?
ACISDA:ACISDA 2686976
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 63
Return traffic - ACI side – Access leaf
bdsol-aci12-leaf1# show bgp vpnv4 unicast vrf ACISDA:ACISDA
BGP routing table information for VRF overlay-1, address family VPNv4 Unicast
BGP table version is 27, local router ID is 10.0.136.64
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup
VM
bdsol-aci12-leaf1# acidiag fnvread | egrep "136.71|136.69"
103 1 bdsol-aci12-leaf3 FDO24310LXQ 10.0.136.69/32 leaf active 0 10.42.42.101
104 1 bdsol-aci12-leaf4 FDO24311CJN 10.0.136.71/32 leaf active 0
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 64
Return traffic - ACI side – Border inside VRF
bdsol-aci12-leaf3# show ip route vrf __mdp_54_DEFAULT_TENANT:Internal03
IP Route Table for VRF "__mdp_54_DEFAULT_TENANT:Internal03"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 65
Return traffic - ACI side – Border inside VRF
bdsol-aci12-leaf3# show ip route vrf __mdp_54_DEFAULT_TENANT:Internal03
IP Route Table for VRF "__mdp_54_DEFAULT_TENANT:Internal03"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
VM
10.42.42.101
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 66
Return traffic - ACI side – Spine
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 67
Return traffic - ACI side – Border outer VRF
bdsol-aci12-leaf3# show ip route vrf __mdp_54_DEFAULT_TENANT:Internal03_outer
IP Route Table for VRF "__mdp_54_DEFAULT_TENANT:Internal03_outer"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 68
ACI / SDA connection - Underlay
Cp-TEP
10.92.92.228/32
Eth 1/41
OSPF
OSPF 10.82.82.6/30
BL
Gi0/1/0.901 Leaf 103
G 1/0/24 10.82.82.5/30
VPC Domain
10.82.82.1/30
B G 0/2/1
F
L3
Anycast-TEP
10.92.92.231/32
10.82.82.2/30 Gi0/2/0.901
Loopback1 / Tunnel0
10.82.83.4/32
10.82.82.9/30
Eth 1/41
BL
10.82.82.10/30
Leaf 104
Cp-TEP
10.92.92.227/32
SDA L3 ACI
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 69
Return traffic - ACI side – Border outer VRF
bdsol-aci12-leaf3# show bgp ipv4 unicast 192.168.1.3 vrf __mdp_54_DEFAULT_TENANT:Internal03_outer
BGP routing table information for VRF __mdp_54_DEFAULT_TENANT:Internal03_outer, address family IPv4 Unicast
BGP routing table entry for 192.168.1.0/24, version 4 dest ptr 0xa03109a4
Paths: (1 available, best #1)
Flags: (0x0c001a 00000000) on xmit-list, is in urib, is best urib route, is in HW, exported
vpn: version 16, (0x100002) on xmit-list
Multipath: eBGP iBGP
BL
Advertised path-id 1, VPN AF advertised path-id 1
Path type: external 0xc0000028 0x0 ref 0 adv path ref 2, path is valid, is best path, remote nh not
installed
Imported from 1:4099:[5]:[0]:[0]:[24]:[192.168.1.0]:[0.0.0.0]/120
AS-Path: 65003 , path sourced external to AS
10.82.83.4 (metric 42) from 10.82.83.4 (172.16.1.254)
Origin IGP, MED 0, localpref 100, weight 0 tag 0, propagate 0
Aggregated by 172.16.1.254, aggregator AS 65003, atomic-aggregate set
Received label 4099
Extcommunity:
ENCAP:8 VM
Router MAC:7035.093c.804b
VNID:4099 10.42.42.101
Normal lookup in
LISP
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 71
Return traffic - SDA side – edge
C
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 72
Return traffic - SDA side – edge C
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 73
Continue your education
Walk-in labs
Related sessions
#CiscoLive BRKACI-2008 © 2021 Cisco and/or its affiliates. All rights reserved. Cisco Public 74
Thank you
#CiscoLive
#CiscoLive