ELAM Packet capture:
Step 1: Kindly check the hardware module to see what inputs must be there in command i.e. tah/roc/app
/// N9K-C*-EX- Leaf ----------------------- tah
N9K-C*FX/FXP/FX2 -Leaf ------------ roc
N9K-C*-GX – Leaf ----------------------app//
L3 traffic
ELAM JI
=========================
Leaf1
=========================
debug platform internal tah elam asic 0
trigger reset
trigger init in-select 6 out-select 1
set outer ipv4 src_ip 192.168.11.11 dst_ip 192.168.12.12
start
status
ereport
Python available. Continue ELAM decode with LC Pkg
ELAM REPORT
======================================================================================================================================================
Trigger/Basic Information
======================================================================================================================================================
ELAM Report File : /tmp/logs/elam_2024-06-29-02m-18h-46s.txt
In-Select Trigger : Outerl2-outerl3-outerl4( 6 )
Out-Select Trigger : Pktrw-sideband-drpvec( 1 )
ELAM Captured Device : LEAF
Packet Direction : ingress
Triggered ASIC type : Homewood
Triggered ASIC instance : 0
Triggered Slice : 0
Incoming Interface : 0x58( 0x58 )
( Slice Source ID(Ss) in "show plat int hal l2 port gpd" )
======================================================================================================================================================
Captured Packet
======================================================================================================================================================
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer Packet Attributes
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer Packet Attributes : l2uc ipv4 ip ipuc ipv4uc
Opcode : OPCODE_UC
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L2 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
Destination MAC : 0022.BDF8.19FF
Source MAC : ACF2.C5F8.2F81
802.1Q tag is valid : yes( 0x1 )
CoS : 0( 0x0 )
Access Encap VLAN : 3311( 0xCEF )
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
IP Version : 4
DSCP : 0
IP Packet Length : 84 ( = IP header(28 bytes) + IP payload )
Don't Fragment Bit : not set
TTL : 255
IP Protocol Number : ICMP
IP CheckSum : 13018( 0x32DA )
Destination IP : 192.168.12.12
Source IP : 192.168.11.11
module-1(DBG-elam-insel6)# show plat int hal l2 port gpd
For Incoming interface
Incoming Interface : 0x58( 0x58 ) <-- Ss (Slice source)
For Outgoing Interface
module-1(DBG-elam-insel6)# report | grep ovec
hom_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x40
leaf1# show lldp neighbors int eth1/52
Device ID Local Intf Hold-time Capability Port ID
Spine1 Eth1/52 120 BR Eth1/29
========================
Spine1
========================
debug platform internal roc elam asic 0
trigger reset
trigger init in-select 14 out-select 1
set inner ipv4 src_ip 192.168.11.11 dst_ip 192.168.12.12
start
module-1(DBG-elam-insel14)# status
ELAM STATUS
===========
Asic 0 Slice 0 Status Triggered
Asic 0 Slice 1 Status Armed
Asic 0 Slice 2 Status Armed
Asic 0 Slice 3 Status Armed
module-1(DBG-elam-insel14)# ereport
Python available. Continue ELAM decode with LC Pkg
ELAM REPORT
======================================================================================================================================================
Trigger/Basic Information
======================================================================================================================================================
ELAM Report File : /tmp/logs/elam_2024-06-29-18m-16h-32s.txt
In-Select Trigger : Outer(l2(vntag)|l3|l4)-inner(l2|l3|l4)-ieth( 14 )
Out-Select Trigger : Pktrw-sideband-drpvec( 1 )
ELAM Captured Device : SPINE_SC
Packet Direction : egress
Triggered ASIC type : Bigsky
Triggered ASIC instance : 0
Triggered Slice : 0
Incoming Interface : 0x58( 0x58 )
( Slice Source ID(Ss) in "show plat int hal l2 port gpd" )
Packet from vPC peer LEAF : yes
Packet from tunnel (remote leaf/avs) : yes
======================================================================================================================================================
Captured Packet
======================================================================================================================================================
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer Packet Attributes
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer Packet Attributes : l2uc ipv4 ip ipuc ipv4uc udp ivxlan
Opcode : OPCODE_UC
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer iEth Header
------------------------------------------------------------------------------------------------------------------------------------------------------
iEth SUP code : NONE
Packet from CPU : no
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L2 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
Destination MAC : 000D.0D0D.0D0D
Source MAC : 000C.0C0C.0C0C
802.1Q tag is valid : yes
CoS : 0
Access Encap VLAN : 2
VN-Tag is valid : no
<-- ( FC tells LC via Vn-Tag about which port the packet needs to go to )
Src VIF (in from leaf/IPN) : 0
<-- ( VIF(dec) in ELTMC or LID(hex) in "show plat int hal l2 port pi" )
Dst VIF (out to leaf/IPN) : 0
<-- ( OIFL in "show forwarding multicast outgoing-interface-list 0" )
------------------------------------------------------------------------------------------------------------------------------------------------------
Inner L2 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
Inner Destination MAC : 000C.0C0C.0C0C
Source MAC : 000C.0C0C.0C0C
802.1Q tag is valid : no
CoS : 0
Access Encap VLAN : 0
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
DSCP : 0
Don't Fragment Bit : 0x0
TTL : 32
IP Protocol Number : UDP
Destination IP : 10.0.248.0
Source IP : 10.0.32.69
------------------------------------------------------------------------------------------------------------------------------------------------------
Inner L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
DSCP : 0
Don't Fragment Bit : 0x0
TTL : 254
IP Protocol Number : ICMP
Destination IP : 192.168.12.12
Source IP : 192.168.11.11
module-1(DBG-elam-insel14)# show platform internal hal l2 port gpd
Incoming interface
Incoming Interface : 0x58( 0x58 )
Outgoing interface
module-1(DBG-elam-insel14)# report | grep ovec
bky_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x148
module-1(DBG-elam-insel14)# show platform internal hal l2 port gpd | grep 148
1a000000 Eth1/1 0 2 3 0 2a 2 9 48 148 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 D-24d - 100 0 1 1 2 0 0
spine1# show lldp neighbors interface ethernet 1/1
Capability codes:
Device ID Local Intf Hold-time Capability Port ID
leaf3 Eth1/1 120 BR Eth1/49
========================
Leaf3
========================
debug platform internal tah elam asic 0
trigger reset
trigger init in-select 14 out-select 1
set inner ipv4 src_ip 192.168.11.11 dst_ip 192.168.12.12
start
module-1(DBG-elam-insel14)# status
ELAM STATUS
===========
Asic 0 Slice 0 Status Armed
Asic 0 Slice 1 Status Triggered
module-1(DBG-elam-insel14)# ereport
Python available. Continue ELAM decode with LC Pkg
ELAM REPORT
======================================================================================================================================================
Trigger/Basic Information
======================================================================================================================================================
ELAM Report File : /tmp/logs/elam_2024-06-29-24m-16h-34s.txt
In-Select Trigger : Outer(l2(vntag)|l3|l4)-inner(l2|l3|l4)-ieth( 14 )
Out-Select Trigger : Pktrw-sideband-drpvec( 1 )
ELAM Captured Device : LEAF
Packet Direction : egress
Triggered ASIC type : Sugarbowl
Triggered ASIC instance : 0
Triggered Slice : 1
Incoming Interface : 0x38( 0x38 )
( Slice Source ID(Ss) in "show plat int hal l2 port gpd" )
Packet from vPC peer LEAF : yes
Packet from tunnel (remote leaf/avs) : yes
======================================================================================================================================================
Captured Packet
======================================================================================================================================================
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer Packet Attributes
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer Packet Attributes : l2uc ipv4 ip ipuc ipv4uc udp ivxlan
Opcode : OPCODE_UC
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer iEth Header
------------------------------------------------------------------------------------------------------------------------------------------------------
iEth SUP code : NONE
Packet from CPU : no
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L2 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
Destination MAC : 000C.0C0C.0C0C
Source MAC : 0000.0000.0000
802.1Q tag is valid : yes
CoS : 0
Access Encap VLAN : 2
VN-Tag is valid : no
<-- ( FC tells LC via Vn-Tag about which port the packet needs to go to )
Src VIF (in from leaf/IPN) : 0
<-- ( VIF(dec) in ELTMC or LID(hex) in "show plat int hal l2 port pi" )
Dst VIF (out to leaf/IPN) : 0
<-- ( VIF(dec) in ELTMC or LID(hex) in "show plat int hal l2 port pi" )
-------------------------------------------------------------------------------------------------------------------------------
Inner L2 Header
-------------------------------------------------------------------------------------------------------------------------------
Inner Destination MAC : 000C.0C0C.0C0C
Source MAC : 000C.0C0C.0C0C
802.1Q tag is valid : no
CoS : 0
Access Encap VLAN : 0
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
DSCP : 0
Don't Fragment Bit : 0x0
TTL : 32
IP Protocol Number : UDP
Destination IP : 10.0.32.67
Source IP : 10.0.32.69
------------------------------------------------------------------------------------------------------------------------------------------------------
Inner L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
DSCP : 0
Don't Fragment Bit : 0x0
TTL : 254
IP Protocol Number : ICMP
Destination IP : 192.168.12.12
Source IP : 192.168.11.11
module-1(DBG-elam-insel14)# show platform internal hal l2 port gpd
Incoming Packet
Incoming Interface : 0x38( 0x38 )
Outgoing Interface
module-1(DBG-elam-insel14)# report | grep ovec
sug_elam_out_sidebnd_no_spare_vec.ovector_idx: 0x20
======================
Traffic from leaf3 to leaf1
=======================
debug platform internal tah elam asic 0
trigger reset
trigger init in-select 6 out-select 1
set outer ipv4 src_ip 192.168.12.12 dst_ip 192.168.11.11
start
status
ereport
================================================
L2 traffic
================================================
vsh_lc
deb pla in <tah/roc/app> elam asic 0
trigger init in-select 6 out-select 1
set out l2 src-mac <Mac> dst-mac <mac address>
start
status
ereport
================================================
Step-by-Step Guide
1. Identify the Target Switch and ASIC
Determine the Switch:
Identify the leaf or spine switch where the packet is being processed.
Use tools like Traceroute, Atomic Counters, or EP Lookup to find the path.
Identify the ASIC:
Know whether you're working with an NFE, ALE, or another ASIC type based on your switch model.
2. Access the Switch
SSH into the Switch:
bash
ssh admin@<switch_ip>
Enter Diagnostic Mode:
For leaf switches:
bash
fabric-switch# vsh_lc
module-1#
For spine switches, the process may differ slightly based on the ASIC.
3. Enter the ELAM Module
Access the ELAM Prompt:
For an NFE ASIC:
bash
module-1# elam asic nfe
For an ALE ASIC:
bash
module-1# elam asic ale
4. Clear Previous ELAM Settings
Ensure a Clean State:
bash
elam> clear
5. Configure Trigger Conditions
Set Packet Matching Criteria:
Source IP Address:
bash
elam> trig ip sa 192.168.1.10
Destination IP Address:
bash
elam> trig ip da 192.168.2.20
Source MAC Address:
bash
elam> trig mac sa 00:11:22:33:44:55
Destination MAC Address:
bash
elam> trig mac da 66:77:88:99:AA:BB
VLAN ID:
bash
elam> trig vlan 100
Ingress Interface:
bash
elam> trig ingress port ethernet1/1
Example: To trigger on packets from source IP 192.168.1.10 to destination IP 192.168.2.20 on VLAN 100:
bash
elam> trig ip sa 192.168.1.10 da 192.168.2.20 elam> trig vlan 100
6. Arm the ELAM
Start Listening for the Packet:
bash
elam> arm
The ELAM is now waiting for a packet that matches your trigger conditions.
7. Generate or Wait for the Packet
Initiate Traffic:
Send traffic that matches your trigger conditions from the source device.
Ensure that the packet traverses the switch you're monitoring.
8. Verify Packet Capture
Check ELAM Status:
bash
elam> status
Confirm that the status indicates a packet has been captured.
9. Display and Analyze the Results
View Captured Packet Details:
bash
elam> show
Important Fields to Review:
Ingress/Egress Ports
VLAN and VNID
Source and Destination MAC Addresses
Source and Destination IP Addresses
Forwarding Decisions (e.g., routed, dropped)
ACL Hits or Drops
Sample Output:
yaml
Ingress Port: Ethernet1/1 VLAN: 100 Source MAC: 00:11:22:33:44:55 Destination MAC: 66:77:88:99:AA:BB Source IP: 192.168.1.10 Destination IP: 192.168.2.20 Forwarding Action: Routed Egress Port: Ethernet1/2
10. Clear ELAM Settings
Reset ELAM for Future Use:
bash
elam> clear
Exit ELAM Module:
bash
elam> exit module-1
# exit fabric-switch#
Yorumlar