The fate of the package. Cisco IOS XE


    You can start diagnosing many problems on a Cisco IOS XE router with Packet Trace . This is a packet processing trace inside the router that appeared not so long ago. Previously, such functionality was only available on ASA firewalls. Those who used packet-tracer on the ASA will agree - a very convenient tool. Now its analogue has appeared on modern routers (ISR 4000, ASR, CSR).

    I will build a note on living examples. It’s easier to get an idea of ​​the IOS-XE Packet Trace. Details can always be found on the vendor's website. It is a pity that there is not much information on this subject yet. During our dive, you will understand what I mean.

    As an experimental, we have an ISR 4000 router (I already wrote about the specifics of the ISR 4000 and IOS XE on Habr) A number of technologies are configured on it: static routing, PfR, PBR, address translation (NAT), ZFW firewall, ACL on interfaces, Flexible NetFlow, NBAR2, IPSec, GRE, VTI and more. All this will make the trace more saturated and closer to the actual operation.

    There are many technologies and each has its own debugging method. In order not to waste time and immediately determine where to look for the cause of the problem, Packet Trace will come in handy.

    We will observe the ICMP packet (echo request) sent from the address 192.168.20.8 to 8.8.8.8.

    Trace activationconsists of two parts. First, run the conditional debug. It is in it that we indicate which packages interest us. In our case, this is the traffic described by ACL 199 and arriving at the router through the GigabitEthernet0 / 0/0 interface:

    access-list 199 permit icmp host 192.168.20.8 host 8.8.8.8
    debug platform condition interf GigabitEthernet0/0/0 ipv4 access-list 199 ingress
    debug platform condition start

    The conditional debugger is used not only for packet trace operation. This tool allows you to effectively filter log messages and debug messages at the stage of their generation. We can set conditions and see records relating only to what we need.

    Next, turn on packet trace directly. Specify the buffer and trace depth. Minimum - 16 packages. Depth: base (path-trace) or extended (fia-trace). In the case of extended, we get a detailed conclusion of the work of all functions within the QFP process. It is he who is responsible for the transmission of packets (datapath).

    debug platform packet-trace packet 16 fia-trace
    debug platform packet-trace enable

    Compared to ASA packet-tracer, the syntax is certainly not that convenient.

    ASA packet-tracer can itself generate packets for further tracing. IOS-XE Packet Trace does not know how to do this. For it to work, it is necessary that the package comes from somewhere.
    Teams for cleaning tails. Useful when we are done with everything.

    no debug platform packet-trace enable
    clear platform packet-trace statistics
    clear platform condition all

    Everything is set up. We start ping so that the packet we need passes through the router.
    We look at the general output of packets that got into packet trace.

    cbs-4000#show platform packet-trace summary
    Pkt   Input             Output            State  Reason
    0     Gi0/0/0           Gi0/0/1.5         FWD 

    We have one. Came through the Gi0 / 0/0 interface and was transferred further (FWD state) via Gi0 / 0 / 1.5.

    We look at the trace of its processing
    cbs-4000#show platform packet-trace packet 0
    Packet: 0           CBUG ID: 8
    Summary
      Input     : GigabitEthernet0/0/0
      Output    : GigabitEthernet0/0/1.5
      State     : FWD 
      Timestamp
        Start   : 6495209991683323 ns (02/18/2017 11:59:43.176192 UTC)
        Stop    : 6495209991814307 ns (02/18/2017 11:59:43.176323 UTC)
    Path Trace
      Feature: IPV4                                             <=================
        Input       : GigabitEthernet0/0/0                      <=================
        Output      : GigabitEthernet0/0/0                      <=================
        Source      : 192.168.20.8                              <=================
        Destination : 8.8.8.8                                   <=================
        Protocol    : 1 (ICMP)                                  <=================
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x8112bfbc - DEBUG_COND_INPUT_PKT
        Lapsed time : 4960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e84 - IPV4_INPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 5280 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e64 - IPV4_INPUT_DST_LOOKUP_CONSUME
        Lapsed time : 1600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4a140 - IPV4_INPUT_ACL
        Lapsed time : 40160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e88 - IPV4_INPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e68 - IPV4_INPUT_FOR_US_MARTIAN
        Lapsed time : 1440 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 236
        cft_bucket_number     : 566799
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8                  <=================
        tuple.dst_ip          : 8.8.8.8                       <=================
        tuple.src_port        : 61609                         <=================
        tuple.dst_port        : 161                           <=================
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP                          <=================
        tuple.l3_protocol     : IPV4                          <=================
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 236
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d83558 - IPV4_INPUT_STILE_LEGACY
        Lapsed time : 226240 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7b508 - IPV4_INGRESS_MMA_LOOKUP
        Lapsed time : 66880 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d59618 - IPV4_INPUT_FME_PROCESS
        Lapsed time : 2560 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000084
        input vrf_idx         : 0
        calling feature       : FNF
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 236
        cft_bucket_number     : 566799
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 61609
        tuple.dst_port        : 161
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 236
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6dc84 - IPV4_INPUT_FNF_AOR_FIRST
        Lapsed time : 21120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6d9d4 - IPV4_INPUT_FNF_FIRST
        Lapsed time : 119520 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e8c - IPV4_INPUT_VFR
        Lapsed time : 1280 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b660 - IPV4_INPUT_CENT_SMP_PROCESS
        Lapsed time : 3840 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000080
        input vrf_idx         : 0
        calling feature       : CENT
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 236
        cft_bucket_number     : 566799
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 61609
        tuple.dst_port        : 161
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 236
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b62c - IPV4_INPUT_CENT_RC_PROCESS
        Lapsed time : 40640 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7ff70 - IPV4_INPUT_PBR              <=================
        Lapsed time : 34720 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d858d0 - IPV4_INPUT_TCP_ADJUST_MSS   <=================     
        Lapsed time : 2560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0                     <=================
        Output      : GigabitEthernet0/0/1.5                   <=================
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS   <=================
        Lapsed time : 4160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6dc88 - IPV4_INPUT_FNF_AOR_FINAL
        Lapsed time : 1280 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6d974 - IPV4_INPUT_FNF_FINAL
        Lapsed time : 218880 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6dc8c - IPV4_INPUT_FNF_AOR_RELEASE
        Lapsed time : 2560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e94 - IPV4_INPUT_IPOPTIONS_PROCESS
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113ac44 - IPV4_INPUT_GOTO_OUTPUT_FEATURE
        Lapsed time : 4480 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e98 - IPV4_OUTPUT_VFR
        Lapsed time : 1920 ns
      Feature: ZBFW                                            <=================
        Action  : Fwd                                          <=================
        Zone-pair name  : in-out1                              <=================
        Class-map name  : CM-FW_in-out                         <=================
        Input interface : GigabitEthernet0/0/0                 <=================
        Egress interface: GigabitEthernet0/0/1.5               <=================
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d70b28 - IPV4_OUTPUT_INSPECT
        Lapsed time : 721760 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d77188 - MC_OUTPUT_GEN_RECYCLE
        Lapsed time : 3680 ns
      Feature: NAT                                             <=================
        Direction   : IN to OUT                                <=================
        Action      : Translate Source                         <=================
        Old Address : 192.168.20.8  00001                      <=================
        New Address : 87.87.87.87 00033                        <=================
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d7c390 - IPV4_NAT_OUTPUT_FIA
        Lapsed time : 54880 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d85d30 - IPV4_OUTPUT_THREAT_DEFENSE
        Lapsed time : 1600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e9c - IPV4_VFR_REFRAG
        Lapsed time : 960 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Output
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 238
        cft_bucket_number     : 566799
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 87.87.87.87
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 61609
        tuple.dst_port        : 161
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 238
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d8359c - IPV4_OUTPUT_STILE_CLR_TXT
        Lapsed time : 137600 ns
      Feature: IPSec                                            <=================
        Result    : IPSEC_RESULT_DENY                           <=================
        Action    : SEND_CLEAR                                  <=================
        SA Handle : 0    
        Peer Addr : 8.8.8.8                                     <=================
        Local Addr: 87.87.87.87                                 <=================
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d761ac - IPV4_OUTPUT_IPSEC_CLASSIFY
        Lapsed time : 50560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e70 - IPV4_OUTPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 7040 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81128eb0 - IPV4_OUTPUT_L2_REWRITE
        Lapsed time : 7040 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e74 - IPV4_OUTPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131ec4 - IPV4_OUTPUT_FRAG
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81133e50 - IPV4_OUTPUT_DROP_POLICY
        Lapsed time : 13600 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6d914 - IPV4_OUTPUT_FNF_FINAL
        Lapsed time : 112800 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113bb40 - MARMOT_SPA_D_TRANSMIT_PKT
        Lapsed time : 41440 ns

    Trace volume directly depends on the configured functions. If we had only routing, there would be much less data.

    Some of the names are clear. But there are stages that are not easy to decode. Documentation of the vendor is not very helpful in this regard.

    We highlight the most interesting points

    1. Information that identifies our data flow:

    Feature: CFT
        …
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 61609
        tuple.dst_port        : 161
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
    

    Data is stored in a CFT (Common Flow Table). They are used by technologies that operate in their work with information about each stream (Netflow, NBAR, PfR, etc.). The CFT table is necessary so as not to store redundant information.

    2. Definition of the outgoing interface:

    When the packet just got on the router, the outgoing interface is not defined. Incoming is substituted:

    
    Feature: IPV4
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Source      : 192.168.20.8
        Destination : 8.8.8.8
        Protocol    : 1 (ICMP)

    Once it is determined where to send the packet further (the routing function is performed), the outgoing interface changes:

      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS
        Lapsed time : 4160 ns

    3. Data on packet processing by ZFW firewall:

      Feature: ZBFW
        Action  : Fwd
        Zone-pair name  : in-out1
        Class-map name  : CM-FW_in-out
        Input interface : GigabitEthernet0/0/0
        Egress interface: GigabitEthernet0/0/1.5

    We immediately see which zones the packet passed between and in which class it fell. This is quite convenient, since the ZFW configuration is often very confusing.

    4. Address translation information:

      Feature: NAT
        Direction   : IN to OUT
        Action      : Translate Source
        Old Address : 192.168.20.8  00001
        New Address : 87.87.87.87 00033

    The destination address in the packet has been replaced by 87.87.87.87.

    5. Since IPSec is configured on our router, it will be noted whether the packet got into it:

      Feature: IPSec
        Result    : IPSEC_RESULT_DENY
        Action    : SEND_CLEAR
        SA Handle : 0
        Peer Addr : 8.8.8.8
        Local Addr: 87.87.87.87

    No, I didn’t.

    In the traces a lot of additional information is presented. For example, IPV4_INPUT_PBR indicates that a packet has passed through PBR. But we won’t find information on whether PBR was applied or if the packet was sent for processing to standard routing rules. In our case, the packet did not fall under the PBR rules. The IPV4_INPUT_TCP_ADJUST_MSS entry indicates that the ip tcp adjust-mss command is configured on the interface. At the same time, as in the previous example, we do not get any details.

    Most of the information displayed by the device is not of interest. However, the situation will change when something goes wrong with the package.

    Situation No. 1. Packet dropped ACL on input interface

    cbs-4000#show platform packet-trace summary
    Pkt   Input             Output            State  Reason
    0     Gi0/0/0           Gi0/0/0           DROP   8   (Ipv4Acl)

    The packet was dropped (DROP) because the ACL (Ipv4Acl) worked.

    Package Processing Trace
    cbs-4000#show platform packet-trace packet 0
    Packet: 0           CBUG ID: 35
    Summary
      Input     : GigabitEthernet0/0/0
      Output    : GigabitEthernet0/0/0
      State     : DROP 8   (Ipv4Acl)
      Timestamp
        Start   : 6515970748260480 ns (02/18/2017 17:45:43.568889 UTC)
        Stop    : 6515970748313558 ns (02/18/2017 17:45:43.568942 UTC)
    Path Trace
      Feature: IPV4
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Source      : 192.168.20.8
        Destination : 8.8.8.8
        Protocol    : 1 (ICMP)
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x8112bfbc - DEBUG_COND_INPUT_PKT
        Lapsed time : 6560 ns
      Feature: FIA_TRACE                               
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e84 - IPV4_INPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 5920 ns                              
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e64 - IPV4_INPUT_DST_LOOKUP_CONSUME
        Lapsed time : 1440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d8375c - STILE_LEGACY_DROP_EXT
        Lapsed time : 3680 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7b554 - INGRESS_MMA_LOOKUP_DROP_EXT
        Lapsed time : 63040 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6e0f8 - INPUT_DROP_FNF_AOR_EXT
        Lapsed time : 8320 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6dc44 - INPUT_FNF_DROP_EXT
        Lapsed time : 324800 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6e6c8 - INPUT_DROP_FNF_AOR_RELEASE_EXT
        Lapsed time : 8320 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81128ebc - INPUT_DROP_EXT                 <=================
        Lapsed time : 1920 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4a140 - IPV4_INPUT_ACL                 <=================
        Lapsed time : 794240 ns

    INPUT_DROP_EXT and IPV4_INPUT_ACL report that the packet was dropped on the inbound interface. Traces turned out to be short, like the life of a package.

    Situation No. 2. Packet dropped ACL on outbound interface

    cbs-4000#show platform packet-trace summary
    Pkt   Input             Output            State  Reason
    0     Gi0/0/0           Gi0/0/1.5         DROP   8   (Ipv4Acl)

    Again, the packet was not transmitted (DROP) due to the ACL (Ipv4Acl). Now, however, Gi0 / 0 / 1.5 appears as the outgoing interface.

    Package Processing Trace
    cbs-4000#show platform packet-trace packet 0
    Packet: 0           CBUG ID: 33
    Summary
      Input     : GigabitEthernet0/0/0
      Output    : GigabitEthernet0/0/0
      State     : DROP 8   (Ipv4Acl)
      Timestamp
        Start   : 6515547984424423 ns (02/18/2017 17:38:40.479689 UTC)
        Stop    : 6515547984571057 ns (02/18/2017 17:38:40.479835 UTC)
    Path Trace
      Feature: IPV4
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Source      : 192.168.20.8
        Destination : 8.8.8.8
        Protocol    : 1 (ICMP)
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x8112bfbc - DEBUG_COND_INPUT_PKT
        Lapsed time : 8320 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e84 - IPV4_INPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 4320 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e64 - IPV4_INPUT_DST_LOOKUP_CONSUME
        Lapsed time : 3520 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4a140 - IPV4_INPUT_ACL
        Lapsed time : 43360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e88 - IPV4_INPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e68 - IPV4_INPUT_FOR_US_MARTIAN
        Lapsed time : 1280 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 5
        cft_bucket_number     : 1591662
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 443
        tuple.dst_port        : 57521
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 5
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d83558 - IPV4_INPUT_STILE_LEGACY
        Lapsed time : 222240 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7b508 - IPV4_INGRESS_MMA_LOOKUP
        Lapsed time : 67200 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d59618 - IPV4_INPUT_FME_PROCESS
        Lapsed time : 2240 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000084
        input vrf_idx         : 0
        calling feature       : FNF
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 5
        cft_bucket_number     : 1591662
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 443
        tuple.dst_port        : 57521
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 5
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6dc84 - IPV4_INPUT_FNF_AOR_FIRST
        Lapsed time : 22080 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6d9d4 - IPV4_INPUT_FNF_FIRST
        Lapsed time : 136320 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e8c - IPV4_INPUT_VFR
        Lapsed time : 1280 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b660 - IPV4_INPUT_CENT_SMP_PROCESS
        Lapsed time : 2560 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000080
        input vrf_idx         : 0
        calling feature       : CENT
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 5
        cft_bucket_number     : 1591662
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 443
        tuple.dst_port        : 57521
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 5
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b62c - IPV4_INPUT_CENT_RC_PROCESS
        Lapsed time : 40160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7ff70 - IPV4_INPUT_PBR
        Lapsed time : 39520 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d858d0 - IPV4_INPUT_TCP_ADJUST_MSS
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS
        Lapsed time : 4320 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6dc88 - IPV4_INPUT_FNF_AOR_FINAL
        Lapsed time : 1920 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6d974 - IPV4_INPUT_FNF_FINAL
        Lapsed time : 274240 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6dc8c - IPV4_INPUT_FNF_AOR_RELEASE
        Lapsed time : 2400 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e94 - IPV4_INPUT_IPOPTIONS_PROCESS
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113ac44 - IPV4_INPUT_GOTO_OUTPUT_FEATURE
        Lapsed time : 2880 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e98 - IPV4_OUTPUT_VFR
        Lapsed time : 1600 ns
      Feature: ZBFW
        Action  : Fwd
        Zone-pair name  : in-out1
        Class-map name  : CM-FW_in-out
        Input interface : GigabitEthernet0/0/0
        Egress interface: GigabitEthernet0/0/1.5
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d70b28 - IPV4_OUTPUT_INSPECT
        Lapsed time : 989760 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d77188 - MC_OUTPUT_GEN_RECYCLE
        Lapsed time : 2720 ns
      Feature: NAT
        Direction   : IN to OUT
        Action      : Translate Source
        Old Address : 192.168.20.8  00001
        New Address : 87.87.87.87 00036
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d7c390 - IPV4_NAT_OUTPUT_FIA
        Lapsed time : 36800 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d85d30 - IPV4_OUTPUT_THREAT_DEFENSE
        Lapsed time : 3200 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e9c - IPV4_VFR_REFRAG
        Lapsed time : 1120 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Output
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 7
        cft_bucket_number     : 1591662
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 87.87.87.87
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 443
        tuple.dst_port        : 57521
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 7
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d8359c - IPV4_OUTPUT_STILE_CLR_TXT
        Lapsed time : 141920 ns
      Feature: IPSec
        Result    : IPSEC_RESULT_DENY
        Action    : SEND_CLEAR
        SA Handle : 0
        Peer Addr : 8.8.8.8
        Local Addr: 87.87.87.87
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d761ac - IPV4_OUTPUT_IPSEC_CLASSIFY
        Lapsed time : 46080 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e70 - IPV4_OUTPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 2560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81128eb8 - OUTPUT_DROP_EXT                  <=================
        Lapsed time : 3360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d4a144 - IPV4_OUTPUT_ACL                  <=================
        Lapsed time : 121760 ns

    In the traces at the very end, we find information about the fate of the package: OUTPUT_DROP_EXT and IPV4_OUTPUT_ACL. The packet almost escaped from the paws of the router, as evidenced by the passage of most stages of processing.

    Situation No. 3. Packet dropped by firewall

    cbs-4000#show platform packet-trace summary
    Pkt   Input             Output            State  Reason
    0     Gi0/0/0           Gi0/0/1.5         DROP   184 (FirewallPolicy)

    The packet is dropped (DROP). The reason is firewall policies (FirewallPolicy).

    Package Processing Trace
    cbs-4000#show platform packet-trace packet 0
    Packet: 0           CBUG ID: 36
    Summary
      Input     : GigabitEthernet0/0/0
      Output    : GigabitEthernet0/0/1.5
      State     : DROP 184 (FirewallPolicy)
      Timestamp
        Start   : 6516783739710881 ns (02/18/2017 17:59:16.560339 UTC)
        Stop    : 6516783739809427 ns (02/18/2017 17:59:16.560438 UTC)
    Path Trace
      Feature: IPV4
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Source      : 192.168.20.8
        Destination : 8.8.8.8
        Protocol    : 1 (ICMP)
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x8112bfbc - DEBUG_COND_INPUT_PKT
        Lapsed time : 8800 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e84 - IPV4_INPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 5440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e64 - IPV4_INPUT_DST_LOOKUP_CONSUME
        Lapsed time : 1600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4a140 - IPV4_INPUT_ACL
        Lapsed time : 47360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e88 - IPV4_INPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e68 - IPV4_INPUT_FOR_US_MARTIAN
        Lapsed time : 1440 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 135
        cft_bucket_number     : 875224
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 56789
        tuple.dst_port        : 514
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 135
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d83558 - IPV4_INPUT_STILE_LEGACY
        Lapsed time : 202560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7b508 - IPV4_INGRESS_MMA_LOOKUP
        Lapsed time : 63360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d59618 - IPV4_INPUT_FME_PROCESS
        Lapsed time : 4640 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000084
        input vrf_idx         : 0
        calling feature       : FNF
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 135
        cft_bucket_number     : 875224
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 56789
        tuple.dst_port        : 514
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 135
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6dc84 - IPV4_INPUT_FNF_AOR_FIRST
        Lapsed time : 20640 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6d9d4 - IPV4_INPUT_FNF_FIRST
        Lapsed time : 127360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e8c - IPV4_INPUT_VFR
        Lapsed time : 1440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b660 - IPV4_INPUT_CENT_SMP_PROCESS
        Lapsed time : 2720 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000080
        input vrf_idx         : 0
        calling feature       : CENT
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 135
        cft_bucket_number     : 875224
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 56789
        tuple.dst_port        : 514
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 135
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b62c - IPV4_INPUT_CENT_RC_PROCESS
        Lapsed time : 43840 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7ff70 - IPV4_INPUT_PBR
        Lapsed time : 37120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d858d0 - IPV4_INPUT_TCP_ADJUST_MSS
        Lapsed time : 1280 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS
        Lapsed time : 4800 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6dc88 - IPV4_INPUT_FNF_AOR_FINAL
        Lapsed time : 1760 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6d974 - IPV4_INPUT_FNF_FINAL
        Lapsed time : 255680 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6dc8c - IPV4_INPUT_FNF_AOR_RELEASE
        Lapsed time : 2240 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e94 - IPV4_INPUT_IPOPTIONS_PROCESS
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113ac44 - IPV4_INPUT_GOTO_OUTPUT_FEATURE
        Lapsed time : 4160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e98 - IPV4_OUTPUT_VFR
        Lapsed time : 1760 ns
      Feature: ZBFW                                           <=================
        Action  : Drop                                        <=================
        Reason  : ICMP policy drop:classify result            <=================                 
        Zone-pair name  : in-out1                             <=================
        Class-map name  : class-default                       <=================
        Input interface : GigabitEthernet0/0/0                <=================
        Egress interface: GigabitEthernet0/0/1.5              <=================
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81128eb8 - OUTPUT_DROP_EXT            <=================
        Lapsed time : 640 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d70b28 - IPV4_OUTPUT_INSPECT        <=================
        Lapsed time : 639200 ns

    The presence of the OUTPUT_DROP_EXT and IPV4_OUTPUT_INSPECT messages indicates that the packet has been discarded by the inspection policies, which are performed by ITU. Details are found in the information on ZFW:

    Feature: ZBFW
        Action  : Drop
        Reason  : ICMP policy drop:classify result
        Zone-pair name  : in-out1
        Class-map name  : class-default
        Input interface : GigabitEthernet0/0/0
        Egress interface: GigabitEthernet0/0/1.5

    Reason reports that the packet has been classified as ICMP. The class into which the package fell and where it was dropped is class-default.

    Situation No. 4. The packet is routed by PBR rules.

    cbs-4000#show platform packet-trace summary
    Pkt   Input             Output            State  Reason
    0     Gi0/0/0           Gi0/0/1.6         FWD

    The packet is transmitted (FWD). Now the outgoing interface is Gi0 / 0 / 1.6.

    Package Processing Trace
    cbs-4000#show platform packet-trace packet 0
    Packet: 0           CBUG ID: 36
    Summary
      Input     : GigabitEthernet0/0/0
      Output    : GigabitEthernet0/0/1.6
      State     : FWD 
      Timestamp
        Start   : 6517659109765260 ns (02/18/2017 18:13:51.930393 UTC)
        Stop    : 6517659109927732 ns (02/18/2017 18:13:51.930556 UTC)
    Path Trace
      Feature: IPV4
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Source      : 192.168.20.8
        Destination : 8.8.8.8
        Protocol    : 1 (ICMP)
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x8112bfbc - DEBUG_COND_INPUT_PKT
        Lapsed time : 10400 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e84 - IPV4_INPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 5440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e64 - IPV4_INPUT_DST_LOOKUP_CONSUME
        Lapsed time : 1600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4a140 - IPV4_INPUT_ACL
        Lapsed time : 265600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e88 - IPV4_INPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e68 - IPV4_INPUT_FOR_US_MARTIAN
        Lapsed time : 3680 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 69
        cft_bucket_number     : 2000178
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 57521
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 69
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d83558 - IPV4_INPUT_STILE_LEGACY
        Lapsed time : 223360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7b508 - IPV4_INGRESS_MMA_LOOKUP
        Lapsed time : 85440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d59618 - IPV4_INPUT_FME_PROCESS
        Lapsed time : 3040 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000084
        input vrf_idx         : 0
        calling feature       : FNF
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 69
        cft_bucket_number     : 2000178
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 57521
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 69
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6dc84 - IPV4_INPUT_FNF_AOR_FIRST
        Lapsed time : 19680 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6d9d4 - IPV4_INPUT_FNF_FIRST
        Lapsed time : 153600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e8c - IPV4_INPUT_VFR
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b660 - IPV4_INPUT_CENT_SMP_PROCESS
        Lapsed time : 2560 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000080
        input vrf_idx         : 0
        calling feature       : CENT
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 69
        cft_bucket_number     : 2000178
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 57521
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 69
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b62c - IPV4_INPUT_CENT_RC_PROCESS
        Lapsed time : 49600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7ff70 - IPV4_INPUT_PBR              <=================
        Lapsed time : 69760 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d858d0 - IPV4_INPUT_TCP_ADJUST_MSS
        Lapsed time : 1440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0                     <=================
        Output      : GigabitEthernet0/0/1.6                   <=================
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS
        Lapsed time : 7840 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d6dc88 - IPV4_INPUT_FNF_AOR_FINAL
        Lapsed time : 1600 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d6d974 - IPV4_INPUT_FNF_FINAL
        Lapsed time : 280480 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d6dc8c - IPV4_INPUT_FNF_AOR_RELEASE
        Lapsed time : 3840 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81131e94 - IPV4_INPUT_IPOPTIONS_PROCESS
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x8113ac44 - IPV4_INPUT_GOTO_OUTPUT_FEATURE
        Lapsed time : 3840 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81131e98 - IPV4_OUTPUT_VFR
        Lapsed time : 5440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d858a0 - IPV4_OUTPUT_TCP_ADJUST_MSS
        Lapsed time : 1280 ns
      Feature: ZBFW
        Action  : Fwd
        Zone-pair name  : in-out2
        Class-map name  : CM-FW_in-out
        Input interface : GigabitEthernet0/0/0
        Egress interface: GigabitEthernet0/0/1.6
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d70b28 - IPV4_OUTPUT_INSPECT
        Lapsed time : 789120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d77188 - MC_OUTPUT_GEN_RECYCLE
        Lapsed time : 11200 ns
      Feature: NAT
        Direction   : IN to OUT
        Action      : Translate Source
        Old Address : 192.168.20.8
        New Address : 62.62.62.62
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d7c390 - IPV4_NAT_OUTPUT_FIA
        Lapsed time : 38400 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d85d30 - IPV4_OUTPUT_THREAT_DEFENSE
        Lapsed time : 4000 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81131e9c - IPV4_VFR_REFRAG
        Lapsed time : 800 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Output
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 71
        cft_bucket_number     : 2000178
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 62.62.62.62
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 57521
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 71
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d8359c - IPV4_OUTPUT_STILE_CLR_TXT
        Lapsed time : 140160 ns
      Feature: IPSec
        Result    : IPSEC_RESULT_DENY
        Action    : SEND_CLEAR
        SA Handle : 0
        Peer Addr : 8.8.8.8
        Local Addr: 62.62.62.62
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d761ac - IPV4_OUTPUT_IPSEC_CLASSIFY
        Lapsed time : 66400 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81131e70 - IPV4_OUTPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 3840 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81128eb0 - IPV4_OUTPUT_L2_REWRITE
        Lapsed time : 13440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81131e74 - IPV4_OUTPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81131ec4 - IPV4_OUTPUT_FRAG
        Lapsed time : 2240 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x81133e50 - IPV4_OUTPUT_DROP_POLICY
        Lapsed time : 18720 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x80d6d914 - IPV4_OUTPUT_FNF_FINAL
        Lapsed time : 113440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1.6
        Entry       : 0x8113bb40 - MARMOT_SPA_D_TRANSMIT_PKT
        Lapsed time : 43680 ns

    If we compare packet tracing for routing with standard rules (static routing) and for routing with PBR rules, we will not see the difference. Only the outgoing interface will change, and the address substituted in NAT.

    Situation No. 5. The packet is transmitted through the VTI interface.

    In this example, we ping the address 172.28.0.1.

    cbs-4000#show platform packet-trace summary
    Pkt   Input             Output            State  Reason
    0     Gi0/0/0           Gi0/0/1.5         FWD

    The packet is transmitted (FWD). Outgoing interface Gi0 / 0 / 1.5.

    Package Processing Trace
    cbs-4000#show platform packet-trace packet 0
    Packet: 0           CBUG ID: 50
    Summary
      Input     : GigabitEthernet0/0/0
      Output    : GigabitEthernet0/0/1.5
      State     : FWD 
      Timestamp
        Start   : 6665377802839987 ns (02/20/2017 11:15:48.257340 UTC)
        Stop    : 6665377803172303 ns (02/20/2017 11:15:48.257673 UTC)
    Path Trace
      Feature: IPV4
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Source      : 192.168.20.8
        Destination : 172.28.0.1
        Protocol    : 1 (ICMP)
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x8112bfbc - DEBUG_COND_INPUT_PKT
        Lapsed time : 5600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e84 - IPV4_INPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 4160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e64 - IPV4_INPUT_DST_LOOKUP_CONSUME
        Lapsed time : 3040 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4a140 - IPV4_INPUT_ACL
        Lapsed time : 19840 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e88 - IPV4_INPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e68 - IPV4_INPUT_FOR_US_MARTIAN
        Lapsed time : 1280 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 186
        cft_bucket_number     : 407373
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 172.28.0.1
        tuple.src_port        : 6603
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 186
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d83558 - IPV4_INPUT_STILE_LEGACY
        Lapsed time : 296480 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7b508 - IPV4_INGRESS_MMA_LOOKUP
        Lapsed time : 43040 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d59618 - IPV4_INPUT_FME_PROCESS
        Lapsed time : 2560 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000084
        input vrf_idx         : 0
        calling feature       : FNF
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 186
        cft_bucket_number     : 407373
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 172.28.0.1
        tuple.src_port        : 6603
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 186
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6dc84 - IPV4_INPUT_FNF_AOR_FIRST
        Lapsed time : 20160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6d9d4 - IPV4_INPUT_FNF_FIRST
        Lapsed time : 134400 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e8c - IPV4_INPUT_VFR
        Lapsed time : 1120 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b660 - IPV4_INPUT_CENT_SMP_PROCESS
        Lapsed time : 3840 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000080
        input vrf_idx         : 0
        calling feature       : CENT
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 186
        cft_bucket_number     : 407373
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 172.28.0.1
        tuple.src_port        : 6603
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 186
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b62c - IPV4_INPUT_CENT_RC_PROCESS
        Lapsed time : 45440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7ff70 - IPV4_INPUT_PBR
        Lapsed time : 14080 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d858d0 - IPV4_INPUT_TCP_ADJUST_MSS
        Lapsed time : 1280 ns
      Feature: FIA_TRACE             
        Input       : GigabitEthernet0/0/0                     <=================
        Output      : Tunnel1                                  <=================
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS   <=================
        Lapsed time : 5920 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d6dc88 - IPV4_INPUT_FNF_AOR_FINAL
        Lapsed time : 1600 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: OCE_TRACE
        Type       : OCE_ADJ_IPV4
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d6d974 - IPV4_INPUT_FNF_FINAL
        Lapsed time : 245440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d6dc8c - IPV4_INPUT_FNF_AOR_RELEASE
        Lapsed time : 1760 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x81131e94 - IPV4_INPUT_IPOPTIONS_PROCESS
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x8113ac44 - IPV4_INPUT_GOTO_OUTPUT_FEATURE
        Lapsed time : 4160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x81131e98 - IPV4_OUTPUT_VFR
        Lapsed time : 3040 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d858a0 - IPV4_OUTPUT_TCP_ADJUST_MSS
        Lapsed time : 1280 ns
      Feature: ZBFW                                       <=================
        Action  : Fwd                                     <=================
        Zone-pair name  : N/A                             <=================
        Class-map name  : N/A                             <=================
        Input interface : GigabitEthernet0/0/0            <=================
        Egress interface: Tunnel1                         <=================
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d70b28 - IPV4_OUTPUT_INSPECT
        Lapsed time : 30080 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d77188 - MC_OUTPUT_GEN_RECYCLE
        Lapsed time : 2560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d85d30 - IPV4_OUTPUT_THREAT_DEFENSE
        Lapsed time : 1600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x81131e9c - IPV4_VFR_REFRAG
        Lapsed time : 800 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x81128eb0 - IPV4_OUTPUT_L2_REWRITE
        Lapsed time : 7360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x81131ec4 - IPV4_OUTPUT_FRAG
        Lapsed time : 640 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d6e1b8 - IPV4_TUNNEL_OUTPUT_FNF_AOR
        Lapsed time : 3520 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d6d8e4 - IPV4_TUNNEL_OUTPUT_FNF_FINAL
        Lapsed time : 1440 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x80d6e640 - IPV4_TUNNEL_OUTPUT_FNF_AOR_RELEASE
        Lapsed time : 800 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d86ce8 - IPV4_TUNNEL_OUTPUT_FINAL
        Lapsed time : 20640 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d86d30 - IPV4_OUTPUT_TUNNEL_PROTECTION_ENCRYPT <=================
        Lapsed time : 7200 ns
      Feature: IPSec                                     <=================
        Result    : IPSEC_RESULT_SA                      <=================
        Action    : ENCRYPT                              <=================
        SA Handle : 98                                   <=================
        Peer Addr : 188.188.188.188                      <=================
        Local Addr: 87.87.87.87                          <=================
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d761ac - IPV4_OUTPUT_IPSEC_CLASSIFY_EXT
        Lapsed time : 44480 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d7641c - IPV4_OUTPUT_IPSEC_DOUBLE_ACL_EXT
        Lapsed time : 11200 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d763ec - IPV4_IPSEC_FEATURE_RETURN_EXT
        Lapsed time : 4960 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x8113ac50 - IPV4_OUTPUT_IPSEC_INLINE_FRAG_CHK_EXT
        Lapsed time : 7680 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d7635c - IPV4_OUTPUT_IPSEC_TUNNEL_RERUN_JUMP_EXT
        Lapsed time : 4480 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d764ac - IPV4_OUTPUT_IPSEC_POST_PROCESS_EXT
        Lapsed time : 12160 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d763ec - IPV4_IPSEC_FEATURE_RETURN_EXT
        Lapsed time : 1600 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d763ec - IPV4_IPSEC_FEATURE_RETURN_EXT
        Lapsed time : 1440 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d86cec - IPV4_TUNNEL_GOTO_OUTPUT
        Lapsed time : 11680 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d86d98 - IPV4_TUNNEL_FW_CHECK_EXT
        Lapsed time : 15040 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x81131e60 - IPV4_INPUT_DST_LOOKUP_ISSUE_EXT
        Lapsed time : 8480 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x81131eb8 - IPV4_INPUT_ARL_EXT
        Lapsed time : 5760 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x81131e6c - IPV4_INTERNAL_DST_LOOKUP_CONSUME_EXT
        Lapsed time : 2880 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : Tunnel1
        Entry       : 0x80d86dc8 - IPV4_TUNNEL_ENCAP_FOR_US_EXT
        Lapsed time : 5600 ns
      Feature: FIA_TRACE                         
        Input       : Tunnel1                                    <=================
        Output      : GigabitEthernet0/0/1.5                     <=================
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS_EXT <=================
        Lapsed time : 4000 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131f20 - IPV4_TUNNEL_ENCAP_GOTO_OUTPUT_FEATURE_EXT
        Lapsed time : 11520 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e98 - IPV4_OUTPUT_VFR
        Lapsed time : 1440 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d70b28 - IPV4_OUTPUT_INSPECT
        Lapsed time : 5120 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d77188 - MC_OUTPUT_GEN_RECYCLE
        Lapsed time : 2240 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d7c390 - IPV4_NAT_OUTPUT_FIA
        Lapsed time : 6400 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d85d30 - IPV4_OUTPUT_THREAT_DEFENSE
        Lapsed time : 1440 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e9c - IPV4_VFR_REFRAG
        Lapsed time : 800 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Output
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01004104
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 186
        cft_bucket_number     : 407373
        cft_l3_payload_size   : 100
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 87.87.87.87
        tuple.dst_ip          : 188.188.188.188
        tuple.src_port        : 6603
        tuple.dst_port        : 443
        tuple.vrfid           : 0
        tuple.l4_protocol     : 50
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 186
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ipsec
        Classification ID: [CANA-L7:9]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d8359c - IPV4_OUTPUT_STILE_CLR_TXT
        Lapsed time : 138080 ns
      Feature: IPSec                                      <=================
        Result    : IPSEC_RESULT_DENY                     <=================
        Action    : SEND_CLEAR                            <=================
        SA Handle : 0
        Peer Addr : 188.188.188.188                       <=================
        Local Addr: 87.87.87.87                           <=================
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d761ac - IPV4_OUTPUT_IPSEC_CLASSIFY
        Lapsed time : 27840 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e70 - IPV4_OUTPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 2880 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81128eb0 - IPV4_OUTPUT_L2_REWRITE
        Lapsed time : 7520 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131e74 - IPV4_OUTPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81131ec4 - IPV4_OUTPUT_FRAG
        Lapsed time : 16800 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8111ea94 - L2_REWRITE_AFTER_FRAG_WITHOUT_CLIP_EXT
        Lapsed time : 11520 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x81133e50 - IPV4_OUTPUT_DROP_POLICY
        Lapsed time : 12000 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x80d6d914 - IPV4_OUTPUT_FNF_FINAL
        Lapsed time : 108320 ns
      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113bb40 - MARMOT_SPA_D_TRANSMIT_PKT
        Lapsed time : 49120 ns

    Traces have changed as packet routing has become more complicated. First, it is passed to the tunnel interface:

    Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : Tunnel1
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS
        Lapsed time : 5920 ns

    Next, firewall rules work. Since our incoming and tunneling interfaces are in the same zone, traffic verification does not occur (we don’t fall into any of the zone-pairs):

    Feature: ZBFW
        Action  : Fwd
        Zone-pair name  : N/A
        Class-map name  : N/A
        Input interface : GigabitEthernet0/0/0
        Egress interface: Tunnel1

    After the packet has entered the tunnel interface, it must be encrypted.

    IPV4_OUTPUT_TUNNEL_PROTECTION_ENCRYPT
      Feature: IPSec
        Result    : IPSEC_RESULT_SA
        Action    : ENCRYPT
        SA Handle : 98
        Peer Addr : 188.188.188.188
        Local Addr: 87.87.87.87

    Once again, routing of a packet that is already encrypted occurs.

      Feature: FIA_TRACE
        Input       : Tunnel1
        Output      : GigabitEthernet0/0/1.5
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS_EXT
        Lapsed time : 4000 ns

    The packet passes through the external interface where IPSec is configured (crypto-map hangs). Although the packet is already encrypted, the system checks to see if it gets into IPSec on the outgoing interface.

    Feature: IPSec
        Result    : IPSEC_RESULT_DENY
        Action    : SEND_CLEAR
        SA Handle : 0
        Peer Addr : 188.188.188.188
        Local Addr: 87.87.87.87

    Situation No. 6. The packet is sent to a non-existent next-hop (or failed)

    cbs-4000#show platform packet-trace summary
    Pkt   Input             Output            State  Reason
    0     Gi0/0/0           internal0/0/rp:0  PUNT   10  (Incomplete adjacency)

    The status PUNT means that the packet cannot be processed by CEF and is transferred to the processor for processing (process switching). The reason is that the router did not find the necessary entry in the adjacency table to transfer the packet to the next next-hop (Incomplete adjacency). Which is logical, since it is not there.

    Package Processing Trace
    cbs-4000#show platform packet-trace packet 0
    Packet: 0           CBUG ID: 55
    Summary
      Input     : GigabitEthernet0/0/0
      Output    : internal0/0/rp:0
      State     : PUNT 10  (Incomplete adjacency)
      Timestamp
        Start   : 6668916530895154 ns (02/20/2017 12:14:46.985396 UTC)
        Stop    : 6668916530979351 ns (02/20/2017 12:14:46.985480 UTC)
    Path Trace
      Feature: IPV4
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Source      : 192.168.20.8
        Destination : 8.8.8.8
        Protocol    : 1 (ICMP)
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x8112bfbc - DEBUG_COND_INPUT_PKT
        Lapsed time : 9760 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e84 - IPV4_INPUT_SRC_LOOKUP_ISSUE
        Lapsed time : 5920 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e64 - IPV4_INPUT_DST_LOOKUP_CONSUME
        Lapsed time : 3200 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4a140 - IPV4_INPUT_ACL
        Lapsed time : 15040 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e88 - IPV4_INPUT_SRC_LOOKUP_CONSUME
        Lapsed time : 960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e68 - IPV4_INPUT_FOR_US_MARTIAN
        Lapsed time : 1440 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x0000008c
        input vrf_idx         : 0
        calling feature       : STILE
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 74
        cft_bucket_number     : 769995
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 443
        tuple.dst_port        : 55391
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 74
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: NBAR
        Packet number in flow: N/A
        Classification state: Final
        Classification name: ping
        Classification ID: [CANA-L7:479]
        Number of matched sub-classifications: 0
        Number of extracted fields: 0
        Is PA (split) packet: False
        TPH-MQC bitmask value: 0x0
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d83558 - IPV4_INPUT_STILE_LEGACY
        Lapsed time : 252800 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7b508 - IPV4_INGRESS_MMA_LOOKUP
        Lapsed time : 48960 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d59618 - IPV4_INPUT_FME_PROCESS
        Lapsed time : 4000 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000084
        input vrf_idx         : 0
        calling feature       : FNF
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 74
        cft_bucket_number     : 769995
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.8
        tuple.src_port        : 443
        tuple.dst_port        : 55391
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 74
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6dc84 - IPV4_INPUT_FNF_AOR_FIRST
        Lapsed time : 20640 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d6d9d4 - IPV4_INPUT_FNF_FIRST
        Lapsed time : 127520 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x81131e8c - IPV4_INPUT_VFR
        Lapsed time : 1280 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b660 - IPV4_INPUT_CENT_SMP_PROCESS
        Lapsed time : 2560 ns
      Feature: CFT
        API                   : cft_handle_pkt
        packet capabilities   : 0x00000080
        input vrf_idx         : 0
        calling feature       : CENT
        direction             : Input
        triplet.vrf_idx       : 0
        triplet.network_start : 0x01003f8e
        triplet.triplet_flags : 0x00000000
        triplet.counter       : 74
        cft_bucket_number     : 769995
        cft_l3_payload_size   : 40
        cft_pkt_ind_flags     : 0x00000000
        cft_pkt_ind_valid     : 0x00000931
        tuple.src_ip          : 192.168.20.8
        tuple.dst_ip          : 8.8.8.7
        tuple.src_port        : 443
        tuple.dst_port        : 55391
        tuple.vrfid           : 0
        tuple.l4_protocol     : ICMP
        tuple.l3_protocol     : IPV4
        pkt_sb_state          : 0
        pkt_sb.num_flows      : 0
        pkt_sb.tuple_epoch    : 74
        returned cft_error    : 14
        returned fid          : 0x00000000
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d4b62c - IPV4_INPUT_CENT_RC_PROCESS
        Lapsed time : 39360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d7ff70 - IPV4_INPUT_PBR
        Lapsed time : 43680 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/0
        Entry       : 0x80d858d0 - IPV4_INPUT_TCP_ADJUST_MSS
        Lapsed time : 1120 ns
      Feature: FIA_TRACE                                          
        Input       : GigabitEthernet0/0/0                      <=================
        Output      : GigabitEthernet0/0/1                      <=================
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS    <=================
        Lapsed time : 135360 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0                       <=================
        Output      : internal0/0/rp:0                           <=================
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS_EXT <=================
        Lapsed time : 30240 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x80d6dc88 - IPV4_INPUT_FNF_AOR_FINAL_EXT
        Lapsed time : 8640 ns
      Feature: OCE_TRACE
        Type       : OCE_ADJ_PUNT
      Feature: OCE_TRACE
        Type       : OCE_ADJ_PUNT
      Feature: OCE_TRACE
        Type       : OCE_ADJ_PUNT
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x80d6d974 - IPV4_INPUT_FNF_FINAL_EXT
        Lapsed time : 277600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x80d6dc8c - IPV4_INPUT_FNF_AOR_RELEASE_EXT
        Lapsed time : 6720 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x81131e94 - IPV4_INPUT_IPOPTIONS_PROCESS_EXT
        Lapsed time : 2560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x8113ac44 - IPV4_INPUT_GOTO_OUTPUT_FEATURE_EXT
        Lapsed time : 11200 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x81131ef4 - IPV4_INTERNAL_ARL_SANITY_EXT
        Lapsed time : 10560 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x80d70b28 - IPV4_OUTPUT_INSPECT_EXT
        Lapsed time : 12160 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x80d85d30 - IPV4_OUTPUT_THREAT_DEFENSE_EXT
        Lapsed time : 1600 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x81131e9c - IPV4_VFR_REFRAG_EXT
        Lapsed time : 2240 ns
      Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x81133e50 - IPV4_OUTPUT_DROP_POLICY_EXT
        Lapsed time : 24320 ns
      Feature: FIA_TRACE          
        Input       : GigabitEthernet0/0/0                   <=================
        Output      : internal0/0/rp:0                       <=================
        Entry       : 0x8112ce90 - INTERNAL_TRANSMIT_PKT_EXT <=================
        Lapsed time : 137440 ns

    An outgoing interface is defined for the package:

    Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : GigabitEthernet0/0/1
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS
        Lapsed time : 135360 ns

    But since there are no necessary entries in CEF, it is sent for processing by the processor (internal0 / 0 / rp: 0):

    Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x8113ac40 - IPV4_INPUT_LOOKUP_PROCESS_EXT
        Lapsed time : 30240 ns

    Record of the fact that the packet was transferred to the processor (INTERNAL_TRANSMIT):

    Feature: FIA_TRACE
        Input       : GigabitEthernet0/0/0
        Output      : internal0/0/rp:0
        Entry       : 0x8112ce90 - INTERNAL_TRANSMIT_PKT_EXT
        Lapsed time : 137440 ns

    Packet Trace provides us with package processing data in QFP. This means that as soon as the package is at the disposal of the CPU, our traces will no longer help. In this case, you can try using debug ip packet. But with this debugger you need to be very careful.

    Conclusion

    These examples clearly demonstrate that the IOS XE Packet Trace in many situations will allow us to quickly understand where it is sugared. Further, owning such information, you can already deal with the problem in more detail, juggling with various variations of the show and debug commands.

    When diagnosing, do not forget about another tool - packet capture. On iOS XE, this functionality has been made more convenient compared to conventional iOS.

    Packet capture
    Activate packet capture:
    monitor capture CAP access-list 199
    monitor capture CAP interface GigabitEthernet0/0/0 in
    monitor capture CAP start
    Shutdown, dump dump to an external PC, deactivation:
    monitor capture CAP stop
    monitor capture CAP export tftp://10.0.0.1/CAP.pcap
    no monitor capture CAP

    Also popular now: