Network device discovery

Published on March 07, 2018

Network device discovery

    Scanning a network with building a list of devices and their properties, such as a list of network interfaces, followed by data removal in monitoring systems, if you do not delve into what is happening, may seem like special, computer magic. How does it work - under the cut.


    Disclaimer
    The author does not have a profile education related to network administration, so there are probably inaccuracies and not all that is mentioned is mentioned.

    Detection


    To detect a device, i.e. determine if there is anything on the selected ip-address, you can apply the following methods:

    • ping scanning
      Oddly enough, this is the easiest and most common way.
    • Checking open TCP ports
      If ping is disabled on the device, you can try to establish a connection using a TCP port. In view of the fact that there are many ports and checking each takes a considerable amount of time, usually only the most common ones are checked, for example. 80 or 443 used for the device web interface.
    • Checking the operation of UDP services
      UDP protocol does not send confirmation of receipt of a request, and therefore, in general, scanning UDP ports is impossible. However, you can try polling services that listen on the UDP port and send a response to the request, for example. SNMP (port 161) or IPMI (port 623). If a response other than a timeout is received, then the device is detected.
    • ARP scanning
      In addition to the usual ICMP requests that use ping utilities, you can use faster arping for devices in the same broadcast L2 domain : broadcast ARP packets of the form “computer with IP address XXX, send your MAC address computer with the MAC address of the requestor, and if a response is received, the device is considered to be detected.
    • CDP / LLDP
      If the network uses LLDP (or an analogue of CDP), the devices can collect information about their neighbors that can be considered discovered. This data is available via SNMP.

      I note that information about neighbors, together with the results, traceroutecan serve as the basis for building a physical map of the network.
    • NetBIOS Scan
      The NetBIOS protocol can be used to search for Windows machines, for example, using a utility nbtscan.
    • DHCP logs (suggested by alexanster )
      DHCP server logs contain information about issuing IP addresses by MAC addresses. For recent log entries, we can assume that these devices are detected.
    • ARPWatch (suggested by alexanster , described by TaHKucT )
      Arpwatch monitors IP address - MAC address pairs, logging new, missing and changed pairs into the log. Devices that fall into the log can be considered detected.
    • Analysis of FDB tables of switches (suggested by mickvav )
      FDB tables (Forwarding DataBase) of managed switches contain data on switching MAC addresses of subscribers and devices, i.e. Compliance MAC address of the device - port of the switch.

      Data is available via SNMP and telnet, and can be used to build a physical map of the network.

    Data collection


    After the device is discovered, you can proceed to collect information about it.
    Using the ARP protocol, you can obtain the MAC address from ip, and from it the probable manufacturer (some equipment allows changing the address, so the method is not very reliable). Next, you can use the nmap utility , which scans open ports, checks its fingerprint database and makes an assumption about the operating system used, its version and device type.

    Getting device type and OS used with nmap
    nmap -O -v 192.168.0.1
    Starting Nmap 7.60 ( https://nmap.org ) at 2018-03-04 01:17 RTZ 2 (ceia)
    Initiating ARP Ping Scan at 01:17
    Scanning 192.168.0.1 [1 port]
    Completed ARP Ping Scan at 01:17, 0.70s elapsed (1 total hosts)
    Initiating Parallel DNS resolution of 1 host. at 01:17
    Completed Parallel DNS resolution of 1 host. at 01:17, 0.00s elapsed
    Initiating SYN Stealth Scan at 01:17
    Scanning 192.168.0.1 [1000 ports]
    Discovered open port 80/tcp on 192.168.0.1
    Discovered open port 49152/tcp on 192.168.0.1
    Discovered open port 1900/tcp on 192.168.0.1
    Completed SYN Stealth Scan at 01:17, 0.13s elapsed (1000 total ports)
    Initiating OS detection (try #1) against 192.168.0.1
    Retrying OS detection (try #2) against 192.168.0.1
    WARNING: OS didn't match until try #2
    Nmap scan report for 192.168.0.1
    Host is up (0.00s latency).
    Not shown: 997 closed ports
    PORT      STATE SERVICE
    80/tcp    open  http
    1900/tcp  open  upnp
    49152/tcp open  unknown
    MAC Address: A0:F3:C1:35:21:58 (Tp-link Technologies)
    Device type: WAP
    Running: Linux 2.4.X
    OS CPE: cpe:/o:linux:linux_kernel:2.4.36
    OS details: DD-WRT v24-sp1 (Linux 2.4.36)
    Network Distance: 1 hop

    To get more detailed information about the device, you will need one of the following methods:

    • SNMP
      protocol SNMP almost always supported routers and switches; available on Windows (the corresponding service is disabled by default); Linux requires the installation of the snmpd daemon. Apparently, the last third version is quite difficult to implement, so the previous version 2c is still relevant, although not recommended due to the lack of encryption during data transfer. The protocol runs on the 161 UDP port of the device.

      You can use the Net-SNMP Utility Package to work with SNMP.. To get, for example, a description of the device, you need to specify the protocol version, the read password (community read, by default public) and the address, in SNMP notation, called OID (object identifier) ​​and consisting of numbers and dots. All device addresses can be represented in the form of a tree, where addresses are sorted in lexicographic order. The protocol allows you to request the current value at the address, as well as the addresses following the current.

      Getting device description with snmpget
      snmpget -v 2c -c public 192.168.0.102 1.3.6.1.2.1.1.1.0
      SNMPv2-MIB::sysDescr.0 = STRING: Linux debian 3.16.0-4-586 #1 Debian 3.16.43-2+deb8u2 (2017-06-26) i686 

      The standard set of addresses is very limited and contains a description of the device, contacts, location and uptime. The remaining addresses depend on the device manufacturer and can be obtained by scanning, for example, with snmpwalk utility. Fortunately, Linux and Windows have typical addresses for network interfaces and CPU / memory load, so they only need to know (or be able to determine) the operating system they are using.

      Retrieving Linux disk descriptions using snmpwalk
      snmpwalk -v 2c -c public 192.168.0.102 1.3.6.1.4.1.2021.9.1.2
      UCD-SNMP-MIB::dskPath.1 = STRING: /
      UCD-SNMP-MIB::dskPath.2 = STRING: /var
      UCD-SNMP-MIB::dskPath.3 = STRING: /
      UCD-SNMP-MIB::dskPath.4 = STRING: /run
      UCD-SNMP-MIB::dskPath.5 = STRING: /dev/shm
      UCD-SNMP-MIB::dskPath.6 = STRING: /run/lock
      UCD-SNMP-MIB::dskPath.7 = STRING: /sys/fs/cgroup
      Getting a description of Windows drives with snmpwalk
      snmpwalk -v 2c -c public localhost 1.3.6.1.2.1.25.2.3.1.3
      HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: C:\ Label:  Serial Number a65ceb77
      HOST-RESOURCES-MIB::hrStorageDescr.2 = STRING: D:\ Label:  Serial Number ded9f83e
      HOST-RESOURCES-MIB::hrStorageDescr.3 = STRING: E:\ Label:  Serial Number 8e764a1
      HOST-RESOURCES-MIB::hrStorageDescr.4 = STRING: I:\
      HOST-RESOURCES-MIB::hrStorageDescr.5 = STRING: Virtual Memory
      HOST-RESOURCES-MIB::hrStorageDescr.6 = STRING: Physical Memory
    • WMI
      technology WMI - an enhanced and adapted to the Windows implementation of the standard the WBEM , which allows not only to remotely read the parameters, but also to manage the device. WMI runs on top of RPC (TCP port 135) and DCOM (on an arbitrary TCP port above 1024), is actively used in Power Shell scripts (formerly Windows Script Host).

      Data can be requested, of course, only from Windows machines.

      Retrieving RAM Information in PowerShell
      Get-WmiObject win32_OperatingSystem |%{"Total Physical Memory: {0}KB`nFree Physical Memory : {1}KB`nTotal Virtual Memory : {2}KB`nFree Virtual Memory  : {3}KB" -f $_.totalvisiblememorysize, $_.freephysicalmemory, $_.totalvirtualmemorysize, $_.freevirtualmemory}
      Total Physical Memory: 2882040KB
      Free Physical Memory : 612912KB
      Total Virtual Memory : 5762364KB
      Free Virtual Memory  : 1778140KB

      There is also a wmic console utility and its Linux port

      Getting a device manufacturer using wmic
      wmic /USER:admin /PASSWORD:mypassword /NODE:"192.168.0.100" computersystem get Manufacturer
      Manufacturer
      Gigabyte Technology Co., Ltd.
    • Monitoring system agent, e.g. Zabbix or Check_MK
      If possible, then a small program is installed on the device that runs in the background and collects data. The advantage of using agents is that the data is unified regardless of the equipment and operating system used by the device, and it is also possible to collect data available only locally (for example, the result of the console program).
    • SSH
      baseline data on SSH can be obtained from the Linux and iOS devices. For Windows and Android, you will need to install an SSH server.

      Getting CPU Information via SSH
      cat /proc/cpuinfo
      processor       : 0
      Processor       : AArch64 Processor rev 4 (aarch64)
      Hardware        : sun50iw1p1
      BogoMIPS        : 48.00
      Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
      CPU implementer : 0x41
      CPU architecture: 8
      CPU variant     : 0x0
      CPU part        : 0xd03
      CPU revision    : 4
      processor       : 1
      ...

    • Request data from a management service , such as a virtual machine hypervisor

    How it works on the example of Zabbix


    As you know, Zabbix can independently detect new devices on the network and automatically poll some of their parameters. This is called Low Level Discovery .

    Device discovery is defined by network discovery rules , which combine the detection methods listed above, determine whether the device is available and which template to apply to it (device description is usually examined). The template contains a list of properties that can be obtained from the device, as well as timer rules for detecting and creating new ones.

    In the case of SNMP, this rule may look something like this: iterate over all the children of the node 1.3.6.1.2.1.2.2.1.8(discovery rule) and for each element found (the number placed in{#SNMPINDEX}) add new metrics defined through prototypes of data elements, 1.3.6.1.2.1.2.2.1.10.{#SNMPINDEX}and 1.3.6.1.2.1.2.2.1.16.{#SNMPINDEX}if they are not already. More details here .

    In the case of an agent, the rule will look a little different: ask the agent for one of the discovery properties, for example system.cpu.discovery, get a list of processors in the form of json

    [
    	{"NUMBER": 0, "STATUS": "online"},
    	{"NUMBER": 1, "STATUS": "online"}
    ] 

    and for each element, add system.cpu.load[{#CPU.NUMBER}]if there is no such metric yet. It is worth noting that the Zabbix agent allows you to create your own items ( UserParameter) that can be requested, and therefore it is easy to implement, for example, detecting files and tracking their size in a given folder. More details here .

    Conclusion


    As you can see, everything is quite simple and there is no magic. If you still know any other methods of detecting devices or obtaining their properties, then please report them.