Continuing on from the "Quick and dirty Python: masscan" diary, which implemented a simple port scanner in Python using masscan to detect web instances on TCP ports 80 or 443.  Masscan is perfectly good as a blunt instrument to quickly find open TCP ports across large address spaces, but for fine details it is better to use a scanner like nmap that, while much slower, is able to probe the port to get a better idea of what is running.

First lets backtrack.  Since the previous diary, I converted the masscan code to a function and created another function to parse the masscan results to return the list of IPs on which masscan detected open ports.  The current script scan_web.py script is:

#!/usr/local/bin/python3
import sys,getopt,argparse
import masscan
import pprint

def scan_masscan(ips):

   try:
      maso = masscan.PortScanner()
      maso.scan(ips, ports='80,443')
   except:
      print("Error:", sys.exc_info()[0])
      sys.exit(1)

   return(maso)

def parse_masscan_host_list(massout):

    #initialize
    host_list = list()

    # Build a list from the massscan output
    for host in massout.all_hosts:
       host_list.append(host)
    return(host_list)

def main():
   # read in the IP parameter 
   parser = argparse.ArgumentParser()
   parser.add_argument('IP', help="IP address or range")
   args=parser.parse_args()
   ip=args.IP

   maso=scan_masscan(ip)

   if int(maso.scanstats['uphosts']) > 0:
      host_list=parse_masscan_host_list(maso)
      pprint.pprint(host_list)

   else:
      print("No Masscan results")
      sys.exit(1)

if __name__ == "__main__":
    main()

Running the script results in a list of IPs where either 80 or 443 were detected open by masscan.

# ./scan_web.py 45.60.103.0,45.60.31.34,1.2.3.4
[2021-05-31 18:28:51,335] [DEBUG] [masscan.py 10 line] Scan parameters: "masscan -oX - 45.60.103.0,45.60.31.34,1.2.3.4 -p 80,443"
['45.60.103.0', '45.60.31.34']

Extending this script to pass the masscan output list to nmap is relatively easy as well.  As somebody pointed out on a comment to the last diary, there are a lot of Python nmap modules and they all provide differing functionality.  After messing with a few of them, as the comment stated, the libnmap module appears to be the most functional and easiest to use.  libnmap does not implement nmap functionality, it needs nmap already installed on the device and interfaces with that version.  I will not be going over nmap functionality in this diary. If you are not clear on the nmap command parameters you can find a quick tutorial in this older diary.

To implement the nmap scan will require two functions.  One to run the scan, and one to parse the results.

The scanning function:

def scan_nmap(ip_list):
    print("Starting nmap for: {0}".format(ip_list))
    nm = NmapProcess(ip_list, options="-Pn -n -A -sT -p80,443 -r --max-retries 2 --host-timeout 2h --open --reason")
    nrc = nm.run()
    if nrc != 0:
        print("nmap scan failed: {0}".format(nm.stderr))
        exit(0)
    try:
        nparse = NmapParser.parse(nm.stdout)
    except NmapParserExcetion as e:
        print("Exception raised while parsing scan: {0}".format(e.msg))

    return(nparse)

and the function to parse and output the scan result.  This example is almost verbatim from the libnmap documentation.

def print_nmap(nmap_report):
    print("Starting Nmap {0} ( http://nmap.org ) at {1}".format(
        nmap_report.version,
        nmap_report.started))

    for host in nmap_report.hosts:
        if len(host.hostnames):
            tmp_host = host.hostnames.pop()
        else:
            tmp_host = host.address

        print("Nmap scan report for {0} ({1})".format(
            tmp_host,
            host.address))
        print("Host is {0}.".format(host.status))
        print("  PORT     STATE         SERVICE")

        for serv in host.services:
            pserv = "{0:>5s}/{1:3s}  {2:12s}  {3}".format(
                    str(serv.port),
                    serv.protocol,
                    serv.state,
                    serv.service)
            if len(serv.banner):
                pserv += " ({0})".format(serv.banner)
            print(pserv)
    print(nmap_report.summary)

The output from the finished script is:

# ./scan_web.py 45.60.103.0,45.60.31.34,1.2.3.4
[2021-05-31 19:00:56,329] [DEBUG] [masscan.py 10 line] Scan parameters: "masscan -oX - 45.60.103.0,45.60.31.34,1.2.3.4 -p 80,443"
Starting nmap for: ['45.60.103.0', '45.60.31.34']
Starting Nmap 7.91 ( http://nmap.org ) at 1622487670
Nmap scan report for 45.60.103.0 (45.60.103.0)
Host is up.
  PORT     STATE         SERVICE
   80/tcp  open          http
  443/tcp  open          https
Nmap scan report for 45.60.31.34 (45.60.31.34)
Host is up.
  PORT     STATE         SERVICE
   80/tcp  open          http
  443/tcp  open          https
Nmap done at Mon May 31 19:01:49 2021; 2 IP addresses (2 hosts up) scanned in 40.03 seconds

In about 80 lines of python code. I have implemented a simple script that can quickly scan a large address space using the very quick masscan and then send the output to nmap to do detailed scanning of a single port. This script is the basic framework I use for dozens of scripts to scan an entire ASN looking for devices that may be at risk for the current vulnerability of the week.

The final version of the scan_web.py script is:

#!/usr/local/bin/python3
import sys,getopt,argparse
import masscan
from libnmap.process import NmapProcess
from libnmap.parser import NmapParser, NmapParserException
import pprint

def scan_masscan(ips):

   try:
      maso = masscan.PortScanner()
      maso.scan(ips, ports='80,443')
   except:
      print("Error:", sys.exc_info()[0])
      sys.exit(1)

   return(maso)

def parse_masscan_host_list(massout):

    #initialize
    host_list = list()

    # Build a list from the massscan output
    for host in massout.all_hosts:
       host_list.append(host)
    return(host_list)

def scan_nmap(ip_list):
    print("Starting nmap for: {0}".format(ip_list))
    nm = NmapProcess(ip_list, options="-Pn -n -A -sT -p80,443 -r --max-retries 2 --host-timeout 2h --open --reason")
    nrc = nm.run()
    if nrc != 0:
        print("nmap scan failed: {0}".format(nm.stderr))
        exit(0)
    try:
        nparse = NmapParser.parse(nm.stdout)
    except NmapParserExcetion as e:
        print("Exception raised while parsing scan: {0}".format(e.msg))
    pprint.pprint(nparse)
    return(nparse)

def print_nmap(nmap_report):
    print("Starting Nmap {0} ( http://nmap.org ) at {1}".format(
        nmap_report.version,
        nmap_report.started))

    for host in nmap_report.hosts:
        if len(host.hostnames):
            tmp_host = host.hostnames.pop()
        else:
            tmp_host = host.address

        print("Nmap scan report for {0} ({1})".format(
            tmp_host,
            host.address))
        print("Host is {0}.".format(host.status))
        print("  PORT     STATE         SERVICE")

        for serv in host.services:
            pserv = "{0:>5s}/{1:3s}  {2:12s}  {3}".format(
                    str(serv.port),
                    serv.protocol,
                    serv.state,
                    serv.service)
            if len(serv.banner):
                pserv += " ({0})".format(serv.banner)
            print(pserv)
    print(nmap_report.summary)

def main():
   # read in the IP parameter 
   parser = argparse.ArgumentParser()
   parser.add_argument('IP', help="IP address or range")
   args=parser.parse_args()
   ip=args.IP

   maso=scan_masscan(ip)

   if int(maso.scanstats['uphosts']) > 0:
      host_list=parse_masscan_host_list(maso)
      nreport = scan_nmap(host_list)
      print_nmap(nreport)
   else:
      print("No Masscan results")
      sys.exit(1)

if __name__ == "__main__":
    main()

Caveat1: Never scan an IP range you don't have permission to scan.  While port scanning is not illegal in most jurisdictions it is questionable ethically to scan things you don't own or have permission to scan.

Caveat2: I am not a professional Python programmer.  My scripting gets the job done that I need it to do.  I know there are many smart people out there who can write way better code than I can. 

-- Rick Wanner MSISE - rwanner at isc dot sans dot edu - Twitter:namedeplume (Protected)