[This is a Guest Diary by Scott Jensen]

Using an ELK (elasticsearch logstash kibana) or Elastic Stack is a great way to get a high-level view of what is being seen with your DShield [1] honeypot.

For those who don’t already have a dedicated ELK stack or are just looking for a way to monitor your honeypot from a separate device (in my case, I didn’t want to put more holes in my internal firewall), follow this step-by-step guide to get up and running quickly.

First of we need to get logs from the honeypot configured to use an ELK beat, if you want a great guide on collecting all the logs see this diary [2] posted on the ISC page.

This is an ELK (Elasticsearch Logstash Kibana) stack that is setup to monitor logs from a DShield honeypot.

It is put together using the Getting started with the Elastic Stack and Docker-Compose [3] project with Linux Ubuntu live server 22.04 and the Install & Configure Filebeat on Raspberry Pi ARM64 to Parse DShield Sensor Logs [2] diary. The only change is to edit the filebeat.yml to send to the IP address of the device you intend to run Dshield-ELK [4] on.

Dshield-ELK assumes the following prior to starting:

• A running DShield Honeypot [1]
• Docker (tested on Ubuntu 20.04 LTS running Docker version 24.0.5, but this should work on any platform)
• Confirm the DShield sensor can reach the logstash port TCP 5044. This port will not be reachable until docker compose has been setup.

Steps

1. Setup filebeat per Install & Configure Filebeat on Raspberry Pi ARM64 to Parse DShield Sensor Logs [2] up to the Setup Logstash Collection & Parsing.
2. Install docker as follow: [3]

• $ sudo apt-get install ca-certificates curl gnupg
• $ sudo install -m 0755 -d /etc/apt/keyrings
• $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
• $ sudo chmod a+r /etc/apt/keyrings/docker.gpg
• $ echo "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
• $ sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

3. Clone Dshield-ELK[4] to the /home user directory.

• $ sudo apt-get install git
• $ git clone https://github.com/fkadriver/Dshield-ELK.git

4. Change to the DShield-ELK directory.

• $ cd ~/DShield-ELK

5. Change any environment variables in .env (hidden file). Any editor can be used (i.e. vi, vim, nano), but recommend using nano if you are not familiar with another editor

• $ nano .env (note the '.' at the front of .env)

Recommend changing at least:

• ELASTIC_PASSWORD is the password that is used for administrative access to both Elastic and Kibana
• KIBANA_PASSWORD is only used as the internal password for Kibana to communicate to Elastic

Optional

• STACK_VERSION is the version of elastic used in this stack. The build has been tested with 8.8.2 but any version should work
• DNS_SERVER Is a list of DNS Sever(s) used in the filters. The default setup in each filter is to use what ever DNS settings that are defined on the host. If you want to use specific DNS server(s) you need to change the DNS_SERVER variable and un-comment the # nameserver lines in the following files:
  • logstash-200-filter-cowrie.conf: line 115
  • logstash-202-filter-cowrie-sqlite.conf: line 284
  • logstash-300-filter-iptables.conf: line 63

Memory Limits are the most memory that docker will allocate for each of the ELK containers. Default to 1073741824 (1GB) but can be expanded if you have the resources

• ES_MEM_LIMIT: Elastic Memory
• KB_MEM_LIMIT: Kibana Memory
• LS_MEM_LIMIT: Logstash memory

6. Save the .env file to the same location

• If using nano, ^o and then ^x

 7. Once the project and any setting changes (if applicable) have been completed, it is time to bring up the stack. This will take a few minutes especially if you have not previously pulled the elastic images.

• $ sudo docker compose up -d (the -d is optional, but without it, when you close the prompt or stop the command the container will shut down). See Overview of docker compose CLI5 for more information about the docker compose cmd

8. Open a browser and confirm that the following ports are your host:

• 5601: Kibana
  • User: elastic
  • Password: ${ELASTIC_PASSWORD}
  • http://localhost:5601 (this can also be http://IP:5601 for remote access)
• 9200: Elastic
  • User: elastic
  • Password: ${ELASTIC_PASSWORD}
  • Since version 8, the connection to ES is secured with a self signed cert, so you will have to use https://localhost:9200. (You might be prompted to use an insecure or self-signed certificate. This is normal)

9. Logstash will also be running on port 5044

• 5044: Logstash
  • This is setup to receive any beats input, but only has filters and output for cowrie* logs from the diary [2].
  • Additional filters can be added to the logstash/pipeline directory.

10. Connect to Kibana on port 5601 (http://localhost:5601 ) using the user elastic and the password {ELASTIC_PASSWORD} from the .env file.

11. If everything worked, you should be able to open [Logs DShield Sensor] Overview dashboard

After you first login, import the DShield dashboard from this source and in Kibana goto:

• Management
• Stack Management
• Kibana
• Saved Objects
• select Import and import dshield_sensor_8.71.ndjson

Now you should see in Kibana, Data Views a mapping to the 3 cowrie* tables. Go back to Dashboard and open [Logs DShield Sensor] Overview to confirm that filebeat and logstash are parsing your data.

12. Here is a snip of the dashboard over the past 24 hours

13. After a reboot, to restart the DShield ELK stack do:

• $ cd ~/Dshield-ELK
• $ sudo docker compose start
• $ sudo docker compose ps

For any updates, check out Scott Jensen's github site.

[1] https://isc.sans.edu/tools/honeypot/
[2] https://isc.sans.edu/diary/Install+Configure+Filebeat+on+Raspberry+Pi+ARM64+to+Parse+DShield+Sensor+Logs/30056
[3] https://docs.docker.com/engine/install/ubuntu/
[4] https://github.com/fkadriver/Dshield-ELK
[5] https://docs.docker.com/compose/reference/
[6] https://docs.docker.com/engine/security/rootless/
[7] https://handlers.sans.edu/gbruneau/elk/DShield/dshield_sensor_8.71.ndjson

-----------
Guy Bruneau IPSS Inc.
My Handler Page
Twitter: GuyBruneau
gbruneau at isc dot sans dot edu