Taking Notes During A Penetration Test

This piece is all about taking detailed notes during a penetration test, and how to organize those notes. As well as the notes you write yourself, you’ll need to organize the vast amount of output generated by tools. One way of keeping everything organized, is to use some software for that exact purpose. But, I want to look at creating some custom bash scripts instead.

Rather than store information in a database, I’m going to use good-old flat files organised into directories. For note taking, I’m going to use markdown. From the markdown, I’ll generate some HTML which will provide a summary of the information acquired from target hosts, and act as a methodology check list too.

Preparation

Before attempting to follow the steps outlined in this document, you should ensure you have a text file containing the IP addresses of targets — hosts on the network you’re assessing. I will use a file called targets.list in the examples that follow. I’m using my home network — my local area network — for demonstration purposes but if you’re worried about breaking something, or you do not have written permission to hack your LAN (from whomever owns it) then you should use a virtual environment running on your localhost.

My targets.list file is just a simple text file, sorted using sort -t . -k 3,3n -k 4,4n so that the IP addresses are in numerical order (I prefer it this way but it’s not essential):

192.168.1.65
192.168.1.66
192.168.1.67
192.168.1.69
192.168.1.70
192.168.1.139
192.168.1.254

I like to use a simple approach of flat files, organized into folders, with each folder representing a host. Within each host’s folder will be stored sub-folders for output from different tools. In each host’s folder there will be a number of text documents (including markdown documents) for different purposes too. A bash script can be used to compile this information into a web page, allowing you to easily view the information.

Making Directories With Bash

Everything related to the current penetration test will be stored under a folder I’m calling the test_root, but this can be named anything you want. You could name this folder based on the client, or the date on which you start the test. Each host in targets.list will have its own directory, named by its IP address, for example 192.168.1.65. Each host folder will contain further folders as shown in the following list. Initial Nmap scans, used for discovering hosts can be stored in a folder called nmap_discovery. The summary.md file will be generated by a bash script, as will summary.html. You will be able to view summary.html in your web browser, in order to give you an overview of your test progress.

  • test_root
    • /nmap_discovery
      • nmap_discovery.gnmap
    • /ip_address
      • /nmap
      • /screenshots
      • hostnames.list
      • notes.md
      • operating_system.txt
    • targets.list
    • summary.md
    • summary.html
    • style.css (optional)

To get started with creating this structure, first make sure you’re already in the directory you want to use for your test_root. Then, to create files and directories based on the previous list, use the following bash code. You could copy and paste (or type) this at the command prompt, or wait until later where I’ll show you how to put everything together into a more usable script file. The following code assumes you already have an nmap_discovery folder set up:

mkdir nmap_discovery
for target in `cat targets.list`; do 
    mkdir $target; 
    mkdir $target/nmap;
    mkdir $target/screenshots;
    touch $target/hostnames.list
    touch $target/notes.md;
    touch $target/operating_system.txt
    touch $target/software.txt
    touch $target/users.txt
done

The summary.md and summary.html files will be generated later, and style.css represents a stylesheet, if you have one, and want to use it (this is not essential, and can be added later). It is assumed your targets.list file is already in your test_root folder.

Running ls -l after executing the previous bash script displays the following on my system:

drwxr-xr-x 4 jim jim 4096 Jun  3 15:37 192.168.1.139
drwxr-xr-x 4 jim jim 4096 Jun  3 15:37 192.168.1.254
drwxr-xr-x 4 jim jim 4096 Jun  3 15:37 192.168.1.65
drwxr-xr-x 4 jim jim 4096 Jun  3 15:37 192.168.1.66
drwxr-xr-x 4 jim jim 4096 Jun  3 15:37 192.168.1.67
drwxr-xr-x 4 jim jim 4096 Jun  3 15:37 192.168.1.69
drwxr-xr-x 4 jim jim 4096 Jun  3 15:37 192.168.1.70
drwxr-xr-x 2 jim jim 4096 Jun  3 15:40 nmap_discovery
-rw-r--r-- 1 jim jim 1322 Jun  3 13:10 style.css
-rw-r--r-- 1 jim jim   93 Jun  3 10:13 targets.list

If any of the directories you’ve attempted to make already exist, then bash will not complete those commands, and instead display an error. But, it will carry on executing further commands regardless. The touch command will not overwrite files that already exist, which is good, because eventually those files will contain important information.

The results of ls will not display your IP address-based directory names in numerical order. If this bothers you then you could set up an alias that sorts the output of ls as shown previously.

Storing Information

Now that you have a basic structure for storing information, you should try using some tools to probe each target host, and save the output in the relevant location. I’ll go through some basic examples now, but you’ll certainly want to expand on these later. The key to storing information, is ensuring that it is stored in the correct place, so that it can be pulled out again later, and manipulated to extract meaningful information. Using meaningful names for files and folders, and being consistent with these names goes a long way.

Hostnames

A simple way to get the hostname for each host, based on the IP address, is to use host. The following loop does this and redirects the result of host to the hostname.txt file.

for target in `cat targets.list`; do 
    host $target | awk -F " " '{print $5}' > $target/hostname.txt; 
done

Running Further Scans

You could run Nmap scans inside a loop, saving results in multiple formats, to each host’s nmap sub-directory. Making each results file read-only afterwards, prevents you accidentally over-writing the scans later with new ones (which is a problem if you specify the same file names, but different scanning options).

for target in `cat targets.list`; do
    sudo nmap -Pn $target -oA ./$target/basic;
    sudo nmap chmod 400 ./$target/basic.*
done;

Making Your Own Notes

The notes.md file is for writing your own notes about each host, that don’t fit anywhere else. This file is meant to be read by you, and can be displayed in your summary document (which we’ll get on to soon) but it is not meant to be parsed by other tools and scripts. Take software names and version numbers for instance, which could be fed to Searchsploit. These should be stored in the software.txt file instead, with one entry per line.

Creating a Summary of Results

Here’s a bash script that combines information from the files you’ve set up so far. The result will be a load of text in markdown format, which you can redirect to a summary.md file.

#!/bin/bash

for target in `cat targets.list`; do

    echo \#\# $target;
    echo "* Hostname:" `cat $target/hostname.txt`;
    echo "* Operating System: " `cat $target/operating_system.txt`;
    echo;
    echo "[Open Local Directory in Web Browser](file://$target/)";
    echo;
    echo "HTTP: [http://$target/](http://$target/)";
    echo;
    echo "HTTPS: [https://$target/](https://$target/)";
    
    cat $target/notes.md;

    echo \#\#\# Basic Nmap Results;
    echo '```';
    cat $target/basic.nmap;
    echo '```';

done;

After creating your results.md file, convert this to an HTML file so that you can view it in your browser. You can do this with pandoc. If you have a stylesheet, you can specify it here using --css=style.css, where style.css is the path to, and filename of your stylesheet.

pandoc --metadata=title:"Penetration Test Summary" --standalone -f commonmark -o summary.html summary.md
2020-06-15