Using Open Source to Satisfy NIST SP 800-171 Requirements

As 2017 comes to a close, many government contractors are working toward the end-of-the-year deadline for compliance with the National Institute of Standards and Technology (NIST) Special Publication (SP) 800-171. NIST 800-171 is a document that specifies how information systems and policies need to be set up in order to protect Controlled Unclassified Information (CUI). This blog addresses section 3.3, “AUDIT AND ACCOUNTABILITY.” The main goal of this section is to require companies to monitor and audit unlawful, unauthorized, or inappropriate system activity.

At Kitware, we use Bro IDS, an open source C++ software system, to monitor network activity. Bro produces running logs of many kinds of network behavior data, including Secure Sockets Layer (SSL) connections, public key certificates, and Simple Mail Transfer Protocol (SMTP) connections. By analyzing these logs both in real time and after suspicious activity occurs, it is possible to perform certain forensics and send alerts in compliance with NIST requirements.

To help analyze Bro logs, we created a Python module called Bro Analysis Tools (BAT). This post describes how to deploy BAT on a corporate network. The first step is to install Bro.

Installing Bro

To install Bro on Ubuntu, you will need the following packages:

• CMake
• Make
• G++
• Flex and Bison
• libpcap-dev
• libssl-dev
• python-dev
• swig
• zlib1g-dev

To set up automated report emails, you will want to install Sendmail. You can also install GeoIP to include geolocation information in the reports for hosts with which you interact. On Ubuntu, the GeoIP package is called libgeoip-dev.

Once you have the above packages, you’re ready to build and install Bro. Start by cloning the Bro Git repository.

$ git clone –recursive git://git.bro.org/bro

Then, build the software using this classic command sequence.

$ cd bro
$ ./configure --prefix=/opt/bro
$ make
$ make install

Next, adjust your PATH.

$ export PATH=/opt/bro/bin:$PATH

Optionally, test if GeoIP is working.

$ bro -e “print lookup_location(8.8.8.8);”

You can now edit the Bro configuration file in /opt/bro/etc/node.cfg to ensure that the correct network interface is listed there.

Finally, launch the Bro service using the BroControl (broctl) program.

$ broctl
[BroControl] > deploy
[BroControl] > start
[BroControl] > exit

You can see the log files that Bro collects in /opt/bro/logs/current. These logs will roll over each day at midnight, so you can go back to perform a post-mortem analysis, in case you suspect something was awry on a previous day.

Installing BAT

BAT makes it easier to analyze Bro logs, as it has built-in log readers. These readers allow an analysis script to stream out the rows of a log file for processing. The readers can also be used to dump an entire log file and convert it to a Pandas data frame for use in statistical processing scripts or other analyses.

BAT is available in the Python Package Index, so installing it is as simple as this:

$ pip install bat

Analyzing Bro Logs

Using BAT, you can write Python scripts to look for oddities and report them to your system administration team via email. For instance, the SSL log may contain hosts that claim to be Google, yet they present Let’s Encrypt certificates. This warrants further analysis; while individuals may use Let’s Encrypt to authenticate their own servers, big players like Google don’t use it on their sites. In this instance, the below script opens the certificate verification log file, scans for certificates that come from the Let’s Encrypt issuer but claim to be Google, and reports them on the console:

import bat

reader = bat.bro_log_reader('/opt/bro/logs/current/x509.log')
for row in reader.readrows():
    issuer = row['certificate.issuer']
    if "Let's Encrypt" in issuer and row['certificate.subject'] == 'CN=google':
        print('<<< SUSPICIOUS CERTIFICATE FOUND >>>')
        print(row)

(You may think this alert filter is a bit too simple to bring Python into the mix, rather than something like Bro scripts — but this is just a simplified example for illustration purposes. We are working on much more complex scenarios involving big data processing and advanced machine learning techniques to detect complex and shifting behaviors as they develop. Stay tuned for future blog posts about those topics.)

To set up a daily email alert, you can use a job scheduler, such as cron on UNIX systems. The job scheduler runs the report script and pipes the results to a Sendmail script like the following:

from subprocess import Popen, PIPE
import sys

msg = MIMEText(sys.stdin.read())
msg["From"] = "bro@example.com"
msg["To"] = "you@example.com"
msg["Subject"] = "Nightly Report"

p = Popen(["/usr/sbin/sendmail", "-t", "-oi"], stdin=PIPE, universal_newlines=True)
p.communicate(msg.as_string())

You’ll receive a daily email, and your organization can respond accordingly.

Taking the Next Step

Not only are we setting up our Bro/BAT system in anticipation of meeting NIST deadlines, but we are forming plans to develop more sophisticated detection and reporting routines. As an example, we would like to use machine learning techniques to detect certain kinds of network events as they develop, such as denial-of-service attacks. Real-time reporting that uses machine learning to distinguish true attacks from other sorts of anomalies would allow us to secure and protect our networks more efficiently.

As we refine our approach to provide our system administration team with the best possible security reporting, we are interested to hear from you. Looking at 2018, what concerns do you have about your network?