MISP

My first post on MISP described how to get MISP installed and get it up and running. This post describes how you can use MISP to your benefit to share threat information with your community.

Basic usage of MISP

The basic features of MISP are described in detail in the documentation at INSTALL/documentation.pdf. I’ll describe the steps needed to create an event and add some useful data.

Create an event in MISP

You can add an event under Event actions, Add event. You’ll have to enter a date, distribution, threat level, analysis and an event description.

The distribution setting defines if you want to share this event with connected servers or only with the local instance.

Once you’ve entered the basic details you can start adding the event details.

You can now add IOCs (IPs, hashes, comments, …) or attributes one-by-one (via 1), template based (via 2) or via free text (via 3).

When you add tags (via 4), for example indicating the TLP-code, you inform the receivers of the event how to process this information.

Once you are finished adding attributes you should publish (5) the event to make it available to your users.

If you want to add different attributes of the same kind then you should select Batch import. This allows you to enter a list of data, line-by-line, and have MISP process them separately. If you do not use Batch import then they will be considered as “one” piece of data. If you want to make the attributes available later for export you should check the option for Intrusion Detection System

MISP allows you to use templates to quickly enter event types that occur often. You can find a list of templates under Event actions, List Templates.

Type of event information

The type of information that you put into MISP is entirely up onto you and depends on your intended audience. One use case of MISP is using it for collecting open source threat intelligence and using the network indicators for a simple “block” or “inspect” list for your customers. You can read all the freely available analysis documents and add that data manually but that is going to be a slow and tedious task.

There’s a tool to automate parts of this process : IOC Parser.

Use IOC Parser to feed MISP

IOC Parser is a tool to extract indicators of compromise from security reports in PDF format. IOC Parser is available on Github.

git clone https://github.com/armbues/ioc-parser.git

The output of IOC Parser returns all the useful IOC information from a PDF in an easy to read format. The APT Notes, Various public documents, whitepapers and articles about APT campaigns contains a repository of open source documents containing useful IOCs. The default output of IOC Parser is like this

./ioc-parser.py pdfs/Regin_Hopscotch_Legspin.pdf 
pdfs/Regin_Hopscotch_Legspin.pdf  1 MD5 6c34031d7a5fc2b091b623981a8ae61c
pdfs/Regin_Hopscotch_Legspin.pdf  1 MD5 42eaf2ab25c9ead201f25ecbdc96fb60
pdfs/Regin_Hopscotch_Legspin.pdf  2 Filename  dllhost.exe
...

I’m only interested in the IP or host information. To extract this data I combine the output of IOC Parser with some bash commands.

./ioc-parser.py pdfs/OperationDoubleTap.pdf | grep IP | cut -f 4 | uniq
192.157.198.103
192.184.60.229
198.55.115.71
210.109.99.64
192.184.60.229
104.151.248.173

./ioc-parser.py pdfs/OperationDoubleTap.pdf | grep Host | cut -f 4 | uniq
join.playboysplus.com
www.playboysplus.com
securitywap.com
www.securitywap.com

I can then use this output to complete a (batch import) of attributes into MISP. Because I want to provide the list of IPs and hosts for a block or inspect list I have to enable the for Intrusion Detection System setting when adding the attributes. If you do not enable this, the data will not be shown in the export. Do not forget to Publish the event afterwards

The only thing you then have to add is meta information about where you found the attributes and their impact.

Exporting the data

MISP can export the IOC data in a number of formats. Have a look under Event actions, Export to see what formats are available.

Newer versions of MISP require that you send the API-key (see your user profile) in the authorization header. The small script below sends the correct http request and will download Suricata events.

#!/usr/bin/python

import urllib2

MISP_HOST="http://misp."
API_KEY=""
EXPORT_DATA="events/nids/suricata/download"
OUTPUT_FILE="misp-suricata"

URL="%s/%s" % (MISP_HOST, EXPORT_DATA)
request = urllib2.Request(URL)
f = open(OUTPUT_FILE,'w')
request.add_header('Authorization', API_KEY)
data = urllib2.urlopen(request).read()
f.write(data)
f.close()

Running this script results in a file that is usable by suricata.

PyMISP

CIRCL published a tool PyMISP on Github that allows you to interact with the MISP REST API. You can use PyMISP to further automate adding attributes to events.

Conclusion

MISP is a very flexible tool to gather threat intelligence from different sources.

My use case of MISP with IOC Parser is limited to feeding IDSs with a block list but that’s only a small subset of its capabilities.