Recon-ng
What is recon-ng?
recon-ng is a tool for open source reconnaissance. Reconnaissance is the first phase in a penetration test and it is the act of gathering preliminary data or intelligence on your target.
Recon-ng has a look and feel similar to the Metasploit Framework and provides an easy to use interface to gather open source intelligence.
This is a post on doing open source intel with recon-ng. The post is split in two parts :
- the first part focuses on gathering user information;
- the second part focuses on gathering domain and host information.
Installation
The installation of recon-ng is very easy on Ubuntu Linux.
git clone https://LaNMaSteR53@bitbucket.org/LaNMaSteR53/recon-ng.git cd recon-ng sudo pip install -r REQUIREMENTS
This will install the latest version of recon-ng. You can then start it with
./recon-ng
How do you use recon-ng?
Open source intel with recon-ng
The best way to demonstrate recon-ng is via a use-case. In this example I will gather as much open source information as possible starting with my company domain name (c[u]de[s]o.be).
Recon-ng is highly database-driven. This means that all the operations are done starting with the information that is already available in the database.
But if you start with an empty database you need to inject a keyword somewhere to get recon-ng started …
Start with a workspace and one domain
I first start with a new workspace. This is not entirely necessary but it keeps the results cleanly contained in one single container. With the use of workspaces you can run multiple different recon operations without having the results getting mixed up with each other.
[recon-ng][default] > workspaces add c[u]de[s]o [recon-ng][c[u]de[s]o] >
My starting point is a domain so I have to add this information manually to the database.
[recon-ng][c[u]de[s]o] > add domains c[u]de[s]o.be [recon-ng][c[u]de[s]o] > show domains +----------------------------------+ | rowid | domain | module | +----------------------------------+ | 1 | c[u]de[s]o.be | user_defined | +----------------------------------+ [*] 1 rows returned
From domains to contacts
I now want to search for information starting with only domain information. Recon-ng has an easy way to get all the modules that can work further on domain information.
[recon-ng][c[u]de[s]o] > search domains-
[*] Searching for 'domains-'...
Recon
-----
recon/domains-contacts/metacrawler
recon/domains-contacts/pgp_search
recon/domains-contacts/salesmaple
recon/domains-contacts/whois_pocs
recon/domains-credentials/pwnedlist/account_creds
By using SEARCH domains- I get everything that adds information starting from a domain. You can also use the search feature the other way around with SEARCH -domains. This will list every module that results in domain information.
I now use the PGP search module to get contact information.
[recon-ng][c[u]de[s]o] > use recon/domains-contacts/pgp_search [recon-ng][c[u]de[s]o][pgp_search] >
Similar to the Metasploit Framework you can get the list of options with the SET command
[recon-ng][c[u]de[s]o][pgp_search] > set Sets module options Usage: set <option> <value> Name Current Value Required Description ------ ------------- -------- ----------- SOURCE default yes source of input (see 'show info' for details)
Source option
This module has only one option, the SOURCE option. This option is something that you’ll also see in the other modules.
Remember that earlier I mentioned that recon-ng is highly database-driven. This option allows you to influence this database-driven behavior. Instead of using the database information as a starting point you can provide your own information as a starting point.
In an earlier step I added a domain manually. I could have skipped that step and used the PGP module directly, feeding it the domain manually as a source. By doing it that way however I would have lost the conceptual relationship “domain -> contacts”.
It is easy to check what information a module needs for a starting point with SHOW INFO
[recon-ng][c[u]de[s]o][pgp_search] > show info
Name: PGP Key Owner Lookup
Path: modules/recon/domains-contacts/pgp_search.py
Author: Robert Frost (@frosty_1313, frosty[at]unluckyfrosty.net)
Description:
Searches the MIT public PGP key server for email addresses of the given domain. Updates the
'contacts' table with the results.
Options:
Name Current Value Required Description
------ ------------- -------- -----------
SOURCE default yes source of input (see 'show info' for details)
Source Options:
default SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL
<string> string representing a single input
<path> path to a file containing a list of inputs
query <sql> database query returning one column of inputs
Comments:
* Inspiration from theHarvester.py by Christan Martorella: cmarorella[at]edge-seecurity.com
The information that is needed to start the crawling of the module is set in the default query string. In this case it is “SELECT DISTINCT domain FROM domains WHERE domain IS NOT NULL” meaning that it needs a domain to start with. You can change the default (database) behavior to using a string as input or also a file. The latter is very useful if you have multiple starting sources for a module (in this case for example imagine a case where you need to conduct a recon for a customer provided domain list).
Run the module and view the results
Running the module is easy with RUN
[recon-ng][c[u]de[s]o][pgp_search] > run --------- c[u]de[s]o.BE --------- [*] Koen Van Impe (koen.van[i]m[p]e@c[u]de[s]o.be) [*] c[u]de[s]o sales (sales@c[u]de[s]o.be) ------- SUMMARY ------- [*] 2 total (2 new) contacts found.
The results will immediately be added to the database. You can have a look at the current content of the database with SHOW DASHBOARD
[recon-ng][c[u]de[s]o][pgp_search] > show dashboard +------------------------------------------+ | Activity Summary | +------------------------------------------+ | Module | Runs | +------------------------------------------+ | recon/domains-contacts/pgp_search | 1 | +------------------------------------------+ +----------------------------+ | Results Summary | +----------------------------+ | Category | Quantity | +----------------------------+ | Domains | 1 | | Companies | 0 | | Netblocks | 0 | | Locations | 0 | | Vulnerabilities | 0 | | Ports | 0 | | Hosts | 0 | | Contacts | 2 | | Credentials | 0 | | Leaks | 0 | | Pushpins | 0 | | Profiles | 0 | | Repositories | 0 | +----------------------------+
This shows I have information on one domain and two contacts. What if I have forgotten the contact details? You can display them (along with information from the other categories) via SHOW CONTACTS
[recon-ng][c[u]de[s]o][pgp_search] > show contacts +-----------------------------------------------------------------------------------------------------------------------------+ | rowid | first_name | middle_name | last_name | email | title | region | country | module | +-----------------------------------------------------------------------------------------------------------------------------+ | 1 | Koen | Van | Impe | koen.van[i]m[p]e@c[u]de[s]o.be | PGP key association | | | pgp_search | | 2 | c[u]de[s]o | | sales | sales@c[u]de[s]o.be | PGP key association | | | pgp_search | +-----------------------------------------------------------------------------------------------------------------------------+ [*] 2 rows returned
Expanding contact information
I’d now like to expand the contact information. What is available with SEARCH CONTACTS-?
[recon-ng][c[u]de[s]o][hibp_paste] > search contacts-
[*] Searching for 'contacts-'...
Recon
-----
recon/contacts-contacts/mailtester
recon/contacts-contacts/mangle
recon/contacts-contacts/unmangle
recon/contacts-credentials/hibp_breach
recon/contacts-credentials/hibp_paste
recon/contacts-credentials/pwnedlist
recon/contacts-domains/migrate_contacts
recon/contacts-profiles/fullcontact
The hibp_ modules can get me useful credential information from (previous) account breaches that is made available via https://haveibeenpwned.com/. First I want to have an overview of what breaches contain useful pointers for my search
[recon-ng][c[u]de[s]o][hibp_breach] > use recon/contacts-credentials/hibp_breach [recon-ng][c[u]de[s]o][hibp_breach] > run [*] koen.van[i]m[p]e@c[u]de[s]o.be => Breach found! Seen in the Adobe breach that occurred on 2013-10-04. [*] sales@c[u]de[s]o.be => Not Found. ------- SUMMARY ------- [*] 1 total (1 new) credentials found. [*] 1 total (0 new) contacts found.
This search returns one breach (the Adobe hack) that has useful account information.
I now search for the pasties where this information was stored. These pasties sometimes can hold extra useful information. By default hibp_paste will attempt to download the pastie.
[recon-ng][c[u]de[s]o][hibp_paste] > set Sets module options Usage: set <option> <value> Name Current Value Required Description -------- ------------- -------- ----------- DOWNLOAD True yes download pastes SOURCE default yes source of input (see 'show info' for details)
You can disable this with the option SET DOWNLOAD False.
[recon-ng][c[u]de[s]o][hibp_breach] > use recon/contacts-credentials/hibp_paste [recon-ng][c[u]de[s]o][hibp_paste] > run [*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-12-08T05:12:00Z (http://pastebin.com/raw.php?i=C4b1t5Db). [*] Paste could not be downloaded (http://pastebin.com/raw.php?i=C4b1t5Db). [*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-12-08T04:12:00Z (http://pastebin.com/raw.php?i=h6b4BRmt). [*] Paste could not be downloaded (http://pastebin.com/raw.php?i=h6b4BRmt). [*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-26T09:11:00Z (http://pastebin.com/raw.php?i=zc3vCANP). [*] Paste could not be downloaded (http://pastebin.com/raw.php?i=zc3vCANP). [*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T22:11:00Z (http://pastebin.com/raw.php?i=0tbNjD4h). [*] Paste could not be downloaded (http://pastebin.com/raw.php?i=0tbNjD4h). [*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T13:11:00Z (http://pastebin.com/raw.php?i=Bi7mv9Kw). [*] Paste could not be downloaded (http://pastebin.com/raw.php?i=Bi7mv9Kw). [*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T11:11:00Z (http://pastebin.com/raw.php?i=j0kXGE6w). [*] Paste could not be downloaded (http://pastebin.com/raw.php?i=j0kXGE6w). [*] koen.van[i]m[p]e@c[u]de[s]o.be => Paste found! Seen in a Pastebin on 2014-11-25T01:11:00Z (http://pastebin.com/raw.php?i=c5a4bb1z). [*] Paste could not be downloaded (http://pastebin.com/raw.php?i=c5a4bb1z). [*] sales@c[u]de[s]o.be => Not Found. ------- SUMMARY ------- [*] 1 total (0 new) credentials found. [*] 1 total (0 new) contacts found.
The module has found a couple of pasties but unfortunately they have already been deleted.
Extending the credentials
What options are available to extend the credentials? I search for the modules that can use credentials as input and output with SEARCH CREDENTIALS:
[recon-ng][c[u]de[s]o] > search credentials
[*] Searching for 'credentials'...
Recon
-----
recon/contacts-credentials/hibp_breach
recon/contacts-credentials/hibp_paste
recon/contacts-credentials/pwnedlist
recon/credentials-credentials/adobe
recon/credentials-credentials/bozocrack
recon/credentials-credentials/hashes_org
recon/credentials-credentials/leakdb
...
For the purpose of this demo I assume a hash was found in a previous run. For this example I add a hash (that could for example have been found via the pasties-information) manually to a record. In order to do this I use the QUERY command
[recon-ng][c[u]de[s]o] > query update credentials set hash = '739c5b1cd5681e668f689aa66bcc254c' [recon-ng][c[u]de[s]o] > show credentials +----------------------------------------------------------------------------------------------------------+ | rowid | username | password | hash | type | leak | module | +----------------------------------------------------------------------------------------------------------+ | 1 | koen.van[i]m[p]e@c[u]de[s]o.be | | 739c5b1cd5681e668f689aa66bcc254c | | | hibp_breach | +----------------------------------------------------------------------------------------------------------+ [*] 1 rows returned
I’ll query https://hashes.org for matches on this hash. In order to do so I have to supply an API key.
Adding API keys for modules
Some modules will access public resources via an API and they require an API key. You have to add this API key to recon-ng with the command keys add.
[recon-ng][c[u]de[s]o] > use recon/credentials-credentials/hashes_org [recon-ng][c[u]de[s]o][hashes_org] > keys add hashes_api replace_with_my_key [*] Key 'hashes_api' added.
Now that an API key has been set we can use the module to extend the credential information.
[recon-ng][c[u]de[s]o][hashes_org] > run [*] 739c5b1cd5681e668f689aa66bcc254c (MD5X5PLAIN) => test [recon-ng][c[u]de[s]o][hashes_org] > show credentials +----------------------------------------------------------------------------------------------------------------+ | rowid | username | password | hash | type | leak | module | +----------------------------------------------------------------------------------------------------------------+ | 1 | koen.van[i]m[p]e@c[u]de[s]o.be | test | 739c5b1cd5681e668f689aa66bcc254c | MD5X5PLAIN | | hibp_breach | +----------------------------------------------------------------------------------------------------------------+ [*] 1 rows returned
The above command shows that a match with the hash has been found in the database of hashes.org. The matching password is automatically added to the credentials table.
Social media profiles
With the help of the module for Fullcontact I’m able to get an overview of other available social media profiles for the accounts that were previously found. Note that this module also requires an API key.
[recon-ng][c[u]de[s]o][linkedin] > use recon/contacts-profiles/fullcontact [recon-ng][c[u]de[s]o][fullcontact] > run [!] FrameworkException: API key 'fullcontact_api' not found. Add API keys with the 'keys add' command. [recon-ng][c[u]de[s]o][fullcontact] > keys add delete list [recon-ng][c[u]de[s]o][fullcontact] > keys add fullcontact_api myfullcontact_api [*] Key 'fullcontact_api' added. [recon-ng][c[u]de[s]o][fullcontact] > run [*] Koen Van Impe - koen.van[i]m[p]e@c[u]de[s]o.be [*] Brugge [*] 812988488 - Facebook (https://www.facebook.com/812988488) [*] 1100812 - Foursquare (https://foursquare.com/user/1100812) [*] c[u]de[s]o - Flickr (https://www.flickr.com/people/c[u]de[s]o) [*] c[u]de[s]o - Gravatar (https://gravatar.com/c[u]de[s]o) [*] c[u]de[s]o - Twitter (https://twitter.com/c[u]de[s]o) [*] Confidence: 89% [*] sales@c[u]de[s]o.be - Searched within last 24 hours. No results found for this Id. ------- SUMMARY ------- [*] 5 total (5 new) profiles found. [*] 1 total (1 new) contacts found.
You can extend the profiling information for the different user accounts with other modules. Just do a search for everything that extends profiles with SEARCH profiles.
[recon-ng][c[u]de[s]o][profiler] > search profiles
[*] Searching for 'profiles'...
Recon
-----
recon/companies-profiles/bing_linkedin
recon/contacts-profiles/fullcontact
recon/profiles-contacts/dev_diver
recon/profiles-contacts/linkedin
recon/profiles-profiles/linkedin_crawl
recon/profiles-profiles/namechk
recon/profiles-profiles/profiler
recon/profiles-profiles/twitter
recon/profiles-repositories/github_repos
End of part 1
This is the first part of a post on doing open source intel with recon-ng. This post focused on gathering open source information for user accounts. The second part on recon-ng focuses on gathering domain and host information.
