tiny Web Url Scanner

For a new project I needed a tool that could scan a web server for the http status code of different URLs and have the results listed in a easily parseable result. The URLs are typical Linux resources (f.e. the password file, the hostname, services file, …) that could lead to disclosing sensitive system information.

There are already a number of tools that can achieve this but none really provided the output that I needed. Hence I wrote my own small script.

You can find it on Github : https://github.com/cudeso/tools/tree/master/twus

It’s a python script that needs one necessary parameter, the URL and two optional parameters. The URL can contain a port number if necessary. So both http://www.myhost.com/ and http://www.myhost.com:995/ are valid.

1usage: twus.py [-h] [-r RESOURCES] [-v] url
2 
3Tiny Web Url Scanner
4 
5positional arguments:
6  url                   The URL to scan (include http://)
7 
8optional arguments:
9  -h, --help            show this help message and exit
10  -r RESOURCES, --resources RESOURCES
11                        File containing the resources to check (default:
12                        twus.input)
13  -v, --verbose         Add verbose output (default: False)
1Start scanning
3 
4"Code", "URL", "Server", "Last Modified", "Content Type", "Cache Control"
5"404", "http://www.changethis.com/tmp", "", "", "", ""
6"200", "http://www.changethis.com/home", "Apache/2 (Linux)", "", "text/html; charset=UTF-8", ""
7"404", "http://www.changethis.com/etc/hostname", "", "", "", ""
8"404", "http://www.changethis.com/etc/motd", "", "", "", ""
9"404", "http://www.changethis.com/etc/issue", "", "", "", ""
10"404", "http://www.changethis.com/var/run/utmp", "", "", "", ""
11 
12Scan finished

The file containing the resources lists one resource per line. You don’t need to pay attention if it starts with / or not. If the resource starts with a / then it’s stripped. If the URL is not ended with a /, then there is a / added.

twus.input, the default resources file, contains mostly Linux interesting resources.

1/tmp
2/home
3/etc/hostname
4/etc/motd
5/etc/issue
6/etc/passwd
7/etc/syslog.conf
8/etc/mailname
9/etc/mail.rc

One thought on “tiny Web Url Scanner

  1. Pingback: Analyze the network traffic of a TV | Koen Van Impe - vanimpe.eu

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.