Analyzing MX records

In a previous post I did an analysis of HTTP headers returned by Belgian websites. The list of websites was based on an old Alexa datafile and more or less reflected the most ‘popular’ Belgian websites. I now trimmed these domains to their top domain only (so www.site.be and alpha.site.be became site.be) and decided to check what type of MX records are defined for the different domains.

MX records are DNS records that specify a mail server that is responsible for accepting email messages on behalf of a recipient’s domain.

Note that this post contains graphs and tables supported by Google Chart, you’ll have to enable javascript for the graphs and tables.

Fault margin

Some domains define different MX records for their sub-domains. For example the domain site.be can have MX record mx.site.be and then extranet.site.be can have mx.google.com as MX record. Because I trimmed the domains to their top domain I query only for the MX record of site.be in the situation above.

Summary of the MX records analysis

• 93% of the domains return MX records
• 32% of the MX records resolve to an IP in Belgium
• 31% of the MX records resolve to an IP in the US
• 23% of the MX records point to Google
• The majority of the domains return 1 or 2 MX records
• Some domains return 7 to 10 MX records
• Some domains return an IP address instead of a hostname in the MX record
• Some domains need a quality check on their MX record (typos or ‘unusual’ data)

Querying for MX records

The data set contained 2925 unique domains. I wrote a small Python script that reads the domains from a file, queries for the MX record and writes out the domain, MX preference and MX handler in a CSV file. If no MX records are returned it writes NOREPLY.

Running the script gave 5874 entries with 5660 valid MX records (not NOREPLY entries).

#!/usr/bin/env python

import sys
import dns.resolver

TARGETLIST="targetlist_domains_full"
OUTPUTLIST="mx_domains.csv"
def main():
  try:
    file = open( TARGETLIST, 'r')
    file_csv = open( OUTPUTLIST, 'w')
    lines = file.readlines()
    for line in lines:
        if line:
            try:
                answers = dns.resolver.query( line.strip(), 'MX')
                for rdata in answers:
                    file_csv.write( '%s,%s,%s\n' % (line.strip(), rdata.preference, rdata.exchange))
            except:
                file_csv.write( '%s,%s,%s\n' % (line.strip(), 0, 'NOREPLY'))
                continue
  except IOError: 
    print "Unable to read %s " % TARGETLIST
    sys.exit(1)
  except:
    print "Unexpected error:", sys.exc_info()[0]

'''
 Jump to the main function
'''
if __name__ == "__main__":
  main()    

I then imported this CSV file in a sqlite database with Valentina Studio. This last step could also have been part of the Python script.

Statistics on MX records

• 2925 unique domains were queried
• 214 of these domains did not return a MX record
• the script output included 5874 entries, including the entry indicating there were no MX records
• 5660 MX records were retrieved
• Tests run between 8-Sep-2014 and 11-Sep-2014

Not every domain provides an MX record

Out of the 2925 domains there are 214 domains that did not return an MX record. This means that 93% of the domains returned one or more valid MX records.

Note that for all other statistics below I omitted the results for domains that did not return a valid MX record.

Three domains returned an IP address as MX record. Two out of these three domains had 2 MX records, one pointing to a hostname, the other to an IP address. The hostname in the MX record did not resolve to the MX-record IP address.

How many MX records are returned?

The number of MX records returned varied between 1 and 10. Only one domain returned 10 MX records : fgov.be.

At the time of my test, the MX records for fgov.be contained some unusual entries. Notice the lines with \032 in the output below. The queries are done against the Google public nameserver 8.8.8.8 but similar results were obtained via OpenDNS (208.67.222.222) and dns1w.fgov.be. No other domain returned something similar in the MX reply.

; <<>> DiG 9.7.0-P1 <<>> @8.8.8.8 -t mx fgov.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 19617
;; flags: qr rd ra; QUERY: 1, ANSWER: 10, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;fgov.be.			IN	MX

;; ANSWER SECTION:
fgov.be.		21599	IN	MX	30 mail.ibz.fed.be.
fgov.be.		21599	IN	MX	30 mail.dofi.fgov.be.
fgov.be.		21599	IN	MX	30 mail.ibz.fgov.be\032.
fgov.be.		21599	IN	MX	30 mail.mibz.fgov.be\032.
fgov.be.		21599	IN	MX	30 mail2.mibz.fed.be.
fgov.be.		21599	IN	MX	30 mail2.ibz.fed.be.
fgov.be.		21599	IN	MX	30 mail2.dofi.fgov.be\032.
fgov.be.		21599	IN	MX	30 mail2.mibz.fgov.be.
fgov.be.		21599	IN	MX	30 mail2.ibz.fgov.be.
fgov.be.		21599	IN	MX	30 mail.mibz.fed.be.

;; Query time: 401 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Sep  9 20:58:19 2014
;; MSG SIZE  rcvd: 289

The majority of the domains return 1 or 2 MX records.

MX Preference value

The preference or distance number defines in which order different mail servers for a single domain have to be tried.

Most MX records have a preference of 10. Some domain administrator use uncommon values as 999, 33, 65535. There are 277 records with preference 0.

Where are the MX records?

The top 20 of MX records is mostly populated with servers pointing to Google and OVH servers.

Administrators make typos, the MX record for anysurfer.be points to ALT1.ASPMX.L.GOOGLE.COMA.. There is no “google.coma” domain.

; <<>> DiG 9.7.0-P1 <<>> @8.8.8.8 -t mx anysurfer.be
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48771
;; flags: qr rd ra; QUERY: 1, ANSWER: 7, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;anysurfer.be.			IN	MX

;; ANSWER SECTION:
anysurfer.be.		3599	IN	MX	10 ASPMX5.GOOGLEMAIL.COM.
anysurfer.be.		3599	IN	MX	10 ASPMX4.GOOGLEMAIL.COM.
anysurfer.be.		3599	IN	MX	10 ASPMX2.GOOGLEMAIL.COM.
anysurfer.be.		3599	IN	MX	5 ALT1.ASPMX.L.GOOGLE.COMA.
anysurfer.be.		3599	IN	MX	10 ASPMX3.GOOGLEMAIL.COM.
anysurfer.be.		3599	IN	MX	5 ALT2.ASPMX.L.GOOGLE.COM.
anysurfer.be.		3599	IN	MX	1 ASPMX.L.GOOGLE.COM.

;; Query time: 401 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Sep 11 17:17:23 2014
;; MSG SIZE  rcvd: 228

There are 1329 MX records that point to a *.google.com or *.googlemail.com server, 137 records pointing to an *.outlook.com. server and 80 to a Postini (*.psmtp.com) mailserver. This means that 23% of the MX records are hosted at Google. The reason why the outlook.com domains do not appear in the table is because these domains have a hostname notation similar to customdomain.*.outlook.com.

If we look at the network block owner then again the majority of the MX records are hosted at Google.

Note that the top domain data (above) counts for 1329 records pointing to google.com and googlemail.com. The network block whois data (according to Team Cymru) however only accounts for 1327 records. I could not easily figure out what caused this difference of two hosts.

Geo-location of MX records

Most of the MX records are hosted on IPs in Belgium (32%) and the United States (31%) but a lot of the MX records are also hosted on IPs in Holland, United Kingdom and France. The geo-location was done via the whois service of Team Cymru.

Not every MX record resolved to an IP address. The Team Cymru whois service returned 5606 valid replies (IPs mapped to a network block / country).

Note that three MX records resolved to 127.0.0.1. There are 5 MX records resolving to an IP in Ukraine and 1 MX record resolving to an IP in Russia.