[Search] [pdf|bibtex] [Tracker] [WG] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02 03 04 05 06 07 08 rfc5782                            
Anti-Spam Research Group                                       J. Levine
Internet-Draft                                      Taughannock Networks
Intended status: Informational                               May 7, 2008
Expires: November 8, 2008


                     DNS Blacklists and Whitelists
                        draft-irtf-asrg-dnsbl-05

Status of this Memo

   By submitting this Internet-Draft, each author represents that any
   applicable patent or other IPR claims of which he or she is aware
   have been or will be disclosed, and any of which he or she becomes
   aware will be disclosed, in accordance with Section 6 of BCP 79.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as Internet-
   Drafts.

   Internet-Drafts are draft documents valid for a maximum of six months
   and may be updated, replaced, or obsoleted by other documents at any
   time.  It is inappropriate to use Internet-Drafts as reference
   material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

   This Internet-Draft will expire on November 8, 2008.

Abstract

   The rise of spam and other anti-social behavior on the Internet has
   led to the creation of shared blacklists and whitelists of IP
   addresses or domains.  The DNS has become a de-facto standard method
   of distributing these blacklists and whitelists.  This memo documents
   the structure and usage of DNS based blacklists and whitelists, and
   the protocol used to query them.

IRTF Notice

   This document is a product of the Anti-Spam Research Group (ASRG).
   Comments and discussion may be directed to the ASRG mailing list,
   asrg@irtf.org.



Levine                  Expires November 8, 2008                [Page 1]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


   This document is not an IETF Internet Standard.  It represents the
   consensus of the Anti-Spam Research Group of the Internet Research
   Task Force.  It may be considered for standardization by the IETF in
   the future.


Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Structure of an IP address DNSBL or DNSWL  . . . . . . . . . .  3
     2.1.  IP address DNSxL . . . . . . . . . . . . . . . . . . . . .  4
     2.2.  IP address DNSWL . . . . . . . . . . . . . . . . . . . . .  4
     2.3.  Combined IP address DNSxLs . . . . . . . . . . . . . . . .  5
     2.4.  DNSxL cache behavior . . . . . . . . . . . . . . . . . . .  6
     2.5.  Test and contact addresses . . . . . . . . . . . . . . . .  6
     2.6.  IPv6 DNSxLs  . . . . . . . . . . . . . . . . . . . . . . .  6
   3.  Domain name DNSxLs . . . . . . . . . . . . . . . . . . . . . .  6
   4.  Typical usage of DNSBLs and DNSWLs . . . . . . . . . . . . . .  7
   5.  Security Considerations  . . . . . . . . . . . . . . . . . . .  8
   6.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . .  9
   7.  Informative References . . . . . . . . . . . . . . . . . . . .  9
   Author's Address . . . . . . . . . . . . . . . . . . . . . . . . .  9
   Intellectual Property and Copyright Statements . . . . . . . . . . 10




























Levine                  Expires November 8, 2008                [Page 2]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


1.  Introduction

   In 1997, Dave Rand and Paul Vixie, well known Internet software
   engineers, started keeping a list of IP addresses that had sent them
   spam or engaged in other behavior that they found objectionable.
   Word of the list quickly spread, and they started distributing it as
   a BGP feed for people who wanted to block all traffic from listed IP
   addresses at their routers.  The list became known as the Real-time
   Blackhole List (RBL).

   Many network managers wanted to use the RBL to block unwanted e-mail,
   but weren't prepared to use a BGP feed.  Rand and Vixie created a
   DNS-based distribution scheme that quickly became more popular than
   the original BGP distribution.  Other people created other DNS-based
   blacklists either to compete with the RBL or to complement it by
   listing different categories of IP addresses.  Although some people
   refer to all DNS-based blacklists as ``RBLs'', the term properly is
   used for the MAPS RBL, the descendant of the original list.  (In the
   United States, the term RBL is a registered service mark of Trend
   Micro[3].)

   The conventional term is now DNS Blacklist or Blocklist, or DNSBL.
   Some people also publish DNS-based whitelists or DNSWLs.  Network
   managers typically use DNSBLs to block traffic and DNSWLs to
   preferentially accept traffic.  The structure of a DNSBL and DNSWL
   are the same, so in the subsequent discussion we use the abbreviation
   DNSxL to mean either.

   This document describes existing practice but does not define a
   standard of any sort.  It describes the structure, operation, and use
   of DNSBLs and DNSWLs but does not describe or recommend policies for
   adding or removing addresses to and from DNSBLs and DNSWLs, nor does
   it recommend policies for using them, nor does it take a position on
   whether the DNS is the best way to distribute such data.  We
   anticipate that management policies will be addressed in a companion
   document.


2.  Structure of an IP address DNSBL or DNSWL

   A DNSxL is a DNS zone containing resource records corresponding to
   hosts present in a blacklist or whitelist.  Hosts were originally
   encoded into DNSxL zones using a transformation of their IP
   addresses, but now host names are sometimes encoded as well.  Most
   DNSxLs still use IP addresses.






Levine                  Expires November 8, 2008                [Page 3]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


2.1.  IP address DNSxL

   An IP address DNSxL has a structure adapted from that of the rDNS.
   (The rDNS, reverse DNS, is the IN-ADDR.ARPA and IP6.ARPA domains used
   to map IP addresses to domain names.)  Each IP address listed in the
   DNSxL has a corresponding DNS entry created by reversing the order of
   the octets of the text representation of the IP address, and
   appending the domain name of the DNSxL.

   If, for example, the DNSxL is called bad.example.com, and the IP
   address to be listed is 192.0.2.99, the name of the DNS entry would
   be 99.2.0.192.bad.example.com.  Each entry in the DNSxL has an A
   record and often a TXT record.  The A record conventionally has the
   value 127.0.0.2, but may have other values as described below in
   Combined IP address DNSxLs.  The TXT record describes the reason that
   the IP is listed in the DNSxL, and is often used as the text of an
   SMTP error response when an SMTP client attempts to send mail to a
   server using the list as a DNSBL, or as explanatory text when the
   DNSBL is used in a scoring spam filter, for example:

   99.2.0.192.bad.example.com    A      127.0.0.2
   99.2.0.192.bad.example.com    TXT
            "Dynamic address, see http://bad.example.com?192.0.2.99"

   Some DNSxLs use the same TXT record for all entries, while others
   provide a different TXT record for each entry or range of entries
   that describes the reason that entry or range is listed.  The reason
   often includes the URL of a web page where more information is
   available.  Some client software only checks the A record, some only
   checks the TXT record, some checks both.

   If a range of addresses is listed in the DNSxL, the DNSxL contains a
   record (or a pair of A and TXT records) for every address in the
   DNSxL.  Conversely, if an IP address is not listed in the DNSxL,
   there is no record for the address.

2.2.  IP address DNSWL

   Since SMTP has no standard way for a server to advise a client why a
   request was accepted, TXT records in DNSWLs are not very useful.
   Some DNSWLs contain TXT records anyway to document the reasons that
   entries are present.

   It is possible and occasionally useful for a DNSxL to be used as a
   DNSBL in one context and a DNSWL in another.  For example, a DNSxL
   that lists the IP addresses assigned to dynamically assigned
   addresses on a particular network might be used as a DNSWL on that
   network's outgoing mail server or intranet web server, and used as a



Levine                  Expires November 8, 2008                [Page 4]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


   DNSBL for mail servers on other networks.

2.3.  Combined IP address DNSxLs

   In many cases, an organization maintains a DNSxL that contains
   multiple entry types, with the entries of each type constituting a
   sublist.  For example, an organization that publishes a DNSBL listing
   sources of unwanted e-mail may wish to indicate why various addresses
   are included in the list, with one sublist for addresses listed due
   to sender policy, a second list for addresses of open relays, a third
   list for hosts compromised by malware, and so forth.  (At this point
   all of the DNSxLs with sublists of which we are aware are intended
   for use as DNSBLs, but the sublist techniques are equally usable for
   DNSWLs.)

   There are three common methods of representing a DNSxL with multiple
   sublists: subdomains, multiple A records, and bit encoded entries.
   Most DNSxLs with sublists use both subdomains and one of the other
   methods.

   Sublist subdomains are merely subdomains of the main DNSxL domain.
   If for example, bad.example.com had two sublists relay and malware,
   entries for 192.0.2.99 would be 99.2.0.192.relay.bad.example.com or
   99.2.0.192.malware.bad.example.com.  Sublist names contain non-
   digits, so there is no problem of name collisions with entries in the
   main domain, where the IP addresses consist of digits.

   To minimize the number of DNS lookups, multiple sublists can also be
   encoded as bit masks or multiple A records.  With bit masks, the A
   record entry for each IP is the logical OR of the bit masks for all
   of the lists on which the IP appears.  For example, the bit masks for
   the two sublists might be 127.0.0.1 and 127.0.0.2, in which case an
   entry for an IP on both lists would be 127.0.0.3:

   99.2.0.192.bad.example.com    A      127.0.0.3

   With multiple A records, each sublist has a different assigned value
   such as 127.0.1.1, 127.0.1.2, and so forth, with an A record for each
   sublist on which the IP appears:

   99.2.0.192.bad.example.com    A      127.0.0.1
   99.2.0.192.bad.example.com    A      127.0.0.2

   There is no widely used convention for mapping sublist names to bits
   or values, beyond the convention that all A values are in the
   127.0.0.0/8 range to prevent unwanted network traffic if the value is
   accidentally used as an IP address.




Levine                  Expires November 8, 2008                [Page 5]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


   DNSxLs that return multiple A records sometimes return multiple TXT
   records as well, although the lack of any way to match the TXT
   records to the A records limits the usefulness of those TXT records.
   Other combined DNSxLs return a single TXT record.

2.4.  DNSxL cache behavior

   The per-record time-to-live and zone refresh intervals of DNSBLs and
   DNSWLs vary greatly depending on the management policy of the list.
   A list of IP addresses assigned to dynamically allocated dialup and
   DHCP users could be expected to change slowly, so the TTL might be
   several days and the zone refreshed once a day.  On the other hand, a
   list of IP addresses that had been observed sending spam might change
   every few minutes, with comparably short TTL and refresh intervals.

2.5.  Test and contact addresses

   Nearly all IP based DNSxLs contain an entry for 127.0.0.2 for testing
   purposes.  DNSBLs that return multiple values often have multiple
   test addresses so that, for example, a DNSBL that can return
   127.0.0.5 would have a test record for 127.0.0.5 that returns an A
   record with the value 127.0.0.5, and a corresponding TXT record.

   Most DNSxLs also contain an A record at the apex of the DNSxL zone
   that points to a web server, so that anyone wishing to learn about
   the bad.example.net DNSBL can check http://bad.example.net.

2.6.  IPv6 DNSxLs

   No DNSxL based on IPv6 addresses has, to the best of our knowledge,
   been deployed yet.  The obvious format for one would use 32-component
   hex nibble-reversed IPv6 addresses in the same places where IPv4
   DNSxLs use four-component decimal byte-reversed addresses.  A single
   DNSxL could in principle contain both IPv4 and IPv6 addresses, since
   the different lengths prevent any ambiguity.  If a DNSxL is
   represented using traditional zone files and wildcards, there is no
   way to specify the length of the name that a wildcard matches, so
   wildcard names would indeed be ambiguous for DNSxLs served in that
   fashion.


3.  Domain name DNSxLs

   A few DNSxLs list domain names rather than IP addresses.  They are
   sometimes called RHSBLs, for right hand side blacklists.  The names
   of their entries contain the listed domain name followed by the name
   of the DNSxL.  If the DNSxL were called doms.example.net, and the
   domain invalid.edu were to be listed, the entry would be named



Levine                  Expires November 8, 2008                [Page 6]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


   invalid.edu.doms.example.net:

   invalid.edu.doms.example.net    A      127.0.0.2
   invalid.edu.doms.example.net    TXT    "Host name used in phish"

   A few name-based DNSBLs encode e-mail addresses using a convention
   adopted from DNS SOA records, so an entry for fred@invalid.edu would
   have the name fred.invalid.edu.doms.example.net:

   fred.invalid.edu.doms.example.net    A      127.0.0.2

   There is no consistent convention for a test entry, but some name-
   based DNSxLs use EXAMPLE.COM as a test entry.  Name-based DNSBLs are
   far less common than IP based DNSBLs.  There is no agreed convention
   for wildcards.

   Name-based DNSWLs can be created in the same manner as DNSBLs, and
   have been used as simple reputation systems with the values of octets
   in the A record representing reputation scores and confidence values,
   typically on a 0-100 or 0-255 scale.


4.  Typical usage of DNSBLs and DNSWLs

   DNSxLs can be served either from standard DNS servers, or from
   specialized servers like rbldns [2] and rbldnsd [4] that accept lists
   of IP addresses and CIDR ranges and synthesize the appropriate DNS
   records on the fly.  Organizations that make heavy use of a DNSxL
   usually arrange for a private mirror of the DNSxL, either using the
   standard AXFR and IXFR or by fetching a file containing addresses and
   CIDR ranges for the specialized servers.  If a /24 or larger range of
   addresses is listed, and the zone's server uses traditional zone
   files to represent the DNSxL, the DNSxL may use wildcards to limit
   the size of the zone file.  If for example, the entire range of
   192.0.2.0/24 were listed, the DNSxL's zone could contain a single
   wildcard for *.2.0.192.bad.example.com.

   DNSBL clients are most often mail servers or spam filters called from
   mail servers.  There's no requirement that DNSBLs be used only for
   mail, and other services such as IRC use them to check client hosts
   that attempt to connect to a server.

   Mail servers that test combined lists most often handle them the same
   as single lists and treat any A or TXT record as meaning that an IP
   is listed without distinguishing among the various reasons it might
   have been listed.  Some mail server and spam filtering software does
   have the ability to apply bit masks to retrieved A values in order to
   select particular sublists of a combined list.



Levine                  Expires November 8, 2008                [Page 7]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


   Mail servers typically check a list of DNSxLs on every incoming SMTP
   connection, with the names of the DNSxLs set in the server's
   configuration.  A common usage pattern is for the server to check
   each list in turn until it finds one with a DNSBL entry, in which
   case it rejects the connection, or a DNSWL entry in which case it
   accepts the connection.  If the address appears on no list at all
   (the usual case for legitimate mail), the mail server accepts the
   connection.  In another approach, DNSxL entries are used as inputs to
   a weighting function that computes an overall score for each message.

   The mail server uses its normal local DNS cache to limit traffic to
   the DNSxL servers and to speed up retests of IP addresses recently
   seen.  Long-running mail servers may cache DNSxL data internally.

   An alternate approach is to check DNSxLs in a spam filtering package
   after a message has been received.  In that case, the IP(s) to test
   are usually extracted from "Received:" header fields or URIs in the
   body of the message.  The DNSxL results may be used to make a binary
   accept/reject decision, or in a scoring system.

   Packages that test multiple header fields need to be able to
   distinguish among values in lists with sublists since, for example,
   an entry indicating that an IP is assigned to dialup users might be
   treated as a strong indication that a message should be rejected if
   the IP sends mail directly to the recipient system, but not if the
   message were relayed through an ISP's mail server.

   Name-based DNSBLs have been used both to check domain names of e-mail
   addresses and host names found in mail headers, and to check the
   domains found in URLs in message bodies.


5.  Security Considerations

   Any system manager that uses DNSxLs is entrusting part of his or her
   server management to the parties that run the lists.  A DNSBL manager
   that decided to list 0/0 (which has actually happened) could cause
   every server that uses the DNSBL to reject all mail.  Conversely, if
   a DNSBL manager removes all of the entries (which has also happened),
   systems that depend on the DNSBL will find that their filtering
   doesn't work as they want it to.

   Since DNSxL users usually make a query for every incoming e-mail
   message, the operator of a DNSxL can extract approximate mail volume
   statistics from the DNS server logs.  This has been used in a few
   instances to estimate the amount of mail individual IPs or IP blocks
   send[5] [6].




Levine                  Expires November 8, 2008                [Page 8]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


   As with any other DNS based services, DNSBLs and DNSWLs are subject
   to various types of DNS attacks which are described in [1].


6.  IANA Considerations

   This memo includes no request to IANA.


7.  Informative References

   [1]  Atkins, D. and R. Austein, "Threat Analysis of the Domain Name
        System (DNS)", RFC 3833, August 2004.

   [2]  Bernstein, D., "rbldns, in "djbdns"".

   [3]  "MAPS RBL+".

   [4]  Tokarev, M., "rbldnsd: Small Daemon for DNSBLs".

   [5]  Ironport Systems, "Senderbase".

   [6]  Levine, J., "The South Korean Network Blocking List".


Author's Address

   John Levine
   Taughannock Networks
   PO Box 727
   Trumansburg, NY  14886

   Phone: +1 831 480 2300
   Email: standards@taugh.com
   URI:   http://www.taugh.com
















Levine                  Expires November 8, 2008                [Page 9]


Internet-Draft        DNS Blacklists and Whitelists             May 2008


Full Copyright Statement

   Copyright (C) The IETF Trust (2008).

   This document is subject to the rights, licenses and restrictions
   contained in BCP 78, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
   OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
   THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
   OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.


Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the procedures with respect to rights in RFC documents can be
   found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   http://www.ietf.org/ipr.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at
   ietf-ipr@ietf.org.











Levine                  Expires November 8, 2008               [Page 10]