DNS Cache-Only Config Advice

[unixanalyst]

Hi Chaps and Chappettes,

I've had a short period of time recently to learn and implement DNS cache-only in our organisation. Trouble is, according to my tcpdumps, the amount of traffic on port 53 has increased. This is of course the exact opposit of the desired effect. Would y'all mind looking over my config?

FYI, the machine is an Exim MTA handling mail traffic. There are 2 internal and 2 external DNS servers as well as this machine. resolv.conf is pointed to local. Stuff I dig appears in the cache dump. I'm not sure if I should have the zone specified as there is no zonefile on this machine, but it did seem to get the internal traffic going again.

RHEL 5 2.6.18-128.1.6.el5PAE
BIND 9.3.4-10.P1.el5

Many, many thanks for any help.

named.conf:

options {
  forwarders { ip_of_external_1; ip_of_external_2; };
  forward first;
  directory "/var/named" ;
  dump-file "dump/named_dump.db";
  allow-query { any; };
  allow-recursion { any; };
  allow-transfer { none; };
  allow-notify { none; };
  listen-on-v6 { none; };
  recursive-clients 3500;
  version none;
  zone-statistics yes;
  notify no;
  auth-nxdomain no;
  };

logging{
  channel simple_log {
    file "named.log" versions 3 size 5m;
    severity dynamic;
    print-time yes;
    print-severity yes;
    print-category yes;
  };
  category default{
    simple_log;
  };
  category lame-servers {
    null;
  };
};

zone "orgname.com" {
        type forward;
        forwarders { ip_of_internal_1; 2ip_of_internal_2; ip_of_internal_1_service_ip; ip_of_internal_2_service_ip; };
};

zone "0.0.127.in-addr.arpa" {
        type master;
        file "db.127.0.0";
};

db.127.0.0:

$TTL 345600
@               IN      SOA     localhost. root.localhost.      (
                                00      ; Serial
                                86400   ; Refresh
                                7200    ; Retry
                                2592000 ; Expire
                                345600) ; Minimum

                IN      NS      localhost.
1               IN      PTR     localhost.

[sknake]

How are you using tcpdump to measure traffic? And if you're doing the recursive lookups then you don't benefit from upstream DNS doing it. You should have a recursive server in front of you that you could send all of your lookups to instead of doing them yourself, unless you want to?

[unixanalyst]

Hi,

I'm running

tcpdump -i eth0 -s 0 -w /tmp/dns.servername.`date +%Y%m%d-%H%M`.cap "tcp port 53 or udp port 53"

and basically noting how many megs it generates per minute/hour.

You've made me think about a couple of things... this is not intended to be a DNS server per se, I just want to cut down on the number of queries it makes to the actual DNS servers. It is a message transfer agent running exim, so does loads of lookups for email purposes. Maybe I should turn off recursion?

I'm sorry but I didn't understand most of the rest, I'm a real noob to DNS. It's just an MTA. It needs to cache DNS for itself. It needs to be able to lookup on the internal DNSs for incoming mail and the external DNSs for outgoing mail.

Do I need to put a "." zone into named.conf? Should I take out the "orgname.com" zone? Should I have a "localhost" zone?

Sorry there's loads more questions there. Thanks big time for the help.

[sknake]

OK well DNS works like this. You have what is called a TLD which is .com .net .org ... etc. So if you want to get the IP of www.apexsoftware.com you have to talk to .com, then .com sends you somewhere, and you keep following it down the line until someone says "OK, here it is". That is called the authoratitive response. Here is a sample. This is an ipv6-enabled DNS request but you can consider IPv6/IPv4 the same here because it doesn't make a difference for what we're talking about.

sk@sk:/tmp$ dig www.apexsoftware.com +trace

; <<>> DiG 9.4.0 <<>> www.apexsoftware.com +trace
;; global options:  printcmd
.                       91066   IN      NS      H.ROOT-SERVERS.NET.
.                       91066   IN      NS      G.ROOT-SERVERS.NET.
.                       91066   IN      NS      E.ROOT-SERVERS.NET.
.                       91066   IN      NS      B.ROOT-SERVERS.NET.
.                       91066   IN      NS      L.ROOT-SERVERS.NET.
.                       91066   IN      NS      C.ROOT-SERVERS.NET.
.                       91066   IN      NS      D.ROOT-SERVERS.NET.
.                       91066   IN      NS      A.ROOT-SERVERS.NET.
.                       91066   IN      NS      J.ROOT-SERVERS.NET.
.                       91066   IN      NS      F.ROOT-SERVERS.NET.
.                       91066   IN      NS      M.ROOT-SERVERS.NET.
.                       91066   IN      NS      K.ROOT-SERVERS.NET.
.                       91066   IN      NS      I.ROOT-SERVERS.NET.
;; Received 336 bytes from 72.16.178.119#53(72.16.178.119) in 4 ms

com.                    172800  IN      NS      L.GTLD-SERVERS.NET.
com.                    172800  IN      NS      G.GTLD-SERVERS.NET.
com.                    172800  IN      NS      E.GTLD-SERVERS.NET.
com.                    172800  IN      NS      B.GTLD-SERVERS.NET.
com.                    172800  IN      NS      D.GTLD-SERVERS.NET.
com.                    172800  IN      NS      H.GTLD-SERVERS.NET.
com.                    172800  IN      NS      F.GTLD-SERVERS.NET.
com.                    172800  IN      NS      K.GTLD-SERVERS.NET.
com.                    172800  IN      NS      I.GTLD-SERVERS.NET.
com.                    172800  IN      NS      C.GTLD-SERVERS.NET.
com.                    172800  IN      NS      J.GTLD-SERVERS.NET.
com.                    172800  IN      NS      A.GTLD-SERVERS.NET.
com.                    172800  IN      NS      M.GTLD-SERVERS.NET.
;; Received 510 bytes from 2001:503:ba3e::2:30#53(A.ROOT-SERVERS.NET) in 140 ms

apexsoftware.com.       172800  IN      NS      beyond.cbeyond.net.
apexsoftware.com.       172800  IN      NS      infinity.cbeyond.net.
;; Received 125 bytes from 2001:503:231d::2:30#53(B.GTLD-SERVERS.NET) in 151 ms

www.apexsoftware.com.   3600    IN      A       72.16.178.114
apexsoftware.com.       86400   IN      NS      beyond.cbeyond.net.
apexsoftware.com.       86400   IN      NS      infinity.cbeyond.net.
apexsoftware.com.       86400   IN      NS      to.cbeyond.net.
;; Received 174 bytes from 64.238.96.11#53(infinity.cbeyond.net) in 6 ms

Now that we know how DNS works lets compare your old/new configuration. In the below examples when it crosses [Your Router] that means its eating up your internet bandwidth.

If you use your ISPs DNS server it would look like this:
Client Machine --> [Your Router] --> ISP's DNS

Which triggers:
ISP's DNS --> . (query for root servers), you are given M.ROOT-SERVERS.NET
ISP's DNS --> M.ROOT-SERVERS.NET (query for .com), you are given B.GTLD-SERVERS.NET
ISP's DNS --> B.GTLD-SERVERS.NET (query for apexsoftware.com), you are given beyond.cbeyond.net
** Note
ISP's DNS --> infinity.cbeyond.net (query for www.apexsoftware.com), you are given the final address: 72.16.178.114!

Now to explain the note. Your DNS servers has the root servers build in to it. You should have a zone like this w/ the root servers:

// prime the server with knowledge of the root servers
zone "." {
        type hint;
        file "/etc/bind/db.root";
};

The *** Note above is placed there because B.GTLD-SERVERS.NET gives you beyond.cbeyond.net --- but how do you know what IP address beyond.cbeyond.net is? In some cases you have to start the DNS all over again at the root servers, then .net, then cbeyond.net, then beyond.cbeyond.net in order to find the IP address of the next DNS server to look up apexsoftware.com. Notice I said in some cases though, this is only if they do not have a glue.

Here is a glue:

sk:/etc/bind# dig apexsoftware.com @b.gtld-servers.net

; <<>> DiG 9.4.0 <<>> apexsoftware.com @b.gtld-servers.net
; (2 servers found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27170
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 2, ADDITIONAL: 2
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;apexsoftware.com.              IN      A

;; AUTHORITY SECTION:
apexsoftware.com.       172800  IN      NS      beyond.cbeyond.net.
apexsoftware.com.       172800  IN      NS      infinity.cbeyond.net.

;; ADDITIONAL SECTION:
beyond.cbeyond.net.     172800  IN      A       66.180.96.11
infinity.cbeyond.net.   172800  IN      A       64.238.96.11

;; Query time: 149 msec
;; SERVER: 2001:503:231d::2:30#53(2001:503:231d::2:30)
;; WHEN: Sat Sep 26 03:56:30 2009
;; MSG SIZE  rcvd: 121

Notice how the IP addresses are in the "Additional Section" and not the "Authority Section"? This is the glue. The root server is basically saying "I'm not responsible for apexsoftware.com ... you need to ask beyond.cbeyond.net, and oh by the way the ip for beyond.cbeyond.net is 66.180.96.11". This stops you from having to start the DNS cycle all over again to resolve cbeyonds nameserver's IP in order to DNS apexsoftware.com's IP.

-----

So what was a single DNS lookup
Mail Server --> [Your Router] --> ISP's DNS

Has been replaced by 4..N DNS lookups. if you have a.really.really.deep.domain.in.com you could easily be doing 12+ lookups.

------

So ... all that being said. You will want to handle DNS requests locally for your internal address space and forward all other queries to your ISPs' DNS server. Take a look at forwarding with BIND.

[unixanalyst]

Thanks Sknake, that's useful info, helps to understand just what all the traffic is.

You know, I think I might be overcomplicating the task at hand. I want the machine to do exactly what it was doing before, query the internal DNS servers... but just cache the responses. The internal DNS servers have all the internal zone files and send everything else to the externals. I can't comment on whether or not this is the ideal setup, that's outside my remit.

So, bit of a rethink here, I'm going to go back to a basic cache-only config (which I still need to understand, there's conflicting info out there) and forget entirely about what to send to which DNS server. It's all going to the internals, like it does at present.

Would this be about right...?

named.conf:

options {
  forwarders { internal_dns_1; internal_dns_2; };
  forward first;
  directory "/var/named" ;
  dump-file "dump/named_dump.db";
  allow-query { any; };
  allow-recursion { any; };  <<< not sure about this one
  allow-transfer { none; };
  allow-notify { none; };
  listen-on-v6 { none; };
  recursive-clients 3500;
  version none;
  zone-statistics yes;
  notify no;
  auth-nxdomain no;
  };

(logging clause deleted)

zone "." {          <<< Does this mean "everything you get asked"??
        type forward;
        forwarders { internal_dns_1; internal_dns_2; };
};

zone "0.0.127.in-addr.arpa" {
        type master;
        file "db.127.0.0";
};

[sknake]

If you're only wanting the machine to query its' local DNS server then you should only allow inbound requests from 127.0.0.1/your internal ips on the machine.

But your root zone "." looks right, although I have never tried to set it up like that before. Give it a shot and if it doesn't work i'll try it out on my DNS server.

[unixanalyst]

Ah, interesting. Yes, I suppose I do only want the localhost to have access to the DNS cache. So that would be...

allow-query { 127.0.0.1; local_ip_address; };
allow-recursion { 127.0.0.1; local_ip_address; };

I have to wait to for change control to try out the 'zone "." forward', I'll be back with the results. Thanks again.

[unixanalyst]

Oh for heaven's sake! Look what I just found in chroot/etc...

//
// named.caching-nameserver.conf
//
// Provided by Red Hat caching-nameserver package to configure the
// ISC BIND named(8) DNS server as a caching only nameserver
// (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//
// DO NOT EDIT THIS FILE - use system-config-bind or an editor
// to create named.conf - edits to this file will be lost on
// caching-nameserver package upgrade.
//
options {
        listen-on port 53 { 127.0.0.1; };
        listen-on-v6 port 53 { ::1; };
        directory       "/var/named";
        dump-file       "/var/named/data/cache_dump.db";
        statistics-file "/var/named/data/named_stats.txt";
        memstatistics-file "/var/named/data/named_mem_stats.txt";
        query-source    port 53;
        query-source-v6 port 53;
        allow-query     { localhost; };
};
logging {
        channel default_debug {
                file "data/named.run";
                severity dynamic;
        };
};
view localhost_resolver {
        match-clients      { localhost; };
        match-destinations { localhost; };
        recursion yes;
        include "/etc/named.rfc1912.zones";
};

And as you would expect, you still need to add your forwarders in the options section (http://gofedora.com/configure-caching-nameserver-named/).

Thanks again sknake.

[sknake]

I'm glad you got it working :)

Please mark this thread as solved if you have found an answer to your question and good luck!

[unixanalyst]

Right, I've implemented the config I previously mentioned. My tcp dump file is smaller than previous captures, but not much, and it is Sunday. Most of the requests are to the internal DNS server. The mail queues are stable.

In the tcpdump there are *lots* of A record lookups to a domain that we own that does not have an A record, only MX. Dig on this domain, from the exim server itself (which is where I'm running cache-only DNS) returns NOERROR but also no IP...

[eximbox]# dig -t A errantdomain.com

; <<>> DiG 9.3.4-P1 <<>> -t A errantdomain.com
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39800
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;errantdomain.com.         IN      A

;; AUTHORITY SECTION:
errantdomain.com.  7633    IN      SOA     our_hidden_master.orgname.com. hostmaster.orgname.com. 2008041200 86400 7200 900000 86400

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Sep 27 12:31:08 2009
;; MSG SIZE  rcvd: 104

dig -t MX...

; <<>> DiG 9.3.4-P1 <<>> -t MX errantdomain.com
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34234
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 3

;; QUESTION SECTION:
;errantdomain.com.         IN      MX

;; ANSWER SECTION:
errantdomain.com.  83252   IN      MX      10 mailhost.eb.orgname.com.

;; AUTHORITY SECTION:
errantdomain.com.  83252   IN      NS      internal_dns1.orgname.com.
errantdomain.com.  83252   IN      NS      internal_dns2.orgname.com.

;; ADDITIONAL SECTION:
mailhost.eb.orgname.com.  81968   IN      A       ip_address
internal_dns1.orgname.com. 2754   IN      A       ip_address
internal_dns2.orgname.com. 2754   IN      A       ip_address

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Sun Sep 27 12:31:00 2009
;; MSG SIZE  rcvd: 177

My guess is that because no record is returned for the A query, nothing gets cached and it (I also guess that *it* is exim) just keeps asking the same question. Is this a sign that my config is not working or is there a way to modify this behaviour? Shouldn't it be coming back with NXRRSET?

[sknake]

Well there are settings in BIND for caching out-of-zone data which is the case here. There are inherent security issues with caching out of zone data unless you know what you are doing (which is the case here ;)). Take a look at this:
http://www.zytrax.com/books/dns/ch7/queries.html

The default behavior should be what you want but double check your configs and make sure they're set up for yes/yes caching. The URL indicates it should only be for CNAME/DNAME out-of-zone following but I would try it just to be sure.

Also take a look at your cache to see what is in it so we don't have to guess:

sk:/var/cache/bind# rndc dumpdb -cache
sk:/var/cache/bind# ls
named_dump.db  stats

From what I can tell MX records aren't cached. If I ran a dump -all I only see MX records for zones where the server is an authority. I have been reading around for half an hour and I can't see a definitive answer on how caching MX records should work. I do not see them in the bind cache and in RFC1912 I see this:

It is a good idea to give every host an MX record, even if it points
to itself! Some mailers will cache MX records, but will always need
to check for an MX before sending mail. If a site does not have an
MX, then every piece of mail may result in one more resolver query,
since the answer to the MX query often also contains the IP addresses
of the MX hosts. Internet SMTP mailers are required by [RFC 1123] to
support the MX mechanism.

It looks like you were right that exim keeps asking regardless. This must be the behavior of the mailer...

[unixanalyst]

Hi Sknake,

A couple of thoughts before bed...

I have no "additional-from*" statements so from the link, I should indeed be "yes yes" (bind 9).

# grep MX cache_dump.db  | wc -l
2802

(counted them to hide the details, they're all MX records)

I wonder why only a few zones behave with the repeated queries, no A record, maybe I should visit their config and have them return a different value? I notice that there is no answer section when I dig these.

ttfn.

[sknake]

Are they "domain.mx" -> mexico, or are they "IN MX" records? My qmail mail server only had domain.mx records in the bind cache, and MX records for my authoratative sites (my domains) -- not cached ones. I think you're on the right track with not having the A record. While its technically correct it may be confusing a resolver.

[unixanalyst]

Just FYI on the thread...

25927   MX      5 mail.nacro.org.uk.
                        25927   MX      10 mail.uksolutions.net.
                        48373   MX      10 mailgate.sawyershall.org.uk.
                        36730   MX      5 mailgate1.networcs.net.
                        36730   MX      5 mailgate2.networcs.net.
                        36730   MX      15 mailgate3.networcs.net.
                        36730   MX      15 mailgate4.networcs.net.
                        62156   MX      0 mx5h.state.nj.us.
                        62156   MX      0 mx6o.state.nj.us.
                        62156   MX      0 mx7h.state.nj.us.
                        62156   MX      0 mx8o.state.nj.us.

Any idea what the number means? Is it TTL?

I'll post more when I've checked out altering the record for the errant domain.

[sknake]

Yes it is the TTL id

[unixanalyst]

Right, it's definitely working. I was unconvinced that Exim was actually benefiting from the local DNS cache but have proved that it is. Here's what I did to test...

[The packet capture bits were Ethereal screenshots but had to be deleted because they contained internal IP addresses. I've included descriptions of what was found instead]

Check yahoo isn't in the cache
[root@eximbox data]# rndc dumpdb
[root@eximbox data]# grep yahoo cache_dump.db
[root@eximbox data]# [nothing returned]
Start capturing TCP traffic on port 53
[root@eximbox data]# tcpdump -i eth0 -s 0 -w /tmp/dns.eximbox.`date +%Y%m%d-%H%M`.cap "tcp port 53 or udp port 53" &
Make Exim generate some DNS queries
[root@eximbox data]# echo test | exim -d+resolver [e-mail]
... [output deleted] ...
Exim reports successful DNS lookup
DNS lookup of yahoo.com (MX) succeeded
...
host_find_bydns yield = HOST_FOUND (2); returned hosts:
  f.mx.mail.yahoo.com 98.137.54.237 MX=1
  a.mx.mail.yahoo.com 67.195.168.31 MX=1
  e.mx.mail.yahoo.com 216.39.53.1 MX=1
  c.mx.mail.yahoo.com 216.39.53.2 MX=1
  c.mx.mail.yahoo.com 216.39.53.3 MX=1
  d.mx.mail.yahoo.com 68.142.202.247 MX=1
  d.mx.mail.yahoo.com 209.191.88.247 MX=1
  b.mx.mail.yahoo.com 66.196.82.7 MX=1
  b.mx.mail.yahoo.com 66.196.97.250 MX=1
  g.mx.mail.yahoo.com 98.137.54.238 MX=1
  g.mx.mail.yahoo.com 206.190.53.191 MX=1
... [output deleted] ...

Kill the capture
[root@eximbox data]# kill %1
93 packets captured
186 packets received by filter
0 packets dropped by kernel

Verify that DNS traffic has occured
[Packet capture:] Shows DNS queries going to upstream DNS servers and replies coming back.

Dump the cache and verify that yahoo has been entered
[root@eximbox data]# rndc dumpdb
[root@eximbox data]# grep yahoo cache_dump.db
yahoo.com.              105668  NS      ns1.yahoo.com.
                        105668  NS      ns2.yahoo.com.
                        105668  NS      ns3.yahoo.com.
                        105668  NS      ns4.yahoo.com.
                        105668  NS      ns5.yahoo.com.
                        105668  NS      ns6.yahoo.com.
                        105668  NS      ns8.yahoo.com.
                        4511    MX      1 a.mx.mail.yahoo.com.
                        4511    MX      1 b.mx.mail.yahoo.com.
                        4511    MX      1 c.mx.mail.yahoo.com.
                        4511    MX      1 d.mx.mail.yahoo.com.
                        4511    MX      1 e.mx.mail.yahoo.com.
                        4511    MX      1 f.mx.mail.yahoo.com.
                        4511    MX      1 g.mx.mail.yahoo.com.
a.mx.mail.yahoo.com.    1024    A       67.195.168.31
b.mx.mail.yahoo.com.    1024    A       66.196.82.7
c.mx.mail.yahoo.com.    1024    A       216.39.53.2
d.mx.mail.yahoo.com.    1024    A       68.142.202.247
e.mx.mail.yahoo.com.    1024    A       216.39.53.1
f.mx.mail.yahoo.com.    1782    A       98.137.54.237
g.mx.mail.yahoo.com.    1024    A       98.137.54.238
ns1.yahoo.com.          105668  A       68.180.131.16
ns2.yahoo.com.          105668  A       68.142.255.16
ns3.yahoo.com.          105668  A       121.101.152.99
ns4.yahoo.com.          105668  A       68.142.196.63
ns5.yahoo.com.          105668  A       119.160.247.124
ns6.yahoo.com.          105668  A       202.43.223.170
ns8.yahoo.com.          105668  A       202.165.104.22

Start a new capture
[root@eximbox data]# tcpdump -i eth0 -s 0 -w /tmp/dns.eximbox.`date +%Y%m%d-%H%M`.cap "tcp port 53 or udp port 53" &
Make Exim generate more DNS queries
[root@eximbox  data]# echo test | exim -d+resolver [e-mail]
... [output deleted] ...
Which again succeed
DNS lookup of yahoo.com (MX) succeeded
...
host_find_bydns yield = HOST_FOUND (2); returned hosts:
... [output deleted] ...

Kill the capture
[root@eximbox  data]# kill %1
32 packets captured
65 packets received by filter
0 packets dropped by kernel

Analyse the traffic
[Packet capture:] No DNS traffic between host and upstream DNS servers.

Big thanks sknake.