[FC3] Sites 'disappearing' from DNS

[Brian Fahrlander]

    I've been seeing a problem growing a little over time.  It's odd,
and I have no idea what's wrong- maybe someone has an idea (or at least
a better idea about DNS than I).

    I have a machine that relies on another to do the DNS for my home;
the one on the ISP seems unable to figure out perl.org, so I had to make
my own.  Following instructions years ago, I created a DNS machine and
it's been fine since then without touching it.

    But every once in a while I get a link to a site and firefox just
gives me "jobsearch.monster.com" not found.  I can run 'host' from the
same machine and it nails it.  I can run it from my DNS machine, and it
nails it again, in a flash.

    Just now I went to 'hardware.slashdot.org' (just like a couple of
days ago) and it didn't find it.

    Is there a chance that firefox does something different? I don't
want to fill up my /etc/hosts with every site it decides not to find...

    Thanks
-- 
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Craig White]

On Sun, 2005-04-03 at 16:31 -0500, Brian Fahrlander wrote:
>      I've been seeing a problem growing a little over time. 
> It's odd,
>  and I have no idea what's wrong- maybe someone has an idea (or at least
>  a better idea about DNS than I).
>  
>      I have a machine that relies on another to do the DNS for my home;
>  the one on the ISP seems unable to figure out perl.org, so I had to make
>  my own.  Following instructions years ago, I created a DNS machine and
>  it's been fine since then without touching it.
>  
>      But every once in a while I get a link to a site and firefox just
>  gives me "jobsearch.monster.com" not found.  I can run 'host' from the
>  same machine and it nails it.  I can run it from my DNS machine, and it
>  nails it again, in a flash.
>  
>      Just now I went to 'hardware.slashdot.org' (just like a couple of
>  days ago) and it didn't find it.
>  
>      Is there a chance that firefox does something different? I don't
>  want to fill up my /etc/hosts with every site it decides not to find... 
----
the fact that it doesn't find it the first time but does on subsequent
tries suggest that you have some problem with the setup and latency so
your client times out before the dns lookup completes.

Probably the best way to fix that is to fix your caching dns server.

Craig

[Brian Fahrlander]

On Sun, 2005-04-03 at 14:43 -0700, Craig White wrote:

>  the fact that it doesn't find it the first time but does on
> subsequent
>  tries suggest that you have some problem with the setup and latency so
>  your client times out before the dns lookup completes.
>  
>  Probably the best way to fix that is to fix your caching dns server. 
    Well, if I had to get it twice to actually make it, sure...the funny
thing is, if I go there first with 'host' and find it, then use Firefox,
it still doesn't. (Denying all reason that *I* know)  Firefox just won't
find it. That's what makes me think Firefox is involved.   These are
sites I've visited every day or so for years...and I've not changed the
local /etc/resolv.conf or anything on my end for about as long.

    How can this be?

-- 
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Bob Chiodini]

On Mon, 2005-04-04 at 06:02 -0500, Brian Fahrlander wrote:
>  On Sun, 2005-04-03 at 14:43 -0700, Craig White wrote:
>  
>  > the fact that it doesn't find it the first time but does on subsequent
>  > tries suggest that you have some problem with the setup and latency so
>  > your client times out before the dns lookup completes.
>  > 
>  > Probably the best way to fix that is to fix your caching dns server.
>  
>      Well, if I had to get it twice to actually make it, sure...the funny
>  thing is, if I go there first with 'host' and find it, then use Firefox,
>  it still doesn't. (Denying all reason that *I* know)  Firefox just won't
>  find it. That's what makes me think Firefox is involved.   These are
>  sites I've visited every day or so for years...and I've not changed the
>  local /etc/resolv.conf or anything on my end for about as long.
>  
>      How can this be?
>  
>  -- 
>  ------------------------------------------------------------------------
>  Brian Fahrländer                 Christian, Conservative, and Technomad
>  Evansville, IN                                http://www.fahrlander.net 
>  ICQ: 5119262                                          AIM: WheelDweller
>  ------------------------------------------------------------------------
>  
>   
Brian,

Do any other browsers have this problem?

I haven't seen any of the IPv6 firefox issues mentioned lately, but have
you turned off IPv6?  IIRC firefox had issues timing out DNS lookups via
IPv6, before trying IPv4.

add:

alias net-pf-10 off

To /etc/modprobe.conf, unfortunately you'll have to reboot to get it to
take affect.

Bob...

Bob...

[Brian Fahrlander]

On Mon, 2005-04-04 at 07:14 -0400, Bob Chiodini wrote:

>  Do any other browsers have this problem? 
    Well, I've tried Galeon, but it wasn't for long...I might have to
try something along those lines.

>  I haven't seen any of the IPv6 firefox issues mentioned lately,
> but have
>  you turned off IPv6?  IIRC firefox had issues timing out DNS lookups via
>  IPv6, before trying IPv4.
>  
>  add:
>  
>  alias net-pf-10 off
>  
>  To /etc/modprobe.conf, unfortunately you'll have to reboot to get it to
>  take affect. 
    Unfortunately it's already there; I was hoping you'd have an answer;
oh,well- the hunt goes on.  :)

    Thanks for your effort!

-- 
Those who entrust life and limb to Microsoft deserve neither. -Me, 2005
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Andy Green]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brian Fahrlander wrote:
| On Sun, 2005-04-03 at 14:43 -0700, Craig White wrote:
|
|
|>the fact that it doesn't find it the first time but does on subsequent
|>tries suggest that you have some problem with the setup and latency so
|>your client times out before the dns lookup completes.
|>
|>Probably the best way to fix that is to fix your caching dns server.
|
|
|     Well, if I had to get it twice to actually make it, sure...the funny
| thing is, if I go there first with 'host' and find it, then use Firefox,
| it still doesn't. (Denying all reason that *I* know)  Firefox just won't
| find it. That's what makes me think Firefox is involved.   These are
| sites I've visited every day or so for years...and I've not changed the
| local /etc/resolv.conf or anything on my end for about as long.
|
|     How can this be?

Your ISP DNS is likely going slow every now and again -- watch it with
tcpdump and see what you see.

Whatever machine at your site talks to the ISP DNS server is often
giving up on the query before the response is received.  Then I guess it
gives up and figures it's an NXDOMAIN.  There's a thing called negative
TTL for DNS, basically if it got a response of NXDOMAIN once, it will
for a fixed time not bother to check again but immediately say NXDOMAIN
to queries.  I guess this is where your "it doesn't exist no matter what
I do" period is coming from.

Then after the negative TTL is exhausted, it will check again with your
ISP DNS, and depending on if your ISP DNS is fast enough or not, you
either get through or have another period of negative TTL timeout.

Here's a suggestion: on the machine that talks to your ISP DNS, edit
resolv.conf to add

nameserver xxx.xxx.xxx.xxx
options timeout:25

This will get your machine to wait up to 25 seconds for a response from
the ISP DNS server and should hopefully make the problem go away, if I
understood it right.

- -Andy


> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFCUSOGjKeDCxMJCTIRAiOYAJ9SXJVpoiB7lmxg+DSRIwQ3PF5aPgCcCa+1
> +QmK/NzofFcB2GRQCSuCyF0=
> =BGEm
> -----END PGP SIGNATURE-----

[Brian Fahrlander]

On Mon, 2005-04-04 at 12:22 +0100, Andy Green wrote:

>  Your ISP DNS is likely going slow every now and again -- watch it
> with
>  tcpdump and see what you see.
>  
>  Whatever machine at your site talks to the ISP DNS server is often
>  giving up on the query before the response is received.  Then I guess it
>  gives up and figures it's an NXDOMAIN.  There's a thing called negative
>  TTL for DNS, basically if it got a response of NXDOMAIN once, it will
>  for a fixed time not bother to check again but immediately say NXDOMAIN
>  to queries.  I guess this is where your "it doesn't exist no matter what
>  I do" period is coming from.
>  
>  Then after the negative TTL is exhausted, it will check again with your
>  ISP DNS, and depending on if your ISP DNS is fast enough or not, you
>  either get through or have another period of negative TTL timeout.
>  
>  Here's a suggestion: on the machine that talks to your ISP DNS, edit
>  resolv.conf to add
>  
>  nameserver xxx.xxx.xxx.xxx
>  options timeout:25
>  
>  This will get your machine to wait up to 25 seconds for a response from
>  the ISP DNS server and should hopefully make the problem go away, if I
>  understood it right. 
    Well, I'll keep this idea in mind; but usually when Firefox is
problematic, the "can't get there" box comes up in about 1s...not as
much/over 25s.  But I appreciate the effort...

-- 
Those who entrust life and limb to Microsoft deserve neither. -Me, 2005
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Nigel Wade]

Brian Fahrlander wrote:
>  On Sun, 2005-04-03 at 14:43 -0700, Craig White wrote:
>  
>  
> >the fact that it doesn't find it the first time but does on subsequent
> >tries suggest that you have some problem with the setup and latency so
> >your client times out before the dns lookup completes.
> >
> >Probably the best way to fix that is to fix your caching dns server.
>  
>  
>      Well, if I had to get it twice to actually make it, sure...the funny
>  thing is, if I go there first with 'host' and find it, then use Firefox,
>  it still doesn't. (Denying all reason that *I* know)  Firefox just won't
>  find it. That's what makes me think Firefox is involved.   These are
>  sites I've visited every day or so for years...and I've not changed the
>  local /etc/resolv.conf or anything on my end for about as long.
>  
>      How can this be?
>   
I would start by looking at nscd. It's responsible for caching all sorts of 
info on the client and quite often gets itself very confused.

The 'host' command queries DNS directly without referring to nscd, whereas 
the resolver library will use nscd if its running. If your DNS server fails 
to resolve an entry this failure is cached by nscd (by default for 300s). 
So, a 'host' command might return a valid response whereas any attempt to 
contact it via the resolver/nscd would fail.

Next time this happens, try flushing the nscd hosts cache:

# nscd -i hosts

and try again.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[FC3] Sites 'disappearing' from DNS [Solved]

[Brian Fahrlander]

On Mon, 2005-04-04 at 13:47 +0100, Nigel Wade wrote:

>  I would start by looking at nscd. It's responsible for caching
> all sorts of 
>  info on the client and quite often gets itself very confused.
>  
>  The 'host' command queries DNS directly without referring to nscd, whereas 
>  the resolver library will use nscd if its running. If your DNS server fails 
>  to resolve an entry this failure is cached by nscd (by default for 300s). 
>  So, a 'host' command might return a valid response whereas any attempt to 
>  contact it via the resolver/nscd would fail.
>  
>  Next time this happens, try flushing the nscd hosts cache:
>  
>  # nscd -i hosts
>  
>  and try again. 
    Bingo; that was the answer.  I guess I have something new to read up
on tonight!

    Thank you so much!

-- 
Those who entrust life and limb to Microsoft deserve neither. -Me, 2005
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Jean Francois Martinez]

What could be happenning is that your ISP is rotating his machines on
DNS and sending your notifications through DHCP.  But you have a static
configuration for named so it is pointing at machines who for some
reason don't have up to date data.  You have to read the DHCP
documentation in order for it updating your named config, and, very 
important, firewall config and then restarting both.


On Mon, 2005-04-04 at 13:47 +0100, Nigel Wade wrote:
>  Brian Fahrlander wrote:
>  > On Sun, 2005-04-03 at 14:43 -0700, Craig White wrote:
>  > 
>  > 
>  >>the fact that it doesn't find it the first time but does on subsequent
>  >>tries suggest that you have some problem with the setup and latency so
>  >>your client times out before the dns lookup completes.
>  >>
>  >>Probably the best way to fix that is to fix your caching dns server.
>  > 
>  > 
>  >     Well, if I had to get it twice to actually make it, sure...the funny
>  > thing is, if I go there first with 'host' and find it, then use Firefox,
>  > it still doesn't. (Denying all reason that *I* know)  Firefox just won't
>  > find it. That's what makes me think Firefox is involved.   These are
>  > sites I've visited every day or so for years...and I've not changed the
>  > local /etc/resolv.conf or anything on my end for about as long.
>  > 
>  >     How can this be?
>  > 
>  
>  I would start by looking at nscd. It's responsible for caching all sorts of 
>  info on the client and quite often gets itself very confused.
>  
>  The 'host' command queries DNS directly without referring to nscd, whereas 
>  the resolver library will use nscd if its running. If your DNS server fails 
>  to resolve an entry this failure is cached by nscd (by default for 300s). 
>  So, a 'host' command might return a valid response whereas any attempt to 
>  contact it via the resolver/nscd would fail.
>  
>  Next time this happens, try flushing the nscd hosts cache:
>  
>  # nscd -i hosts
>  
>  and try again.
>  
>  -- 
>  Nigel Wade, System Administrator, Space Plasma Physics Group,
>               University of Leicester, Leicester, LE1 7RH, UK
>  E-mail :    nmw(a)ion.le.ac.uk
>  Phone :     [phone], Fax : [phone]
>   
-- 
Jean Francois Martinez <jfm512(a)free.fr&gt;

[Brian Fahrlander]

On Mon, 2005-04-04 at 23:36 +0200, Jean Francois Martinez wrote:
>  What could be happenning is that your ISP is rotating his machines
> on
>  DNS and sending your notifications through DHCP.  But you have a static
>  configuration for named so it is pointing at machines who for some
>  reason don't have up to date data.  You have to read the DHCP
>  documentation in order for it updating your named config, and, very 
>  important, firewall config and then restarting both. 
    No, I'm on a static IP, and the problem was two different
applications not sharing the same information- "host" could find it in a
flash, but Firefox wouldn't...even immediately after it.

    Thankfully, "ncsd -i host" on the command line nailed it.

    Thanks, though!

-- 
Those who entrust life and limb to Microsoft deserve neither. -Me, 2005
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Nigel Wade]

Brian Fahrlander wrote:
>  On Mon, 2005-04-04 at 23:36 +0200, Jean Francois Martinez wrote:
>  
> >What could be happenning is that your ISP is rotating his machines on
> >DNS and sending your notifications through DHCP.  But you have a static
> >configuration for named so it is pointing at machines who for some
> >reason don't have up to date data.  You have to read the DHCP
> >documentation in order for it updating your named config, and, very 
> >important, firewall config and then restarting both.
>  
>  
>      No, I'm on a static IP, and the problem was two different
>  applications not sharing the same information- "host" could find it in a
>  flash, but Firefox wouldn't...even immediately after it.
>  
>      Thankfully, "ncsd -i host" on the command line nailed it.
>  
>      Thanks, though!
>   
Given that nscd is the "problem", you can alter the time for which nscd 
caches failed DNS lookups. In /etc/nscd.conf change the timeout value in

negative-time-to-live hosts 300

to a smaller value than 300 seconds.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Andy Green]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nigel Wade wrote:

| Given that nscd is the "problem", you can alter the time for which nscd
| caches failed DNS lookups. In /etc/nscd.conf change the timeout value in
|
| negative-time-to-live hosts 300
|
| to a smaller value than 300 seconds.

As I understand it the behaviour of ncsd is just a symptom of the "real"
problem which is wrongful intermittent failed lookups, presumably due to
a timed-out lookup.  One might imagine it would be better to treat that
disease than just try to minimize the pain by reducing the negative time
to live?

- -Andy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFCU6wxjKeDCxMJCTIRAq1SAJ9sTImskJ3uyxcb7akanTzQsVs5yACbB2Fs
> gOmo+zYJPRctWDcb+IbQ8pg=
> =LuEa
> -----END PGP SIGNATURE-----

[Nigel Wade]

Andy Green wrote:
>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>  
>  Nigel Wade wrote:
>  
>  | Given that nscd is the "problem", you can alter the time for which nscd
>  | caches failed DNS lookups. In /etc/nscd.conf change the timeout value in
>  |
>  | negative-time-to-live hosts 300
>  |
>  | to a smaller value than 300 seconds.
>  
>  As I understand it the behaviour of ncsd is just a symptom of the "real"
>  problem which is wrongful intermittent failed lookups, presumably due to
>  a timed-out lookup.  One might imagine it would be better to treat that
>  disease than just try to minimize the pain by reducing the negative time
>  to live?
>   
DNS timeouts are a fact of life, they are not "wrongful", you have to live 
with them. If your DNS server doesn't have an entry for the hostname you 
request (all entries have a fixed TTL), then it has to request it from its 
upstream server. This goes all the way up to a top level domain, and then 
back down to the authoritative server for the domain in question. If this 
takes longer than 30s (the ususal timeout for DNS lookup) then you get a 
failed lookup returned by your DNS server. You can't treat it, it isn't a 
"disease", it's part of the way DNS works.

The root of this particular problem is that nscd caches this failed lookup 
for you, DNS does not.


-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Andy Green]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nigel Wade wrote:

| The root of this particular problem is that nscd caches this failed
| lookup for you, DNS does not.

I respectfully disagree.  I do not experience these "fact of life"
timeouts and fake NXDOMAIN results; I use my ISP DNS cached on a
separate machine here.

The DNS cache is behaving as designed, the problem seems to me to be the
timeout is set too low for the behaviour of the original poster's
upstream DNS, or put another way, the upstream DNS may be overloaded and
not always responsive.  I would do a

tcpdump port 53

(despite the name this gets UDP too) and look for SERVFAIL or slow
response, and if seen, complain to whoever it is that I pay for the
upstream DNS in the one case and in the other case add to /etc/resolv.conf

options timeout:xx

where xx is the timeout in seconds; my DNS cache machine has it set to
25.  If you are hanging around for more than 25 seconds to get DNS that
is not what I would call normal or a "fact of life".

- -Andy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFCVQ4sjKeDCxMJCTIRAjXXAJ9r+dUXMIbpTmDIjap7xl4TLw7z0gCeMq69
> JmkbdszH43A4ZmPEKhhfPO8=
> =7Nzs
> -----END PGP SIGNATURE-----

[Nigel Wade]

Andy Green wrote:
>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>  
>  Nigel Wade wrote:
>  
>  | The root of this particular problem is that nscd caches this failed
>  | lookup for you, DNS does not.
>  
>  I respectfully disagree.  I do not experience these "fact of life"
>  timeouts and fake NXDOMAIN results; I use my ISP DNS cached on a
>  separate machine here.
>  
>  The DNS cache is behaving as designed, the problem seems to me to be the
>  timeout is set too low for the behaviour of the original poster's
>  upstream DNS, or put another way, the upstream DNS may be overloaded and
>  not always responsive.  I would do a
>  
>  tcpdump port 53
>  
>  (despite the name this gets UDP too) and look for SERVFAIL or slow
>  response, and if seen, complain to whoever it is that I pay for the
>  upstream DNS in the one case and in the other case add to /etc/resolv.conf
>  
>  options timeout:xx
>  
>  where xx is the timeout in seconds; my DNS cache machine has it set to
>  25.  If you are hanging around for more than 25 seconds to get DNS that
>  is not what I would call normal or a "fact of life".
>   
You can complain all you like to your ISP, but that won't help one jot if 
the authorititive server for the domain is down/overloaded etc. You will get 
timeouts, and nscd will cache that. A repeat request will get a failure even 
if the information is now available again, until the nscd 
negative-time-to-live is reached.

Like I said, it's nscd that's the problem, not DNS. If you cache DNS using a 
proper DNS server I expect you will be ok. If you rely on nscd then you will 
  see this problem.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw(a)ion.le.ac.uk
Phone :     [phone], Fax : [phone]

[Andy Green]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nigel Wade wrote:

| You can complain all you like to your ISP, but that won't help one jot
| if the authorititive server for the domain is down/overloaded etc. You

Sure, both the way I describe (poor upstream DNS performance) and the
way you describe (poor performance from the guy at the end of the DNS
road for a particular query) can lead to the slow DNS response.

Hey Brian, can you give us an example domain that you experienced the
problem with, and I guess that way we can see if it is the DNS for that
domain that is struggling, or infer that it is your upstream DNS
provider's server that is slow.

- -Andy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFCVRN0jKeDCxMJCTIRArk1AJ9B3yuC/zqI/VGVmij7wCsK2yx+7QCgk4Ay
> f3bw7y94gri2vVT2zQDOk6I=
> =4qvL
> -----END PGP SIGNATURE-----

[Andy Green]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andy Green wrote:

| Hey Brian, can you give us an example domain that you experienced the
| problem with, and I guess that way we can see if it is the DNS for that
| domain that is struggling, or infer that it is your upstream DNS
| provider's server that is slow.

I reviewed the thread and see the first post has some addresses.

I get the nameserver addresses from whois.


# dig @ns2.osdn.com hardware.slashdot.org

; <<>> DiG 9.2.5 <<>> @ns2.osdn.com hardware.slashdot.org
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27430
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 5, ADDITIONAL: 5

;; QUESTION SECTION:
;hardware.slashdot.org.         IN      A

;; ANSWER SECTION:
hardware.slashdot.org.  7200    IN      A       66.35.250.151

;; AUTHORITY SECTION:
slashdot.org.           7200    IN      NS      ns2.osdn.com.
slashdot.org.           7200    IN      NS      ns2.vasoftware.com.
slashdot.org.           7200    IN      NS      ns3.vasoftware.com.
slashdot.org.           7200    IN      NS      ns1.osdn.com.
slashdot.org.           7200    IN      NS      ns1.vasoftware.com.

;; ADDITIONAL SECTION:
ns1.osdn.com.           7200    IN      A       66.35.250.10
ns1.vasoftware.com.     72924   IN      A       12.152.184.135
ns2.osdn.com.           7200    IN      A       66.35.250.11
ns2.vasoftware.com.     72924   IN      A       12.152.184.136
ns3.vasoftware.com.     72924   IN      A       66.35.250.12

;; Query time: 164 msec
;; SERVER: 66.35.250.11#53(66.35.250.11)
;; WHEN: Thu Apr  7 12:12:41 2005
;; MSG SIZE  rcvd: 244


# dig @ns1.tmpw.net jobsearch.monster.com

; <<>> DiG 9.2.5 <<>> @ns1.tmpw.net jobsearch.monster.com
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4501
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 2

;; QUESTION SECTION:
;jobsearch.monster.com.         IN      A

;; ANSWER SECTION:
jobsearch.monster.com.  600     IN      A       63.112.169.3
jobsearch.monster.com.  600     IN      A       63.121.29.3

;; AUTHORITY SECTION:
monster.com.            4800    IN      NS      ns1.tmpw.net.
monster.com.            4800    IN      NS      ns2.tmpw.net.

;; ADDITIONAL SECTION:
ns1.tmpw.net.           3600    IN      A       63.121.28.53
ns2.tmpw.net.           3600    IN      A       63.112.168.53

;; Query time: 118 msec
;; SERVER: 63.121.28.53#53(63.121.28.53)
;; WHEN: Thu Apr  7 12:14:45 2005
;; MSG SIZE  rcvd: 147



After this somewhat unscientific and one-shot test I continue to suspect
the problem is with the performance of his upstream DNS, or some
communication problem that can lose packets including his DNS ones,
since the servers are both responding in well below 200ms.

- -Andy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFCVRaDjKeDCxMJCTIRArs7AJ48UCultq6qLrRPWVJ2GnsTZqNFIgCeKCsr
> qexzTT6aTJ/rsMeRy53K0To=
> =Bwi3
> -----END PGP SIGNATURE-----

[Brian Fahrlander]

On Thu, 2005-04-07 at 12:03 +0100, Andy Green wrote:

>  Hey Brian, can you give us an example domain that you experienced
> the
>  problem with, and I guess that way we can see if it is the DNS for that
>  domain that is struggling, or infer that it is your upstream DNS
>  provider's server that is slow. 
    Sure: hardware.slashdot.org, jobsearch.monster.com...there may have
been others, but I don't recall'em.  These are both sites I'd visit at
least once a day, and 'host' called'em up, but Firefox didn't- that's
what made it seem so weird.  (And I'd completely forgotten that I'd
turned on nscd....)

-- 
Those who entrust life and limb to Microsoft deserve neither. -Me, 2005
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Les Mikesell]

On Thu, 2005-04-07 at 06:21, Brian Fahrlander wrote:
>  > Hey Brian, can you give us an example domain that you
> experienced the
>  > problem with, and I guess that way we can see if it is the DNS for that
>  > domain that is struggling, or infer that it is your upstream DNS
>  > provider's server that is slow.
>  
>      Sure: hardware.slashdot.org, jobsearch.monster.com...there may have
>  been others, but I don't recall'em.  These are both sites I'd visit at
>  least once a day, and 'host' called'em up, but Firefox didn't-
> that's
>  what made it seem so weird.  (And I'd completely forgotten that I'd
>  turned on nscd....) 
Does your /etc/resolv.conf contain more than one nameserver entry?  If
you are having trouble with failures, I'd try more entries even if you
have to point them back at the same server to get another chance.

-- 
  Les Mikesell
   les(a)futuresource.com

[Brian Fahrlander]

On Thu, 2005-04-07 at 11:21 -0500, Les Mikesell wrote:

>  Does your /etc/resolv.conf contain more than one nameserver entry? 
> If
>  you are having trouble with failures, I'd try more entries even if you
>  have to point them back at the same server to get another chance. 
    The local machine (the one I'm on) points only to the DNS machine
in /etc/resolv.conf.  The DNS machine points to itself, and the DNS
there is made aware of the root server changes, etc so it can find
everything by itself.

    I wonder; I'm not really clear on this 'forwarding' concept. Would
this be the place for forwarding on my local machine, to this DNS
machine?

-- 
Those who entrust life and limb to Microsoft deserve neither. -Me, 2005
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Les Mikesell]

On Thu, 2005-04-07 at 15:02, Brian Fahrlander wrote:

>  > Does your /etc/resolv.conf contain more than one nameserver
> entry?  If
>  > you are having trouble with failures, I'd try more entries even if you
>  > have to point them back at the same server to get another chance.
>  
>      The local machine (the one I'm on) points only to the DNS machine
>  in /etc/resolv.conf.  The DNS machine points to itself, and the DNS
>  there is made aware of the root server changes, etc so it can find
>  everything by itself. 
Where does the error occur?  On your machine or the DNS machine?  If
it is your machine you could try adding more nameserver entries in
/etc/resolv.conf even if you have to point them to the same DNS server.

>      I wonder; I'm not really clear on this 'forwarding'
> concept. Would
>  this be the place for forwarding on my local machine, to this DNS
>  machine? 
I'd try to fix the network problem first.  DNS uses UDP, so if you
are overloaded or dropping packets the answers can be dropped.  You
can work around it by doing more retries but you really should not
be dropping the answers in the first place.   You could run a DNS
server on your own machine configured to use the local DNS server
as a fowarder but unless you have a huge amount of traffic you
should not need that.

-- 
  Les Mikesell
   les(a)futuresource.com

[Brian Fahrlander]

On Thu, 2005-04-07 at 16:38 -0500, Les Mikesell wrote:

>  Where does the error occur?  On your machine or the DNS machine?  If
>  it is your machine you could try adding more nameserver entries in
>  /etc/resolv.conf even if you have to point them to the same DNS server. 
   You must not have heard; the error was a popup window from Firefox.
"host" (or I suppose nslookup) could get the IP addresses, but for some
reason Firefox would pop up an error, quickly, and be the problem.  It
doesn't look like any of the 'real' DNS stuff was ever a problem.

>  I'd try to fix the network problem first.  DNS uses UDP, so if
> you
>  are overloaded or dropping packets the answers can be dropped.  You
>  can work around it by doing more retries but you really should not
>  be dropping the answers in the first place.   You could run a DNS
>  server on your own machine configured to use the local DNS server
>  as a fowarder but unless you have a huge amount of traffic you
>  should not need that. 
    Network problem?  No, this problem went away when I messed with
nscd, I don't think there was ever a network problem or DNS problem...

-- 
Those who entrust life and limb to Microsoft deserve neither. -Me, 2005
------------------------------------------------------------------------
Brian Fahrländer                 Christian, Conservative, and Technomad
Evansville, IN                                http://www.fahrlander.net 
ICQ: 5119262                                          AIM: WheelDweller
------------------------------------------------------------------------

[Andy Green]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brian Fahrlander wrote:
| On Thu, 2005-04-07 at 16:38 -0500, Les Mikesell wrote:
|
|
|>Where does the error occur?  On your machine or the DNS machine?  If
|>it is your machine you could try adding more nameserver entries in
|>/etc/resolv.conf even if you have to point them to the same DNS server.
|
|
|    You must not have heard; the error was a popup window from Firefox.
| "host" (or I suppose nslookup) could get the IP addresses, but for some
| reason Firefox would pop up an error, quickly, and be the problem.  It
| doesn't look like any of the 'real' DNS stuff was ever a problem.

That's not what I understood from reading the thread.  The situation is
that the DNS cache is a more noticable "detector" of the problem,
because it extends the lack of resolvability to 300s or whatever due to
the "negative ttl" feature of the cache.  But something ELSE caused the
DNS cache to see a failure to resolve in the first place.  The problem
is highly unlikely to be local to the DNS cache itself I would imagine.
~ So....

|     Network problem?  No, this problem went away when I messed with
| nscd, I don't think there was ever a network problem or DNS problem...

... the next guess is that you may have a systemic network packetloss
issue.  When the DNS request or response packet gets dropped and lost,
then the DNS cache is provoked into "negative ttl" mode and will
immediately tell you NXDOMAIN until the "negative ttl" is used up.  So
there's no mystery why firefox comes back immediately with the bad news,
it's the DNS cache doing its job after it sees a failure to resolve a
site.  The mystery is how the DNS cache got the idea it couldn't resolve
your site in the first place.

Why not try floodpinging your DNS server and see if you see any dots
before your eyes.  Dots represent lost packets somewhere along the line.

ping -f -i 0 -s 1490 192.168.0.1

Replace 192.168.0.1 with your DNS server IP.

- -Andy
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
> Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
>
> iD8DBQFCVis5jKeDCxMJCTIRArgZAJ9yVkyEmbWsIGAZ57fGkeZktkDAyACbBrsa
> nLcy2PTPraX3T461g61egGI=
> =Ari2
> -----END PGP SIGNATURE-----