Is there a file I can examine that will show me what URL's are presently cached in my caching-nameserver ?
Bob Goodwin
On Sat, Apr 28, 2007 at 10:54:12 -0400, Bob Goodwin - W2BOD bobgoodwin@wildblue.net wrote:
Is there a file I can examine that will show me what URL's are presently cached in my caching-nameserver ?
Name servers cache DNS records not URLs. I think bind caches records in memory, but there may be a way to get a dump of what's there. Try doing a google search or reading the bind documentation.
If you are really interested in your browser cache, those files are normally saved in your home directory. However the URLs are usually hashed, so finding out which web pages have been cached requires having a set of URLs you are interested in or looking at the cached web pages.
There is also a browser history file that can have recently used URLs in it. However, if that is what you are looking for, it is easiest to get that information in the web browser rather than the file system.
We might be able to give you a better answer for your real problem if you tell us what it is, rather than asking about how execute one approach you have thought of to try to solve the problem. (You might really just be curious about what's in your DNS cache, but it seems likely there is something else you are trying to do, for which looking at the cache might help.)
On Sat, 2007-04-28 at 10:54 -0400, Bob Goodwin - W2BOD wrote:
Is there a file I can examine that will show me what URL's are presently cached in my caching-nameserver ?
FQDNs, not URIs, as Bruno pointed out. But check out the "rndc" command. That man file for it is lacking, but running the command without any parameters produces this:
# rndc Usage: rndc [-c config] [-s server] [-p port] [-k key-file ] [-y key] [-V] command
command is one of the following:
reload Reload configuration file and zones. reload zone [class [view]] Reload a single zone. refresh zone [class [view]] Schedule immediate maintenance for a zone. retransfer zone [class [view]] Retransfer a single zone without checking serial number. freeze zone [class [view]] Suspend updates to a dynamic zone. thaw zone [class [view]] Enable updates to a frozen dynamic zone and reload it. reconfig Reload configuration file and new zones only. stats Write server statistics to the statistics file. querylog Toggle query logging. dumpdb [-all|-cache|-zones] [view ...] Dump cache(s) to the dump file (named_dump.db). stop Save pending updates to master files and stop the server. stop -p Save pending updates to master files and stop the server reporting process id. halt Stop the server without saving pending updates. halt -p Stop the server without saving pending updates reporting process id. trace Increment debugging level by one. trace level Change the debugging level. notrace Set debugging level to 0. flush Flushes all of the server's caches. flush [view] Flushes the server's cache for a view. flushname name [view] Flush the given name from the server's cache(s) status Display status of the server. recursing Dump the queries that are currently recursing (named.recursing) *restart Restart the server.
* == not yet implemented Version:
Note the "dumpdb" information, around about the middle. There's also statistics options which give you a summary about what it's got, without masses of explicit detail.
On 4/29/07, Tim ignored_mailbox@yahoo.com.au wrote:
On Sat, 2007-04-28 at 10:54 -0400, Bob Goodwin - W2BOD wrote:
Is there a file I can examine that will show me what URL's are presently cached in my caching-nameserver ?
FQDNs, not URIs, as Bruno pointed out. But check out the "rndc" command. That man file for it is lacking, but running the command without any parameters produces this:
# rndc Usage: rndc [-c config] [-s server] [-p port] [-k key-file ] [-y key] [-V] command
command is one of the following:
reload Reload configuration file and zones. reload zone [class [view]] Reload a single zone. refresh zone [class [view]] Schedule immediate maintenance for a zone. retransfer zone [class [view]] Retransfer a single zone without checking serial number. freeze zone [class [view]] Suspend updates to a dynamic zone. thaw zone [class [view]] Enable updates to a frozen dynamic zone and reload it. reconfig Reload configuration file and new zones only. stats Write server statistics to the statistics file. querylog Toggle query logging. dumpdb [-all|-cache|-zones] [view ...] Dump cache(s) to the dump file (named_dump.db). stop Save pending updates to master files and stop the server. stop -p Save pending updates to master files and stop the server reporting process id. halt Stop the server without saving pending updates. halt -p Stop the server without saving pending updates reporting process id. trace Increment debugging level by one. trace level Change the debugging level. notrace Set debugging level to 0. flush Flushes all of the server's caches. flush [view] Flushes the server's cache for a view. flushname name [view] Flush the given name from the server's cache(s) status Display status of the server. recursing Dump the queries that are currently recursing ( named.recursing) *restart Restart the server.
- == not yet implemented
Version:
Note the "dumpdb" information, around about the middle. There's also statistics options which give you a summary about what it's got, without masses of explicit detail.
-- (This box runs FC6, my others run FC4 & FC5, in case that's important to the thread.)
Don't send private replies to my address, the mailbox is ignored. I read messages from the public lists.
-- fedora-list mailing list fedora-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
Tim, I am also curious about this question. Previously I had tried enabling logging via the named.caching-nameserver.conf file and had creeated empty files for the logs but these files have never been updated by named. From your message I have tried running the mdc command but it is not found. Tried to 'locate mdc' and no useful result. I do have the caching-name-server installed... named 2364 1 0 Apr27 ? 00:00:09 /usr/sbin/named -u named -c /etc/named.caching-nameserver.conf which should be the same basic setup as Bob Goodwin's. Now I am trying to see where the mdc command should come from using "yum whatprovides mdc".
My logging (which is not working as yet) configuration from named.caching-nameserver.conf is as follows:
# specify log files for different categories # logging { channel ch_default { file "/var/named/log_default"; severity info; print-category yes; print-severity yes; print-time yes; }; channel ch_security { file "/var/named/log_security"; severity info; print-severity yes; print-time yes; }; channel ch_queries { file "/var/named/log_queries"; severity info; print-time yes; }; category default { ch_default; }; category security { ch_security; }; category lame-servers { null; }; category xfer-in { null; }; category queries { ch_queries; }; };
UPDATE: I have finaly figured out the logging issue as being an ownership problem with the log files. Now I have changed them to named user and group. Bob, using logging may be a partial answer for for you about what queries are hitting your nameserver. This will not tell you what information is still current in your cache though. I do hope that this may be of some help to you. (Still am waiting for the results of my yum query....)
Fennix wrote:
Tim, I am also curious about this question. Previously I had tried enabling logging via the named.caching-nameserver.conf file and had creeated empty files for the logs but these files have never been updated by named. From your message I have tried running the mdc command but it is not found. Tried to 'locate mdc' and no useful result. I do have the caching-name-server installed... named 2364 1 0 Apr27 ? 00:00:09 /usr/sbin/named -u named -c /etc/named.caching-nameserver.conf which should be the same basic setup as Bob Goodwin's. Now I am trying to see where the mdc command should come from using "yum whatprovides mdc".
My logging (which is not working as yet) configuration from named.caching-nameserver.conf is as follows:
# specify log files for different categories # logging { channel ch_default { file "/var/named/log_default"; severity info; print-category yes; print-severity yes; print-time yes; }; channel ch_security { file "/var/named/log_security"; severity info; print-severity yes; print-time yes; }; channel ch_queries { file "/var/named/log_queries"; severity info; print-time yes; }; category default { ch_default; }; category security { ch_security; }; category lame-servers { null; }; category xfer-in { null; }; category queries { ch_queries; }; };
Where did these lines come from? I don't see the in my conf file, thought they might be there commented out but no.
UPDATE: I have finaly figured out the logging issue as being an ownership problem with the log files. Now I have changed them to named user and group. Bob, using logging may be a partial answer for for you about what queries are hitting your nameserver. This will not tell you what information is still current in your cache though. I do hope that this may be of some help to you. (Still am waiting for the results of my yum query....)
And the following did not make it to this list:
On Sat, Apr 28, 2007 at 14:44:33 -0400, Bob Goodwin - W2BOD bobgoodwin@wildblue.net wrote:
After a lot of groping around I finally realized that while they use the dns provided by Wildblue our isp I use some different servers that I found, open dns or some such, and apparently they have collected the data before the isp's server?
Each ISP will have its own caching servers. You don't even need to use those if you don't want. If you are running your own caching nameserver locally, you can just use that. It can be configured to search from the root down instead of going through your ISP's caching server. However some ISP's will set up hidden proxies and force you to use their caches. This can cause problems if you want to use one of the alternate DNS roots.
Does the data collected by the caching-nameserver survive a shutdown and reboot over night? If the computer is not shutdown how long do the records remain? Presumably they are deleted at some point in time since they are subject to change?
Whether the cached data survives a reboot depends on the cache. However, there may not be much point if you shut down overnight. Many DNS records have a timeout in the range of minutes to hours and they would need to be fetched again in any case.
I've been reading whatever I can find but haven't found the answers to these questions. Again there is no real problem, just questions.
You can read some stuff about DNS at: http://cr.yp.to/djbdns.html
Now another question comes to mind, my resolv.conf is as follows:
cat /etc/resolv.conf
nameserver 127.0.0.1 nameserver 208.67.222.222 # OPENDNS.COM nameserver 208.67.220.220 # OPENDNS.COM nameserver 12.189.32.61 # Wildblue dns
I am assuming that it will go through the local cache first, then drop down through the servers specified sequentially until it hits? Am I right in that assumption?
This computer finds the new .com web address, but from my daughters Mac and the others on the Wildblue dns they still have to enter the address numerically. The hosting company said it should work within 48 hours, worst case 72 ... Mine worked almost immediately.
Whatever this seems to be working well especially since getting /var/named/chroot/etc/blocking.conf populated with bothersome advertisers. Initially it would take a minute or so to display a news item and now that's reduced to 3 to 6 seconds typically without collecting all those ad's [which were not displayed but still sent].
Other than that I am overwhelmed with information on this subject and it will take me some time to sort it out. Thanks to everyone.
Bob Goodwin
On 4/30/07, Bob Goodwin - W2BOD bobgoodwin@wildblue.net wrote:
Fennix wrote:
Tim, I am also curious about this question. Previously I had tried enabling logging via the named.caching-nameserver.conf file and had creeated empty files for the logs but these files have never been updated by named. From your message I have tried running the mdc command but it is not found. Tried to 'locate mdc' and no useful result. I do have the caching-name-server installed... named 2364 1 0 Apr27 ? 00:00:09 /usr/sbin/named -u named -c /etc/named.caching-nameserver.conf which should be the same basic setup as Bob Goodwin's. Now I am trying to see where the mdc command should come from using "yum whatprovides mdc".
My logging (which is not working as yet) configuration from named.caching-nameserver.conf is as follows:
# specify log files for different categories # logging { channel ch_default { file "/var/named/log_default"; severity info; print-category yes; print-severity yes; print-time yes; }; channel ch_security { file "/var/named/log_security"; severity info; print-severity yes; print-time yes; }; channel ch_queries { file "/var/named/log_queries"; severity info; print-time yes; }; category default { ch_default; }; category security { ch_security; }; category lame-servers { null; }; category xfer-in { null; }; category queries { ch_queries; }; };
Where did these lines come from? I don't see the in my conf file, thought they might be there commented out but no.
UPDATE: I have finaly figured out the logging issue as being an ownership problem with the log files. Now I have changed them to named user and group. Bob, using logging may be a partial answer for for you about what queries are hitting your nameserver. This will not tell you what information is still current in your cache though. I do hope that this may be of some help to you. (Still am waiting for the results of my yum query....)
And the following did not make it to this list:
On Sat, Apr 28, 2007 at 14:44:33 -0400, Bob Goodwin - W2BOD bobgoodwin@wildblue.net wrote:
After a lot of groping around I finally realized that while they use
the
dns provided by Wildblue our isp I use some different servers that I found, open dns or some such, and apparently they have collected the data before the isp's server?
Each ISP will have its own caching servers. You don't even need to use those if you don't want. If you are running your own caching nameserver locally, you can just use that. It can be configured to search from the root down instead of going through your ISP's caching server. However some ISP's will set up hidden proxies and force you to use their caches. This can cause problems if you want to use one of the alternate DNS roots.
Does the data collected by the caching-nameserver survive a shutdown
and
reboot over night? If the computer is not shutdown how long do the records remain? Presumably they are deleted at some point in time
since
they are subject to change?
Whether the cached data survives a reboot depends on the cache. However, there may not be much point if you shut down overnight. Many DNS records have a timeout in the range of minutes to hours and they would need to be fetched again in any case.
I've been reading whatever I can find but haven't found the answers to these questions. Again there is no real problem, just questions.
You can read some stuff about DNS at: http://cr.yp.to/djbdns.html
Now another question comes to mind, my resolv.conf is as follows:
cat /etc/resolv.conf
nameserver 127.0.0.1 nameserver 208.67.222.222 # OPENDNS.COM nameserver 208.67.220.220 # OPENDNS.COM nameserver 12.189.32.61 # Wildblue dns
I am assuming that it will go through the local cache first, then drop down through the servers specified sequentially until it hits? Am I right in that assumption?
This computer finds the new .com web address, but from my daughters Mac and the others on the Wildblue dns they still have to enter the address numerically. The hosting company said it should work within 48 hours, worst case 72 ... Mine worked almost immediately.
Whatever this seems to be working well especially since getting /var/named/chroot/etc/blocking.conf populated with bothersome advertisers. Initially it would take a minute or so to display a news item and now that's reduced to 3 to 6 seconds typically without collecting all those ad's [which were not displayed but still sent].
Other than that I am overwhelmed with information on this subject and it will take me some time to sort it out. Thanks to everyone.
Bob Goodwin
-- fedora-list mailing list fedora-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
Bob,
Those lines I had added (found via google) to to enable logging for tracking what queries were hitting my DNS server along with the 3 log files in /etc/named/ to capture the data. The rest of my configuration I have left as the stock install. My only other change was to resolv.conf to insert the server directive to point to 127.0.0.1.
I also really had wanted to add Tim's configuration for blocking.conf but I seem to have missed some important detail in his instructions to do this successfully.
Fennix wrote:
Bob,
Those lines I had added (found via google) to to enable logging for tracking what queries were hitting my DNS server along with the 3 log files in /etc/named/ to capture the data. The rest of my configuration I have left as the stock install. My only other change was to resolv.conf to insert the server directive to point to 127.0.0.1 http://127.0.0.1.
I also really had wanted to add Tim's configuration for blocking.conf but I seem to have missed some important detail in his instructions to do this successfully.
Ok, I figured those lines came from somewhere else. I may try just using what you have there and see if I can obtain some usable log files. As I said my questions really stemmed from an interest in understanding better what is happening.
I simply followed what Tim suggested in this mailing list and the result was excellent. I also use the "adblock" extension for Firefox and obtained addresses from it that were causing me grief and added those to the blocking.conf file, in fact I just added another a few days ago and the result is immediately apparent after doing "service named restart" I suspect it would be easy to miss that step, not sure if it was suggested or if it just occurred to me to try that?
Bob Goodwin
On Mon, 2007-04-30 at 03:18 +0800, Fennix wrote:
I also really had wanted to add Tim's configuration for blocking.conf but I seem to have missed some important detail in his instructions to do this successfully.
If you haven't worked it out yet, sing out and we can go through it again.
On Sun, 2007-04-29 at 13:08 -0400, Bob Goodwin - W2BOD wrote:
another question comes to mind, my resolv.conf is as follows:
cat /etc/resolv.conf
nameserver 127.0.0.1 nameserver 208.67.222.222 # OPENDNS.COM nameserver 208.67.220.220 # OPENDNS.COM nameserver 12.189.32.61 # Wildblue dns
I am assuming that it will go through the local cache first, then drop down through the servers specified sequentially until it hits? Am I right in that assumption?
Your computer will try using the first DNS server in the list (first being the topmost). If it gets an answer, even a "there is no answer for your query" type of answer, it won't ask anything else. But if a DNS server is unresponsive, after the time out period (which can be quite long), it'll start using the next server. And slowly walk down the list trying each in turn. The next query will do the same (start from the top, and work down). Because of that delay, you don't want a bad entry in your list.
This computer finds the new .com web address, but from my daughters Mac and the others on the Wildblue dns they still have to enter the address numerically. The hosting company said it should work within 48 hours, worst case 72 ... Mine worked almost immediately.
What's this "new" address? One recently registered? The company is right that some things will take a while to trickle through. Some DNS servers cache things longer than they should, including "no answer" results. And if your query goes through them in the middle, it's a while before you get the right answers.
If you want to check sooner, you can do a dig query against the name server actually hosting the records. That should work straight away, unless they're slack and not enterred the data.
e.g. dig example.com @a.iana-servers.net
Tim wrote:
What's this "new" address? One recently registered? The company is right that some things will take a while to trickle through. Some DNS servers cache things longer than they should, including "no answer" results. And if your query goes through them in the middle, it's a while before you get the right answers.
If you want to check sooner, you can do a dig query against the name server actually hosting the records. That should work straight away, unless they're slack and not enterred the data.
e.g. dig example.com @a.iana-servers.net
My daughter finished the new web page and transferred service to a new host. There was some confusion when it would display on this computer with it's different dns than on the three others we tried and it took me a while to realize she was talking about a dns problem.
She questioned the hosting company and they pointed out to her that the numerical address should work, then I understood, should have tried that myself but wasn't thinking clearly, if at all?
The result of her effort is at http://finepianoservices.com/index.html should you be interested. It took nearly 48 hours to show up on the other computers using the Wildblue dns.
Bob Goodwin
Tim:
What's this "new" address? One recently registered? The company is right that some things will take a while to trickle through.
Bob Goodwin:
My daughter finished the new web page and transferred service to a new host. There was some confusion when it would display on this computer with it's different dns than on the three others we tried and it took me a while to realize she was talking about a dns problem.
She questioned the hosting company and they pointed out to her that the numerical address should work, then I understood, should have tried that myself but wasn't thinking clearly, if at all?
That'll only work if the IP isn't used as a host for many different websites, that are only distinguished by the hostname in your connection attempt (named-based virtual hosting). In your case, it does appear to work. So either you've got your very own IP (most likely), or yours is the default for a virtual host carrying many (very unlikely).
The result of her effort is at http://finepianoservices.com/index.html should you be interested. It took nearly 48 hours to show up on the other computers using the Wildblue dns.
Most people use their ISP's DNS server, and it can add propagation delays. If you run your own, it should go directly to the authoritative source, and get the most current address (as long as whatever's it's currently been cached for it hasn't yet expired).
It's working here. I saw a few nifty animations. I got a bit a surprise by one link opening a new browser (the restoration gallery), which probably not a good idea for a framed site (even a psuedo framed site). That's probably as far as I should go critiquing something, I could probably pick any site to death.
But, otherwise, interesting to look at. It's been years since I played the piano, it hurts the fingers too much. These days I stick to playing the organ, the keyboard requires less effort to play, it's much easier on the joints. And I can maintain the electronics, myself (up to a point). ;-)
On Sun, 2007-04-29 at 20:31 +0800, Fennix wrote:
From your message I have tried running the mdc command but it is not found. Tried to 'locate mdc' and no useful result.
It was "rndc" (the lower case letters R N D C). I suspect you have a less than useful font displaying your mails, mine only just shows the rn as rn rather than as m. Cut and paste would have worked, though.
Tim wrote:
On Sat, 2007-04-28 at 10:54 -0400, Bob Goodwin - W2BOD wrote:
Is there a file I can examine that will show me what URL's are presently cached in my caching-nameserver ?
FQDNs, not URIs, as Bruno pointed out. But check out the "rndc" command. That man file for it is lacking, but running the command without any parameters produces this:
# rndc Usage: rndc [-c config] [-s server] [-p port] [-k key-file ] [-y key] [-V] command
command is one of the following:
reload Reload configuration file and zones. reload zone [class [view]] Reload a single zone. refresh zone [class [view]] Schedule immediate maintenance for a zone. retransfer zone [class [view]] Retransfer a single zone without checking serial number. freeze zone [class [view]] Suspend updates to a dynamic zone. thaw zone [class [view]] Enable updates to a frozen dynamic zone and reload it. reconfig Reload configuration file and new zones only. stats Write server statistics to the statistics file. querylog Toggle query logging. dumpdb [-all|-cache|-zones] [view ...] Dump cache(s) to the dump file (named_dump.db). stop Save pending updates to master files and stop the server. stop -p Save pending updates to master files and stop the server reporting process id. halt Stop the server without saving pending updates. halt -p Stop the server without saving pending updates reporting process id. trace Increment debugging level by one. trace level Change the debugging level. notrace Set debugging level to 0. flush Flushes all of the server's caches. flush [view] Flushes the server's cache for a view. flushname name [view] Flush the given name from the server's cache(s) status Display status of the server. recursing Dump the queries that are currently recursing (named.recursing) *restart Restart the server.
- == not yet implemented
Version:
Note the "dumpdb" information, around about the middle. There's also statistics options which give you a summary about what it's got, without masses of explicit detail.
Tim:
rndc dumpdb -cache Produces a lot of data in /var/named/chroot/var/named/data/cache_dump.db. It looks interesting and seems to be related to places I have been so I assume that it is either what is in the nameserver cache when the command was given or is it a history of what has been?
And then the notation "ttl" I assume is time to live for that record, is it expressed in seconds or some arbitrary unit peculiar to the computer? Where is the ttl value assigned, by the creator or is it a value determined in this computer?
"Unassociated entries" Is it caching addresses that were received in numeric form?
I'm sure there are many questions, some will yield to google I suppose ...
Thanks.
Bob Goodwin
--------- --------- --------
; cbru.br.ns.els-gms.att.net [v4 TTL 39566] [v4 success] [v6 unexpected] ; 199.191.128.105 [srtt 12] [flags 00000000] [ttl -7638] ; udns1.ultradns.net [v4 TTL 39688] [v4 success] [v6 unexpected] ; 204.69.234.1 [srtt 333975] [flags 00000000] [ttl -44908] ; C.ROOT-SERVERS.NET [v4 TTL 37456] [v4 not_found] [v6 unexpected] ; 192.33.4.12 [srtt 6676070] [flags 00000000] [ttl -43329] ; M.ROOT-SERVERS.NET [v4 TTL 37456] [v4 not_found] [v6 unexpected] ; 202.12.27.33 [srtt 6542549] [flags 00000000] [ttl -43329] ; ; Unassociated entries ; ; 64.233.179.9 [srtt 1493225] [flags 00000000] [ttl 1321] ; 209.85.139.9 [srtt 1651330] [flags 00000000] [ttl 1321]
On Sun, 2007-04-29 at 16:45 -0400, Bob Goodwin - W2BOD wrote:
rndc dumpdb -cache Produces a lot of data in /var/named/chroot/var/named/data/cache_dump.db. It looks interesting and seems to be related to places I have been so I assume that it is either what is in the nameserver cache when the command was given or is it a history of what has been?
It's a history of what BIND has done (what it's cached). Addresses its resolved, and everything it had to do to resolve them.
And then the notation "ttl" I assume is time to live for that record, is it expressed in seconds or some arbitrary unit peculiar to the computer?
I don't know, but TTL is done in seconds in DNS records, so I expect this is the same.
Where is the ttl value assigned, by the creator or is it a value determined in this computer?
The original DNS records have it as a parameter. A proper host lets you set these things, many just let you put A, CNAME and MX records into the zone, and you're stuck with them setting other parameters.
If the record is accessed through another server, it may muck around with them, and return different values (they shouldn't do that, but some might).
"Unassociated entries" Is it caching addresses that were received in numeric form?
Don't know off the top of my head.
-
Bob Goodwin -
Bruno Wolff III -
Fennix -
Tim