Whenever I mistype a URL, it gets turned into food for Midcontinent's search engine. What is really annoying is that the back button does not take me back to what I typed. Also, the search results rarely includes the site I wanted.
My suspicion is that this is done at least partly through DNS corruption. Anyone know for sure?
I'm running F33 connected directly to Midco's modem/router. Assuming DNS corruption is at least part of the issue, how, if at all, do I bypass Midco's DNS?
The problem seems not unique to Midco or F33: My girlfriend has the same problem with CenturyLink and Ubuntu. They send her to yahoo.
On Tue, 19 Jan 2021 15:08:33 -0600 (CST) Michael Hennebry hennebry@web.cs.ndsu.nodak.edu wrote:
Whenever I mistype a URL, it gets turned into food for Midcontinent's search engine. What is really annoying is that the back button does not take me back to what I typed. Also, the search results rarely includes the site I wanted.
My suspicion is that this is done at least partly through DNS corruption. Anyone know for sure?
I think your ISP is monetizing mistypes. It is the same thing that people who park websites that are close to popular websites, say one keystroke off, do by putting ads on those pages. Every time someone mistypes, they earn ad money. In your ISP case, they are checking for invalid domain names, and instead of sending a 404, they send you to a site that earns them money.
I'm running F33 connected directly to Midco's modem/router. Assuming DNS corruption is at least part of the issue, how, if at all, do I bypass Midco's DNS?
I did this by installing knotresolver (there are other similar applications), telling it to use another DNS site, turning off NetworkManager management of DNS as part of dhcp, and turning off the default DNS servers from the ISP in the router. This bit is critical. If you don't have access to that in the interface to your router, I don't think you can make this work.
I know that things have changed significantly with F33, and systemd-resolved becoming the default DNS manager, so the whole process might be different for you. But with some sleuthing, you *can* bypass your ISP's DNS servers (if your router allows it).
On Tue, 2021-01-19 at 15:08 -0600, Michael Hennebry wrote:
Whenever I mistype a URL, it gets turned into food for Midcontinent's search engine. What is really annoying is that the back button does not take me back to what I typed. Also, the search results rarely includes the site I wanted.
My suspicion is that this is done at least partly through DNS corruption. Anyone know for sure?
When you type in an address your browser first tries to find its IP address, then it connects to that IP and asks whatever's at that IP for the hostname you're wanting to view.
That IP lookup can be done through your browser requesting DNS lookups, or if there's a proxy involved, the proxy can instigate the lookup. These days there can also be DNS queries through HTTP, and I'm not sure how you'd bypass this kind of issue in that situation.
I'm running F33 connected directly to Midco's modem/router. Assuming DNS corruption is at least part of the issue, how, if at all, do I bypass Midco's DNS?
You can run your own DNS server, though that presumes your ISP won't intercept its attempts to resolve domain names. There are some relatively pain-free packages you can install to do this for you. And you need to configure your network to use your own DNS server.
You can configure your network to use an outside server (such as google's at 8.8.8.8 and 8.8.4.4, and there's plenty of other open DNS servers). Again, that presumes that your ISP won't intercept attempts to consult it. Also, when using an outside DNS server, it can be a problem if you want to resolve domain names within your LAN (if you have a network of computers and other network devices). You have to do that another way.
On Wed, 2021-01-20 at 11:35 +1030, Tim via users wrote:
Also, when using an outside DNS server, it can be a problem if you want to resolve domain names within your LAN (if you have a network of computers and other network devices). You have to do that another way.
Supplemental: It can also be a problem, with some ISPs, for accessing their services. They may give customers on their own network different IPs to connect to for their own services (mail, etc).
Though I expect that kind of thing is not as prevalent as it used to be. A lot of people don't use their ISP's mail service anymore, they use something like gmail, or their phone. So they've got to be able to carry on using services when not directly using their ISP.
I think the proper term is DNS hijacking. Corruption generally implies accidental in computer terms, not by design.
CenturyLink also does dns hijacking. Not sure how many other slimy ISP's there are out there. MidCo is my other ISP choice where I am.
CenturyLink sends me to an amazingly bad Mediacom search engine and I have not figured out why the dns servers that CenturyLink gives me via dhcp send me to mediacom (a cable company), and apparently a competitor, unless Mediacom is paying a bit for each lookup or providing the dns service.
You can either use someone else dns servers that do not pull this crap, or go to the dns via https via the browser.
On Tue, Jan 19, 2021 at 3:08 PM Michael Hennebry hennebry@web.cs.ndsu.nodak.edu wrote:
Whenever I mistype a URL, it gets turned into food for Midcontinent's search engine. What is really annoying is that the back button does not take me back to what I typed. Also, the search results rarely includes the site I wanted.
My suspicion is that this is done at least partly through DNS corruption. Anyone know for sure?
I'm running F33 connected directly to Midco's modem/router. Assuming DNS corruption is at least part of the issue, how, if at all, do I bypass Midco's DNS?
The problem seems not unique to Midco or F33: My girlfriend has the same problem with CenturyLink and Ubuntu. They send her to yahoo.
-- Michael hennebry@web.cs.ndsu.NoDak.edu "Sorry but your password must contain an uppercase letter, a number, a haiku, a gang sign, a heiroglyph, and the blood of a virgin." -- someeecards _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
On Wed, 2021-01-20 at 06:46 -0600, Roger Heflin wrote:
I think the proper term is DNS hijacking.
No, that's a malicious thing. Like someone trying to commit fraud. This sort of misdirection is better described as DNS manipulation.
Anyway, apart from inconvenience, it can lead to all sorts of networking nightmares. Things on your computer, or whole network, can't easily tell that you haven't actually got what you wanted when you don't get true error messages.
It's a disaster waiting to happen, and there are far better ways to help users who've got an address wrong. Most web browsers, by now, let you search from the address bar. So, if you don't get the address right, you're likely to end at a *NORMAL*, and useful, search engine page that will help you find what you're after.
Apparently BIND and DNSmasq have options to filter out these bogus results. Google has open DNS servers that don't return spoofed results.
On Wed, Jan 20, 2021 at 10:53:34AM -0700, Joe Zeff wrote:
dns servers that CenturyLink gives me via dhcp
Stop using dhcp.
(Note that you can still use dhcp to get your IP address, just then ignore the DNS servers.)
On 1/20/21 12:37 PM, Matthew Miller wrote:
On Wed, Jan 20, 2021 at 10:53:34AM -0700, Joe Zeff wrote:
dns servers that CenturyLink gives me via dhcp
Stop using dhcp.
(Note that you can still use dhcp to get your IP address, just then ignore the DNS servers.)
Checking, you can set both IPV4 and IPV6 to use DHCP only for your IP address and set your DNS servers manually.
On Wed, 20 Jan 2021, Joe Zeff wrote:
Checking, you can set both IPV4 and IPV6 to use DHCP only for your IP address and set your DNS servers manually.
Thanks folks, not just JZ.
I expect doing that is not a new thing and directions for it already exist somewhere. Would someone be kind enough to point me to them?
Also, I rather like the idea of my F33 also be its DNS server. Mostly I would have it punt to 8.8.8.8 or something. doubleclick would die.
I'm going to be away from my computer for a while, so it might be some time before I actually try anything.
If you control the router, you can override the settings to use a set you specify.
That way all hosts on the network will use those.
On Sat, Jan 23, 2021 at 9:25 AM Michael Hennebry hennebry@web.cs.ndsu.nodak.edu wrote:
On Wed, 20 Jan 2021, Joe Zeff wrote:
Checking, you can set both IPV4 and IPV6 to use DHCP only for your IP address and set your DNS servers manually.
Thanks folks, not just JZ.
I expect doing that is not a new thing and directions for it already exist somewhere. Would someone be kind enough to point me to them?
Also, I rather like the idea of my F33 also be its DNS server. Mostly I would have it punt to 8.8.8.8 or something. doubleclick would die.
I'm going to be away from my computer for a while, so it might be some time before I actually try anything.
-- Michael hennebry@web.cs.ndsu.NoDak.edu "Sorry but your password must contain an uppercase letter, a number, a haiku, a gang sign, a heiroglyph, and the blood of a virgin." -- someeecards _______________________________________________ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
On Sat, 2021-01-23 at 09:25 -0600, Michael Hennebry wrote:
Also, I rather like the idea of my F33 also be its DNS server. Mostly I would have it punt to 8.8.8.8 or something. doubleclick would die.
If you install your own DNS server, like BIND, it works like a DNS server ought to. Walking up the tree to get records.
There's little point in installing a server if you're just going to forward all queries to another DNS server, you might as well make your life easier by just configuring your network to use that server.
But, yes, you can blackhole various annoying domain names so that they fail quickly. I've done that for many years with BIND.
On 1/23/21 12:57 PM, Tim via users wrote:
But, yes, you can blackhole various annoying domain names so that they fail quickly. I've done that for many years with BIND.
And, if you're not hosting your own DNS, you can use /etc/hosts to do the same thing on a machine by machine basis. Of course, this isn't practical if you're running a large LAN, but it's just great for a home user.
Tim:
But, yes, you can blackhole various annoying domain names so that they fail quickly. I've done that for many years with BIND.
Joe Zeff:
And, if you're not hosting your own DNS, you can use /etc/hosts to do the same thing on a machine by machine basis. Of course, this isn't practical if you're running a large LAN, but it's just great for a home user.
I used to do that, but using the hosts file only leaves you with two choices: Give annoying domains a wrong IP to connect to that either tries to load non-existent files from a real server (wasting traffic and filling logs), or tries to connect to a server that isn't there (and waits a long time for a timeout).
For many years I've used a zone file in my DNS server that instantly returns a "no answer" for any forbidden domains (and subdomains), that kills annoyances without any delays.
Have you ever tried to load a page, only for there to be some big long delay before parts of the page loaded? The server is waiting for something before it continues. On pages full of crap, that can add up to a long wait. Removing those delays is as important as removing the annoying adds (and malware).
There are proxies and plugins that can do the same kind of thing of removing annoying content from webpages. But I found that my DNS deadzone that I've pointed about a dozen domain names at has worked well for many years without needing updating all the time.
On 1/23/21 9:30 PM, Tim via users wrote:
Tim:
But, yes, you can blackhole various annoying domain names so that they fail quickly. I've done that for many years with BIND.
Joe Zeff:
And, if you're not hosting your own DNS, you can use /etc/hosts to do the same thing on a machine by machine basis. Of course, this isn't practical if you're running a large LAN, but it's just great for a home user.
I used to do that, but using the hosts file only leaves you with two choices: Give annoying domains a wrong IP to connect to that either tries to load non-existent files from a real server (wasting traffic and filling logs), or tries to connect to a server that isn't there (and waits a long time for a timeout).
I point them to some variation on a localhost IP, e.g. 127.x.x.x. I'm not running a web server, so it immediately fails with can't connect. No delays, no traffic, and doesn't bother anyone.
On 1/23/21 10:30 PM, Tim via users wrote:
I used to do that, but using the hosts file only leaves you with two choices: Give annoying domains a wrong IP to connect to that either tries to load non-existent files from a real server (wasting traffic and filling logs), or tries to connect to a server that isn't there (and waits a long time for a timeout).
Or, you do the sensible thing and point said domain to 127.0.0.1, so that it times out almost instantly.
On Sun, 2021-01-24 at 00:07 -0700, Joe Zeff wrote:
Or, you do the sensible thing and point said domain to 127.0.0.1, so that it times out almost instantly.
It doesn't. The web browser waits for something to answer it. Go on, try to get your web browser to connect to a non-existent server, it doesn't immediately stop looking. If you load up a page that might try waiting for a dozen different things before it will proceed and let you read something, it's a pain.
And, in my case, I do have a local webserver running.
On Sun, Jan 24, 2021, at 3:21 AM, Tim via users wrote:
On Sun, 2021-01-24 at 00:07 -0700, Joe Zeff wrote:
Or, you do the sensible thing and point said domain to 127.0.0.1, so that it times out almost instantly.
It doesn't. The web browser waits for something to answer it. Go on, try to get your web browser to connect to a non-existent server, it doesn't immediately stop looking.
My hosts file setup is working very quickly.
I can't remember why I stopped pointing to 127.* but something had me change to:
route -n
Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface 0.0.0.0 192.168.100.254 0.0.0.0 UG 600 0 0 wlp0s26f7u3 111.72.252.91 - 255.255.255.255 !H 0 - 0 - [snip]
So you see that "111.72.252.91" is null routed.
In my hosts file I point to that IP:
grep "^111.72.252.91" /etc/hosts | wc -l
11811
Using a sample name from that list I get a very quick fail with firefox:
Unable to connect
Firefox can’t establish a connection to the server at xads.zedo.com.
The site could be temporarily unavailable or too busy. Try again in a few moments. If you are unable to load any pages, check your computer’s network connection. If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.
On 1/24/21 3:21 AM, Tim via users wrote:
On Sun, 2021-01-24 at 00:07 -0700, Joe Zeff wrote:
Or, you do the sensible thing and point said domain to 127.0.0.1, so that it times out almost instantly.
It doesn't. The web browser waits for something to answer it. Go on, try to get your web browser to connect to a non-existent server, it doesn't immediately stop looking. If you load up a page that might try waiting for a dozen different things before it will proceed and let you read something, it's a pain.
That's why you don't use a non-existent server. localhost exists, so you get an immediate rejection.
And, in my case, I do have a local webserver running.
That could be a problem. It depends on if it answers to only 127.0.0.1 or any localhost IP.
Tim:
It doesn't. The web browser waits for something to answer it. Go on, try to get your web browser to connect to a non-existent server, it doesn't immediately stop looking. If you load up a page that might try waiting for a dozen different things before it will proceed and let you read something, it's a pain.
Samuel Sieb:
That's why you don't use a non-existent server. localhost exists, so you get an immediate rejection.
You can only get an immediate *rejection* if there's something there to reject it. If you don't have a webserver listening on the HTTP and HTTPS ports, and don't have a firewall rejecting connections, then the browser is going to wait for a connection, that's how it works. A rejection is very different from a no response (yet).
Surely all of you have sat there looking at the page loading spinner at some stage, wondering if a page is ever going to show up?
If you try browsing to a non-existent site, you're going to get a quickish failure. That's down to a quick domain doesn't exist DNS server response. If you try to connect to a domain that exists, but no server is responding (e.g. site failure) then you're going to get a delay while the browser waits.
If you're browsing an existing site, and that page pulls in things from a plethora of different services, which is what most ad-laden sites do, every one of those is a connection that will try to connect and wait for a timeout. If you're browsing a site that tries to stop you looking at the pages until the extra content has loaded, or is simply not authored well enough to load progressively, you'll get those delays compounding together.
On 25/01/2021 11:52, Tim via users wrote:
Tim:
It doesn't. The web browser waits for something to answer it. Go on, try to get your web browser to connect to a non-existent server, it doesn't immediately stop looking. If you load up a page that might try waiting for a dozen different things before it will proceed and let you read something, it's a pain.
Samuel Sieb:
That's why you don't use a non-existent server. localhost exists, so you get an immediate rejection.
You can only get an immediate*rejection* if there's something there to reject it.
Really? I just told my browser to connect to https://127.0.5.6/%C2%A0 and it came back immediately with...
This site can’t be reached 127.0.5.6 refused to connect.
Am I missing something?
On 1/24/21 7:52 PM, Tim via users wrote:
Tim:
It doesn't. The web browser waits for something to answer it. Go on, try to get your web browser to connect to a non-existent server, it doesn't immediately stop looking. If you load up a page that might try waiting for a dozen different things before it will proceed and let you read something, it's a pain.
Samuel Sieb:
That's why you don't use a non-existent server. localhost exists, so you get an immediate rejection.
You can only get an immediate *rejection* if there's something there to reject it. If you don't have a webserver listening on the HTTP and HTTPS ports, and don't have a firewall rejecting connections, then the browser is going to wait for a connection, that's how it works. A rejection is very different from a no response (yet).
That's not correct. If there's a computer at the IP address and nothing is listening on the port and the firewall is not configured to drop packets, then you will get an immediate reply saying that no one's home. If there's no computer at the IP address and it's not on your local subnet, then TCP will keep trying until it times out which will be quite a while. If there's a computer at the IP address and the firewall is configured to drop packets, same thing.
localhost is the first case. There's a computer at the IP address, nothing listening (if you're not running a web server on your computer), and no firewall. So you get an immediate reply that you can't connect, no delays at all.
Back at last. I've logged into the router, but I'm not at all sure how to do what I want. For now, I'd settle for bypassing midco's router. Would turning off dynamic DNS help? I expect I'd have to tell something what DNS server to use. The router is labeled ARRIS and is at 192.168.0.1 . Anyone else stuck with it?
On Tue, 2021-02-23 at 21:12 -0600, Michael Hennebry wrote:
Back at last. I've logged into the router, but I'm not at all sure how to do what I want. For now, I'd settle for bypassing midco's router.
*Often* you *can* ignore an ISP's router, remove it and substitute your own.
Would turning off dynamic DNS help?
No, that's a different thing (unless they've named something badly). It's for giving you a public domain name that points to your IP, and handles updating your domain name record whenever your IP changes (that's the dynamic part of it). People use things like that so they can connect to their computer when they're away, and not have to know its current IP address.
I expect I'd have to tell something what DNS server to use. The router is labeled ARRIS and is at 192.168.0.1 .
Perhaps you should search: https://www.google.com/search?&q=arris+router+configuration https://www.google.com/search?q=arris+router+dns+settings
1. Log into your router's IP address at 192.168.0.1, use the admin/password (default password is password)
2. Go to LAN Setup Tab
3. Scroll down to DNS Override
4. Enable the feature by clicking the check mark option at that section
5. Enter the DNS IP addresses you want to use into the fields.
6. Save your settings.
Of course I got that information without knowing your particular model, so the instructions may need some interpretation.
You can set domain name server IPs of your own choosing. They could be your ISP's, you could use Google's (8.8.8.8 & 8.8.4.4). There are anti-censorship servers, filtered servers (removing ads or other unwanted content).
https://www.google.com/search?&q=public+dns+servers
Have a look at this page: https://www.lifewire.com/free-and-public-dns-servers-2626062
On 24/02/2021 11:12, Michael Hennebry wrote:
I've logged into the router, but I'm not at all sure how to do what I want. For now, I'd settle for bypassing midco's router. Would turning off dynamic DNS help? I expect I'd have to tell something what DNS server to use. The router is labeled ARRIS and is at 192.168.0.1 . Anyone else stuck with it?
I don't recall what issue you're trying to resolve. However, if you want your Fedora system to use a different DNS server/service than your router is supplying via DHCP it is easy to accomplish.
If you're using the NetworkManager GUI you can select a "Method" of "Automatic (Addresses Only)". This will result in your system getting its IP address assignment from the router as it normally would. But, you'd manually enter the DNS server you want to use.
Examples of public DNS servers are....
Cloudflare: 1.1.1.1 and 1.0.0.1 Google: 8.8.8.8 and 8.8.4.4
Then, assuming your ISP doesn't block using them, you'd be using a public DNS server.
On 24/2/21 20:13, Ed Greshko wrote:
On 24/02/2021 11:12, Michael Hennebry wrote:
I've logged into the router, but I'm not at all sure how to do what I want. For now, I'd settle for bypassing midco's router. Would turning off dynamic DNS help? I expect I'd have to tell something what DNS server to use. The router is labeled ARRIS and is at 192.168.0.1 . Anyone else stuck with it?
I don't recall what issue you're trying to resolve. However, if you want your Fedora system to use a different DNS server/service than your router is supplying via DHCP it is easy to accomplish.
If you're using the NetworkManager GUI you can select a "Method" of "Automatic (Addresses Only)". This will result in your system getting its IP address assignment from the router as it normally would. But, you'd manually enter the DNS server you want to use.
Examples of public DNS servers are....
Cloudflare: 1.1.1.1 and 1.0.0.1 Google: 8.8.8.8 and 8.8.4.4
Then, assuming your ISP doesn't block using them, you'd be using a public DNS server.
Just further to this check the wifi settings in your router, they may let you turn off using your ISP's dns servers and hard code your own servers using the address supplied by Ed. That way you can change what dns servers you are using without having to change anything in Fedora.
regards. Steve
On Wed, 24 Feb 2021, Ed Greshko wrote:
On 24/02/2021 11:12, Michael Hennebry wrote:
I've logged into the router, but I'm not at all sure how to do what I want. For now, I'd settle for bypassing midco's router. Would turning off dynamic DNS help? I expect I'd have to tell something what DNS server to use. The router is labeled ARRIS and is at 192.168.0.1 . Anyone else stuck with it?
In other people's posts, I've occasionally seen what I think was supposed to be whitespace as question marks in black diamonds.. This time my own blanks came back to me that way. They stay that way in alpines default editor. Using its vim editor, I see blue vertical strokes followed by blanks. Any idea what is happening?
I suspect it might have something to do with me using alpine through ssh. That said, I thought blanks were immune to that sort of thing.
On Wed, 2021-02-24 at 08:17 -0600, Michael Hennebry wrote:
In other people's posts, I've occasionally seen what I think was supposed to be whitespace as question marks in black diamonds.. This time my own blanks came back to me that way. They stay that way in alpines default editor. Using its vim editor, I see blue vertical strokes followed by blanks. Any idea what is happening?
I suspect it might have something to do with me using alpine through ssh. That said, I thought blanks were immune to that sort of thing.
It may well depend on what those white spaces are.
If they're tabs or linebreaks, or a person has sent a non-breaking space (sometimes by accident, as they've hit shift+space), they can do funny things through email. Especially when messages get transcoded (7-bit to 8-bit, utf-8, quoted-printable, format=flowed [with ignorable but marked linewrapping], plain text, HTML, etc).
Seeing as you mention SSH, is your encoding scheme the same on both ends (server and client)? That kind of thing is common with mangled UTF-8 coding/decoding.
And it can be your email client. Mine (evolution) seems to stuff up on some content, and the dots in domain names sometimes disappear when it renders the text (breaking links). I don't know if it's screwing up decoding, or the original messages were badly encoded. The raw email looks fine, I can see them in there.
On Wed, 24 Feb 2021 at 10:18, Michael Hennebry < hennebry@web.cs.ndsu.nodak.edu> wrote:
[...] In other people's posts, I've occasionally seen what I think was supposed to be whitespace as question marks in black diamonds.. This time my own blanks came back to me that way. They stay that way in alpines default editor. Using its vim editor, I see blue vertical strokes followed by blanks. Any idea what is happening?
I suspect it might have something to do with me using alpine through ssh. That said, I thought blanks were immune to that sort of thing.
If the original text was Unicode, there are many new possible types of "blanks": n-space, m-space, non-breaking space, etc https://jkorpela.fi/chars/spaces.html. Conversions to other encodings often have problems. This can be particularly annoying in PDF handouts were cutting and pasting a block of code into an editor looks "correct" but won't compile/execute. It is one more reason to avoid spaces in file and directory names that may be shared with others.
On 24/02/2021 22:17, Michael Hennebry wrote:
On Wed, 24 Feb 2021, Ed Greshko wrote:
On 24/02/2021 11:12, Michael Hennebry wrote:
I've logged into the router, but I'm not at all sure how to do what I want. For now, I'd settle for bypassing midco's router. Would turning off dynamic DNS help? I expect I'd have to tell something what DNS server to use. The router is labeled ARRIS and is at 192.168.0.1 . Anyone else stuck with it?
In other people's posts, I've occasionally seen what I think was supposed to be whitespace as question marks in black diamonds.. This time my own blanks came back to me that way. They stay that way in alpines default editor. Using its vim editor, I see blue vertical strokes followed by blanks. Any idea what is happening?
I suspect it might have something to do with me using alpine through ssh. That said, I thought blanks were immune to that sort of thing.
You really should have started a whole new thread on this subject instead of hijacking your own post. All the messages with this subject are lumped in with the DNS thread and my exists here with nothing that I wrote and even then the quoted section is irrelevant to this display issue.