Hello!
I have just pushed a change, which modifies default configuration of dnsmasq package. I think it should not create any regressions, but it may happen in few special cases.
What has changed:
It is described in bug #2258062 [1]. It no longer contains in default configuration bind-interfaces and interface=lo options. Instead, it contains now local-service=host. Exact change is in PR #15 [2].
It should work exactly the same way. Except if you configure interface explicitly, for example by executing
echo interface=eth0 > /etc/dnsmasq.d/iface.conf
local-service=host will deactivate itself. It is active only when dnsmasq is otherwise unconfigured. When interface or listen-address option is used, it deactivates and behaves like commented out.
That might create a regression in special case. If you are running by default systemd-resolved, it listens already on domain port on address 127.0.0.53 address. But if bind-interfaces or bind-dynamic is not used explicitly, dnsmasq will try to listen on wildcard address 0.0.0.0 and just filter incoming requests, accepting only those arriving on interface eth0. But if any service already listens on port domain, it will fail to listen on it and fail to start.
Solution is simple: add your own explicit bind-interfaces, because it is not present by default. Use instead this:
(echo bind-interfaces; echo interface=eth0) > /etc/dnsmasq.d/iface.conf
There are some packages, which depends on dnsmasq in various way. libvirt or NetworkManager should not be affected, but won't hurt checking it. I would like to ask their maintainers depending on dnsmasq to check this change will not break their functionality and cause regressions. Use bug [1] to report problems found with this change, don't be afraid to set needinfo? on me.
Unless some regression appears, I would like to backport this change also into Fedora 39 stable after two weeks without reported regressions.
Thank you for reading it so far!
Best Regards, Petr
1. https://bugzilla.redhat.com/show_bug.cgi?id=2258062 2. https://src.fedoraproject.org/rpms/dnsmasq/pull-request/15
Petr Menšík wrote:
That might create a regression in special case. If you are running by default systemd-resolved, it listens already on domain port on address 127.0.0.53 address. But if bind-interfaces or bind-dynamic is not used explicitly, dnsmasq will try to listen on wildcard address 0.0.0.0 and just filter incoming requests, accepting only those arriving on interface eth0. But if any service already listens on port domain, it will fail to listen on it and fail to start.
But we run systemd-resolved by default these days, don't we? So making dnsmasq attempt by default to serve the same requests does not sound like a good idea to me.
On a server I administer for work, I have dnsmasq serving the DNS for an ocserv (OpenConnect) VPN, listening only on the VPN interface. Any request for a host not within the VPN network (coming in from clients with no or broken split DNS support, e.g., old GNU/Linux distros without systemd- resolved, or Windows, where the OpenConnect client is still unable to set up split DNS) is forwarded to systemd-resolved, which in turn forwards it to the upstream DNS from the datacenter. Relying instead on the filtering would not have worked exactly for the reason you describe above. But that server is not running Fedora anyway.
Kevin Kofler
systemd-resolved is unfortunately known to broken. I would recommend having systemd-resolved forwarded to dnsmasq, which can then be forwarded further.
Dnsmasq does not break DNSSEC, systemd-resolved does. But this change should create conflict with systemd-resolved only in case it was improperly configured. In all other cases, it should work independently on the same machine just fine.
Anyway, dnsmasq will listen by default on 127.0.0.1, as every standard resolver does. You can use listen-address=127.0.0.53 if you like, but then it will conflict with systemd-resolved.
On 14. 01. 24 1:57, Kevin Kofler via devel wrote:
Petr Menšík wrote:
That might create a regression in special case. If you are running by default systemd-resolved, it listens already on domain port on address 127.0.0.53 address. But if bind-interfaces or bind-dynamic is not used explicitly, dnsmasq will try to listen on wildcard address 0.0.0.0 and just filter incoming requests, accepting only those arriving on interface eth0. But if any service already listens on port domain, it will fail to listen on it and fail to start.
But we run systemd-resolved by default these days, don't we? So making dnsmasq attempt by default to serve the same requests does not sound like a good idea to me.
No, dnsmasq will serve addresses 127.0.0.1 and ::1. If you do not want it listening on wildcard address, please configure it explicitly by using bind-interfaces or bind-dynamic options. Otherwise make sure no other program listens on port 53 (domain), be it systemd-resolved, named, unbound or anything similar.
On a server I administer for work, I have dnsmasq serving the DNS for an ocserv (OpenConnect) VPN, listening only on the VPN interface. Any request for a host not within the VPN network (coming in from clients with no or broken split DNS support, e.g., old GNU/Linux distros without systemd- resolved, or Windows, where the OpenConnect client is still unable to set up split DNS) is forwarded to systemd-resolved, which in turn forwards it to the upstream DNS from the datacenter. Relying instead on the filtering would not have worked exactly for the reason you describe above. But that server is not running Fedora anyway.
Kevin Kofler--
Unfortunately broken are clients having systemd-resolved enabled.
I would recommend to skip systemd-resolved stub and using resolv-file=/run/systemd/resolve/resolv.conf
in such case. It would use servers configured by systemd-resolved, but without using broken port domain at address 127.0.0.53. Alternatively use server=127.0.0.54, which should not break incoming queries so much.
Consider using unbound as a cache for other VPN clients. dnsmasq is great for its integration with DHCP server, but is targeted to use minimal resources. Unfortunately at cost of some design issues. Unbound is a high quality cache, while still relatively small compared to bind's named.service.
Cheers, Petr
Petr Menšík wrote:
systemd-resolved is unfortunately known to broken.
[snip]
Dnsmasq does not break DNSSEC, systemd-resolved does.
[snip]
Unfortunately broken are clients having systemd-resolved enabled.
How exactly is it broken? If you refer to: https://github.com/systemd/systemd/issues/25676 fixes for that are finally coming in now (as of 3 weeks ago).
I would recommend having systemd-resolved forwarded to dnsmasq, which can then be forwarded further.
If you think dnsmasq should replace systemd-resolved by default, then please propose that through the Changes process, which will also ensure the glibc resolver, NetworkManager, and the like get configured properly for it. Simply shipping dnsmasq with a default configuration that conflicts with systemd-resolved is not a productive approach.
If systemd-resolved is really broken, then it either needs to be fixed or replaced. The former needs to be handled through systemd upstream, the latter through the Fedora Changes process.
But this change should create conflict with systemd-resolved only in case it was improperly configured.
But the default configuration you ship will conflict.
Anyway, dnsmasq will listen by default on 127.0.0.1, as every standard resolver does. You can use listen-address=127.0.0.53 if you like, but then it will conflict with systemd-resolved.
You just wrote that you make it listen by default on all interfaces, and then filter. This means it will conflict over the port 53. That said, listening on the lo interface only will also conflict with systemd-resolved or any other local resolver, so you are probably right that your change does not change much for the default configuration, it just makes it harder (more settings to change) to set up coexistence. 127.0.0.53 is unfortunately not an independent interface, it is still the lo interface.
On 14. 01. 24 1:57, Kevin Kofler via devel wrote:
On a server I administer for work, I have dnsmasq serving the DNS for an ocserv (OpenConnect) VPN, listening only on the VPN interface. Any request for a host not within the VPN network (coming in from clients with no or broken split DNS support, e.g., old GNU/Linux distros without systemd- resolved, or Windows, where the OpenConnect client is still unable to set up split DNS) is forwarded to systemd-resolved, which in turn forwards it to the upstream DNS from the datacenter. Relying instead on the filtering would not have worked exactly for the reason you describe above. But that server is not running Fedora anyway.
I would recommend to skip systemd-resolved stub and using resolv-file=/run/systemd/resolve/resolv.conf
in such case. It would use servers configured by systemd-resolved, but without using broken port domain at address 127.0.0.53. Alternatively use server=127.0.0.54, which should not break incoming queries so much.
Well, I do not see a good reason to disable systemd-resolved for the server's own queries (which includes the forwarding queries from dnsmasq, if the domain is not one it knows). It just works.
Consider using unbound as a cache for other VPN clients. dnsmasq is great for its integration with DHCP server, but is targeted to use minimal resources. Unfortunately at cost of some design issues. Unbound is a high quality cache, while still relatively small compared to bind's named.service.
Using minimal resources is exactly what I want here. Which is why I do not want to use dnsmasq for what systemd-resolved can do, nor unbound for what dnsmasq can do.
And sending the server's own queries through dnsmasq is not going to work (not without a second instance, at least), because the VPN server is not a VPN client, so I have the server's /etc/hosts resolve its own domain to 127.0.0.1 (not the public IP, because services listen only on localhost and the VPN, that is what the VPN is for), which is honored by systemd-resolved, whereas my dnsmasq configuration overrides this to return the VPN IP to the VPN clients querying that same domain. Sounds hackish, but works great.
Kevin Kofler
On Mon, Jan 15, 2024 at 11:32 AM Kevin Kofler via devel devel@lists.fedoraproject.org wrote:
Petr Menšík wrote:
systemd-resolved is unfortunately known to broken.
[snip]
Dnsmasq does not break DNSSEC, systemd-resolved does.
[snip]
Unfortunately broken are clients having systemd-resolved enabled.
How exactly is it broken? If you refer to: https://github.com/systemd/systemd/issues/25676 fixes for that are finally coming in now (as of 3 weeks ago).
I would recommend having systemd-resolved forwarded to dnsmasq, which can then be forwarded further.
If you think dnsmasq should replace systemd-resolved by default, then please propose that through the Changes process, which will also ensure the glibc resolver, NetworkManager, and the like get configured properly for it. Simply shipping dnsmasq with a default configuration that conflicts with systemd-resolved is not a productive approach.
If systemd-resolved is really broken, then it either needs to be fixed or replaced. The former needs to be handled through systemd upstream, the latter through the Fedora Changes process.
But this change should create conflict with systemd-resolved only in case it was improperly configured.
But the default configuration you ship will conflict.
Anyway, dnsmasq will listen by default on 127.0.0.1, as every standard resolver does. You can use listen-address=127.0.0.53 if you like, but then it will conflict with systemd-resolved.
You just wrote that you make it listen by default on all interfaces, and then filter. This means it will conflict over the port 53. That said, listening on the lo interface only will also conflict with systemd-resolved or any other local resolver, so you are probably right that your change does not change much for the default configuration, it just makes it harder (more settings to change) to set up coexistence. 127.0.0.53 is unfortunately not an independent interface, it is still the lo interface.
On 14. 01. 24 1:57, Kevin Kofler via devel wrote:
On a server I administer for work, I have dnsmasq serving the DNS for an ocserv (OpenConnect) VPN, listening only on the VPN interface. Any request for a host not within the VPN network (coming in from clients with no or broken split DNS support, e.g., old GNU/Linux distros without systemd- resolved, or Windows, where the OpenConnect client is still unable to set up split DNS) is forwarded to systemd-resolved, which in turn forwards it to the upstream DNS from the datacenter. Relying instead on the filtering would not have worked exactly for the reason you describe above. But that server is not running Fedora anyway.
I would recommend to skip systemd-resolved stub and using resolv-file=/run/systemd/resolve/resolv.conf
in such case. It would use servers configured by systemd-resolved, but without using broken port domain at address 127.0.0.53. Alternatively use server=127.0.0.54, which should not break incoming queries so much.
Well, I do not see a good reason to disable systemd-resolved for the server's own queries (which includes the forwarding queries from dnsmasq, if the domain is not one it knows). It just works.
Consider using unbound as a cache for other VPN clients. dnsmasq is great for its integration with DHCP server, but is targeted to use minimal resources. Unfortunately at cost of some design issues. Unbound is a high quality cache, while still relatively small compared to bind's named.service.
Using minimal resources is exactly what I want here. Which is why I do not want to use dnsmasq for what systemd-resolved can do, nor unbound for what dnsmasq can do.
And sending the server's own queries through dnsmasq is not going to work (not without a second instance, at least), because the VPN server is not a VPN client, so I have the server's /etc/hosts resolve its own domain to 127.0.0.1 (not the public IP, because services listen only on localhost and the VPN, that is what the VPN is for), which is honored by systemd-resolved, whereas my dnsmasq configuration overrides this to return the VPN IP to the VPN clients querying that same domain. Sounds hackish, but works great.
Based on my reading of this thread, this change is going to break the default configuration and needs to be reverted immediately. Petr, please file a Change Proposal for Fedora *41*. You have missed the deadline for F40 System-Wide Changes (Dec. 26th) and this is absolutely NOT a self-contained Change.
Please, Stephen, would you please read what I wrote, not what Kevin wrongly deducted?
The change I made in dnsmasq does not conflict with systemd-resolved in default configuration or if configured correctly. I have wrote scenario, when it can conflict and how to fix it. So no, there is no need to revert my change. If it is, *PLEASE* report it with exact step to reproduce. I have tested those changes do not cause regressions. If you (or anyone else) know something I have missed, I need much more precise steps than those below to fix it.
I would suggest everyone to try that change in rawhide actually, verify what is listening on and whether it does conflict with anything. I use "lsof -np $(pidof dnsmasq)" command for quick checks.
On 15. 01. 24 20:04, Stephen Gallagher wrote:
On Mon, Jan 15, 2024 at 11:32 AM Kevin Kofler via devel devel@lists.fedoraproject.org wrote:
Petr Menšík wrote:
systemd-resolved is unfortunately known to broken.
[snip]
Dnsmasq does not break DNSSEC, systemd-resolved does.
[snip]
Unfortunately broken are clients having systemd-resolved enabled.
How exactly is it broken? If you refer to: https://github.com/systemd/systemd/issues/25676 fixes for that are finally coming in now (as of 3 weeks ago).
I have reported issues known to me:
https://github.com/systemd/systemd/issues?q=is%3Aopen+is%3Aissue+author%3Ape...
https://bugzilla.redhat.com/buglist.cgi?quicksearch=reporter%3Apemensik%40re...
There are some others, which I consider important enough. If it works for you, good for you. It does not work well for others, I do not want to discuss it here. This thread is about dnsmasq only and I would like to focus to that only.
I would recommend having systemd-resolved forwarded to dnsmasq, which can then be forwarded further.
If you think dnsmasq should replace systemd-resolved by default, then please propose that through the Changes process, which will also ensure the glibc resolver, NetworkManager, and the like get configured properly for it. Simply shipping dnsmasq with a default configuration that conflicts with systemd-resolved is not a productive approach.
If systemd-resolved is really broken, then it either needs to be fixed or replaced. The former needs to be handled through systemd upstream, the latter through the Fedora Changes process.
Yes, I plan to propose changes to default system wide resolver, because long reported bugs to systemd-resolved were mostly ignored. For years. But the solution I want to propose is not replacing it with dnsmasq. I am dnsmasq maintainer, I know it has own issues. In my opinion it is less broken, but not perfect. We are preparing different solution, which should be better. But it is not yet ready and therefore there is no Fedora Change filled. We fill create that once we are sure it brings features without causing regressions.
But this change should create conflict with systemd-resolved only in case it was improperly configured.
But the default configuration you ship will conflict.
Anyway, dnsmasq will listen by default on 127.0.0.1, as every standard resolver does. You can use listen-address=127.0.0.53 if you like, but then it will conflict with systemd-resolved.
You just wrote that you make it listen by default on all interfaces, and then filter. This means it will conflict over the port 53. That said, listening on the lo interface only will also conflict with systemd-resolved or any other local resolver, so you are probably right that your change does not change much for the default configuration, it just makes it harder (more settings to change) to set up coexistence. 127.0.0.53 is unfortunately not an independent interface, it is still the lo interface.
Based on my reading of this thread, this change is going to break the default configuration and needs to be reverted immediately. Petr, please file a Change Proposal for Fedora *41*. You have missed the deadline for F40 System-Wide Changes (Dec. 26th) and this is absolutely NOT a self-contained Change.
I am sorry my intention were not understood correctly. Miroslav already mentioned that.We will fill System-Wide Change, but this change does not need it. If it breaks something, please use bug #2258062 to report details or create a new one for rawhide. But reproducible details are needed, not just assumptions.
Cheers,
Petr
PS: I consider it rude to be commented on issue closed for discussion, especially if the comment is misleading and false.
On Mon, Jan 15, 2024 at 05:31:36PM +0100, Kevin Kofler via devel wrote:
Petr Menšík wrote:
Anyway, dnsmasq will listen by default on 127.0.0.1, as every standard resolver does. You can use listen-address=127.0.0.53 if you like, but then it will conflict with systemd-resolved.
You just wrote that you make it listen by default on all interfaces, and then filter.
If I understand it correctly, it will do that only if the configuration was modified (an additional interface was specified). The default configuration should work exactly as before.
This means it will conflict over the port 53.
They bind to different addresses by default.
Thanks, that is exactly the message I wanted to tell.
Perhaps I should keep corner cases to myself until someone actually hits them. It has clearly confused people and created unnecessary reactions. Conflict will arise only when configuration change done by administrator were not complete.
On 16. 01. 24 10:18, Miroslav Lichvar wrote:
On Mon, Jan 15, 2024 at 05:31:36PM +0100, Kevin Kofler via devel wrote:
Petr Menšík wrote:
Anyway, dnsmasq will listen by default on 127.0.0.1, as every standard resolver does. You can use listen-address=127.0.0.53 if you like, but then it will conflict with systemd-resolved.
You just wrote that you make it listen by default on all interfaces, and then filter.
If I understand it correctly, it will do that only if the configuration was modified (an additional interface was specified). The default configuration should work exactly as before.
This means it will conflict over the port 53.
They bind to different addresses by default.
-
Kevin Kofler -
Miroslav Lichvar -
Petr Menšík -
Stephen Gallagher