I have set up spf, dmarc, and dkim for my email domain. It *seems* to work well. I tested it by sending an email to my GMail account. When I look at the headers of the email, GMail says that it passes all three tests:
ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@camerontech.com header.s=default header.b=My0caSvG; spf=pass (google.com: best guess record for domain of thomas.cameron@camerontech.com designates 3.138.45.83 as permitted sender) smtp.mailfrom=thomas.cameron@camerontech.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=camerontech.com
But then, when I do something like send email to this list, I suddenly get a TON of error messages saying that the email failed spf tests because it's coming from the server of the mailing list instead of my email server. Is that normal? It's kind of frustrating. I added the ip address of the Fedora list server to my spf record, but that seems really hackish.
What do folks do to set up email with dmarc, spf, and so on?
Thomas
Thomas Cameron writes:
But then, when I do something like send email to this list, I suddenly get a TON of error messages saying that the email failed spf tests because it's coming from the server of the mailing list instead of my email server. Is that normal?
I would guess the mail system is behaving correctly given your server configuration.
Since you don't like what's happening, you probably have a misconfiguration. But exactly what the problem is, I don't know because I don't know what you do want and you haven't provided any configuration or even quoted the error message.
It's kind of frustrating. I added the ip address of the Fedora list server to my spf record, but that seems really hackish.
And insecure. Anybody can now spoof your mail by sending it through the list's MTA. (It wouldn't stand up under close examination, I guess, but neither would most successful phishing mails.)
What do folks do to set up email with dmarc, spf, and so on?
Depends on what else your server is doing, how paranoid you are, and several other things.
Your DNS TXT record says "v=spf1 a:you.com ip4:1.2.3.4 ip4:5.6.7.8 ~all". Based on that and a wild guess, I think the issue is probably the "~all". While the SPF RFC doesn't specify what receivers should do on matching "~all" (aka softfail), and does say it's not sufficient to reject a message, it does imply you're asking for feedback. If you're not all that paranoid, I suggest changing "~all" to "?all". See https://datatracker.ietf.org/doc/html/rfc7208#section-8.5 for details (they're pretty gory if you're not a regular denizen of RFC-world). I can't guarantee that will reduce the error messages but it's the only thing to try with information provided. (You could also simply not use SPF and rely entirely on DKIM which has fewer failure modes.)
The other WAG about the source of the error messages is that you enabled the reporting feature for DMARC. In that case I suggest you shut it off. :-)
Your list posts should be well-enough protected by DKIM. Your lists can improve handling of your mail by implementing ARC, but of course that's up to them, not you. And it depends on ultimate receivers supporting ARC, too, although most of the majors already do. https://en.wikipedia.org/wiki/Authenticated_Received_Chain https://datatracker.ietf.org/doc/html/rfc8617
Steve
On 6/17/22 10:56, Thomas Cameron wrote:
I have set up spf, dmarc, and dkim for my email domain. It *seems* to work well. I tested it by sending an email to my GMail account. When I look at the headers of the email, GMail says that it passes all three tests:
ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@camerontech.com header.s=default header.b=My0caSvG; spf=pass (google.com: best guess record for domain of thomas.cameron@camerontech.com designates 3.138.45.83 as permitted sender) smtp.mailfrom=thomas.cameron@camerontech.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=camerontech.com
But then, when I do something like send email to this list, I suddenly get a TON of error messages saying that the email failed spf tests because it's coming from the server of the mailing list instead of my email server. Is that normal? It's kind of frustrating. I added the ip address of the Fedora list server to my spf record, but that seems really hackish.
What do folks do to set up email with dmarc, spf, and so on?
Thomas
I have spf, dmarc, etc., set up. I don't recall what happens when I post to the list, so I'll reply now and see. I'll follow up here if I also get failure notifications. It doesn't sound familiar.
-- Mark
On 6/17/22 11:01, Stephen J. Turnbull wrote:
Thomas Cameron writes:
But then, when I do something like send email to this list, I suddenly get a TON of error messages saying that the email failed spf tests because it's coming from the server of the mailing list instead of my email server. Is that normal?
I would guess the mail system is behaving correctly given your server configuration.
Since you don't like what's happening, you probably have a misconfiguration. But exactly what the problem is, I don't know because I don't know what you do want and you haven't provided any configuration or even quoted the error message.
It's kind of frustrating. I added the ip address of the Fedora list server to my spf record, but that seems really hackish.
And insecure. Anybody can now spoof your mail by sending it through the list's MTA. (It wouldn't stand up under close examination, I guess, but neither would most successful phishing mails.)
What do folks do to set up email with dmarc, spf, and so on?
Depends on what else your server is doing, how paranoid you are, and several other things.
Your DNS TXT record says "v=spf1 a:you.com ip4:1.2.3.4 ip4:5.6.7.8 ~all". Based on that and a wild guess, I think the issue is probably the "~all". While the SPF RFC doesn't specify what receivers should do on matching "~all" (aka softfail), and does say it's not sufficient to reject a message, it does imply you're asking for feedback. If you're not all that paranoid, I suggest changing "~all" to "?all". See https://datatracker.ietf.org/doc/html/rfc7208#section-8.5 for details (they're pretty gory if you're not a regular denizen of RFC-world). I can't guarantee that will reduce the error messages but it's the only thing to try with information provided. (You could also simply not use SPF and rely entirely on DKIM which has fewer failure modes.)
The other WAG about the source of the error messages is that you enabled the reporting feature for DMARC. In that case I suggest you shut it off. :-)
Your list posts should be well-enough protected by DKIM. Your lists can improve handling of your mail by implementing ARC, but of course that's up to them, not you. And it depends on ultimate receivers supporting ARC, too, although most of the majors already do. https://en.wikipedia.org/wiki/Authenticated_Received_Chain https://datatracker.ietf.org/doc/html/rfc8617
Yeah, I changed it to soft fail (~all) instead of -all before. Also, to be clear, I only added the list server address for testing purposes. I wouldn't leave it that way. I was just trying to figure out why I was getting errors and how I could fix them.
But my point is, setting up spf works as expected. I've verified it via my emails to known correctly configured mail servers like GMail. What I don't understand is why, when it is apparently set up correctly, are there mail servers which throw errors when I send email through a mailing list. Is it a misconfiguration of the mailing list? Is it a misconfiguration of the receivers?
I mean, setting up spf isn't rocket science. There are tons of tutorials, and I am reasonably certain it's set up correctly since my emails come through with PASS ratings when I check them via e.g. GMail. Why are they failing when I send them through an email list server? What is the misconfiguration that you are saying I have?
On 6/17/22 11:16, Thomas Cameron wrote:
Yeah, I changed it to soft fail (~all) instead of -all before. Also, to be clear, I only added the list server address for testing purposes. I wouldn't leave it that way. I was just trying to figure out why I was getting errors and how I could fix them.
But my point is, setting up spf works as expected. I've verified it via my emails to known correctly configured mail servers like GMail. What I don't understand is why, when it is apparently set up correctly, are there mail servers which throw errors when I send email through a mailing list. Is it a misconfiguration of the mailing list? Is it a misconfiguration of the receivers?
I mean, setting up spf isn't rocket science. There are tons of tutorials, and I am reasonably certain it's set up correctly since my emails come through with PASS ratings when I check them via e.g. GMail. Why are they failing when I send them through an email list server? What is the misconfiguration that you are saying I have?
And here's what I get every time I post to this list. It's a small handful of mail servers that send me what appears to be incorrect errors because the ip address of the Fedora mail server is different from my mail server. I think it's stale DNS data now, since I've changed the TXT record.
On 6/17/22 11:50, Thomas Cameron wrote:
On 6/17/22 11:16, Thomas Cameron wrote:
Yeah, I changed it to soft fail (~all) instead of -all before. Also, to be clear, I only added the list server address for testing purposes. I wouldn't leave it that way. I was just trying to figure out why I was getting errors and how I could fix them.
But my point is, setting up spf works as expected. I've verified it via my emails to known correctly configured mail servers like GMail. What I don't understand is why, when it is apparently set up correctly, are there mail servers which throw errors when I send email through a mailing list. Is it a misconfiguration of the mailing list? Is it a misconfiguration of the receivers?
I mean, setting up spf isn't rocket science. There are tons of tutorials, and I am reasonably certain it's set up correctly since my emails come through with PASS ratings when I check them via e.g. GMail. Why are they failing when I send them through an email list server? What is the misconfiguration that you are saying I have?
And here's what I get every time I post to this list. It's a small handful of mail servers that send me what appears to be incorrect errors because the ip address of the Fedora mail server is different from my mail server. I think it's stale DNS data now, since I've changed the TXT record.
Hopefully I've nailed it down. I've stripped some weird/extraneous stuff out of the TXT entries in DNS and gone to the bare minimum. I'll see if I get bounces when I send this.
Thomas
On Fri, 2022-06-17 at 11:16 -0500, Thomas Cameron wrote:
But my point is, setting up spf works as expected. I've verified it via my emails to known correctly configured mail servers like GMail. What I don't understand is why, when it is apparently set up correctly, are there mail servers which throw errors when I send email through a mailing list. Is it a misconfiguration of the mailing list? Is it a misconfiguration of the receivers?
I don't think there's a way around this (for you). These records are used to say who can post your mail (only *your* mail servers).
That, in itself, doesn't stop spam. It relies on other servers refusing mail appearing to be from you, but coming from an unauthorised source. And it wouldn't stop someone sending mail forged as coming from you, going through your authorised server. But it's a better spam identifier than many other schemes.
But when you post to a mailing list, it reposts your mail through them, still identified as coming from you. Their server is not on your authorised list (and shouldn't be).
The only way I can see to avoid that problem is for the mailing list to not distribute your message from *you*, but rewrite the "from" address as coming from itself. People don't like that, because it anonymises mail (people behave worse when anonymous), and they can't send private replies (not that some of us want them).
I preferred usenet to mailing lists. You posted to a group, people subscribed (or browsed it) if they wanted to see it. You didn't need to use an email address, so no spam could come in your direction (only to the group, which may have reasonably good anti-spam systems).
I've yet to come across an anti-spam system that doesn't stuff something up (false negatives, false positives, not detecting spam). If you have to check your (suspected) spam folder each time you get your mail, what's the point of using it?
System-wide ISP systems are able better than personal spam detection systems. In the sense that an ISP gets thousands of emails, and when scads of identical spam hit their server, it can be flagged and deleted as spam. This is completely different from any system (ISP-supplied or not) that only assesses your inbox in isolation.
Really what's needed to actually stop spam is for all SMTP servers to require their clients to authenticate, for the servers to verify their client's identities when they join the service, and to refuse anyone to post spam in the first place. But that's never going to happen. People don't want that level of identity control, anonymity is needed for some circumstances, and there are service that are set up solely for spewing spam (they won't agree to do anything to stop spam).
I've said it for many years - the only way to stop spammers is to chop off their hands.
Thomas Cameron writes:
But my point is, setting up spf works as expected. I've verified it via my emails to known correctly configured mail servers like GMail. What I don't understand is why, when it is apparently set up correctly, are there mail servers which throw errors when I send email through a mailing list. Is it a misconfiguration of the mailing list?
The mailing list can't do anything about this. It's entirely a conversation between your server and the recipient's. SPF provides a list of authorized IPs. If you put your mailing lists' hosts in there, they're authorized, and the recipient should accept. If you don't, they're not authorized, and the recipient should reject. That's the basic idea in a nutshell.
Is it a misconfiguration of the receivers?
No, it's a misunderstanding of what SPF can do. SPF was designed for what IETF mail geeks call "transactional mail flows", such as between you and your bank. By having their own mail server, and mailing you directly from that server, you can have pretty high confidence that it's from your bank, as typical script kiddies are unlikely to be able to spoof your bank's IP.
But third-party mailing lists obviously break this model. So if you want to post to mailing lists you must not have their servers match "-" or "~" entries in your SPF configuration (including "all").
What in theory could work is DKIM, which depends on cryptographic signatures, not on (somewhat spoofable) IP source addresses, and therefore is designed to work for indirect mail flows, that have a relay host between your server and the recipient. Unfortunately, DKIM signatures are typically broken by mailing lists because they usually include Subject (which mailing lists add tags and serial numbers to) and the whole body (which mailing lists frequently add footers to). Users have different opinions about tagging Subject, but almost everybody likes footers, especially list admins. *sigh*
ARC helps with this by implementing a "transitive trust" model. If a recipient trusts lists.fedoraproject.org, and they say your DKIM verified, this solves both the breakage of your DKIM signature and the DMARC From mismatch because the recipient will use the results from the mailing list's MTA instead of its own for authenticating you.
Why are they failing when I send them through an email list server?
Because that's the whole point of SPF. If you do not explicitly authorize an IP to send your email, you want mail coming from that IP to be considered a spoof. That's how SPF provides security, that's the whole design.
For a normal individual who does lots of "stuff" with mail from their server, the value in SPF is that when you send *direct* mail, the recipient can be pretty sure that it's someone with a valid account on your server.
What is the misconfiguration that you are saying I have?
Using "-all" or "~all" in your SPF configuration. They are saying "reject mail whose last hop source IP isn't explicitly authorized", with "~all" being less strict but any receiver is within their rights to reject (for example, you probably want your bank and your customers to do so, right?) If you use "?all" (or no "all" at all), you're saying "you can trust mail direct from my server to be me, but sometimes I send indirect mail, so use your best judgment if it's not direct from my server."
You might think "but why not check the Received fields for my server?" Unfortunately that's very easily, and very frequently, spoofed -- unless you sign them, which is exactly what ARC does.
Steve
On Sat, 2022-06-18 at 11:39 +0930, Tim via users wrote:
System-wide ISP systems are able better than personal spam detection systems. In the sense that an ISP gets thousands of emails, and when scads of identical spam hit their server, it can be flagged and deleted as spam. This is completely different from any system (ISP-supplied or not) that only assesses your inbox in isolation.
That's why I turned off local spam detection years ago, and have never looked back.
poc
On 6/17/22 07:56, Thomas Cameron wrote:
I have set up spf, dmarc, and dkim for my email domain. It *seems* to work well. I tested it by sending an email to my GMail account. When I look at the headers of the email, GMail says that it passes all three tests:
ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@camerontech.com header.s=default header.b=My0caSvG; spf=pass (google.com: best guess record for domain of thomas.cameron@camerontech.com designates 3.138.45.83 as permitted sender) smtp.mailfrom=thomas.cameron@camerontech.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=camerontech.com
But then, when I do something like send email to this list, I suddenly get a TON of error messages saying that the email failed spf tests because it's coming from the server of the mailing list instead of my email server. Is that normal? It's kind of frustrating. I added the ip address of the Fedora list server to my spf record, but that seems really hackish.
What do folks do to set up email with dmarc, spf, and so on?
Hi Thomas,
Here's how I setup SPF.
Create a separate subdomain in DNS to contain the spf TXT record that points to the mailserver:
_spf.hostisimo.com. TXT "v=spf1 ip4:78.138.24.13 ~all"
Now add an spf redirect TXT record to the mail server's domain and any other domains and subdomains that use the mail server.
mailserver's domain: hostisimo.com. TXT "v=spf1 redirect=_spf.example.com"
a subdomain: nospam.hostisimo.com. TXT "v=spf1 redirect=_spf.example.com"
Here's what google says about it:
ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of nobody@nospam.hostisimo.com designates 78.138.24.13 as permitted sender)...
That's it. ezpz.
Mike Wright
On 6/18/22 09:23, Mike Wright wrote:
mailserver's domain: hostisimo.com. TXT "v=spf1 redirect=_spf.example.com"
a subdomain: nospam.hostisimo.com. TXT "v=spf1 redirect=_spf.example.com"
CORRECTION: (replaceing example.com with hostisimo.com) mailserver's domain: hostisimo.com. TXT "v=spf1 redirect=_spf.hostisimo.com"
a subdomain: nospam.hostisimo.com. TXT "v=spf1 redirect=_spf.hostisimo.com"
:/
On 6/18/22 02:20, Stephen J. Turnbull wrote:
What is the misconfiguration that you are saying I have?
Using "-all" or "~all" in your SPF configuration. They are saying "reject mail whose last hop source IP isn't explicitly authorized", with "~all" being less strict but any receiver is within their rights to reject (for example, you probably want your bank and your customers to do so, right?) If you use "?all" (or no "all" at all), you're saying "you can trust mail direct from my server to be me, but sometimes I send indirect mail, so use your best judgment if it's not direct from my server."
OK, I've changed the spf to ?all and we'll see if that works.
Thanks.
-
Mark C. Allman -
Mike Wright -
Patrick O'Callaghan -
Stephen J. Turnbull -
Thomas Cameron -
Tim