To keep this off-list as much as possible, the rant is here:
http://reiisi.blogspot.com/2016/07/to-gil-tim-fedora-et-al.html
(The blame lies elsewhere. I wish I had the network and social cred to get a real movement started, away from the current faceless CA system and towards a different identity assurance system that depends on actual, existing day-to-day trust relationships.)
Allegedly, on or about 01 July 2016, Joel Rees sent:
(The blame lies elsewhere.
Yes. While I can't answer to the causes for other people's mail turning up misidentified as spam, I can say that when I mail this list using a Yahoo address, it's sent through the Yahoo SMTP servers, and I have logged into them to send it (it won't let you send, if you don't). There is nothing more that *I* can do to send that mail in a better way to avoid it being misidentified as spam.
I wish I had the network and social cred to get a real movement started, away from the current faceless CA system and towards a different identity assurance system that depends on actual, existing day-to-day trust relationships.)
Well, you're participating on a list for Fedora, and many services are managed by those people. If it's the Fedora list that's misidentifying spam on the way through, its software needs looking at. But I seem to recall the conversation pointing the finger at gmail not properly understanding mailing lists and the to/from addressing being different from personal mail. Over the years, there's been a lot of things said against Google's mail service when using it with list mail, and it appears you've be snagged by the latest WTF! You do have the option to vote with your feet and use a different mail service provider. Leaving gmail, and hotmail, and their ilk, to those who don't understand the difference between webmail and email.
I don't think we'll ever get truly certified email (the sender is who they say, spam is forbidden and accurately killed, etc.). Many people would fail the intelligence test to actually make use of it (they wouldn't manage to set it up, wouldn't comprehend status notices about the mail being good/no-good - clearly shown by the mammoth number of people who get conned by mail, never picking up on the fact that the addresses are wrong and that the grammar and spelling is worse than a grade 3 dropout). And mail client software can be diabolic at supporting security features.
Though, it could be an interesting experiment done with Fedora by the Fedora users. i.e. Set up a secured mail service for Fedora users. But obviously no good for a technical support list, where users often come to sort out their problems, not having to deal with a new one before they can even ask.
There probably needs to be something during the mailing list registration set up that lists a number of services that are known to be problematic. Sure, you can't possibly list every little service provider, but if one of the world's biggest has serious false spam triggering, I think it ought to get mentioned.
On 07/01/16 13:16, Tim wrote:
Well, you're participating on a list for Fedora, and many services are managed by those people. If it's the Fedora list that's misidentifying spam on the way through, its software needs looking at. But I seem to recall the conversation pointing the finger at gmail not properly understanding mailing lists and the to/from addressing being different from personal mail.
It is a "gmail" issue and it is *easily* solved within gmail.
Simply create a file with "To:" equal to users@lists.fedoraproject.org and check the box that says "Never send to spam". Problem solved.
On 07/01/16 13:39, Ed Greshko wrote:
On 07/01/16 13:16, Tim wrote:
Well, you're participating on a list for Fedora, and many services are managed by those people. If it's the Fedora list that's misidentifying spam on the way through, its software needs looking at. But I seem to recall the conversation pointing the finger at gmail not properly understanding mailing lists and the to/from addressing being different from personal mail.
It is a "gmail" issue and it is *easily* solved within gmail.
Simply create a file with "To:" equal to users@lists.fedoraproject.org and check the box that says "Never send to spam". Problem solved.
Simply create a *filter* !!!
And stop watching baseball games while responding to posts.
Since gmail isn't letting me reply directly to Ed,
On Fri, Jul 1, 2016 at 2:39 PM, Ed Greshko ed.greshko@greshko.com wrote:
On 07/01/16 13:16, Tim wrote:
Well, you're participating on a list for Fedora, and many services are managed by those people. If it's the Fedora list that's misidentifying spam on the way through, its software needs looking at. But I seem to recall the conversation pointing the finger at gmail not properly understanding mailing lists and the to/from addressing being different from personal mail.
It is a "gmail" issue and it is *easily* solved within gmail.
Simply create a file with "To:" equal to users@lists.fedoraproject.org and check the box that says "Never send to spam". Problem solved.
I have a filter.
Don't have a TV to watch baseball, FWIW. :)
I didn't have "Never send to spam" checked. Guess I missed it somehow. :(
Still, there is a problem with the way e-mail is handled.
Once upon a time, Ed Greshko ed.greshko@greshko.com said:
On 07/01/16 13:16, Tim wrote:
Well, you're participating on a list for Fedora, and many services are managed by those people. If it's the Fedora list that's misidentifying spam on the way through, its software needs looking at. But I seem to recall the conversation pointing the finger at gmail not properly understanding mailing lists and the to/from addressing being different from personal mail.
It is a "gmail" issue and it is *easily* solved within gmail.
No, it isn't specifically a Gmail issue, it is an issue from the combination of DMARC strict policies, sites that enforce DMARC policies, and mailing lists.
Yahoo publishes DMARC policies that say messages from a Yahoo domain in the From: header should only come from the Yahoo servers. Gmail (and other sites) recognize and follow those policies. When a Yahoo user sends email to a mailing list, and the list server resends the message, it doesn't come from a Yahoo server, so sites that follow DMARC policies reject the message.
The correct solution is for the mailing list software to be changed to rewrite From: addresses. Newer versions of Mailman support this. The address rewriting is annoying, but is the only true solution to being in between sites that publish and honor DMARC policies.
On Fri, Jul 1, 2016 at 7:16 AM, Chris Adams linux@cmadams.net wrote:
No, it isn't specifically a Gmail issue, it is an issue from the combination of DMARC strict policies, sites that enforce DMARC policies, and mailing lists.
DMARC, that was it. Thank you for the more detailed (and more correct) explanation.
As a Google user, I most often see this in mail coming from Yahoo. And Ed also makes a good point. I already filter my mailing list mail into separate labels for each list, and check "Never send to spam" for these, which removes the DMARC issue. Setting "never send to spam" means you are relying on the server for the mailing list to do the spam filtering.
--Greg
Once upon a time, Greg Woods woods@ucar.edu said:
On Fri, Jul 1, 2016 at 7:16 AM, Chris Adams linux@cmadams.net wrote:
No, it isn't specifically a Gmail issue, it is an issue from the combination of DMARC strict policies, sites that enforce DMARC policies, and mailing lists.
DMARC, that was it. Thank you for the more detailed (and more correct) explanation.
As a Google user, I most often see this in mail coming from Yahoo.
Yes, Yahoo is probably the largest user base with a DMARC "p=reject" policy (AOL also has that, but who uses AOL these days? :) ).
And Ed also makes a good point. I already filter my mailing list mail into separate labels for each list, and check "Never send to spam" for these, which removes the DMARC issue. Setting "never send to spam" means you are relying on the server for the mailing list to do the spam filtering.
The problem with whitelisting senders is that malware is smart enough to abuse it (has been for years, but is getting smarter lately). There have been several spam/virus senders lately that recognize mailing lists in people's address books, and then send garbage from a recognized list member address to other members, setting the headers to look like they went through the list server.
As soon as you whitelist the list, there's no way for your server to block such garbage messages. Even if the list server sets SPF, DKIM, and/or DMARC, your whitelist entry says to ignore them.
It is a tough problem. Ideally, list servers would rewrite the From: address (to avoid DMARC issues), and they'd all have SPF records (which Fedora's list server does), sign messages with DKIM, and have DMARC policies. Then you shouldn't need to whitelist (or at most whitelist from content filters, but not SPF/DKIM/DMARC checks).
As mail servers are moving to IPv6, some servers are requiring SPF and/or DMARC to accept email over IPv6, so maybe we'll get a little better experience there.
On Fri, 1 Jul 2016, Chris Adams wrote:
Once upon a time, Ed Greshko ed.greshko@greshko.com said: No, it isn't specifically a Gmail issue, it is an issue from the combination of DMARC strict policies, sites that enforce DMARC policies, and mailing lists.
Yahoo publishes DMARC policies that say messages from a Yahoo domain in the From: header should only come from the Yahoo servers. Gmail (and other sites) recognize and follow those policies. When a Yahoo user sends email to a mailing list, and the list server resends the message, it doesn't come from a Yahoo server, so sites that follow DMARC policies reject the message.
The correct solution is for the mailing list software to be changed to rewrite From: addresses. Newer versions of Mailman support this. The address rewriting is annoying, but is the only true solution to being in between sites that publish and honor DMARC policies.
Yeah, this was a big issue on another mailinglist I belong to (NAME-L at Emory U). We had a bunch of people who simply disappeared from the list for a couple of months. Then they all reappeared with obviously re-written addresses.
That meant that none of them could get backchannel responses by just choosing "reply" because Emory didn't decode the addresses for response in a reply. Since that list is associated with an organization that keeps a list of public email addresses of members, the workaround was to have to look up the address if you wanted to respond individually. But that doesn't happen much. All in all, it turned out to be a blessing in disguise because a lot of people were in the habit hitting "reply all" and posters would get two responses -- one to the mailinglist and one personally. And it turned out that most of the people who were either offenders *or* victims of the double mails were mostly Yahoo/Gmail folk. Win. win.
billo
On Fri, Jul 01, 2016 at 08:16:10 -0500, Chris Adams linux@cmadams.net wrote:
The correct solution is for the mailing list software to be changed to rewrite From: addresses. Newer versions of Mailman support this. The address rewriting is annoying, but is the only true solution to being in between sites that publish and honor DMARC policies.
I disagree. The correct solution is to detect that this is a list message and evaluate it some other way. For example the envelope sender address could be checked instead of the from address. Some signature systems will pass through mailing lists and still be verifiable.
Modying from headers is going to cause problems with replies and it really doesn't do anything but flag the message was resent, whivh can be figured out with better methods.
This does solve the spam problem in any case as plenty of spam gets sent from places like yahoo and gmail because end users get their credentials stolen by spammers a lot.
Once upon a time, Bruno Wolff III bruno@wolff.to said:
On Fri, Jul 01, 2016 at 08:16:10 -0500, Chris Adams linux@cmadams.net wrote:
The correct solution is for the mailing list software to be changed to rewrite From: addresses. Newer versions of Mailman support this. The address rewriting is annoying, but is the only true solution to being in between sites that publish and honor DMARC policies.
I disagree. The correct solution is to detect that this is a list message and evaluate it some other way. For example the envelope sender address could be checked instead of the from address. Some signature systems will pass through mailing lists and still be verifiable.
At which point, malware makes the same headers to avoid DMARC policies, and DMARC is useless. I'm not the biggest fan of DMARC (haven't implemented it for any of my domains for example), but it is a decent solution.
Modying from headers is going to cause problems with replies and it really doesn't do anything but flag the message was resent, whivh can be figured out with better methods.
There are a bunch of lists already doing it, and newer versions of Mailman have an option for it. Basically though, old-school mailing lists are such a niche market, large-scale mailers just don't care (and that is not going to change).
This does solve the spam problem in any case as plenty of spam gets sent from places like yahoo and gmail because end users get their credentials stolen by spammers a lot.
I know you left out a "not" (as in "does NOT solve"), but nobody claims DMARC (or anything else) is THE solution. There are incremental steps, and malware will always get around some of them in some form or fashion. Saying something is not a 100% solution so we should give up is just abandoning traditional email.
Most mailers apply filtering, rate limits, etc. to authenticated email to cut down on being a source of garbage email (again, not a 100% solution but a big help).
On Thu, Jun 30, 2016 at 11:39 PM, Ed Greshko ed.greshko@greshko.com wrote:
On 07/01/16 13:16, Tim wrote:
Well, you're participating on a list for Fedora, and many services are managed by those people. If it's the Fedora list that's misidentifying spam on the way through, its software needs looking at. But I seem to recall the conversation pointing the finger at gmail not properly understanding mailing lists and the to/from addressing being different from personal mail.
It is a "gmail" issue and it is *easily* solved within gmail.
Simply create a file with "To:" equal to users@lists.fedoraproject.org and check the box that says "Never send to spam". Problem solved.
No because it means everyone has to do that, and now when there's spam it doesn't get filtered. So this breaks things worse.
Basically Google came up with an idea, Yahoo is following it and asking for their list relayed messages to fail per that idea, and Google is honoring it. So the solution is to not use Yahoo and other email addresses that insert the dmarc failure indication, on lists. It's broken crap, but that's the solution both companies are advertising.
On Thu, Jun 30, 2016 at 11:16 PM, Tim ignored_mailbox@yahoo.com.au wrote:
But I seem to recall the conversation pointing the finger at gmail not properly understanding mailing lists
Google and Yahoo is a well-known issue. There is some anti-spam method (which I can't remember the name of now) that Yahoo's servers claim that they do, but then they don't do it right, which is why Google often rejects e-mail from Yahoo servers as spam. At least, this is the argument from the Google side.
In the end though, all spam filters are imperfect, so it's never going to be possible to completely avoid the need to look in your spam folder for false positives.
--Greg
-
Bruno Wolff III -
Chris Adams -
Chris Murphy -
Ed Greshko -
Greg Woods -
Joel Rees -
Tim -
William Oliver