Hello everyone,
I am not sure what the exact name of it but I will try to explain it.
Say you reply to am email to a person and it comes back saying that you need to verify that you are a real person and not a spammer.
Is there anything out there similar to that under the GPL or free??
Hopefully I explained that right. I would have used google but I do not know the exact name of it.
Thanks, Will
CH schrieb:
Hello everyone,
I am not sure what the exact name of it but I will try to explain it.
Say you reply to am email to a person and it comes back saying that you need to verify that you are a real person and not a spammer.
Is there anything out there similar to that under the GPL or free??
Hopefully I explained that right. I would have used google but I do not know the exact name of it.
Thanks, Will
http://en.wikipedia.org/wiki/Challenge-response_spam_filtering
Please, please don't use this nonsense! http://linuxmafia.com/faq/Mail/challenge-response.html
http://tmda.net/ - released under the GNU General Public License
Alexander
On Fri, 27 Apr 2007, CH wrote:
Hello everyone,
I am not sure what the exact name of it but I will try to explain it.
Say you reply to am email to a person and it comes back saying that you need to verify that you are a real person and not a spammer.
And how do you verify if that message is from a real person and not a spammer. "Click here" is a good way to install malware.
I personally find that sort of "spam blocker" to be incredibly invasive and annoying. I also refuse to respond to any of them.
Also think about what would happen if everyone use this method. Everytime you posted to a mailing list for the first time you would be mailbombed by every member of the list.
You are better off using greylisting or some other nonintrusive solution.
CH writes:
Hello everyone,
I am not sure what the exact name of it but I will try to explain it.
Say you reply to am email to a person and it comes back saying that you need to verify that you are a real person and not a spammer.
Is there anything out there similar to that under the GPL or free??
There are some crappy scripts that do this, but if you choose to use it, you yourself will be blacklisted as a spammer. That's because return addresses on spam are always forged, so if you use such a stupid script, you'll end up forwarding all the spam that you receive to forged return addresses, that belong to other people, and before long you yourself will get blacklisted.
These kinds of scripts are used only by people who are too stupid to be allowed access to the Internet.
El Viernes, 27 de Abril de 2007 23:28, CH escribió:
Hello everyone,
I am not sure what the exact name of it but I will try to explain it.
Say you reply to am email to a person and it comes back saying that you need to verify that you are a real person and not a spammer.
Is there anything out there similar to that under the GPL or free??
Hopefully I explained that right. I would have used google but I do not know the exact name of it.
Thanks, Will
Do you mean those _annoying_ emails? It's not too late to keep you in the good way, do not set up that.
To be honest, those emails screwed me up, especially when you're in a mailing list and you send several emails to it, and you get all those mails back asking you to click...it's crap
Thanks for the reminder everyone!! :) They are a pain. I did not think about mailing lists and such, thanks for setting me straight! :)
I will look into greylisting and see what I can come up with.
Thanks again. Will
On Fri, 27 Apr 2007, CH wrote:
Thanks for the reminder everyone!! :) They are a pain. I did not think about mailing lists and such, thanks for setting me straight! :)
I will look into greylisting and see what I can come up with.
I use "postgrey". Works pretty well.
At 6:53 PM -0400 4/27/07, CH wrote:
Thanks for the reminder everyone!! :) They are a pain. I did not think about mailing lists and such, thanks for setting me straight! :)
I will look into greylisting and see what I can come up with.
Also look at Early Talker, called Greet Pause in sendmail. It's when the sender has sent the whole message at once, rather than having a proper SMTP conversation. Such messages can be presumed spam, and discarded with no further action.
On Fri, 2007-04-27 at 22:04 -0400, Tony Nelson wrote:
At 6:53 PM -0400 4/27/07, CH wrote:
Thanks for the reminder everyone!! :) They are a pain. I did not think about mailing lists and such, thanks for setting me straight! :)
I will look into greylisting and see what I can come up with.
Also look at Early Talker, called Greet Pause in sendmail. It's when the sender has sent the whole message at once, rather than having a proper SMTP conversation. Such messages can be presumed spam, and discarded with no further action.
Hmm... That looks like a reasonable idea.
This could work similarly as a Postfix SPAM/Virus deterrent
smtpd_delay_reject = no smtpd_helo_restrictions = reject_unauth_pipelining smtpd_data_restrictions = reject_unauth_pipelining
On Mon, 30 Apr 2007, Guy Fraser wrote:
On Fri, 2007-04-27 at 22:04 -0400, Tony Nelson wrote:
I will look into greylisting and see what I can come up with.
Greylisting is one of the biggest winners here. We've cut the spam load so much that spam assassin now catches almost nothing.
Also look at Early Talker, called Greet Pause in sendmail. It's when the sender has sent the whole message at once, rather than having a proper SMTP conversation. Such messages can be presumed spam, and discarded with no further action.
Hmm... That looks like a reasonable idea.
The general consensus on the postfix mailing idea is that Greet Pause is a bad idea (TM). What it ends up doing is (a) delay legitimate mail and (b) DoS your own server as you now take longer to handle legitimate mail. Any mail source that would fail greet pause will also fail numerous other checks that don't inconvenience your intended users (and your own system).
Steve Friedman
At 12:40 PM -0400 4/30/07, Steve Friedman wrote:
On Mon, 30 Apr 2007, Guy Fraser wrote:
On Fri, 2007-04-27 at 22:04 -0400, Tony Nelson wrote:
I will look into greylisting and see what I can come up with.
Greylisting is one of the biggest winners here. We've cut the spam load so much that spam assassin now catches almost nothing.
Also look at Early Talker, called Greet Pause in sendmail. It's when the sender has sent the whole message at once, rather than having a proper SMTP conversation. Such messages can be presumed spam, and discarded with no further action.
Hmm... That looks like a reasonable idea.
The general consensus on the postfix mailing idea is that Greet Pause is a bad idea (TM). What it ends up doing is (a) delay legitimate mail and (b) DoS your own server as you now take longer to handle legitimate mail. Any mail source that would fail greet pause will also fail numerous other checks that don't inconvenience your intended users (and your own system).
Well, don't forget that greylisting will delay each new legitimate mail sender by a while (maybe a few hours), requires maintaining whitelists for the server farms of large email providers (AOL, etc.) or email from them may take much more than 4 hours to get through, and the mail must be handled twice by your server rather than just once.
Tony Nelson wrote:
Well, don't forget that greylisting will delay each new legitimate mail sender by a while (maybe a few hours), requires maintaining whitelists for the server farms of large email providers (AOL, etc.) or email from them may take much more than 4 hours to get through, and the mail must be handled twice by your server rather than just once.
You are incorrect on several counts.
1. The time to delay is configurable in a good greylist milter. Mine is set to 15 minutes since this is pretty much the default retry interval of most MTAs.
2. No whitelist maintaining is needed. The sending system either tries again or it doesn't. If it is a legitimate sender, it will retry. Also, when a sender/system is allowed it will be cached. So, even if you have multiple servers from AOL, etc. they will eventually be cached.
3. The email itself will only be handled once. When a server to be delayed first contacts your server the milter will check the cache with the initial information supplied and simply close the connection and not allow the DATA portion to be sent.
Tony Nelson wrote:
Well, don't forget that greylisting will delay each new legitimate mail sender by a while (maybe a few hours), requires maintaining whitelists for the server farms of large email providers (AOL, etc.) or email from them may take much more than 4 hours to get through, and the mail must be handled twice by your server rather than just once.
Ed Greshko wrote:
You are incorrect on several counts.
- The time to delay is configurable in a good greylist milter. Mine is
set to 15 minutes since this is pretty much the default retry interval of most MTAs.
Really? The standard says The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; (RFC 2821 section 4.5.4.1)
Calling half an hour "a while" seems reasonable to me...
I'd argue that your first sentence is misleading, too -- the delay is a result of the configuration of both sending and receiving MTAs.
- No whitelist maintaining is needed. The sending system either tries
again or it doesn't. If it is a legitimate sender, it will retry. Also, when a sender/system is allowed it will be cached. So, even if you have multiple servers from AOL, etc. they will eventually be cached.
Tony calling it a "whitelist" may be misleading.
But you are missing a detail here, and confusing "sending system", "computer", and "IP address". For major providers, the sending system may involve lots of computers, with lots of IP addresses. Retries may come from any of those computers -- this is perfectly legitimate under SMTP. So it may take a while (especially if they use an "exponential back-off") before the same server retries the same e-mail. With enough sending IP addresses, it's possible that the e-mail might never be retried from the same IP address.
There are two ways around this -- either you can (as Tony said) maintain a list of senders which use this sort of system, or hope that the senders put their sending MTAs in no more than a few /24 subnets. You then get the greylist to consider that one sending attempt from 127.36.5.1[1] and a retry from 127.36.5.2 is Good Enough.
- The email itself will only be handled once. When a server to be delayed
first contacts your server the milter will check the cache with the initial information supplied and simply close the connection and not allow the DATA portion to be sent.
This is true, but possibly not the best response to Tony's post. The *real* point is that although the server has to "think about" the message twice, the first time takes up nearly no bandwidth and nearly no processor time.
But you're missing another point -- the more people use greylisting, the less reliable it becomes (because spammers start retrying on any error). If Tony and I choose not to use greylisting, that makes it more usable for you!
James.
[1] Yes, I know there's a slight problem with that IP address!
James Wilkinson wrote:
You are incorrect on several counts.
- The time to delay is configurable in a good greylist milter. Mine is
set to 15 minutes since this is pretty much the default retry interval of most MTAs.
Really? The standard says The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; (RFC 2821 section 4.5.4.1)
Calling half an hour "a while" seems reasonable to me...
I'd argue that your first sentence is misleading, too -- the delay is a result of the configuration of both sending and receiving MTAs.
Whatever.... It is certainly not 4 hours.....
You need to understand the meaning of "should" v.s. "must".
- No whitelist maintaining is needed. The sending system either tries
again or it doesn't. If it is a legitimate sender, it will retry. Also, when a sender/system is allowed it will be cached. So, even if you have multiple servers from AOL, etc. they will eventually be cached.
Tony calling it a "whitelist" may be misleading.
But you are missing a detail here, and confusing "sending system", "computer", and "IP address". For major providers, the sending system may involve lots of computers, with lots of IP addresses. Retries may come from any of those computers -- this is perfectly legitimate under SMTP. So it may take a while (especially if they use an "exponential back-off") before the same server retries the same e-mail. With enough sending IP addresses, it's possible that the e-mail might never be retried from the same IP address.
There are two ways around this -- either you can (as Tony said) maintain a list of senders which use this sort of system, or hope that the senders put their sending MTAs in no more than a few /24 subnets. You then get the greylist to consider that one sending attempt from 127.36.5.1[1] and a retry from 127.36.5.2 is Good Enough.
I think you have no idea of what you speak.
- The email itself will only be handled once. When a server to be delayed
first contacts your server the milter will check the cache with the initial information supplied and simply close the connection and not allow the DATA portion to be sent.
This is true, but possibly not the best response to Tony's post. The *real* point is that although the server has to "think about" the message twice, the first time takes up nearly no bandwidth and nearly no processor time.
Huh?
But you're missing another point -- the more people use greylisting, the less reliable it becomes (because spammers start retrying on any error). If Tony and I choose not to use greylisting, that makes it more usable for you.
For every point, there is a counter point.
All I know is that greylisting or graylisting and spamassisssin has reduced the amount of spam I get by 95%.
You can chose to do as you wish. I will do as I do and be happy that I get very little spam.
Oh, and BTW, that is not to do stupid things like blanket rejects of upper level domains.
James.
[1] Yes, I know there's a slight problem with that IP address!
Ah, yes, well spoken from someone with no idea as to how things work.
Ed Greshko wrote:
You are incorrect on several counts.
- The time to delay is configurable in a good greylist milter. Mine is
set to 15 minutes since this is pretty much the default retry interval of most MTAs.
Really? The standard says The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; (RFC 2821 section 4.5.4.1)
Calling half an hour "a while" seems reasonable to me...
I'd argue that your first sentence is misleading, too -- the delay is a result of the configuration of both sending and receiving MTAs.
Whatever.... It is certainly not 4 hours.....
You need to understand the meaning of "should" v.s. "must".
The point here is that it is up to the sender to retry. If you tempfail the first attempt you have no control over how long it will be until the next attempt happens. If the sender has a big queue, it could be 4 hours or more.
There are two ways around this -- either you can (as Tony said) maintain a list of senders which use this sort of system, or hope that the senders put their sending MTAs in no more than a few /24 subnets. You then get the greylist to consider that one sending attempt from 127.36.5.1[1] and a retry from 127.36.5.2 is Good Enough.
I think you have no idea of what you speak.
At the very least you should permit a sending host once it is known to retry. Some schemes match up senders/recipients - which is appropriate for the first connection, but once you know a host is going to retry you might as well let it through.
Les Mikesell wrote:
The point here is that it is up to the sender to retry. If you tempfail the first attempt you have no control over how long it will be until the next attempt happens. If the sender has a big queue, it could be 4 hours or more.
That *could* happen. In practice, I've not seen it. In any event, it would happen only once for a given sending MTA. If that is the price I have to pay for reducing incoming SPAM by over 80% it is well worth it. In a short period of time all of those MTA's will long delays will be cached.
I bet those people that fret over this also have their user agents set to poll for new mail every minute.
At the very least you should permit a sending host once it is known to retry. Some schemes match up senders/recipients - which is appropriate for the first connection, but once you know a host is going to retry you might as well let it through.
That is what the cache does....without any human intervention.
Ed Greshko wrote (separately):
You are incorrect on several counts.
You need to understand the meaning of "should" v.s. "must".
I think you have no idea of what you speak.
Ah, yes, well spoken from someone with no idea as to how things work.
I've been trying to formulate a response for some time. I think I'd better be blunt.
Ed, I think these statements are combative, unhelpful, and basically rude. I can put up with the first and the last, but if you're going to make such statements, please back them up.
In particular:
I think you have no idea of what you speak.
was your sole comment to a long paragraph of mine. You didn't even begin to tell me or the list *why* you thought I was wrong. Neither I nor anyone else had the opportunity to benefit from your vast knowledge.
Ah, yes, well spoken from someone with no idea as to how things work.
was, at best, cheap, belittling, and designed to Put Me In My Place.
These e-mails are counter-productive. If you had addressed them to a new user, they may well have driven them out of the community. If read by new users, they may give the impression that the list is hostile.
It hasn't been, in the past. Don't make it change.
James.
James Wilkinson wrote:
I've been trying to formulate a response for some time. I think I'd better be blunt.
Sounds like a good idea. Better than pussy footing around.
Ed, I think these statements are combative, unhelpful, and basically rude. I can put up with the first and the last, but if you're going to make such statements, please back them up.
OK.....
Let's cut to the chase then...
Really? The standard says The sender MUST delay retrying a particular destination after one attempt has failed. In general, the retry interval SHOULD be at least 30 minutes; (RFC 2821 section 4.5.4.1)
The "MUST" part is understood as a rule while the "SHOULD" part is a guideline.
Since its inception "sendmail" has had its retry interval defaulted to 15 minutes and the retry limit set to 3 days. Some admins change the defaults and I've seen many times where the a retry is set to 5 minutes. There are also MTA's that increase the retry intervals at each failure of a given email to be delivered. Your phrase "exponential back-off" is a good description.
So, "SHOULD" is a guideline and having set my greylist interval to 15 minutes is perfectly fine.
Calling half an hour "a while" seems reasonable to me...
Never said it was "unreasonable". I only stated what I have as my settings.
I'd argue that your first sentence is misleading, too -- the delay is a result of the configuration of both sending and receiving MTAs.
I'm not sure about "misleading" but certainly "incomplete". I did make the mistake of assuming that the discussion was the delay imposed on accepting a message. But, yes, if I have my greylist set such that I will accept the next retry after 15 minutes and your MTA resends the message 30 minutes later then the delay in deliver will be 30 minutes for the first time.
But you are missing a detail here, and confusing "sending system", "computer", and "IP address". For major providers, the sending system may involve lots of computers, with lots of IP addresses. Retries may come from any of those computers -- this is perfectly legitimate under SMTP. So it may take a while (especially if they use an "exponential back-off") before the same server retries the same e-mail. With enough sending IP addresses, it's possible that the e-mail might never be retried from the same IP address.
You said, "Retries may come from any of those computers" and this is an incorrect statement. While a major provider has many systems sending out emails when an individual email is placed in the queue of a sending system it stays in that system's queue. It doesn't get passed to a different system at the provider's end. So, the retry will come from the same system.
Would you like me to send you my greylist logs to prove it to you?
But you're missing another point -- the more people use greylisting, the less reliable it becomes (because spammers start retrying on any error). If Tony and I choose not to use greylisting, that makes it more usable for you!
There is a word/phrase for that type of "argument", I think it is a "Red Herring" but not sure....
Of course spammers will react to *any* defensive measures put in place and a given defense will reduce in value in time. Why do you think we are seeing more and more spam with single image attachments that are designed to fool OCR programs?
Yet, I don't see the effectiveness of greylisting going down. The greylist's main role is to combat spambots. I'm sure you know what they are, so there is no reason to explain. If creators of spambots would start to build in complexity of retries into their process the return on investment would be small and the users of the infected systems would more likely detect things have slowed down.
These e-mails are counter-productive. If you had addressed them to a new user, they may well have driven them out of the community. If read by new users, they may give the impression that the list is hostile.
Hopefully, a new user would not pass themselves off as an authority on a subject.
It hasn't been, in the past. Don't make it change.
At times it has been...and much more so. In some cases I've seen things spiral down (sometimes rather quickly) to the point of name calling.
Ed Greshko wrote:
- The time to delay is configurable in a good greylist milter. Mine is
set to 15 minutes since this is pretty much the default retry interval of most MTAs.
then:
So, "SHOULD" is a guideline and having set my greylist interval to 15 minutes is perfectly fine.
OK -- I think you missed my point there, but I will agree that your point is also good.
My point -- the standard recommends a retry interval of 30 minutes. That doesn't mean that a shorter interval doesn't follow the standard, merely that a longer retry period should be considered normal, according to the standard.
Your point -- in practice, few MTAs follow the recommendation in their default configuration.
I wrote:
But you are missing a detail here, and confusing "sending system", "computer", and "IP address". For major providers, the sending system may involve lots of computers, with lots of IP addresses. Retries may come from any of those computers -- this is perfectly legitimate under SMTP. So it may take a while (especially if they use an "exponential back-off") before the same server retries the same e-mail. With enough sending IP addresses, it's possible that the e-mail might never be retried from the same IP address.
Ed replied:
You said, "Retries may come from any of those computers" and this is an incorrect statement. While a major provider has many systems sending out emails when an individual email is placed in the queue of a sending system it stays in that system's queue.
You pointed out "SHOULD", I'll point out "MAY" in my statement. For many major senders, what you right is absolutely true. I maintain that it is not universally true, and there are some major exceptions.
I understand that a number of major senders (who have their own, custom-written SMTP engines) do resend from different servers. There is a fair amount of evidence to support this:
http://www.merakmailserver.com/forum/Greylisting_Bypass_Info/m_1441/tm.htm http://en.wikipedia.org/wiki/Greylisting makes this point. http://www.dataenter.co.at/doc/xwall_greylisting_exclusions.htm
I wrote:
But you're missing another point -- the more people use greylisting, the less reliable it becomes (because spammers start retrying on any error). If Tony and I choose not to use greylisting, that makes it more usable for you!
Ed replied:
There is a word/phrase for that type of "argument", I think it is a "Red Herring" but not sure....
Of course spammers will react to *any* defensive measures put in place and a given defense will reduce in value in time. Why do you think we are seeing more and more spam with single image attachments that are designed to fool OCR programs?
Yet, I don't see the effectiveness of greylisting going down. The greylist's main role is to combat spambots. I'm sure you know what they are, so there is no reason to explain. If creators of spambots would start to build in complexity of retries into their process the return on investment would be small and the users of the infected systems would more likely detect things have slowed down.
I think we're pretty nearly saying the same thing -- the more greylisting is used, the greater the return on investment would be. If everyone used greylisting, then spambots would be worthless unless they learned to retry.
It looks as though most e-mail providers who are likely to use greylisting already have it in place, and that most spammers either aren't collecting or analysing reject rates, or they reckon the extra complexity of retrying isn't worth the hassle.
But I am seeing some evidence that a few spammers are retrying even on 5xx permanent rejects (for example, identical e-mails, down to To: From: and Message-ID: fields, from the same IP address).
James.
James Wilkinson wrote:
My point -- the standard recommends a retry interval of 30 minutes. That doesn't mean that a shorter interval doesn't follow the standard, merely that a longer retry period should be considered normal, according to the standard.
One could also say "a shorter retry period should be considered normal" as well. Better still, "it is normal that admins don't change the default settings of a given MTA". I think the use of the word "normal" requires a definition of what is "normal".
Your point -- in practice, few MTAs follow the recommendation in their default configuration.
In actuality, my point is that the RFC only makes a recommendation buy use of the keyword SHOULD and not all MTA's follow the recommendations.
You said, "Retries may come from any of those computers" and this is an incorrect statement. While a major provider has many systems sending out emails when an individual email is placed in the queue of a sending system it stays in that system's queue.
You pointed out "SHOULD", I'll point out "MAY" in my statement. For many major senders, what you right is absolutely true. I maintain that it is not universally true, and there are some major exceptions.
I understand that a number of major senders (who have their own, custom-written SMTP engines) do resend from different servers. There is a fair amount of evidence to support this:
http://www.merakmailserver.com/forum/Greylisting_Bypass_Info/m_1441/tm.htm http://en.wikipedia.org/wiki/Greylisting makes this point. http://www.dataenter.co.at/doc/xwall_greylisting_exclusions.htm
Sorry, I don't consider those "evidence" since they are merely statements by some individuals. The wikipedia entry simply says "or if the retry comes from a different IP address than the original attempt" but it doesn't offer any proof that it does happen in reality. Also, the section this comes from has a disclaimer of " This does not cite its references or sources."
If you really want evidence, I'll send you my logs and you can see for yourself.
I think we're pretty nearly saying the same thing -- the more greylisting is used, the greater the return on investment would be. If everyone used greylisting, then spambots would be worthless unless they learned to retry.
So, greylisting is a good thing to implement.
It looks as though most e-mail providers who are likely to use greylisting already have it in place, and that most spammers either aren't collecting or analysing reject rates, or they reckon the extra complexity of retrying isn't worth the hassle.
But I am seeing some evidence that a few spammers are retrying even on 5xx permanent rejects (for example, identical e-mails, down to To: From: and Message-ID: fields, from the same IP address).
So, you are now making a case for a blacklist. Yes?
Ed Greshko wrote:
You said, "Retries may come from any of those computers" and this is an incorrect statement. While a major provider has many systems sending out emails when an individual email is placed in the queue of a sending system it stays in that system's queue.
I replied:
For many major senders, what you right is absolutely true. I maintain that it is not universally true, and there are some major exceptions.
I understand that a number of major senders (who have their own, custom-written SMTP engines) do resend from different servers. There is a fair amount of evidence to support this:
http://www.merakmailserver.com/forum/Greylisting_Bypass_Info/m_1441/tm.htm http://en.wikipedia.org/wiki/Greylisting makes this point. http://www.dataenter.co.at/doc/xwall_greylisting_exclusions.htm
Ed replied:
Sorry, I don't consider those "evidence" since they are merely statements by some individuals.
<snip>
If you really want evidence, I'll send you my logs and you can see for yourself.
What would that show? I'm attempting to show that it does happen in general, not that it happens to you.
The non-spam e-mail that people get is different. If you don't get e-mail from those major senders, then you won't see the phenomenon in your logs. Good! That means greylisting works especially well for you. (It *may* just also be that your greylisting software handles this automatically for you -- what are you using, by the way?)
It sounds as though I'm going to have to come up with logs from people who use greylisting who have seen retries from different servers. Would enough examples of that convince you? Or are you essentially unconvinceable on this point unless you see it in your own logs?
Part of the problem is that these logs are necessarily going to look anecdotal. Part of the problem is that I'm going to have to rely on logs other people have posted.
http://midori.shacknet.nu/OpenSourceProjects/Setting_up_Greylisting_with_Sen... shows eleven different addresses that Paypal is using to send one message.
http://forum.powweb.com/archive/index.php/t-38837.html includes greylisting logs: hawkers@isecard.com 68.99.120.72
hawkers@isecard.com 68.99.120.69
hawkers@isecard.com 68.99.120.71
hawkers@isecard.com 68.99.120.68
You can see there that the same e-mail from the same individual (the poster's mother) was retried from four different, similar IP addresses. But then, they're excerpts of logs "from some individual".
How much Googling do you want me to do?
Ed wrote:
So, greylisting is a good thing to implement.
Maybe I should have clarified my position, then. Greylisting is a good and valid anti-spam practice that has a few downsides. That's fair enough -- every anti-spam practice has a few downsides. Greylisting has comparatively few downsides -- many have a lot more. It also has a number of upsides that we haven't mentioned[1].
If I've been discussing the downsides of greylisting, that merely means I'm interested in the technicalities. Depending on the particular situation, the best combination of spam techniques may not involve greylisting (because the downsides are greater and the benefits less than for another
I wrote:
But I am seeing some evidence that a few spammers are retrying even on 5xx permanent rejects (for example, identical e-mails, down to To: From: and Message-ID: fields, from the same IP address).
Ed asked:
So, you are now making a case for a blacklist. Yes?
No. I'm making an observation that a few spammers' practices are changing in a way that would defeat greylisting.
What we *do* about that is another question. It certainly doesn't make greylisting worthless, and won't do unless most spammers change.
James.
[1] Spammers using "real" MTAs that do retry may not be on DNSBLs the first time they try sending a spam, but have found their way onto automated DNSBLs by the time the greylist time-out has expired. In this case, the greylist is effectively delaying mail until the sender's had a chance to get themselves caught by the spamtraps.
On average, for one domain that I manage, we get 2500~3000 incoming connections/day.
I use smf-grey, smf-spamd, RBL, and I permanently reject all incoming mails that are not valid recipients in the domain.
About 10% of the incoming mails are delivered the remaining 90% are rejected. Of the delivered messages, less than 0.8% are spam.
All strategies for fighting spam have their advantages and disadvantages. It is not, IMHO, a good idea to rely on a single defense.
I'm happy with my setup. And, I've no complaints from the user community.
Oh, and BTW, smf-grey uses only the Class C subnet for matching the sender's IP. So, even if the same message came from multiple IP's the chances of it causing grief is minimal.
On Mon, 2007-04-30 at 12:40 -0400, Steve Friedman wrote:
The general consensus on the postfix mailing idea is that Greet Pause is a bad idea (TM). What it ends up doing is (a) delay legitimate mail and (b) DoS your own server as you now take longer to handle legitimate mail. Any mail source that would fail greet pause will also fail numerous other checks that don't inconvenience your intended users (and your own system).
How does it work? If it pauses the current connection with that server, independently of any other system trying to send you mail, then only one thing at a time gets delayed, so it shouldn't be a DOS. But if sendmail pauses completely while one thing talks to it, and won't do anything else until that task is completed, yes, I see potential problems.
On Tue, 1 May 2007, Tim wrote:
On Mon, 2007-04-30 at 12:40 -0400, Steve Friedman wrote:
The general consensus on the postfix mailing idea is that Greet Pause is a bad idea (TM). What it ends up doing is (a) delay legitimate mail and (b) DoS your own server as you now take longer to handle legitimate mail. Any mail source that would fail greet pause will also fail numerous other checks that don't inconvenience your intended users (and your own system).
How does it work? If it pauses the current connection with that server, independently of any other system trying to send you mail, then only one thing at a time gets delayed, so it shouldn't be a DOS. But if sendmail pauses completely while one thing talks to it, and won't do anything else until that task is completed, yes, I see potential problems.
It's a DoS because the system can have only a finite number of sockets open (this is both a kernel limit and a postfix tuning parameter limit), and greet pause ties them up doing nothing for a period of time. Recall that postfix is written to support many operating systems and not all OSs (especially the older ones, e.g., linux 2.4) support epoll (enabling greater than 1024 elements in the select()). Consequently, on an active server, legitimate connections will be denied because of a lack of an available socket and thus you've denied service to a legit user.
Steve
Steve Friedman wrote:
How does it work? If it pauses the current connection with that server, independently of any other system trying to send you mail, then only one thing at a time gets delayed, so it shouldn't be a DOS. But if sendmail pauses completely while one thing talks to it, and won't do anything else until that task is completed, yes, I see potential problems.
It's a DoS because the system can have only a finite number of sockets open (this is both a kernel limit and a postfix tuning parameter limit), and greet pause ties them up doing nothing for a period of time. Recall that postfix is written to support many operating systems and not all OSs (especially the older ones, e.g., linux 2.4) support epoll (enabling greater than 1024 elements in the select()). Consequently, on an active server, legitimate connections will be denied because of a lack of an available socket and thus you've denied service to a legit user.
Good luck at explaining that to rabid anti-spam fanatics who don't care how much damage they cause others in their quest to avoid having to hit the delete key.
On Tue, 2007-05-01 at 09:41 -0400, Steve Friedman wrote:
On Tue, 1 May 2007, Tim wrote:
On Mon, 2007-04-30 at 12:40 -0400, Steve Friedman wrote:
The general consensus on the postfix mailing idea is that Greet Pause is a bad idea (TM). What it ends up doing is (a) delay legitimate mail and (b) DoS your own server as you now take longer to handle legitimate mail. Any mail source that would fail greet pause will also fail numerous other checks that don't inconvenience your intended users (and your own system).
How does it work? If it pauses the current connection with that server, independently of any other system trying to send you mail, then only one thing at a time gets delayed, so it shouldn't be a DOS. But if sendmail pauses completely while one thing talks to it, and won't do anything else until that task is completed, yes, I see potential problems.
It's a DoS because the system can have only a finite number of sockets open (this is both a kernel limit and a postfix tuning parameter limit), and greet pause ties them up doing nothing for a period of time. Recall that postfix is written to support many operating systems and not all OSs (especially the older ones, e.g., linux 2.4) support epoll (enabling greater than 1024 elements in the select()). Consequently, on an active server, legitimate connections will be denied because of a lack of an available socket and thus you've denied service to a legit user.
Then you must also consider connection limiting and throttling DoS as well. Your facts don't line up with reality. This system can and does work well, when sendmail and the system are configured to make allowances for the delay, even when each server is processing over a million messages per month.
On Tue, 1 May 2007, Guy Fraser wrote:
It's a DoS because the system can have only a finite number of sockets open (this is both a kernel limit and a postfix tuning parameter limit), and greet pause ties them up doing nothing for a period of time. Recall that postfix is written to support many operating systems and not all OSs (especially the older ones, e.g., linux 2.4) support epoll (enabling greater than 1024 elements in the select()). Consequently, on an active server, legitimate connections will be denied because of a lack of an available socket and thus you've denied service to a legit user.
Then you must also consider connection limiting and throttling DoS as well. Your facts don't line up with reality. This system can and does work well, when sendmail and the system are configured to make allowances for the delay, even when each server is processing over a million messages per month.
Your server; your rules; however, I am interested neither in debating semantics nor foolish configurations. One million messages per month -> 10^6 / (30 * 86400) = 0.385 messages / second. This is a low traffic site. As an aside, I would suggest that you whitelist servers from which you've already accepted mail to avoid foolishly penalizing your intended correspondents.
Steve Friedman
Tim (about sendmail greet pause):
How does it work? If it pauses the current connection with that server, independently of any other system trying to send you mail, then only one thing at a time gets delayed, so it shouldn't be a DOS. But if sendmail pauses completely while one thing talks to it, and won't do anything else until that task is completed, yes, I see potential problems.
Steve Friedman:
It's a DoS because the system can have only a finite number of sockets open (this is both a kernel limit and a postfix tuning parameter limit), and greet pause ties them up doing nothing for a period of time.
This is a genuine question: Is that actually worse than having the server tied up dealing with lots of spam?
I would imagine that anyone who wanted to try this approach, would also want to increase the number of sockets that could be handled, to avoid getting DOSd.
It would also seem prudent to reset a connection if more traffic came through when you'd told it to wait.
On Wed, 2 May 2007, Tim wrote:
Tim (about sendmail greet pause):
How does it work? If it pauses the current connection with that server, independently of any other system trying to send you mail, then only one thing at a time gets delayed, so it shouldn't be a DOS. But if sendmail pauses completely while one thing talks to it, and won't do anything else until that task is completed, yes, I see potential problems.
Steve Friedman:
It's a DoS because the system can have only a finite number of sockets open (this is both a kernel limit and a postfix tuning parameter limit), and greet pause ties them up doing nothing for a period of time.
This is a genuine question: Is that actually worse than having the server tied up dealing with lots of spam?
The key part of my initial response is "and doesn't affect legitimate mail". For example, postfix can detect pipeline violations without a greet pause -- using marginally more WAN bandwidth than greet pause but doesn't affect legitimate mail. My other point was "just how much spam is blocked using greet pause that wouldn't be caught using other means". I claim (without any data to back me up :-) that any spambot so poorly written as to be trapped by greet pause would also be trapped by less intrusive means (e.g., reject_rbl_client zen.spamhaus.org, greylisting, reject_unauth_pipelining). Thus, you and your legitimate peer servers (which also must wait for the greet pause) are paying a price without reaping any benefit.
I would imagine that anyone who wanted to try this approach, would also want to increase the number of sockets that could be handled, to avoid getting DOSd.
If one truly wanted to go down this path, then the "correct" solution would be to populate a database with the IP addresses of all peers from whom you have accepted e-mail and not greet pause those servers. Then, like greylisting, you wouldn't affect your usual correspondents after the initial training period.
It would also seem prudent to reset a connection if more traffic came through when you'd told it to wait.
Agreed, but if you wait to detect a pipelining violation until the DATA phase (rather than simply at the HELO phase), you can accomplish something during that time instead of just pausing.
Steve Friedman
Tim wrote:
How does it work? If it pauses the current connection with that server, independently of any other system trying to send you mail, then only one thing at a time gets delayed, so it shouldn't be a DOS. But if sendmail pauses completely while one thing talks to it, and won't do anything else until that task is completed, yes, I see potential problems.
Steve Friedman:
It's a DoS because the system can have only a finite number of sockets open (this is both a kernel limit and a postfix tuning parameter limit), and greet pause ties them up doing nothing for a period of time.
This is a genuine question: Is that actually worse than having the server tied up dealing with lots of spam?
Remember, there are two sides of the connection. The other end may be a trying to deliver hundreds of mail list messages to millions of subscribers while you sit there intentionally delaying it, wasting one of it's sockets and the RAM associated with your connection attempt.
I would imagine that anyone who wanted to try this approach, would also want to increase the number of sockets that could be handled, to avoid getting DOSd.
It would also seem prudent to reset a connection if more traffic came through when you'd told it to wait.
You don't tell it to wait - it is supposed to wait for your response and many spam senders don't. However, there are legitimate mail sources that will drop your connection and move on if you delay very long before responding.
-
Alan -
Alexander Dalloz -
CH -
Ed Greshko -
Guy Fraser -
James Wilkinson -
Les Mikesell -
Manuel Aróstegui -
Sam Varshavchik -
Steve Friedman -
Tim -
Tony Nelson