Spam was killing us! Here is what we did to help!
I work for a smallish ISP in the Midwest. In late September and the month of October we began getting blasted with spam and DHA's from all over the world. We had been utilizing a spam filtering service but it was not keeping up. We billed the customers for the service and they were starting to complain. They were getting so much spam in their inboxes that they felt like they were wasting their money. In October when the problem became so bad that it started affecting our mail servers ability to process mail any longer we knew we had to do something. We had been "test driving" a spam filter device by Red Condor. The accounts that had been moved over to the Red Condor filter were virtually spam free. We decided to implement the Red Condor solution across the board on the server that was being hammered the worst. This server has just over 9,000 accounts on it. We turned up the Red Condor box at about 4pm and by 7:00am the next morning the quarantine boxes had been created for all customers. No interaction required, it simply verified each inbox as the emails arrived for the account. If the account did not exist it threw the spam away, if the account did exist it created the inbox and then determined whether the email was spam or was legit (autodiscover does not work with Exchange Servers).
We decided to "give the service away" as part of the customers Internet service. In reality we have been the ones to benefit from the service. The mail server has been purring along for months now and our customers are much happier. They literally have had no spam hit their inboxes. We have been in the learn mode for a while and slowly started migrating other customers over to the device. It has not missed a hit. The other thing that is amazing is the ease in setting up the "accounts" on Red Condor. With the previous service it was about a 15 minute process to setup each domain. It was a series of long drawn out steps to setup the accounts. With Red Condor it takes less than a minute to setup a new account/domain. If I can use autodiscover to create the inboxes then the setup task is done. Change the MX record and I am good to go.
Now here is the amazing part. The reporting available with the product is unbelievable. At a glance I can see just how much work this single device is doing. Here is a report for the domain that has just over 9,000 accounts. This is a summary of the transactions handled for the domain since March 1, 2010. You see that out of almost 20 million emails handled only 713,222 (3.6%) were actually delivered.
March 2010
|
Disposition |
|
||||||
Category |
Deliver |
Markup |
Quarantine |
Block |
Total |
|
Size |
|
OK |
638,116 |
|
|
|
638,116 |
3.2% |
108GB |
32.1% |
Unprotected |
2,905 |
|
|
|
2,905 |
0.0% |
60MB |
0.0% |
Friends |
72,201 |
|
|
|
72,201 |
0.4% |
17GB |
5.2% |
Enemies |
|
|
176 |
|
176 |
0.0% |
31MB |
0.0% |
Virus |
|
|
|
55,587 |
55,587 |
0.3% |
7,109MB |
2.1% |
Phish |
|
|
434,661 |
2,218 |
436,879 |
2.2% |
1,165MB |
0.3% |
Keyword |
|
|
|
|
0 |
0.0% |
0 |
0.0% |
Adult |
|
|
|
106,296 |
106,296 |
0.5% |
270MB |
0.1% |
Spam |
|
919 |
13,412,089 |
42,939 |
13,455,947 |
68.1% |
154GB |
45.9% |
Junk |
|
1,718 |
349,796 |
697 |
352,211 |
1.8% |
9,223MB |
2.7% |
Blank |
|
|
489 |
2 |
491 |
0.0% |
1,073KB |
0.0% |
Foreign |
|
|
12,707 |
33 |
12,740 |
0.1% |
159MB |
0.0% |
Risky Attachment |
|
|
16 |
|
16 |
0.0% |
18MB |
0.0% |
Unresolved Sender |
|
|
|
|
0 |
0.0% |
0 |
0.0% |
Invalid Recipient |
|
|
|
4,623,107 |
4,623,107 |
23.4% |
38GB |
11.3% |
Total |
713,222 |
2,637 |
14,209,934 |
4,830,879 |
19,756,672 |
|
335GB |
|
|
3.6% |
0.0% |
71.9% |
24.5% |
It isn't hard to understand now why my poor mail server was weeping on a daily basis. We are now in the process of moving the remaining customers, accounts and domains over to the Red Condor system.
Spam and viruses have become such a big problem for ISP's world wide. Until we can clean up the infected machines that are generating this spam and shut down the bad guys that are pushing this garbage at us, it is good to know that these types of systems exist.
I would like to hear from our reader's. What has helped your organization deal with spam and the pr
Comments
Cricket
Mar 17th 2010
1 decade ago
BillR
Mar 17th 2010
1 decade ago
$.02 deposited
Greg
Mar 17th 2010
1 decade ago
ChadS
Mar 17th 2010
1 decade ago
EdE
Mar 17th 2010
1 decade ago
Emails sent between accounts hosted at the same provider can possibly be identified as spam even *after* delivery (after other recipients have complained), so that too gives the larger email providers an edge over external victims of their spam.
I think spam will continue to hit smaller providers hardest, nudging people toward the major webmail providers and probably suffering a CAPTCHA every other outgoing message (or pay to 'go pro'). And business users would be pushed toward their outsourced email offerings, probably paying by volume of sent mail. I think that's a sad place for any Internet-based service to end up.
So I guess SMTP was fatally flawed. Maybe its next incarnation would define a good, standard 'feedback loop' for reporting spam and maybe even the ability to 'recall' a message after sending it. Systems downstream from it would need to be prepared for messages to be recalled also.
In the meantime it might be smart to reduce dependency on email. In the context of a web application, do you really *need* the user to provide you with an email address? Can you offer alternate contact methods? Can you handle account signup and authentication some other way, such as OpenID, providing unique URIs, client certificates or cryptographic tokens? Maybe you could still provide your service without requiring login at all? Could you offer your email subscriptions or notifications via RSS or other means?
Steven Chamberlain
Mar 17th 2010
1 decade ago
Subelman
Mar 17th 2010
1 decade ago
Winders
Mar 18th 2010
1 decade ago
craigF
Mar 18th 2010
1 decade ago
Papdheen
Mar 18th 2010
1 decade ago