|
Ascent Member Posts: 64 From: Registered: 01-20-2001 |
Hey everyone, I'm working on a new anti-spam program that goes out to the (pop) email server, detects which emails are spam, and then according to the level you set, deletes them or warns you, etc. I'm primarily focusing this on non-outlook email clients, as there seem to be a handful of outlook addons available already, but it works with a normal pop server, so if you're *not* using exchange with outlook, it should work fine. It should also work fine for netscape, mozilla, eudora, etc. So... here's my thing... I'd like to get some people to help me test it *soon* (not yet) if anyone tends to get a lot of spam. I've got a mailbox that I've all but abandoned, and it gets probably around 500 messages a week, mostly spam. So far, from processing the subject and sender only (at this point) it determines that 430 of the 530 currently there are probably spam, and of those, 300+ of them are almost definitely spam. I'm still working on it, but I should be ready to start giving it out to some people soon. So why am I posting it here? Well, among other reasons, I think something like this might help believers who end up getting the kind of spam they don't want to, and I'm sure most of you know exactly what I mean by that. So, is anyone interested in helping me? If so, post a message here, and let me know what you use for email, and maybe how much spam you get (few a day, hundred a day, etc) Thanks, |
||
Klumsy Administrator Posts: 1061 From: Port Angeles, WA, USA Registered: 10-25-2001 |
i am very keen.. spam gets up my nose , and immoral, and blatently evil spam is the worst. ------------------ |
||
Mack Administrator Posts: 2779 From: Registered: 01-20-2001 |
I'm willing to give it a go, I get quiet a few spam, more so viruses sent to my e-mail. ------------------ |
||
Believer Member Posts: 80 From: British Columbia, Canada Registered: 01-30-2001 |
I created a spam filtering system called 'PEP' (which stands for Personal E-Mail Processor). It's a server-side thingy that our customers can use to filter out spam and viruses before it gets to their mailbox in the first place. Currently in use by about 10,000 people. You can read about it here if you want, might give you some ideas for your own project. | ||
CobraA1 Member Posts: 926 From: MN Registered: 02-19-2001 |
You could take a look at Pegasus Mail and Mercury Transport also - they have some powerful filtering capabilities. http://www.pmail.com/index.cfm That PEP thing looks pretty powerful also. I'm afraid I can't help you much with testing - I don't get much spam. God be with you on your project. |
||
|
Klumsy Administrator Posts: 1061 From: Port Angeles, WA, USA Registered: 10-25-2001 |
woah that pegasis mail and mercury think look awesome and because you can write plugins for almost every thing, you can use it as a base, and just write your algoritmns for your advanced filtering if you want, or even try to write a plugin to integrate that peer to peer spam filtering software... ------------------ |
||
|
graceworks Member Posts: 455 From: Corvallis, Oregon, USA Registered: 03-03-2001 |
Sign me up for testing. I'll check out the other links too. Ascent's program sounds somewhat like a program called MailWasher, which I only used for a bit. MACK - are you getting much spam that is titled, "When is IT shipping?" yet? :-) -Tim ------------------ |
||
|
Ascent Member Posts: 64 From: Registered: 01-20-2001 |
It looks like there's more interest and more programs out there than I thought. That's good. Tell me, GraceWorks, why did you only use that program for a little while? -Ascent |
||
|
Mack Administrator Posts: 2779 From: Registered: 01-20-2001 |
quote: Those are always welcome ------------------ |
||
c h i e f y Member Posts: 415 From: Surrey, United Kingdom Registered: 03-07-2002 |
spammers only SPAM because ppl ACTUALLY reply to their unsolicited e-mails  if they never GOT any replies, the whole SPAM thing would be a no go so just WHO ARE the complete GIMPS who REPLY to SPAM ?? let's get a LIST together ! ------------------ |
||
D-SIPL Moderator Posts: 1345 From: Maesteg, Wales Registered: 07-21-2001 |
quote: Not me, i've had a hotmail account for two years, and rarely get any spam at all, and thats with the filter turned off. So yeh lets get a list of the ppl who reply, we can then sell the list to a third party and generate some revenue for the christian games industry!  --D-SIPL |
||
InsanePoet Member Posts: 638 From: Vermont, USA Registered: 03-12-2003 |
I've used yahoo mail for about a year and I never gotten spam myself. ------------------ |
||
|
Believer Member Posts: 80 From: British Columbia, Canada Registered: 01-30-2001 |
quote: This is partially true, but plenty of them do it whether they get responses or not. They make their money by spamming for others, or they're members in some silly afilliate marketing system (DHS club, for example) and have convinced themselves that it all helps, even if there's no direct response. Sadly, some spammers are pushing OPINIONS and BELIEFS and CAUSES instead of products. There are even a few "christian" outfits out there using spam to spread the gospel. They don't understand that spam = theft (of time, bandwidth, disk space) and that they are damaging reputations. One of these is Crosswalk. You can sign anyone up for any of their mailing lists without any confirmation. I've had to blacklist them here to stop getting stuff I never asked for in the first place (they ignored all my e-mails and even the web feedback forms).
quote: Not quite so simple, unfortunately.
quote: Well, this morning I had 10 misdirected unsubscribe requests in one of my postmaster accounts. All attempts by the same person to get off spam lists. She's a nun at a local monastery. I bet she'd forgive you for calling her a GIMP though  Seriously though, I think you're referring to the folks who actually respond POSITIVELY to spam, and yes those people are twits. Folks who reply in an effort to get removed are only confirming that their address is valid, and they need to be educated, but they aren't putting money in the spammer's pockets or anything like that at least. Personally I refuse to do business with anyone who spams. I'll never buy a book from Amazon, for example.
[This message has been edited by Believer (edited November 29, 2002).] |
||
|
c h i e f y Member Posts: 415 From: Surrey, United Kingdom Registered: 03-07-2002 |
hi Believer, I quite agree with you squire  in fact, I propose an ACTUAL CHANGE IN THE LAW !! yes, check out my mob handed approach to this above, I reckon spamming should be made illegal ! ------------------ |
||
|
Christian Member Posts: 400 From: Australia Registered: 09-15-2002 |
Wow - count me in for any testing you want done. I hate the stuff, and I get heaps of it. My email is cgraus@viewbuild.com. |
||
rowanseymour Member Posts: 284 From: Belfast, Northern Ireland Registered: 02-10-2001 |
"The web activists have found the personal details of the man behind a federal surveillance system and an e-mail spammer and are giving them a dose of their own medicine." http://news.bbc.co.uk/1/hi/technology/2580089.stm I recently implemented a system of hiding email addresses on the [url=]GODCENTRIC[/url] site. All the email addresses are stored as name*NOSPAM*godcentric.com, and then converted by some client side Javascript to real addresses before they are displayed. This means they can't be read by email spiders. I'm going to try this http://www.mailwasher.net . It can send faked "delivery failure reports" to spammers so that it looks like your address doesn't exist anymore, and it works with both Hotmail and POP3. I'll tell you all if it works... ------------------ |
||
|
CobraA1 Member Posts: 926 From: MN Registered: 02-19-2001 |
quote: LOL
quote: Cool. Could I use the script on my webpages? Including pages for a small business I might start up? MailWasher looks cool. I'll recommend it to any friends having troubles with spam. |
||
|
rowanseymour Member Posts: 284 From: Belfast, Northern Ireland Registered: 02-10-2001 |
Help yourself - it's really simple In your Javscript...
In your HTML...
------------------ |
||
|
Believer Member Posts: 80 From: British Columbia, Canada Registered: 01-30-2001 |
Shameless plug (but it's totally free)... If anyone is interested in playing around with my PEP product, you can create a free e-mail account on www.christian.net - it now supports PEP (and it provides both a webmail interface as well as POP3 access). PEP lets you create a set of rules that get applied to incoming mail before it even reaches your mailbox. You can generate false bounces with a customized error message, delete messages, forward or copy them, check them through Spam Assassin and test the resulting spam score, and so on. One feature it has is a "challenge" system. A database is maintained (if you enable this feature) for each user that contains addresses of people known to you. If a message arrives from one of these people, the message is delivered normally. Otherwise, the message is "frozen" and a challenge message (that you can write) is sent back to the sender. They must then reply to the message to "thaw" it. It doesn't get delivered to you until they reply. Works great because I've found that *most* spam comes from either completely bogus addresses, or from temporary mailboxes that don't get read, etc. So a filter like this can be really effective (lines with hash marks are comments):
Of course you can add any number of additional rules. You can "bounce", "keep", "forward", "copy", etc. You can test any header or even multiple headers. You can do exact matches, wildcard matches, regex matches, file lookups, etc. The problem with Mailwasher and similar products is that the viruses and spam get delivered to your mailbox, so it's possible for you to lose legit mail due to a full mailbox. When you run Mailwasher, it connects to your POP server just like your mail client does, which means it has to download at least the headers for each message in order to determine whether or not it is spam, which wastes bandwidth. And determining if a message is spam based solely on the headers is NOT very accurate these days, so it might have to download the entire message anyway, which means more wasted bandwidth. I'd rather not get the spam/viruses deliveries in the first place, which is what PEP does. It's been in use by some 10,000 users for many years now, so it's pretty robust and solid. |
||
|
Ascent Member Posts: 64 From: Registered: 01-20-2001 |
Finding spam from the header and subject is probably quite a bit more accurate than you think. That's the way my program works, and out of 850 emails or so, it identified 500 as most definitely spam, and another 100 as probably. On that particular email address, spam is *almost* all I get, so that's around 75% accuracy. Just from the subject and sender. I think if you use a combination of looking for words and patterns (as spam assassin does) as well as looking for / blocking certain domain names, you can do quite a bit. Of course, your solution is a more complete solution, and I know that quite a few good ones exist for linux and exchange server, my program (and mailwasher, from what I can tell) target someone who already has an existing email account that they still need to use, but want to clean up the incoming mail so they don't have to do it manually. Some poeple don't have the option of switching email addresses (or using a server based program) and may be stuck with what they've got. And I guess if you got a new email address, you wouldn't really get spam there, would you? For those (still) interested, my program is coming along pretty well, and I should have a test version out in the next month or so. -Ascent |
||
|
Believer Member Posts: 80 From: British Columbia, Canada Registered: 01-30-2001 |
quote: ;-) Perhaps. I've been writing spam filtering and prevention software for 8 years or more now (ranging from custom SMTP server software, to delivery mechanisms like PEP, to custom POP clients). My personal observation is that it's becoming harder to identify spam based strictly on the headers.
quote: Certainly in some (many) cases the headers alone are sufficient, but that's becoming less and less so, unfortunately.
quote: Yes, absolutely. A lot of spam can indeed be filtered based on the sender's domain and/or IP address alone. And in your tests you seem to get a high percentage, but in mine I'd wager that far less than 50% of spam can be caught that way (without a lot of false positives). Still a lot better than nothing. I presume your software can be configured to consult the various DNSbls like SPEWS, RFC-Ignorant, etc, yes?
quote: Good points, I totally agree. I didn't mean to cut down your solution, it's perfectly valid and definitely worth pursuing. I just wanted to point out that it's more of an "after the fact" solution, that some of the damage from spam has already occured (mainly waste of bandwidth and time).
quote: Heh, you might be surprised. With the increase in dictionary style spam runs, you'd have to pick a rather bizarre username to avoid it for any length of time. Depending on the name you pick, you might instantly be the recipient of tons of spam, or you might go for quite a while without any. Looking at my mail logs for rejected mail, I can see certain specific christian.net email addresses that would be swamped with spam if someone chooses one of them. Odd, since those e-mail addresses have NEVER existed at any time (I'm the only person to ever have this domain, and until a couple months ago it was home to only a few email addresses). It's due to the fact that many spammers are known to invent e-mail addresses by combining a domain with usernames drawn from other domains, spelling lists, and even randomly generated words.
quote: Cool, look forward to seeing it. [This message has been edited by Believer (edited December 19, 2002).] |
do you ??