Whitelists, Blacklists, and the Great Spam Filter Scam

By Duane Thresher, Ph.D.          June 15, 2019

Spam is unwanted email and is often dangerous and offensive. Dangerous because it is a leading hacking method (e.g. phishing) and offensive because it often involves porn. Spam has become such a big problem that many people have drastically reduced or even stopped using email. This is an incredible waste since email is the best form of communication — you can inexpensively and instantly send large amounts of text, documents, photos, audio, and video at any time and it can all be sent securely. Spam filtering is used to address the problem but the most common methods — because they are the most profitable to the companies that produce them — are frustratingly bad, often causing the loss of important wanted emails while still allowing dangerous and offensive spam. There is a simple, free, highly effective spam filtering method, whitelists, but spam filter producers and email service providers don't want you to use them because they are free and highly effective.

Email considered spam is filtered by automatically being put in a spam folder (a.k.a. quarantine), or rejected outright, by the email service provider using that spam filter, which may be its own, like Google Mail (Gmail), or some spam filter company's. The outright rejection is often done without notifying anybody, which violates a founding principle of email: each email should be accounted for. A few years ago I had an extremely IT incompetent email service provider, 3 Rivers, that used a spam filter, Barracuda, that could not be turned off, and I only found out it was rejecting emails — legitimate important ones — because their senders notified me via U.S. Mail. How many other emails I didn't get I have no way of knowing.

Spam filters can filter emails based on content or sender.

Spam filters based on content are ridiculously ineffective and of highly questionable ethicality. The emails are read — technically by a computer program — for key words or phrases supposedly indicative of spam, but spammers can easily trick these programs and they get better at doing so faster than the programs get better, regardless of all the nonsense spewed about artificial intelligence; see Artificial Intelligence: Savior, Antichrist, or Hyperbole?

I say "technically by a computer program" because, regardless of what Google says, these programs are written by humans so your email is in effect being read by a human. Gmail reads your email both for its spam filter and for targeting ads at you (and via Google's collusion with the National Security Agency (NSA), for supposed national security reasons; see Google: Invasion of the Email Snatchers). None of this can be turned off by you. Being human, programmers, particularly those at leftist liberal Google, often incorporate their biases, particularly political, into their programs; for example, automatically putting politically conservative campaign emails into the online-only spam folder, which most rarely check and in which emails are automatically deleted after 30 days. I've experienced this personally.

Spam filters based on content are also processor-intensive — every word of the email has to be read and analyzed. Whether the spam filter program is run on the spam filter company's computer or your email server (your email service provider's computer) or your email client (your computer), it can dramatically slow down getting your email. I've suffered through quite a few spam filters I couldn't turn off and some of them delayed emails by a full day. When the speed of email falls to that of the U.S. Mail (see The U.S. Mail SHOULD Be Worried About Email Competition), of course people stop using email.

As ineffective as they are, spam filters based on content are still very profitable. Companies often point out their complexity as justification for their high price, completely ignoring their ineffectiveness!

Much better are spam filters based on sender. Primarily, the sender can be identified by his email address or his email server computer IP address. (There is also the email server computer hostname but this is not as useful as, and is closely tied to, IP addresses, so is not used much.)

Everyone knows what an email address is; for example, sergey.brin@gmail.com (Sergey Brin is the Russian co-founder of Google).

Computers on the Internet, including email server computers, have numerical addresses, IP addresses, like 172.217.8.5. When you email someone, you have to tell your email server computer the IP address of that someone's email server computer. People have a hard time remembering numbers like that but are much better with names, like gmail.com, which is a domain name. The Domain Name System (DNS) is a system of computers on the Internet, queried by email server computers, that translate domain names into IP address numbers, like gmail.com into 172.217.8.5. See ICANN Do Whatever I Wants.

A sender email address is more specific than the IP address of an email server computer, which may have thousands of senders, so would seem the best to use for spam filtering — and it is; see ahead — but it is far easier than an IP address to spoof, i.e. fake, and many spam filter companies use this as an excuse not to use it.

Companies that make spam filters based on sender IP address create huge lists, i.e. blacklists, of IP addresses of email server computers they consider — for whatever reason — to be spammers. Similarly as for spam filtering based on content, the reason a spam filter company puts the IP address of an email server computer on its blacklist may not be what you assume and could even be political.

And the blacklisted email server computer needn't be dedicated to sending spam. It could just have one bad apple — albeit encouraged by the greedy email service provider (see ahead) — among thousands of innocent senders. They all get tarred with the same brush.

Many years ago I used to have an IT incompetent email service provider, Sovernet, where this happened regularly — they happily allowed a few spammers and then all their customers had their email filtered. It was one of the reasons I quit Sovernet, but also one of the reasons I became an email expert and started setting up my own email servers. (I also quit Sovernet because several times when I had problems downloading my emails, due to their IT incompetence, their tech support offered to read my private emails to me over the phone!)

If an email service provider is put on one of these blacklists, it is probably put on many. To get off the blacklists, the email service provider has to individually contact the spam filter companies and convince them they are not spammers. This is time-consuming frustrating work so email service providers are reluctant to do it, and don't unless they get a lot of complaints from their own customers. And most of their customers have no idea what is going on.

A blacklist then, is a list of those not allowed to send email to you. A whitelist is a list of those allowed to send email to you.

Spam filter companies make blacklists because they can't sell you a whitelist and they make millions selling blacklists. Only you can make a whitelist for you and can do so very easily and for free.

Think about how absurd a blacklist is compared to a whitelist. Do the guest lists for your parties list the 7.7 billion people who are not invited or those few who are invited?

The world has millions of ever changing senders you don't want to get email from compared to just a stable few that you do. Trying to maintain the huge ever changing list of the former is absurd, if it weren't so profitable.

Whitelists are how nature works. For example, your kidneys don't recognize the millions of bad things (toxins) to pull out of your blood to excrete; they recognize just the few good things to keep and excrete everything else. (Mother Nature is very clever, although she has had millions of years to figure it out.)

Blacklists often have high false positives, i.e. they reject many senders you want to get email from. A whitelist never rejects someone you want to get email from; that's why they are on the whitelist. And what are the chances you would have been pleasantly surprised by an email from someone not on your whitelist? Probably zero. Further, senders who know they are on your whitelist are more likely to email you because they are more confident you will get the email and it won't just end up in a spam folder somewhere. It's also flattering to be on someone's A-list, which you can also think of a whitelist as.

Conversely, blacklists also often have high false negatives, i.e. they allow many senders you don't want to get email from. A whitelist never allows someone you don't want to get email from; that's why they are not on the whitelist. When someone not on your whitelist emails you they get a rejection email. This is a very satisfying effective way to say "screw you spammers!" (admit it, you've yelled that at your computer, pointlessly) or reject someone who used to be on your whitelist; much better than unfriending or blocking them on Facebook or Twitter.

Email service providers say (disingenuously; see ahead) whitelists are too hard for customers. These same customers may have created phone number lists on their cell phones longer than their email whitelist would be and that includes individual photos and ring tones for each number.

Whitelists using IP addresses might indeed be too hard for customers but sender email addresses would not be; they are probably already in the customer's email address book. These email service providers, and their spam filter companies, will then say that sender email addresses are too easily spoofed. But that is nonsensical because a spammer/spoofer doesn't know who is on each individual recipient's whitelist and is not going to go to the immense effort to find this out; spammers depend on huge lists of recipients. In short, a whitelist can act as sort of a password for those who want to email you.

Despite what they say, all email service providers love spammers. Spammers are not just Nigerian get-rich-quick scammers and porn sellers. Almost all so-called legitimate businesses are spammers. They constantly try to trick you into giving your email address and agreeing to allow them to spam you. Contacting a business online involves giving your email address and searching for the tiny obscurely-placed pre-checked "I agree to being spammed" box (phrased a little less obviously of course; usually like they are doing you a favor). If you have ever frustratingly wondered why you are getting spam from a company, that tiny obscurely-placed pre-checked box is the reason.

These business spammers are where email service providers make their big money, their cash cow. Most don't make any money from individuals like you — you probably got your email account free. They charge these spammers to send out large quantities of spam. The more the spammers send, the more the email service provider makes.

Obviously, email service providers are not really going to help you to block these business spammers, even if they do offer you spam filtering. Why would the spammers pay them if they knew the spam they sent was just going to be blocked by the same email service provider's spam filter?

Similarly, and most importantly, why would email service providers let you have a whitelist to block these spammers?

Apscitu Mail exclusively uses whitelists. Since Apscitu Mail is a custom service, you simply supply us with the list of email addresses you want to be allowed to send you email — probably already in your email address book and we can help you figure it out — and we quickly take care of the rest. Update whenever you need to, although you'll find it won't be that often; your allowed sender list will be rather stable.