I have used real-time blocklists myself since a dozen or so years ago.  I've worked for companies and managed servers that have been listed on blocklists more than once unwarranted. I can't help but notice some huge changes between the granddaddy lists I did support and some of the current breed I'd stay away of.

As with all things the most negative experiences will stand out, but there's a lesson to be learned in how to detect the "bad" blocklists and how to avoid them.

The unintentional user

First, how do you know you are using a blocklist? You don't, unless you start to hunt for it. E.g. your google toolbar has a blocklist of sites it thinks are a bad idea to surf to^(*). It'll warn you about a supposedly bad website you really might be willing to avoid. But how many more blocklists are you using without having intentionally configured, chosen and vetted the processes behind it?

If you use e.g. a sendmail configuration file that you didn't write, how do you know it isn't using some blocklist to tune down the volume of spam.

If you're an unintentional user, you're not in control of the choices being made and you and your peers might in the end suffer badly. So the advise is to seek out what blocklists you are using and go from the unintentional to the intentional user.

False positives - false negatives

True positives and true negatives will mostly go unnoticed but the other two can be problematic. A false positive is e.g. a blocklist for spammers that contains well behaving Internet users. Those users (might be your supplier, your customers, ...) can't communicate anymore with you, and might give up on you as you just seem to be ignoring them rudely.

The false negatives are what will prompt some into searching ever more strict rules as there is still spam sneaking through. We know that getting ever more strict measures will also increase the false positives rate dramatically.

For things like spam where the spam outnumbers the genuine messages dramatically if your address is well known, getting spam free with a blocklist is likely to cost you most if not all genuine messages as well. Basically blocking all email will guarantee you no false negatives, but it'll also guarantee all genuine messages turn out to be false positives.

Measuring false negatives is terribly easy for e.g. spam lists, while measuring false positives is next to impossible. Just measuring how much email got blocked says nothing about it all, and if you need to read the messages in order to be able to measure the effectiveness , you might just as well delete the spam by hand.

Criteria

Some criteria we could suggest to choose blocklists:

• Speed of reaction: The faster (the more real-time) a list is updated, the more easier it is to deal with false positives and with false negatives.
• Selection criteria: How are the sources added to the blocklist, based on what criteria? How sure are the blocklist admins that the one they are listing is bad? How sure are you they will not add your partners, customers, suppliers and other business critical peers. Similarly how sure are you they will not list yourself (from experience: this is extremely painful)
• Goal of the blocklist: Does the list have an agenda (hidden or not) that you might not share with them? Do they aim to have 0 false negatives without care for false positives?
• Ease of getting unlisted: How easy is it to contact the list administration for those listed ? Is there 24x7 (remember the Internet is worldwide so thy need to cover all timezones) support on getting back out for those unjustly listed ?
• Working Email contact to get unlisted: This is very tricky for e.g. spam blocking list that are using their own blocklist.
• Try contacting them to get unlisted: if you cannot reach them, remember what your communicating partner that got listed by accident will feel like. And while it might reflect mostly on the blocklist provider, it will also reflect on you and your organization due to your choice and implicit support of their (failing) processes.
• Is there somebody who feels responsible enough behind it to put up out of band contact details such as phone numbers, working snail-mail addresses etc. Of course this means they'll feel exposed to the scam artists they are blocking, but it also means those being blocking without reason have a way to complain.
• Blocking for the right reasons: E.g. some anti-spam lists are blocking with as reason the IP addresses sent unwanted TCP/IP traffic (not just unwanted email). Some might have political reasons or other things you don't want to be associated with.
• Duration of a block: many IP addresses that get infected by bots etc. are home users on a (somewhat) dynamic IP address. Blocking such an IP address for a long time won't help as the IP isn't fixed and the next one to come after it will get blocked unwarranted. Similarly, infected machines do eventually get cleaned up by the rightful owners. So short durations are better.
• Automatic delisting: How automated is the unblocking? Based on what tests is it done? Some listed entities might not know what blocklist they are being listed on. Hence asking them to jump through hoops on their own might not be hard, they might not even know what hoops they need to seek out.
• Granularity of the block: Unless there are clear signs of malice, most regular users will clean up intrusions and malware instead of hopping about the IP address in an address space to avoid blocklists. Hence only very bad neighborhoods should get blocked indiscriminately. Similarly "punishing" an ISP for having a single misbehaving customer will not work as the ISP is hardly punished at all, it's the other (innocent) customers of the ISP that get hit.
  While there are people who are going to say they only deal with a specific country/continent and don't need anything outside, think a bit longer: none of the employees of your partners, customers, ... will ever go out of the country/continent on business or holiday and get a phone call to do something or try to make a decision on the road?
• Provide reason for listing: Blocklist operators sometimes refuse to give any proof (such as email headers and spam contents) to those managing the networks of listed entities. Something that for certain larger environments (e.g. .edu's) makes it impossible to find the culprit on that network and actually shut it down.
• Security of the blocklist provider: Who can submit data to the blocklist, and how is the data authenticated? The bad guys could poison lists by creating fake data and submitting it in order to block even more addresses. Don't forget Availability is part of security: what happens to your processes if the blocklist were to become unavailable or just slow?
• One practice I found to be impossible to deal with from an business point of view: was a blocklist demanding money to get unlisted. Any self-respecting business will feel this is extortion and will not give in. No matter that they send it to their charity of choice, no matter the small amount it actually is, this remains a show stopper. For you this means you'll find contacts who get listed and have no way of getting out again.
• Do the blocklist administrators actually warn those getting listed? Since many of the evil actions a machine does is more often than not done without the knowledge of the rightful owner, a word to the ISP connecting the machine or the business hosting the machine, can in fact be a big step towards detecting the rootkitted botnet and starting the clean-up.

If your favorite blocklist fails many of these criteria, perhaps it's time to urgently switch blocklists, or move to another solution as to avoid the false positives you might not be aware of.

Sometimes just reading the FAQ of a blocklist will set of so many alarms that you might choose not to use their blocklist at all.

Email: using extra caution

The above goes for any blocklist, email, DNS based, or any other form or method. Still most use of blocklists seems to remain in the anti-spam war.

A few readers are using email blocklists -despite their lack of interest in avoiding false positives-  not as a real blocklist, but as a indication the messages has potentially dubious origins. Such input can then be used together with other signs in the message itself to come to a decision to mark or delete it as spam. The key to success probably lies in finding a good balance in weight given to sources being in a certain blocklist (although it's not really a blocklist anymore then).

When not delivering email to a destination cause you think it is spam, please do not send back non delivery reports. Most spam has spoofed addresses as sender and the spammers rotate the sender victims they spoof, so when it's your turn in the great rotation you get thousands, if not millions of incoming messages containing non delivery, out of office, automatic replies and whatnot. The non delivery reports are makign the spam problem worse, not better. See an older diary on spam backscatter.

Thanks to John and Dr. Neal Krawetz for submitting some more criteria.

(*): I've never seen a false positive on the google toolbar myself, so I'm not criticizing them, just using it as one of the examples where you or your users might have picked up a blocklist without having the intention of doing so.

--
Swa Frantzen -- NET2S