Bad Behavior and Today’s Reputation Analysis

Subscription Options

The first spam message was sent via the Arpanet—the precursor to the Internet—30 years ago last month. At that time, everyone knew the sender, Gary Thuerk, however, that cannot be said today. In an effort to track perpetrators known to abuse the Internet, blacklists are used to publish addresses linked to spam. In the words of Stephen Pao, VP of product management for Barracuda Networks, “blacklists are the bread and butter of email security.” Kept by independents, like Spamhaus or Mail Abuse Prevention System (MAPS), as well as a host of others, Internet Service Providers (ISPs) subscribe to such lists, in addition to developing their own. They are controversial because of unclear listing criteria, plus delisting can be difficult. “I think they are less than ideal,” says Arvel Hathcock, founder and CEO of Alt-N Technologies. “I’m speaking of IP-based DNS blacklists. When run responsibly, these services are useful. When run irresponsibly, they are a complete nightmare. However, on the whole as things stand today, we are better off with them than without.”

Having reached a certain level of maturity, blacklists are well understood by those trying to dodge the list. “One of the fascinating things happening now is that even the guys who are renting their botnets are protecting their assets,” observes Pao. “Even though they have so many bots available, you will see fewer than 50 actual bots contributing to any single campaign. Our hypothesis is that they see no reason to expose the entire botnet, if it is not required to proliferate the spam.” Instead Pao notes the spammer saves those bots for later use, in an effort to stay off anyone’s blacklist. “Blacklists still work a good percentage of the time,” thinks Pao. “Today over 70 percent of the email blocked by Barracuda Spam Firewall globally is blocked at the IP reputation layer. So it is still significant, it is just not where the innovation is happening.”

Beyond Blacklisting

While blacklists do help to establish bad behavior and assign reputation status, few would argue they are perfect. “Existing IP-based reputation schemes are based upon a less than ideal characteristic—the sender’s IP address,” explains Hathcock. “This is not ideal because IP’s can (and often do) change. When this happens, all existing reputation data, whether good or bad, is lost. Also, IP use fragments the picture. By this I mean that a single sending identity can send email using many different IPs forcing an IP-based reputation to have a fragmented overall picture of the sender.” Hathcock believes that a better approach is to track reputation based on the domain name of the sender. “This solves the problems I just mentioned. The barrier to this has been the inability to authenticate an identity like a domain name. Now we’ve solved that problem with DomainKeys Identified Mail (DKIM). DKIM is the foundation that makes domain-based reputation services possible.”

Verification by either DKIM, Sender Policy Framework (SPF) or Sender ID Framework (SIDF) when coupled with reputation data, allows ISPs and receiving networks to make enhanced decisions on whether or not to deliver email into the inbox, junk or bulk mail folder, or to quarantine and/or block the email altogether. According to Richi Jennings of Ferris Research, “SPF isn’t exactly a ‘reputation mechanism,’ although it can be used to help identify the sender, in order to make improved reputation-based decisions. SPF, DKIM, and other “sender authentication” schemes help a receiving MTA decide if it knows which domain sent a message. For example, SPF can tell if the sending IP address 1.2.3.4 is authorized to send mail claiming to be from example.com and DKIM can tell if the incoming message was signed by example.com’s private key. If the receiving MTA knows the sending domain, it doesn’t need to rely on the reputation of the sending IP address, which can be a blunt instrument,” acknowledges Jennings. He notes that sender authentication allows domains themselves to have reputations. “It’s especially useful for whitelisting known-good domains, so that mail from them doesn’t fall victim to the false-positive problem.”

Pao offers a comparison to credit card theft. “Reputation relies on identity,” he states. “I can prevent a fraud on my credit card, if I am really good about protecting my identity—like shedding all my documents, and not publishing my Social Security number. I am not going to be the victim of fraud as often as if I did not do these things.” Pao goes on to say that DKIM and SPF are all about protecting identity and to prevent spoofing. “That helps me a lot, as a legitimate email sender to do that. However, just like the credit card example, the fact that someone shreds documents and takes other actions, it does not help me, as a merchant from receiving stolen credit cards. It helps me to not receive that particular individual’s stolen credit card, but doesn’t help me not receive someone else’s. It is true that IP-reputation and sender authentication absolutely go hand and hand, but they go hand and hand from the perspective of the person who wants to be protected. Not from the perspective of the person who is trying not to be the victim of fraud.”

Additional Solutions

Companies like Alt-N Technologies and Barracuda Networks are adding layers to aid in establishing senders’ reputation. Continuing his credit card analogy, Pao explains, “If I am the merchant, I expect my security provider to act like VISA or MasterCard and profile the behavior of the cardholder. That is essentially what Barracuda is doing for our customers. We profile the behavior of the bad guys so that no matter whose identity that bad behavior is seen on we can block the emails. It is the same with VISA, who will question someone who buys 50 flat screen TVs in one day. Likewise, we question anyone that shows bad emailing behaviors.”

For Alt-N Technologies, Hathcock reports an additional technique used with DKIM that has been incorporated into its MDaemon Email Server using Vouch By Reference (VBR) to enable email certification in a new way. “Alt-N runs a VBR server at vbr.emailcertification.org. This VBR server lists the domains of some of our own MDaemon customers whom have asked to be listed and whom we have vetted. We also list domains like PayPal, eBay, banks, etc. However, about 90 percent of the domains we are certifying are our own customers. Adding message identification values as input for our certification service enables matches to be made quickly and easily. Consequently, users of the service have the option to skip spam filtering.” Hathcock notes that Alt-N is not trying to get into the certification business, but rather is demonstrating what is possible. “We have been very successful over several months of use,” reports Hathcock. “The criteria to qualify a message for certification is currently a DK or DKIM verified domain. In our forthcoming MDaemon 10 release we are opening it up to SPF and Sender ID verified domains, as well.”

While not alone in adding layers to secure email and score reputations, Pao does see a shift coming. “Our security space is very fragmented, there are lots of vendors offering security. What is beginning to happen is the anti-spam problem is becoming complex enough where you have to have a certain amount of scale in order to do it well,” he believes. “What we will see from an industry perspective is that a set of folks with the expertise, and access to the resources, will become much fewer and farther between.” IP reputation is the third in Barracuda’s 12 layers. Sender authentication is the fourth layer. “The more sophisticated and innovative techniques to block spam are layered on along the process. You cannot ignore doing the easy things before doing the more complex. It is like not locking your door, just because you have an alarm system.”

Moving Forward

The whole measure, counter measure game has made this a very complex problem to solve because each side keeps escalating. Pao notes that spammers are now required to spend greater and greater engineering and technical resources in order to get their mail through. “They have been forced to do stuff to their mail, which reduces their efficacy at marketing promotions—like having to misspell words or images or garbage text in the email. This leads to inferior marketing pieces, and ultimately not as good a conversion rate.” He optimistically hopes that as the war continues to escalate it will simply not make economic sense for a spammer to continue. “At some point, like with any war, it just makes sense for someone to put down their arms. Over time, the level of sophistication required to send out emails will continue to increase, and the response from consumers will continue to decrease, causing people who are currently spamming to find something else to do.” Pao recognizes however, that the reality of the situation is quite different. “This escalation in sophistication is creating the breeding ground for new forms of business to outsource this technical expertise. We are seeing the malware being distributed as Software as a Service (SaaS). It used to be it was the penny stock guys, or the grey market Viagra guys that were putting the most money into it. Now, we are seeing the more sophisticated techniques being used by the mainstream, like the fake luxury product guys, or fake education degrees, because you can now rent the botnets—that expertise—on a SaaS bases. We will see more and more spammers using the techniques that used to be reserved for the more sophisticated spammers.”

Will there ever be a time when we can truly connect identify and reputation, consistently and accurately? Experts think so, including Hathcock. “Once the deployment of techniques like DKIM are widely adopted and a shift is made from IP-based to domain-based reputation services, the email world will be a much better place.”

Online Business Practices to Avoid

If you are a business that sends out email campaigns your reputation should be of vital importance to you. The recent release of a 2008 study by Habeas, Inc., of consumer attitudes towards email and online interaction with businesses identified the following practices to avoid.

Daily email messages ranked with pop-up advertisements as the most damaging online tactics to a company's online reputation.
As many as one in four respondents lose some degree of faith in an organization that is unable to deliver email reliably.
On average, about 80 percent of respondents are not comfortable with businesses sharing their email address.
Internet users believe that about two thirds of companies are likely to share their email addresses with third parties.
More than 80 percent feel that a business’ reputation is negatively affected if it shares customer email addresses with third parties.

While these are consumer thoughts, there are lessons to be learned for any business-to-business organization trying to avoid a negative reputation.