What is the Measure of Success?
by David Daniels
Imagine upon balancing your checkbook you determined you had thirty percent less money in the bank than you thought. Upon further investigation, you learn that each place that took your debit card had a different methodology for debiting your account. To your shock, you found some merchants defined a debit as subtraction plus the number of purchasing attempts multiplied by the average number of credit card transactions. Confusing? Thankfully for us, this fictional account is not a real issue, as our banking system established standard methodologies long ago. However the lack of consistent measurement is prevalent within the email marketing industry, which obscures the real efficacy of the channel and renders cross industry benchmarks useless.
In 2004, JupiterResearch identified that the methodology for determining key metrics (e.g., delivery, click) were not consistent across Email Service Providers (ESPs). (See E-mail Marketing Buyer's Guide: JupiterResearch E-mail Service Provider Evaluation, August 20, 2004.) This metric manipulation is particularly driven by the manner in which bounces and gross mailing numbers are dealt with, which ultimately impacts every metric thereafter. While the industry has made good progress in the last three years creating standards for email authentication, recent research indicates that little progress has been made in standardizing email marketing measurement. Without standard email marketing measures, the confusion surrounding success metrics will not abate.
E-mail Measurement Accuracy Coalition Announced
In April 2007, JupiterResearch along with twenty charter supporters including the Email Service Provider Coalition (ESPC), Email Experience Council (eec) and many leading email service providers, launched the E-mail Measurement Accuracy Coalition (EMAC). With an international footprint, EMAC aims to create a consistent framework for measuring email marketing performance. This will entail producing standard definitions for common metrics such as delivered, clicked, opened and other engagement oriented metrics. The success of the EMAC will be based on broad participation and collaboration. Solving the measurement issue transcends creating consistent definitions; it involves implementation issues for many vendors.
The stakes are real and it is time for the industry to act and come to agreement on what success means. Everyone's voice can be heard in the process to standardize metrics. It is easy to get involved. Sign-up at www.emacoalition.org for a free membership to the EMAC at and provide your comments directly via email at feedback@emacoalition.org. The EMAC will provide regular updates on its progress to its members via email and its website.
For the email marketing channel to continue to flourish, we must first have common consensus by defining how the industry measures success. Staying engaged in this dialog will help all of us more accurately understand how efficient and effective the email channel truly is. DD/TMP