Last year our firm was fortunate enough to have a client that liked and encouraged our desire to experiment and test.

Situation

We were attempting to test the validity and cost effectiveness of obtaining petitions via direct mail.  To do so, we created a self contained mail piece with perforated petitions.  To further reduce friction, we paid for return postage and used variable printing to pre-fill the petitions with all needed information.  Literally “all” the voter needed to do was sign, date, tear off, and drop in the mail.

Our test universe were households with voters with a history of voting in primaries. It was a large enough test universe to allow for testing multiple conditions.

We were brainstorming different creative elements (social proof, colors, calls to action) and then Ben asked “Why are we limiting ourselves to only direct mail?”

A great question.   We approached the client and said since we are testing direct mail, let’s see if we can push it.   The client agreed with the push and added some additional funds to the project.

We separated three test groups.

• Group 1 – Control – received direct mail only.
• Group 2 – Direct mail with a layered a digital campaign over the top.
• Group 3 – Direct mail, layered digital campaign, and received a personalized text message.

Most direct mail response rates are 1-2%.  We hypothesized that the control group’s response rate would be 1% and Group 2 & Group 3 would be lifted to 2.5%.

Once again, we were dead wrong.

We had a mail date of mid-month.

Digital Media

10 days before the mail went out, groups 2 & 3 began seeing a targeted digital campaign. The creative essentially said “Look for your petition in the mail” and it clicked through to a landing page specifically about the petition.

Both display and facebook were used. The digital audience mirrored the mail groups exactly using proprietary methods.

The digital campaign continued until 10 days after the mail drop. 20 days in total.

Text campaign

The day of the mail drop, Group 3 began to receive personalized text messages:

{first_name}, this is Al from the xx campaign.In the coming days,you will receive a petition in the mail.Please read,sign & return.”

The text was repeated 2 days later, minus the opt outs of course.

Waiting

And then we waited. 1 day…2 days….7 days….nothing.

The panic began to set in. Did we just waste a a ton of money?

No, we did not.

Then came the glorious day of first returns. The PO Box was stuffed AND it contained a note from the post master “please see us at the counter.” At that time, we were handed a basket of returns. We were excited, and couldn’t wait to run the stats. But how long could we wait for returns before computing the stats? Within 10 business days of returns beginning, we received 78% of what were in the end going to receive. However, we received returns for 45 days. Yes, they slowed to a trickle, but I know from the date of signature, the petitions were being signed up to 40 days after the mail drop. We were extremely pleased with the returns, but couldn’t wait to explore the data to explore was there a real difference between the groups? When we finally went two consecutive business days, we allowed ourselves permission to compute the “final” stats:

Cost Effectiveness

Yes, we did spend more on Groups 2 & 3. So was the increased response rates more cost effective? Yes! Group 2 (digital media only), even with an increase in spending was a 1% reduction in cost per returned petition when compared to the control group. Group 3 (digital media and texting), Even with an increase in spend, it was a 3.6% reduction in cost per returned petition when compared to the control group.

Conclusion

The secret sauce of this entire experiment was limiting the waste and precisely targeting the digital. As we briefly mentioned, it was limiting the digital audiences to as close of an exact overlay as possible. We were targeting super-voters in a specific geographical area (not everyone with an interest in politics). It was this precision that drove response rates AND made it cost effective. How did we do that exactly? Call us about your next project……