Most likely, you cant afford BIG DATA

I am in a mood, so buckle up, but we need to talk.

We need to talk about data

We need to talk about data

We need to talk about data.

I am putting out a warning:  the next time a candidate walks into or calls the Ozean office asking for “big data”, a web, or a social media presence ‘like President Obama’s’, they are going to get slapped.  HARD!

Unless such candidate has a budget to match President Obama’s, they should take a look at THIS Obama AWS diagram of their computer network infrastructure  (HINT:  The graphic has a zoom!). then read this Obama case study.

If all of this is Greek to you, then you can’t possibly imagine the cost, time, and talent it takes to assemble just the network infrastructure – let alone the people to run the damn thing.

And the network infrastructure was the easier part!

So just STOP it with the big data!

Dirty secret About Big Data

Here is the dirty little secret, if you are running for state house, state senate, congress or even US Senate in some states, you can’t afford big data.

Before you rip my head off, please read this: Most data isn’t “big,” and businesses are wasting money pretending it is.

Let’s be honest – most likely, your campaign can’t afford true big data (yet) unless you are operating at a presidential or party level.  If we continue to be honest, in most gerrymandered districts, the successful use of big data would be negligible.

Big Data

Here is another dirty secret, Big Data is used to shave the margins.

In a presidential campaign or a large campaign where the winning margins are slim AND THE BUDGET ALLOWS, big data can be extremely useful.


If you talk to anyone seriously involved in predictive analysis they will tell you that their work is on the margins.

So think about it, if you are in a district that averages 60% of the vote (like most districts) for one party is it wise to pay for ‘big data’ to push it to 63%?

Next, if you are in a district that is competitive (within 10%), you have to ask yourself: “Are you willing or able to pay for BIG DATA?”

I have been involved in enough of these campaigns – even at the Congressional level – to know that candidates complain about paying for one more mail piece or 100 GRPs on TV,  let alone BIG DATA.

BIG DATA & Fieldwork

But, here is the REAL unspoken part – Big Data doesn’t work without robust field work.

I am a numbers GEEK. I love them. I love data. I love multivariate regression, etc. I love SPSS.  However, the real benefits of big data are NOT just a one time purchase of data.

Big Data or “Obama data” – ONLY works when you have robust field operations feeding real time data BACK into a system creating a closed feedback loop.

Big Data is NOT a one time purchase of credit card company data and calling it “big data”

The true genius of the Obama campaign was:

  • having initial data —>
  • purchasing data to augment the initial data set —->
  • having real time field intelligence —>
  • creating hypothesis —>
  • creating predictive models —->
  • having real time field intelligence feed into data set—->
  • running experiments against predictive models —->
  • comparing results to hypothesis —>
  • refining predictive models —->
  • repeat over and over again until its over.

The Obama campaign had on staff PhD level data analysts, a robust field operation, a unmatched Internet infrastructure, and the money to finance it all.

You most likely don’t.

The Saving Grace

There is a saving grace in all of this. There are only two real numbers your campaign should care about.

1) What is the likelihood the voter will vote?
2) What is the likelihood the voter will vote for you or your campaign?

Those two numbers for 95% of the campaigns in American are fairly easy to compute for any political consultant worth their weight in salt.  They are not easy to perfect, but they are easy to compute. (if they cant understand that sentence, please contact Ozeans’ Political Consultants)


The Value of Experimentation

The most interesting take away for me  in this BIG DATA debate is the value of experimentation in campaigns.

If you look at big data in campaigns- big data is using large data sets to test hypothesis in order to create and refine predictive models.

However, here is the kicker – ONLY by testing hypothesis, can a campaign refine their models.

The issue is that testing means FAILURE and FAILURE means a ‘waste’ of resources, right?

We must embrace experimentation

We must embrace experimentation

BS – You should value the political consultant that says, “I have read the most recent literature, but you know what?  I don’t know. Let’s test that hypothesis by running an experiment.”

The problem is, you don’t read the press articles on the experiments that the Obama or Romney campaign ran that failed (okay, there was orca), and I will bet you my entire net worth there were many, many, many failed disasters.  But the campaign learned from these failures.

The Big Data Truth

In most all cases, your campaign can’t possibly afford true BIG DATA.

If your campaign budget won’t allow you to fund small scale experiments, your campaign can not develop nor afford BIG DATA.


What your campaign should strive for is building models that are good, embracing experimentation, and developing a robust field operation that refines your data.

Commit to that and your campaign will be better off.

About Alex Patton