Recently I’ve been seeing lots of polls showing 50% or 64% or 49% of a group supports or opposes something. In particular i’ve been seeing it applied to voters, that is Democrats, Independents and/or Republicans. What I rarely see is the number behind the number. Like how many of the “voters” are ‘D’ or ‘I’ or ‘R’? This can and is important.

Why do I say this? Let me show you. Lets say that 25% of all voters are ‘D’, 25% are ‘R’ and 50% are ‘I’. Lets also say that the total number of voters is 100. (Not the case as the actual number is in the 10’s of millions but 100 is an easy number to work with.). Given 100 voters we have 25 ‘D’ & ‘R’ and 50% ‘I’. Next lets say the issue I question only needs 50% + 1 voters to pass. If all the voters cast a ballot that means 51 voters need to vote yes. Got it? Good.

Now lets look at a poll that says 45% of the ‘D’s support issue ‘X’. 55% of ‘R’s oppose ‘X’ and 50% of ‘I’s support X’. That means (25*.45)+(25*.45)+(50*.50) will vote “Yes). Given that 25*.45 = 11.25 twice plus 25 giving us giving us just 47.5 votes “yes”. So ‘X’ doesn’t pass. But now lets change the number just a bit with 55% of the ‘D’s voting yes, 55% of the ‘R’s voting no and the ‘I’ splitting again 50/50 now the numbers are (25*.55)+(25 * .45)+50*.5) giving a total vote of 50 yes and 50 no giving us a tie.

Now lets get into the deep grass. Lets change the number of ‘I’ to 30, the ‘D’d to 45 and leave the ‘R’s at 25. How do the numbers change? In the first case ‘X’s still looses with only 46.5 yes votes but in the second case ‘X’ wins with 51 votes. This demonstrates quite clearly why knowing actual number is critical understanding just what the poll numbers mean. Besides the actual numbers we need to know just how the numbers are being counted.

By how they are being counted I mean are we talking registered voters, likely voters, or just asking the person being polled what party affiliation the have. Just a casual use of Google show just how wide a swing in numbers we have being reported. It is no wonder that anyone watching politics has full time teams analyzing the polling data and just what it is based on. So what are you to do?

My recommendation is to remember what I’ve said here and do the following. First and foremost check the predictions against actual election results. Not only did they call the election correctly for who won and lost but did the numbers match up with the prediction. Then, if the results are off, especially off by big numbers (I go by 5 percentage points or more but you are free to go by any numbers you want), do they publish any studies/analysis/etc on why they were off. What conclusions did they draw and do you agree/disagree with the conclusions. Once you find a place that you believe does a good job, stick with them. But remember “Trust but Verify” is still good advice.

An before you ask I do have a site on the Web I trust, it is 538 both it’s website and it’s podcasts. But just take my word for it, do your own work.