When you hear the results of another public opinion poll (an overdose of them during any election cycle), the media forgets to caution audiences that those numbers should be taken with a grain of salt. Polls don’t necessarily reflect broadly-held views unless they are based on a large enough sample.
This is not to question the credentials or analysis of the researchers. They’re just doing their job according to the limits of time and budget. But the reporting of results in the media can give the impression that everyone in the country fits into those rankings. Looking at a couple samples, you might be surprised by how small a group is actually involved.
Quick, Not Quantitative
There are two categories of research. Qualitative is easy, fast and cheaper because it surveys a small number of people for their opinion. Quantitative research means large numbers of opinions that produce an equally large amount of data, more accurately representing the broad population. Obviously, that requires a lot more time and money.
Sample News Polls
Consider a poll sponsored by NBC News and The Wall Street Journal after the 2010 mid-term elections. The headline on the MSNBC Web site reads: “Poll: Nation Wants Change, Skeptical Washington Can Deliver,” followed by a subhead that reads: “76 percent believe the country is headed for division, little compromise.” Well, guess what. The poll reflected opinions from one thousand interviews of males and females in a spectrum of ages and political preferences. The margin of error, that you will find in the actual results but not in the text of the story, was plus or minus 3.10%. So the 76 percent came from a tiny sampling and was not 76 percent of everyone in the voting population.
Another poll conducted in 2010 by Langer Research Associates for ABC News and the Washington Post involved opinions on the airport screening controversy. The ABC story headline reads: “Enhanced Scanners Win 2-1 Support, But Half Say Hands Off to Pat-Downs.” Once again, the poll was conducted by telephone, sampling only 514 adults. Margin of error this time was 5 points. The report refers to sixty-four percent supporting the use of full body scanners but again, it’s 64 percent of 514 people!
Carnegie Mellon University did a computer analysis of a billion tweets from 2008 and 2009. Researchers found that results came close to those from opinions polled using established methods, particularly related to consumer confidence and presidential job approval. That finding has led to talk of the possibility that Twitter could serve one day as a cheap and fast way of learning about the public’s opinion.
So in the future, you may not need to wait for a call. Just log on to any social media and say what’s on your mind! A pollster may be taking note.