stay connected

FrumForum Facebook FrumForum YouTube Update Twitter FrumForum Flickr

Fudging Poll Numbers 101

November 23rd, 2009 at 7:45 am by Andrew Gelman | 18 Comments |

Mark Blumenthal links to Rush Limbaugh accusing Gallup of “upping the sample [of] black Americans to keep [Obama] up at 50%” in the polls.  (For the context, see the last paragraph of the transcript.)

Frank Newport of Gallup responds here.  Newport denies it all, but he would, wouldn’t he?

Seriously, though, it’s hard to believe that Limbaugh really believes that Gallup is fudging the numbers.  As a big-time radio host, he’s gotta know all about marketing surveys, right?  I’m just assuming he said that “upping the sample” bit as more of a joke or an off-the-wall speculation.  It did raise two interesting questions in my mind, though:

1.  The assumption behind Limbaugh’s argument — as with many arguments about polls — is that the published poll results have an effect of their own, beyond the president’s underlying popularity.  For example, maybe some senator would vote for the healthcare bill if he read that Obama’s approval rating was 51% but would vote no if he read that Obama only had 49% approval.  This might very well be true — it makes sense — I just don’t really know.

2.  What if you were a pollster and really did want to cheat and over-represent Democrats?  How would you do it?  Contra Limbaugh’s suggestion, I don’t think you’d oversample blacks.  Gallup does telephone surveys, and it’s not like there’s a separate telephone directory for blacks.  Also, as several commenters to Newport noted, the percentage of blacks among the survey respondents is easy enough to check.  And, for that matter, many survey organizations (possibly including Gallup) do post-sampling weighting adjustments for race, anyway, in which case oversampling blacks won’t do anything for you at all.

If you’re doing a telephone poll and want to oversample Democrats, you can just call states and area codes where more Democrats live.  Call New York, L.A., Chicago, etc.  You can even call people in Democratic-leaning white areas if you want to mix things up a bit.  That’ll do the trick.  Bury it deep enough in the sampling algorithm and maybe nobody will notice!

Recent Posts by Andrew Gelman



18 responses so far

  • 1 greglane // Nov 23, 2009 at 8:33 am

    Everything’s racial with Rush.

  • 2 cpanza // Nov 23, 2009 at 8:43 am

    Your point is taken, but in checking a pollster’s methodology only an idiot would miss a bias sampling error like calling mostly urban population centers with predominantly left oriented people or moderate right-oriented ones. In the age of the million blogs on either side (especially run by people like Nate Silver, people with statistical know-how) all focused on picking apart everything little thing that anyone does in order to discredit that person’s project, a dishonest pollster will need to find much better things than that to “bury” in the algorithms.

  • 3 teabag // Nov 23, 2009 at 9:05 am

    Limpballs is a fool, who knew!

  • 4 Raider1 // Nov 23, 2009 at 9:10 am

    Does anyone want to ask the obvious? er…if you can call more heavily democratic area codes to weigh the sample more democratic, what prevents you from calling more heavily Black area codes to do the same with race? Honestly, where do you think you are more likely to find a Black household? certain sections of Harlem or East Hampton? Come on now. Stop with the p.c. crap already.

  • 5 Raider1 // Nov 23, 2009 at 9:13 am

    Everything is NOT racial with Limbaugh. But he recognizes race when it is an issue. Such as the only factor that got the most unqualified man in history elected as president.

  • 6 MI-GOPer // Nov 23, 2009 at 9:27 am

    Radier1, I attended a press briefing last Thursday held by the Michigan Democrat Party alleging that the 3-4 state pollsters doing local polling were bigots and racists for under-representing both the black and urban public opinion.

    The proof? Well, the pollsters are finding that Governor Granholm, US Senators KarlMarxLevin and DebbieStupidCow aren’t as loved as they are at Democrat Party HDqtrs or at the Mother House in Detroit. That means the pollsters are skewing the sample… first, they poll registered voters instead of anyone likely to vote or –better yet– any adult irrespective of whether they vote or not.

    The Democrats in Michigan want a larger alternative sampling because they argue that poor black households without phones and urban households without electricity should be included in the tallies by making a mathematical adjustment based on the notion that there’s a theoretical 13-17% homeless population in Michigan and they would vote straight Democrat if they could get to the polls (enter ACORN, stage far, far, far Left). If polling companies don’t want to do the work to reach out to poll those “voters”, then the pollsters –according to the Democrats– need to mathematically adjust any result to reflect the impact of these “Obama voters”… once of the Democrat Party hacks slipped and called the target group that phrase: Obama voters.

    Transparent, eh?

    Democrat Party officials also said that they are interested in joining a NY Cuomo suit to force Arbitron to mathematically adjust their numbers in a similar fashion.

    Heck, we’ve done it for years on the census when Democrats controlled the WH… it seems perfectly partisan to apply the same logic to private polling firms.

    At the press briefing, Democrats went a bit further to imply that polling done by state university organizations may find appropriation bill sanctions against the university if they don’t comply with the Democrat Party’s wishes. Chicago Thuggery comes to Michigan!

  • 7 MI-GOPer // Nov 23, 2009 at 9:30 am

    TeaBagged at #3 spews: “Limpballs is a fool, who knew!”

    Wisdom on topic from the Troll Tribe and Frum Forum Village Idiot. What a great intellect.

  • 8 ottovbvs // Nov 23, 2009 at 9:52 am

    MI-GOPer // Nov 23, 2009 at 9:27 am

    …….the paranoia attacks pretty bad this morning?……haven’t you taken the VA meds that Teabag keeps reminding you about?

  • 9 MI-GOPer // Nov 23, 2009 at 10:32 am

    AutomaticBSer chimes in from the Retirement Lounge at the local union goon shop “the paranoia attacks pretty bad this morning”… I’m thinking you migth want to read some of the other threads, automaticBSer… it seems that lots of folks are calling you and TeaBagged and all the TrollTribe out into the public square as irritants, activists and disrupters.

    There’s no paranoia for me. I’m just reporting what the Michigan Democrat Party is trying to do to help the Obama Messiah… and NY’s AttyGen Cuomo.

    I think you’ve been reading Saul Alinsky’s book a little too late into the night and it shows. “When confronted with the superior argument, community activists must use parody or ridicule to try to destroy their opponent’s better thinking”.

    You nailed the text book definition with #8, automaticBSer. And everyone here now knows what your pathetic, tired game is all about. And still you post? That’s just your lameness informing your actions.

  • 10 balconesfault // Nov 23, 2009 at 10:49 am

    Radier1, I attended a press briefing last Thursday held by the Michigan Democrat Party alleging that the 3-4 state pollsters doing local polling were bigots and racists for under-representing both the black and urban public opinion.

    If this was a press briefing … do you have any links to media reporting of the subject of the meeting?

  • 11 Raider1 // Nov 23, 2009 at 11:34 am

    Alleging…there ya go. It was not news because the DNC sees racism at the root of everything. I did notice to the right here though that Jesse Jackson very much keeps fanning the flames of racial strife (so he can keep his very existence justified and the extortion fees rolling in to pay off his mistresses) by saying that if you are anti-healthcare legislation and are Black you are a traitor to your race.

    Democratic Myth of Myths: They want a “post-racial” society. Nothing could be further from the truth. The DNC keeps its ill-served Black constiuency in line by feeding them the line that we are still a very racist country and only they look out for Blacks…by feeding them the crack of government dependency. Jesse Jackson, Al Shorpton, et al. what on earth would they do in a true color-blind society–besides fade to oblivion and face taxe evasion charges?

    Trust me, no one wants racial strife in the country more than that very party who claim to be oh-so-color bind…the election of the wholley unqualified, un-vetted, far left-radical with ZERO executive experience Barack Hussein Obama being case in point.

  • 12 Raider1 // Nov 23, 2009 at 11:36 am

    MI-GOPer…be careful. YOu don’t know which Otto you are talking to. I have identified three, possibly four, parallel life stories he has manifested here. But there may be the world-feared internation assassin and man of intrigue about to appear. Tread lightly my friend.

  • 13 Arch // Nov 23, 2009 at 3:04 pm

    Huh. I just went to fivethirtyeight.com to see if they had a piece up about this, and found this same piece. Mr. Gelman published it there too.

  • 14 tdawg11870 // Nov 23, 2009 at 4:18 pm

    MI-GOPer, everyone else: Can we call people by their real names? Does it really grate you so much to use names like “Stabenow” and “Limbaugh”? If you want to get your digs in, just use the rest of the sentence. You sound like children and undermine the rest of your argument.

  • 15 MI-GOPer // Nov 23, 2009 at 4:42 pm

    why tdawg? we listened for years to far Left democrat activists call Bush McChimpey and Cheney Darth Vader and Rove all sorts of slanderous gay-bigoted terms… and during the campaign it was Rudy McRomney and The Pig with Lipstick… hmmmm.

    Now the shoe is on the other foot and you suddenly get squimish?

    It doesn’t sound childish –it’s accurate. BessieStupidCow is nearly the size of a full Heffer… and KarlMarxLevin is certainly to the left of Karl Marx on lots of issues… a guy who had some strong devotees inside the Democrat Party, no?

    tdawg, it would be more compelling if I’d watched you mouth similar concerns when TeaBagged or AutomaticBSer were in high gear on this site.

    You didn’t, of course. Then, when it was your team doing it, it was fun and good sport. When the arrow hits too close to home, then you start crying? Spare us.

  • 16 balconesfault // Nov 23, 2009 at 5:32 pm

    and Rove all sorts of slanderous gay-bigoted terms

    gay bigoted terms?

    Like what … turd blossom?

    By the way, MI-GOPer … you do know that Karl Rove is an atheist, right?

  • 17 MI-GOPer // Nov 23, 2009 at 5:47 pm

    BlankHead, I remember posters on the Mall in DC with “FudgePacker” next to Rove plugging Cheney or Bush from behind… I remember posters on the Mall in DC with “Bug Chasers” next to a picture of the Bush Cabinet and Rove bent over Bush and pointing at something in a briefing packet… both those posters were in a line of posters for the Easter Egg Roll and carried by gayLeft democrat activists. I remember pictures of Bush helping the ailing Saudi King down some steps and the SEIU activists scrawled “Hey McChimpy, does Karl know you have a new boyfriend with benefits?”.

    The people you call political allies can be a pretty scummy lot, BlankHead. But then, that doesn’t matter because you’ve got them here with TeaBagged and AutomaticBSer and OldSkwerl. You’re use to slumming it with political pals.

    I think you need to stop asking rhetorical questions, BlankHead –you’re earning your title. BlankHead.

  • 18 About support, rasmussen obama poll, health care, obama gallup poll, obama approval, president obama | Find me About // Nov 24, 2009 at 1:01 pm

    [...] Mark Blumenthal links to Rush Limbaugh accusing Gallup of “upping the sample [of] black Americans to keep [Obama] up at 50%” in the polls. (For the context, see the last paragraph of the transcript.) Frank Newport of Gallup responds here. …Read Original Story: Fudging Poll Numbers 101 – FrumForum [...]

You must log in to post a comment.