Poll results from Rasmussen Reports consistently overstated Republican performance in the midterm elections, according to The New York Times' Nate Silver. Rasmussen polls and analysis have been criticized by polling experts who note that they often harmonize with narratives favored by conservatives and Republicans. Indeed, the polling company is a favorite of Fox News, which has cited Rasmussen polls at least 94 times in the past three months.
Fox News cited Rasmussen 94 times in months leading up to election
Fox was a Rasmussen client in 2008. In 2008, Fox News hired Rasmussen to conduct general election polls in 2008.
Fox News is a client of Rasmussen's Pulse Opinion Research. Throughout September and October, Fox News released a series of surveys “conducted by Pulse Opinion Research,” which “licenses methodology developed by veteran pollster Scott Rasmussen, providing a survey platform for a host of clients, from individuals to special-interest groups" and “provide[s] the field work for all Rasmussen Reports surveys,” according to its website. The firm allows anyone to commission a survey using the same “automated computer phone-calling system” used by Rasmussen Reports.
Fox News cited Rasmussen polls at least 94 times since August 3. According to a search of the Nexis database, which excludes most of Fox News' daytime programming, Fox News shows available in Nexis have cited Rasmussen polls 94 times in the past three months. In addition, president and CEO Scott Rasmussen has been interviewed live on Fox and has appeared in several pre-taped reports during that period. On November 3, he was quoted in a report on Fox News' Special Report stating, “Quite frankly, if the Republican House doesn't repeal the [health care] law early in the tenure, they will disappoint an awful lot of voters.” Rasmussen also appeared in a report aired during the September 7 edition of Fox News' Special Report, stating of the health care law, “Every week since it became law, every single week a majority of voters have said yes, we want it repealed.”
Rasmussen has appeared at least 86 times on Fox News dating back to 2008. A search for “Fox News” on the Rasmussen Reports website returns 95 matches, 86 of which are Fox News appearances by Scott Rasmussen. The earliest appearance listed is August 29, 2008. Rasmussen was occassionally identified as a Fox News contributor in 2008.
Fox News defended Rasmussen. In a January 2 article, Politico reported that many Democrats believe “that Rasmussen Reports polls are, at best, the result of a flawed polling model and, at worst, designed to undermine Democratic politicians and the party's national agenda.” On the January 4 edition of Special Report, host Bret Baier said that “Democrats and liberal bloggers” were making a “target” of “independent pollster Scott Rasmussen.” After reporting on the criticism, Baier provided examples of accurate polls the polling company released.
Rasmussen polls “exhibited a considerable bias toward Republican candidates” in 2010
Silver: Rasmussen polls “badly missed the margin in many states, and also exhibited a considerable bias toward Republican candidates.” Nate Silver, a polling analyst for The New York Times wrote on November 4 that Rasmussen Reports polls “badly missed the margin in many states, and also exhibited a considerable bias toward Republican candidates.” Silver stated that “the methodological shortcuts that the firm takes may now be causing it to pay a price in terms of the reliability of its polling.” From his analysis:
The 105 polls released in Senate and gubernatorial races by Rasmussen Reports and its subsidiary, Pulse Opinion Research, missed the final margin between the candidates by 5.8 points, a considerably higher figure than that achieved by most other pollsters. Some 13 of its polls missed by 10 or more points, including one in the Hawaii Senate race that missed the final margin between the candidates by 40 points, the largest error ever recorded in a general election in FiveThirtyEight's database, which includes all polls conducted since 1998.
Moreover, Rasmussen's polls were quite biased, overestimating the standing of the Republican candidate by almost 4 points on average. In just 12 cases, Rasmussen's polls overestimated the margin for the Democrat by 3 or more points. But it did so for the Republican candidate in 55 cases -- that is, in more than half of the polls that it issued.
If one focused solely on the final poll issued by Rasmussen Reports or Pulse Opinion Research in each state -- rather than including all polls within the three-week interval -- it would not have made much difference. Their average error would be 5.7 points rather than 5.8, and their average bias 3.8 points rather than 3.9.
Nor did it make much difference whether the polls were branded as Rasmussen Reports surveys, or instead, were commissioned for Fox News by its subsidiary Pulse Opinion Research. (Both sets of surveys used an essentially identical methodology.) Polls branded as Rasmussen Reports missed by an average of 5.9 points and had a 3.9 point bias. The polls it commissioned on behalf of Fox News had a 5.1 point error, and a 3.6 point bias.
Rasmussen poll missed mark on Hawaii Senate race by 40 points. An October 17 Rasmussen survey found that Sen. Daniel Inouye (D-HI) had a 13 point leade over his Republican opponent Cam Cavasso. Inouye actually won by 53 points. Silver stated that Rasmussen's error was the largest “ever recorded in a general election in FiveThirtyEight's database, which includes all polls conducted since 1998.” Salon reported on November 3 that Scott Rasmussen “did not specifically remember the Hawaii poll showing Inouye ahead by just 13 points. 'I'll have to go back and look at that one,' Rasmussen said.”
Rasmussen poll of New York gubernatorial race ignored Lazio. On September 20, Silver highlighted a poll by Rasmussen Reports showing Republican New York gubernatorial candidate Carl Paladino gaining on Democratic frontrunner Andrew Cuomo (Cuomo won the race). However, as Silver pointed out, the poll completely ignored Rick Lazio, the Conservative Party candidate at the time, who could have potentially lured away conservatives who favored Paladino. (Lazio subsequently dropped out of the race.) From the post:
[W]ithout geting into a larger debate about whether it is proper for pollsters to also serve as prognosticators, Mr. Rasmussen's prediction of a tightening race is likely to have seemed less prescient had he also included Mr. Lazio in his poll. Since Mr. Lazio will generally draw from the same pool of conservative-leaning voters that Mr. Paladino does, a poll with him included might have read something like -- and this is just a rough guess -- Mr. Cuomo 52 percent, Mr. Paladino 26 percent and Mr. Lazio 14 percent. Such a result would not have provided much of an indication of momentum for Mr. Paladino.
At the very least, Rasmussen Reports could have tested the contest both as a head-to-head matchup between Mr. Cuomo and Mr. Paladino -- and also as a three-way matchup with Mr. Lazio included. Pollsters are sometimes too reluctant to run multiple versions of their poll, like reporting results both among registered voters and those most likely to vote.
Mr. Rasmussen is sometimes accused of wanting to “push a narrative” at the expense of what should be a pollster's goal, which is to reflect public opinion in the electorate he is testing as fairly and accurately as possible. Conducting this poll without including Mr. Lazio might not quiet his critics.
Polling experts have previously criticized Rasmussen
Blumenthal: Rasmussen “manages to violate nearly everything I was taught what a good survey should do.” A June 17 Washington Post article that described Rasmussen as “a driving force in American politics” also noted that there are those “in the political polling orthodoxy who liken Scott Rasmussen to a conjurer of Republican-friendly numbers.” The Post further reported:
“The firm manages to violate nearly everything I was taught what a good survey should do,” said Mark Blumenthal, a pollster at the National Journal and a founder of Pollster.com. He put Rasmussen in the category of pollsters whose aim, first and foremost, is “to get their results talked about on cable news.”
Nate Silver, who runs the polling analysis site FiveThirtyEight, soon to be hosted on the Web site of the New York Times, faults Rasmussen for polling only likely voters, which reduces the pool to “political junkies.”
“It paints a picture of an electorate that is potentially madder than it really is,” agreed Scott Keeter, director of survey research at Pew Research Center and vice president of the American Association for Public Opinion Research (AAPOR). “And potentially more conservative than it really is.”
Silver: Rasmussen's question wording is “a concern” and choice of subject matter “tends to dovetail with conservative media narratives.” In a January 3 post, Silver wrote:
I'm not saying that Rasmussen's question wording is always biased. It isn't. And I'm sure you could find a couple of cases where the wording tend to portray the liberal argument more favorably. But cases like these happen consistently enough with Rasmussen that I'd say it's a concern. And when they do use unorthodox question wording, nine times out of ten it favors the conservative argument. I would describe this as a form of bias -- although it should generally implicate only the poll in question, and not their overall enterprise. In other words, if Rasmussen uses some misleading wording in a health care poll, that means I'm not likely to take that health care poll very seriously -- but it doesn't particularly mean that you should throw out their presidential approval polling, or their Arkansas polling, or their polling on gay marriage, or whatever. Yes, this does mean a bit of extra work -- we have to scrutinize each particular poll for potentially misleading wording -- but that's something that we should be doing more of anyway.
I also have some questions about Rasmussen's choice of subject matter. In particular, they have a knack for issuing polls at times which tend to dovetail with conservative media narratives. Rasmussen, for instance, recently decided to issue a poll about Ben Nelson's standing in Nebraska in light of his vote for health care, which is unpopular in the state. But did they issue a similar poll for Joe Lieberman, who until recently looked like he might vote against the health care bill -- and who opposed the public option, a policy which is very popular in Connecticut? No. They did poll Connecticut in December, but they asked only about Chris Dodd, and not Lieberman. Certainly, there's nothing wrong with polling on Ben Nelson -- or with not polling on Joe Lieberman, who gets polled frequently by home-state pollster Quinnipiac. But if you see this sort of pattern consistently, then it may reflect a certain kind of bias.
Pollster.com: Rasmussen polls have significant pro-Republican house effect. According to Silver, “house effects” are “systematic differences in the way that a particular pollster's surveys tend to lean toward one or the other party's candidates.” In a July 13 post, Harry Enten wrote on Pollster.com that “Rasmussen had higher pro-Republican house effects during important news cycles in 2008.” Enten had previously noted:
My friend David Shor, who I am currently working on projects with, documented that Rasmussen had a “summer” reversal of its Republican house effect in 2008. Looking at all polling from presidential, senatorial, gubernatorial, and house races, David found “Rasmussen polls have a statistically significant Pro-Republican house-effect that appears during primary season in the beginning of the year, disappears during the summer, and then very rapidly appears right before the Republican National Convention” .
Enten further wrote that Rasmussen's generic ballot polls in 2010 had a significantly pro-Republican house effect when compared to other surveys. According to Silver, Rasmussen's use of a likely voter model does not explain most of Rasmussen's pro-Republican house effect. He added, “The bottom line is this: the sample included in Rasmussen's polling is increasingly out of balance with that observed by almost all other pollsters. This appears to create a substantial house effect, irrespective of whether Rasmussen subsequently applies a likely voter screen.”
Rasmussen has history of misleading issue questions and analysis
Rasmussen repeatedly polled a question on terrorism that was based on false premise. In November 2009, Rasmussen Reports released a poll in which it asked: “Should the [Fort Hood] shooting incident be investigated by military authorities as a terrorist act or by civilian authorities as a criminal act?” However, terrorism is a violation of criminal law and the Justice Department investigates terrorist acts. Numerous terrorists have been brought to justice through the federal justice system. Nevertheless, Rasmussen included the same question in another poll released a month later, and Scott Rasmussen himself highlighted the findings on the January 4 edition of Fox News' The O'Reilly Factor.
Rasmussen baselessly trumpeted claim that Americans are skeptical of science behind climate change. A December 3, 2009, Rasmussen press release declared that Americans doubt the science behind climate change, but failed to provide any evidence. Instead, it said that 52 percent of Americans “believe that there continues to be significant disagreement within the scientific community over global warming.”
Rasmussen misrepresented proposed government policy on America's sodium intake. An April 26 Rasmussen poll asked: “Should the government set limits on how much salt Americans can eat?” However, as Blumenthal pointed out: “The Institute of Medicine is proposing to limit the amount of salt in processed food. No on is urging the government to restrict the sale of salt or 'set limits on how much salt Americans can eat.' ”
Rasmussen conducted misleading poll on nuclear reductions. On the day President Obama and Russian President Dmitry Medvedev signed the New START treaty, Rasmussen released a poll finding that 53 percent of Americans think the United States should not reduce its nuclear arsenal. But the poll only asked, “Should the United States reduce the number of nuclear weapons in its arsenal?” without mentioning that Russia would be reducing its own arsenal in kind. A related question asked, “How likely is it that other countries will reduce their nuclear weapons arsenal and development in response to the actions taken by the United States?” Fifty-four percent replied that it was either “not very likely” or “not at all likely” that other countries would reduce their arsenal. The poll did not mention the agreements made in the START treaty. Nevertheless, Fox News featured the poll on Special Report, with guest host Chris Wallace saying that “53 percent oppose the reduction of the U.S. nuclear arsenal agreed to in a new treaty with Russia.”
Rasmussen poll criticized for question on spending and taxes. Politico reported in January that Rasmussen was criticized for asking respondents if they agreed or disagreed with the statement: “It's always better to cut taxes than to increase government spending because taxpayers, not bureaucrats, are the best judges of how to spend their money.” Politico reported that critics believe “Rasmussen designs its polling questions to elicit negative responses about Obama and Democrats,” and, as Democratic pollster Mark Mellman put it, “they write their questions in a way that supports a conservative interpretation of the world.” Rasmussen used the same question in a poll released on October 28.
Health care reform question used suspect phrasing. An August 2009 Rasmussen poll asked: “Suppose that Democrats agreed on a health care reform bill that is opposed by all Republicans in Congress. Should the Democrats pass that bill or should they change the bill to win support from a reasonable number of Republicans?” Sargent commented that the poll “includes the amusing caveat that the pollster did not quantify what constitutes 'a reasonable number' of Republican supporters” and was “clearly designed to suggest public opposition to Dems going it alone on health care.”
Sargent: Rasmussen buried “striking” poll result supporting argument for health reform while highlighting anti-reform finding. On March 3, 2009, Washington Post blogger Greg Sargent highlighted criticism that Rasmussen “games its poll questions and presents its results in a way designed to gain maximum pick-up by the conservative media” and stated:
Well, this week brings a new example. Here's the headline on Rasmussen's latest press release touting its poll on health care:
49% Say Obama Should Delay Health Care Reform Until Economy Is Better
And it's true that the Ras poll found this. But if you dig down into the poll, you find that it also found another number that's just as newsworthy, if not more so. From the Ras release:
Seventy-eight percent (78%) of voters acknowledge, however, that reining in health spending is at least somewhat important to improving the nation's economy. That includes 46% who say it is Very Important.
That's a striking, headline-worthy finding. An enormous majority -- nearly 30 points more than the number who said Obama should delay health care reform -- believes the central argument being made for reforming health care right now.
It's odd enough that this is presented as an acknowledgment on the part of voters, as if they were somehow reluctant to believe this. But put that aside for a sec. The larger point is that Ras decided to headline the less interesting and dramatic number in its own polling -- one that clearly would have much more appeal to conservative media and their audiences.
Sargent later highlighted a report by the Weekly Standard promoting the 49 percent number. He wrote:
As I noted, Ras emphasized the number that supposedly suggested reluctance to health care reform, rather than the one buried in their survey finding overwhelming support for the central argument in favor of overhauling the system right now.
Barely 15 minutes later, the Weekly Standard grabs on to Rasmussen's poll and slaps this headline on it:
Rasmussen: Many Believe Health Care Reform Should Be Delayed Until Economy Improves
The Standard reads the poll and concludes that President Obama should “rethink” the argument that the current economic crisis is a “justification to reform health care.”
Rasmussen promoted Sarah Palin's Going Rogue. A November 16, 2009, Rasmussen press release claimed that 20 percent of the voting public “say they are likely to” read Sarah Palin's book, Going Rogue. This statistic was followed with a link on where to buy the book: "(To order the book, Going Rogue, click HERE)."
Rasmussen aligns himself with Republicans and conservatives
Rasmussen reportedly worked for RNC and Bush. As Think Progress has noted, according to the nonpartisan Center for Public Integrity, Rasmussen received $45,500 from President Bush's re-election campaign in 2004 for “survey research,” as well as $95,500 from the Republican National Committee between 2003 and 2004 for “survey cost,” “voter data,” and “survey.”
Rasmussen wrote a column for right-wing website, WorldNetDaily.com. In a February 2009 Washington Independent article, Dave Weigel wrote of Rasmussen: “For a short time around the 2000 elections he wrote a column for WorldNetDaily.com, once arguing that President Bill Clinton had 'ratified the Reagan Revolution' by declaring the end of big government in Clinton's 1996 State of the Union speech.” WorldNetDaily is an extreme right-wing website that routinely pushes conspiracy theories, anti-gay attacks, and birther stories.
Rasmussen wrote book calling for privatization of Social Security. In his article, Weigel further reported:
In other columns, and in a 2001 company-published book titled A Better Deal! Social Security Choice, Rasmussen made the case for privatizing the nation's oldest entitlement program. “In fact, 46 percent of American adults say that relying on the government is riskier than letting workers invest for their own retirement,” wrote Rasmussen in a Jan. 10, 2001 column arguing that incoming President Bush should push for private accounts. “Just 36 percent say letting workers invest is more risky, while 18 percent are not sure.” In the book -- not a huge seller, but promoted by Rasmussen at an August appearance at the libertarian Cato Institute -- the pollster argued that “giving workers more control over their 'contributions' will put the 'Security' back in Social Security.”
Rasmussen will reportedly attend cruise hosted by conservative National Review. Salon reported in July that Rasmussen “will in November speak for no fee on a post-election National Review cruise to raise money for the conservative magazine,” alongside “Karl Rove, Andrew Breitbart, Phyllis Schlafly and Jonah Golderg.” Salon later asked Rasmussen “if he is worried about creating perceptions of bias by attending a cruise made up largely of professional conservatives, he said: 'If Keith Olbermann and Markos had a cruise trip and gave me a free trip, I'd go and speak there the same way.'”