WRAL Poll – Not Insightful, Widely Misinterpreted | Eastern North Carolina Now

Publisher's note: This post, by Leah Byers, was originally published in Civitas's online edition.

    A recent poll commissioned by Raleigh news station WRAL has some serious flaws - but that's not stopping the media and left-leaning groups from promoting the results as gospel truth.

    There are two major failings with the survey as it's been presented to the public - the validity of the questions and the interpretation of the results.

    Let's start with the former. Poll validity is defined as its ability to measure what it claims to measure. The survey asks four questions related to policy in North Carolina. Each has some fatal flaw that likely influenced the way that people respond to the question, thus invalidating the question's ability to measure what it intended to measure. This is known as "measurement error," and I discuss it in more detail in this polling overview here. Responsible poll creators try to mitigate this type of error through clear, neutral wording and follow-up questions in order to get the most valid poll results possible.

    I would make the case that each question in the WRAL poll contains measurement error. The measurement error of each question is highlighted below, and you can read the full poll results HERE for yourself.

    Question 1 - Medicaid is a government program that provides health care to low-income families. North Carolina has the option to expand Medicaid to include additional low-income adults not currently covered, with the federal government paying 90 percent of the cost. Or, North Carolina may instead leave Medicaid covering only those people it covers today. What should North Carolina do?

• The wording of this question is fine - in fact, Civitas Polls from November 2018 and February 2019 show similar initial support for Medicaid expansion. The problem in this question is that the general public does not tend to know a lot about the pros and cons of this policy proposal. When presented with the negative effects of expansion on healthcare access and the state budget, around half of respondents in the Civitas Polls say they'd be less likely to support expansion. Asking only one initial question, with no follow up, provides no valuable assessment of public sentiment.

    Question 2 - If North Carolina legislators had to choose between giving more money to schools or cutting taxes on businesses to boost the economy, what should NC lawmakers do?

• This question presents a false dichotomy. Education spending in North Carolina has been on an upward trajectory for the past decade, even though the 2013 state tax reforms provided tax relief to businesses. In fact, the state has had several revenue surpluses since the reforms went into effect. Perhaps more importantly, polling almost always reveals that people want to spend "more" on education, even when they know little about how much is currently spent and even when they think increased spending won't lead to better outcomes. This overly-simplified WRAL poll question doesn't represent any of these nuances of the policy conversation.

    Question 3 - The average North Carolina public school teacher earns $54,000 a year. North Carolina ranks 29th out of 50 states for teacher pay. Should North Carolina teachers be paid less? More? Or the same?

• The problem with this question comes from the insertion of the 29th out of 50th It is a credit to WRAL that they include the average salary in the question, but then they muddy the water by adding in the negative ranking. The rankings, of course, do not take cost of living or experience of the workforce into consideration. Would people's opinions on North Carolina's average teacher salary be different if they didn't know the ranking? The only way to know would've been to ask two separate questions.

    Question 4 - There are 2 proposals to raise teacher salaries. One would raise salaries by 4% over two years. The other would raise salaries by 9% over two years. Which proposal should be passed into law?

• This question is ridiculous. You could insert nearly any two numbers and respondents would likely pick the larger of the two. As our friends at the John Locke Foundation pointed out, the more accurate question to compare support for the governor and legislatures positions would be to ask if respondents preferred a 4 percent raise or a zero percent raise, since that's the actual result of the governor's budget veto.

    Even if the given poll topics were asked in a way to validly measure what they are attempting to measure, the poll has been largely misinterpreted by both the poll creators and other outlets.

    The WRAL headline reads "WRAL News poll: NC residents back Cooper's positions in state budget debate." Not a single one of the questions from the poll mentioned the governor or the budget stalemate. Although the topics of the questions relate tangentially to the positions of the governor and the legislature, the questions themselves provide no concrete evidence that - if given the full story - North Carolinians would side with the governor.

    What makes this even worse is that the results get trumpeted. The far-left NC Policy Watch was able to simply re-print the words of the WRAL piece, advancing their agenda without any original analysis. It is disturbing - although perhaps not surprising - that the media and such a radical left-leaning organization are that closely aligned.