Chapter 11: Good Polls, Good Pollsters, Good Sponsors
By Alan F. Kay, PhD
© 2004, (fair use with attribution and copy to author)
Aug. 16, 2004
Good sponsors must take responsibility for hiring good pollsters and conducting good polls. But how does a sponsor become a good sponsor and how can a good sponsor convince the public audiences for its polls that its poll findings can be trusted? This book has given many tips on how to spot the spin in polls and more generally what, besides the absence of spin, is needed to make a good poll. But what does a sponsor that wishes to produce high quality political polls have to do to be judged of top quality, and how can it demonstrate that it is aspiring to do something that is much larger and more difficult than current commercial level polling? Such a sponsor is obligated to demonstrate that it is trustworthy, and that sponsor may have to go to great lengths to succeed. Let’s look at some of the things that a public-interest polling sponsor could do to help convince others it is performing at a high-standard level.
A Public-interest Polling Sponsor’s Obligations
Primary Purpose Public-interest polling has been defined as political polling where the sponsor’s primary intention is to uncover, as reliably and stably as possible, what the public most wants for governance. The sponsor is presumed competent and sincere. It matters little if the sponsor has other desires, hopes, or intentions for surveys, as long as all decisions as to the design, conduct, analysis, promotion and distribution of the findings give the highest priority to finding the public’s wants for governance. Uncovering the “public-interest” has to be the primary purpose, with priority over all other purposes.
Credibility and Trustworthiness. It is up to the sponsoring organization(s) to establish a relationship with the audiences for its polls to persuade them of its true intentions and thus to demonstrate that it is a credible public-interest polling organization. This can be done in many ways. ATI has used the following ways in varying degrees. It is hard to see why anyone sincerely wishing to be a public-interest poll sponsor would not be pleased to follow most, if not all of them and, where practical, even strengthen them.
| 1. | The methods used to perform the surveys should conform as a minimum to the highest generally accepted commercial polling standards. |
| 2. | Limitations of the methods used should be acknowledged frankly and fully, while efforts should be made to remove or overcome significant limitations. |
| 3. | The survey and question design should be carried out by a team that includes experts in political polling and in the issue area(s) under investigation, representing a wide range of viewpoints across the political spectrum. The team should include some with knowledge of, or easy access to, the enormous range of policy proposals available from the government, political leaders, policy organizations, and others. The process of designing, conducting, and analyzing the survey data should be considered a collaboration between experts and the public. The important and amazing results of public-interest polling will not be achieved without both the experts fulfilling their assigned role and the public making its judgments. An important duty of the team is to create questions that are fair, balanced, and accurate, as defined in Locating Consensus for Democracy, pp. 350-353. |
| 4. | One of the most important functions of the team is seeking, culling and refining for inclusion in the surveys a wide range of policy-choice options. This is an absolutely essential feature of public-interest polling. If the choices offered are only those that leading politicians and the major news media poll on, it cannot be public-interest polling. To the extent that the survey team does not fairly and fully carry out its role, the full benefits of public-interest polling will not be realized. |
| 5. | Every member of the team should sign off on the survey report, should be recognized for his/her role in the report, and if team members disagree with any finding, those members’ views should be welcomed, acknowledged, and carried in some section of the report. |
| 6. | The sponsor should make clear in its written materials, particularly in each survey report, what its intentions are. It should make plausible why it wishes to conduct surveys in issues and subjects it has chosen. It should explain how it is financed, and how it intends to use the resulting information. All results, including the full and complete wording of the interviewers’ script and its own analysis, should be made public shortly after the results are available. If the sponsor can state that it does not conduct special-interest polling as well, that will add to the sponsor’s credibility. |
| 7. | The sponsor should be open to cooperating with any other public-interest polling organization by a willingness to share data and methodology used so that others can confirm, build on, or find limitations to the sponsor’s findings. |
| 8. | The sponsor, as long as it remains an active public-interest polling organization, should encourage members of the public with unusual interests in the issues it surveys to become involved. Because they are leaders, pundits, or experts, or have had considerable practical or theoretical experience in the issue or because the issue impinges on them more than on others, such people should be invited to submit new versions of policy proposals or previously untested proposal arguments, both pro and con, for testing public support, as soon as such can be scheduled into the sponsor’s ongoing survey research program. If there is any charge at all for such inclusion, it should not exceed a fair allocation of the survey’s full costs. The sponsor should be open to including questions in upcoming surveys that others challenge them to include. |
| 9. | Individuals who are key in the sponsoring organizations or are themselves sponsors should make clear their own biases on the policy questions in each survey by saying honestly how they would respond to these questions. Sponsors should voluntarily take the test that they ask others to voluntarily take! |
| 10. | The sponsor(s) must take responsibility for managing the design team, especially the pollsters, to make sure that those of the preceding items that the sponsor claims to be following actually are being followed. |
If a sponsor conforms to these, or to most of these, suggestions, I believe that its credibility as a bona fide public-interest polling organization will be quite readily established and will increase as long as it continues to abide by these suggestions. Then, others who claim to poll in the public’s interest would not be able to successfully fake it for very long.