Regulating Hong Kong Exit Polls?
According to Hong Kong University Public Opinion Programme director Robert Chung Ting-Yiu, the response rate to the exit poll in the Legco election this September fell down drastically compared to previous years. Here is the table for the four Legco elections in Hong Kong While there is an overall trend of dropping response rates by 1% or 2%, the drop this time is beyond the pale.
Robert Chung pointed out that this situation was happening because unprofessional exit polling organizations were clamoring and deceiving voters outside the voting stations. On election day, these polling organizations sent out large numbers of interviewers with no concept of random sampling. Sometimes, there were even more interviewers than voters, who were getting annoyed. Even worse yet, someone posed as a government worker in order to elicit opinion.
Robert Chung said that some of these exit pollsters were connected to certain candidates. "These candidates can obtain the poll data immediately in order to allocate the votes. The candidates do not have to report this as campaign expense, and therefore the other candidates felt that this is unfair."
Robert Chung proposed that there ought professional regulations for exit poll organizations. The industry should voluntarily subject itself to independent monitoring and be penalized for violations. This will increase the common acceptance of the polling organizations by the citizens. Alternately, the organizations can be certified through an independent process to establish their reputation.
Apart from the exit polls, there has also been a surfeit of so-called public opinion polls in Hong Kong recently. "There are too many polls, so the citizens just laugh when they read the results. This is a huge problem." Robert Chung is concerned that too many poll results may diminish the value of social research and cause the people not to take public opinion polls seriously.
Some university scholars believe that exit polls are just commercial activities with no academic value. Robert Chung responded: "You would be stupid to make that statement question overseas. Everybody knows that exit polls are a multiple-win situation." He said that even if Hong Kong University [Public Opinion Programme] does not perform exit polls, it would not be a blow to the finances of the organization. Exit polls are expensive to conduct with a small profit margin. But without the exit polls, there would be a lot less data to analyze, because many of these things cannot be replaced by telephone interviews.
If it is stupid overseas to denounce exit polls as commercial activities with no academic value, then it is just as stupid to establish an quasi-government commission to regulate polling organizations. In the United States of America, the free market rules. Anyone can start a polling company anytime, and success is gauged by the utility and accuracy of the data. If the interviewers are untrained with no concept of random sampling and that is actually a real problem, the poll results will diverge from the other existing polling data. No one will buy or believe in inaccurate data, and the company will be eliminated sooner or later. Certification by an independent organization means nothing if the data are inaccurate. I would like to address how the Hong Kong University Public Opinion Programme perform in these aspects.
The polls serve the public interest
The pre-election opinion polls allow the public to understand the evolving situation of the election. Who is hot? And who is not? They may guide the public in making the strategic voting decisions. For example, a voter may have liked candidate X, who is apparently hopeless according to all polls, but it seemed that the second choice Candidate Y has a better chance to beat the much reviled Candidate Z. Therefore, the voter chooses Candidate Y rather than waste it on Candidate X. As such, it is desirable to have as many polls out there as possible (including those with likely political biases).
But what did the Hong Kong University Public Opinion Programme do during the 2008 Legco election? It conducted a daily tracking poll, and it released its data only to its newspaper sponsors. The general public did not get to see the full data until AFTER the election. The general public only knew what the newspapers selectively choose to print every day. It was quite clear that some newspapers held certain political biases and they selectively published whatever suited them each day (such as information that favors their preferred candidate in one district while ignoring everything else going on elsewhere).
The Hong Kong University Public Opinion Programme treated the daily tracking polls as a commercial activity. The net result is that part of the public may be misinformed because of the selective publication of data in the media. The public interest is not being served here. In the United States, the polling organizations usually partner with media organizations (e.g. Gallup/Harris etc with ABC/CBS/CNN/New York Times/Washington Post/Newsweek/TIME etc) but the polling data are released in full. We don't see games such as McCain is doing great in Oklahoma/Nebraska today according to the XXX national poll while the data for the rest of the country go missing. But we did see games in Hong Kong such as the support for candidate XXX of district YYY has just plummetted in the latest 3-day tracking poll while the data for the entire month and the sample size qualifications (it was based upon 6 responses out of 150 persons) go missing.
The exit polls allow the public know what is likely to be going on before the official tallies are released. Did things work out as the pre-election opinion polls say? Or were there upsets?
But what did the Hong Kong University Public Opinion Programme do during the 2008 Legco election? It conducted an exit poll, and it released its data only to its electronic media sponsors at 8pm (note: the polls close officially at 10:30pm). The general public did not get to see the data except as qualitative statements through the electronic media.
The Hong Kong University Public Opinion Pgoramme treated the exit polls as a commercial activity. This is now two weeks AFTER the election, and the Hong Kong University Public Opinion Programme has not yet released the full data. If and when the full data ever get released, it will be ancient history for the public. The public interest is not being served here.
I submit that the public interest is served by as many public opinion polls as their sponsors feel like financing. The data should be published in full, and not selectively according to political preferrences. This is not a case of the bad products chasing out the good products in a market. Rather, this is a case of the good products chasing out the bad products.
For the 2008 Legco elections, there were four tracking polls (Hong Kong University Public Opinion Programme, Institute of Asia-Pacific Studies at the Chinese University of Hong Kong, Lingnan University Public Governance Programme and Hong Kong Research Association). The first three are affiliated with academic institutions and the fourth is an independent organization that is alleged to have ties to mainland Chinese interests. But no matter, because it is the end results that matter (see the final tracking poll data here). Each poll has its sources for biases -- question wording; sample frame; sample coverage; differential response rates; interactive voice system versus live personal interviewing; random respondent selection; weighting; etc. But the results don't look significantly different -- nobody got it all right and nobody was way wrong. Isn't it better to have more sources to consult from than just a single authoritative source? And if one of them should be way different from the others, that would be a sign to give it less credibility.
The polls serve academic research
Public opinion polls (especially tracking reports over extended periods of time) are of great academic research. The book above titled <Looking at Hong Kong society and politics from public opinion polls> comes from the Institute of Asia-Pacific Studies at the Chinese University of Hong Kong based upon its public opinion polling data over many years. The book below titled <The 2004 Legislative Council Elections in Hong Kong> also comes from the Institute of Asia-Pacific Studies at the Chinese University of Hong Kong. Eventually there may be books about the 2008 Legislative Council elections as well.
The Hong Kong University Public Opinion Programme released its tracking poll data after the election here. The exit poll data have not been released as yet. For the 2004 Legco election, the Hong Kong University Public Opinion Programmed has published related reports here, which are very interesting and informative (and therefore serves the public interest) but not widely known.
I submit that very few people have read these and other academic publications.
The polls are used for political purposes
When the pre-election public opinion polls are released only selectively to the public, political massaging occurs. The antidote is to publish everything.
The political application appears to be a greater issue with the exit polls, for which a number of obscure organizations were accredited by the Election Affairs Commission to hold polling of voters outside the voting stations. The idea is that these exit poll data are being used by a certain large political party to allocate its supporters among various candidate lists.
There is an urban legend that more than 100,000 'iron votes' are held back until 8pm on election day. Then the masters in Sai Wan issues an order to tell them to vote in accordance to the latest exit poll data. The late hour of voting meant that the opponents have no time to react. For many reasons, I don't believe in that. This is a bogeyman.
Here are some reasons why I don't believe this.
First, how do you maintain contact with 100,000+ people? You could tell them to sit and wait for your phone call at 8pm. But how do you make 100,000 calls at 8pm? You can't do it alone, so you will probably need at least 1,000 people doing 100 calls each just after 8pm. You could not rent a central location to house those 1,000 people. So you need a hierarchical organizational structure: You call 5 lieutenants, who call 10 of their deputies each on the average, who call 10 of their managers each on the average, who call 10 of their assistants each on the average, who call 20 'iron voters' each on the average. 5 x 10 x 10 x 10 x 20 = 100,000. How would you like to manage that contact spreadsheet about who is calling who? Have you considered the risks associated with this organizational structure? One of your lieutenants gets run over by a bus at 7:30pm and 20,000 votes are left hanging!
Alternately, you can require that everyone be ready to receive a SMS for voting instructions. At 8pm, 100,000 SMS messages go out. Have you considered the risks associated with this method of communication? You send the SMS and you get the return message: "Our network is experiencing traffic congestion at this time. Your message could not be delivered. Please try again later." This leaves with you with a total disaster on your hands.
Secondly, if this project requires such a tremendous effort that involves more than a hundred thousand persons, then why hasn't any concrete information ever be leaked out in the press or the Internet forums? I have never seen anything published in this regard. By that, I mean something as simple as someone telling a story such as: "I asked my parents whether he wanted to join us for dinner. My dad said no because he must stay home and wait until 8pm for the instruction on how to vote." On this election day, an allegation from a voter that an exit poll worker claimed falsely to be working for the government was aired repeatedly on the television coverage. Can you imagine how much airtime these other stories would be getting? But there has been no such story.
Thirdly, let us suppose that someone out there actually controls the way how 100,000+ voters vote. Couldn't these same people be mobilized to do something less. For example, they can be mobilized to join the patriotic march on July 1st as a counter-weight to the pro-democracy march on the same day. Traditionally, the patriotic march always had between 10,000 to 20,000 marchers of which a large proportion were high school marching bands. If these supporters are willing to cast their votes however they are told, why couldn't they be asked to also volunteer a couple hours of their time once a year?
Fourthly, let us see if the number of 'iron votes' could be smaller than reported 100,000+. 100,000 divided by five districts is 20,000 each. That is big enough to affect the district election outcome. If the number of 'iron votes' is 2,000 or even less, then it is too small to change anything. It seems that you still have to be talking about 10,000 or more per district to affect the outcome, for which the three problems listed above still apply.
So what are these exit polls being used for? I have no idea. But for the reasons above, I see neither the theoretical feasibility nor the actual evidence for 100,000+ 'iron votes' after 8pm. By the way, such actual evidence would be present in the exit polls because the exit interviews should be time-stamped and it is possible to see an anomalous surge in the final 2-1/2 hours of the voting.
How are public opinion polls regulated in the United States of America?
It isn't. PERIOD.
And they don't feel that it is necessary because a free market will take of things by itself. If your poll numbers are inaccurate (compared to the myriad of options out there), you will be eliminated by natural selection.
If you insist, there are two regulatory models in the United States of America.
One is organizational membership in a professional association. The most significant professional association is the American Association for Public Opinion Research. However, it has only individual membership but no organizational members. [This blogger is a long-time individual member of AAPOR, mostly for the sake of the journal Public Opinion Quarterly.] AAPOR states: "It is appropriate and acceptable for individuals and organizations to state that they subscribe (if individual members) or adhere (if not AAPOR members) to the AAPOR Code of Professional Ethics and Practices. However, the reference to AAPOR membership must not state or imply that membership in itself is evidence of professional competence." Furthermore, the name AAPOR means nothing to the general American population. Thus, AAPOR membership will not confer credibility and increase cooperation rate. There is no such comparable organization in Hong Kong, nor will it have enough membership or clout to be significant.
The other is to seek accreditation on the basis of an extensive audit conducted by an independent organization such as the Media Rating Council. This is largely used to audit media measurement services (such as television people meter panels, Internet user panels, print readership surveys, etc). Public opinions do not go through this process because it is very expensive. [This blogger is responsible for working on the MRC audits of the various services of his company, and therefore understands the costs and the complexity of the process.] There is no such comparable organization in Hong Kong, nor is it affordable.
Of course, Hong Kong does not have to follow an American model. It can develop its own model. But who is going to follow the call of Robert Chung Ting-yiu, especially given the way that his programme has served the public interest during the Hong Kong Legislative Council election?
