How Useful Are Pre-Election Public Opinion Polls and Exit Polls in Hong Kong?

The Legislative Council elections were held on September 9, 2012.  Prior to that, the media were reporting on the election races through public opinion polls.  On election night, exit polls were quoted extensively after the polling stations closed but before the official tallies were announced.

How useful are the pre-election public opinion polls and the election-night exit polls?

This can be regarded as a simple technical question.  But in Hong Kong, it is also a political question. 

One question is: Are the pre-election public opinion polls (both the openly disclosed ones and the privately conducted proprietary ones) being used to distribute votes.  For example, the DAB party has two candidates in the so-called District Council (second) Functional Constituency race.  They apparently asked their supporters in the Hong Kong and Kowloon areas to vote for DAB candidate Starry Lee Wai King, and those in the New Territories to vote for DAB candidate Lau Kong Wah.  This is a case of a party coordinating votes among its supporters to maximize effects.  Another question is whether the public opinion polls are being used to manipulate voter intentions.  That is, if one poll indicates that a certain candidate is likely to lose, his supporters may abandon him and vote instead for another like-minded candidate with better prospects.

When these kinds of political interests are involved, people would right to suspect the poll results.  Is the public opinion poll really fair, neutral and objective?  Or is it serving certain nefarious purposes?

With respect to the exit polls, it is one thing to say that the electronic media organization want these results for discussion while waiting for the official tallies to be announced.  There does not seem anything nefarious about it.  But it is possible that the exit polls are being used on election day to coordinate vote.  The polling stations open at 730am.  At 730pm, a certain political party has some exit poll data on hand.  They observe that they have more votes necessary for one candidate and not enough votes for another candidate.  So they immediately dial up 5,000 of their supporters who were told to wait at home for the signal to vote as directed.  Thus, the election was won because that political party had access to exit poll data that nobody else had.  That would be an unfair advantage.

In Hong Kong, the pre-election public opinion polls and the election-night exit polls are primarily conducted by two organizations: the Hong Kong University Public Opinion Programme and the Hong Kong Research Association.  The former has been criticized for sometimes wandering away from fair, neutral and balanced polling into political activism; the latter has been criticized for having a mysterious background and using inferior technology (automated telephone interviewing).  Periodically, people have called to boycott one or both of these organizations.  Notwithstanding the hubbub, these two organizations continue with their work.

The following is an assessment of the pre-election public opinions polls and election-night exit polls.  Our approach is not based upon scientific theories.  We are assessing the polls on theoretical grounds (e.g. one poll has higher response rate; etc).  However the polls were conducted, we are simply going to look at how well the results match the final outcome.  If the poll results are drastically different from the election outcomes, then theoretical purity serves no purpose and we should ignore the polls altogether.  If one polling organization performs better than the other, we should give more credence to the one and less credence to the other, irrespective of any theoretical advantages/disadvantages or political biases.  If you like, this is a case of finding the cat that can catch mice, whether it be a black or white cat. 

Of course, if you are intellectually curious, you may want to think about how canceling errors in a multiply flawed methodology can actually produce useful results.  However, that may be a topic that is even deeper than a doctoral thesis.

Hong Kong Research Association

In the table below, we show the Hong Kong Research Association results.  The first column of numbers are their pre-election public opinion polls conducted between August 21 and September 2.  The answers also included certain "Undecided" responses.  We have re-percentaged everything to share of votes among those who chose any candidate to give the share of votes (which adds up to 100%).  Applying the election rules, we determined the winners.  We compared it against the actual election tallies.  Out of 40 seats, the Hong Kong Research Association called the outcomes correctly in 34 cases for a 85% success rate.  The right farmost column of numbers is the Hong Kong Research Association exit poll classification.  They did not post any numbers this time.  All those except 2 cases classified as "Very likely" and "Somewhat likely" won.

Hong Kong University Public Opinon Programme

In the table below, we show the Hong Kong University Public Opinion Programme results.  The last rolling pre-election public opinion poll was taken between September 3 and September 7, 2012.  We converted the raw numbers into share of votes and we applied the election rules to find the winners.  Out of 40 seats, the Hong Kong University Public Opinion Programme poll was right in 36 cases (90% success rate).

Conclusion

The two pre-election public opinion polls were successful 85% or more of the time.  The incorrectly predicted cases tend to be the more marginal candidates.  So the next time when you see more poll results from these two organizations, it is "very likely" that that they are accurate.  You may prefer to see a different result, but that is "very likely" just wishful thinking on your part.