An Analysis Of A Hong Kong District Council Election Exit Poll

(The Standard)  Voters told to shun poll questions  Diana Lee   November 3, 2011.

Voters are being urged not to respond to exit polls outside voting stations during Sunday's district council elections as information may be used in last-minute campaigning. Wearing surgical masks adorned with an X to help make the plea for silence, pan- democrat lawmakers said they are concerned about data from some pollsters going to certain groups - presumably pro-establishment candidates - before voting ends.

Polling stations will be open from 7.30am and close at 10.30pm.

Democratic Party legislator Emily Lau Wai-hing, who is not a candidate, said pan- democrats have for years expressed concerns about exit polls to the Electoral Affairs Commission but have been ignored.

Civic Party lawmaker Audrey Eu Yuet- mee, who is also not a district council candidate, said she had suggested that only universities be allowed to conduct exit polls and there be clear guidelines about data collected not being used on polling day. But this was rejected by the commission, she said.

Lau admitted that their call may affect genuine academic research, such as that by the Public Opinion Programme at the University of Hong Kong.  But she understood from POP director Robert Chung Ting-yiu that members of his team would be in outfits to identify themselves in the 19 constituencies where they plan to conduct exit polls. Still, the public must be cautious, Lau said. She had come up against a data collector claiming falsely to be a government worker.

Commission chairman Barnabas Fung Wah said after visiting a mock polling station yesterday that his office has approved nine of 10 groups or individuals who have applied to conduct exit polls. All said their polling is for academic research. If any of them were known to have close links to candidates, Fung added, the commission would not have approved an application. But he declined to say why one application was refused.

Speaking for some exit pollsters, the Hong Kong Research Association's Mak Yin-mei said an independent institution will not pass data to particular parties.

About 2.9 million people are eligible to vote on Sunday.  There will be 839 candidates contesting 336 seats across the 18 districts. But candidates in 76 of the 412 constituencies are standing uncontested.

The list of organisations and persons allowed to conduct exit poll by district for 2011 District Council can be found here.

The polls closed at 10.30pm.  At 10.50pm, the Hong Kong Research Association posted its exit poll results in 25 districts (see link).  These districts contains the most discussed races which involves well-known politicians.

Since the actual vote tallies would not be known until several hours later, these exit poll results were used extensively in the televised coverage that night.  The research question of interest here is whether these exit poll results were skewed by the pan-democrats' boycott call to the point where they become useless. 

In the following, I have compiled a table in which the exit poll information is posed together with the actual vote tallies (see link).  I have colored the vote leader/poll leader in yellow.  If you are interested in running your own analyses, you can use my Excel spreadsheet which contains some additional information on response rate.

First of all, it is noted that the Hong Kong Research Association reported that they made attempts to interview 30,776 voters on their way out.  Only 12,172 (or 39.5%) responded.  This is not a good response rate for public opinion polling.  However, it is under-appreciated that this does not automatically invalidate the data.  After all, it is theoretically possible for a poll with 39.5% response rate to be closer to the truth than a poll with 60.0% response rate (see Footnote 1).  In the final analysis, the criterion will still have to be whether these data are accurate enough for calling the winners and gauging the margins of victory/defeat at a time when the vote tallies were not yet available.

Out of 25 cases, the Hong Kong Research Association called the races in the right direction in 22 cases (88% accuracy).  In other words, it was wrong in only 3 cases:

Southern Horizons West: HKRA called 36.3% for Fung Wai Kwong and 57.6% for Sze Chun Fai.  The vote tallies were 47.4% for Fung Wai Kwong and 47.1% for Sze Chun Fai.

Olympic: The HKRA called 37.0% for James To Kun Sun and 58.0% for Vincent Lau Kai Kit.  The vote tallies were 50.6%f for James To Kun Sun and 44.3% for Vincent Lau Kai Kit.

Fu Yan: The HKRA called 52.5% for Lee Cheuk Yan and 47.5% for Lau Kwai Yung.  The vote tallies were 48.0% for Lee Cheuk Yan and 52.0% for Lau Kwai Yung.

In practice, the case of Fu Yan would be characterized as "too close to call."

In conclusion, the Hong Kong Research Association exit poll was serviceable notwithstanding the pan-democrats' boycott call.

There are policy implications here.  The premise is that certain organizations are running exiting polls on election day for the purpose of optimizing resource allocation on election day.  The antidote is to call for everyone to boycott the exit polls or lie about their votes ("always say that you voted for the DAB candidate").  But is that working or not?  The action seems reasonable but that does not guarantee that it will work.  The assumption has to be checked against reality.  The analysis here shows that it is not working, because the Hong Kong Research Association certainly has serviceable exit poll data that revealed patterns that were not obvious before election day.  Now what?  How do you eliminate the advantages accrued to whoever is funding these types of exit polls?  Well, that requires some thinking.  What is for certain that the problem was not solved by some piece of wishful thinking (such as a boycott call).

Footnote 1: Consider a population of 1,000 persons, of which 500 are RED and 500 are BLUE.  A poll conducted by one organization (e.g. Joint University Public Opinion Research Institute) reached out to these 1,000 persons and only 400 responded (at 40% response rate), with 200 RED and 200 BLUE.  This poll therefore estimates a 50%/50% split which is perfect.  Another poll conducted by a different organization (e.g. RED-IS-BEST OUTREACH CAMPAIGN) reached out to the same 1,000 persons and 600 responded (at 60%) response rate, with 500 RED and 100 BLUE.  This poll therefore estimates a 83%/17% split which is way wrong because it attracts the REDs and alienates the BLUEs.  So having a higher response rate does not automatically guarantee more accurate data.  This is not a suggestion that we should avoid high response rates.  This is a comment that a higher response rate is not a panacea.