Public Opinion Polls And The Hong Kong Legislative Council Elections

The Hong Kong Legislative Election will be held on September 7, 2008.  There is a great deal of interest about the developments.  Who is leading?  Who is lagging?  Who is gaining momentum?  Who is losing momentum?

Those answers have to come through public opinion polls.

Some candidates and political parties may be conducting their own public opinion polls, but they don't usually share the results with the public.  Even if they publish some proprietary polling results, the public ought to be skeptical about selective leaking to satisfy certain agenda.

Therefore, the public has to rely on the public opinion polls that are conducted by independent third parties (such as university research institutions) and released to the general public. 

The three major public opinion polls on the Legco elections are the following:

• Hong Kong University Public Opinion Programme.  Unfortunately, you won't find any real data because of this situation:

The above data links contain findings from POP's Legislative Council Election rolling and exit polls. They are provided to sponsors for their first use. The links will be open for public consumption after the election.

You will have to read the sponsoring media (e.g. Apple Daily, Ming Pao, South China Morning Post, etc) to find the actual poll results.  But please be careful because some of the sponsoring media have been selective in publishing partial data to satisfy certain agenda.  That is, they do not release all of the poll results; they only publish selective slices every day in conjunction with editorial statements (e.g. vote for democrat XXX in district YYY because there is an emergency situation!).

• Hong Kong Institute of Asia-Pacific Studies, The Chinese University of Hong Kong (in Chinese only).
  　
• Hong Kong Research Association (in Chinese only).

In the following, I will run an analysis of these public opinion polls.  I am particularly interested in certain aspects of the polling process that may impact the end result.

The first question is: what is the universe of persons that is getting measured?  Many public opinion polls gauge attitudes on public issues (such as the domestic helper tax, inflation, satisfaction with the job performance of the HK SAR Chief Executive, fire safety, etc) and it is reasonable to use the universe of all adults (age 18 or over) in Hong Kong.  But since this is about the Legislative Council elections, it seems reasonable to restrict the universe to all registered voters in Hong Kong.  But if the purpose is to gauge the possible election outcome, it seems reasonable to further restrict the universe to all registered voters who are likely to vote.

For the HKIAPS-CUHK polls, the voter preference results exclude those who indicate that they do not intend to vote.  That leaves those who "will definitely vote," "may vote" or "are undecided/unsure."  During August 18-24, 9.1% said that they won't vote, 22.5% said that they may vote, 66.4% said that they will definitely vote and 1.9% said that they are undecided/unsure.

For the Hong Kong Research Association, the voter preference results are restricted only to those who say that they will vote.  That excludes those who "won't vote," "don't know" and "have no opinion."  During August 1-8, 62% of registered voters said that they will vote, 12% said that they won't vote, 15% are unsure and 11% have no opinion.

The HKU POP site does not publish any information on the universe because the data is reserved for first use by the sponsors.

Immediately, you can see that there are different choices for the universe.

The next question is about the polling methodology.  Due to cost considerations, the only cost-feasible choice that delivers data of decent quality is the telephone methodology.  Face-to-face, door-to-door personal interviews are cost-prohibitive, while mail/internet surveys deliver poor quality data.

This leads to questions about the coverage of the telephone sampling frame.  The starting point may be the telephone directory.  There are known deficiencies about the coverage based solely upon the telephone directory.  The most important factor is that telephone penetration is not universal.  At one point back in time, those without telephones are most likely poor people who cannot afford to have a telephone.  There is also a small number of people (such as celebrities) who opt out of the telephone directory listing.  More recently, there are young, affluent people who use mobile phones exclusively (note: the telephone directory only covers landline phones and not mobile phones).

Another important question is how the telephone interviewing is conducted.  The preferred method is interviewing conducted by a real person.  HKU POP explicitly adds this footnote (see, for example, this poll):

The data of this survey is collected by means of random telephone interviews conducted by real interviewers, not by any interactive voice system (IVS). If a research organization uses "computerized random telephone survey" to camouflage its IVS operation, it should be considered unprofessional.

This dig is aimed at the Hong Kong Research Association, which uses an interactive voice system.  This is somewhat abstract, but I can make it more concrete with my personal experience when I was called up by HKRA for one of their interviews:

The telephone rings.  I pick up the telephone.  I am asked: "Are you age 20 or older? If yes, press 1; if no press 2."  I press 1 and the interview continues.

Thus, they are using the first adult that picks up the telephone.

In my household, there are three adult residents.  When the telephone rings, I always pick up because there is a strong likelihood that it is for me (e.g. business associates, friends, service calls, etc).  The other two adult residents are both over 80 years old, barely receive any calls and would not know how to respond to my calls.  Therefore, they will always let me pick up the phone if they know I am home.

Therefore, this first-adult methodology gives me a 100% chance of selection and 0% of them.  The resulting polling results will be biased.

By the way, Apple Daily and Oriental Daily/The Sun also use interactive voice systems to conduct quick-and-dirty polls.

There are many issues that may bias poll results, including: sampling frame coverage; response rate; respondent selection within household; weighting; question phrasing; etc.

Given that these three polls differ in many aspect, there is no reason to expect that they should deliver in the same poll results.

How different are they?  Here are the most recent poll results that I can find:

The polls do not yield the same results (in the sense that they do not yield the same list of people being elected to the 30 Legco seats).  But then again they are not expected to due to time period differences, survey biases (including those listed above) and sampling error.

How do we know which poll is correct (or more correct)?  We don't.  And we cannot know because survey bias is defined as the expected value of the survey estimate minus the true number.  If we know the true number, we would not have to conducting any polling.  This is tautologically unknowable.

So what good is any of this?  Well, each and every one of the polls may be flawed in many ways.  But sometimes it is possible to be right for all the wrong reasons when errors cancel themselves out.  How do we know that?  We can check with the past performances.  When a poll did not work out last time, can we determine why and make an appropriate adjustment?

Let us review what happened in the 2004 Hong Kong Legislative Council election results.

The final voter turnout was 1,784,140 out of 3,207,227 at a rate of 55.6%.  This was better than the two previous Legco elections in 2000 and 1998, but it is less than the 60% forecast (and never mind the 70% wished for by the Democratic Party).  The results of the directly elected geographical constituencies are shown in the table (with the winners in bold letters), and you can find the results of the functional constituences in this other post.

Geographical District/candidate	%HKU 9/11 poll	# of actual votes	% of actual votes
Hong Kong (5 seats) 1. Ma Lik/Choy So-yuk (DAB) 2. Rita Fan (independent) 3. Tsang Kin-shing 4. Yeung Sum/Martin Lee (DP) 5. Kevin Wong 6. Audrey Eu (Article 45 Concern)	15.5 % 23.6 %   1.3 % 27.4 %   0.7 % 31.6 %	74,662   65,660     5,312 131,785     2,829   73,841	21.1 % 18.5 %   1.5 % 37.2 %   0.8 % 20.9 %
Kowloon East (5 seats) 1. Fred Li (DP) 2. Chan Kam-lam (DAB) 3. Albert Cheng (independent) 4. Alan Leong (independent) 5. Chan Yuen-han (HK FTU)	16.9 % 13.3 % 21.5 % 23.1 % 25.2 %	56,409 55,188 73,424 56,161 52,520	19.2 % 18.8 % 25.0 % 19.1 % 17.9 %
Kowloon West (4 seats) 1. Lau Yu-shing 2. Frederick Fung (ADPL) 3. Lau Chin-shek 4. Tsang Yok-sing (DAB) 5. James To (DP) 6. Bruce Liu (ADPL)	0.4 % 23.3 % 23.7 % 19.5 % 28.0 %   5.1 %	1,824 46,649 43,460 61,770 60,539 13,452	0.8 % 20.5 % 20.4 % 27.1 % 26.6 %   5.9 %
New Territories East (7 seats) 1. Leung Kwok-hung (April 5 Action) 2. Tso Wung-wai (HK Progressive Alliance) 3. Andrew Cheng, Emily Lau, Ronny Tong 4. James Tien (Liberal) 5. Andrew Wong 6. Lau Kong-wah/Li Kwok-ying (DAB)	7.9 %   3.5 % 43.6 % 20.2 %   7.5 % 17.3 %	60,925   14,174 168,833   68,560   23,081   95,434	14.1 %   3.3 % 39.2 % 15.9 %   5.4 % 22.1 %
New Territories West (8 seats) 1. Albert Chan 2. Lee Wing-tat (DP) 3. Albert Ho (DP) 4. Leung Yiu-chung (Neighborhood & Workers) 5. Chow Ping-tim 6. Stephen Char 7. Tam Yiu-chung/Cheung Hok-ming (DAB) 8. Ng Tak-leung 9. Selina Chow (Liberal) 10. Lui Sau-tuen (New Century Forum) 11. Lee Cheuk-yan (HK CTU) 12. Yim Tin-sing (ADPL)	5.4 % 15.2 % 12.2 % 12.6 %   0.2 %   2.9 % 18.7 %   0.8 % 14.6 %   1.0 % 12.7 %   3.6 %	36,278   62,500   62,342   59,033     1,725     9,116 115,256     1,920   50,437     4,511   45,725   14,570	7.8 % 13.5 % 13.5 % 12.7 %   0.4%   2.0 % 24.9 %   0.4 % 10.9 %   1.0 %   9.9 %   3.1 %

I have included the HKU final public opinion poll numbers to illustrate a longstanding complaint from the DAB.  I have extracted these numbers:

• Hong Kong: Ma Lik (DAB): HKU 15.5%, actual 21.1%.  Relative shortfall of 27%.
• Kowloon East: Chan Kam-lam (DAB): HKU 13.3%, actual 18.8%.  Relative shortfall of 29%.
• Kowloon West: Tsang Yok-sing (DAB): HKU 19.5%, actual 27.1%.  Relative shortfall of 28%.
• New Territories East: Lau Kong-wah/Li Kwok-ying (DAB) : HKU 17.3%, actual 22.1%.  Relative shortfall of 22%.
• New Territories West: Tam Yiu-chung/Cheung Hok-ming (DAB): HKU 18.7%, actual 24.9%.  Relative shortfall of 25.0%.

Why are the HKU POP poll numbers so wrong for the DAB?  This is a matter of conjecture.  One explanation that is often given is that the DAB supporters are not included in sampling frame (i.e. they do not have telephones) or else they hide their opinions when interviewed.  My preferred explanation is that the DAB has a superior GOTV (Get Out The Vote) effort.  According to the public opinion polls, about 90% of the people said that they will vote.  In reality, just over 55% voted.  The DAB is simply better able to get their supporters to vote because they have more manpower and resources (as in reminder phone calls, personal visits, etc).  Therefore, the DAB end up with a higher share of the actual votes than the polls would indicate.

Here is an illustration with a hypothetical example:

Constant Model (60% voter turn out for all candidates)

Candidate	Projected Votes According to Poll	Actual Votes	% Votes
1	20,000	12,000	20%
2	20,000	12,000	20%
3	20,000	12,000	20%
4	20,000	12,000	20%
5	20,000	12,000	20%
Total	100,000	60,000

Differential Voter Turnout Model (80% for candidate 1 and 55% for candidates 2-5)

Candidate	Projected Votes According to Poll	Actual Votes	% Votes
1	20,000	16,000	27%
2	20,000	11,000	18%
3	20,000	11,000	18%
4	20,000	11,000	18%
5	20,000	11,000	18%
Total	100,000	60,000

This means that the poll results that you see in media reports are suspect.  All this is just of academic interest to those people who are interested in the horse race results. 

The more important issue is what should you do as a voter?  How should you maximize the impact of your vote in light of the current trends as indicated by poll results?

I suggest that the answer is very simple.  As a voter, you should just ignore the poll results.  Instead, you should look at the credentials, records, policy platforms, debate performances and other direct aspects of the candidates, and you make your choice.  As for the self-selected spins on the poll results, pleas to distribute the votes within your households, last-minute emergency appeals to save 'democracy' in freely distributed newspapers ... well, they are just statistics, lies and damned lies.  If you cannot even vote according to your own conscience and judgment, what is the point of even casting a vote?  You might as well as sign a proxy card and let someone decide for you.