home
Homenavigate_nextIssuesnavigate_nextBackissuesnavigate_nextVolume 31, No.1navigate_nextIs Weighting a Routine or Something that Needs to be Justified?
Is Weighting a Routine or Something that Needs to be Justified?
Survey research as a method of collecting sample data is supposed to produce sample statistics which can estimate the corresponding population parameters if the sampling design is appropriate. However, for reasons such as unit non-response, survey data is usually weighted by the institutes that collect the data or by researchers who analyse the data in order to correct or diminish the discrepancies between sample and population. Sample statistics based on weighted data are more representative of the population parameters than unweighted data in terms of some demographic characteristics. Therefore, to some extent, it seems legitimate to weight data and this manipulation has become a routine when dealing with survey data. It is true that to weight data could be helpful, but this manipulation needs justifications. This paper therefore tries to argue that to weight data is no panacea and should not be taken for granted when considering the examples in Taiwan's Election and Democratization Studies (TEDS) surveys. The first section discusses why weighted data is not necessarily representative of the population. As the TEDS surveys show, the turnout, the vote shares of parties, and marital status become more deviant from the population parameters after weighting the data. If the focus is the relationships between variables, the correlations may be changed by weighting the data in bivariate or multivariate analysis. However, it is not clear whether we manufacture relationships which do not exist or if weighting the data actually helps us approximate the relationships that already exist in the population. Besides, it should be noted that to weight data set as a whole only deals with the problem of unit non-response, but does not solve the problem of item non-response. The third section discusses why most efforts should be devoted to examining and improving questionnaires, sampling designs, and interviewers' training and supervision, instead of simply appealing to post-weighting. If everything necessary has been tried, weighting data may be the last resort to improve the estimates. But the justifications for the selection of auxiliary variables and the methods of calculating weight factors should be provided rather than doing it without any explicit considerations. It is also important to consider whether the consequence of weighting is positive or negative.
