Three Lessons Learnt in My AsiaBarometer Survey Experience, 2002-2022 from Three Sub-disciplines: Cultural Psychology, Statistics, and Experimental Psychology

บทคัดย่อ

This paper presents how I attempted to escape from, and try to overcome, some pitfalls of public opinion research that were born in the West and have diffused to the entire globe. This paper focuses on how we can mitigate some Western biases in questionnaire formulation, data analysis, and publications. I have learnt from three sub-disciplines: cultural psychology, statistics, and experimental psychology about cultural biases, misuse of statistics, and experiment misapplication in public opinion research. Here, I provide my concrete experiences in formulating the questionnaire for the AsiaBarometer Survey 2000s, in executing a trial and error of statistical data analysis for writing five monographs from 2010 and onward, and in reflecting on the whole process, especially focusing on questionnaire formulation, execution of field work, book publications with three methodological issues kept in mind (Inoguchi, forthcoming in 2022; Inoguchi, 2017; Inoguchi and Tokuda, 2017; Inoguchi and Fujii, 2013; Shin and Inoguchi, eds., 2010).

 This Abstract points out how we can avoid and try to overcome some pitfalls learning from three other subdisciplines: cultural psychology, statistics, and experimental psychology. (1) Cultural psychology: a: DATA CREATION: Survey methodology was born in the West and diffused to the whole globe but questionnaires are mostly created in the western mode, if not in toto. Nisbett (2003) argues that Westerners and Asians think differently about the same questions. Westerners think analytically while Asians think holistically; Westerners focus on leading actors and their universalist logics while Asians concentrate on contexts and backgrounds and they look toward large masses (cf. Henrich, 2020). b: ANALYSIS MODE: questionnaire in the United States, Canada and Australia, which employ the western mode, dominantly use psychology undergraduate students as respondents accounting for 70% of their manuscripts published in major leading journals. c: PUBLICATION SPACE: These responses from the undergraduates are used in psychological data analysis that led to publications in leading academic psychology journals in the United States and Western Europe. At each of the three processes of survey targeting, respondents selection, and choice of publication outlets, Asians have been misrepresented. (2) Statistics: a: Contributions to our knowledge depend on good questionnaire formulation: questionnaire should not be packed with narrowly focused questions with exceedingly abstract and/or ambiguous words (like happiness or trust) which coincide with prevalent western universalist (modernist) thinking. One example is to ask respondents: Do you agree with the sentence: On the whole, do you think that people can be trusted? One of the think tanks based in the UK reports that 80 percent of Westerners reply in the positive, probably because the Bible says so. b: Contributions to our knowledge derive from inappropriate statistical data analysis: When we ask questions on values and norms without some domain clues or adjectives attached to such abstract concepts, such as quality of life and then attempt to reveal key dimensions, say by factor analysis, the first dimension to emerge is bound to be the super-factor (Fog 2020), meaning that many questions are closely related to each other so that factor-analyzing them is bound to yield the one key dimension that explains almost everything. Such data analysis does not reveal new knowledge and thus contributes little. (3) Experimental psychology: Those questions experimental psychologists formulate are bound to be complicated because they set up control variables and to identify the differences in outcomes between control groups and non-control groups. Undergraduate students majoring in psychology may be happy to undergo this process. But those respondents on the street or those respondents chosen randomly may not understand much of such questions. The results are either they do not accept answering questions at all (e.g., DK) or they give their answer to hide their thoughts by choosing middle-point answers like 'neither agree nor disagree.' Those academics who use experimental methods must explain in their questionnaire in non-academic language. My paper summarizes the three lessons drawn from the Asia Barometer Survey experiences in parallel to the aforementioned three pitfalls.