Cross-Cultural Research

Research in industrial-organizational psychology is increasingly being conducted across cultural boundaries to test the generalizability of Western findings and to train managers to be more effective in multicultural contexts. Although cross-cultural research involves many of the same methods that are used in typical I/O research, many unique issues arise in the cross-cultural research process—from the level of theory to the sampling of people, constructs, and  Cross-Cultural Research Methods and methods to the analysis and interpretation of data— each of which will be discussed here.

Developing Cross-Cultural Research Questions: Levels of Analysis

Cross-cultural research is inherently multileveled, and the first step in any cross-cultural research project is to decide on the level of analysis that is inherent in the research question. By way of illustration, Figure 1 shows the levels of analysis that might be involved in studies of societal culture. Societal culture reflects variation across societies in values, norms, beliefs, and assumptions, among other elements. Linkages A and B in Figure 1 represent single-level models, which examine the macro antecedents and consequences of national culture. Linkages C through E represent cross-level direct effect models, which examine the direct effect of societal culture on organizations and individuals. Linkages F through I represent cross-level moderator effect models, which examine how societal culture moderates relationships at the organizational and individual levels. Later, we will give illustrations of cross-cultural research questions that span these multiple levels of analysis. (Note: The dashed arrows in the figure are not discussed for reasons of space.)

Macro Antecedents and Consequences of Societal Culture

Linkages A and B represent research on the antecedents and consequences of societal culture at the macro level. Cross-cultural psychologists have long argued that societal cultures develop through adaptations to the ecological, historical, and sociopolitical context. Accordingly, one might examine how factors such as temperature, natural resources, population density, economic structure, or a history of conflict between nations affect societal values, norms, or beliefs. Research may also focus on culture’s consequences at the societal level, for example, examining how societal values, norms, or beliefs affect societal crime rates, conformity, or innovation.

Cross-Level Direct Effects of Societal Culture

Research questions may be concerned with the cross-level direct effects of societal culture on organizations and individuals. Scholars in I/O psychology recognize that organizations are open systems that reflect and adapt to the larger societal context. This is reflected in Linkage C, or cross-level research that examines the influence of societal culture on organizational culture. Other research at this level of analysis might examine how societal culture affects the human resource practices—such as selection and job analysis techniques, performance appraisal methods, and training methods—that are implemented in organizations. Research might also examine the indirect effect of societal culture on organizational outcomes as mediated by cross-cultural differences in organizational culture and practices. Finally, Linkage D in Figure 1 represents research that examines how societal culture directly affects the institutional context of organizations. For example, research might examine whether the prevalence of certain industries or ownership structures (e.g., private versus public) varies across societies.

Although Linkages C and D represent societal cross-level effects on organizations, Linkage E represents research that examines how societal culture affects individual-level phenomena, such as cognitions, motives, or emotions. Research in I/O psychology, for example, has examined how societal culture affects employees’ achievement motivation, self-efficacy, job attitudes, and perceptions of effective leadership. Alternatively, one might be interested in the indirect effect of societal culture on individual behavior as mediated by individual perceptions, motives, or emotions. For example, research might examine whether there are cross-cultural differences in cooperative behavior in negotiations, teams, or organizations (e.g., organizational citizenship behaviors) as mediated by cross-cultural differences in individuals’ perceptions, attitudes, or motives.

Cross-Level Moderator Effects of Societal Culture

Cross-cultural research questions focus on how societal culture moderates relationships at lower levels of analysis. Linkage F represents cross-level research that examines how societal culture interacts with features of the organizational context (e.g., industry, technology) to predict organizational culture and practices. For example, one might be interested in whether organizational cultures are more similar across societies within some industries (e.g., manufacturing) as compared with others (e.g., services). Linkage G illustrates that the relationship between organizational culture and practices and organizational outcomes can be moderated by societal culture. Research might examine whether the success of implementing different practices (e.g., innovation versus quality control) varies across societies. Alternatively, research might address whether diversity in organizations is beneficial for organizational performance and how this relationship varies across societies.

Figure 1 illustrates the moderating effect of societal culture on relationships at lower levels. For example, Linkage H represents research on the way societal culture moderates the impact of organizational practices on individual cognitions, motives, or emotions. Research might address, for example, whether giving voice (an organizational justice practice) has similar effects on satisfaction in different societal cultures or whether working in teams similarly affects motivation across cultures. Finally, Linkage I illustrates that societal culture moderates the relationship between psychological states and behavior. For example, research might examine whether societal culture moderates the strength of attitudes as a predictor of behavior or whether needs (e.g., the need for closure) differentially affect behavior across cultures.

Figure 1 illustrates the complexity of levels of analysis in cross-cultural research. Accordingly, before conducting cross-cultural research, it is important to specify the level at which the theory is operating and implement a design and analysis strategy that matches the level of theory. Figure 1 need not represent all possible multilevel linkages that pertain to societal culture. For example, research might focus on how societal culture affects variance at multiple levels of analysis rather than just mean differences. Figure 1 also does not include indigenous research questions, which are interested in examining emic (culture-specific) phenomena that may not be applicable beyond a certain cultural context.

Sampling Participants

After deciding on a research question and determining the appropriate level of theory, the next step is to determine which cultures, organizations, or individuals to include in the research. As in unicultural research, these decisions should ideally be guided by theory. For example, if the researcher is testing a theory that relates one or more dimensions of culture to organizational phenomena, cultures should be sampled so that there is wide variation along those dimensions. Quantitative and qualitative data sets on cross-cultural differences abound and can be consulted when making sampling decisions. Conducting cross-cultural organizational research is further complicated by the need to consider the similarity of organizations and individuals within organizations across cultures. If a sample contains individuals from industry A in one culture but individuals from industry B in a second culture, culture is confounded with industry, thus creating an alternative explanation for observed cultural differences. Matching industry, as well as individual characteristics such as job level or other demographics, across cultures can reduce alternative explanations for observed cross-cultural variation.

Assessing the Constructs of Interest

After deciding on a sampling strategy, researchers need to consider how they will assess the constructs of interest. Typically, researchers rely on what J. W. Berry calls imposed etic constructs, or constructs that are developed in one culture and simply applied to another culture. Ideally, researchers should consider whether there is construct contamination and construct deficiency to avoid comparing apples to oranges in cross-cultural studies. Consider a construct that is developed in culture A and then applied to culture B. It is possible that the construct will be contaminated with aspects of culture A that are not meaningful in culture B. To assess the possibility of construct contamination, researchers might use factor analytic techniques to determine whether the factor structures are invariant across cultures. Even if the construct is not contaminated, however, it may possess construct deficiency in culture B if there are culture-specific (emic) aspects of culture B that are not encompassed by the etic construct. Pilot studies and feedback from local collaborators are critical to identifying missing elements of constructs. An alternative approach is to develop an emic instrument without regard to other cultures and then compare results using the local instrument with results using instruments developed elsewhere.

Sampling Methods

A wide range of methods are available for cross-cultural research, and each has particular advantages and disadvantages.

• Laboratory experiments provide a controlled research environment and facilitate tests of causal assumptions. Laboratory research is also beneficial because it enables the researcher to assess implicit attitudes in addition to explicit self-reported attitudes. Yet it is critical to ensure that laboratory tasks and procedures are equally understood and motivating to individuals across different cultures. Extensive pilot testing, coupled with detailed feedback from local collaborators, can help to ensure that the laboratory situation is equivalent across cultures.
• Questionnaires may be less intrusive than laboratory experiments, and they provide the ability to collect data on a wide range of questions at any one time. Cross-cultural challenges to surveys abound, however, including potential differences in motivation, understanding of instructions, validity, reliability, and response sets. Pilot testing, discussions with local collaborators, and statistical controls are critical to ensuring equivalence.
• Interviews and focus groups enable researchers to gain depth in a research question and afford more of an understanding of emic perspectives that are especially useful in the early stages of cross-cultural research. Difficulties may arise, however, in standardizing interviews across cultures, and there may be differential reactions to the interviewer across cultures (e.g., women interviewing men may be viewed as inappropriate in some cultures) that can threaten the validity of the results. Interviewers should possess characteristics similar to those of the participants, and cross-validating interview findings with additional methods is critical for gaining confidence about the results.
• Cultural documents such as newspapers, proverbs, or speeches provide useful and unobtrusive cross-cultural data that can be content analyzed. Researchers should involve local collaborators to identify documents that are most relevant to the research question, to ensure the comparability of documents across cultures, and to develop detailed coding schemes that are reliable and valid across cultures.
• Observations of behavior can provide an unobtrusive method for collecting cross-cultural data. Researchers, however, should confer with local collaborators to ensure that the observed situation has equivalent meaning across cultures. Furthermore, as with cultural documents, detailed coding schemes and hypothesis-blind coders are necessary to produce reliable data.
• Large archival databases provide another unobtrusive source of cross-cultural data. Examples include ethnographies, which provide in-depth information about a given culture, and cross-cultural databases on ecological, sociological, economic, or political variables. Preexisting databases, however, may only be available for a limited number of countries or variables. Furthermore, at times, the researcher may not be sure of the extent to which the data set is characterized by errors or biases, making the use of multiple sources critical to establishing validity.

In sum, many methods for cross-cultural research exist, and all have distinct advantages and disadvantages. Moreover, each method varies in terms of how appropriate or ethical it is in a particular cultural context, how much depth it affords, and ultimately, its validity and reliability across cultures. In light of these complexities, feedback from local collaborators, as well as the triangulating across multiple methods, is critical in cross-cultural research.

Translations

The most commonly employed cross-cultural research methodologies often require translating materials into other languages. Translation, however, may result in variations in meaning across cultures. The translation-back translation technique is often used in cross-cultural research to examine problems with translations. In this method, an individual translates materials from the original language to a second language. A second individual then retranslates the material back into the original language, and the two versions are compared to reveal any meaning that was lost in translation. Although frequently used, the translation-back-translation methodology may result in awkward translations. Another option is to use the “decentering”

approach. In decentering, changes are made simultaneously to the original and the translated version to improve the quality of the materials in both languages.

Assessing Rival Hypotheses

When conducting research across cultures, extraneous variables that may influence the results should be measured and controlled. R. S. Malpass referred to these variables as rival hypotheses and noted that they are often unaccounted for in cross-cultural research. For example, asking people about their own opinions may be more commonplace in some societies than others, or people may be differentially motivated to please experimenters across societies. Cultural differences in perceptions of the method, task, or instructions can also affect results. In his studies of Kpelle farmers in Liberia, Joseph Glick showed that participants were able to sort objects in the “Western intelligent” way (i.e., by taxonomic category) compared to the “superficial” way (i.e., by color), but only when they were told to sort the objects in the “stupid” way! Pilot tests and discussion with local collaborators are critical to identifying which rival hypotheses need to be measured or controlled.

Analyses and Interpretations

Unique issues arise in the analysis and interpretation phases of cross-cultural research. Fons Van de Vijver and Kwok Leung addressed such issues by establishing equivalence and dealing with response sets. Cultural response sets, or systematic tendencies in the use of response scales, can pose a rival hypothesis for substantive results. In some societies, individuals may avoid the extremes of the scales, whereas in others, individuals may be more likely to show agreement. Additionally, people in different societies may be thinking of different reference groups when answering questions. Van de Vijver and Leung, as well as Kaiping Peng and colleagues, discuss a number of techniques to deal with such issues, including the use of different response formats, standardization techniques, or structural equation modeling to correct for response biases. Finally, when testing hypotheses, it is critical to align the level of theory with the level of data analysis and to avoid making ecological fallacies, wherein a phenomenon at a higher level of analysis is attributed to a lower level.

Summary

Cross-cultural research adds a number of complexities to the research process that require additional informed judgment calls at all research stages. Although the task appears formidable, using multiple methods, extensive piloting, and feedback from collaborators can greatly increase confidence in the findings.

References:

1. Berry, J. W. (1969). On cross-cultural compatibility. International Journal of Psychology, 4, 119-128.
2. Brett, J. M., Tinsley, C. H., Janssens, J., Barsness, Z. I., & Lytle, A. L. (1997). New approaches to the study of culture in industrial/organizational psychology. In P. C. Earley & M. Erez (Eds.), New perspectives on international industrial/organizational psychology (pp. 75-125). San Francisco: New Lexington Press.
3. Gelfand, M. G., Raver, J. L., & Ehrhart, K. H. (2002). Methodological issues in cross-cultural organizational research. In S. G. Rogelberg, Handbook of research methods in industrial and organizational psychology (pp. 216-246). Malden, MA: Blackwell.
4. Glick, J. (1970, November 19). Concept formation: Kpelle and American style. Paper delivered at the Annual Meeting of the American Anthropological Association, San Diego, CA.
5. Hofstede, G. H. (1980). Culture’s consequences: International differences in work-related values. Beverly Hills, CA: Sage.
6. Malpass, R. S. (1977). Theory and method in cross-cultural psychology. American Psychologist, 32, 1069-1079.
7. Peng, K., Nisbett, R., & Wong, N. (1997) Validity problems comparing values across cultures and possible solutions. Psychological Methods, 2, 329-344.
8. Schwartz, S. H. (1994). Beyond individualism/collectivism: New cultural dimensions of values. In U. Kim, H. C. Triandis, C. Kagitcibasi, S. C. Choi, & G. Yoon (Eds.), Individualism and collectivism: Theory, method, and applications (pp. 85-122). Thousand Oaks, CA: Sage.
9. Sinha, D. (1997). Indigenizing psychology. In J. W. Berry, Y. H. Poortinga, & J. Pandey (Eds.), Handbook of cross-cultural psychology: Vol. 1. Theory and method (2nd ed., pp. 129-169), Needham Heights, MA: Allyn & Bacon.
10. Triandis, H. C., & Berry, J. W. (Eds.). (1980). Handbook of cross-cultural psychology. Boston: Allyn & Bacon.
11. Van de Vijver, F., & Leung, K. (1997b). Methods and data analysis for cross-cultural research. Thousand Oaks, CA: Sage.