Multilevel Modeling

As scholars of human behavior in organizations, industrial and organizational psychologists often find themselves trying to understand phenomena that are inherently nested, hierarchical, and multilevel. From private industry to universities to the military to nearly all forms of government, organizations comprise hierarchical structures that loosely resemble pyramids (even if the structures resemble flat pyramids, they are still hierarchical).

Consider a large chain of department stores: individual employees are nested within different departments (e.g., sporting goods, men’s clothing, women’s clothing), which are, in turn, nested within a store in a particular location, which is, in turn, nested within the overall organization. In this example, influences and consequences occur at the individual, department, store, and organizational levels of analysis, including relationships that cross these levels. Furthermore, the people within these units (department, store, organization) are not there randomly; rather, they share some similarities that make them distinct from other units. There are many consequences of this natural nesting, which will be described shortly. But first, let us consider the consequences of ignoring hierarchical structures in organizational research.

Consequences of Ignoring Multilevel Influences

A number of theoretical and methodological fallacies may occur as a result of ignoring multilevel structures. This entry focuses on the following theoretical fallacies:

  • Misspecification fallacies involve assigning the wrong level of theory to a construct. For example, a researcher may wish to study organizational flexibility, but using only measures of individual employee flexibility does not operationalize the construct at the appropriate (organizational) level.
  • Cross-level fallacies involve inappropriately equating findings at one level to other levels. If a researcher adopts a theory of individual flexibility to understand organizational-level flexibility, he or she must show how the theory adequately explains phenomena at the organizational level or risk committing a cross-level fallacy.
  • Contextual fallacies involve instances in which a researcher ignores important contextual (higher-level) influences on lower-level relationships and outcomes. This is a big one in psychology: For years, the field studied individuals as if situations did not matter. For example, if organizational flexibility constrains or enhances individual-level flexibility, then both levels should be studied to truly understand how individual flexibility relates to individual-level outcomes.

The bottom line is that ignoring multilevel relationships when they exist may lead us to inappropriate conclusions from our research. This is not a trivial issue because the validity of our science depends on adequately understanding these relationships.

Properties of Emergence

Measuring individual-level cognitions, attitudes, and behaviors is relatively straightforward. But how do such constructs manifest themselves in higher-level units such as teams and organizations? Sometimes, higher-level constructs are easy to measure (e.g., organizational size, location, or money spent on human resources). But more often then not, researchers want to understand psychological phenomena that exist at higher levels (e.g., climate, customer satisfaction, or turnover). Perhaps one of the most important theoretical advancements in recent years has been the clarification of the process of emergence—that is, how lower-level psychological constructs manifest as higher-level constructs. There are numerous ways this can occur, but for simplicity’s sake, two extreme forms will be discussed.

First, composition represents similarity, consensus, or “sharedness” among within-unit observations. For example, if lower-level observations are hypothesized to be sufficiently similar to form some aggregate, higher-level construct, one is dealing with a composition model. In such a model, the higher-level construct is the unit average of within-unit members’ scores. For example, organizational climate may be the average of all employees’ climate perceptions within that organization. Notice what just happened: The mean scores within the organization were used to create an organization-level score based on that mean. Hence, there is no direct measure of organizational climate in this case but instead an indirect measure based on an average of employees’ scores. Beyond having a strong theory to justify aggregation, one must statistically demonstrate it through indexes of agreement and consensus. A composition model, by focusing on similarity, frequently makes a claim that the lower- and higher-level constructs are reasonably isomorphic (i.e., similar in their nature).

Second, compilation represents dissimilarity, dissensus, or, simply put, within-unit disagreement. If it is hypothesized that the unit-level score is based on differences among unit members, then one need not justify similarity to create the aggregate-level variable. Instead, one directly estimates the amount of within-unit disagreement. For example, climate strength is a variable that is hypothesized to explain how strongly climate perceptions are held. It has been operationalized as the within-unit standard deviation in climate perceptions. The higher the within-unit standard deviation, the more disagreement there is among unit members. There are other ways of operationalizing the unit-level compilation model, but the within-unit standard deviation appears to be the most popular.

Thus, composition and compilation processes represent two forms of emergence. They are not simply different sides of the same construct; with a perfectly normal distribution, means and variances are uncorrelated. In practice, they are often correlated with each other, and the researcher must clearly justify why one or both forms of emergence are relevant in a given setting.

Types of Multilevel Relationships

At the risk of oversimplifying a detailed topic, multilevel relationships can be categorized into three types. The first relationship is single level, wherein all predictors and criteria reside within a single level of analysis. This is the dominant approach in psychology, particularly industrial psychology. The second relationship is cross-level, wherein the predictors exist at multiple levels, but the criteria exist at a single level (alternatively, the predictors exist at a single level, and the criteria exist at multiple levels). For example, one might hypothesize that organizational climate moderates the relationship between individual service provider attitudes and individual customer’s satisfaction. The final relationship is homologous, meaning that the same set of relationships is found at each level of analysis. For example, individual service provider attitudes may influence individual customer satisfaction, and aggregate organizational service climate influences aggregate customer satisfaction. A variety of advanced statistical techniques are capable of modeling the multilevel relationships described here, but the real difficulty is articulating a theory that makes such statistics meaningful.

Summary

Industrial and organizational research continues to shift from single-level questions to multilevel questions. This research is challenging yet ultimately necessary if we are to truly understand how the behavior of the people within organizations contributes to the behavior of organizations. Many advancements have already been made, and this is likely to be an active area of research for the next several years.

References:

  1. Bliese, P. D. (2000). Within-group agreement, non-independence, and reliability: Implications for data aggregation and analysis. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 349-381). San Francisco: Jossey-Bass.
  2. Chen, G., Mathieu, J. E., & Bliese, P. D. (2004). A framework for conducting multilevel construct validation. In F. J. Dansereau & F. Yamarino (Eds.), Research in multilevel issues: The many faces of multi-level issues (Vol. 3, pp. 273-303). Oxford, UK: Elsevier Science.
  3. Hofmann, D. A., Griffin, M. A., & Gavin, M. B. (2000). The application of hierarchical linear modeling to organizational research. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 467-511). San Francisco: Jossey-Bass.
  4. Klein, K. J., & Kozlowski, S. W. J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey-Bass.
  5. Ployhart, R. E. (2004). Organizational staffing: A multilevel review, synthesis, and model. In J. Martocchio (Ed.), Research in personnel and human resource management (Vol. 23, pp. 121-176). Oxford, UK: Elsevier.
  6. Von Bertalanffy, L. (1972). The history and status of general systems theory. In G. J. Klir (Ed.), Trends in general systems theory (pp. 21-41). New York: Wiley.