Nonexperimental Designs

Nonexperimental Designs Definition

Nonexperimental designs are research methods that lack the hallmark features of experiments, namely manipulation of independent variables and random assignment to conditions. The gold standard for scientific evidence in social psychology is the randomized experiment; however, there are many situations in social psychology in which randomized experiments are not possible or would not be the preferred method for data collection. Many social psychological variables cannot be manipulated, or ethics would keep one from doing so. For example, a researcher cannot randomly assign people to be in a relationship or not or to stay in a relationship for a long versus short period of time. Similarly, research participants cannot be randomly assigned to be male or female, homosexual or heterosexual, or Black or White. Therefore, the impact of important variables such as relationship status, culture, and ethnicity must be studied using nonexperimental designs.

Characteristics of Nonexperimental Designs

Nonexperimental DesignsMany nonexperimental studies address the same types of research questions addressed in experiments. They are aimed at testing whether the variable of interest causes people to react in certain ways to social stimuli. When this is the goal, nonexperimental studies often measure the variable of interest, often by asking people to report their beliefs or perceptions (such as measures of amount of self-confidence, of commitment to one’s relationship, or of identification with one’s ethnic group). Statistical analyses are then used to relate people’s ratings to measures of other variables thought to be influenced by the initial variable. Consider a simple example in which a researcher wants to learn whether being committed to remaining in a romantic relationship leads people to be happier than not being committed to a relationship. This researcher might survey research participants who are in relationships, asking them to report their current level of commitment to the relationship and their current level of general happiness. A typical type of statistical analysis in this case might be to correlate relationship commitment with level of happiness. Because correlation is a common type of analysis in these designs, many people use the term correlational designs when they are actually referring to nonexperimental designs. The term nonexperimental is preferred primarily because the same correlational analyses could be performed on either nonexperimental or experimental data. The status of the study is determined by the research methods, not by the type of statistics used to analyze the data. Yet, the reader should understand that the terms correlational and nonexperimental are often used interchangeably.

If, in the previous example, the data show that people currently committed to their relationships are happier than are people not committed to their relationships, does this mean that being committed to a relationship makes people happier? Maybe, but maybe not. One of the major problems with nonexperimental designs is the result might have occurred in many ways. In this example, it could be that commitment to their current relationships does make people generally happier. However, it could be that people who are generally happier also make more attractive mates. People may flock to those who seem happy (and may want to stay with them), but may shy away from people who seem sullen and unhappy (and may want to leave them). If commitment loves company, being happy may also make people more likely to be committed to a relationship, rather than relationship commitment making people happier. It could also be that a third variable might encourage people to be committed to relationships and might also make people happy. For example, if the research participants are students, it could be that people who are doing well in school are happier than people not doing well in school. It could also be that people who are doing well in school have the time for social activities that draw them closer to their relationship partners. However, if people are doing poorly in school, spending more time outside of class studying to catch up (or the stress of struggling to catch up) may pull them farther away from their relationship partners. Third variables could also be called confounding variables, because they confound the original causal link that is hypothesized to exist between the two variables of interest.

In a nonexperimental study, it can be difficult to tell which of a variety of explanations is the best. Because of this, researchers should include additional study features that help determine which explanations are best supported by the data. For instance, if our relationship researcher is concerned that happiness might lead to relationship commitment rather than commitment leading to happiness, he or she might measure people’s happiness and relationship commitment over time. If it is true that happiness precedes commitment to a relationship, it should be possible to see that happy uncommitted people are more likely to become highly committed than are unhappy uncommitted people. It would also be possible to look at effects of relationship commitment controlling for one’s level of happiness before committing to the relationship. That is, even if happier people want more to stay with their partners, it could be that commitment to the relationship provides an additional boost to happiness beyond the original level of happiness. Measuring the variables over time does not always identify the ordering of the variables in their causal chains, but it can help.

Measuring possible third (confounding) variables can also help in identifying the most likely causal relations among the variables. When these third (confounding) variables are measured, specific alternative explanations can be tested. For example, if a researcher is concerned that class performance influences both the likelihood of relationship commitment and overall happiness, then a measure of class performance can be used to predict both of these variables. If class performance fails to predict one of the original variables, then it can be rejected as an explanation for the original relation between the two. Even if good class performance was correlated with relationship commitment and with increased happiness, analyses could be conducted using both relationship commitment and class performance to predict happiness. If commitment predicted happiness beyond class performance, this would undermine any concerns about class performance providing the best explanation for a relation between commitment and happiness.

Nonexperimental research can be conducted in laboratories or in naturalistic settings. In general, it might be more likely to see nonexperimental designs when research is conducted in natural (field) settings because the natural settings themselves might make it difficult or impossible to randomly assign people to conditions or to manipulate variables, even though one might still observe or measure the variables in that setting. Yet, it is important to realize that the distinction between experimental and nonexperimental research is not the same as the distinction between lab and field research. Either the laboratory or the field may serve as settings in which to conduct nonexperimental or experimental research.

It is equally important to realize that nonexperimental research includes a wide variety of research methods. Research questions similar to those described earlier (i.e., research aimed at addressing causal relations among variables) can use procedures other than asking research participants to directly report their beliefs or perceptions. For example, researchers might use archival data or direct observation to categorize a research participant’s gender, ethnicity, or occupation. If so, the researcher might treat differences between these known groups as reflecting effects of the variables thought to differ between the groups. The problem, of course, is that these known groups might differ in many ways. Therefore, there are many potential third (confounding) variables to consider and possibly to test. Because of this, even if known groups are identified, the study should include direct measurement of the variables thought to differ across the groups to account for the effects of the third (confounding) variables.

Although many nonexperimental studies ask the type of causal questions described earlier, there are also other kinds of research questions. Some research asks whether a set of measures all tap one underlying psychological dimension. Correlational analyses of this type are used to create many of the multi-item scales that are used across areas of psychology. For example, if a researcher wants to create a multi-item measure of political affiliation, research participants might be asked to respond to a large set of measures asking about their liking or disliking of political figures, about political behaviors in which they have engaged, and about social policies they support or oppose. When determining which of these measures best fit together to assess overall political preferences, the research question is not which variables cause the others but instead what the best set of measures is to assess a person’s political preferences. Other research questions might simply address which variables are correlated with which other variables or might attempt to identify what a certain group of people does or thinks about a certain issue. These research questions are both nonexperimental and noncausal, though forms of these studies can also be the building blocks for creating hypotheses about the causal relations among the variables of interest.

References:

  1. Pelham, B. W., & Blanton, H. (2003). Conducting research in psychology: Measuring the weight of smoke (2nd ed.). Toronto: Thompson/Wadsworth.
  2. Wegener, D. T., & Fabrigar, L. R. (2000). Analysis and design for nonexperimental data: Addressing causal and noncausal hypotheses. In H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social and personality psychology (pp. 412—150). New York: Cambridge University Press.

Read More: