Generalizability

Experiments in human development attempt to identify the influence of variables or events on human behavior. These experiments contribute to the shared knowledge of the community that seeks to understand human development by describing the course of human development, predicting how individual humans will develop,  and  arranging  optimal  environments  for growth. Before experiments can contribute to an understanding of human development, they must fulfill two criteria: accuracy and generalizability. How accurate the results of an experiment are can be determined by asking, “How well did the experimenters rule out other possible explanations for the observed results?”  If  it  can  be  reasonably  assured  that  the results are accurate for a given experiment, we may then begin to ask questions of generalizability.

When an experiment is conducted, it would be impossible (in terms of time, effort, and finances) to measure all of the important behavior of all the world’s humans. Instead, researchers observe some important behavior of a smaller group of participants, called a sample, in a limited number of settings. Generalizability is the extent to which outcomes of observations with a sample represent the performance of the larger population under similar conditions. Typically, consumers of research are concerned with generalizability across different groups, but they may also be interested in generalizability across settings and behaviors.

If a large group of infants was observed to all begin walking by about 10 months of age, would it be appropriate to conclude that all infants learn to walk by 10 months? If mothers reacted with joy and glee each time their babies smiled in a laboratory setting, and the babies smiling increased dramatically when this type of parental response was consistent, would it be appropriate to conclude that all babies learn to smile through social interaction with mothers? If babies do learn to smile from their mother’s approval,  would it be safe assume that they also learn other behaviors from their  mothers’ approval, such as crawling or walking?

These are questions relevant to the generalizability of results. They are important questions because it is crucial to be able to apply what is learned under one set of conditions in the past to many of the important situations occurring in the present. Nevertheless, the tentative nature of science suggests that the answer to all of the questions above is “No.” Infants raised in cultures other than those represented in the original study may not learn to walk until they reach 12 months. Mothers may respond to smiling in different ways outside of the laboratory setting, and other important infant behaviors may develop independent of parental social interaction.

Increased generalizability can be obtained in one of two ways. The first involves replicating studies (i.e., doing them again) with different participants, in different settings, or while measuring different behaviors. This method is direct in that similar analyses are replicated across relevant new dimensions. The second involves including a large and diverse sample of participants, settings, or behaviors into the initial experiment. Generalizations of findings from samples to populations are made with the most confidence when the critical dimensions of the population are equally represented in the sample; this is usually achieved through systematic sampling and assignment techniques. Ultimately, generalizability, the successful extension of findings to people and situations beyond those of the original experiment, is achieved by assessing the impact of the same or similar variables on an ever-widening subset of participants, behaviors, and settings.

References:

  1. American Psychological Association, http://www.apa.org
  2. Kirk, R. E. (1995). Experimental design: Procedures for behavioral sciences (3rd ). Pacific Grove, CA: Brooks/Cole.
  3. Sidman, M. (1960). Tactics of scientific research: Evaluating experimental  data  in  psychology.  Boston:  Authors Cooperativ