Longitudinal Research

Longitudinal research involves the collection and analysis of comparable data at more than one point in time. The exact number of time points and the interval between time points depend on the nature of the investigation. A longitudinal design is desirable for many research investigations because it enables an assessment of change over time. The history of research methods in industrial/organizational psychology shows the increasing importance of longitudinal designs (although they are still relatively rare) and advances in associated analytical methods. Experience sampling is one method of longitudinal research in which people are asked to report on particular aspects of their everyday experiences at frequent intervals (which may be more than once a day). This form of longitudinal research has become more commonplace in industrial/organizational psychology in recent years because of its potential for capturing the dynamic nature of work experiences.

Longitudinal Research

The defining characteristic of longitudinal research is that it examines how data change over time. This contrasts with cross-sectional research, in which data are collected and analyzed at only one point in time in order to provide a single snapshot of the variables being studied. Research in which cross-sectional data collection is repeated over time—for example, when a survey is readministered in an organization—is considered longitudinal only if the analysis compares the data from different time points (rather than examining the data from each time point in isolation).

Data can be collected over time from the same sample or from different samples in longitudinal research. Retaining the same sample can be difficult from a practical viewpoint, but it allows the researcher to identify how individuals change over time and eliminates the possibility that identified changes are attributable to differences between samples rather than changes over time. Longitudinal data can be quantitative, qualitative, or both. There are many methods for collecting longitudinal data, but the measures used should be consistent over time so that identified changes cannot be attributed to changes in instrumentation. There are a variety of techniques for analyzing longitudinal data, including time-series analysis, longitudinal multilevel modeling, and latent variable modeling. Usually, studies that sample a large number of equally spaced time points (time-series analysis) offer the greatest potential for identifying the causes and nature of effects.

Types of Longitudinal Research

Five main types of longitudinal research study can be identified:

  1. A trend study compares different groups from the same population at different points in time. It can provide information about net change but not individual change.
  2. A panel study tracks the same individuals at fixed intervals over a period of time, which often covers many years.
  3. A cohort study follows a sample of individuals from a particular group (e.g., an age or an employment group) at fixed intervals over a period of time. It is usually a form of panel study, but it becomes a trend study if a new sample is observed at each time point. A number of cohorts can be simultaneously observed and compared over time, which can be useful for separating the effects of maturational and external influences. For example, comparing different age cohorts measured at the same point in time and the same age cohorts measured at different points in time can help distinguish changes that normally occur as people get older from changes resulting from shifts in employment. If a cohort has received a treatment or intervention, the study is also a quasi-experiment because the individuals have not been assigned randomly to the groups being compared.
  4. An intervention study—which is also a form of quasi-experiment—examines the effects of a naturally occurring intervention (e.g., occupational retirement) or an imposed intervention (e.g., a stress-prevention program) by comparing data collected from a group on one or more occasions, both before and after it receives the intervention. Collecting data at the same times from a comparison or control group that has not received the intervention can help to establish whether the changes resulted from the intervention.
  5. An experience sampling study obtains reports from the same individuals on many occasions at frequent intervals. It is distinguished by its focus on individuals’ immediate experiences in their natural environment and its high frequency of reporting.

Retrospective studies, in which participants recall aspects of their life histories or recent past, are sometimes also labeled longitudinal, but this is not strictly accurate because, although they cover periods of time, they do not usually involve real-time sampling at more than one time point. In genuinely longitudinal retrospective studies, participants report at consecutive time points on their experiences since the last time point. Prospective studies are often preferred, however, because they are less susceptible to the problem of memory recall.

Advantages and Disadvantages

Longitudinal research is advantageous in situations in which the investigator wishes to (a) focus on change or patterns of change; (b) measure the duration of an effect; (c) investigate causal processes, including sleeper or delayed effects; (d) separate maturational and external influences; or (e) establish the effects of an intervention.

However, a number of disadvantages are also associated with longitudinal research. First, it can be difficult to retain participants between time points— for example, they may leave their job or no longer wish to participate. Sample sizes, therefore, commonly become smaller over time, and changes in the findings may actually reflect this attrition (or mortality) rather than true change if individuals drop out on a systematic basis. Second, there is the potential for reactivity—that is, responses may be influenced by previous participation, or respondents may actually change themselves as a consequence of participation. Third, longitudinal research has practical disadvantages in that it takes more time and resources to complete. Finally, although longitudinal research goes some way toward addressing the fact that psychological processes unfold in time, it still relies on the selection of discrete time points; therefore, what occurs between those time points must be inferred. Experience sampling is less prone to this problem because it uses frequent time points.

Experience Sampling

The term experience sampling method (ESM) was coined during the late 1970s by Mihalyi Csikszentmihalyi to describe the technique of obtaining subjective reports of people’s current experiences by signaling them at random times in their natural environment (using beepers, for example). Now, the term is used more broadly to refer to any research method that involves collecting subjective reports of episodic experiences from people on frequent occasions during their everyday life. This technique is also known by other names, including diary methods, time sampling, and ecological momentary assessment.

The interval between sampling points in ESM varies from study to study, but the majority use either a number of sampling points within each day or a once-daily schedule. In designing an ESM study, researchers must strike a balance between sampling frequency, duration of study, and time needed to complete measures so as not to place excessive demands on participants. The intense nature of experience sampling means that studies usually involve a smaller sample size and a smaller number of measures than cross-sectional survey studies, but the number of time points contributes to the method’s statistical power and reliability.

Experience sampling uses many means to signal participants to respond (pagers, telephone calls, e-mail messages, and palmtop computer alarms) and to collect data (paper diaries, telephone interviews, online surveys, and palmtop computer programs). Computerized devices have certain advantages for signaling and collection—for example, they are able to record the exact times when participants respond and provide greater efficiency in data processing. A variety of measurement formats can be used in ESM studies to record the thoughts, activities, and feelings of respondents. Analysis of ESM data is complicated by the fact that observations are not independent.

Types of Experience Sampling Study

There are three main types of experience sampling studies:

  1. In signal-contingent studies, respondents report on their current experiences whenever they are prompted by a signal sent at quasi-random intervals. This procedure minimizes memory recall problems but may miss important low-frequency events. This type of study corresponds to the original intent of ESM.
  2. In interval-contingent studies, a signal is sent at regular, fixed intervals, and respondents report on their experiences since the last signal. This procedure covers all events but relies on respondents’ memory of them.
  3. In event-contingent studies, respondents report on their experiences whenever a prespecified event occurs (e.g., a work meeting, a conflict). This procedure investigates selected events in detail but is less amenable to time-series analysis.

Advantages and Disadvantages

Experience sampling offers four distinct advantages for research. First, the method enables an in-depth study of everyday experiences in a natural setting, which gives it high ecological validity. Second, experience sampling is suitable for examining how different kinds of individuals react to different kinds of everyday events and situations. Third, experience sampling allows researchers to study within-person processes and the temporal nature of their experiences (for example, how long it takes people to recover from negative events at work). Finally, the use of real-time assessment in experience sampling reduces the problem of memory biases, which are inherent in global retrospective reports.

Yet, like all research methods, experience sampling also has a number of disadvantages. The high frequency of reporting required of participants means that the method can cause a selection bias in the study sample (for example, an investigation of time demands at work will be compromised if only people with an abundance of time volunteer to participate). For the same reason, the method is also intrusive, heightens the risk of reactivity and habituation in responses, and is prone to missing data. Despite these problems, experience sampling is becoming an established method in industrial/organizational psychology because of its capacity to address temporal variations in the psychological experience of work.

References:

  1. Alliger, G. M., & Williams, K. J. (1993). Using signal-contingent experience sampling methodology to study work in the field: A discussion and illustration examining task perceptions and mood. Personnel Psychology, 46, 525-549.
  2. Bolger, N., Davis, A., & Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579-616.
  3. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis for field settings. Chicago: Rand McNally.
  4. Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175, 526-536.
  5. Menard, S. (2002). Longitudinal research (2nd ed.). London: Sage.
  6. Rogelberg, S. G. (2002). Handbook of research methods in industrial and organizational psychology. Oxford, UK: Blackwell.
  7. Scollon, C. N., Kim-Prieto, C., & Diener, E. (2003). Experience sampling: Promises and pitfalls, strengths and weaknesses. Journal of Happiness Studies, 4, 5-34.