Outcomes of Counseling

Counseling and psychotherapy outcomes are the benefits (or harms) that derive for clients (patients or mental healthcare consumers) as a result of their experiences or treatment in therapy or counseling. Although it is generally assumed that therapy is an effective treatment for mental health concerns, this is not always the case. The determination of therapy outcomes involves a variety of issues and considerations.

How Are Counseling and Psychotherapy Outcomes Evaluated?

Efficacy and effectiveness are two ways in which the outcomes of counseling and psychotherapy are discussed. Efficacy refers to the therapeutic benefits found in comparison of the treatment and a no-treatment control group within the context of a controlled clinical study. In contrast, effectiveness refers to the benefits of therapy that occur within a mental health practice context. In the former instance, the question is whether a treatment or intervention is found to achieve a greater benefit for clients than no treatment. If so, the treatment is said to be “efficacious.” In the latter instance, the question is one of how effective counseling is for those clients who seek and receive treatment within the community.

It is alleged that clinical studies create an artificial context in which the therapy that takes place is not characteristic of how treatments are provided in an actual community context. Consequently, finding that a treatment is efficacious cannot be assumed to mean that it is effective (i.e., is beneficial to clients in practice settings). Although there is merit in this criticism, effectiveness findings are generally compromised by the absence of a control group within a practice setting against which to compare client therapeutic gains. As a result, it may not be possible to convincingly determine whether the benefits derived by clients receiving counseling in community settings are due to the treatment or to some other extraneous factors.

Significance of Therapy Outcomes

In the consideration of counseling and psychotherapy outcomes, it is important to ask, “When is an outcome significant?” The significance of therapy outcomes can be evaluated in several ways. Outcomes can be evaluated in terms of their statistical significance, and they can be evaluated in terms of the clinical significance or clinical relevance.

Statistical Significance

There are two types of statistical significance that may be considered when evaluating therapy outcomes. The first has to do with differences between or among treatment groups. The second has to do with the changes experienced by individuals within those groups.

Between-group differences are examined by comparing the outcomes of two different approaches to therapy (e.g., a new approach to therapy vs. an established approach), or by comparing the outcome of a specific therapeutic approach with a placebo treatment or a nontreatment (wait list) group (i.e., a control group). Whatever the comparison, if the research is designed so as to rule out extraneous factors as competing explanations for the change, statistical procedures may be used to determine whether the observed differences that appear between groups can reasonably be attributed to differences in the administered treatments or whether it is more reasonable to conclude that the differences are due to chance (e.g., sampling differences). If the difference between the outcomes of the treatment group and the comparison group is in the expected direction and unlikely to be due to chance sampling differences, then it can be concluded that the difference is statistically significant. In other words, the treatment group was more efficacious and yielded a statistically better outcome than did the comparison group.

Although the treatment outcome of one group may differ significantly from that of another group, this does not necessarily mean that the change that occurred was itself significant. Indeed, it is conceivable that the treatment group did not change at all, but rather the comparison group became significantly worse, relative to the treatment group. In order to evaluate the statistical significance of change within the treatment group (i.e., the statistical significance of its outcome), a different approach is needed. In this approach, a group’s pretreatment performance on some relevant outcome variable is evaluated against its posttreatment performance on the same variable. If the difference between the pre- and posttreatment assessments is in the expected direction and not attributable to chance differences in the measurement of the outcome variable (measurement error), then the change (or outcome) is said to be statistically significant.

Clinical Significance (Clinical Relevance)

While the statistical significance of outcome research findings can provide empirical support for different treatment approaches, it is important to note that just because there are statistically significant differences between groups does not mean that the groups differ in clinically significant ways. That is to say, although the treatment outcome for one group may differ from that of another and be in the desired direction, such a finding may not be clinically meaningful. For example, a treatment for depression might produce therapeutic change in a group of clients that is significantly different statistically from that of a placebo treatment, but this does not necessarily mean that those who received the treatment are no longer depressed. It simply means that as a group, they changed more than those who received a placebo treatment did. Furthermore, a statistically significant within-group pre-post difference does not necessarily mean that the individuals who received the treatment are meaningfully improved. It simply means that their posttreatment scores are reliably different from their pretreatment scores.

Several approaches to the evaluation of clinically relevant change have been proposed. Evidence that treated clients are indistinguishable from a nondisturbed reference group is probably the most convincing evidence of clinically meaningful change. Neil S. Jacobson and Paula Truax extended this notion by proposing a standardized statistical method involving two criteria for assessing clinical significance. First, the treated client should be more likely identifiable as belonging within a distribution of healthy persons than a distribution of troubled individuals. Second, the client change must be reliable; that is, it must be large enough that the pre- to posttreatment change cannot be attributable to measurement error—a criterion for which there is a reliable change index that can be sta-tistically computed. Notwithstanding the above discussion, statistical significance, rather than clinical significance, is the manner in which outcome efficacy is generally reported.

Perspectives on Therapy Outcomes

The determination of the clinical significance of a therapy outcome is invariably related to the perspective of the outcome evaluator. Early efforts at examining therapy outcomes generally relied exclusively on therapists’ impressionistic ratings of client improvement. Over time, outcome evaluation moved from the reporting of therapists’ clinical impressions to the use of standardized therapist rating scales to measure general improvement.

Although the use of such scales constitutes an important improvement to the evaluation of therapy outcomes, the possibility of therapist bias in the provision of ratings is problematic. As an additional source of information, client self-reports of improvement are frequently included in outcome evaluations. Often these consist of global measures of improvement framed as judgments of the benefit or value to therapy or client satisfaction on posttherapy questionnaires. Other client self-report efforts include structured personality measures or symptom rating scales—generally collected pre- and posttreatment. Client self-report measures provide a unique and important source of information and perspective regarding change resulting from therapy that can be especially useful to therapists in gauging their effectiveness in providing treatment. Concerns about objectivity that may result from the use of therapist- and client-rated measures has led to the use of persons not involved in therapy who, as raters, are blind to the treatment status of research participants or clients. It has also led to the use of physiological measures and unobtrusive measures such as student grades, judicial records, sales records, and the like.

Hans H. Strupp and Suzanne W. Hadley proposed a tripartite model of therapeutic outcomes that captured three important, yet distinct, perspectives from which the relevance of therapy outcomes could be evaluated: (a) the perspective of the client as consumer, (b) the perspective of the therapist or mental health professional, and (c) society. In discussing their model, they provided a variety of different therapy outcome evaluation scenarios. For example, it is possible for the client to report improvement, while neither the therapist nor others support that conclusion. Alternatively, the therapist may believe the client has improved, but the client does not. Another possibility is that both the client and therapist may report improvement on the part of the client, but others in the community with whom the client interacts do not see the same improvement. In other words, agreement of these perspectives on the outcome of therapy cannot be assumed. These differences in perceptions of the outcome and therapeutic benefit raise important concerns in terms of the evaluation of therapy outcomes. Suffice it to say, when considering therapy outcomes, it is important to consider that there are multiple legitimate perspectives to consider and often the perspectives are not consistent.

Benjamin M. Ogles, Michael J. Lambert, and Kevin S. Masters incorporated the notion of different legitimate perspectives on therapy outcome into a multidimensional model for considering therapy outcomes. Expanding on earlier perspectives, these researchers include the perspectives of relevant others such as the client’s spouse, friends, and work colleagues, plus those of trained observers and institutional referents (i.e., school, work or hospital records, public records of arrest, etc.). In addition to this social dimension of outcome measurement and evaluation, they also proposed several additional dimensional perspectives: (a) content, (b) social level, (c) technology, and (d) time orientation.

The content dimension refers to the psychological area or aspect of the person on which outcome is being measure and against which efficacy is evaluated. It includes client behavior, cognitions/thoughts, and emotions. The social level dimension refers to the degree to which the outcome construct being measured is intrapsychic or internal to the person or interpersonal and external to the person. Intrapsychic constructs include moods, self-concept, cognition or beliefs, and behavioral deficits. Interpersonal constructs include such things as social and work adjustment, delinquency, and group interaction.

The technology dimension refers to the variability in the types of instruments that are used for the collection of outcome/change data. This dimension captures the fact that different measures and instrumentation may yield differentially favorable or unfavorable outcome findings. These discrepancies in therapy outcome findings may arise depending on whether outcome is assessed globally or specifically, directly or indirectly. Differences in the sensitivity of the measurement technology to change on the construct being assessed may also result in outcome discrepancies. The final dimension, time orientation, reflects the degree to which the outcome assessment attempts to measure stable/enduring, trait-like features of the person rather than features that are unstable and more dependent on the state or contemporary circumstance of the client.

This multidimensional model suggests that therapy outcomes are best understood and most reasonably evaluated over a number of different dimensions and from multiple perspectives. However, in this regard, it is important to note that conclusions about the effects or outcome of counseling and therapy depend on the particular measure that is used in assessing change in therapy and from what perspective(s) that change is evaluated. In outcome assessment, the measurement tools that are used often are connected with particular theoretical orientations or schools of therapy. Notwithstanding the differences among various orientations toward counseling and psychotherapy, clinicians and researchers recognize the need to assess the outcome of therapy on aspects broader than the dimensions or aspects immediately specific to a particular theory or approach to therapy.

Is Therapy Better Than No Therapy?

The fundamental question in terms of therapy outcome is whether clients receiving therapy are, at the conclusion of their experience, better off than those who need therapy but do not receive it. In other words, “Does counseling/psychotherapy work?” This is a question about the absolute efficacy of therapy. As suggested earlier, it is generally assumed that therapy is an effective treatment for mental health concerns; however, it is an assumption that has been and continues to be open to the scrutiny of researchers.

A notable first attempt to examine the evidence relating to the effects of therapy was conducted in 1952 by Hans J. Eysenck. As previously mentioned, the evaluation of the efficacy of therapy requires that the effects of treatment be compared with a no-treatment control group. To conduct his evaluation, Eysenck compared the outcomes found in 24 studies of psychodynamic and eclectic psychotherapy with spontaneous remission rates (i.e., rate of improvement in client functioning without benefit of therapeutic intervention) using two control groups. The results of Eysenck’s study were disconcerting, finding that clients who received psychodynamic or eclectic therapy improved less than those in his control/comparison no-treatment condition. Not only did it appear that therapy was ineffective, it might also be harmful.

Eysenck’s study was not without critiques, as there were serious design problems with his research method. Responding to the challenge to therapy implied by Eysenk’s study, numerous reviews of aggregated efficacy studies of counseling and psychotherapy were conducted during the 1960s and 1970s. While having their own methodological problems, these subsequent studies generally yielded findings supportive of therapy’s efficacy.

Over the years, examinations of the efficacy of counseling and psychotherapy have reached different and even contradictory conclusions. It is noteworthy that these earlier reviews of the outcome literature often lacked objectivity and replicability. They generally involved narrative descriptions of each study included in the review, an evaluation of the results in terms of the type of evidence offered with respect to therapy outcome, and then an implicit summary of the findings to render an overall conclusion about therapy’s effectiveness. However, with the hundreds of outcome studies now available for consideration in evaluating therapy’s effectiveness (outcome), turning the thousands of pieces of evidence that derive from all of these studies into an integrated summary of the benefits of counseling and psychotherapy is problematic.

Although a single outcome study will reveal information about the benefits received by the participants of that study, the answer to the broad question of “Is counseling/psychotherapy effective?” requires the examination of the body of research that has addressed this question. More recent inquiries into therapy efficacy have used the statistical method of meta-analysis to examine the aggregated results of hundreds of different studies that have compared a group receiving counseling or psychotherapy with a control group. Briefly, meta-analysis consists of a set of statistical procedures that allow researchers to gain a comprehensive picture of the research on a question and an unbiased answer to the research question. Through meta-analysis, outcome data from many individual counseling and psychotherapy outcome studies are systematically aggregated, allowing the findings to be analyzed to achieve an answer to the larger question of whether therapy is effective. Unlike the research methods used in individual studies for which the client-participants serve as data points for analysis, meta-analysis uses the summary statistics from individual studies as the data points for analysis. Although not without detractors, meta-analytic procedures provide a methodology to assemble an overall picture of therapy’s effectiveness (relative to no therapy or a placebo treatment) and for comparing across studies of different approaches to therapy in order to investigate the relative efficacy of different treatments.

The first meta-analysis of the outcome of psychotherapy was conducted by Mary Lee Smith and Gene V. Glass. They analyzed the results of 375 published and unpublished therapy outcome studies. The results of their study produced an effect size of .68, which suggests that an average client receiving therapy would be better off (i.e., improved) than 75% of untreated (control group) clients. Although their results suggest that a proportion (34%) of untreated clients also improved (i.e., spontaneous remission), the success rate for those receiving treatment was 66%, leading them to conclude that the research showed the beneficial effects of counseling.

As with the challenges to Eysenck’s methodology and findings, there have been critics of and challenges to Smith and Glass’s meta-analytic findings. Subsequent meta-analyses of the therapy outcome literature have challenged the validity of those criticisms, while at the same time providing rather convincing support for the absolute efficacy of counseling and psychotherapy. Although it cannot be said that therapy is effective for everyone who seeks it, the likelihood of improvement is high for those in therapy, and much higher than for those left untreated.

Negative Outcomes in Therapy

Research does support a conclusion that there are negative outcomes in therapy; that is, some clients do get worse. This research suggests that a relatively consistent 5% to 10% of clients deteriorate while participating in therapy. However, this does not mean that all instances of worsening that occur during counseling/therapy are the result of the treatment. Some clients may be on a progressive decline that a therapist is unable to stop. There are a number of factors related to the client, therapist, and treatment that are associated with negative outcomes during counseling/psychotherapy. Negative outcomes are more likely in clients being treated for borderline personality disorders and obsessive-compulsive disorders, as well as in clients with interpersonal difficulties and more severe problems at the start of therapy. Therapist characteristics that are associated with negative outcomes include lack of empathy, underestimation of the severity of clients’ problems, and negative countertransference. Experiential therapies are more likely to produce negative outcomes, as are minimal interventions for severely distressed clients.

Are Some Therapies Better Than Others?

One of the interesting, but perplexing, aspects of counseling and psychotherapy is the great diversity of different schools and orientations that exist in the field. In the mid-1960s, one list documented over 60 different approaches to therapy. In 1975, a report of the Research Task Force of the National Institute of Mental Health (NIMH) noted over 130 different types of psychotherapy. Five years later, the number had grown to over 200 different forms of therapy, and in 1986, over 400 different therapeutic techniques were listed. Whether the list of therapeutic approaches has grown or shrunk since that time probably depends on who is doing the counting.

In light of the proliferation of approaches to therapy, it is reasonable to ask whether some approaches are better than others. Such a question is one of the relative efficacy of different treatments. Specifically, does Treatment X produce a better outcome than Treatment Y? In contrast to the evaluation of the absolute efficacy of a treatment (“Is Treatment X better than no treatment?”), researchers evaluate the relative efficacy of different approaches to counseling by contrasting one treatment against another.

In 1936, upon considering the claims of success (and eventually of superiority) of the proponents of the various then-current approaches to therapy, Saul Rosenzweig conjectured that despite purported differences in these various approaches to counseling and psychotherapy, they were essentially equivalent in terms of their outcomes. Borrowing a phrase drawn from Lewis Carroll’s 1865 book Alice in Wonderland, Rosensweig remarked, “Everybody has won and all must have prizes”—a phrase now referred to as the Dodo Bird conjecture, after the character in the book that proclaimed the winners of the oddly run Caucus Race.

Although a history of comparative outcome reviews will reveal mixed results in regard to the outcome superiority of various counseling and psychotherapy approaches, contemporary meta-analytic reviews of the comparative outcome research reach a conclusion remarkably similar to that reached in the Dodo Bird conjecture. In general, and in terms of proportions of clients who improve by the end of therapy, differences between various forms of therapy are generally insignificant, but consistent with the absolute efficacy outcome finding noted above—that is, across different approaches to counseling and psychotherapy, there appears to be uniform efficacy.

Are Some Therapies Better Than Others for Certain Problems or Disorders?

While the results of treatment comparison studies suggest little in the way of outcome differences between different approaches to counseling and psychotherapy, it is reasonable to speculate that outcome differences might exist for different therapies, depending on the problems or disorders toward which they were applied. Such a supposition is reflected in the question of what treatment works best for what specific problem. The implication in this question is that the comparison of various treatment outcomes is too gross a comparison to capture meaningful differences among therapy approaches for specific types of presenting concerns. Instead, it may be that one approach is best for treating depression, while another is best for treating anxiety, and so on. In this regard, detractors of the finding of uniform treatment efficacy (the Dodo bird conjecture) contend that the lack of findings of the superiority of certain various treatments is the result of researchers failing to take into consideration the effect of different treatments on different client problems or concerns.

That there is a vast array of different approaches to therapy has already been noted. Similarly, there are many different sorts of concerns, problems, and disorders that clients present to counselors and therapists. A cross-tabulation of treatments by disorders would be enormous. Adding therapist, client, and circumstantial variables to this mix would result in an impossibly large number of combinations for researchers to test and compare and for counselors and therapists to master.

Notwithstanding the enormity of the challenge posed by considering disorder-specific therapy, there is an increasing large body of research that supports the efficacy of particular treatments for particular problems. These studies provided the evidentiary basis for what are referred to as empirically supported treatments. The results of meta-analyses comparing different treatments approaches for different problems, however, generally do not support an interpretation that certain treatments are more effective than others for specific client problem areas.

What Makes Therapy Effective?

The question of what makes therapy effective is not an “outcome question.” Rather it a question about what goes on during therapy—the “process of therapy”—that leads to a therapeutic outcome. Considerable research has been and continues to be conducted that addresses this issue. It is beyond the scope of this entry to review that literature. Suffice it to say that different approaches to counseling and psychotherapy postulate factors specific to each approach as the mechanisms of therapeutic change. However, the findings that different approaches to therapy tend to be equally effective, and that there is little evidence to support the contention that different treatments are differentially effective for different client problems, do not support the argument that factors specific to particular counseling approaches account for change. These findings have led to research on aspects of the therapy process that are common to different approaches to counseling and therapy that might account for the consistency in therapy’s beneficial effects across these different approaches. These two perspectives on the therapy process—change as a function of orientation-specific factors and change a function of common factors—are central to contemporary process research.

References:

1. Chambless, D., & Ollendick, T. H. (2000). Empirically supported psychological interventions: Controversies and evidence. Annual Review of Psychology, 52, 685-716.
2. Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16, 319-324.
3. Frank, J. D. (1982). Therapeutic components shared by all psychotherapies. In J. Harvey & M. Parks (Eds.), Psychotherapy research and behavior change (pp. 9-37). Washington, DC: American Psychological Association.
4. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.
5. Lambert, M. J., & Ogles, B. M. (2004). The efficacy and effectiveness of psychotherapy. In M. Lambert (Ed.), Handbook of psychotherapy and behavior change (5th ed., pp. 139-193). New York: Wiley.
6. Mohr, D. C. (1995). Negative outcome in psychotherapy: A critical review. Clinical Psychology: Science and Practice, 2, 1-27.
7. Ogles, B. M., Lambert, M. J., & Masters, K. S. (1996). Assessing outcome in clinical practice. Needham Heights, MA: Allyn & Bacon.
8. Rosenzweig, S. (1936). Some implicit common factors in diverse methods of psychotherapy: “At last the Dodo said, ‘Everybody has won and all must have prizes.'” American Journal of Orthopsychiatry, 6, 412-415.
9. Smith, M. L., Glass, G. V., & Miller, T. (1980). The benefits of psychotherapy. Baltimore: Johns Hopkins University Press.
10. Strupp, H. H., & Hadley, S. W. (1977). A tripartite model of mental health and therapeutic outcome: With special reference to negative effects in psychotherapy. American Psychologist, 32, 187-196.
11. Wampold, B. E. (2001). The great psychotherapy debate: Models, methods, and findings. Mahwah, NJ: Lawrence Erlbaum.

See also:

• Counseling Therapy