One of psychology’s most important and enduring contributions to civilization is the development of the psychological test. Psychologists have invented and refined psychometric procedures (i.e., tests) for assessing a breathtakingly wide array of constructs. Among the topics reviewed in this volume of the Encyclopedia of Counseling, for example, are the assessment of academic achievement, adaptive behavior, affect, counseling process, counseling outcome, depression, intelligence, language, memory, mental status, neuropsychological functioning, personality, problem solving, psychopathology, and self-esteem. Also reviewed are assessment procedures such as the use of clinical interviews and projective techniques. Psychologists are presently developing advanced technologies such as computer-assisted assessment, computer test interpretation, and item response theory.
Most counseling psychologists regard psychological assessment as an important activity regardless of their work setting or the type of clientele. A national survey of American Psychological Association members revealed that the work activities of counseling psychologists include collecting information about their clients (80.3%) and identifying and diagnosing their clients’ problems (76.6%). More than 20% of their total professional time is devoted to assessment and diagnosis. Psychological tests are so critical to the work of psychologists that Rene V. Dawis and David Lubinski have described them as serving the same function as the microscope to microbiologists and the telescope to astronomers.
Academic Writing, Editing, Proofreading, And Problem Solving Services
Get 10% OFF with 24START discount code
Assessment is not an end unto itself; the critical intent is to obtain information that will be useful in addressing the practical problems confronting the individual and society. However, the incremental utility of assessment has not been established and the belief in the value of assessment rests more on assumptions of utility than on empirically demonstrated benefits. Although studies have examined the efficacy and effectiveness of a variety of therapeutic approaches, scant empirical attention has been paid to the process of assessment.
Test interpretation, in particular, is one assessment procedure that has received relatively little scientific scrutiny. This entry describes approaches to assessment and interpretation, explains how a skilled psychologist prepares for and delivers an interpretation, and summarizes the evidence pertaining to the effectiveness of test interpretation.
Approaches to Assessment
Clinical Interview
The assessment technique most widely used by counseling psychologists is the clinical interview. However, psychometric assessment procedures have numerous advantages over the interview. For example, therapists gain experience one client at a time, but tests provide information based on large numbers of people in the form of norm groups. Given the fallibility of human memory, therapists are likely to forget some cases and to give too much weight to memorable, but atypical, cases. The norm groups provided by tests are not overly influenced by unusual or graphic cases.
Administering a test to an individual can be thought of as analogous to conducting a standard interview. At the conclusion of the procedure, the test reports descriptive information (scores) having an approximately known level of reliability. In contrast, psychologists conduct semistandard interviews and obtain descriptive information having an unknown reliability. The test scores suggest inferences having an approximately known validity. In contrast, psychologists draw inferences that have less validity than the inferences drawn from the tests. Furthermore, supporters and critics repeatedly examine the reliability and validity of tests, while the reliability and validity of the interview, as an assessment device, is virtually never examined.
Clinical Versus Statistical Prediction
The end goal of assessment is to obtain information that can be used to guide the important decisions psychologists and examinees must make. That means the information obtained from tests must be useful in predicting important real world phenomena. The issues that confront individuals and require predictions are as diverse as life itself: whether to get married or divorced, which course of study or job to pursue, whether an incarcerated individual should be permitted to rejoin the community, and what approach will most effectively treat the symptoms that interfere with an individual’s ability to function in society. In short, the usefulness of test scores lies in the real life criteria they can predict.
In 1954, Paul Meehl demonstrated that the inferences (i.e., predictions) trained therapists draw from psychological tests are not as accurate as those made using statistical algorithms. Well over 100 studies have been published demonstrating that statistical algorithms are usually more accurate than trained therapists in predicting people’s future behavior.
The greater accuracy of statistical algorithms is attributable to several of the factors explained above (i.e., norm groups, reliability, and validity). An additional problem occurs when combining multiple pieces of information. Therapists have limited ability to determine the proper weight to give each factor, and they are likely to be influenced by irrelevant considerations. Statistical algorithms derive weights for each factor that lead to the most accurate predictions possible.
Despite overwhelming evidence that actuarial predictions are more accurate, the use of test information in clinical practice has remained virtually unchanged in the half century since the publication of Meehl’s book. A challenge to psychology is to develop procedures for better integrating the results of assessment procedures into clinical practice. A careful examination of this issue is long overdue.
Computer Test Interpretation
The development of computer programs capable of preparing test interpretations was pioneered in the area of vocational interest measurement three decades ago. Since that time the practice has expanded to include personality tests and tests used in making psychological diagnoses. Essentially, the computer program compares the test results to the normative data available for the test and determines the range within which the score falls (e.g., very low, average, slightly elevated). It then selects relevant descriptive phrases, sentences, and paragraphs from a library of interpretive materials and organizes these into a narrative summary. When completed, that summary can be displayed on a computer monitor and printed, as desired.
Some interpretations consist of little more than normative statements that indicate whether a score could be considered low, average, or high. For example, a statement such as, “Your score on the introversion scale is in the normal range when compared to the scores of other high school freshmen” provides a normative interpretation.
Interpretations that are more elaborate explain the meaning of the score. For example, the meaning of introversion could be explained using statements like the following:
Introversion refers to a predisposition to be concerned with your own thoughts and feelings. Contrary to what many think, introversion is not the same as shyness. Shyness involves an element of apprehension or anxiety that is not necessarily present in introverts. Although introverts may be less inclined to seek companionship, for some it is merely because they are energized by being alone.
Some test interpretation programs take the process a step further and suggest implications of the scores for future behavior:
People who obtain similar introversion scores tend to avoid conflict and unpleasantness. They may enjoy social interactions, but also find them draining and typically feel a need for time to recharge their batteries after social interactions. They may be especially uncomfortable in social interactions with the opposite sex. They form social relationships slowly and deliberately, but once formed those relationships tend to be highly stable.
Although computer-generated interpretations sound authoritative, their reliability and validity require careful scrutiny. However, the cost of evaluating these algorithms is high and most psychologists do not possess the resources necessary to undertake such an evaluation.
Self-interpreting Tests
Another major innovation in the last half century is the self-administering, self-scoring, self-interpreting test. Personality inventories such as the Allport-Vernon-Lindzey Study of Values and vocational interest inventories such as John Holland’s Self-Directed Search pioneered this approach. The practice is now widely used in assessing constructs of interest in vocational psychology, it has been incorporated into computer-based guidance programs such as SIGI, and it is enjoying increasing use in personality tests such as the Neuroticism, Extraversion, Openness Personality Inventory-Revised (NEO PI-R).
Psychologists were originally concerned about the possibility that examinees would make mistakes in scoring their tests that would lead to inaccurate information. Early research revealed that some scoring problems did exist, but modifications of the tests have reduced these problems to an acceptable level. There have also been numerous expressions of concern that examinees will make serious mistakes in attempting to interpret these tests for themselves. Despite this, a review of the literature reveals only seven investigations in which the accuracy and effectiveness of self-interpretation was considered. Those studies generally revealed no differences among the modes of test interpretation, but the lack of scrutiny of this important test interpretation innovation is cause for caution and concern.
Preparing and Delivering a Test Interpretation
Preparing the Interpretation
Expert test interpretation requires the integration of technical information about the test with an intimate understanding of the individual who has taken the test. This requires preparation. Despite extensive experience with the tests to be interpreted, conscientious psychologists review reference sources to refresh their memory of the nuances that are critical to insightful test interpretation. Furthermore, they realize that test scores must be interpreted in the context of the personality and life history of an individual. For example, a comment reflecting doubt or pessimism about the prospects for a happy life takes on different meanings when made by a young person whose parents just died in a automobile accident and one who is undergoing treatment for leukemia. Psychologists avoid interpreting test results in isolation, if possible.
Understanding the Assessment Results
Psychologists’ initial question when preparing a test interpretation is, “What do these test results mean?” The objective is to develop hypotheses that can be tested when discussing the results with the client. Test scores provide statistical summaries of statements the client made in responding to test items or evaluations of the client’s performance on tasks required by the test. As such, psychologists may ask clients to explain some of their responses to the test during a subsequent interview.
Counselors give special attention to the meaning of information that appears to be contradictory. Resolving apparent contradictions is important and often requires discussion with the examinee to arrive at a clear under-standing. Low or socially undesirable scores can be threatening or embarrassing to clients and require careful consideration. Psychologists think carefully about how to discuss this information with the client.
Psychologists may use a nomothetic (norm-based) or an idiographic (individual-focused) approach to understanding the meaning of the test results. Nomothetic interpretations tend to be quantitative. They compare the scores of the individual to a relevant comparison group, as when a ninth-grade student’s performance on a math test is compared to that of other ninth-grade students. Placing the results in a relevant context allows the psychologist to distinguish average or typical scores from various types of exceptional scores. Psychometric testing and other quantitative methods are consistent with a nomothetic approach to test interpretation.
Psychologists working with individuals are also interested in understanding the examinee as a unique individual without reference to others. The idiographic approach to test interpretation involves an exclusive focus on the individual. Case studies, informal interviews, unstructured observation, and other qualitative methods are more consistent with idiographic approaches. The assessment of vocational interests is one area in which idiographic approaches are common. Vocational psychologists typically want to know how an individual’s interests and abilities compare to those of a reference group, but also how each interest and ability compares to the other interests and abilities of the person. For example, while it is useful to know that a person’s interest in music is strong when compared to the interests of other people, it may be even more important to know that the person has even stronger interests in business, teaching, and military service.
Integration of Results
Psychologists must integrate their understanding of the test results with the other available information. This information may include others’ comments, work-or school-based performance indicators, additional assessment results, and family background or work history information. At a minimum, most psychologists will have information gained from the interviews conducted prior to selecting the assessments for the individual to take.
The first step is to evaluate the consistency of the information. Psychologists identify any inconsistencies and attempt to determine why they occurred. Mistakes in administering the test can result from errors such as using the wrong form of the test, the wrong answer sheet, or failing to follow the directions properly. Mistakes in scoring the test can result from a simple mistake when evaluating the test responses, using the wrong scoring procedure, using the wrong norm group for converting the raw scores to standard scores, or making a mathematical mistake when calculating the standard scores. Skilled psychologists do not assume the test result is correct when it contradicts the other information that is available, nor do they automatically disregard test results that do not confirm their expectations. They investigate the inconsistency and attempt to resolve the paradox. Often, this involves discussing the seeming inconsistency with the examinee.
Presentation Order
Once psychologists have a clear understanding of the test results, they consider the order in which to present the information. This involves adopting the client’s frame of reference. Psychologists develop a tentative plan that is subject to revision during the test interpretation interview. For example, the examinee might ask about a particular set of results that are of special interest, or express an interest in discussing something different than the test results.
Psychologists often begin the interpretation with more concrete information (e.g., achievement test results) and then move to information that is more abstract (e.g., personality test results). When the psychologist is uncertain about the best order or believes the order is unimportant with this particular client, he or she may review the tests to be interpreted with the client and ask the client’s preference.
Psychologists realize that the test interpretation must be guided, in part, by the emotional needs of the client. They consider whether the client is psychologically ready to think about the test results effectively, able to handle threatening information constructively, and whether this is the most important use of interview time at this point.
Preparing the Client
Despite the fact that the therapist and client have jointly agreed on the tests to administer and the timing of the test interpretation, psychologists realize that it is still important to prepare the client for the interpretation.
Therapists give clients an opportunity to raise any issues they regard as important prior to discussing the test results. The therapist begins the test interpretation only after determining what will be the most effective use of the interview time.
Psychologists lead the interview into a discussion of the test results when it is clear the client is ready. Often they describe the test briefly so the client can associate the results with the test he or she took. This is more important when the client took more than one test or the time lapse between completing the test and interpreting the results is prolonged.
Psychologists also review with the client the types of information the test provides and the types of interpretations that are possible. For example, a score indicating an interest in a particular occupation does not mean the person has the ability to be successful in that occupation. Psychologists explain the norm group used to interpret the test; they may interpret the results in relation to several norm groups. Before beginning the test interpretation psychologists also take care to make sure the client understands the inexact nature of test scores and the limits on the precision with which inferences can be drawn. Often they reiterate this point periodically throughout the interpretation process.
Delivering the Test Interpretation
Client’s Goals
A given test score may have several possible interpretations. The most useful interpretation addresses the issues confronting the client and therapist. For example, a given score on an academic aptitude test could indicate the client’s intellectual ability is probably adequate for completion of a B.A. in history, or that completing a master’s degree in international relations would be quite difficult. Psychologists make the test interpretation personally meaningful by relating the results to the issues and decisions confronting the client.
Test Precision
Many individuals believe that tests can provide precise answers to their questions. They often report test results—particularly flattering or socially desirable results—using precise scores or unqualified statements (e.g., “My IQ is 130” or “I am an altruistic extravert”). Psychologists avoid the use of technical terms such as the standard error of measurement, but they do explain that a test score (e.g., an IQ of 130) actually indicates an approximate level (e.g., an IQ somewhere between 124 and 136). Likewise, when relating the test score to an external criterion psychologists explain that the test score actually predicts a range of possible outcomes. It is not always necessary or desirable to give examinees exact scores or detailed technical information. Psychologists view the purpose of the test interpretation as conveying meaningful information.
Defensive Reactions
Interpreting low or potentially undesirable scores can be difficult. Psychologists do not scold, disapprove, moralize, or use pejorative labels when interpreting such scores, but they are sensitive to the possibility that examinees will impose these reactions upon themselves. Psychologists do not avoid these scores nor downplay their importance, but they do remind clients of the limited accuracy of these scores. They point out the implications of the scores and relate them to the decision the client must make. The objective is to get clients to discuss what this information means for them.
Examinees need accurate information about their weaknesses and areas needing improvement as well as information about their strengths. Low or negative scores can be useful in raising issues the client needs to address. Talking about these scores allows the examinee to acknowledge these areas and to develop realistic plans. At the same time, psychologists remain sensitive to the client’s feelings about these scores and deal with those feelings in a sensitive and constructive manner.
Plain Language
Psychologists avoid technical terms and explain test results in language the typical individual can understand. They also avoid the use of words such as neurotic, maladjusted, masculinity or femininity, and intelligence that are likely to arouse unfavorable connotations. Once a counseling relationship has been established, psychologists also identify and avoid the use of individually threatening terms.
Encouraging Feedback
Psychologists interpret tests results in a manner that allows and encourages client participation. They accomplish this goal by using a statement such as, “How does that compare with what you expected (or think about yourself)?” “Does that seem right to you?” or “You seem to be disappointed with that score.”
Eliciting client feedback allows the psychologist to determine whether the client understands the information being presented, and it provides information about the clients’ attitudes and feelings toward the information. Instead of asking a direct question, the psychologist periodically may ask the client to summarize the information being presented. This helps create a therapist-client discussion rather than a psychologist monologue, and it allows the psychologist to determine whether the client understands the information. Furthermore, this gives the client an opportunity to organize and integrate the information and to express his or her feelings about the information.
Psychologists may also ask the client to speculate on the implications of the test results. This requires that the client organize the information presented, integrate it with his or her existing knowledge, and draw conclusions about the implications of the information for the decisions he or she must make.
As noted earlier, skilled psychologists take care to ask clients about any unusual responses they noted when reviewing the test responses. For example, the psychologist might say, “I noticed on one of the items you replied that you sometimes think about injuring yourself. Can you tell me more about that?” This leads to a discussion of an issue of direct concern to the client and psychologist.
Test Profiles
Many tests provide graphical representations of test scores. Use of these visual aids helps clients better understand the test interpretation. Visual aids help clients clarify, simplify, and organize the test results. Clients typically appreciate a graphical depiction of their test results that they can take home. Many psychologists also encourage clients to take notes throughout the test interpretation.
One effective test interpretation strategy involves having clients rate themselves on the traits on which they are to be tested before they take the tests. The psychologist plots these self-estimates on a blank test profile and then during the test interpretation plots the actual test scores on the same profile. This shows the relation between the client’s self-estimates and actual test scores and leads to a useful discussion to any discrepancies. Clients find it useful to take this profile home for further study.
Follow-Up
The client’s assimilation of test information continues after the test interpretation interview. Psychologists continue to refer to the information in subsequent interviews and to review and reinforce the clients’ understanding of that information whenever necessary.
Research on Test Interpretation Outcomes
Purpose of Assessment
It is not possible to evaluate the practical utility of assessment without specifying the purpose of the assessment procedure; however, psychologists have given this issue little attention. It appears that most practicing psychologists use assessment procedures for one of three purposes. One is to obtain information for making predictions. For example, therapists perform assessments to obtain information for use in deciding how to work effectively with their clients or for use in advising their clients about future courses of action. Psychologists also perform assessments to obtain descriptive information to help them better understand their clients. Prediction is inherent as a secondary objective in this purpose. Finally, assessment procedures are used as an intervention. For example, psychologists sometimes interpret test results to increase their clients’ sensitivity to important developmental issues. Evaluating the practical utility of assessment requires a case-by-case specification of the purpose of assessment, but this information is virtually never available.
In addition to the three general purposes specified above, psychologists may anticipate a variety of specific benefits. Several objectives are possible, including the following: learning factual information, changing attitudes or behaviors, and influencing future decisions. Unfortunately, the criteria typically used in past research on the utility of assessment are the recall of specific scores and changes in the accuracy of self-estimates. These criteria do not adequately represent the many potential benefits of testing.
Effects of Test Interpretation
Howard E. A. Tinsley and Serena Chu reviewed 65 research studies that examined test interpretation outcomes. Most of the research focused on the interpretation of aptitude and ability tests. Virtually no research examined the interpretation of vocational interest tests or the use of tests in individual psychotherapy, couples counseling, family counseling, substance abuse counseling, or any of the many other specialty areas in which psychologists provide services. They found that few studies have been competently done or adequately reported, and that the research is fraught with methodological weaknesses (e.g., flawed criteria, use of an immediate follow-up, lack of random assignment, and lack of a control group). They concluded that there is no coherent body of evidence demonstrating the efficacy of test interpretation as an intervention.
They reported tenuous support for only three conclusions. First, the use of visual aids improves the effectiveness of test interpretation. Second, group test interpretations are as effective as individual test interpretations. Third, examinees prefer individual test interpretations to group test interpretations. Since individual interpretations are six times more costly than group interpretations, however, providing individual interpretations when group test interpretations are feasible does not appear to be justified.
Conclusion
Adroit test interpretation requires that psychologists extract accurate information from their assessment procedures, draw accurate inferences from that information, and accurately convey the results to their clients in a manner they can understand. There is little evidence documenting the ability of counseling psychologists to satisfy these requirements. Furthermore, a convincing case has not yet been made for the practical utility of testing even when these requirements are met. Despite this, psychologists who are conscientious in their preparation of the test interpretation, who take care to prepare the client for the interpretation, and who are proficient at delivering the interpretation find that test interpretation provides a valuable adjunct to therapy.
References:
- Goldman, L. (1971). Using tests in counseling (2nd ed.). Santa Monica, CA: Appleton-Century-Crofts.
- Goldman, L. (1972). Tests and counseling: The marriage that failed. Measurement and Evaluation in Guidance, 4, 213-220.
- Hambleton, R. K., Robin, F., & Xing, D. (2000). Item response models for the analysis of educational and psychological test data. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling (pp. 553-581). San Diego, CA: Academic Press.
- Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. Northvale, NJ: Jason Aronson.
- Meehl, P. E. (1956). Wanted—a good cookbook. American Psychologist, 11, 262-272.
- Tinsley, H. E. A., & Bradley, R. W. (1986). Test interpretation. Journal of Counseling & Development, 64, 462-466.
- Tinsley, H. E. A., & Chu, S. (1999). Research on test and interest inventory interpretation outcomes. In. M. L. Savickas & A. R. Spokane (Eds.), Vocational interests: Meaning, measurement and counseling use (pp. 257-276). Palo Alto, CA: Davies-Black.
See also: