Using Alternative Assessments in Vocational Education
Student assessment has always played an important role in
vocational education, and recent changes in assessment practices
may hold great promise for vocational educators. A RAND study was
conducted to examine alternative forms of assessment from the
perspective of vocational educators. Two products resulted. The
first is this report, which describes a variety of assessment
alternatives, reviews examples from extended case studies, and
discusses criteria for choosing among the alternatives. This
report should be of interest to vocational educators at the state
and local level, particularly those responsible for decisions
about the form and use of assessment systems. The second product
is a set of training materials to help vocational educators make
effective decisions about assessments. These materials are to be
offered as a supplement to Getting to Work: A Guide for Better
Schools (Rahn et al., 1995), a recent National Center for
Research in Vocational Education (NCRVE) training package.
The purpose of this study was to evaluate the utility of nontraditional forms of assessment for vocational education. This issue is particularly important at the present time because both vocational education and educational assessment are undergoing significant changes. In education, enrollment in high school vocational courses is dropping, the nature of vocational students is changing, and employers are calling for applicants who possess skills different from those of the past. In assessment, new forms of constructed-response measures, including performance tasks, portfolios, and senior projects, are gaining popularity, and assessment is being used more prominently as a policy tool. The confluence of these factors makes this an opportune time to take a careful look at the potential value of alternative forms of assessment for vocational education.
For our study, which ran from 1994 to 1996, we selected six cases that reflected a wide range of assessment options, with particular emphasis on operational programs using constructed-response measures. In each case, we conducted a critical review of the assessment system based on descriptive materials provided by the program, research literature, telephone interviews, and, in all but two cases, a one- to two-day site visit. Our investigations focused on program definition, implementation, and administration; the quality and feasibility of the assessments; and the potential usefulness of the assessment approach for vocational educators. Most, but not all, of our cases were drawn from vocational education. They include the Career-Technical Assessment Program, the Kentucky Instructional Results Information System, the Laborers-Associated General Con-tractors environmental training and certification programs, the National Board for Professional Teaching Standards certification program, the Oklahoma Department of Vocational-Technical Education competency-based testing program, and the Vocational/Industrial Clubs of America national competition.
By comparing these assessment systems, we were able to identify a
number of important design and implementation considerations that
should inform decisions about the use of alternative assessments
in vocational education (and, more broadly, assessment planning
in any educational context). First, the user must clarify the
purpose of the assessment because the intended use (e.g.,
improving learning and instruction, certifying mastery) affects
the choice of assessment method. Second, the user must understand
the nature of the skills being assessed. Different cognitive
demands (e.g., factual knowledge, integrated problem solving) may
be better measured with one type of assessment than with another.
Third, the user should become familiar with the range of
alternative assessment options available. We identify four
categories of assessments that may be of interest to vocational
educators: written assessments, portfolios, performance tasks,
and senior projects. Fourth, the user should understand the
advantages and disadvantages of each assessment strategy. We
describe several dimensions along which alternative assessments
may differ, the two most important being quality (which includes
reliability, validity, and fairness) and feasibility (which
includes cost, time, complexity, and credibility). We discuss the
quality and feasibility of the assessment types identified.
Fifth, the user should be aware of certain other issues involved
in designing assessment systems that might be relevant to a
particular program or setting. We review the relative advantages
of single versus multiple measures, low versus high stakes,
standardization versus adaptability, embedded versus stand-alone
assessments, single versus multiple purposes, and voluntary
versus mandatory participation.
Finally, we illustrate how vocational educators can use this review to think about the usefulness of alternative assessments for a particular situation. We describe a procedure for evaluating the utility of alternative assessments that includes analyzing the purposes of the assessment and the skills to be assessed, considering alternative types of assessment, reviewing their quality and feasibility, and considering other factors identified in the case studies. This approach is illustrated by two scenarios that describe assessment problems faced by vocational educators and show how the results of this study can contribute to their resolution.
Stecher, B. M., Rahn, M. L., Ruby, A., Alt, M. N., Robyn, A., & Ward, B. (1997, July). Using alternative assessments in vocational education. Berkeley, CA: National Center for Research in Vocational Education.