10. What are the critical aspects
of the Assessment Plan?
ASSESSMENT OF CANDIDATE
Principles for Performance-Based
Assessment Systems in Professional Education Programs
Assessing what professional educator candidates know and can do is critical
to implementing the performance-based standards of the National Council for
Accreditation of Teacher Education (NCATE) and its affiliated national professional
specialty organizations. Given the complexities of teaching and other educational
professions, the range of knowledge, skills, and dispositions to be assessed,
the multiple purposes for which assessment results are used, and the stakes
associated with the outcomes, assessment in professional education programs
and units needs to include multiple measures implemented on a systematic and
ongoing basis as part of a comprehensive system. This document outlines principles
set forth by the NCATE Specialty Area Studies Board for performance-based assessment
systems at the program level.
Although assessment systems will vary across programs and units, they generally
should a) address the knowledge, skills, and dispositions to be acquired by
professional educator candidates as set forth in program goals; b) be consistent
with the standards of relevant national and state accrediting/approval bodies;
c) have multiple means for measuring candidate performance and impact; and d)
provide on-going, systematic information useful for decision-making. It is particularly
critical that assessment systems provide credible results that are collected
and used in a fair, valid manner consistent with their intended purpose(s).
Assessment systems should have the following characteristics:
1. The system is driven by a conceptual framework and program values which espouse
assessment as a vehicle for both individual and program self-evaluation and
improvement. Assessment is planned and implemented by key stakeholders in a
manner consistent with the method of inquiry in the discipline and is considered
a means to an end rather than an end in itself.
2. The system includes components which work together in a synergistic manner
to address the knowledge, skills, and dispositions of candidates across program
goals, objectives and curriculum consistent with the performance-based standards
of the respective national professional specialty. Assessment is a goal-oriented
process linked to program purposes/goals and national standards.
3. Multiple measures are planned and administered on a systematic, ongoing basis
throughout the program beginning with the admissions process. The system includes
quantitative and qualitative measures useful for formative and summative assessment.
One or more measures designed to yield evidence of positive candidate impact
on students is included in the system.
4. The system includes one or more measures which have been created, reviewed,
and/or scored by specialty professionals external to the program. Such professionals
include those with relevant specialized expertise whose primary responsibility
is not to the program/unit, such as field-based master teachers, clinical teachers,
intern supervisors, and/or supervisors/employers of program candidates/graduates.
5. The system is clearly delineated. Measures and associated criteria or rubrics
(including minimal proficiency levels), as well as policies and practices for
obtaining and using results, are described in program documents in a manner
which candidates and other stakeholders can understand. Candidates are made
aware of program standards and assessment requirements to which they will be
held and are provided with models and/or examples of performance and the instruction
and support needed to attain such levels.
6. The assessment methods and corresponding criteria included in the system
are sufficiently comprehensive and rigorous to make important decisions about
the proficiencies of candidates and to safeguard those they may potentially
serve. Critical decision-making points are delineated in the system. Decisions
that are made reflect the application of relevant criteria and use of results
in a manner which discriminates acceptable versus unacceptable performance.
7. The system includes policies and procedures for the gathering, use, storage,
and reporting of individual results. Such policies address the rights of individuals
(e.g., those afforded candidates by the Family Educational Rights and Privacy
Act; confidentiality/anonymity of survey responses). Individual candidate results
are reported in a clear manner which acknowledges the source(s) and limitations
of the data, individual strengths, and areas of needed or potential improvement.
8. The system includes a structure and procedures for sampling, analyzing, summarizing,
and reporting aggregated results. Data are gathered on an ongoing basis and
are summarized in a manner which reflects pass rates, the range of performances,
and/or the "typical" or "average" performance (e.g., mean,
median, or modal performance) as appropriate to the types of measures. Summaries
of results are provided to key program stakeholders in a clear manner which
acknowledges the source(s) and limitations of the data, data collection and
reporting time frame, program strengths, and areas of needed or potential improvement.
9. The program and its assessment system foster the use of results for individual
candidate and program improvement. Assessment results are regularly reviewed
in relation to program goals and objectives as well as to relevant state and
national standards and stimulate changes designed to optimize success.
10. The system has a mechanism and procedures for evaluating and improving itself
and its component assessment methods. Evidence of the reliability and validity
of the system and its component measures is gathered and used to make decisions
about their ongoing use and/or revision. Evidence should address the ability
of the system to comprehensively assess performance in a credible manner which
is valid, fair, and unbiased.
Characteristics of Sound Evidence
Sound evidence usually exhibits several
- Results from planned, purposeful,
and continuing evaluation of candidate proficiencies, drawing on diverse sources;
Monitoring of candidate performance
is embedded in the ECIT preparation program and conducted on a continuing basis.
This monitoring is planned in response to faculty decisions about the points
in the program best suited to gathering candidate performance information, consistent
with the institutions own context and mission. Typically such information
is gathered at candidate entry, in coursework, in connection with field experiences,
prior to the start of practica, and at completion of the program.
All information about candidates proficiencies, from all sources, is drawn
on by the unit for continuous evaluation of candidate progress and program success.
Excerpts, summaries, and samples from this array of information are provided
for use by NCATE in its program quality reviews. Institutions will usually begin
to plan their assessment system around activities that are the direct responsibility
of the ECIT preparation unit. Examples of assessments that might be used or
created within the program include end-of-course evaluations but also
tasks used for instructional purposes such as projects, journals, observations
by faculty, comments by supervisors, samples of candidate work, and other information
that would commonly be available for faculty use in determining the adequacy
of the candidates accomplishments in a course.
The monitoring information from the ECIT preparation program can be complemented
by evaluations originating from external sources that supply information on
candidate proficiencies. Examples from outside the unit are candidate performance
evaluations during induction years and follow-up studies; performance on state
licensure exams that assess candidates knowledge and skills; and academic
subject knowledge end-of-course examinations, essays, or other demonstrations
- Represents the scope of the
standards for ECIT preparation;
Candidate performance evidence is
congruent with the knowledge and skills in the AECT standards. Institutions
determine the best way to demonstrate that all aspects of the standards are
covered, but avoid treating each individual statement in the standards and supporting
explanations in an individual, serial, and fractionated way. Instead, faculty
think through how all their existing assessment information can be marshaled,
and what additional information is needed, to demonstrate candidate proficiency
across the standards. The usefulness and value of information derived from tests
are the key determinants in decisions to use or exclude them from an institutions
performance measurement system.
- Measures the different "attributes"
of standards in appropriate and multiple ways;
One conclusion about the current
state-of-the art in assessment is that no single test or measurement of teacher
candidates is sufficient by itself to represent these different attributes and
the full scope of the standards. Multiple measures provide wide opportunities
for candidates to demonstrate their accomplishments in relation to the standards.
It is anticipated that institutions will draw on the extensive range of available
assessment forms, including objective tests (which may be useful to gauge proficiencies
in standards calling for candidate knowledge) and also portfolios, observations,
reflections, teaching demonstrations, analytic work, candidate work samples,
and other forms of evaluative information demonstrating proficiency in technology
use. Consider as well external evidence of graduate success (surveys, licensure
tests, employer induction year assessments), artifacts produced by the candidates
(products, plans, assessments, case studies), reflective essays, attestations
of accomplishments by supervisors, awards and recognitions, professional service,
and scholarly activities.
- Results from rigorous and systematic
efforts by the institution to set performance levels and judge accomplishments
of its candidates;
Faculty establish written and shared
explanations of what is valued in a candidates response to an assessmentthe
qualities by which levels of performance can be differentiatedthat serve
as anchors for judgments about the degree of candidate success. The terms "rubrics"
and "criteria" are frequently used in assessment to designate these
explanations for levels of performance. These may be stated in generic terms
or may be specific to particular assessment tasks. They may define acceptable
levels of performance for the institution and one or more levels below (such
as borderline, or unacceptable) and above (such as exemplary), or they may be
in the form of criteria defining the institutions expectations for success.
The rubrics or criteria are "public," that is shared with candidates
and across the faculty. Faculty teach, advise, and prepare candidates for success
in meeting critical external performance expectations, as expressed, for example,
in state licensure test pass scores.
The institution judges individual candidate proficiencies, and also summarizes
and analyzes the proportions of candidates who reach levels expressed in the
rubrics or criteria. These results are used both for advisement of individual
candidates, and also for strengthening of the courses and experiences offered
by the institution to prepare elementary teacher candidates. The summary of
results from the faculty judgments in applying the rubrics or criteria are used
for the NCATE submission. Examples of candidate work are attached to the institutional
submission where that is a useful way to assist reviewers understanding
of the levels of proficiency reached by candidates.
- Provides information that is
credibleaccurate, consistent, fair and avoiding bias;
The institution gathers information
on the accuracy (or validity) and consistency (or reliability) of its assessments.
Accuracy is an expectation that the assessment information measures what is
important for the decision to be made and that it represents the performances,
competencies, and dispositions that are intended (that is, included in the AECT
standards). Consistency is an expectation that successive samples of performances
from the same candidate are reasonably related. Assessment systems must also
be fair, avoiding bias and providing equitable treatment. These are matters
that require professional judgment and are often determined through peer review,
evaluations by external experts, or formal validation studies.
- Makes use of appropriate sampling
and summarizing procedures.
In preparing the program submission,
the institution samples and summarizes information about candidate proficiencies.
Sampling refers both to representing the domain of the standards and representing
the full range of the programs candidates. The candidate sample might
be taken from the cohort of candidates completing the program in a specific
academic year and previous completers so that information about performance
of candidates from their entire preparation experience and into employment can
be available for demonstration of candidate proficiency. Of course, anonymity
of individual candidates and the students of those candidates must be protected.
Candidate proficiency results are summarized through averages, spread of scores,
and distributions of rubric scores. Summary results are requested because NCATEs
interest is in making decisions about program quality, rather than decisions
about individual candidates. These summaries are made meaningful through illustrations
such as samples of examination questions, examples of written responses, and
analytic materials intended to inform reviewers of the proficiencies that candidates
achieve in relation to the standards.
Of course, institutions that have sound evidence systems use the data
to advise individual candidates and to strengthen teaching, courses, experiences,
These qualities of assessment evidence are not, themselves, the requirement
for submission. The submission is developed to describe the results of the
An Explanation of the NCATE Transition Plan
NCATE understands that the development
of an assessment system an take several years. It has made provisions for the
development of assessment systems in its Transition Plan. The Transition Plan
identifies levels at which the first two standards must be addressed at the
time of the unit's on-site visit. During visits, NCATE expects the unit to
make available performance assessment data that are currently collected.
These data include assessments conducted internally by the unit and external
data such as state licensing tests, program review reports, and graduate and
employer surveys. In addition to performance data that are already collected,
the unit must, at a minimum, present a plan of the unit's assessment system
during the first year that the standards are effective, fall 2001 and spring
2002. Increasing implementation of the plan is required in each subsequent year.
Consequently, all units1 should follow the schedule outlined in the Transition
Plan for the development and implementation of their assessment systems. Units
may move at a faster pace at their discretion.
This paper is written to detail NCATE expectations for institutions that are
developing assessment systems and have visits scheduled for fall 2001 and beyond.
At a minimum, all institutions must have a plan for their assessment system
during the 2001-2002 academic year. Plans for assessment systems should
include the six items detailed in this paper and should be developed collaboratively
by the units' educational communities. In addition, units should review the
appropriate components of an assessment system detailed in the rubric and supporting
explanation for Standard 2 of the NCATE Unit Standards.
According to the Transition Plan, units with visits in fall 2001 and spring
2002 should have a plan for an assessment system that:
- Identifies transition points
at the unit and/or program level.
Transition points are key points in a program when the unit assesses candidate
knowledge, skills, and dispositions, and determines if candidates are ready
to proceed to the next stage. The NCATE standards require transition points
upon entry, prior to entering clinical practice, prior to exiting clinical
practice, and upon program completion. While the four transition points stated
above must be in place for all programs, institutions may have, at their discretion,
additional transition points in their programs.
The unit should anticipate a course of action if assessments indicate that
candidates are not yet ready to proceed to the next stages of their programs.
Possible actions might include remediation, r~taking assessments, denial of
advancement, and academic probation.
- Identifies the major assessments
to be used at the stated transition points.
Major assessments can include tests, portfolios, essays, student teaching/internship
evaluations, student work samples with reflections, etc. The assessments should
be linked to the learning outcomes in the conceptual framework and should
reflect institutional, state, and national standards. The assessments must
address candidate content knowledge, pedagogical knowledge, professional knowledge,
pedagogical content knowledge, dispositions, and candidates' positive impact
on K-12 student learning.
At this stage in the development of the assessment system, the unit must decide
what assessments will be used and when. NCATE does not dictate the types of
assessments used, however, evaluations of student teaching/internship and
state licensing assessments, where applicable, are expected to be two of the
components of the assessment system. NCATE supports the use of multiple assessments
in evaluating candidates. The types of assessments may change over time based
on considerations of fairness, accuracy, and consistency.
While a review of all assessments in all courses may be in order to address
unit and program coherence and the modeling of best practices by faculty,
only major assessments should be identified as part of the assessment system.
The instruments for these assessments may not yet be developed, but the unit
must indicate a timeline for their development. See Item 3 below.
The units should specify which elements of the standards each assessment is
designed or being designed to evaluate. The same assessments must be administered
to all candidates in a given cohort in a given program/unit. This means that
all candidates in a program that reach the designated transition points should
be required to complete the same assessments. When including course-based
assessment(s) as part of the assessment system, the unit should ensure that
candidates in different sections of the same course are administered the same
assessment(s). Inevitably, assessments will differ across programs. The unit
must coordinate the effort and ensure that each program is operating in the
context of an overall assessment system.
Assessments must be fair, accurate, and consistent. To ensure this, the unit
may need to provide time and resources for the review of curricula to ensure
that candidates have the opportunity to learn the materials assessed. In addition,
the unit must provide time and resources for piloting assessments, developing
benchmarks, ratings assessments, and analyzing the extent to which the assessments
were successful in measuring targeted candidate knowledge, skills, and dispositions.
- Identifies a timeline for the
development and implementation of the major assessments.
The timeline should include major steps in the development and implementation
of assessments. These steps may include preparation, the development of drafts,
piloting of drafts, benchmarking, ensuring that the assessments meet their
intended goals, and refining the assessment instruments. Steps may also include
revising graduate surveys, re-thinking faculty evaluations, and developing
better focused employer surveys. The timeline should indicate what is to be
done, by when, and what committee/person will be responsible for completing
the tasks. It can be written in increments such as weeks, months, quarters,
- Identifies the design for the
collection, analysis, summarization, and use of data.
Once the system is in place, data from the system, particularly student assessment
data, must be compiled at regular intervals. It is anticipated that these
intervals will correspond with the transition points discussed above. Decisions
should be made about student progress at each interval. The plan for the unit's
assessment system should identify how the data will be generated and when
the data will be collected. Will portfolios be submitted and evaluated by
a committee? What is the content for the portfolios? Will candidates have
to sit for an exam that is then graded by two faculty members? Will they be
required to complete a student teaching/internship assignment evaluated by
Further, once the assessments are evaluated, the data must be summarized and
analyzed. The purpose of summarization and analysis is to enable the unit
to look across the cohort to examine strengths and weaknesses, to identify
trends in comprehension of knowledge, skills, and dispositions, and to pinpoint
where additional support and academic work needs to be done. The summarization
and analysis of graduate and employer surveys can further help the unit identify
programmatic strengths and weaknesses. Similarly, the summarization and analysis
of faculty evaluations can inform the unit of professional development needs.
Summarization and analysis shifts focus from individuals to programs. Programmatic
analysis, in turn, should lead to program change and improvement. For more
information on the summarization of data, see Sampling and Summarizing Candidate
Performance Information, a paper by Emerson Elliot available on the NCATE
The Transition Plan indicates that units should have a design for systematic
collection, analysis, summarization, and use of data. The unit should describe
in its plan for an assessment system what that system is expected to look
like when it is operational. When will data be collected? How will it be collected?
Who will do the evaluating? Who will summarize and analyze it? When will this
take place? How will it be done? When will it be shared, with whom will it
be shared, and what mechanisms will be in place to ensure that the data is
used to improve the programs?
- Identifies aspects of the system
that address unit operations.
While the direct assessment of candidates is important, other mechanisms can
and should be used to gauge the quality of the unit and its programs. The
NCATE 2000 standards require that the unit collect data from applicants, candidates,
recent graduates, faculty and other members of the professional community.
The collection, analysis, and use of these data must be built in to the unit
assessment system. The evaluations and surveys used to gather information
are instruments that should be revised, when necessary, to reflect the unit's
mission and philosophy. These instruments should also reflect the extent to
which the unit is meeting the learning expectations stated in the conceptual
Other aspects of unit operations that could also be evaluated as part of the
unit assessment system include the effectiveness of advisement, record keeping,
the admissions system, student teaching placement, governance structures,
In addition, provisions should be made to study the extent to which the requirements
at the various transition points are adequate predictors of candidate success.
The types of questions that might be posed by these studies include: are candidates
who barely met admissions requirements scoring as well on the assessments
as those who exceeded the requirements? Do candidates with high scores on
the assessments have higher scores on employer surveys than those who had
- Identifies how information
technology will be used in the maintenance of the assessment system.
Tracking student progress and unit operations will likely require the use
of computers and computer programs. The unit must have the capacity to retrieve
and manipulate data. The unit should describe the role of information technology
in the assessment system. The type and complexity of data management systems
will depend, in part, on the size of the unit. Small units may be able to
store assessment system data is Excel spreadsheets, while larger units may
require more sophisticated software.
Units may be at different points in the development of assessment systems;
at a minimum, all of the items listed above should be described in one document
that includes all programs.
During fall 2001 and spring 2002,
BOE members will base their assessment of Standard 2 on the extent to which
institutions have addressed the Transition Plan, which is explained in the items
above. If an institution exceeds these minimum requirements in the development
of its system, then that institution should include the additional information
in the document. This document could be the unit's response to Standard 2 in
the Institutional Report, or it could be a separate document that the unit makes
available to the team via the web or at the time of the visit.
Visits in Fall 2002 through Spring
Institutions with visits in subsequent years should be implementing their plans
as delineated in the Transition Plan. For example, institutions with visits
in fall 2002 and spring 2003 should have a plan for their assessment systems
and they should have developed some performance assessments. Rubrics/criteria
for scoring the assessments should be under development and steps or activities
to ensure fairness, accuracy and consistency should have been planned. Some
data collection should be underway.
Similarly, institutions with visits in fall 2003 and spring 2004 should have
an assessment system plan that is inclusive of the six items above. In addition,
the assessments should be fully developed. The rubrics/criteria for scoring
should also be developed. The unit should be implementing the assessment system
by using the assessments and rubrics/criteria with candidates and other members
of the educational community. Data from the assessments should be collected
and analysis should have begun. These steps should be in the timelines articulated
in the assessment system plan.
Finally, institutions with visits in fall 2004 and spring 2005 should have fully
developed systems that are being implemented, evaluated, and refined. The system
should be clearly articulated and steps or activities to ensure accuracy, fairness,
and consistency should be ongoing. In addition, institutions should have evidence
that data from internal assessments, as well as external assessments (state
tests scores, graduate and employer surveys, etc.), are being used to make changes
and improve programs.