|
|
15. How does the Program Review
process work once the Program Report is submitted to NCATE?
The program reviews and proposals
prepared by the various institutions or states are reviewed by a team of program
reviewers trained by the AECT Accreditation Committee. The program reviews or
state proposals are forwarded by NCATE to the AECT Program Review Coordinator
who distributes each document to three program reviewers. The reviewers independently
evaluate each program/proposal, noting its strengths and weaknesses using the
NCATE critique form and determine the compliance of the program/proposal. The
reviewers do not know which other individuals are reviewing the same program.
The Program Review Coordinator does not actually evaluate any program reviews
but serves in a coordination role. Approximately three weeks elapse while the
program reviews are being independently evaluated.
When the reviews are completed they are returned to the Program Review Coordinator.
If two or more reviewers agree in their evaluation of a program/proposal, then
that recommendation is sent to NCATE. When there is disagreement between the
readers the Coordinator conducts a phone interview with the program reviewers
in an effort to reach consensus. If there is still disagreement among the three
readers then the materials are sent to three new readers and the process described
above is repeated.
Once there is agreement between at least two reviewers the Program Review Coordinator
compiles the three reviews into one report. This report, including recommendations,
is then returned to NCATE for subsequent return to the institution.
Program Reviewer Selection and Training
All program reviewers participate in a training seminar. The Association also
holds training sessions for institutions preparing AECT program reviews. These
sessions are held during the annual AECT convention so that they alternate with
the program reviewer training sessions. Institutions that are preparing to submit
an AECT program review are invited to send a representative to these sessions.
Selection. An invitation to apply for positions as Board of Examiner
(BOE) site-visitors and Program/State Partnership Reviewers is distributed to
all AECT members through the organizational publications, Tech Trends and Educational
Technology Research and Development, and on the AECT web page (http://www.aect.org).
Nominations are also sought from current reviewers, committee members, and the
AECT leadership. Candidates are asked to submit a vita to the AECT Accreditation
Committee. Those who respond are sent a letter specifying the duties and responsibilities
of the positions and inviting them to attend the session on accreditation training
at the next AECT conference. The vitae are evaluated by the committee and invitations
to participate as program reviewer and/or BOE member are issued.
Training and Evaluation of Reviewers and Responses to Standards. The
BOE training is provided by NCATE and the site visits are coordinated by the
NCATE Office. Once the request for BOE trainees is received by AECT, the names
and vitae are submitted and all further contact for training and site visits
is handled through NCATE.
The purposes of the program review/state partnership training are to prepare
program/state partnership proposal reviewers to evaluate the state proposals
and programs in educational communications and instructional technologies (ECIT);
to provide guidance for teacher preparation institutions and states that are
developing or updating ECIT programs; and to assist applicants in preparing
program reviews/proposals for the National Council for Accreditation of Teacher
Education (NCATE) program evaluation.
The program/proposal review training and procedures are directed by the AECT
Accreditation Committee. Program reviewer training is managed by the Program
Review Coordinator, a member of the Accreditation Committee. The training session
for new program reviewers is conducted every other year in conjunction with
the annual AECT convention. The nature of this training includes presentations,
simulation, role playing, and review of AECT/NCATE standards. The session is
conducted by the Chair of the AECT Accreditation Committee, assisted by one
or more other members of the committee. Training session participants receive
materials beforehand with which they are asked to familiarize themselves (the
standards, the policy and procedures for program approval, and a sample program
documentation). During training, the role of program review within the NCATE
accreditation process is explained. Then the NCATE approved standards for AECT
programs are reviewed and discussed. Expectations for program consistency with
the standards also are discussed, as are common explanations and responses by
programs, as well as potential sources of evidence of consistency with the standards.
Sources of confusion regarding standards and program responses are explored.
Examples of actual program responses to standards, blinded for identification,
are reviewed with the participants.
A sample program review is distributed and evaluated by the participants. The
practice evaluations and the sample program review are then discussed by the
trainers. Individual participants receive feedback regarding their practice
evaluation. Finally, there is a discussion of issues that emerge during the
practice program review process.
Upon the conclusion of the session, the performance of the participants is evaluated
by the trainers. Although it happens very rarely, some individuals could be
and have been eliminated as reviewers at this point.
Upon completion of the training, readers are given a certificate/letter indicating
that they have completed reviewer training for particular sets of guidelines
and stipulating the date by which they must renew their training (based on renewal/revision
of guidelines).
When the Program Review Coordinator distributes proposals/program reviews, newly
trained reviewers are grouped with experienced reviewers so that comparisons
can be made regarding elements that may not be clear to particular individuals
and that may require follow-up training. The term of service for reviewers is
three years, but experienced reviewers are encouraged to serve again. Reviewers
who do not respond to reviews appropriately or do not submit evaluation materials
on time will not receive additional reviews at the discretion of the Program
Review Coordinator and will not be invited to continue as reviewers. This decision
is made by the Program Review Coordinator in consultation with the Chair of
the AECT Accreditation Committee.
Procedures for Evaluation. The NCATE/AECT program and state review process
(including development of procedures and the training of program reviewers)
is directed by the AECT Accreditation Committee under the auspices of the AECT
Board. The procedures for program evaluation are as follows:
- Institutions and states submit
accreditation documents to NCATE.
- NCATE forwards ECIT program reviews/proposals
to the AECT Program Review Coordinator.
- The Program Review Coordinator
assigns three independent reviewers to evaluate each program/proposal.
- Reviewers use the appropriate
rubrics and forms (see Appendix E) to either evaluate programs or determine
the alignment of state standards.
- The Program Review Coordinator
compiles reviewer responses and writes the final report which includes the
evaluation recommendation.
- The report is submitted to NCATE,
with a copy to the Chair of the AECT Accreditation Committee, and one copy
is kept on file by the Program Review Coordinator.
- Notations are made regarding the
consistency among the three evaluations and of any obvious misunderstandings
of standards that should be addressed with the individual reviewers or in
future training sessions. This information is conveyed to reviewers and to
the Chair of the AECT Accreditation Committee.
- NCATE reviews the reports and
forwards them to the university or state.
- A summary of results of all reviews
is reported yearly to the AECT Accreditation Committee and to the AECT Board.
- AECT submits an annual report
of reviewer data to NCATE.
If the institutional program/state
proposal is not approved, a rejoinder may be filed by the institution or state
to make further explanations, add documentation, or otherwise address areas
of weakness. When NCATE receives a rejoinder it is forwarded to the appropriate
specialty organization for a second review.
|
|
