February 2002
Jeroen Breman Wins Outstanding
Practice in an International Setting
Award by a Student of Educational Technology
1. What attracted you to apply for
the award?
Actually, I did not apply for the
award but was nominated by a former
President of AECT, Dr. Kyle Peck. As my advisor at the Pennsylvania
State
University he knows about my work at the World Wide Fund for nature
(WWF)
College of Conservation Leadership, and he pointed out the award. What
attracted me about the award is that a group of your peers at the AECT
International Council recognizes the good work that we do.
2. Could you give us a brief
description of your practice submitted
for the award and describe what made it an exemplary project?
What
makes it an applied educational communications and technology practice?
The WWF College is an interactive
learning network. It combines
new technology with traditional face-to-face training and offers a
unique
learning environment. Participants learn from each other, and
analyse
and improve their own performance to become better leaders and thereby
achieve greater impact in their work. The College promotes the
ideas
of self-directed learning, peer learning, learning on the job and
continual
improvement.
Participants enrol in an 18-month
programme designed to upgrade competencies
in four overall themes:
· Leadership & Management
· Strategy & Planning
· Campaigning & Advocacy
· Communications & Networking
Each year, through a competitive
selection process that involves both
the applicants and their supervisors, two groups of 25 participants,
balanced
in terms of geographic regions, gender and functional specialities,
start
the Conservation Leadership programme. The programme begins with a one
week workshop which sets the stage for the rest of the 18 months.
Through role plays, exercises and discussions, this interactive
workshop
introduces the themes of leadership, communications, and information
communications
technology. Participants evaluate themselves on the key
competencies
within each of the four themes, and then in consultation with a coach,
plan a customised study programme based on their needs. This week
also provides an opportunity to get all participants familiar with the
College's on-line learning environment, the College Campus.
3. How did you design and develop
your practice and what was the
nature of the performance data regarding its effectiveness?
I began my work with the WWF halfway
through the trial period of the
college. Four groups had started the 18-month program and were getting
ready for their mid-point workshops. Online performance was poor for
several
reasons, including technical and motivational domains. Just
before
I started, a new version of the online learning environment had been
completed,
and was much better than the previous version, but many problems
remained.
At the time of my arrival, the content
of the program was not completely
developed, and what was developed was not always very well aligned with
participants needs. Participants were not prepared for Distance
Learning,
and some were probably not really representative of the target
population
for which the program was designed. Participants were generally
either
asked to join by their supervisors without an intrinsic desire to take
the courses, or were too experienced, or too busy to study while on the
job.
Because of the problems inherent in
the system he inherited, we had
to beg people to return for their second workshop, telling them it was
very important for them to share their negative experiences with peers
and course designers, and convincing them that it was still
possible
to finish the program. Even with this effort to retain students for
multiple
courses, only 50 out of the original 70 participants attended their
follow-up
workshop, and only 30 finished the minimum requirements and graduated
with
a Certificate of Conservation Leadership. Others did benefit from the
program
since they attended the second workshop and selected online modules
that
helped them in their jobs.
In my first year at WWF, the focus was
on finishing the total18-month
program, including the re-design of the follow-up workshops, and
increasing
the graduation rate among the initial cohort of students. We
identified
problem areas and ways to solve them.
In September 2001 a new group of 25
new participants were starting the
program. To improve the overall success rate, several new steps were
initiated.
Before arriving for their Kick-off workshops the participants went
through
a new application process, which included more information, a test of
learning
styles, opportunities to talk to alumni and current participants, an
optional
intake interview by telephone, and the requirement to have full support
from their supervisors who were required to agree to do whatever is in
their power to provide the opportunity for the participant to finish
the
program.
A Study Guide including descriptions
of the program, the underlying
competences, descriptions of the way they were going to learn, and
descriptions
of the online modules including learning objectives for each. The Guide
also included a self-assessment tool to assess the level of experience
in the different competences underlying the different modules as well
as
the perceived importance of these competences for their jobs. This
assessment
then formed a basis for the development of a Personal Learning Plan
that
students would start to develop during the Kick-off Workshop.
The team learned that for an
inexperienced group of distance learners,
it was best not start with the most controversial and difficult topic
in
the first online module. We now start with a well-liked and highly
valued
module on Effective Communication. Of the group that started in Costa
Rica
in 2001, 95% finished that module (80% on time). After that good start,
they were able to offer a relaxed schedule in which there are always 8
to 10 weeks to finish a 6 week online module. We adjusted requirements
to a more realistic level, given the other demands in the learners’
lives.
We also discovered that many of the mandatory modules appeared not to
be
of importance to everyone, as the backgrounds of the participants is
very
diverse. (Our learners come from roles in conservation,
communications
and marketing, and financial management.) We rearranged the curriculum
so that the certificate is earned by successfully completing only two
mandatory
modules and five elective modules, and many students now elect to
complete
more than five.
Another improvement in the WWF
curriculum was inspired by an AECT roundtable
by Lya Visser in Long Beach, CA, in February of 2000. Her dissertation
was on supporting distant learners, and Jeroen used some of her results
to formulate a support structure for WWF’s online learners. All staff
members
of the WWF College now follow the progress of a group of participants.
Progress is checked at least every 2 weeks so that problems are noticed
quickly and discussed by email or phone. Once a month the progress is
reported
to the mentors, who are also notified if serious problems seem to
occur.
When nothing else seems to help, the supervisors are the last resort,
since
they promised to create the necessary opportunity for the participant
to
do their College work.
Communications to and from
participants are now recorded in a small
knowledge management system as part of the online learning environment.
This makes it possible for any staff member to quickly look up
background
information on a participant that might be useful in monitoring a
person's
progress.
In the second year (which started with
the Costa Rica group) the focus
shifted to improvement of the content of the modules. The previous
materials
included lots of texts and lots of assignments. However, it was very
clear
that very little instructional design had been involved. By now,
almost all modules have been revised by Instructional Designers and
development
of new materials is always coordinated by instructional designers
(in-house
or consultants). By identifying the learning objectives, providing
text,
cases, and examples that explain the underlying concepts and
procedures,
and by making sure that assignments measure the required behaviour, and
providing clear directions the learning experiences were transformed.
Many lessons have been learned.
And many principles of instructional
design have proven valuable. Last June 2002 the Costa Rica cohort
came back together for their Follow-up Workshop in Vienna, Austria.
What
a wonderful change from the workshops the WWF College had experienced
with
the pilot! Twenty-two of the 24 participants showed attended the
Follow-up session. One was on sick-leave, one dropped
out.
All of them finished their two core modules and did at least 1 elective
module online, and many of them already finished 2 or 3. They shared
success
stories and focused on the content of the workshop instead of crisis
management.
4. What did you learn from the
award process?
I see the award from AECT’s
International Council as recognition and
a reward. Our gut feeling tells us that we have a great world-wide
programme
of which many not-for-profit organization as well as businesses would
be
jealous of. This is acknowledged by a group of my peers in the field.
Now
we just need the empirical evidence, see below.
5. What are your practices now?
At all times now, between 70 and 75
people are enrolled in the WWF College.
We start two new groups a year, two groups return for their mid-term
workshop
and two groups graduate. We are currently implementing an evaluation
study
that will evaluate the programme at five levels: Kirkpatrick’s 4 levels
of evaluation, from satisfaction to business impact, and Jack Phillips’
additional level, the return on investment (ROI). Pat Phillips in
overseeing
this evaluation executed by a Dutch business partner. The goal is not
only
to measure the impact and ROI of the programme but also to design a
method
to structurally include evaluation in the programme, as this is too
often
done on an ad hoc basis. We hope to present about the design,
implementation
and results of the evaluation at the next AECT Conference.
