Guidelines for
Writing the Evaluation Summary
For questions
contact:
Curriculum Evaluator
Julie Foertsch, Ph.D., foertsch@wisc.edu
1. Keep the goals of the
Evaluation Summary and the Committee
in mind:
The purpose of the Evaluation
Summary is to clearly and concisely identify the aspects of a course
that, if adjusted or improved, would lead to a more effective learning
experience for a greater number of students. The document needs
to be well-written, constructive, and persuasive if it is to succeed
in encouraging course directors to listen to students’ perspectives
on how they learn best and make improvements to their courses that are
mindful of students’ needs. As an Evaluation Committee member,
you have a chance to represent the views of your peers and make a real
difference in how a particular medical school course evolves over time.
The faculty, the administration, and your fellow students appreciate
your giving this important task your time, your thoughtfulness, and
your problem-solving skills. By participating in an Evaluation
Committee, you will be developing your own competency in the ACGME areas
of Systems-Based Practice, Problem-Based Learning and Improvement, and
Professionalism, skills that will serve you well throughout your medical
career.
As an Evaluation Committee
member, your job is to be a course reviewer who:
uses the available
resources to gain perspective on a course’s strengths, weaknesses,
and logistical constraints;
accurately represents
the varied opinions of the students, not just your own opinions;
is respectful and
constructive in your writing and commentary;
is responsive to
requests from your fellow committee members and timely in meeting your
committee’s deadlines.
2.
Review all the information available and summarize
the balance of what you have learned:
You will be receiving a complete
course evaluation report about two weeks after the course’s final
exam that summarizes the ratings for each rating question and lists
all the responses to each text question, ordered by topic (see Appendix
for a complete list of the questions). Although you will only
be responsible for attending to the two sections of the report that
match your subcommittee assignments, you will find it useful to look
at the ratings in other sections and perhaps even skim the comments
in sections that seem related to your sections. After you have
an overall sense of how the course was evaluated, you should focus on
the two sections to which you have been assigned to write a summary.
The comments will already be
ordered (to the extent that they can be) by the topic they cover so
that comments which say something similar are clustered together.
A suggested process for writing the section summaries is:
Read
all the comments for a section, jotting notes about the theme or main
point of a given cluster of comments. This gives you a sense
of the main issues you will need to discuss.
Note how many
students felt a certain way, giving the numerical ratings somewhat
more weight than the written comments, since not all students will choose
to comment on the same topic, but most do give as rating. When summarizing
comments, your summary should prioritize the opinions of the majority
over comments made by only a small number of students, but it is not
unusual to have fairly large numbers of students with opposing opinions
about the same issue. For example, you may have 25 students who
thought the course book was way too detailed and found themselves overwhelmed
and unable to discern the key points, 15 who liked having the details
but wanted more guidance on which parts to focus on for an exam, and
5 who loved the book and relied upon it more than any other resource.
Your summary should reflect this difference of opinion and look for
a resolution that will help the greatest number of students get what
they need. For example, if the course director used highlighting
within the coursebook or a clearer, more concise set of course objectives
to draw students’ attention to key sections, the needs of all three
groups of students could be met.
Draft
some sentences about the key issues and
approximately how many students felt that way (almost all, the vast
majority, the majority, some, a few). You can state the actual
number of students who commented on an issue if you think that is helpful,
but keep in mind that not all students who feel a certain way bother
to comment. (Note:
If you are on the Overall Course Effectiveness Committee, Question 28
is the one for which listing the actual
% of commenters who made each suggestion
is most useful.) Comments that are made by a very small number
of students shouldn’t make it into your summary unless the points
are particularly good ones or the suggestions especially effective and
easy to do. In those cases, you should make it clear that was
a minority opinion by saying something like “A few students thought
it would be helpful if” or “one student suggested.”
Write objectively
and in the third-person. Remember, you are representing all
students who rated or commented on this course, not just stating your
own opinions.
Write a one-paragraph
summary of each section you are reviewing, keeping in mind that
a paragraph can be as short as a 1-2 sentences if students had few suggestions
for improvement or were all in agreement on what to do. Give enough
detail so that course directors know why a particular aspect
of the course is problematic, but there is no need to belabor a point
when they can read the comments for themselves. If many students
comment about how strong that aspect of the course is, you should mention
that and why students think it is helpful, but focus your
summary on areas where improvements can be made. For each
statement you make about something that students had trouble with, offer
their suggestions for improvement. You can offer a suggestion
of your own if it aligns with the majority opinion and would address
the issues of most students.
Reread your summary
and edit it for grammar, constructive word choice, and concision.
A well written, carefully worded summary is much more persuasive than
a haphazardly written, harshly worded one.
Share your summary
with your 1-2 subcommittee members by the
end of the week. Your subcommittee then has a week to read
the summaries of each member, see where they align and misalign, and
choose/construct a summary that incorporates the key issues or best
suggestions from all the summaries. This
paragraph-long consensus section summary
should be emailed to the Committee Chair by the section summary deadline.
Here’s an example of what
a section summary should look like:
Course Materials:
Many students commented on
how helpful it was to have all of the course materials available online.
Of the 55 students who commented on the coursebook, the majority felt
it contained too many details whose importance was unclear. They
said they had a hard time keeping up with the reading and thought the
coursebook’s key points were lost among the details. A significant
minority said they enjoyed having those details available, but even
they wanted more guidance on which parts to focus on for exams.
Suggestions for how to improve this included highlighting key concepts
in the coursebook or clarifying through the course objectives or lectures
which processes were the most important to memorize in detail.
3. Use the responses to
the final evaluation question and the section summaries to discuss and
write the 1-3 priorities for course improvement.
During the Course Evaluation
Committee Meeting (or “meeting period” if the committee cannot actually
meet and this work is done online), the committee convenes to discuss
what was learned by reading the course evaluations. All committee
members should have read the compilation of section summaries and Medical
Advisor’s notes sent out by the Committee Chair. The Chair then
facilitates a discussion about the top priorities for course improvement.
Each section summary will be a part of the final report, so committee
members needn’t be concerned that the suggestions from their section
won’t be seen if they are not considered one of the top priorities.
Nonetheless, the group does need to discuss and decide upon 1-3 suggestions
for improvement that could have the biggest impact for the greatest
number of students. Generally, these suggestions will address
issues that interfered with learning for a significant number of students,
but they can also be suggestions that, while mentioned by only a minority,
are simple, effective adjustments to make. The job of the Evaluation
Committee is not only to draw attention to the course aspects that were
the most problematic for learners but to offer reasonable and well-considered
suggestions for how to address those issues.
Hence, when writing the priorities for course improvement, briefly describe
the problem and why it’s a problem, but then focus on one or two ways
that problem could be solved.
After the Evaluation Committee
has reached a consensus as to what the priorities for improvement should
be, the Committee Chair will draft a clear statement of those priorities
and put it at the end of an Evaluation Summary that follows the format
described below.
4. Follow a standard
report structure for the Evaluation Summary:
The Evaluation Summary written
by each course committee should use a similar structure and formatting
in order to maintain consistency across courses and make it easier for
course directors to find the information they need for their course
review. On the next page is the outline and formatting that all
the Evaluation summaries should use. Like this guideline, your
report should be a Word document in 12-point Times New Roman.
An electronic copy of these guidelines and a report template that you
may copy and paste is available on the Medical Students Website under
Academic Resources/Course Evaluation/Guidelines for Writing the Evaluation
Summary.
Evaluation Summary for [Name of Course]
Fall [or
Spring] of 20XX
This report, provided by the
course’s 1X-student Evaluation Committee, summarizes the evaluation
comments received from XX students [the total N stated in the evaluation
report] who took the course in the Fall of 20XX. The purpose of
this summary is to provide guidance in making course improvements that
will have positive impacts on student learning and the overall effectiveness
of this course. First we provide summaries of the comments received
on eight key aspects of the course, followed by a list of the most popular
suggestions for the one change that would have the biggest impact on
overall course effectiveness. We conclude with the Evaluation
Committee’s consensus on the top X priorities for course improvement,
which should be taken into account when making adjustments for next
year.
Summaries of
student suggestions regarding various course aspects:
Learning Objectives:
One paragraph summaries submitted
by each subcommittee
Content:
Course Materials:
Learning Activities:
Assessments:
Schedule:
Faculty:
Integration:
Overall Course Effectiveness:
Top Priorities for Course
Improvement:
After discussing the strengths
and weaknesses of the course and the suggestions made above, the Evaluation
Committee feels the following changes would have the most positive impact
on the effectiveness of this course:
Reduce the number
of…
UWSMPH
COURSE EVALUATION QUESTIONS
To be completed online via
OASIS between the day of the final and 7 days later. Students
will not be able to see their final exam grade online until it is completed.
The scale for all non-text
questions:
Strongly AgreeStrongly
Disagree
1234567
Text questions are in bold.
LEARNING OBJECTIVES
The learning objectives
clearly stated the important concepts that I should learn.
The learning objectives
were at an appropriate level of detail.
How could the
objectives be more helpful in directing your studying and learning?
CONTENT
The content was
presented in a logical sequence that facilitated learning.
Each topic was allotted
an appropriate amount of time.
I understood the
relevance of the content to medical practice and/or research.
How could the
presentation and organization of course content be improved?
COURSE MATERIALS (includes
all written materials like coursebooks, texts, online references)
The course materials
clearly explained important concepts.
The course materials
are well-edited, with few errors, good formatting, and logical organization.
Please suggest
improvements in the course materials.
LEARNING ACTIVITIES
Taken as a whole,
the course’s learning activities helped me to understand the important
concepts.
This course actively
engaged me in the content through learning activities such as problem-solving,
hands-on learning, and peer discussions.
The learning activities
gave me insight into how I will apply that learning to clinical practice
and/or research.
How could the
learning activities be made more effective?
ASSESSMENTS
The assessments
measured my understanding and application of important concepts.
To be successful
in this course, I needed to demonstrate an understanding of the content
that went beyond the memorization of facts.
I was given a reasonably
clear idea of what to expect on the assessments.
Please suggest
ways that the assessments could promote more effective learning of course
content.
SCHEDULE
Course activities
were scheduled in a way that facilitated my learning.
My workload during
this course was manageable.
Please suggest
improvements in the course schedule.
FACULTY
The teaching in
this course enhanced my understanding of course concepts.
Questions were
welcome and help was available if I had difficulty with course material.
Please suggest
ways for improving the effectiveness of the teaching in this course.
INTEGRATION
This course was
well integrated with related curriculum offerings.
What
topics in this course were too redundant with other parts of the curriculum
or not linked effectively to related topics in other courses?
OVERALL COURSE EFFECTIVENESS
Overall, this course
provided a good learning experience.
What one improvement
would have the biggest impact on the effectiveness of this course?