When it comes to feedback about faculty or courses, one size does not fit all

Considering the Design of Evaluations of Faculty or Courses

I saw yet another item in the POD listserv whether and how to ‘grade’ teaching, student feedback.  Obviously, any such system requires collecting best evidence and in several different ways, "triangulating" from them to get a more valid, reliable sense of what's been going on. 

The fallacy I see in most such discussions is the assumption that one procedure must fit all kinds of courses and faculty.  (The IDEA Center is a noteworthy exception to 'one size fits all.')

Before getting to judging faculty, let's be clear: the most important target for intensive evaluation is the performance of academic programs (e.g., degree programs, general education, writing or other essential learning outcomes as they are mastered across the curriculum).

• What should students have learned by the time they leave (not just in content and capabilities but also in motivation and perspectives)? 
• What did students learn by the time they left? Compared with the goals? compared with students graduating five years earlier?
• Have equity gaps increased or decreased for students who have been in the program?  
• Has the program made responsible, effective uses of its resources including students' time and money?

Now, let's talk about making judgments about faculty. We can learn more, and waste less time and money, by having different inquiries for different situations

	Purpose -Formative	Purpose - Summative	Sample criteria
The top educators  (or courses) (~5%)	Cross-fertilization among most engaged innovators, SoTL practitioners. Forge direction and leadership cadre for future programmatic improvements	Rewards of distinction	In what ways has your work benefited the practice of other faculty and staff, here and at other institutions?
Broad middle (90%)	Provide feedback useful for faculty growth and course improvement	Use evidence of growth or no growth over the years as one of many inputs into personnel decisions	Use highly engaging practices? With desired effects?
Bottom of the scale (~5%)	Identify and then fix problematic teachers/courses if possible	Identify teachers/courses performing at unacceptably low levels, term after term.	Is instructor showing up? Providing feedback in a timely fashion?

The "broad middle" is the most important group because they do most of the teaching and have far more impact, collectively, on students. Because so many faculty are in this group, any central committee or office will be able to spend comparatively little time with each instructor, teaching assistant, and learning assistant.  So these kinds of feedback need to come from standardized processes (e.g., standard data about course performance; standard review from peers who have been educated about what's worth noticing and how to coach).

The bottom of the scale is the most important for a "high touch" approach because of the risks that the intervention might make things worse.  Each case requires sensitivity and a unique approach.

The top of the scale is also worth high touch, to harvest, interpret and disseminate important lessons and new challenges arising from the work done in these very best courses.

Does your institution have a faculty/course improvement program that treats the worst performers differently than the others -uses different criteria to judge them, gathers different data about them?

The Book in Progress

As you might know, I'm working on a book tentatively titled, Quality, Access, and Affordability: Pursuing 3Fold Gains in Higher Education.  About 2/3 of the planned book is in at least first draft shape.  The book describes 5-6 institutions that have been working for years to enhance how well students learn, to make access more equitable, and to control affordability in time and money, for the students, the institutions, and their benefactors. Even more important, these institutions have all been trying to accomplish such gains in ways that are scalable (engage more than boutique numbers of students) and sustainable for many years to come.

If you'd like to hear more, write a comment below or email me (ehrmannsteve gmail).