Assessment - Improving Learning AND Accountability?

In a recent op-ed in the New York Times, "Testing the Teachers," David Brooks contrasted the increasing price of higher education with the evidence that results are getting worse.   For example, he cited studies showing that the reasoning ability of graduating seniors is disappointingly poor, significantly worse than a couple of decades ago. A co-author of one of those studies, Josipa Roksa, spoke at our Teaching Day this past October. (Click here to see a video of her talk.) People have faith that going into years of debt is worth the price. Taxpayers have faith that spending large amounts of tax money on student aid and research is justified. What if that faith begins to falter? Brooks suggests that academics study learning in order to figure out the problems, and fix them, before it's too late.

Fortunately, many faculty (like Nobel Laureate Carl Wieman who spoke here at GW last October) have been working on the same problem because of their own doubts about whether their students are actually mastering even the most elementary ideas and capabilities in their disciplines.

As Wieman showed, the problem is that hard-working faculty can spend enormous effort on teaching with methods that sometimes don't work very well.  (As a rueful Gregory Peck remarked in "Twelve O'Clock High," "I'm choppin' but no chips are flyin'.")

Wieman reported research that compared two sections of physics, each of about 270 students.  In one, students were taught by an experienced faculty member using traditional, respected methods of instruction. The other section was led by an inexperienced GTA who had been trained to use new methods that continually challenged students to think and work together throughout the class rather than silently taking notes.  The worst of the GTA's students learned as much as the best students learning from traditional methods.  Attendance in the GTA's section was substantially higher, too.   Wieman's study appeared in Science on May 13, 2011 (p. 862-864)  (Click here to see a video of Wieman's talk.)  These kinds of results are not unusual, but, when a Nobel Laureate does the research, it tends to draw attention to the new possibilities.

To oversimplify, here are a few ideas, all of which are already being tried by at least a few GW departments:
1. Figure out what capabilities your program is trying to develop in all your students by the time they graduate.  For example, an engineering program might be trying to educate students who can figure out a design that can solve a problem, convert that design into a production plan, produce a product and test it.
2. Don't expect that one course can develop one capability by itself ("We need our graduates to be ethical so we will require them to take an ethics course.")
3. Assign students projects and other activities in course after course: activities that reveal whether those capabilities are actually developing satisfactorily.  Use capstone courses, senior projects, and student portfolios to see whether, by graduation, students have learned what they need to learn.
4. Where many students are having problems making progress, don't get waste energy on the blame game. Instead find different ways to teach them. While you're at it, look for methods that can also save faculty and students time.
5. Of course, each student will also learn many things that beyond those core capabilities. Those same student projects and activities (#3) can help the student and faculty see how each student's distinctive achievements (and problems). Taking a good look at that evidence may suggest ways of improving this aspect of students' education, too.

I heard about an example of these ideas in action a few days ago when I talked with Prof. Jay Shotel, Chair and Prof. of Special Education and Disability Studies in GSEHD.   Jay and his colleagues have identified 14 capabilities that all their master's students in Early Childhood Special Education should master.  In each course, students do projects that help them and faculty see how they're progressing in developing those capabilities. And the students continually write about that learning ("reflection").  As part of those courses,  students are also each developing an online portfolio -- a collection of their projects that, along with those reflections, documents the cumulative development of those 14 capabilities.  They present their portfolios as a requirement for graduation. And they can use that evidence to help get jobs.  (Jay, did I get this right?)

PS. I was also fascinated to see that this GSEHD program is a 'hybrid,' i.e., it combines learning at Foggy Bottom, online learning, and clinical experience at off-campus sites.  If the Innovation Task Force approves our plans, we'll be selecting a couple of faculty teams each year that want to take advantage of the hybrid format in order to develop world class master's and graduate certificate programs.  We will support the winning proposals with grants and expert consulting help.  Let me know if you'd like to learn more about these ideas.

One Reason Why Faculty Resist

I've been enjoying helping Natalie Milman by writing contributing several "Ends and Means" columns for the magazine Distance Learning this year.  Today I'm submitting my most recent contribution (and my last one for a while) called "One Reason Why Faculty Resist."  It begins:

"Have you ever heard the phrase “resistance to technology” used to imply that some faculty are irrational dinosaurs? I have, and I don’t like it. In my experience, most such resistance is quite reasonable.  The following story about online discussion in real time suggests what worries these instructors."

My story's point is that, when the terrain of teaching and learning changes, faculty are quite likely to encounter unexpected problems in their courses.  I don't just mean technical problems.  I mean problems with teaching and learning that are frustrating, embarrassing and sometimes potentially threatening. And, when they encounter such problems, they may well blame themselves. And they may feel that student reaction may put them at risk.

We know all this. But most institutions do little or nothing to prepare faculty for those problems.

So one reason for faculty "resistance" is that they sense that the effort to get them to teach online is a bit of a con game: "Come on in, the water's fine!"  Young technology staff,  leading training workshops, probably aren't aware of the problems. And, when workshops are led by faculty enthusiasts, they often  paint a rosy picture because they discount once-painful problems and don't want to scare their colleagues away.

My column concludes with some suggestions for how to organize self-sustaining, scalable, inter-institutional faculty conversations about teaching a particular course (e.g., "Econometrics 202; ). Their online and face-to-face discussions should be comparatively brief, brisk, relaxed, and helpful enough (trading tips, insights and moral support about what happened last week) that faculty would look forward to next week. That's the theory. Perhaps we can start a few such groups from GW.

If you'd like to learn more, the column should be published in a few months. Or contact me and I'll send you the draft.