Assessment - Improving Learning AND Accountability?

In a recent op-ed in the New York Times, "Testing the Teachers," David Brooks contrasted the increasing price of higher education with the evidence that results are getting worse.   For example, he cited studies showing that the reasoning ability of graduating seniors is disappointingly poor, significantly worse than a couple of decades ago. A co-author of one of those studies, Josipa Roksa, spoke at our Teaching Day this past October. (Click here to see a video of her talk.) People have faith that going into years of debt is worth the price. Taxpayers have faith that spending large amounts of tax money on student aid and research is justified. What if that faith begins to falter? Brooks suggests that academics study learning in order to figure out the problems, and fix them, before it's too late.

Fortunately, many faculty (like Nobel Laureate Carl Wieman who spoke here at GW last October) have been working on the same problem because of their own doubts about whether their students are actually mastering even the most elementary ideas and capabilities in their disciplines.

As Wieman showed, the problem is that hard-working faculty can spend enormous effort on teaching with methods that sometimes don't work very well.  (As a rueful Gregory Peck remarked in "Twelve O'Clock High," "I'm choppin' but no chips are flyin'.")

Wieman reported research that compared two sections of physics, each of about 270 students.  In one, students were taught by an experienced faculty member using traditional, respected methods of instruction. The other section was led by an inexperienced GTA who had been trained to use new methods that continually challenged students to think and work together throughout the class rather than silently taking notes.  The worst of the GTA's students learned as much as the best students learning from traditional methods.  Attendance in the GTA's section was substantially higher, too.   Wieman's study appeared in Science on May 13, 2011 (p. 862-864)  (Click here to see a video of Wieman's talk.)  These kinds of results are not unusual, but, when a Nobel Laureate does the research, it tends to draw attention to the new possibilities.

To oversimplify, here are a few ideas, all of which are already being tried by at least a few GW departments:
1. Figure out what capabilities your program is trying to develop in all your students by the time they graduate.  For example, an engineering program might be trying to educate students who can figure out a design that can solve a problem, convert that design into a production plan, produce a product and test it.
2. Don't expect that one course can develop one capability by itself ("We need our graduates to be ethical so we will require them to take an ethics course.")
3. Assign students projects and other activities in course after course: activities that reveal whether those capabilities are actually developing satisfactorily.  Use capstone courses, senior projects, and student portfolios to see whether, by graduation, students have learned what they need to learn.
4. Where many students are having problems making progress, don't get waste energy on the blame game. Instead find different ways to teach them. While you're at it, look for methods that can also save faculty and students time.
5. Of course, each student will also learn many things that beyond those core capabilities. Those same student projects and activities (#3) can help the student and faculty see how each student's distinctive achievements (and problems). Taking a good look at that evidence may suggest ways of improving this aspect of students' education, too.

I heard about an example of these ideas in action a few days ago when I talked with Prof. Jay Shotel, Chair and Prof. of Special Education and Disability Studies in GSEHD.   Jay and his colleagues have identified 14 capabilities that all their master's students in Early Childhood Special Education should master.  In each course, students do projects that help them and faculty see how they're progressing in developing those capabilities. And the students continually write about that learning ("reflection").  As part of those courses,  students are also each developing an online portfolio -- a collection of their projects that, along with those reflections, documents the cumulative development of those 14 capabilities.  They present their portfolios as a requirement for graduation. And they can use that evidence to help get jobs.  (Jay, did I get this right?)

PS. I was also fascinated to see that this GSEHD program is a 'hybrid,' i.e., it combines learning at Foggy Bottom, online learning, and clinical experience at off-campus sites.  If the Innovation Task Force approves our plans, we'll be selecting a couple of faculty teams each year that want to take advantage of the hybrid format in order to develop world class master's and graduate certificate programs.  We will support the winning proposals with grants and expert consulting help.  Let me know if you'd like to learn more about these ideas.