"Technology as Lever": the story of my most influential article is as instructive as the article itself

"Implementing the Seven Principles: Technology as Lever," written by Art Chickering and me, is undoubtedly my most influential article - cited thousands of times.

• The article begins by listing the Seven Principles, in case you're not already familiar with them. They were formulated by Chickering and Gamson in the mid 1980s as a summary of teaching & learning activities that research showed were particularly powerful.

Personally, I've been influenced almost as much by how the article came to be written.

PROLOGUE

For technology to influence educational outcomes, it isn't enough to have and use the technology. That technology must be used to carry out an activity that is, even without that technology, educationally powerful. The question should not be"Does technology X improve learning outcomes." Instead it should be "When technology X is used to support a particular teaching and learning activity, does that activity improve learning outcomes?"

So what might those powerful teaching and learning activities be?

At conferences and in campus visits, I started asking faculty what technology-assisted activities seemed most powerful and most likely to have widening use in the coming years.

Eventually, I came up with a list of about seven such patterns of activity (e.g., active learning, working in teams on a realistic and motivating project). A little later, my colleagues Robin Zuniga and Trudy Banta pointed out that my list paralleled Chickering and Gamson's "Seven Principles of Good Practice in Undergraduate Education." So I started using the Seven Principles instead because of the extensive research showing their effectiveness.

Robin and I used that insight to develop an item bank for student surveys on their uses of technology for specific activity: The Flashlight Current Student Inventory. Thanks to Gary Brown and his colleagues at Washington State University, an early online survey tool was created to manage and extend those survey questions. Use of the Flashlight Online tool spread eventually to a couple hundred institutions.

GENESIS
In the mid 1990s, I bumped into Art Chickering. He knew of how I was using the Seven Principles. "Why don't we write an article together about how to use digital technology to implement them in undergraduate education?" he asked. .

I was highly skeptical. Examples of technology enhancing seven principle activities seemed so self-evident. I literally could not imagine anyone reading or valuing such an article. Chick persisted so I asked him to send me a first draft. If, when I'd read it, I thought I could help, I'd send him a rewrite. He did, and then I did.

Eventually, our article appeared in the paper Bulletin of the American Association for Higher Education, where I worked. It was 1996 and AAHE had just created its first site on the World Wide Web. There was very little content on it yet. I suggested to the websmith that they link the text of our article to the AAHE home page.  The initial URL: http://aahe.org/ehrmann.

VIRUS
Readership started strong and multiplied each month. For at least a decade, the article drew over 4,000 hits a month. Impressive for an article I thought no one would read!

I got some insight into why the article was so influential on a visit to the University of South Carolina (USC) in the late 1990s. In a talk, I described one way in which technology could be used to implement a Principle: giving rapid, useful feedback to students. When I was in high school, teachers had given the same comment on papers so many times that explanations and suggestions had disappeared. If a passage seemed awkward to the teacher, they'd write "K" in the margin. Not helpful!

 Today, I said, USC faculty can ask for student work to be submitted electronically. Then they can use the "Comments" feature to type detailed feedback about specific elements of the student's work. Better yet, the next time you want to make a similar comment, you can copy the initial version of the feedback and, if necessary, adapt or improve it. The more times similar feedback is given, the better and more helpful it can become.

Here's the kicker: Later in my visit, Bill Hogue, the CIO, introduced me to another audience of faculty. Bill remarked that he'd heard my talk the previous day.Then he explained the time-saving idea for improving feedback on student work. Bill added that he'd never considered using a computer that way.

The ideas I'd thought too obvious to need explanation were in fact 1) powerful, 2) easy to comprehend, 3) easy to spread from person to person by word of mouth, and 4) could be useful to the listener no matter how much or how little relevant experience they'd had.

MORAL
I'm writing my first book in quite a few years. The hidden assumptions beneath my draft may well be the most important ones for the book to explain. Perhaps those insights will be simple and powerful enough that readers will want to share them with their friends.

When it comes to feedback about faculty or courses, one size does not fit all

Considering the Design of Evaluations of Faculty or Courses

I saw yet another item in the POD listserv whether and how to ‘grade’ teaching, student feedback.  Obviously, any such system requires collecting best evidence and in several different ways, "triangulating" from them to get a more valid, reliable sense of what's been going on. 

The fallacy I see in most such discussions is the assumption that one procedure must fit all kinds of courses and faculty.  (The IDEA Center is a noteworthy exception to 'one size fits all.')

Before getting to judging faculty, let's be clear: the most important target for intensive evaluation is the performance of academic programs (e.g., degree programs, general education, writing or other essential learning outcomes as they are mastered across the curriculum).

• What should students have learned by the time they leave (not just in content and capabilities but also in motivation and perspectives)? 
• What did students learn by the time they left? Compared with the goals? compared with students graduating five years earlier?
• Have equity gaps increased or decreased for students who have been in the program?  
• Has the program made responsible, effective uses of its resources including students' time and money?

Now, let's talk about making judgments about faculty. We can learn more, and waste less time and money, by having different inquiries for different situations

	Purpose -Formative	Purpose - Summative	Sample criteria
The top educators  (or courses) (~5%)	Cross-fertilization among most engaged innovators, SoTL practitioners. Forge direction and leadership cadre for future programmatic improvements	Rewards of distinction	In what ways has your work benefited the practice of other faculty and staff, here and at other institutions?
Broad middle (90%)	Provide feedback useful for faculty growth and course improvement	Use evidence of growth or no growth over the years as one of many inputs into personnel decisions	Use highly engaging practices? With desired effects?
Bottom of the scale (~5%)	Identify and then fix problematic teachers/courses if possible	Identify teachers/courses performing at unacceptably low levels, term after term.	Is instructor showing up? Providing feedback in a timely fashion?

The "broad middle" is the most important group because they do most of the teaching and have far more impact, collectively, on students. Because so many faculty are in this group, any central committee or office will be able to spend comparatively little time with each instructor, teaching assistant, and learning assistant.  So these kinds of feedback need to come from standardized processes (e.g., standard data about course performance; standard review from peers who have been educated about what's worth noticing and how to coach).

The bottom of the scale is the most important for a "high touch" approach because of the risks that the intervention might make things worse.  Each case requires sensitivity and a unique approach.

The top of the scale is also worth high touch, to harvest, interpret and disseminate important lessons and new challenges arising from the work done in these very best courses.

Does your institution have a faculty/course improvement program that treats the worst performers differently than the others -uses different criteria to judge them, gathers different data about them?

The Book in Progress

As you might know, I'm working on a book tentatively titled, Quality, Access, and Affordability: Pursuing 3Fold Gains in Higher Education.  About 2/3 of the planned book is in at least first draft shape.  The book describes 5-6 institutions that have been working for years to enhance how well students learn, to make access more equitable, and to control affordability in time and money, for the students, the institutions, and their benefactors. Even more important, these institutions have all been trying to accomplish such gains in ways that are scalable (engage more than boutique numbers of students) and sustainable for many years to come.

If you'd like to hear more, write a comment below or email me (ehrmannsteve gmail).

Impact of Online Teaching Upon Campus Teaching

Some months ago, with help from Camille Funk and Patty Dinneen of the Teaching & Learning Collaborative, I surveyed faculty from George Washington University who had worked with instructional designers to develop and teach online courses during the summer over the last 15 years. Our research question: had this experience influenced their subsequent teaching on campus. In a word: it did.

Faculty who had been through the program once were influenced in many dimensions of their teaching.  For example these kinds of changes in campus teaching were reported by at least half of the respondents:

My syllabus, instructions and directions for students are more clear and complete.

Development tools I learned about for summer I now sometimes use for my campus course materials.

I've re-used or adapted materials from my online course.

I use images, animations or video.

I've started designing a campus course or assignment by first figuring out what students should be able to do as a result.

I assign online discussion among my students.

Faculty who had participated two or more times were influenced much more.

I've heard anecdotal reports of such influence for years but we got a 53% response rate. Of those respondents, 85% reported influence on their campus teaching in at least one dimension.

Here's my full report and the survey that was sent to faculty.  Feel free to use or adapt the survey; if you're doing so, I'd appreciate it if you'd let me know (ehrmannsteve at gmail.com)

Note: One reason this program of developing summer courses has lasted 15 years, attracting both faculty participation and GW resources to support them, was that teaching online summer courses made money for both the university and for the participating faculty. As faculty interest in developing summer courses increased, GW had the incentive to hire more instructional designers to help them.  In effect, improving teaching on campus was being rewarded (via the intervening step of improving teaching, and increasing revenue, online).  That fact suggests a policy I'll describe in my next post.

Propagating Teaching Improvement by making it More Rewarding for Faculty and the Institution

For teaching techniques that are needed institution-wide, seek opportunities where those techniques can be incorporated in programs that can earn the institution, the department and the individual faculty some extra money (e.g., summer courses, new degree programs)  (Sharing revenue with faculty isn't automatic, but, to provide incentives for widespread teaching improvement, it's essential.)

For example, at almost every university and college, it's important to find ways to encourage students to invest more time and effort in assignments outside class.  Today's full-time students spend only about half the amount of time on assignments they did thirty years ago, and about half the time that faculty think students need to spend, according to Arum and Roska in Academically Adrift.

One of several mutually reinforcing ways to do that is by quizzing students online in ways that require them to reason about what they've been assigned to learn. These online quizzes and assignments can both (a) provide instructors with advance notice about students' readiness for class, (b) enable instructors to prompt students about what else they need to do to prepare for class, (c) enable instructors to call on students by name when they get to class, and (d) encourage students to be (even) more ready for the next class meeting.  If there is faculty-staff agreement about the power and flexibility of this use of online preparation of students (and instructors) for upcoming classes, then it makes sense to strongly encourage any new revenue-generating courses and degree programs to include that practice. Once faculty try out the technique in that new degree program or summer course, they may well begin using the approach in their other courses as well.  As these pioneering faculty use the technique, other faculty may follow suit. And the more often students see it in classes, the more likely they will be to accept it as a normal feature of studying.

Make sense? or not?

An Incremental Approach to Improving Learning Outcomes (Better than "Flipping")

Arguably the most important facing higher education is that, for a variety of reasons, the ways students aren't mastering higher order skills (ability to use knowledge to analyze, create, innovate, act wisely and ethically, etc.) nearly as much as needed nor as much as they once did.  Arum and Roksa's Academically Adrift is one persuasive source of empirical evidence on that point.  Just one of many important reasons for declining student achievement: full-time students are investing only about half the time in academic work than they did thirty years ago.

Against this backdrop comes a rush of discussion and confusion about 'flipping courses,' a notion that is sometimes justified as a strategy for improving higher order learning.  I think the term 'flipping' is being used to point to some good changes but, as a label, it's more misleading than helpful.

For example, a recipe for improving learning needs to include more than one way to assure that students learn from their assignments, including doing the kind of work, and enough work, to reorganize their minds.

I recently gave a featured address at the International Conference on College Teaching and Learning on this set of questions, entitled "Bit by Bit: An Incremental Approach to Improving Learning."

Briefly, many elements are needed for a successful, sustainable approach to teaching what is actually college-level learning, including:

1. Reconsidering the course (and academic program) in light of what graduates need to be able to do with the content they've learned, in later courses and after graduation ("backward design");
2. Stealing the students' beer time - assignments and other academic work that are so engaging that students will choose to spend more time studying (this is necessary but not sufficient for students to study more);
3. Leveraging face-to-face opportunities for what can be done best face-to-face;
4. At least four kinds of feedback (what Schön and Argyris called 'learning loops'), including:
• Feedback that keeps students going (farther and deeper), helping them see progress, helping them get unstuck;
• Feedback from that student work going to faculty, to help plan the next class session;
• Feedback to faculty that helps them tweak the design of the course;
• Feedback to faculty that helps all this work seem more rewarding, encouraging those faculty to keep on improving their courses.

I called the talk 'Bit by Bit' because these kinds of improvements don't all need to be done simultaneously or only in this order.  In fact there are even smaller chunks of improvement that can be tried (and supported by the institution), one at a time.

The long term goal, however, is cumulative, major changes in how we teach, how students invest in their own learning, and what our graduates are able to do.

I've recorded a version of the address on YouTube, in two installments. The whole talk is about 50 minutes long:

Part 1: http://youtu.be/Z-1fayCy69k

Part 2: http://youtu.be/VgaFTXCYtzc 

Does it Take More Time to Teach Distant Learners? (No, really...)

Does teaching online take more time than teaching on campus?

For many faculty, the answer seems obvious.  But is the obvious answer correct?

I've personally come across only one research study that went beyond personal impressions.  Years ago Christine Geith and Michele Cometa did a pilot study that worked closely with nine faculty who were each experienced in teaching both campus students and also distant students.

In order to compare 'apples with apples,' each faculty member interviewed was experienced in teaching both distant learners and campus learners in comparable courses,

They began by asking each faculty member which type of teaching was more time-consuming (per student): teaching distant learners or learners on campus. Everyone agreed that teaching distant learners was more time-consuming.

But when these nine faculty each thought about it some more, most decided they'd been mistaken.

Each spent about 45 minutes estimating how much time they'd spent in each of the different activities that make up "teaching a course."

• One third concluded they'd spent more time (per student) teaching distant learners (as they'd expected);
• One third concluded they'd spent more time teaching campus learners; and
• One third decided it was about the same.

From this pilot study, it appears that teaching distant learners isn't necessarily more or less time consuming than teaching students on campus. 

How can that be?

Experienced faculty can adjust elements of the course over a period of years. Eventually, 'time invested per student is more a result of time available.  (The same thing is true of university budgets, for example; the cost of teaching a student is determined by how much money over time has been available to teaching them.  I've written more about this surprising result and its implications in Ehrmann (2010).  When trying a completely new mode of teaching, a faculty member may feel compelled to invest an uncomfortable amount of time per student But after a while the workload can be tweaked until it's tolerable.

The big difference in time invested, surprisingly, was from one faculty member to another.  These nine faculty varied enormously in how much time they spent per student.  

In other words, personal preferences and circumstances have a major effect on how faculty invest their time in each student.  The mode of teaching apparently doesn't.  

*****
Ehrmann, S.C. (2010) “Improving Higher Learning by Taking the Long View: Ten Recommendations about Time, Money, Technology and Learning.”  Change Magazine, Taylor & Francis, September/October, pp. 16-22. (This article was reprinted in the January 2011 issue of Planning for Higher Education, published by the Society for College and University Planning, pp. 34-40.)

What Kinds of Teaching Improvement are Most Important?

Which kinds of teaching improvement are most important for the Teaching & Learning Collaborative (TLC) to support over the coming years?  To advance that discussion, I'll suggest three interdependent goals now, and a couple more in an upcoming post:

1. Support learning-centered teaching.  There's a lovely cartoon showing a boy with his dog. The boy is talking to his friend, and boasting that he's taught Spot to whistle.  The friend responds dubiously, "I don't hear him whistling."  The first boy retorts, "I said I taught him. I didn't say he learned it!"

I hope we can help an increasing number of faculty teach in a way that's guided by the actual learning that occurs in their academic programs.  Faculty who understand teaching in this way ('the learning paradigm') believe that virtually every student is capable of excellence, if teaching can guide and stimulate them appropriately.  Because it's not obvious at the start what needs to be done to help students achieve excellence, there's a certain amount of trial and error every term - the instructor tries something and, if it works for some students, do more of it. If it doesn't work for some students, try something else.

• The alternative view of teaching, sometimes called the 'instruction paradigm', is reflected in the comments of Spot's owner.  Teaching and learning are independent activities.  If you organize and present the content clearly, then you've taught it.  If the students learn it, that's entirely to their credit. If they don't, that's on them, too.  

Of course, it's not simple for faculty (or students) to see whether and how the student is learning, which leads to a second goal for TLC and units like it.

2. Support evidence-guided teaching. Because teaching needs to be guided by actual learning, then the faculty need to gather evidence of how the learning process is going.

More easily said than done.  For example, if the faculty member assesses the wrong thing --  testing whether the student can remember an expert's analysis when the real goal is to help students learn to analyze for themselves -- then scores or grades will mislead both the instructor and the student.

Testing what students have learned is necessary but not sufficient.  Analogy: people learning to bat in the game of baseball. Measuring their batting average, no matter how precisely, won't help them learn to hit better.  A very different kind of feedback is needed for that, e.g., slow motion video of how they swing at a pitch.  Faculty need to learn both how to provide two very different kinds of feedback for themselves and for their students: what the student has learned and how the student is learning.  Few faculty have received any training in the many ways in which these things have been done.  We can help with that.

3. Support faculty collaboration to improve learning.   I mean "faculty collaboration" in two different ways.

The first is illustrated by a research finding about composition courses.  In many evaluations of learning in composition, students are asked to write a composition at the beginning of the term and a second composition on a similar topic at the end of the term.  External graders then assess each paper without knowing when it was written.  The papers written at the beginning and at the end of the writing course often get similar grades. This does not mean that the students learned nothing about writing: over two or more courses, the essays do measurably improve.  The important lesson: the kinds of capabilities useful in life are often so complex and personal that one course often can't do not enough to create improvements large enough to measure.  As a senior once told me about something he'd learned, "I don't think I could have learned how to think that way in any one course, but, over the years, it gradually sank in."

That's why capstone courses and major projects in upper division courses can be so important: as sources of insight for faculty teaching lower division courses on where they're succeeding and where they need to improve.  

The second important kind of collaboration is between faculty and others, e.g.,  instructional designers, assessment experts, publishers and other materials developers outside the university.  I'll choose just one example: instructional designers in a number of departments at GW are currently testing ePortfolio products that faculty can use, individually and as teams, to gather and analyze evidence of student learning.

So those are my first three suggestions for where we should focus our support: learning-centered teaching, evidence-guided teaching, and collaboration to improve learning.

Over the next 5-10 years, what (additional) kinds of teaching improvement do you think that units like TLC should support?

One Reason Why Faculty Resist

I've been enjoying helping Natalie Milman by writing contributing several "Ends and Means" columns for the magazine Distance Learning this year.  Today I'm submitting my most recent contribution (and my last one for a while) called "One Reason Why Faculty Resist."  It begins:

"Have you ever heard the phrase “resistance to technology” used to imply that some faculty are irrational dinosaurs? I have, and I don’t like it. In my experience, most such resistance is quite reasonable.  The following story about online discussion in real time suggests what worries these instructors."

My story's point is that, when the terrain of teaching and learning changes, faculty are quite likely to encounter unexpected problems in their courses.  I don't just mean technical problems.  I mean problems with teaching and learning that are frustrating, embarrassing and sometimes potentially threatening. And, when they encounter such problems, they may well blame themselves. And they may feel that student reaction may put them at risk.

We know all this. But most institutions do little or nothing to prepare faculty for those problems.

So one reason for faculty "resistance" is that they sense that the effort to get them to teach online is a bit of a con game: "Come on in, the water's fine!"  Young technology staff,  leading training workshops, probably aren't aware of the problems. And, when workshops are led by faculty enthusiasts, they often  paint a rosy picture because they discount once-painful problems and don't want to scare their colleagues away.

My column concludes with some suggestions for how to organize self-sustaining, scalable, inter-institutional faculty conversations about teaching a particular course (e.g., "Econometrics 202; ). Their online and face-to-face discussions should be comparatively brief, brisk, relaxed, and helpful enough (trading tips, insights and moral support about what happened last week) that faculty would look forward to next week. That's the theory. Perhaps we can start a few such groups from GW.

If you'd like to learn more, the column should be published in a few months. Or contact me and I'll send you the draft.