Interesting opening at LaGuardia CC

Bret Eynon and his colleagues at LaGuardia are highly respected for their work on integrative learning and ePortfolios.  Bret asks for help in spreading the word about an exciting new position now open at the College:

"LaGuardia Community College was recently awarded a prestigious $2.9
million US ED "First in the World" grant. “Project COMPLETA:
Comprehensive Support for Student Success” links digital technology
with exciting pedagogical and co-curricular innovations to build success
for more than 25,000 of LaGuardia’s diverse, low-income students.
COMPLETA links a range of High Impact Practices, including the First
Year Experience, into a transformative, college-wide, integrative
design.   We're inviting applications for a Project Coordinator who will
play a critical role in helping to lead this showcase project.

"We're looking for candidates who have experience with higher education
innovation and project management.  COMPLETA links pedagogy, assessment,
professional development, and technology, including ePortfolio and
learning analytics.  We're seeking a smart, energetic innovator who will
be committed to our students, someone who can learn and grow as an
educational leader as they help to advance our nationally-recognized
change initiatives.  Detailed position description and application
information are both available at
https://www.rfcuny.org/hr/pvn/cgi-bin/show_job.asp?pvn=RMP-1141

"Please share this announcement with anyone you know who you might be
interested and appropriate.  The position is open till filled; resume
review will begin on January 27th.  We will circulate this announcement
now and again after the holidays.   Thanks for any assistance you can
provide."

How regional teaching institutions can compete more successfully

Tom Carey has just written a provocative blog post in Inside Higher Ed, describing how regional four-year institutions and two-year institutions might reshape and market themselves as teaching institutions of distinctive value in their area.  I'd summarize the message as 1) identify learning outcomes of distinctive value in your region (to some degree, the less universal the better) and that can be linked to some teaching/learning strength in which you're particularly strong (preparing for work in a certain type of job, after preparation by a co-op program with numerous, fascinating placements in that same field, for example). His strategy is much more detailed than this, but it's still a quick read.  Once you've read it, let us know what you think!

What does "Learning Outcome" mean? No, really....

"Confusors" are terms such as "teaching," "assessment," and "interactive,"  each of which has  several widely-held, conflicting definitions. When clashing definitions remain hidden, bad things can happen, needlessly.

For example, imagine a group of faculty hotly arguing the consequences of 'technology in classrooms.'  Perhaps some have a lecture hall in mind, others a seminar room, and still others are thinking about the library because, for them, "classroom" is a synonym for course. Meanwhile some are assuming 'technology' only means computers, a problem if some of the others are imagining cell phones or laboratory demonstration equipment.

Similarly it's dangerous to draw many conclusions from an article about what's wrong with MOOCs if the author didn't explicitly select one of the many accepted, conflicting definitions of "MOOC".

"Learning outcome" is also a confusor, I now realize. Earlier today I read this sentence in a report addressed to provosts and other senior administrators about the value of a new teaching strategy, " ...72% of institutions showed improved student learning outcomes."

Nowhere in this report is the term 'learning outcome' defined. That made me more conscious that most of the time when I hear "learning outcome" used in conversation, it's left undefined.  Set aside the question of whether that 72% improvement was measured in a single course, or across many degree programs.  There are at least three widely-used definitions of 'learning outcomes':

1. Good test results or grades at the end of a unit or a semester. Often good marks can be achieved through memorization or applying routine problem solving skills in a routine way.  Teaching to the test is one way to improve this kind of learning outcome. Cramming for exams is another.  "Here today, gone tomorrow," is what many faculty and educational researchers agree happens to much of this kind of learning.
2. Reduction in failure and dropout rates from a course.  This definition overlaps #1, but sometimes a student might drop a class for reasons unrelated to learning. 
3. Learning that lasts. In this definition, achieving a learning outcome means that students developed a capability and then showed they could use it in a later course, for example.  This definition implies that, if a student has achieved a learning outcome during college, they can use it in graduate school or a job the next year.

So the report I was reading asserted that " ...72% of institutions showed improved student learning outcomes"  What does the author mean? From here on, "learning outcome" joins my list of hot buttons - confusors that can cause real trouble.

PS. This web site describes many more confusors. Do you have a favorite?

I'm starting a new role

I've spent the last few months thinking hard about exactly what I wanted to do next.  A vision of the kinds of improvement that were possible in the long term for university teaching, and a path for getting there began to take shape. (An earlier version of some of those ideas was tested in this blog, in entries like this one.

Once I realized that working with those ideas was my goal, the next challenge was to find a place to put these notions to the test (somewhere near Silver Spring, MD where Leslie and I have long lived) and to find a good team to work with. 

The great news is that there was such a place and they were looking for someone with ideas and a track record like mine. 

So I'm happy to announce that I've just accepted the position of Assoc. Director for Research and Evaluation at the University System of Maryland's Center for Academic Innovation, working with MJ Bishop. The System has a delightful range of distinctive institutions as members, which should provide a good fire in which to cook my half-baked ideas.  And it gives me a chance to give back to the state where we've lived for 35 years.  I'll be starting in my new role on Sept. 22.

Limits to Interactive MOOCS (?)

In a recent Wired article, "The MOOC Completion Conundrum: Can 'Born Digital' Fix Online Education,", Dror Ben-Naim points to the potential of more interactive MOOCs to provide a more individualized, interactive educational experience.

Experience with interactive tutorials over the last half century suggests at least two questions to ask about any new strategy of this type:

1) How many person hours does it take to create, and debug, an hour's worth of student experience?  Historically that figure has been "hundreds to thousands," and the time requirement has changed less than one might guess over the decades. That's partly because the standards for the user experience have increased as the technologies have improved. But an even bigger factor has been the need for human judgment to assess how satisfactory each branch will be for various kinds of learners (learners with different preconceptions, learners with different reading skills, etc. etc.)

2) How many person hours does it take for a faculty member to decide whether this package offers good experience for all students?  Textbooks are generally comparatively easy to judge because tables of contents and thumbing through the book make triage easy. Instructional videos generally take much more time because faculty may feel the need to watch virtually the whole video, or video sequence, before their students see it. Interactive software can take even more time when instructors, unfamiliar with the package, want to see how it reacts to students with varied preparation, misconceptions, etc.  It's that same problem of judging the branching that makes development so expensive.

3) How soon might changes in underlying technology make the user interface seem old-fashioned, or interfere with the operation of the software itself? In that event, where will the money and effort come from to update the software? 

Interactive tutorials have often demonstrated impressive educational results, compared with less interactive ways of teaching the same content or skills.  However most such packages have died without ever progressing to Version 2.0 because there was no satisfactory answer to at least one of those three questions.

So, if you hear about a new technology for creating highly interactive and individualized instruction, it's wise before you leap to ask those three questions.

Impact of Online Teaching Upon Campus Teaching

Some months ago, with help from Camille Funk and Patty Dinneen of the Teaching & Learning Collaborative, I surveyed faculty from George Washington University who had worked with instructional designers to develop and teach online courses during the summer over the last 15 years. Our research question: had this experience influenced their subsequent teaching on campus. In a word: it did.

Faculty who had been through the program once were influenced in many dimensions of their teaching.  For example these kinds of changes in campus teaching were reported by at least half of the respondents:

My syllabus, instructions and directions for students are more clear and complete.

Development tools I learned about for summer I now sometimes use for my campus course materials.

I've re-used or adapted materials from my online course.

I use images, animations or video.

I've started designing a campus course or assignment by first figuring out what students should be able to do as a result.

I assign online discussion among my students.

Faculty who had participated two or more times were influenced much more.

I've heard anecdotal reports of such influence for years but we got a 53% response rate. Of those respondents, 85% reported influence on their campus teaching in at least one dimension.

Here's my full report and the survey that was sent to faculty.  Feel free to use or adapt the survey; if you're doing so, I'd appreciate it if you'd let me know (ehrmannsteve at gmail.com)

Note: One reason this program of developing summer courses has lasted 15 years, attracting both faculty participation and GW resources to support them, was that teaching online summer courses made money for both the university and for the participating faculty. As faculty interest in developing summer courses increased, GW had the incentive to hire more instructional designers to help them.  In effect, improving teaching on campus was being rewarded (via the intervening step of improving teaching, and increasing revenue, online).  That fact suggests a policy I'll describe in my next post.

Propagating Teaching Improvement by making it More Rewarding for Faculty and the Institution

For teaching techniques that are needed institution-wide, seek opportunities where those techniques can be incorporated in programs that can earn the institution, the department and the individual faculty some extra money (e.g., summer courses, new degree programs)  (Sharing revenue with faculty isn't automatic, but, to provide incentives for widespread teaching improvement, it's essential.)

For example, at almost every university and college, it's important to find ways to encourage students to invest more time and effort in assignments outside class.  Today's full-time students spend only about half the amount of time on assignments they did thirty years ago, and about half the time that faculty think students need to spend, according to Arum and Roska in Academically Adrift.

One of several mutually reinforcing ways to do that is by quizzing students online in ways that require them to reason about what they've been assigned to learn. These online quizzes and assignments can both (a) provide instructors with advance notice about students' readiness for class, (b) enable instructors to prompt students about what else they need to do to prepare for class, (c) enable instructors to call on students by name when they get to class, and (d) encourage students to be (even) more ready for the next class meeting.  If there is faculty-staff agreement about the power and flexibility of this use of online preparation of students (and instructors) for upcoming classes, then it makes sense to strongly encourage any new revenue-generating courses and degree programs to include that practice. Once faculty try out the technique in that new degree program or summer course, they may well begin using the approach in their other courses as well.  As these pioneering faculty use the technique, other faculty may follow suit. And the more often students see it in classes, the more likely they will be to accept it as a normal feature of studying.

Make sense? or not?

Example of a flipped course

Mike Moore, Prof. of Economics and International Affairs at George Washington University, has problems with asthma. So this past semester he taught two courses to GW students (mostly) from Santa Fe, New Mexico.  He did it by prerecording his lectures, posting them on YouTube, conducting live online office hours using Blackboard Collaborate, and visiting GW monthly for 90 minute face-to-face sessions with his students.

Mike didn't jump from traditional teaching to this strategy in one huge leap.

Some years ago, he taped some lectures on background material so that students could study and review them as often as they liked.  A few years after that, he tried GW LectureCapture - a system for automatically recording all class sessions and making recordings available only to students in that class. He'd worried that students wouldn't come to class if recordings were available but (as with every other faculty member I've found who's used a system like this), his worries were unfounded.  Attendance was unaffected.

Recently, he realized it would be better for him to spend some time in New Mexico and got permission from Mike Brown, his Dean, to teach a hybrid course - a mix of online and face-to-face instruction.

Early on he decided to use YouTube so that anyone could see his lectures.  He then took a look at examples of classes on YouTube. Some had pretty poor audio or picture quality; he knew he wanted to do better.

To create the lectures with HD quality, he pointed the camera of an iPad at a whiteboard.  He was no longer limited to the normal time constraints of a campus class. So he created about 75 small lectures, averaging 15 minutes in length. Some was material that would normally have been given in class but he also added additional material.  This was the big investment.  "I hope I can do this again because creating those videos took maybe 400 hours of work," he commented: if he teaches the course again, he can reuse some or all of those videos. He likes this approach to creating chunks of video because he can edit them if he likes. That removes the pressure to do each one perfectly the first time.

Even though the material is abstruse, the videos are popular. His videos had already been viewed over 65,000 times and over 60% of those views were requested from outside the United States.

Mike also created some interactive homework online, using Blackboard: multiple choice questions that provide hints to students if their initial answer is incorrect. There are also four online quizzes each semester: multiple choice and only a small fraction of the class grade.

For his live online office with Blackboard Collaborate, Mike uses a tablet so that he can draw. Those sessions are recorded so students can take part live and/or review them later.

Monthly Mike comes to GW for live, 90-minute sessions with students, a mix of lecture and discussion  The midterm and final are administered during these sessions as well. After the midterm he polled students and about 2/3 responded. "They love it, the structure, the delivery.  The most telling set of comments is that they loved being able to listen to the lecture whenever they want, pause, replay to get the details."  One student could take the course despite the fact that he had to spend a month in Malawi during the semester.  I interviewed Mike before the final was administered but he was already pleased with the quality of the students' work. I've heard since that his end-of-course student feedback scores were among the best he's ever received.  

I take particular note on that last point.  Student feedback is not proof of superior learning. But, if the learning itself could be assessed, it would not be surprising to find that this design could be both more accessible to both faculty and students (they don't need to all be in the same place, and there's less need to coordinate schedules) and also produce superior learning outcomes.  There's a long history in education of using technology in ways that can produce gains in quality and access simultaneously, going back to printed books as a way of both increasing an author's audience and enable readers to study the author's words more closely than would be possible in an auditorium.

if you'd like to learn more, Mike's email is mom@gwu.edu. 
 

An Incremental Approach to Improving Learning Outcomes (Better than "Flipping")

Arguably the most important facing higher education is that, for a variety of reasons, the ways students aren't mastering higher order skills (ability to use knowledge to analyze, create, innovate, act wisely and ethically, etc.) nearly as much as needed nor as much as they once did.  Arum and Roksa's Academically Adrift is one persuasive source of empirical evidence on that point.  Just one of many important reasons for declining student achievement: full-time students are investing only about half the time in academic work than they did thirty years ago.

Against this backdrop comes a rush of discussion and confusion about 'flipping courses,' a notion that is sometimes justified as a strategy for improving higher order learning.  I think the term 'flipping' is being used to point to some good changes but, as a label, it's more misleading than helpful.

For example, a recipe for improving learning needs to include more than one way to assure that students learn from their assignments, including doing the kind of work, and enough work, to reorganize their minds.

I recently gave a featured address at the International Conference on College Teaching and Learning on this set of questions, entitled "Bit by Bit: An Incremental Approach to Improving Learning."

Briefly, many elements are needed for a successful, sustainable approach to teaching what is actually college-level learning, including:

1. Reconsidering the course (and academic program) in light of what graduates need to be able to do with the content they've learned, in later courses and after graduation ("backward design");
2. Stealing the students' beer time - assignments and other academic work that are so engaging that students will choose to spend more time studying (this is necessary but not sufficient for students to study more);
3. Leveraging face-to-face opportunities for what can be done best face-to-face;
4. At least four kinds of feedback (what Schön and Argyris called 'learning loops'), including:
• Feedback that keeps students going (farther and deeper), helping them see progress, helping them get unstuck;
• Feedback from that student work going to faculty, to help plan the next class session;
• Feedback to faculty that helps them tweak the design of the course;
• Feedback to faculty that helps all this work seem more rewarding, encouraging those faculty to keep on improving their courses.

I called the talk 'Bit by Bit' because these kinds of improvements don't all need to be done simultaneously or only in this order.  In fact there are even smaller chunks of improvement that can be tried (and supported by the institution), one at a time.

The long term goal, however, is cumulative, major changes in how we teach, how students invest in their own learning, and what our graduates are able to do.

I've recorded a version of the address on YouTube, in two installments. The whole talk is about 50 minutes long:

Part 1: http://youtu.be/Z-1fayCy69k

Part 2: http://youtu.be/VgaFTXCYtzc 

Three steps forward for GW

The following three recent events have at least one thing in common, which I'll get to in a moment.

First, George Washington University agreed to host a program to host undergrads and high school students from Brazil in an exciting and innovative summer program.  The Brasilia Program exemplifies a crucial strength of GW education - creating a rich interaction between formal instruction and real world activity. In this case, students are taking classes in the mornings and doing field work in the afternoons.  Georgette Edmondson-Wright, Assistant Provost for Summer and Special Programs, pulled together a coalition of faculty and staff from across GW to plan this superlative initiative.  (And it doesn't hurt that, at a time, when GW is working hard to balance the budget for next year, the Brasilia project will help the entire institution financially.)

This past Monday, the Faculty Honors program showcased what's great about our faculty and graduate assistants.  Awards were given for teaching, scholarship and service and some of our longest-serving faculty were honored.  If you weren't able to be there, take a look at these inspirational videos of the recipients of the Trachtenberg Prizes, the Bender Awards for teaching excellence, and the Amsterdam awards for best teaching by graduate students. The ceremony drew a packed house to the Morton Auditorium.  It was particularly exciting to see the students who'd come to help honor their instructors.  Kaithlyn Kayer of the Office of Teaching & Learning (OTL) coordinated preparations for the ceremony and Aaron Kramer of the Teaching and Learning Collaborative (TLC) organized the teaching awards competitions from beginning to end.   Folks seemed to think it was the biggest, best celebration of its kind that GW had ever had.

"Biggest and best" was also a phrase that came to mind the very next day on the first day of Research Days at GW, when research by over a hundred undergraduate and many grad students was displayed and judged. (To get a sense of how excited I got, and why, check out this video, starting at the 3:00 mark.)  The Center for Undergraduate Fellowships and Research (CUFR) worked closely with the Office the the Vice Provost for Research to create this event.  For CUFR, Paul Hoyt O'Connor and Prof. Margaret Gonglewski led the way.  The exhibit halls at the Marvin Center were jammed with faculty, students and staff who'd come to marvel at the work and talk with the student researchers.  Two years ago, Research Day for undergraduates drew 69 student researchers, last year 89, and this year about 115.  What impressed me even more was the uniformly high quality of the work this year. I don't envy the 80+ faculty who had to pick the winners from this stellar group but I certainly do envy the hundred faculty who got to mentor them!

What do these three events have in common? Aside from the fact that all of them were significant for the entire university, they were each pulled together by people from the Office of Teaching and Learning; OTL includes Summer and Special Programs, TLC, CUFR, Assessment, and Academic Technologies.  In each instance, our folks worked closely with other units and individuals all across GW. But, OTL is my unit so I'm especially proud of the people I work with.

Improving teaching & learning for non-native speakers of English

Like many other universities, George Washington is attracting more students, faculty and teaching assistants with different native languages.  I'm not an expert in either second language learning or teaching international students.  What follows are the speculations by a generalist. If you have better information and contrasting ideas, I hope you'll post them below.  In the meantime, here's how things look to me.

Faculty-student interaction, rapid feed back, and student teamwork can each improve learning, but only to the extent that those people can easily understand one another, which includes being able to think quickly in English.  It's not surprising that a study at the New Jersey Institute of Technology (NJIT) in the 1980s found that non-native speakers of English got lower grades, on average, than native speakers.  I'll get back to that study in a moment.

I have a suggestion for how we can improve education for all our students, but especially for non-native speakers of English.

Hypothesis: learning can improve if a campus makes more, better use of vigorous, critical, online, written discussion (e.g., dialogue using tools such as email, threaded discussions in Blackboard, and more specialized tools such as VoiceThread).  

Why might that be true?

Remember that NJIT finding that non-native speakers were getting worse grades in campus courses?  Let me tell you more about that.

in the mid 1980s, NJIT had pioneered online courses. In one study, Prof. Roxanne Hiltz compared the grades of native English speakers with non-native speakers across many courses (Hiltz, 1988). As I mentioned earlier, the native speakers got better grades in their NJIT courses.  One might imagine many reasons for that.  Perhaps their high schools were inferior to ours, for example.

But Hiltz also compared the grades of native and non-native speakers taking wholly online courses.  No significant difference in the academic performance of native- versus non-native speakers of English (nor do students in online courses do worse than students taking campus courses).  

Hiltz concluded that, in the campus classroom, non-native speakers find it difficult to listen, think and speak fast enough in English to fully participate in learning.  

In contrast, in written online exchange, with plenty of time to interpret and respond, non-native speakers were able to participate fully, demonstrating their talent, preparation, and motivation. 

The benefits of online discussion for non-native speakers aren't limited to learning the substance of the course.

Karen Smith, then at the University of Arizona, studied non-native speakers of Spanish:  American students who were taking Spanish IV.  (Smith, 1990) 

In one section of Spanish, the students used a language laboratory, listening to Spanish on headsets and practicing pronunciation. 

In another section, students were assigned to converse online in written Spanish. They were graded only on their fluency in expressing complex thoughts, not on syntax or spelling.  And they had the option of initiating their own discussions, and of conducting some of the discussions in private. (These students also were assigned to work in the language lab but for less time each week, to make time for their work online.)

I visited that section in mid-semester. The students loved that they could converse in Spanish about issues they cared about. In one online discussion for example, a number of students were communing with a classmate who was feeling depressed.  In another discussion they discussed a film that had been shown in class, while in another they planned a class party, all in Spanish.  Students liked the fact that, online, they had time to interpret and consider what they were 'hearing' and then to plan how to reply. And, for these discussions, they were not being graded on their spelling or grammar. 

Here's the surprise. At the end of the semester, the two groups of students were each given an oral examination. And the students who'd been practicing oral Spanish in the language laboratory lost.  The students who, instead, had been conversing asynchronously in written Spanish were better able to think in Spanish. So now, in real-time face-to-face conversation, they found it easier to speak fluently (and grammatically).

This suggests that online discussion can prepare our non-native speakers to listen and speak face-to-face in the languages of their disciplines as well. 

Adding online discussion to campus courses can help all students learn, not just the non-native speakers of English.

Some years ago, I was talking with a young professor of philosophy at another institution. "I can't talk philosophy with an undergraduate," he remarked. "They just don't know the subject well enough." He paused and then continued, "Ah, but email! When we're conversing by email, we can have quite a satisfactory discussion. The student has enough time to interpret what I've said and to plan their response."

Think about it: in every discipline, students are learning to think in, and speak, a new language: the language of philosophy, or physiology, or law.  It's easier for novices to learn to think, speak and see in a new field if first they can do it at a thoughtful pace. (In a future post, I'll suggest a strategy for fostering  more critical, searching online discussions.)

Bottom line: By incorporating the right kind of vigorous, critical online discussion in our campus courses, we can improve learning for non-native speakers of English, for others with issues listening or speaking in a group talking at classroom speed, and for all our other students too.

If you're interested in learning more about how to use technology in these ways in your own courses at GW, contact our Instructional Technology Laboratory in the basement of Gelman Library about the technology and our Teaching & Learning Collaborative (our teaching center) to talk about how to organize and moderate such discussions. If you'd like to talk with someone from the TLC about these issues, you can use this form to arrange for a meeting.

REFERENCES

1988    Hiltz, Starr Roxanne, "Learning in a Virtual Classroom" (Executive Summary and two volumes), Research Report #25 and 26, Computerized Conferencing and Communications Center, New Jersey Institute of Technology.

1990      Smith, Karen L. Collaborative and Interactive Writing for Increasing Communication Skills, Hispania, LXXIII:1, pp. 77-87.