Backed by 20 years of research, Peer Instruction is my favorite way to engage students in class when using a flipped approach. Developed by physics Professor Eric Mazur at Harvard University in the 1990s, Peer Instruction is now used by teachers across the disciplines and in classrooms all over the world. Many such teachers are registered as members of the Peer Instruction Network, a global community of innovative educators.

A frequently posed comment about Peer Instruction (PI) is: “Well, my students are very different than Harvard students. I am not sure PI will work in my school and in my classroom.”

Recently, Akshay Saxena of the Avanti Learning Centre in Kanpur India set out to test this theory for himself.  Avanti provides after school services to low income, high school students in India to better prepare them to excel in college.

“These are students with an average annual household income of less than $4,000,” says Saxena. ” Half the class doesn’t speak even conversational English.”

In this incredible two minute video, Avanti demonstrates the power of Peer Instruction, implemented successfully with no technology, in Kanpur. Saxena reports, “You’ll see [the students] like it, are super engaged and better still, their answers converge!”

To learn more about Peer Instruction, see the Quick Start Guide to Flipping your Class with Peer Instruction (available in nine languages) or watch this 2 minute video. 

What is a flipped classroom?

Last week, I plopped down for Sunday brunch in New York City with some non-edu obsessed friends and acquaintances I had not seen in a long time.  About 10 seconds after our formal greetings, the person sitting across from me leaned forward and said, “So…not to be too business like, but what is a flipped classroom?”

Surprised, I tilted my head and narrowed my eyes quizzically, the person sitting two seats over followed up with a smile and explanation: “We follow you on LinkedIn.”

Not wanting to bore my friends with a long dissertation on the flipped class, I tried to explain the basics in an elevator speech. While I am not sure I succeed at brunch, the experience inspired me to challenge myself to define the flipped classroom in 60 seconds or less.The result of that challenge is the below video (with 14 seconds of intro and conclusion).

Important note: I do not provide a comprehensive definition of or address the many issues of definitional clarity with the term flipped classroom (that would take longer than 60 seconds). For more on those important issues, see how Turn to Your Neighbor defines a flipped classroom here and our interactive question series on 7 myths about flipped classrooms here.

In this video, I explain the basic cycle of learning in a flipped classroom using a visual from the Center for Teaching and Learning at the University of Texas at Austin, developed by Josh Walker and Manny Oliverez. Click  here to download a transcript of the video.

Stealth Flipper’s class, Fall 2012 (blur purposeful)

Won’t students skip my class if my lectures are available online?

This is a question that comes up often in the world of higher education, where class attendance is usually not compulsory.  One fine day early Fall of 2012, I took this question with me on my walk from my office in the University of Texas at Austin tower to one of the largest auditoriums on campus.

I was visiting Stealth Flipper’s  class, a large enrollment (n=400) Humanities course for non majors, called Introduction to Ancient Rome.  Stealth put all pre-recorded lectures online for students and required them to watch three to four, 20 minute lectures per week. In class, Stealth used several innovative strategies,  such as mini lectures,  clicker questions, guided large class discussion, and Peer Instruction.

Within the first few minutes of arriving, as I had to jockey for a seat, the answer to my question seemed pretty clear. I wrote the following statement in my notebook: “Yes! Students will attend even when the lectures are online!” and I took the above snapshot as proof.

Now, as I think back on this, I ask myself – “So what? Is attendance really a measure of how well a course is going?”

Stealth had not always taught a flipped class. Indeed, she originally taught Intro to Ancient Rome more traditionally –  by assigning readings, lecturing, and “trying to push the class to think deeply about the complexities of the content,” she says on her blog, Teaching without Pants.

Stealth emphasizes that she liked teaching a large class and “even enjoyed lecturing.” That said, she also felt that something just did not sit quite right with her and the traditional approach: “I hated the feeling that I was in cahoots with my students–I’d make the class entertaining and not too demanding and they’d humor me by cramming a bunch of facts (from a study guide I handed out) and then purging them on the midterms.  I knew they weren’t really learning, but didn’t know what else to do.  I also realized that I was going to become bored very quickly with giving the same lectures every fall.”

So, when Stealth learned lecture capture via Echo360 was available in her classroom, she decided to try to flip her class. I have been eagerly following Stealth’s flip quest for almost a year. Through a ton of trial and error, she’s come up with two game changing tips that I think everyone considering flipping their classroom should know.

1. Don’t tell students you are “flipping” or “experimenting”

In the first implementation of her flip, Stealth used the word “flip” to describe her class to her students. Everything in the literature says to spend time upfront describing exactly what you are doing as a means of meeting the inherent student resistance that will come when you try flipping for the first time.

“I told the students that they were in a ‘flipped’ class and tried to make them partners in creating the learning environment” she says.

When I heard about what happened next, it caught me like a deer in headlights. I was stunned and had no idea how to help.

Students in Stealth’s class started a Facebook Page with a thread titled “I hate the flipped class.” This thread was not only active, it had quite a bit of disturbing content.  Apparently, students did not complain about the content or the teacher but their dissatisfaction with the “flipped class” was vocal and aggressive.   Comments included plans to blast the class in the end of course evaluations and that students were not paying to go to a top university to watch their teacher on a video or to talk to their peers in class.

Such student resistance can be a huge turn off for instructors who are spending an inordinate amount of care, time, energy, and emotion toward creating a better learning experience for their students. Albeit small, an uprising of vocal, angry, dissenting students can be enough to send some teachers packing their flipped-class suitcases back to the land of lecture for good. I’ve seen it happen myself, at Harvard, and heard about it elsewhere.

Not so for Stealth Flipper, my new hero.

In several conversations over the past year, Stealth told me that in reading between the lines, she felt students actually had some valid concerns. Instead of chalking it up to the flip class itself and abandoning ship, she sifted through their comments, took them seriously and listened carefully.

Through this exercise she got an idea that would change the trajectory of her flip and her teaching.

She discovered that her students seemed to be latching onto the word “flip.”  She made several tweaks to her flip approach, but the most interesting to me is that the following semester she did not decide to give up on the flip class. Rather, she decided to give up on using the word flip.  ”I haven’t used the word flipped or flip once in the course or in talking with my students,” she recently told me–hence the moniker, Stealth Flipper.

According to Stealth, this tweak has worked brilliantly. ”Student resistance hasn’t just lessened, it has entirely disappeared,” she says.  Students now come to her office and report how much they enjoy how she teaches, whereas in the fall, they would come in and complain  about their “flip class.”

So, she must have just made the class easier, and that’s why they liked better, right? On the contrary, she made it harder and added many more formal assessments.

2. Don’t teach in new ways and assess in old ways, add frequent low stakes assessments 

The other tip that changed the flipped game for Stealth was her approach to assessment. In her first implementation, she used the same approach to assessment that she had in her traditional class. Students had three midterm exams and a final exam. In her second implementation, however, she added nine weekly quizzes plus a portfolio project in addition to three midterms. She administered the quizzes with Scantrons.

Nine weekly quizzes? WHAT? Certainly this would cause a revolt?

It seems not. Students have indicated that the quizzes have motivated them to change their approach to learning – ie. not cram before the midterm. This is also reflected in their viewing patterns. Figure 1 shows Stealth’s students’ lecture-video viewing patterns for both semesters in the week before the second midterm.  Observe that in the first implementation (Fall 2012) there was a huge spike in views in the week before the exam, which was not the case for the students in the stealth flip class (Spring 2013)…they were watching all along.

Fig 1. Viewing patterns in Stealth Flipper’s class 1 week before 2nd exam

Engagement and less student resistance is not the only outcome that Stealth is realizing through these two tweaks. She has also observed a gain in the average on the first two midterms, as demonstrated in Figure 2. One thing to note is that the exam in the stealth flipped class was significantly harder than Fall 2012 exam.

Fig 2. Exam performance across implementations

As Stealth says, “there’s such a difference in the spring class.  I think these two points are important because they counter a couple of orthodoxies–that you should tell students they are in a flipped class to encourage buy in; and that you can’t do low stakes assessment in a large intro class because the logistics are too messy. ” For a more detailed version of how she runs class based on these two ideas click here. 

In closing, my original question about whether students will attend class if I put all my lectures online seems trivial. Who cares? The real question is will they learn to learn better and will they show greater success in so doing. It seems with these two key yet simple tweaks, they just might.

“If you wanted to create an education environment that was directly opposed to what the brain was good at doing, you probably would design something like a classroom.”

John Medina, Brain Rules

In a flip class, educators have dedicated a lot of attention and valiant effort to redesigning approaches to direct classroom instruction (e.g. lecture) so that class time is transformed into something more…well…effective.  Such effort has resulted in an incredible movement where teachers and students are radically changing the out-of-class experience.  We have software and systems galore - ranging from free to expensive, screencasts, interactive whiteboards, online homework systems, social annotation systems, learning management systems, MOOCs, question management systems, and more.  This is very different from when I went to school and played Zelda all night because I knew whatever I read would be covered in lecture the next day.

It seems to me that less public attention is directed toward what happens during class time. (I’m a culprit of this as well!) Eric Mazur describes Peer Instruction as one way to do something innovative with your flip in class. Indeed, PI offers one robust, low-threshold way to transform student learning during class time backed by 20 years of research.

I put together this self-paced, interactive tutorial on Peer Instruction via TedEd.  Check it out, quiz yourself, discuss, and in 5 minutes learn to flip your class using a pedagogy that works. Change how you teach. (Clicking the red play button will take you directly to the TedEd Lesson.)

TedEd Video on Peer Instruction

Peer Instruction Network member Claire, a K-12 English teacher from Michigan, worries that far too many of her students are falling through the cracks.  She is looking for ways to reach them.  Could student-generated questions be one possible low-threshold intervention?  This post digs into this question.  However, if you are looking for a quick protocol for student-generated questions, skip to the bottom. 

It’s that moment.  An hour before class starts and for a variety of reasons (committee meetings, chair beating down your door for this that or another thing, kids with the flu, etc.) you didn’t have time to prepare the totally engaging, mind blowing experience that usually rocks your students’ worlds during class.

What to do?

I faced this predicament when I last taught my education theory course at TC. Being someone who takes their teaching extraordinarily seriously, I was panicked and feeling guilty about being under prepared. So, I whipped open Keynote and started to eek out a few clicker questions. Halfway through, I stopped, dejected, when I realized that in rushing I could only come up with a few lame memorization or fact recall prompts. Scratching my head but determined to figure something worthwhile out, I decided to try an idea I had been batting around in my mind for weeks.

At this point in the semester, the students were used to my  flipped approach: they were diligently doing their readings and Just-in-Time Teaching (JiTT) exercises before class and working on application questions in class via Peer Instruction and ConcepTests.   For one of their JiTT exercises, I planned to have students write ConcepTests/clicker questions about the material. My goal was to engage students in the higher levels of Bloom’s (e.g. creating, synthesizing material) by getting them to create questions in a low-stakes environment.

I also planned to use some of the questions as ConcepTests during class.

On this one bad day,  I decided to fast track that plan and have students spend the first 15 minutes in groups writing clicker questions that we would then run in class.

While a few of the student questions were terrific, most were too easy, a small handful were terrible. I tried this exercise a few more times, but soon took it out of my active learning tool box.  I simply did not find enough student-generated questions that hit that sweet spot of not too hard and not too easy.

Did I abandon ship too quickly? Maybe.

What does research say about student-generated questions?

Recently, a colleague at the University of Texas at Austin, Zachary Williamson, pointed me to a study by Berry and Chew (2008),  which evaluated the potential benefits of student-generated questions. Berry and Chew found that in a study of undergraduate psychology students (n=102),  ”generating questions related positively to improved student performance,” particularly for low- performing students.

A caution: The students who opted to participate by generating questions about course content, had performed worse on their first two exams than those who opted out. As such, there is some noise there – was it motivation to improve, more intense study, or was it the act of generating the questions? Hard to tell.

However, Berry and Chew also documented that for the students who generated questions, there was an upward trajectory in their exam performance (they did better on the third exam than would be predicted based on their previous performance). The students who did not participate in the student-generated question exercises did worse than statistically expected.  Furthermore, the more questions the lower performing students generated, the more they improved their performance. The improvement appeared to be related to frequency of question and independent of the depth or quality of their questions. Other similar studies in other disciplines such as physics found the opposite effect. For example, Harper, Etkina, and Lin (2003) found that the number of questions posed was unimportant, but the quality or depth of the question was. Berry and Chew explain this by suggesting that the effects of question generation may have some discipline-specific differences.

A possible intervention to help struggling students?

While the jury is still out, I am intrigued by the possibility of such a low-threshold instructional intervention to help teachers like Claire and her lower-performing students. In Berry and Chew’s study, instructors used the same prompt to solicit questions every time and did not even give the students feedback about their questions, they simply responded with an email awarding the extra credit. (Note:  I make a hard stop at advocating lack of feedback and I think feedback experts would probably discount this tactic as well.)

I do think it is worth trying student-generated questions to see if they might help under-performing students in our classes improve.  Even in interactive teaching environments, with master teachers,  there are students who show early warning signs of struggle. It is possible that creating incentives for generating questions could be an easy way to help them succeed. 

Protocols 

If you want to test student-generated questions in your class – here is the prompt Berry and Chew (2008) used:

“Provide us with three questions that you would like answered concerning the topics covered in your textbook readings or in lecture. These can be any questions you might have, as long as the questions are about the material or are stimulated by the material. They can be questions about concepts you are still unclear about, about further information you would like to have, or questions about how some issue applies to your own life or to other course concepts” (p. 306).

A recent blog post by Jackie Gerstein also has some protocols developed by others.

Here is my distilled protocol based on my research into this topic:

Protocol for student-generated questions (c) Julie Schell

Online systems for student-generated questions 

There are fantastic online systems that facilitate student-generated questions in a higher-threshold way. My favorite is PeerWise. PeerWise is free and also provides a platform for peer and instructor social interaction.

IN CLOSING, I do not know if my efforts to get students to generate clicker questions had any statistically significant effect on their actual learning. When I abandoned ship, I assumed that the lack of depth of the questions my students generated was a sign that the exercise was not very useful. But maybe I was being too teacher centered–just because most of the questions were not useful for me does not mean creating them was not useful for my students.

Claire, I think trying out a structured approach to student-generated questions may be worth a shot.

READERS: Provide us with three questions you would like answered concerning the topics covered in this post in the comments section below…

Berry, J. & Chew, S. (2008). Improving learning through interventions of student-generated questions and concept maps. Teaching of Psychology, 35, 305-312.

Harper, K. A., Etkina, E., & Lin, Y.-F. (2003). Encouraging and analyzing student questions in a large physics course: Meaningful patterns for instructors. Journal of Research in Science Teaching, 40,  776–791.

Many teachers are using Peer Instruction and classroom response systems (CRSs) to flip their classrooms and to engage students in deep learning and subject-matter understanding. After trying a range of CRSs throughout his career, in 2011 Eric Mazur teamed up with Brian Lukoff and Gary King at Harvard University to develop Learning Catalytics (LC). LC is a new CRS that allows instructors to ask a wide range of open-ended questions, can automatically group students based on their responses to questions, provide out-of-class and in-class quizzing, and fully facilitate Team-Based Learning. (Math Professor Robert Talbert writes a lot about LC here.)

In a new book by Emerald Publishing, “Increasing Student Engagement and Retention Using Classroom Technologies: Classroom Response Systems and Mediated Discourse Technologies,” we have a chapter titled “Catalyzing Learner Engagement Using Cutting-Edge Classroom Response Systems in Higher Education” (Schell, Lukoff, & Mazur, 2013).

The chapter poses two questions:

1) How can LC help students engage with subject matter in ways that will help them learn?

2) How can instructors use LC to measure student engagement in new ways?

We begin with an overview of key learning science principles, such as metacognition, self-monitoring, self regulation,  conceptual change, transfer of learning, and feedback, and provide examples of how LC relates to those principles.  A brief outline of Peer Instruction is also included, as is some screen shots of nifty features native to LC–such as a wide range of question types that go beyond multiple choice and intelligent grouping of students. The chapter concludes with pilot data from a course using LC at Harvard University we used to identify possible examples of student metacognition as well as “natural teachers”– students who are consistently able to convince their peers of the correct answer.

Contrary to popular belief, there is actually no “ONE way to flip a class” (Bergmann, 2012).  However, after visiting flipped classrooms all over North America and talking with lots of teachers who have tried flipping across the globe, I have come to view the cycle of flipping as occurring through a common approach.  Below, I synthesized this basic cycle into 4 best-practice steps. While there is certainly more to the method than this distillation acknowledges, I hope the sticky acronym can serve as a quick, simplified FLIP guide– whether you are using a Flipped 101 model (e.g. putting lecture videos online and having students do homework in class, see Bergmann and Sams, 2012) or you are using more innovative approaches, such as Peer Instruction. Special thanks and attribution to Josh Walker  for the inspiration for this visual.

4 Steps to Implementing JiTT 

1. Select an out of class coverage assignment for students - The assignment can be reading text, watching a video of your lectures or someone else’s, or both through a pdf with an instructional video embedded (click here to watch our 3 min video on how to create pdf lessons with embedded video.)

2. Embed Assessment – Pose 3 questions to students about the coverage assignment- 2 questions specific to the content and one feedback question. For each question, students must include a rationale or delineate the reasoning for their responses.

3. Review feedback before class –  Spend time reviewing student responses to the three questions

4. Address themes in student difficulty or misunderstanding during class time  – We do this using Peer Instruction but you can use any activity to do this, really.

Tips on out-of-class question design 

We are always careful to design the first two JiTT questions in a way that goes beyond fact recognition or recall. Here’s a 2:40 minute video we made that begins with a 40 second overview of JiTT and then provides a simple example of the types of questions we like to use in our JiTT exercises.

Video credits: Embedded from http://www.youtube.com/watch?v=9yEylIfDkms

Two ways to evaluate students’ out-of-class work  

One of the most frequently asked questions we get about JiTT is how we score students’ assignments. We always give students credit (points) for doing their assignments and we base the scoring on effort not on correctness.  For JiTT, we want to emphasize student effort and reward development of reasoning, not just getting the right answer. We provide feedback on the “correct answer” in JiTT and we do grade homework on correctness. Here is our rubric for evaluating their effort on JiTT exercises.

Mazur Group JiTT Rubric

Many instructors do grade JiTT exercises on correctness - Peer Instruction Network member and Director of the Academy for Teaching and Learning  at North Carolina A & T,  Scott Simkins recently pointed me to a set of fantastic resources on Just-in-Time teaching here. I pulled out the below rubric by Kathy Marrs (which appears to be adapted from de Caprariss et al., 2001). 

Kathy Marrs downloaded from: http://serc.carleton.edu/introgeo/justintime/step3.html

JiTT experts recommend sharing the rubrics you will use to evaluate work with students at the beginning of the semester. If you have a hack that you like to use for JiTT, include it in the comment section.

Like any popular phenomena, as the flipped classroom continues to gain attention, myths about it spreads freely. In this interactive series, we introduce and debunk 7 myths through a set of opinion polls.

Instructions

1. On each page, starting with this one, you will see a True or False poll. Don’t worry, we don’t have a record of who answered what, so answer honestly!

2. Select your response and/or view others’ results

3. Move to the next myth by clicking the NEXT MYTH at the bottom of each poll, this will direct you to the next poll in the series.

4. At the end of the series, click Finish to read a debunking of the myths and share your thoughts with others.

START NOW: Test your beliefs about flipped classrooms with this first question: Is it only about the videos?

Myth 1 about the flipped classroom – flipped teaching is primarily about putting videos online.

NEXT MYTH>

In a recent conversation with Peer Instruction Network member Ives Araujo from IF-UFRGS in Brazil, I learned a simple approach to enlivening the readings we ask students to do outside of class: embedding video into pdfs. Of course, this approach will only work with content you create, versus preexisting materials or texts.

Figure 1 is an example of a pdf with video instruction from another Peer Instruction Network member, Vagner Oliveira, who flips his high school classes using  Just-in-Time Teaching and Peer Instruction. The pdf contains a few different types of content:  images (top left), text content (right column), graphics (bottom left), and in the middle, there is a clickable video that students can watch of their instructor explaining a concept, covered in the pdf, on a whiteboard. Before coming to class, Vagner’s students prepare by reading and watching the pdf, and answering questions about concepts in an online reading assignment facilitated by a learning management system.

To see how complicated embedding a video into a pdf would be for teachers, I tried it out myself. I was surprised that I was able to do this  in less than two minutes. Admittedly, I am “good at the computer,” but even still, I think this is a low threshold, terrific way to do something innovative with text.  (The only barrier is the cost of Adobe Acrobat Professional if your school doesn’t provide it.)

Fig 1. Pdf with video of instructor teaching content embedded

What you need:

1. A lesson or content for students to read, which you have prepared with word processing software
2. An instructional video or lesson to embed, either that you have made yourself or that is free use
3. Adobe Acrobat Professional (there are probably other ways to do this, but this is what
   I am using, Dr. Araujo also found this link for embedding video using  LaTeX. 

What your students need: Free Adobe Acrobat Reader (Preview won’t work), hardware that runs Acrobat Reader.

What to do: Watch this quick 3 min instructional video, and try breathing life into your lessons in a new way.

Have cool examples of lessons with videos embedded in pdfs? Share in the comments section!