How to write effective questions for evaluating deep learning

Authors

“Are my students really learning anything?”

When I ask teachers from all corners of the world about what worries them most about their teaching, this is by far the most popular response…they wonder, “Am I making any difference in my students’ learning lives? Have my students gotten anything out of my course that will help them succeed in their subsequent classes and eventual careers? Have they gained an appreciation for the beauty or power of this discipline or subject matter?”  These are certainly the questions that weigh heavily on my mind at the end of a semester of teaching.

packice2003final

The Surface

Here are 5 of my favorite strategies for writing effective questions for evaluating deep learning.  I use these strategies to help me write questions or assessments that provide evidence of deep learning (learning that is permanent, strong, and enduring) versus surface learning (learning that is temporary, weak, and short-lived).  As the semester comes to a close and you are thinking about how to write questions that will assess your students’ learning, see if you can dive below the surface with one or two of these strategies. I have given some tips, also, for question types for each strategy.

  • 1      Assess for CHANGE
    There are many ways to assess for change and many different kinds of change you can assess.  The best way to do this is to collect pre-course or early course data from a conceptual inventory, attitudinal survey, or prior knowledge assessment you develop yourself. However, you can also do this by evaluating students on their demonstrations of learning about a concept early in the course and their demonstrations at the end. Question type tip: You could pick a set of clicker questions (or ConcepTests) that you asked early in the course and see how students do on a set of isomorphic questions in the last week of class or on an exam.  Don’t use clicker questions? Try some homework problems or early exam questions; think about asking students to explain their rationales for their responses in an open-ended format.
  • 2      Assess for ORGANIZATION OF KNOWLEDGE
    Surface learning is generally evidenced by performance resulting from memorization of discrete facts. Assess students’ deep learning by evaluating how they organize knowledge and how they describe the relationships between facts, concepts, or ideas.  Question type tip: Try asking students to contrast two or three cases and describe similarities and differences or have them prepare a Concept Map (don’t forget to ensure they define the relationships between concepts).
  • 3      Assess SELF-ASSESSMENT
    Accurate self-assessment is a hallmark of mastery. Give students some self-assessment questions for extra credit at the end of their final exam or final paper.  Question type tip: My favorite way to do this at the end of the term is to ask:  “If you had more time on this exam or paper, explain five things you would improve upon in this assignment/assessment if you had more time.” This also gives me a leg up on problem areas to look out for. You could also have them fill out a self-evaluation rubric if you are using one to assess their work. Either way, compare their self-assessments with yours and pay attention to their accuracy.
  • 4     Assess for  SOPHISTICATION OF THE BASICS: Learners with surface knowledge only can usually successfully perform well on basic tasks, but if you compare their performance with that of someone who has mastery of the material, cracks in that surface knowledge will become obvious. Question type tip: Ask students to perform a task, give a brief presentation, or write a short essay demonstrating or explaining in detail a single, very basic idea or concept from earlier in the semester. How sophisticated is their explanation?
  •  5    Assess for KNOWLEDGE TRANSFER
    I think the ability to transfer knowledge from one context to another new or unfamiliar context is the signpost of deep learning.  Question type tip: Use an “application card” – an assessment technique proposed by Angelo and Cross in their book Classroom Assessment Techniques. Ask students to write out in detail how they could apply an abstract concept to a real world context of their own choosing. ConcepTests are also great for testing knowledge transfer, find examples on Learning Catalytics  or follow the instructions here. 

Of course, there are hundreds of other strategies and question types you can use to assess deep learning. These are a few of my favorites that I’ve tried out in my own classroom and that I think provide the most convincing evidence. If you use them, I think you can break through surface learning to observe some pretty amazing things underneath.

photo 1

Under The Surface (Nat Geo)

These resources deeply informed this list of strategies and tips:

5 Elements of Effective Thinking
Classroom Assessment Techniques
How Learning Works
How People Learn
Peer Instruction: A User’s Manual
Time for Telling

What do you think?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 646 other followers

%d bloggers like this: