The Smith Study: Peer Discussion

Top

00:00:16.11 So, I want to tell you about a study that convinced us of...
00:00:19.01 further convinced us of the value of using clickers in class.
00:00:23.27 This study was the work of Michelle Smith,
00:00:27.15 who was a very talented postdoc
00:00:30.13 at the University of Colorado at that time,
00:00:32.12 as a teaching fellow,
00:00:35.21 and a professor of genetics, Tin Tin Sue,
00:00:37.24 who was teaching the introductory genetics course.
00:00:41.02 And Michelle was working with Tin Tin
00:00:42.29 to try to help her transform her course
00:00:45.05 to a more active learning mode,
00:00:47.25 and she urged Tin Tin to include clicker questions.
00:00:51.14 And showed her the data that, often,
00:00:53.16 when you ask a question that's difficult, and students...
00:00:55.25 many students get it wrong, and then you let them discuss with each other,
00:00:58.19 and then you ask again, and most of them get it right,
00:01:02.11 that the clickers are really a valuable learning tool.
00:01:05.29 And Tin Tin, being a good skeptical scientist,
00:01:08.08 said, well, how do you know that the students are actually learning anything in this process?
00:01:13.04 Isn't it just as possible that the students
00:01:15.15 are simply listening to whoever they think is the most knowledgeable person in their group
00:01:20.14 and changing their answer accordingly?
00:01:23.01 First, I'll ask you what you think
00:01:25.12 about why students do better
00:01:28.25 after discussing a question that many of them got wrong
00:01:33.10 than they did the first time around.
00:01:41.08 And so, Michelle and Tin Tin dreamed up an experiment
00:01:44.00 to try to test that.
00:01:45.15 The basis for their study was to
00:01:48.20 develop pairs of questions that were isomorphic,
00:01:52.13 which means that it's the same...
00:01:54.20 testing the same concept but in a different context.
00:01:57.23 So, for example, you could have a genetic...
00:02:00.01 genetics question first with fruit flies and then with mice.
00:02:05.03 And it's getting at the same principle,
00:02:07.17 but it's a different experimental system,
00:02:09.16 and the names of the genes are different, and so on.
00:02:12.03 So, it doesn't really look, superficially,
00:02:13.22 like the same question, but it's testing the same concept.
00:02:18.16 And the experimental design was to
00:02:20.26 ask students the first question of a pair
00:02:23.19 and then let them vote
00:02:26.18 and not show them the result,
00:02:28.23 then let them discuss it,
00:02:31.13 and then ask the question again and let them vote again,
00:02:34.07 and don't show them the result.
00:02:36.14 And then a little bit later,
00:02:39.14 ask the isomorphic question and let them record their result,
00:02:44.28 and then give them the correct answer,
00:02:48.17 explain the correct answer.
00:02:51.03 Before I show you the results,
00:02:52.27 take a moment to think about the experimental design.
00:03:14.01 Here's the experimental protocol again,
00:03:15.24 outlined in the next few slides.
00:03:18.10 The results were based on student averages
00:03:20.20 on a total of 16 such question pairs
00:03:23.01 over the course of the semester.
00:03:31.29 And I'll show you the actual numbers in a minute.
00:04:03.25 So, after posing the second question and recording the results,
00:04:07.25 here are the patterns of results one could get.
00:04:13.28 So, now, here's the question for you to think about.
00:04:16.10 Which of the data sets shown here
00:04:19.02 would allow you to answer the question at the top of the slide?
00:04:37.03 And here's the answer and the rationale
00:04:39.11 in the next few slides.
00:05:20.17 So, the correct choice is A,
00:05:22.18 or more precisely, comparing A and D with B and E.
00:05:30.14 And here are the actual results Michelle and Tin Tin obtained.
00:05:35.20 Here's a picture of the data,
00:05:37.25 and you can see that about 48% of students
00:05:41.08 missed the question on the first go-around.
00:05:44.00 When the question was re-asked after discussion,
00:05:46.12 you can see that 41% of the students got it right, now,
00:05:51.01 and 59% still got it wrong.
00:05:55.07 And then, if you look at the isomorphic question,
00:05:57.14 the people who changed their mind as a result of discussion
00:06:00.11 got 77% correct on the isomorphic question.
00:06:04.03 And the people who stayed with their incorrect answer only got 44% correct.
00:06:18.02 So, what they found was evidence that seemed to clearly indicate
00:06:21.23 that students were in fact increasing their understanding
00:06:24.22 through the discussion process.
00:06:26.26 Because if you'd pick the people who got it wrong the first time,
00:06:29.28 and then changed their vote and got it right after the discussion,
00:06:35.26 they do much better on the subsequent isomorphic question
00:06:38.22 than the students who did not change their mind on the first...
00:06:44.15 after the discussion.
00:06:46.17 People who stayed with their incorrect answer
00:06:48.25 only got 44% correct,
00:06:51.18 which incidentally is much better than guessing,
00:06:54.14 so maybe they learned something too during the discussion process.
00:06:59.09 The other interesting thing that we learned from this...
00:07:02.14 from this exercise is that if you look at the difficult questions,
00:07:06.29 where only about 20% of the students got it right the first time,
00:07:11.07 then statistically it was clear that
00:07:14.20 more than half of the discussion groups
00:07:17.19 did not contain anybody in them who knew the correct answer.
00:07:21.08 And yet, on the iteration after discussion,
00:07:24.20 and then on the subsequent follow-up question,
00:07:28.04 many of these groups arrived at the right answer.
00:07:31.15 So, this is a validation of the constructivist idea
00:07:35.06 that the discussion process itself
00:07:37.07 is really helping students get to the right answer,
00:07:39.23 rather than the transmissionist view
00:07:43.11 that what's happening is that the good students
00:07:45.17 are telling the weaker students what the answer is.
00:07:50.16 So, an interesting aspect of this for us
00:07:53.04 was that the students really understood
00:07:55.12 what was going on in these discussions.
00:07:57.10 We asked them a question in a survey at the end of the course:
00:07:59.28 Is it important to have somebody who knows the answer in your group
00:08:04.07 in order to have a productive discussion?
00:08:06.12 One student said,
00:08:08.06 "Often, when talking through the questions,
00:08:09.27 the group can figure out the questions
00:08:12.02 without originally knowing the answer,
00:08:13.23 and the answer almost always sticks better that way
00:08:16.20 because we talked through it instead of just hearing the answer."
00:08:20.09 And another said,
00:08:22.03 "Discussion is productive when people do not know the answers
00:08:24.29 because you explore all the options
00:08:27.01 and eliminate the ones you know can't be correct."
00:08:29.18 So, we felt that this was a real validation of the...
00:08:33.10 first of all, the clicker approach in general,
00:08:36.15 but also particularly the value of having the students
00:08:40.11 argue about them and discuss in a group among themselves
00:08:43.22 as a process... as a part of the learning process.

This Talk

Audience:

Educators of H. School / Intro Undergrad
Educators of Adv. Undergrad / Grad
Educators

Recorded: July 2014

Talk Overview

In a peer-discussion, do well-prepared students simply tell other students the answer?

Bill Wood (University of Colorado Boulder) describes the Smith et al. study, which examines whether students who change their mind as a result of peer-discussion have actually learned the concepts being tested.

Educators featured in this video:Michelle Smith, University of Maine; Tin Tin Su and Bill Wood, University of Colorado Boulder.

What did you think about this video?