Engaging students through educational science videos: Paper discussion and Q&A

Top

00:00:07:06 Hi, everyone. Welcome to this event.
00:00:09:15 We are very happy to be here and to share some insights about using
00:00:14:16 videos that are freely available online for use in undergraduate education.
00:00:19:02 We'll start with some introductions and then we'll get into the presentation we
00:00:21:20 have. And then we'll spend plenty of time in this hour,
00:00:24:10 the next hour or so to get into a Q and A with any attendees that are here.
00:00:29:08 So my name is Anderton. I am an Associate Director at iBiology.
00:00:34:03 And for those of you that might not be familiar with us. iBiology is a small,
00:00:37:14 independent nonprofit we're based in San Francisco.
00:00:40:15 And largely what we do is we provide free online videos that support the life
00:00:44:12 sciences research community.
00:00:46:06 So largely we produce videos that tell the stories and the process of science.
00:00:50:07 But beyond that, we also produce videos that support, um,
00:00:53:22 professional development in science.
00:00:55:04 So we produce courses and videos that provide professional development
00:00:58:22 opportunities as well. So that's my introduction,
00:01:02:00 and I'll hand it over to my co-host Laci to introduce herself as well.
00:01:06:09 Good morning, everyone. I'm Laci Gerhart-Barley.
00:01:08:12 I'm an assistant professor of teaching in the evolution and ecology department
00:01:12:05 at UC Davis. Um, and Brittany, should we just get started?
00:01:17:10 Yeah, one thing I will add before Laci begins with the presentation, feel free,
00:01:20:20 everyone to introduce yourself in the chat function.
00:01:24:04 We'll use the chat for kind of that side chat function.
00:01:27:17 If people want to kind of just share comments, et cetera.
00:01:30:11 And then the Q and A function in the zoom window is what we'll use for the
00:01:33:19 actual question and answer period. After the presentation,
00:01:37:06 Laci will give just to kind of set those ground rules and we heard you all to
00:01:40:16 participate. And thank you again for joining.
00:01:43:11 Yes. And as Brittany mentioned, um,
00:01:45:08 we will be using the chat during the presentation.
00:01:47:14 So I have a couple of little interactive questions,
00:01:49:12 one of which is to type something in the chat. So, uh, when we pull that up,
00:01:53:11 make sure you have the chat visible so you can see everyone's comments. Um,
00:01:56:10 and then Brittany will be keeping an eye on the Q A for the, uh,
00:01:59:09 panel part at the end. All right. And then let's get started,
00:02:05:11 uh, first to introduce you to what Brittany and I were working on here.
00:02:08:08 I want to talk a little bit about the class that we were working in. Uh,
00:02:11:08 we did this project before the pandemic began.
00:02:13:18 So this was actually an in-person class.
00:02:15:07 This photo is me in the class that we actually studied. Um,
00:02:18:07 this photo makes me kind of nostalgic because I kind kinda miss classrooms. Uh,
00:02:21:19 but part of why we chose this particular class is for a couple of reasons.
00:02:25:17 One it's rather large. Uh, the section was about 350. Uh,
00:02:29:17 it ranges from 300 ish to over 500.
00:02:33:07 And so it's a nice large class. And that means, first of all,
00:02:36:23 that you can get a lot of data out of the students. Uh,
00:02:39:03 but it also means that the demographics of the class pretty closely follow the
00:02:42:21 demographics of the university with just a couple of exceptions.
00:02:46:07 So this table I have here shows the different categories of student identities
00:02:50:13 that the university tracks,
00:02:52:05 and then this individual section compared to campus as a whole,
00:02:56:19 there's a couple of exceptions campus as a whole skews, a little bit female,
00:03:00:09 and this class tends to skew even more female. Um,
00:03:03:17 and then we also have the most noticeable difference is that we have less than
00:03:07:12 half the number of transfer students.
00:03:09:14 And that's because this course serves as the intro biology course for majors in
00:03:13:08 the college of biological sciences and the college of agriculture and
00:03:16:23 environmental studies.
00:03:18:07 But the college of bio-sciences requires that incoming transfer students have
00:03:21:17 completed the intro bio series already.
00:03:24:09 And so we only see transfer students from the college of ag majors in this
00:03:27:06 class. So our transfer percentages, uh,
00:03:29:13 consistently significantly lower than the campus average,
00:03:32:21 but otherwise pretty closely tracks the campus. Uh,
00:03:36:14 Brittany first approached me about doing a pilot study of, uh,
00:03:40:08 faculty and student feedback using iBiology videos. Uh,
00:03:44:04 we initially discussed the evolution flipped course,
00:03:46:08 which is a set of videos that iBiology has put together on evolution topics. Um,
00:03:50:21 since my class covers evolution, and ecology, I drew some videos from, uh,
00:03:54:15 other sources as well from iBiology and there's, uh,
00:03:58:07 a number of other sort of topics and, uh,
00:04:01:08 channels that iBiology has videos on. Um,
00:04:04:14 if you're curious about what all exists Brittany's happy to give some details on
00:04:08:11 that when we get to the Q and A, but there's a lot of variety there. So if the,
00:04:12:01 the topics that I'm discussing in this video,
00:04:13:20 don't really link with the classes that you teach, uh,
00:04:16:04 recognize that there's a lot more variety on the iBiology library than just what
00:04:19:16 I used for this class. Uh, so how we structured it, I chose,
00:04:24:24 uh, videos from iBiology and from other sources to use for weekly homework
00:04:29:01 assignments in the class. Uh, we ended up doing seven weekly assignments, um,
00:04:33:12 at UC Davis we're on the quarter system. So our classes are 10 weeks long.
00:04:37:05 And then since they didn't have homework assignments on exam weeks,
00:04:39:17 that came out to seven assignments and a couple of the weeks had more than one
00:04:43:22 video.
00:04:44:07 So this is a screenshot of the different videos to get a sense of what the
00:04:47:20 students were watching. Uh,
00:04:49:12 the ones with the iBiology logo here are from the iBiology library.
00:04:54:04 And then I also drew a couple from a HHMI Bio Interactive series and
00:04:59:02 one, the phases of meiosis,
00:05:00:15 one from a YouTube educational channel called Bozeman science.
00:05:05:09 And I picked these for a variety of reasons.
00:05:07:06 I wanted to have studies that videos that related to studies that
00:05:12:05 covered the content that we were discussing that week in class, um,
00:05:15:24 in particular,
00:05:16:17 because part of my goal with how these integrated into the course related to the
00:05:21:06 discussion activities. So, uh,
00:05:23:11 the class has three 50 minute lectures a week and then a 50 minute discussion.
00:05:28:08 And I put that in air quotes because it's still the full enrollment of the
00:05:31:09 class. So with 356 students in a room,
00:05:33:19 you can't really do what anyone would consider a true traditional discussion.
00:05:38:04 And so it's often treated as an additional fourth lecture,
00:05:41:15 and I've been playing around with some of the structure to try to do something a
00:05:43:25 little more student-centric and a little more focused on, um,
00:05:49:01 learning the process of science and learning about scientists, um,
00:05:52:16 in their everyday lives.
00:05:54:06 And so these homework assignments linked with that every week they had a video
00:05:58:15 homework that had one or two videos and was linked with a seven or eight
00:06:02:02 multiple choice question assignment, um, on the learning management system. Uh,
00:06:06:23 and that was due by the start of the discussion period so that the students
00:06:09:24 would have watched the video gone through the questions, um, and be, uh,
00:06:13:10 somewhat prepared and thinking about how scientists study that particular topic
00:06:16:15 for that week. And as I mentioned,
00:06:18:14 my central goal in a lot of these was to get students thinking about the process
00:06:22:20 of science, how we currently study the various topics we discuss in this class.
00:06:26:23 And on current researchers, um,
00:06:29:08 the discussion sessions were focused around the research programs of individual
00:06:33:17 Davis faculty. So there was also, uh,
00:06:35:15 a push to get students to be thinking about if they were interested in research
00:06:38:15 opportunities, what types of opportunities exist on this campus?
00:06:43:24 And then we had two sources of student feedback.
00:06:45:22 The first was that each homework assignment.
00:06:48:06 So the weekly homework assignments on a learning management system included an
00:06:52:06 open-ended voluntary ungraded zero point question, um,
00:06:56:16 asking for feedback on that particular homework assignment and video,
00:07:00:06 since these were on the assignment, they were not anonymous.
00:07:03:11 And so for that reason,
00:07:04:08 we also included an open-ended question on the end of quarter evaluations asking
00:07:08:08 about the assignment structure as a whole, and that was anonymous.
00:07:12:12 And then Brittany and I coded the responses to both of these across each
00:07:16:05 homework assignment. And to the end of quarter evaluations,
00:07:18:22 I've included just part of our coding schema here to give you an idea of what
00:07:22:15 sorts of things we were tracking. We use three parent codes,
00:07:26:05 one of which was usage.
00:07:27:16 So students talked a lot about how they were interacting with the videos. Uh,
00:07:31:09 and then we had positive categories and negative categories.
00:07:35:12 The table here shows all of the positive categories that we used.
00:07:38:16 And then the negative were often just the reverse of the positive category. Uh,
00:07:43:06 the gray lines were found in all of the homework assignments and on the end of
00:07:47:02 quarter evils, the blue categories,
00:07:49:09 the organism and the homework questions we just found on the weekly homework
00:07:53:02 assignments.
00:07:54:02 And then the orange colored ones were just found on the end of quarter
00:07:56:13 evaluation.
00:07:57:03 So there was a little bit of a difference in what students were commenting on
00:08:01:12 and, and what sort of, uh, statements they were making to us.
00:08:04:09 But for the most part, they, they were pretty consistent.
00:08:07:15 So for the rest of the talk, I'll get into the actual data.
00:08:09:24 There's a lot of graphs here.
00:08:11:10 I'll try to be really clear on what each of these graphs are showing. Um,
00:08:14:17 and if at any point it's not clear what a graph is showing,
00:08:17:10 please pop something up in the chat and I'm happy to slow down and back up on
00:08:20:10 what we're actually looking at.
00:08:22:11 Most of these figures are in the paper that's published, uh,
00:08:26:04 but some of them are new, so not all of them will be in that paper.
00:08:30:19 So the first one I'll show you is graph from the end of quarter evaluations.
00:08:34:24 These were anonymous responses, and I've included the question prompt there.
00:08:38:02 So you can see what the students were asked to respond to.
00:08:40:17 And the response rate was about 68%.
00:08:43:04 That was a little bit lower than the overall responses for,
00:08:46:08 to the evaluations as a whole,
00:08:47:09 not every student that completed the evaluations responded to this particular
00:08:50:14 question. Um,
00:08:51:19 and that's a tiny bit low for our response rate on end of quarter evils, um, uh,
00:08:56:05 for this class in the panel here,
00:08:58:10 we have it split into overall percent of comments with green being positive
00:09:02:02 comments, uh, orangeish being negative.
00:09:05:05 And then we did have some ambivalent comments, uh,
00:09:07:11 which to be considered ambivalent. It had to literally just say they were fine,
00:09:11:12 or it was okay, or something like that without any sort of clarifying detail on,
00:09:15:04 on what the students thought about it in more depth and then gray,
00:09:19:03 there's a little bit of a gray here, and those comments were irrelevant.
00:09:22:06 And the comments that fell into the irrelevant category tended to be, uh,
00:09:25:15 talking about something else other than the discussion videos.
00:09:28:03 And so they weren't really relevant to the particular question we were asking
00:09:30:17 here,
00:09:32:01 panel B then shows the percent of the comments in individual categories that we
00:09:36:02 were tracking color coded green for positive comments and orange for negative.
00:09:41:07 And so you'll see here,
00:09:42:10 we have a variety of positive and negative comments falling out into the top 10.
00:09:46:16 Um, and if you look closely at the titles, you'll see that, uh,
00:09:48:16 some of them are directly contradictory.
00:09:50:06 So intellectually stimulating and intellectually unstimulating are in the top 10
00:09:55:09 as are emotionally stimulating and emotionally unstimulating, et cetera.
00:09:59:18 And so what Brittany and I took from this,
00:10:00:25 as we were looking through these data, uh, were two things. First of all,
00:10:04:06 the student body in the class is not a monolith, right?
00:10:06:14 They have different opinions about the assignments. Um,
00:10:09:15 and having read through a lot of these comments,
00:10:12:14 individual students would often report both feelings depending on what video
00:10:16:11 they were talking about.
00:10:17:09 So it was not uncommon for students to say something like, you know,
00:10:20:04 I found this video and this video really,
00:10:22:16 really interesting and great for these reasons.
00:10:24:13 And then I found this video and this video, not so much for these reasons.
00:10:28:02 And so for, for those purposes,
00:10:30:07 we then decided to dive into the individual responses on each homework
00:10:33:24 assignment.
00:10:34:17 Cause the end of quarter evils came off as being a little bit too coarse to get
00:10:38:04 a lot of specifics about what the students were responding to, um, week by week,
00:10:42:17 especially when we were asking them to reflect on it at the end of the quarter,
00:10:45:05 you know, a video that occurred 10 weeks ago. So for the rest of the talk,
00:10:49:06 I'll focus on just the responses to each week's individual assignment.
00:10:53:06 Remember that those are not anonymous, um,
00:10:56:07 which may or may not have influenced how students responded as you'll see,
00:10:58:20 they were not necessarily shy about criticizing things that they didn't care
00:11:02:00 for. So I do mostly trust their responses on that. Um,
00:11:06:09 the first thing I'll show you is those, uh, usage patterns that I mentioned, uh,
00:11:10:07 these show by each video. So these will be the codes we use for the video.
00:11:14:03 HW stands for homework. So homework one, and then 2, 3, 4, et cetera.
00:11:19:04 And then this is the title. So as we go through the different types of videos,
00:11:22:15 uh, you'll see the different specifics about the different videos.
00:11:26:08 And what we noticed here was that, uh, first of all,
00:11:28:19 the first assignment had much more usage comments, uh, than the others did.
00:11:32:14 So you see,
00:11:32:24 we have a separate axis here because there were a lot more comments of this type
00:11:35:17 on the first one. And part of that was,
00:11:37:16 I think there was a bit of a learning curve on a new assignment, um,
00:11:40:12 accessing the video through the LMS, et cetera. Uh,
00:11:43:06 this first video was rather long.
00:11:44:21 And so I had the students just watch a part of it.
00:11:47:00 And so they started in the middle of the video and many of the students found
00:11:49:10 that very confusing. Uh,
00:11:51:01 so we got more sort of technical issue comments on the first one than on the
00:11:54:03 others. Um, other things of note, the students watch the videos multiple times,
00:11:58:21 every single video students reported, uh, reviewing the video more than once.
00:12:02:17 And that was especially true for videos that the students found generally more
00:12:06:05 difficult or confusing.
00:12:08:21 They also will use and need to know how to access captions and transcripts. Uh,
00:12:13:08 so I, um,
00:12:14:13 somewhat unfairly assumed that students would know how to access captions and
00:12:18:12 transcripts on YouTube and on the iB iology videos.
00:12:22:09 And I realized after I read through the comments on the first video,
00:12:25:21 that many students, all the ones in this light blue bar here, uh,
00:12:30:07 didn't know how to access those.
00:12:31:09 And weren't able to see that they were on there. And so the next week in class,
00:12:34:14 I showed students in lecture how to pull up the captions and transcripts on the
00:12:37:20 videos. And for the most part, then those comments went away.
00:12:40:14 Students still reported using the captions. Um,
00:12:42:24 and in a couple of cases appear to have forgotten how to access those.
00:12:45:15 But for the most part that helps students access them.
00:12:51:24 Additionally, many students reported changing the video speed, um,
00:12:56:07 difficult videos.
00:12:57:07 They reported watching a little bit slower so that they could follow along.
00:13:00:09 Step-by-step a little bit more, uh,
00:13:02:10 but then speakers that students considered to be slow talkers were watched
00:13:06:09 faster, uh, to speed up to the timing of the video a little bit.
00:13:10:02 So this gives us kind of an idea about how students were engaging kind of
00:13:13:14 technologically with the videos. Uh,
00:13:16:11 and now we'll move on to their comments about what they liked and disliked about
00:13:19:06 the videos. So much like the previous slide that I showed you,
00:13:23:13 this is a top 10 about what comments were most common across all of the homework
00:13:28:04 video, weekly assignments. And I've included the question prompt here,
00:13:31:20 as you can see what the students were asked to respond to. This was,
00:13:37:15 um, the same comment for each homework video each week.
00:13:40:00 So they were familiar with it after the first week. And here's similarly,
00:13:44:23 we have the top 10 comments.
00:13:46:05 So the most common across all of the weekly assignments was intellectually
00:13:49:21 stimulating. We have a number of positive ones here.
00:13:52:02 You'll notice I have covered up the comments for what was most common in the
00:13:55:18 negative categories, because that is our first interactive question.
00:14:01:02 So in a second here, Brittany,
00:14:03:08 will open up the poll question and there are two,
00:14:06:15 you'll see there's two negative categories here.
00:14:09:10 And so you'll be able to pick more than one option on the poll. Um,
00:14:13:01 and my question for you then is which of these negative comments, uh,
00:14:17:01 were the two that were most common.
00:14:37:14 All right. I'll give you a few more seconds.
00:14:39:01 We've got about two thirds of the folks have voted,
00:14:54:24 all right. And then I'll end the poll and show you the results.
00:15:01:18 All right. So our, our most voted option was,
00:15:05:01 the videos were too long and then it looks like intellectually unstimulating,
00:15:09:07 which we included comments like dense or confusing.
00:15:12:11 There are two most common.
00:15:17:03 So the actual results are that it was
00:15:21:21 intellectually unstimulating. So that one was correct.
00:15:24:01 And then that they disliked the homework questions. Um,
00:15:26:15 this arising in the top 10 here was due primarily to a single video.
00:15:31:01 And so we'll come back to that comment at the end about, uh,
00:15:33:11 why the homework questions themselves, uh, appeared in the top 10 like this. Um,
00:15:38:01 I will tell you, I wrote all of the questions that went with the assignments.
00:15:41:11 So iBiology does provide some instructor resources as does HHMI for some of the
00:15:45:23 videos. Uh, but for these homework assignments,
00:15:48:01 I wrote them in some cases specifically to link to other things that we were
00:15:51:05 talking about in the class. Uh, so those students maybe necessarily,
00:15:54:20 weren't a big fan of that. All right.
00:15:58:09 Most of the rest of the graphs I'll show look at responses week by week to
00:16:02:09 individual videos on the left here is, uh,
00:16:05:14 the number of student comments per video, uh, broken out by that positive,
00:16:09:06 negative, ambivalent and irrelevant.
00:16:11:10 And you'll see the response rate varies quite a bit across the quarter. Um,
00:16:14:03 it was the largest for the first week. Um,
00:16:16:01 and then as students kind of got into the, um, you know,
00:16:18:14 into the quarter and into the workload,
00:16:19:25 the response rate did slow down a little bit when it got really low in week
00:16:23:12 three, I reminded students that their comments were of value to me.
00:16:26:21 And it bumped up a little bit when week four and then tapered off again towards
00:16:30:02 the end of the quarter panel B is the same data just presented as the percent of
00:16:34:19 student comments. So it's a little bit easier to compare, uh,
00:16:37:09 which videos the students preferred, or maybe didn't prefer quite as much.
00:16:42:24 I want to look a little bit into the videos that we consider generally
00:16:48:02 well-liked by the students that have the really high green bars.
00:16:51:04 And those were videos two over the elephant census five over lizard evolution,
00:16:55:25 and then the two videos for week seven,
00:16:58:05 which were about different fossil organisms.
00:17:01:05 And the next graph that I'm going to show you is a little bit messy.
00:17:03:11 So prepare yourself. It's, uh,
00:17:05:12 all of the positive comments across all of these videos,
00:17:08:14 so that we can see the breakdown of the different positive comments within these
00:17:11:24 green bars. Um, so as I said, this is a little bit of a mess. Uh,
00:17:16:10 the colors are not all the different blues are not necessarily super visibly
00:17:19:16 different,
00:17:20:01 but these tall sort of medium blue ones are intellectually stimulating.
00:17:23:19 And you'll see that those are pretty common across all of the videos,
00:17:26:08 even the ones that we don't identify here as being necessarily the most liked by
00:17:30:17 the students, but the ones that are well-liked especially video two,
00:17:34:15 you can see that while their most positive comments are not
00:17:39:14 really higher than a lot of the others.
00:17:41:10 Students liked a lot of different things about this video.
00:17:44:04 So they're getting a lot of comments across all of these positive categories.
00:17:47:24 Whereas some of the other videos didn't rate quite as highly with the students
00:17:50:24 are getting a lot of comments in a couple of categories,
00:17:53:05 but not consistently across all of those positive things. Uh,
00:17:57:09 video seven students consistently found really intellectually stimulating, uh,
00:18:01:19 and in the Finding Tiktaalik video,
00:18:03:10 students responded really positively to the speaker in this video.
00:18:06:01 And so those couple of bars stand out really highly.
00:18:10:06 Brittany and I hypothesized that students might prefer the documentary style
00:18:13:08 videos,
00:18:13:18 which were the HHMI bio interactive video compared to the ones that were more of
00:18:17:05 a lecture style. And so we broke down some of those responses, uh,
00:18:21:05 just by the video style.
00:18:23:07 So this figure shows the percent of student comments that specifically commented
00:18:27:06 on the speaker or video style both positively and negatively,
00:18:31:11 and then we've organized them by video style.
00:18:33:14 So the documentary style videos are in the left-hand here.
00:18:36:16 The middle is the more lecture style videos.
00:18:39:15 And then there was one video that was the handwriting on the whiteboard.
00:18:42:18 And so we consider that a different style since there wasn't really a person
00:18:45:06 visible in it at all. Um,
00:18:47:11 and you'll see here that the documentary style videos are not consistently
00:18:50:25 well-liked at least in terms of explicitly stating that they preferred that
00:18:54:14 style. Uh, when I first made this graph, I was really surprised because as I,
00:18:58:22 as you may have noticed on the previous slides,
00:19:01:09 the elephant census video was consistently pretty well liked by students,
00:19:04:22 but not necessarily explicitly because, because of it being a documentary style,
00:19:09:14 uh, if we pull the top 10 common categories for that video,
00:19:12:19 you'll see that the style comes out as the seventh,
00:19:15:07 most popular thing students were commenting on.
00:19:17:23 And so they saw a number of other things that they responded to positively that
00:19:21:19 they commented on much more than the explicit documentary style of the video,
00:19:26:00 um, including things like, uh,
00:19:27:09 the process of science and the link to course content.
00:19:30:12 Those were particularly common in this video because the researchers are using
00:19:35:04 transect and quadrats sampling methods in the video and the students had just
00:19:39:08 learned how to use those sampling procedures themselves in the lab that week.
00:19:43:01 And so, um,
00:19:43:24 a lot of them were responding more positively to those sorts of connections to
00:19:47:11 their experience in the course than they necessarily were to the documentary
00:19:50:14 style. So even though this was a very well-liked video,
00:19:53:16 it wasn't necessarily because of that. And similarly,
00:19:56:07 other videos in different styles can receive as many positive comments or indeed
00:20:00:18 some more positive comments about the speaker and video style,
00:20:05:04 even if it's not, um, a documentary style,
00:20:12:07 uh, moving on to some of the things they disliked.
00:20:13:22 There were three videos that seem to be a little bit less
00:20:18:11 positively responded to by the students. Um, and those were homework,
00:20:21:24 one on the virus adaptation, number three on the Hardy Weinberg and number,
00:20:25:16 number four on, uh, genetics of morphology.
00:20:29:09 And so for the same kind of graph,
00:20:31:02 we'll look at the negative comment categories here. Um, and in this case,
00:20:35:07 unlike the previous ones where we saw that they were, um,
00:20:38:04 mostly responding positively to things with the sort of general categories of
00:20:41:13 intellectually stimulating or emotionally stimulating, et cetera. Um, here,
00:20:45:11 they tend to pick out with the exception of video one, in videos three and four.
00:20:49:05 They pick out much more specific things that they didn't care for in particular,
00:20:52:14 in video three, they really did not care for the homework questions.
00:20:55:11 This one is driving the fact that that category appeared on the top 10 across
00:20:58:23 all videos and in homework four,
00:21:01:15 they were responding primarily to the length that they found this video much too
00:21:04:20 long. Um,
00:21:07:09 I want to look though a little bit in detail at the top 10 response categories
00:21:11:08 to each of these videos,
00:21:12:24 because if we look just at this breakdown of positive to negative comments,
00:21:16:18 these videos might seem, uh, somewhat less well-liked than the others.
00:21:20:04 But if we look at these top 10 categories for homework one,
00:21:23:12 homework three and homework four,
00:21:26:06 you'll see that there are a number of other positive comments that are appearing
00:21:29:17 in the top 10 most common. Um,
00:21:32:18 even though these yellow bars are a little bit bigger than the others,
00:21:35:15 there is a lot of green here occurring as well, right? If you remove the, uh,
00:21:39:21 ambivalent and irrelevant comments, then of the comments that had an opinion,
00:21:44:02 uh, over half of them are positive,
00:21:45:24 even in the videos that were somewhat less well-liked. In fact,
00:21:48:21 in two of these videos, the most common category is a positive category. Uh,
00:21:53:04 it's just that there were other negative categories that were fairly common as
00:21:56:13 well. All right.
00:21:59:24 So I want to look at each of these two specific, uh,
00:22:03:02 criticisms that the students made in a little bit more detail,
00:22:05:12 the first being the length of videos and the second being the homework
00:22:08:12 questions. So this is our second poll question,
00:22:11:19 which Brittany will open here in a second. Um,
00:22:14:00 and this relates to how long is it,
00:22:17:06 is it recommended that videos be, I've included a couple of references here,
00:22:21:20 if you're interested in this.
00:22:23:01 And then how many of the videos that I used in this class were of that length.
00:22:26:18 Um, and as you're responding, I'll give you a hint
00:22:31:08 too long,
00:22:31:24 did appear in the top 10 comments on the responses to the assignments as a
00:22:36:02 whole, at the end of the, uh, end of quarter evaluations,
00:22:39:14 but did not appear in the top 10 most common response categories across
00:22:44:13 each individual homework assignment. So take that as you will. Um,
00:22:49:06 and tell me how many of these videos you think fell in that range?
00:22:53:04 I'll give you a second to respond.
00:23:11:03 All right. I'll give you a few more seconds. We've got a little bit over. Oh,
00:23:14:05 two-thirds of you responding now.
00:23:20:20 All right. All right.
00:23:25:03 You can now see that most folks responded that, uh,
00:23:29:02 the recommended length is about 12 to 15 minutes and that three of my videos
00:23:32:18 fell in that range. Uh, the correct answer here is actually letter B, uh,
00:23:36:12 the recommendations from these sources recommend no longer than nine or 10
00:23:40:22 minutes. Um, and only one of my videos, uh, fell in that, uh,
00:23:45:18 particular length, which at first concerned me a little bit. I was like,
00:23:50:17 when we run the numbers on how many complaints there were about length,
00:23:53:23 how bad is it going to be? And it was actually not as bad as I was anticipating.
00:23:58:04 Uh, this graph is arranged a little bit differently than the others.
00:24:00:08 So our X axis here instead of being the order of the homeworks is the total
00:24:04:21 video runtime. So our shortest video was homework two,
00:24:08:13 which was the elephant census video at a little under eight and a half minutes.
00:24:12:14 And then our longest one was homework assignment four at over 36 minutes.
00:24:17:01 And then this shows you the percent of comments about length, um,
00:24:20:16 on those student homework assignments with positive ones showing in green, uh,
00:24:25:01 negative comments showing in yellow, uh,
00:24:27:00 and then statements that didn't have any comment about length in gray.
00:24:30:02 And you'll see the vast majority of students didn't state an opinion either way
00:24:33:04 on, on length. And as you might expect, uh,
00:24:36:13 generally speaking longer videos receive more complaints about the length and
00:24:40:12 shorter videos received more positive comments on it,
00:24:42:20 not being too time consuming. Uh, but there was, uh,
00:24:46:13 two things that stood out to Brittany and I,
00:24:48:12 when we looked at this and the first was the difference in the two longest
00:24:52:00 homeworks here.
00:24:53:14 So these two homework assignment videos differed only at about 50 seconds worth
00:24:57:10 of length, and yet received rather different student perceptions on length. Uh,
00:25:02:03 the longest one homework four, uh,
00:25:04:19 had many complaints about length and no positive comments,
00:25:08:04 whereas homework seven at less than a minute,
00:25:10:03 shorter had a few negative comments about length,
00:25:12:16 but several positive comments that this one was not too time consuming and was a
00:25:15:19 good length according to students. And the primary difference between these two,
00:25:20:14 uh, that we identified was that homework seven was two videos,
00:25:24:05 which totaled 35 minutes and 23 seconds while homework four was one video,
00:25:28:17 which was itself, uh, over 36 minutes long.
00:25:31:24 And so this led us to look at the two homework assignments that had two sets of
00:25:35:05 videos and that's homework seven and homework three. And in both cases,
00:25:39:19 these received fewer complaints on length from students than we might expect
00:25:44:17 compared to other videos in this series.
00:25:47:10 And so this led us to believe that students maybe find multiple short videos,
00:25:52:02 a little bit more palatable than a single long video.
00:25:55:07 And this particular result led to me, uh,
00:25:58:12 a change in how I taught the asynchronous classes that I've taught during the
00:26:01:15 pandemic, uh, where instead of recording just one standard 50 minute lecture,
00:26:05:14 like I would've done in front of the students,
00:26:07:16 I recorded 15 to 20 minute chunks that totaled about 50 minutes.
00:26:13:03 And, uh, I can back up that on my evals from those classes.
00:26:16:06 Students seem to prefer having a series of shorter videos that they could watch.
00:26:21:04 One, take a break and come back to rather than having one big,
00:26:24:20 long behemoth video. Uh, and so this is, uh,
00:26:29:01 something that came out of this study and influenced my teaching during the
00:26:31:18 pandemic, which I found really useful. Uh, the other complaint that we saw,
00:26:37:01 um, that stood out on one of the videos was this idea around the questions
00:26:40:03 themselves, which as I mentioned, I wrote myself. And so that was a, uh,
00:26:45:20 interesting criticism. Uh, here we have by homework assignment,
00:26:51:03 uh, negative and positive comments on the questions.
00:26:53:20 You'll see each homework assignment did receive some positive comments on the
00:26:57:22 questions, uh,
00:26:58:22 which some proportion of those might have been some degree of buttering me up a
00:27:02:04 little bit since these were not anonymous and students' names were associated
00:27:05:11 with them, but they were not shy about, uh,
00:27:08:18 reporting that they did not care for the homework assignment,
00:27:10:23 the questions on homework assignment three,
00:27:12:21 especially relating to the Hardy-Weinberg video. And so my question for you,
00:27:16:22 and this is where we're going to open up the chat.
00:27:18:05 So if you haven't had the chat open, uh, not the Q and A, but the chat itself.
00:27:23:02 Um, and tell me, take a second and think about it,
00:27:26:11 and then tell me what you think was different about this homework assignment,
00:27:30:02 uh, from the other ones. I'll give you a second to think about it.
00:27:55:03 All right. We've got a vote for, uh,
00:27:57:05 the theme or the topic of the video is a little less interesting.
00:28:00:09 Maybe there were more complicated math calculations.
00:28:04:01 Maybe there were free response questions, um,
00:28:06:13 some more votes for math or computational skills.
00:28:11:20 Um, I agree that the,
00:28:12:19 if there had been free response questions that probably would have been more
00:28:15:03 frustrating to students, um, all of the assignments were all multiple choice,
00:28:18:12 so that one is not applicable here. Um,
00:28:21:23 you're right in saying that math was related to this.
00:28:24:08 So if you teach Hardy-Weinberg,
00:28:26:07 you maybe are familiar with the Hardy-Weinberg equilibrium calculations that
00:28:29:09 relate to allele and genotype frequencies,
00:28:31:24 and students generally find those a little bit frustrating and confusing.
00:28:35:10 And that was a component of this assignment was the only one that required
00:28:39:01 direct calculations of data presented in the video. Uh,
00:28:42:08 and this was the only video that asked students to kind of work beyond what was
00:28:46:22 just presented in the video. So, uh,
00:28:49:15 this video follows a fictional population of squirrels and calculates some
00:28:53:08 allele and genotype frequencies, uh, in this squirrel population. Uh,
00:28:56:18 and then the video ended and I asked the students to consider, uh,
00:28:59:14 in that population, if a mutation arose,
00:29:01:12 how might that change various allele and genotype frequencies.
00:29:04:14 And since the video didn't discuss what happens when m utations arise in a
00:29:08:12 population and how that affects frequencies, uh,
00:29:10:18 the students felt a little bit stranded on having to work through that without
00:29:13:10 any sort of assistance in the video. Um, and so in retrospect,
00:29:16:18 this is one place where, uh,
00:29:18:06 some framing or maybe a little bit more guidance on my part on the questions,
00:29:21:22 uh, could have helped, um,
00:29:23:20 guide them on what was expected there a little bit better.
00:29:29:22 And then there was a little bit of an unexpected result that just makes me
00:29:32:20 really happy.
00:29:33:14 And it's one of my favorite little findings from this particular study and that
00:29:37:12 relates to the organisms or regions that were presented in the videos.
00:29:42:01 So all of the videos related to some sort of, um, organismal or,
00:29:46:02 or biological structure, right,
00:29:47:18 even though meiosis video was about cells and cell division, uh,
00:29:51:04 the virus adaptation video focused on viruses, which, you know,
00:29:53:20 you may or may not consider alive. Um,
00:29:55:23 but many of the videos focused on a particular taxonomic group or, or region ,
00:30:01:04 uh, the genetics of morphology video, which covered, uh,
00:30:03:18 Hopi Hoekstra's beach mice, which are just super adorable.
00:30:07:10 And then the elephant census video of course had a lot of like sweeping, uh,
00:30:10:17 panoramic views of, uh, elephants in the field.
00:30:15:19 And, uh, the Amazon deforestation video, uh,
00:30:19:15 he focused on how do you experimentally manipulate rainfall in an entire
00:30:24:04 ecosystem.
00:30:24:23 And so he spent some time showing how you set up these plots in order to catch
00:30:29:13 rainfall and simulate a drought, even if one isn't actually occurring. Uh,
00:30:34:03 Jonathan Losos's, uh, evolution, lizard evolution, video.
00:30:37:16 He actually captured lizards from the surrounding area. And, and then, uh,
00:30:41:08 put them on different types of substrates in the ecosystem to show how different
00:30:44:16 sizes of lizards could move on different surfaces.
00:30:47:05 And he jokingly in the video refers to this as the lizard Olympics to see which
00:30:50:14 ones are better running in different parts of the habitat. Uh,
00:30:54:11 and then the last two videos for homework seven,
00:30:56:11 both focused on fossil organisms and what we can learn from, uh,
00:31:00:24 how organisms are preserved and the traits that we can identify in fossils.
00:31:04:15 So my question for you is which of these organisms,
00:31:07:00 or in the case of the rainforest region or habitat were best liked by the
00:31:11:07 students.
00:31:11:21 So Brittany will open the chat here in a second and tell me which of these you
00:31:15:08 think students preferred.
00:31:34:05 All right. We're at about, two-thirds give you a little bit more time to vote.
00:31:49:21 All right. Let's see what you put.
00:31:54:25 I like that there's a tie here. We have, uh,
00:31:59:06 a debate between elephants being obviously the most interesting and the lizard
00:32:03:24 Olympics. So I can show you that the actual results here,
00:32:09:10 um, and you were smart to, to think that these would be both in the top.
00:32:12:09 They were both, uh, much more liked than the others,
00:32:14:24 but the lizards did eke out the elephants by just a little bit. Um,
00:32:21:08 the fossils were still well-liked and Hopi's mice are extra cute. Um,
00:32:25:18 and then the Amazon rainforest did receive a number of comments about liking to
00:32:29:15 see such a habitat that the students find very iconic and enjoy learning about.
00:32:34:08 Um, but the lizards did come out ahead.
00:32:35:20 You'll notice there were a couple of negative comments about, um,
00:32:39:13 not really enjoying seeing these organisms. And so this,
00:32:42:13 this pattern could have been really different if we had chosen videos around,
00:32:46:22 uh, organisms that students might have some sort of phobia around.
00:32:49:20 I do use a spider mating behavior example in class,
00:32:53:08 and I do sometimes have students who are arachnophobic and, uh,
00:32:56:13 respond very negatively to that. So I've had to remove a lot of the spider, uh,
00:32:59:10 images from that lecture. Uh,
00:33:01:10 but so fortunately we chose organisms that for the most part,
00:33:04:10 students responded really positively to.
00:33:06:21 And this led to me thinking about how important the imagery and the connection
00:33:10:21 to the taxonomic group of study is to students when they're watching something.
00:33:14:08 A lot of the videos,
00:33:15:04 since we're focused on the process of science and the results of the study,
00:33:18:20 do have a lot of data in them,
00:33:20:11 but backing that up with some introduction to the organism itself to the habitat
00:33:25:08 itself and to how these beings operate in the real world was something that
00:33:28:24 students responded pretty positively to.
00:33:32:06 And then the last graph I want to show you,
00:33:33:24 it was what Brittany and I were most interested in,
00:33:36:04 which is why we really started this. And that was,
00:33:38:21 do students generally engage with these or,
00:33:41:13 or do they find it some sort of busy work? And so on the top panel here in a,
00:33:45:18 we have the positive categories that I was most personally interested in.
00:33:49:07 Do they find them intellectually or emotionally stimulating?
00:33:52:02 Do they find the content accessible?
00:33:53:24 Do they see how it links to the things that we're learning in class and the
00:33:57:05 world around them?
00:33:58:15 And I mentioned part of my explicit goal was to show them how researchers study
00:34:02:25 these topics.
00:34:03:15 And so are they seeing those components in these videos and then panel B is the
00:34:07:17 reverse of that. Uh, do they find them intellectually,
00:34:10:08 emotionally unstimulating or they're boring or overwhelming or dense are they
00:34:13:25 inaccessible in terms of, um,
00:34:16:14 students maybe not being ready to talk about the specifics of how we set up
00:34:20:21 study designs and how we test hypothesis and,
00:34:23:11 and some of the jargon related to the specifics of those studies,
00:34:27:03 which we may not have yet covered in a lot of detail in an intro class, um,
00:34:30:06 or do they generally find them irrelevant to the course and to their life.
00:34:33:19 And so I found this, uh, this panel really promising in that, um,
00:34:37:13 all of the videos had at least some students find them intellectually and
00:34:41:14 emotionally stimulating,
00:34:43:02 considered them accessible and could see how they linked to the course.
00:34:45:20 And in many cases also to non course events,
00:34:48:15 you'll notice the orange bar is particularly high here in the Amazon
00:34:52:03 deforestation.
00:34:53:16 Many of the students were familiar with the fires that had been occurring in the
00:34:56:12 Amazon.
00:34:57:05 And were really interested in learning more about that ecosystem and about what
00:35:00:05 drives these dynamics of change in the Amazon.
00:35:03:12 And so they were really interested in that. Um, and,
00:35:07:19 even the videos that students maybe didn't didn't connect to quite as strongly,
00:35:11:07 still had some positive reactions.
00:35:13:04 And so there was nothing that was really universally disliked,
00:35:16:03 which was promising. And that leads to our take-home points.
00:35:20:03 The first of which was our central question, right?
00:35:22:05 That these sorts of videos can promote student engagement with the content, uh,
00:35:26:12 though some more than others, uh, which is not surprising. Uh,
00:35:29:10 they do seem to prefer documentary style videos over lecture, style videos. Uh,
00:35:33:24 however,
00:35:35:11 you could still get a lot of positive responses to non-documentary style videos.
00:35:39:22 If they had other things that the students respond positively to such as a
00:35:44:02 charismatic speaker or a charismatic organism of focus,
00:35:47:20 or a really obvious link to the course content or to the world at
00:35:52:17 large students do prefer shorter videos, which is not at all surprising.
00:35:57:19 Um, but if you want to assign more content,
00:36:00:04 multiple shorter videos might be a little more palatable for them than a single
00:36:04:04 long video that's, uh, even if they're the same total length.
00:36:08:09 And then lastly, scaffolding and framing are important. In retrospect,
00:36:11:15 I wish I had scaffolded the first assignment a little bit better for them.
00:36:15:14 Cause that seems like a pretty big first step for the students. Um,
00:36:19:09 and we outlined in the paper,
00:36:20:09 some of the ways that I did frame the assignments that I think was useful, uh,
00:36:23:21 and some of the things I would do a little differently, um,
00:36:26:09 if we use this type of homework assignment again, um, and with that,
00:36:31:10 I want to remind you of our email addresses here. Uh,
00:36:33:18 Brittany and I are definitely happy to respond to any questions about the
00:36:37:08 project or the study.
00:36:39:03 And then I have a link to the full paper there it's open access,
00:36:42:02 so everyone should be able to access the full paper.
00:36:46:06 Great, thank you, Laci, for that really exciting presentation,
00:36:50:01 one thing I will add from our perspective and publishing this paper and sharing
00:36:54:16 it as a resource and an open access publication, um,
00:36:57:25 for those of you that are interested in seeing exactly what videos were used.
00:37:02:03 I know we touched on that briefly on a slide here, but I believe it's table two.
00:37:05:22 In our publication has that information really well laid out in a table
00:37:10:19 in the publication, um, with,
00:37:12:10 I believe hyperlinks to the videos so that you can access those and understand
00:37:16:01 why Laci chose them and what topics and concepts they can link to in your own
00:37:19:11 classes, if you teach evolution and ecology.
00:37:22:24 So now we're going to switch from focusing on the chat.
00:37:26:06 I tried to respond to questions in real time, just for clarification,
00:37:29:16 to focusing on questions in the Q and A,
00:37:31:19 and I'll serve as the emcee here and feed Laci,
00:37:34:21 some questions for you and unsurprisingly, I can see,
00:37:37:21 we have a lot of active educators on this webinar,
00:37:41:13 and a lot of the questions are centered around the homework assignments.
00:37:45:13 So let's start with the logistics. Um,
00:37:49:03 one question came in Laci,
00:37:52:04 what was being used to collect the responses to the homework questions.
00:37:58:02 Uh, I set up all of the homework assignments in our learning management system,
00:38:01:04 which at Davis is Canvas.
00:38:02:18 And so they were canvas quizzes that the students completed on their own time
00:38:06:18 prior to the discussion. So I'm trying to think back to,
00:38:09:03 I think the discussion that quarter was on Fridays.
00:38:11:12 And so I opened it up Monday morning and they had until they started the
00:38:15:08 discussion late Friday afternoon to watch the videos and answer the questions.
00:38:20:04 Um, I also saw there was a question there about how do you make seven or eight
00:38:24:01 multiple choice questions on a nine minute video? Um,
00:38:26:15 and that's a good question.
00:38:27:11 So most of the videos were much longer than that and gave me a lot more content
00:38:30:15 to deal with. Uh, but the, uh,
00:38:32:19 elephant census video was only eight and a half minutes and it did have seven or
00:38:36:03 eight questions about it. And how I did that,
00:38:38:03 one was that that video focuses on how they are gathering the data and those
00:38:43:01 data have since been published.
00:38:44:05 So I pulled the papers that came from the elephant census project that was
00:38:47:07 highlighted in the video.
00:38:48:14 And the students answered mostly questions about the data from the study and not
00:38:52:20 necessarily just the video, because the video just follows, um,
00:38:55:22 what they are doing, how they are gathering data. And so some of it,
00:38:59:14 I included questions on, you know, why did they set up the study this way?
00:39:03:08 You know, why are they making this choice instead of this choice?
00:39:05:16 Cause they outline some of those decision-makings in the video and then the rest
00:39:08:13 of it was about data that they were gathering in the video that were later
00:39:11:15 published.
00:39:14:12 Great. And just to clarify, I know this question was briefly touched on,
00:39:17:22 but it's good to always reiterate here. Um,
00:39:20:06 for the homework questions that you added,
00:39:21:18 you added seven to eight per homework.
00:39:24:20 Did you take those from existing resources or did you kind of write those on
00:39:29:10 your own?
00:39:30:20 Um, I mostly wrote them on my own,
00:39:32:02 but I did refer for ideas and inspiration to some of the instructor materials
00:39:36:15 that both HHMI, bio interactive and iBiology provide. Um,
00:39:40:13 I tried to make them rather specific to the way we present material in the
00:39:44:15 course.
00:39:44:24 And so a lot of times I was writing questions about the video they were watching
00:39:49:10 that links to things we had talked about before,
00:39:51:18 which of course the instructor provided materials don't do because they're very
00:39:54:12 focused on the video,
00:39:55:20 but I did refer to those for some ideas about what types of things to ask,
00:39:59:06 but I didn't, I then did end up writing all of them. I think,
00:40:02:12 I can't remember using anyone that was pre prepared by anyone else.
00:40:06:18 Yeah. And just from the perspective of, of iBiology as a resource,
00:40:09:17 we find that to be the case that educators use questions as a starting point,
00:40:14:03 and then will modify them to their own teaching needs and assessment needs. Um,
00:40:18:12 there is a question from somebody who would like to know whether you can share
00:40:22:09 the questions that you use for your homework assignments and how you might do
00:40:25:16 that. If that's possible.
00:40:27:05 I can, um, I don't know what would be the easiest way to do that. Brittany,
00:40:31:15 we have talked about me at least sending you the ones for the iBiology videos,
00:40:34:17 which could be included in your instructor materials, uh, um, that you provide.
00:40:39:07 I don't know that HHMI would necessarily care about what I wrote for their
00:40:42:02 videos. Um, but I'm happy if you want,
00:40:43:17 like shoot me an email and I'm happy to just send you the word doc that has all
00:40:46:10 of them in it. If you're curious.
00:40:48:03 Yeah. I think what we can do on the iBiology side of things.
00:40:50:18 So five of the nine videos were from our own resource.
00:40:53:06 We can post Laci's questions on our videos, on our website, ibiology.org,
00:40:57:22 But for the other ones you'll need to reach out to Laci directly or myself
00:41:00:14 directly. Let me share the email there.
00:41:05:23 So did you do any other post assessments besides those multiple choice questions
00:41:09:21 with the homeworks?
00:41:11:20 I did. So each video had, um,
00:41:14:07 a related question that appeared on the exam that was related to that content.
00:41:18:10 And usually that question was a figure that had been presented in the video or
00:41:22:24 was included in the homework assignment and asked students to interpret or draw
00:41:26:08 conclusions from that, that data figure. Um,
00:41:28:17 that's a major part of the purpose of the homework assignments and the
00:41:31:13 discussion sessions is getting comfortable with interpreting data figures and
00:41:35:06 tables and, you know, getting kind of savvy with that sort of skill.
00:41:39:03 And so that was the primary goal on the exam questions, um,
00:41:42:18 as they related to the videos later.
00:41:46:02 Great. And this is a great question.
00:41:49:01 I think it's obvious that everyone's thinking about the logistics of,
00:41:52:17 of teaching students, engaging with students.
00:41:54:22 How did you make sure students really watched the videos?
00:41:56:21 I think the homework assignment was one part,
00:41:58:18 but was there any other thing that you were using in the learning management
00:42:01:00 system to double check?
00:42:03:02 No. The questions were the type that you couldn't really answer them without
00:42:05:18 watching the videos for the most part. Um, a couple of them. So on the,
00:42:10:08 I mentioned on the elephant census video, some of the questions were, uh,
00:42:13:24 data that were not presented in the video. And so those,
00:42:16:02 I suppose you could have answered without watching it,
00:42:18:22 but the vast majority of them were things that were discussed in the video.
00:42:22:24 So for example, in, uh, the mice,
00:42:26:24 the genetics of morphology video with Hopi Hoekstra's mice, um,
00:42:30:12 a lot of the questions there related to like this was her hypothesis.
00:42:33:20 What evidence does she present as to whether or not that hypothesis was
00:42:36:10 supported or, you know, did, did she determine that that hypothesis was correct?
00:42:40:13 So you couldn't really answer that question without having watched the video.
00:42:44:21 I mean, you could answer it, it would just be hard to answer it correctly.
00:42:50:18 And a question, another logistical question.
00:42:54:02 So did you provide the questions during the video or were they interpolated
00:42:57:16 using some sort of software or were they posed at the end?
00:43:00:18 Excellent question. They were just, uh, separate. So the,
00:43:03:21 the canvas quiz was on the LMS and then it just had a link to the video.
00:43:07:12 And so they could have them open in different pages and they could follow
00:43:10:04 through the questions as they watch the video,
00:43:12:08 but it wasn't embedded in a video. Um,
00:43:14:18 I had not used that type of technology at that point in the class.
00:43:18:13 I've now done quite a bit of that, um, teaching asynchronously, uh, for,
00:43:22:13 for the pandemic. So now I think if I was doing this sort of thing again,
00:43:26:25 I would use something like PlayPosit and embed the questions in the video. Um,
00:43:31:04 the only thing that makes me hesitate on that a little bit is that, um,
00:43:35:11 not all of the questions related explicitly to what was in the video, right?
00:43:38:18 Like I mentioned, some of them were about data that were published later,
00:43:41:07 which where I put that might be a little difficult in the video.
00:43:44:10 And I suspect then students would just fast forward to where the question is and
00:43:48:00 watch the 15 seconds prior to it, to answer the question.
00:43:51:05 And I wanted them to get the full narrative of what was happening in the study
00:43:54:22 and watch the whole video start to finish. And so I'd have to do some,
00:43:58:06 some thinking about if I think embedding the questions would engage students
00:44:02:09 better or would encourage them to like, game-ify,
00:44:05:14 I'm not watching the whole video. I'm not sure how I feel about that.
00:44:09:09 Right. So one question comes in,
00:44:13:06 were students reaching out to you while watching the videos or were they not
00:44:17:10 really reaching out to you as an instructor or were they maybe going to other
00:44:21:02 online resources?
00:44:22:10 And an analogous question from a different attendee is,
00:44:26:08 were students working individually or do you think they were working together?
00:44:30:18 Um, they were expected to work primarily on their own. Um,
00:44:35:05 that's difficult to manage, right. Since this was open for a whole week. Um,
00:44:40:04 I'm sure that they were talking to each other. Um,
00:44:43:02 we were worried about answers being posted somewhere.
00:44:47:14 And so we, we discouraged them from getting together in a group and doing it.
00:44:51:18 Um, but I don't know for sure how much of that happened. Uh,
00:44:55:04 what was the other question? I forgot the first part. Were
00:44:58:14 they reaching out to me, um, not very often.
00:45:02:19 So sometimes those topics would come up since they were linked to the
00:45:07:19 discussion. Sometimes in the discussion,
00:45:09:16 students would want to start our discussion session talking about some of the
00:45:12:19 things in the video. Uh, but they primarily worked through it on their own.
00:45:16:24 And didn't really reach out to me with, with much going on there.
00:45:20:02 We do also have a course discussion board on campus wire.
00:45:23:19 And so occasionally questions would come up on there about, you know,
00:45:26:15 I don't know what this is, you know,
00:45:28:17 what this particular component is talking about.
00:45:30:14 I'm not sure what this question is asking me and we would respond to those on
00:45:33:05 there, but that wasn't very common.
00:45:36:09 Right. So a couple Of questions are still relating to the,
00:45:40:10 to the homework questions. Um,
00:45:43:14 so one is how did you interpret the understanding of students with the homework
00:45:47:04 videos? Um,
00:45:49:09 and I think maybe that relates to assessing which multiple choice questions were
00:45:53:19 wrong and how that informed how you responded in lecture or on exams.
00:45:57:23 And another question is,
00:45:59:12 was accuracy in answering the questions part of their grade.
00:46:03:04 But my understanding was that it was just right or wrong for those multiple
00:46:06:11 choice questions. So can you kind of address those?
00:46:09:05 Correct. So this was part of their grade. Um,
00:46:13:15 I'm trying to remember the exact percentage that it accounted for.
00:46:15:24 It was relatively small. Um, but it was, I think a little over,
00:46:20:02 I think it was about 12%. Um, so it was much lower than the exams, for instance.
00:46:24:06 Um, but it, it wasn't, um, they did need to do them, right.
00:46:26:23 If you wanted to do well in the class,
00:46:30:21 they were graded right or wrong. And the students only got one attempt,
00:46:35:25 um, in part, because,
00:46:40:20 so how we structured it was that, um,
00:46:43:22 students got to drop their lowest homework assignment grade. Um,
00:46:46:21 and so that we found is a little bit more helpful,
00:46:49:04 especially in as large a class as this, but if a student just, you know,
00:46:52:17 forgets about the assignment that week, for whatever reason, or,
00:46:54:20 or is ill or something, it's easier to just let them drop their lowest. Um,
00:46:58:07 and we do that instead of giving them, uh,
00:47:00:06 repeated trials or extensions on deadlines, et cetera.
00:47:03:24 And that's just sort of a course that's, um,
00:47:07:16 that all of the instructors have to be, tend to follow. Uh,
00:47:11:06 and then the other question was what was the other part of that -.
00:47:17:10 A question elated to interpreting understanding of students from,
00:47:20:13 from the videos? So you had the multiple choice questions,
00:47:23:20 but were there other ways in class or in the discussion sections that you
00:47:27:10 expanded on that and how, how did you approach that?
00:47:30:05 Yeah, it was mostly just those assignments. Um, the homework,
00:47:32:18 the video questions were just on the homework assignment and then would appear
00:47:37:01 again on an exam on a couple of examples. Um, but beyond that,
00:47:40:15 there was not a broader conversation,
00:47:43:07 except if students brought it up in the discussion session,
00:47:45:17 which only happened a couple of times. Um,
00:47:47:22 well the one exception to that is that homework assignment where they,
00:47:50:20 they really disliked the questions. Um,
00:47:52:25 cause I realized I had pushed them a little bit too far beyond what they were
00:47:55:06 comfortable doing. And so in response to those,
00:47:58:14 I posted an extra study guide that had a number of three allele problems.
00:48:02:06 Cause that tends to be something that students really struggle with in the Hardy
00:48:04:15 Weinberg is when you have more than two alleles, um,
00:48:07:06 it gets much more complicated to them.
00:48:09:07 And so that was a sign to me that we weren't ready for that step on the exam
00:48:13:01 yet. Um, and I, I pointed that out to them and said,
00:48:15:17 this is going to be on the exam, right.
00:48:17:05 But here's some extra practice if you're not comfortable with those types of
00:48:19:21 questions yet. And we can talk about them in office hours.
00:48:22:14 We can talk about them on the discussion board and we can spend a little bit
00:48:24:20 more time on that as well.
00:48:27:14 Great.
00:48:29:20 And one question from an attendee about kind of an extension of
00:48:34:17 these homework assignments, did you,
00:48:36:03 in the course require or recommend students to read the original,
00:48:39:24 the primary literature?
00:48:42:03 I don't require it. Um, I do encourage it.
00:48:44:19 So all of my lecture slides and the discussion slides, I,
00:48:48:04 I have the citation for, uh, whatever paper we're discussing in the moment,
00:48:52:01 much like my lecture today, my slides today,
00:48:54:17 my actual lecture slides in the class are really heavy with tables and figures
00:48:58:11 from actual research papers. And I tell the students throughout the quarter,
00:49:01:20 if you're interested in diving into this, um, I'm happy to send you papers,
00:49:05:17 you know, and it happens on a couple of topics. Interestingly, the,
00:49:08:02 the topic they most commonly email me to want to read the papers on is whether
00:49:12:03 or not dinosaurs were endo or exothermic, which if you're familiar with that,
00:49:16:08 literature is really interesting.
00:49:17:12 There's a variety of ways to try to figure that out from just fossil evidence.
00:49:21:11 Um, since this is the first class in the intro bio series,
00:49:26:20 um, I don't want to push students if they don't feel confident yet, um,
00:49:30:18 to dive into the original literature, if,
00:49:33:05 if they're comfortable with that and are interested by all means, do it,
00:49:35:22 but that's why I don't require it. Uh,
00:49:37:08 the students come into this class with a wide variety of backgrounds in biology
00:49:41:13 that range from, you know, 15 AP classes. And they're totally,
00:49:45:17 they've read original research before, you know,
00:49:47:10 they'd totally be ready to dive into that to students who maybe haven't taken a
00:49:51:03 biology class in five years and maybe don't remember much of what they did take
00:49:55:05 before,
00:49:55:14 even if they did well in it and are a little bit less confident on that front.
00:49:59:13 So I encourage it, but I don't require it. Um,
00:50:02:05 and I do sometimes share text or examples from literature to introduce
00:50:07:05 them to it. In what I hope is a less intimidating way, um,
00:50:12:01 which I think is successful.
00:50:15:00 Great. So One question,
00:50:17:06 did you share the weekly feedback with students on a weekly basis?
00:50:21:08 I know that you did it at least midway, but how, what was that frequency?
00:50:25:16 I only did it twice,
00:50:26:22 so I was very serious about them being open to criticizing
00:50:31:20 the videos and the homework if they didn't find value in it.
00:50:34:24 So after the first video,
00:50:36:12 I pulled a handful of some positive and some negative comments and showed them
00:50:40:07 to the students in class and was really explicit about how the negative comments
00:50:44:08 were useful for me in understanding how they were interacting with it. And then,
00:50:48:16 um, I didn't weeks two and three. And I mentioned that,
00:50:51:05 that you saw how the response rate declined a little bit.
00:50:53:19 And so after the third homework assignment, um,
00:50:56:23 which was also after the first midterm, I shared some comments, again,
00:51:01:15 reminded the students that, uh,
00:51:03:10 this question was a value that I was reading them all the time. Um,
00:51:06:14 and that this was important to me.
00:51:07:16 And then the rate went back up and then tapered off a little bit throughout the
00:51:10:15 end of the quarter.
00:51:11:10 So I only explicitly shared or talked about the feedback twice in the class.
00:51:17:12 And this last question we have in the Q and A,
00:51:19:04 and I encourage you all to keep feeding them.
00:51:20:21 We have a few minutes left in this hour,
00:51:23:00 but the last question I think is this really nice kind of philosophical
00:51:26:01 question.
00:51:27:01 Do you think that students in an introductory evolution and ecology course are
00:51:30:21 really looking for insights into researching exotic species? Um, and they,
00:51:35:16 and they are predisposed to that?
00:51:37:08 And maybe how can we set expectations for what the reality is of,
00:51:42:14 of ecology research?
00:51:44:16 Yeah, that's a good question. Um, I didn't choose the videos based on, um,
00:51:49:01 how charismatic I thought the organism was. Right.
00:51:51:12 I was totally interested in whether these were topics that matched with a
00:51:55:08 discussion activity I had planned or whether it was a good illustration of
00:51:59:09 research on a particular topic that we were discussing.
00:52:01:14 So I wasn't caring about whether they were exotic or charismatic organisms at
00:52:05:03 all. Um, we do,
00:52:09:20 so I do spend a little bit of time in each of the discussions as each of the
00:52:14:04 videos did talking about the organismal system, uh,
00:52:17:07 because in part that's important for understanding like what research questions
00:52:20:14 you can address with this particular, uh, you know,
00:52:22:21 habitat or taxonomic group or whatever,
00:52:25:07 and the sort of biology of the organism is important for like how we study it.
00:52:29:18 And so I do think students got interested in that. Um, I do,
00:52:34:20 I guess philosophically, I have,
00:52:37:16 I don't want to cater to the types of organisms that students already love.
00:52:41:15 I like to highlight things that they don't know as much about. Um,
00:52:44:12 my background is in plants and so I tend to use a lot of, uh,
00:52:47:11 plant examples because students tend to think that plants aren't as interesting
00:52:51:09 and don't do as much, which is just patently false.
00:52:53:09 And so I like to really break down that particular misconception. Um,
00:52:58:10 I would have expected,
00:52:59:13 I did expect when we first started looking at the organism responses that
00:53:02:17 elephants would be by and large, the most loved thing, right?
00:53:05:19 They're an incredibly charismatic organism.
00:53:08:02 The sweeping views of the herds of elephants in the video were like,
00:53:12:08 were really great.
00:53:13:08 And students were aware of the conservation concerns with elephants.
00:53:16:08 And so I was sure that that was going to be by far the most loved organism. Um,
00:53:20:14 and I was wrong, right? It was the, uh,
00:53:22:07 it was the lizards and even the other organisms got a lot of comments
00:53:26:00 specifically saying these mice are so cute.
00:53:28:17 I loved watching this video about these adorable mice and what's happening in
00:53:31:16 their coat color. And like the fact that it was,
00:53:33:25 there were photos of the organisms there that they could identify with,
00:53:36:18 made them more interested in the research. And so it wasn't just,
00:53:39:15 I liked this cute thing. It was, I like this cute thing.
00:53:42:18 And now I want to know why this cute mouse is brown and this cute mouse is
00:53:45:19 white, right? Like that actually did get them invested in the research question,
00:53:49:17 which surprised me a little bit.
00:53:52:15 That's great. Thank you for sharing.
00:53:54:09 So there are no more questions coming in through the Q and A box,
00:53:57:09 and we're almost at the end of the hour here.
00:53:59:04 I'm sure people are ready to move on to the next zoom meeting they have of the
00:54:02:02 day. Um, thank you all for attending.
00:54:05:02 This was a very fun webinar to host. We will, uh,
00:54:09:09 share this recording on YouTube. So that will come in our newsletter.
00:54:11:24 If you subscribe to the iBiology newsletter, one thing I want to say,
00:54:16:03 just a quick plug for our organization. If you're not familiar with us,
00:54:19:13 I encourage you to go to ibiology.org and check out our resources.
00:54:23:18 We have everything curated by topic. We even have a search function.
00:54:26:23 We also have a popular YouTube channel.
00:54:29:11 If you search the ibiology channel on YouTube, you'll find us right away.
00:54:32:20 And all of our videos are there for easy sharing and searching. So yeah,
00:54:36:18 I'll put my email in the chat here.
00:54:41:16 We are happy to take any questions.
00:54:44:02 And the question came up about Laci sharing her homework questions that she put
00:54:48:15 together. We'll do that for those five iBiology videos. Ideally,
00:54:52:02 we'll aim to do that by the end of the summer, but for the other ones,
00:54:54:22 such as the HHMI and the Bozeman science,
00:54:57:12 you'll need to reach out to her for those.
00:54:59:17 So thank you so much and I hope you enjoy your summer.
00:55:03:22 Thank you everyone.
00:55:05:02 Thanks Laci.
00:55:06:06 Thanks Brittany.
00:55:07:06 Bye bye.

This Talk

Speaker: Laci Gerhart-Barley

Audience:

Educators of H. School / Intro Undergrad
Educators of Adv. Undergrad / Grad
Educators

Recorded: June 2021

Talk Overview

Educational science videos are commonly used in undergraduate biology instruction, but what do we know about the student perspective on them? In this paper discussion and live Q&A, Dr. Laci Gerhart-Barley and Dr. Brittany Anderton present findings from their open-access paper “Engaging students through online video homework assignments: A case study in a large-enrollment ecology and evolution course.” Undergraduate biology educators will find a wealth of useful information on how to effectively implement science videos in homework assignments, develop assessment questions, and use the content in educational science videos to promote student engagement and learning of the process of science.

Speaker Bio

Laci Gerhart-Barley

Laci Gerhart-Barley is an Assistant Professor of Teaching (LPSOE) in the Department of Evolution and Ecology at UC Davis. She received her PhD in Ecology and Evolutionary Biology from the University of Kansas, and has taught introductory biology and botany at the University of Hawai’i. At UC Davis, Dr. Gerhart-Barley studies the role of educational… Continue Reading