November 2009 Site of the Month
Peer Review in Five Minutes
Authors: Hyun-Duck Chung and Kim L. Duckett
Interviewees: Hyun-Duck Chung and (where noted) Kim L. Duckett
Institution: North Carolina State University (NCSU) Libraries
Interviewer: Sarah Cantrell
Peer Review in Five Minutes is a brief animation video that is designed to primarily help undergraduate college students to quickly understand the concept of peer review and its importance in higher learning. The goal of this video is to provide greater contextual background on this topic so that students can think more critically about how they use scholarly sources.
Q: I really enjoyed reviewing your tutorial and am delighted that it has been chosen as a PRIMO Site of the Month. My first question for you is: what was the impetus for creating a tutorial on the peer review process and what purpose does it serve?
A: We often have students coming to the reference desk looking for peer reviewed journals without knowing what they are, how they are put together or what purpose they serve. We thought it would be great to share this kind of contextual or background information with students, so that they can have a better sense of what they are being asked to use and therefore hopefully make better use of the resources we provide. We also recognized that short animated videos hosted on popular video sharing channels like YouTube and social networks like Facebook, offered a lot of potential for disseminating information like this. And although we didn't know exactly how much work such a project would entail, we knew it wouldn't be a small feat to produce a video, so we were always hoping to create something that could be disseminated broadly and reused at other institutions since this topic is something that any academic library might deal with.
Q: Who is the target audience of this tutorial and how have they been using it?
A: The video is designed to 'initiate' undergraduate students into an important aspect of academic culture and how peer reviewed articles are related to the work on college campuses. It has been used in a few different ways: librarians have sent a link to the tutorial while providing virtual reference services, it has been linked from library course pages, and faculty instructors have been assigning them in their classes. One of the most interesting uses was when one instructor assigned students to view the video and review it by posting a comment on a class blog.
Q: Who was part of the design and creation process and how did this team come together? Did you collaborate with anyone outside of the library? How would you characterize the working relationship of the group?
A: (KLD) I had the basic idea and a draft script for a “peer review” video before Hyun-Duck started at NCSU Libraries and it seemed like a perfect project for someone with a strong interest in visual design and learning. Hyun-Duck was a perfect fit! (HDC) When I arrived as a NCSU Libraries Fellow I took the lead in creating the video under Kim's supervision– including scripting, visual storyboarding, recording the narration and general project management. But we also had a lot of help from students and colleagues. We were very fortunate in having Susan Baker, a graphic design student intern, working with us. I worked closely with her to turn a basic storyboard of ideas into polished-looking moving images. As for the background music, I approached Chris Hill, one of our Digital Media Lab students to see if he was interested in creating a track for us. I sent him some music clips I'd found online to give him a sense of the tone and mood I was going for. He went away with a copy of the animation and a few weeks later, came back with the sound track timed to the video. We relied on the expertise of Jason Walsh, our departmental web technologist, to get the video to play from the library website at a decent bandwidth and quality. We also consulted a group of multimedia designers on campus to learn how to provide captioning and use progressive downloading to make the video play properly. Andreas Orphanides, another NCSU Libraries Fellow, worked with Jason and Susan to create the web-presence for the video. As you can see, it was a highly collaborative effort.
Q: Did you use or adapt any design model to assist you in designing and planning the tutorial?
A: We used the ADDIE model which comes from the area of instructional design. The acronym stands for the different phases of Analysis, Design, Development, Implementation and Evaluation. For this project the analysis phase included going out to informally interview faculty to explore how the topic of peer review was relevant in their instruction and pitched our idea of a video module. I also conducted a brief environmental scan of existing tutorials. There is some good content out there in terms of articles and tutorials written out in text, rarely incorporating any graphics, but most importantly, none of them seemed to tie the concept back to the role of college education or local library services – which are important aspects of this video. I also feel strongly that the online environment favors the visual image and audio. I think text only works if it is broken up enough using headings and bullets to be scan-able. For these reasons, we thought a video module on this topic would be a worthwhile project to pursue.
Q: The tutorial has a wonderfully colloquial script that, I think, must appeal to students. Did you involve students in the script-writing process or get their feedback via focus groups or other methods?
A: Getting student feedback is something I guess we could have done more of in the design and implementation phases but I don't know if a script without the visuals would have made sense. The production phase is also the most labor-intensive stage of the process so you typically want to have a pretty solid script and storyboard before investing the time to create the graphics and animations. My thinking was to view faculty and library instructors as the primary stakeholders since we wanted to provide material they could use in a flexible way in their interactions with students. We did get a lot of feedback from other librarians and instructors on early versions of the script and made revisions. This was our first attempt at this kind of video module so we were mostly hoping to get something out there for students to react to. Reading the comments on that one course blog was interesting – a number of students were surprised about how long the peer review process could take and how well established it is, since it was a concept that was fairly new to them.
Q: I read a blog post that you both authored entitled “Narrating the back story through e-learning resources in libraries” and was very intrigued and excited by the back story or background approach. Could you please tell our readers about this and how you incorporated it into the tutorial design?
A: (KLD) We believe that librarians are uniquely positioned to teach people about where information comes from -- how it is created, debated, shared, stored, and accessed. Often the contexts surrounding information gets left out of instruction by both librarians and faculty, however. When we focus on the social life of information and teach students about information rather than how to search or evaluate, we call it “back stories.” The Peer Review in 5 Minutes tutorial is pure “back story.” It doesn’t teach how to search for peer review articles or how to distinguish a peer reviewed research article from a magazine. It exists to enrich the viewer’s understanding of what the term “peer review” even means and why the process is important. We wanted to create this video so librarians and instructors can give students the big picture about the peer review process as an aspect of teaching them to find and use peer reviewed research literature.
Q: What technology did you use to create the tutorial? Why did you choose it and did you consider other technology?
A: We used a combination of media tools to create the tutorial. Our graphic design student intern Susan was familiar with the tools in Adobe Creative Suite so we went with that – I don't know that we would have used it otherwise. Susan used the suite of tools in that package to create graphics and animations and to integrate the various media files to produce the final video format. I used Audacity to record the narration and create audio files. Chris used GarageBand for the sound track. For captioning we used a service called Automatic Sync Technologies.
Q: The tutorial is only five minutes long but packs in so much great information. I imagine, however, that this must have taken you and your colleagues a long time to complete! How long did it take you to complete the tutorial, from idea to final product? Were there any parts of the tutorial that took longer than expected or that were easier than you thought they would be?
A: (HDC) A lot of thought and energy went into creating the video. The hardest part was carving out time beyond the day-to-day work and pushing the project forward. It took about a year from writing the script to being able to play the video online – but even now we are continuing to work on the delivery, promotion and assessment of it. The audio recording was a bit of a challenge for me and took much longer than I expected. I had to practice and re-record several times and I still feel like there are segments that could be improved. I also had to do some minor editing of the files to reduce background noise and optimize the sound quality, which I hadn't anticipated. That was another thing: I had to create several smaller files of narration to make it easier to synchronize it with the visual component. (KLD) This was our first experiment with a multimedia project beyond screencasting in Camtasia, so Hyun-Duck and our other team members were really pioneers. We had to sort out a lot of details along the way – how to get the best audio, for example. This project was a huge learning experience for all of us, but we’ve been able to apply what we’ve learned to other projects – some equally complex, others more straightforward.
Q: I really like the simple animations and background music that were used in the tutorial. Could you tell me more about them and how they were created?
A: (HDC) I think the simplicity of style came from a variety of factors: trying to lessen the cognitive load of the viewer, the personal style and skill that Susan brought to the creation, and wanting to keep the production process as simple as possible. Selecting the music was interesting. I knew it would be more engaging to use some background sound but I had to select something that wasn't distracting. I like video surfing online so I looked at some videos that seemed to strike that balance well and thought about what made it work. Then I listened to samples on some of the sites that sell royalty-free music loops and picked out a few samples that I thought captured the pace and mood of the video – somewhat happy, upbeat but still mellow. I then shared these with Chris – and unfortunately I have no idea what magic he worked to create the track that he did. Chris was a graduating student who was seeking a career in this area so we were fortunate to have him contribute his talent and skills to the project before he left. (KLD) I also know from watching Hyun-Duck interact with Susan that Hyun-Duck played a big role in shaping the visuals for the animation. Many of the graphic concepts originated with Hyun-Duck. Susan brought them to life.
Q: What experience or training did you bring to creating this tutorial? What did you need to learn “on the job”?
A: I've always been interested in working with digital media but had no experience – and I still don't really since our students did most of the production. I think my personal talent is in the graphical communication of information and I was able to work on developing that into a skill and creating something of value through this project. Project management skills would be another on-the-job-training example: figuring out what to do and how to do it, setting priorities, communicating clearly with everyone, pushing the project forward – all of that take practice to do well, so I’m glad I got to practice some of those skills.
Q: Have you done any formal assessment of the tutorial? Or, what plans do you have for evaluating and assessing the impact of the tutorial?
A: (HDC) The assessment piece is something we're looking to work on going forward. So far we've tried to keep a record of the informal feedback we've been getting from librarians, faculty and students. Generally they have been very positive. Putting it on YouTube and the A.N.T.S. Site has helped us gauge some statistics for use but we're trying to come up with better assessment strategies not only for this video but some of the other modules we've created as well. (KLD) During the upcoming spring semester we hope to do some more thorough assessment using the module with certain classes.
Q: What feedback have you received from students who have taken the tutorial or from faculty who use it in their classes? A: (KLD) One freshman writing instructor here at NC State recently emailed to say, “I really like the peer review animation. It is very well done and my students seemed to get a lot out of it in connection with other activities on peer review.” We’ve also received many kind emails from librarians across the U.S. as well as Canada and the Caribbean thanking us for the video and asking if they can use it with their students. The video is available under a free, noncommercial use, share-alike Creative Commons license, so you don’t actually need to ask for specific permission to use it in these ways. Of course, we always do love getting any feedback that users have to offer.
Q: How have you publicized the tutorial?
A: (HDC) We wanted to market Peer Review in 5 Minutes along with other “back story” modules we’ve created as a suite of tutorials so we've been waiting until we had a small collection of them. (KLD) You can see more of our tutorials at the NCSU Libraries Tutorials and Online Learning Guides website: http://www.lib.ncsu.edu/tutorials. We’re working on the best strategies for marketing the tutorial at our university and beyond. So far we’ve shared it with librarians and faculty through listservs and personal emails. We’ve contributed it to learning object repositories such as MERLOT, A.N.T.S. and now PRIMO. Some of our projects such as Peer Review in 5 Minutes have also been put into the YouTube channels for our university and library.
Q: How is the tutorial used at NC State? Is it a "stand alone" lesson for students to complete on their own, is it integrated into face-to-face library research sessions, or is it a requirement of your institution’s general curriculum?
A: (KLD) It’s used in various ways. We created the tutorial so librarians can use it in tandem with their instruction sessions or instructors can use it on their own, such as the instructor who had her students watch it and post reactions on a course blog. We also plan to incorporate it into our more comprehensive information literacy tutorial, LOBO, which is currently being redesigned. We hope to reach our largest audience at NC State through LOBO.
Q: What was the biggest challenge you encountered during the creation of the tutorial? And did you have any pleasant surprises?
A: (HDC) This was our first experience with creating a multimedia project of this complexity so we had a lot to learn about how to manage such projects. (KLD) For instance, it’s much easier to create an instructional project that teaches a procedure – such as a screen-cast that demonstrates how to use a particular database -- than one that tells a story or teaches a concept. With a narrative-style project you need to figure out what’s in and out of scope. You need to be careful about sticking to your learning objectives and keeping the amount of content manageable for the learner. I’d say our biggest challenge was finding the time between all our other activities to keep the momentum of the project going. In public services it’s rare to have the big blocks of time you need for development work, but you just have to stick with it. It was a pleasant surprise to find out that Hyun-Duck had been consulting with Chris, the student who created the music, and that he came up with a cool soundtrack. I had never anticipated that the project would include music.
Q: Is there any advice or recommendations that you would make to someone interested in creating a similar tutorial?
A: (HDC) Get a pretty good script and storyboard established before even considering what technology you will use to make the video or who will do the work of creating it. You only need paper, pen, your mind and maybe some background research to create the script and storyboard, but this can take a lot of time to nail down. There will always be tweaking later down the line but the more solid your script and storyboard, the more smoothly the rest of the production will go, as a general rule of thumb. On the other hand you do want to push the project forward so it actually gets made. (KLD) I’d also say that you should look to your students as a source of creativity and to supplement the skills of your library’s team. Many students are eager to take on projects that will enhance their portfolios and build their resumes. They are a pleasure to work with and to learn from!
Q: Is there anything you'd like to add about your tutorial that you think our readers would like to know?
A: (KLD) We've released the video under a Creative Commons license so we encourage people to take a look, link to it, embed it into their local web presence and remix it if they are so inclined. You can find ways of doing so at http://www.lib.ncsu.edu/tutorials/pr/adopt
November 2009 PRIMO Site of the Month