primo - ala acrl is

February 2008 Site of the Month

Appalachian State University Library Research Tutorial

Authors: Kelly McBride and members of the Library Research Tutorial Taskforce
Institution: Appalachian State University Library and Information Commons

Interviewer: Krystyna Mrozek

Description: The Appalachian State University Library Research Tutorial is an interactive, web-based tutorial designed for use within the Freshman Seminar Program. In 2006 it was completely re-designed to incorporate information literacy more prominently within the content.

  • Module 1 introduces students to a variety of locations and services within the library.
  • Module 2 compares types of information sources: popular vs. scholarly, primary vs. secondary and internet vs. database sources.
  • Module 3 teaches students to search effectively by developing thesis statements, using background sources, identifying key concepts, combining search terms, identifying synonyms and using truncation.
  • Module 4 helps students find materials using the catalog, reserves, and library databases.
  • Module 5 presents fives steps in evaluating information sources.
  • In Module 6 students learn to avoid plagiarism.

Q. I see that an earlier iteration of your tutorial is in our PRIMO database dated Winter 2004. What prompted you to re-design the original tutorial and what is its current purpose?
A. In 2003 we launched the original tutorial, the ASU Library Tutorial. It worked just fine with its original elements - the coverage was the basics. The tutorial was designed to be an introduction and was directed towards first semester incoming freshmen. It had been in place for three years and although we were happy with the outcome, it was just dated and the time had come for a formal review of the product. We had new technology available to us in 2006 that wasn’t available in 2003. Also we wanted to imbed more information literacy and wanted to make it a little bit more interesting for the students. The feeling was that the first tutorial was fine and covered the basics but it wasn’t very flashy or interesting. The purpose of the re-design was to make it more interactive and to have modules that could be completed individually.

Q. Who is the target audience of this tutorial and how have they been using it?
A. The target audience is the first semester incoming freshmen primarily enrolled in the University’s freshman seminar program. The tutorial is part of a three part library component. The students complete the Library Research Tutorial, afterwards they come in for a formal library instruction session, which is based on an assignment, and then use the skills introduced in the library session to complete the assignment. So their first introduction to the library is the tutorial, then they come over for library instruction that is tied to an assignment. The last part of it is that they actually complete the assignment, hopefully using what they learned. The assignment primarily covers the introductory elements required for library research, i.e. searching the library catalog, database searching, and evaluation of their information and internet searching. The end result is a college level paper which would include a bibliography.

Q. How have you incorporated information literacy more prominently into the tutorial?
A. In the first tutorial we covered the basics. In the revision we wanted to help students identify types of information sources, evaluate information critically, and avoid instances of plagiarism. We had areas in each one of these segments where students had to test their knowledge. That was a big difference. For example, in Module Six – Avoiding Plagiarism, we give them an example of something that is plagiarized and something that is cited correctly and they have to identify the correct answer. If they identify it correctly that’s a reaffirmation. If they identify it incorrectly that’s an additional learning opportunity for them.

Q. Who was part of this redesign process? Did you collaborate with anyone outside of the library? How did this group of people work together?
A. The Library Research Tutorial Taskforce consisted of the web librarian, three reference librarians, one catalog librarian, the distance learning librarian, one instructional technology design staff person, and two technology support staff. It takes a village. We did a lot of research on the PRIMO site looking at what others had done because we did not want to reinvent the wheel. We used PRIMO to identify best practices. It was often times challenging, but really collaborative and in the end rewarding. We worked together as a big group and also as smaller subgroups. We had a pretty tight timeline. The Taskforce formed in June 2006 and our deadline was to complete the tutorial by spring of 2007 and have it ready for use in the fall of 2007. We started with usability testing to look at the old tutorial, with various populations of Appalachian State University students and incoming students. We compared the original tutorial with other tutorials that we found (for example ones that used Flash or audio) and asked students for their impressions. This was a long process to begin to get a sense of what students reacted to - what they liked.

Q. What made you decide on this format?
A. We found that students liked tutorials that are self paced - that contain graphics, that are interactive and non linear. We didn’t meet all of those objectives. We decided to go with six modules - once you start a module you go through it linearly but you can choose any module to start on. That was our compromise.

Q. What technology did you use to create the tutorial? Why did you choose it and did you consider other technology? Why did you decide not to use audio?
A. We used Adobe Captivate and Flash. That was the recommendation of the computer consultants. We didn’t have a lot of outside funding or release time so it was a matter of using technology that was scalable for us to learn so that when we talked we were using the same language. I think we decided not to use audio because of a variety of reasons including time and ADA compliance requirements. We would be willing to explore the use of audio in the future.

Q. Can you talk more about usability testing and what was the result?
A. We anecdotally found out what students preferred but that did not necessarily drive all the decisions that we made. Initially we had the original tutorial and we had some ideas already about content, so it was more changing the flavor of it. Once we developed the tutorial we kept testing and at that point we were looking at things like the effectiveness of the information we were presenting. Was it too long? Was it confusing? Were they getting the quizzes right? And at least at three distinct points we were doing studies that were designed to gather information from different populations of students. For example, students at ASU in the Gear Up program who were not yet affiliated with the university and currently enrolled students who might have some familiarity with library resources. Once the tutorial was just about finished we again conducted usability testing under controlled conditions, using TechSmith Morae software. This screen capture software recorded all the movements of the students - it looked at the mouse movements and clicks, voice, and provided video of the user via a webcam. Students were also given a written evaluation and a questionnaire and they recorded their impressions of each section. We found out what they liked and what they didn’t like. They particularly liked when they had to drag and click. They didn’t like the plagiarism section which they thought was too long and that the three examples that we used there were too long.

Q. How have you publicized the tutorial?
A. We didn’t do much publicizing of the re-design. The tutorial was already established and there was a new crop of students so they didn’t know that anything had changed. We just used our normal publication to faculty through faculty participating in freshman classes in addition to using the library’s web page. Anyone who wants to learn the basics about library research can use it. It’s available on the library home page and it’s not restricted to freshman students.

Q. What feedback have you received from students who have taken the tutorial or from faculty who use it in their classes?
A. We have a couple of mechanisms for gathering feedback. I teach a lot of freshman courses. I have them take the tutorial before they come to their first library instruction session. I ask them “what do you think?” I get very nonscientific but amazingly useful feedback that way. They are happy to tell me! We do individual testing in our usability lab. We also do an analysis of the statistical data from the quiz - we can generate the number of correct and incorrect answers for each question. Students have to score a minimum of seventy in order to pass.

Q. How much time does it normally take a student to work on your tutorial? And what do you do with the quiz at the end?
A. I think a student takes thirty minutes to an hour to complete the tutorial. One popular use in English 1000 or the Freshman Seminar is that the faculty member assigns the tutorial and it is an activity that they need to complete. After completing the quiz the student receives a certificate of completion.

Q. Have you done any formal assessment of the revised tutorial? What plans do you have for evaluating and assessing the impact of the tutorial?
A. We haven’t done it yet. And it’s time. The tutorial went live in September 2007 and you know how busy fall semester can get. We were so involved with it. I think that there is still some quirkiness in there that needs to be worked out. We do have just a page for comments and we’re hoping that people have comments and suggestions. We need to assess because the way we have been using the tutorial is not necessarily the way it’s going to continue to be used. The University has just completed a revision of its general education goals and the Freshman Seminar program will be significantly changed as a result. My hope is that the tutorial becomes more than a stand alone product but is more integrated into the curriculum. It is a wonderful first step in introducing students to the library resources and services. I want it to be a first step in a multi step information literacy component. And I’d like it to be more robust. We changed the entire tutorial which was a massive amount of work. We could have benefited from smaller more discreet projects. Maybe that would be a different approach to assessment and re-design.

Q. What did you learn through redesigning the tutorial?
A. What we learned is that assessment is an important step. You need to set a reasonable timeline for any project of this sort. You need to schedule regular meetings. You don’t want to reinvent the wheel. Our first tutorial was the result of collaboration with the University of North Carolina at Chapel Hill. I’d say collaboration is key - there are lots of libraries doing great things. The three lessons learned are: 1) You don’t have to start from scratch, you can customize it for your institution. 2) Keep your eyes on the prize. Remember that the end result will hopefully add value to your instruction program. Don’t get bogged down in the details. Typically when you have an instruction program the tutorial is one of many instructional opportunities so the tutorial shouldn’t have to do everything. 3) Perfection is not the goal.

Q. What was the biggest challenge you encountered during this revision of the tutorial. And did you have any pleasant surprises?
A. It was just the amount of work. I don’t know that we were completely prepared for the undertaking. It was a lot of time. Really finding the time to actualize our vision was the biggest challenge.

We did have a pleasant surprise. It allowed us to really come together, and it was a tremendous learning opportunity, individually and collectively. As a group meeting together we often had to meet in smaller subgroups. And that we learned all of this. I didn’t know any of this before I started. All these questions that you’re asking me I would not have been able to answer before we started the re-design. It was a positive experience.

Q. Do you have any new revisions planned?
A. Not right now. I’d probably call them enhancements. We’d probably approach it in discrete chunks. There are sections that could benefit from future revision.

Q. Is there any advice or recommendations that you would make to someone interested in creating a similar tutorial?
A. I’d just go back to the three lessons learned. It would be different for different institutions. What is the objective for what you’re creating? What is it you really want to do? I think tutorials are hot and trendy now, not that they don’t have staying power. There is a long term commitment to having a tutorial. It never ends. I don’t know what students will be like in five more years.

Q. Is there anything you'd like to add about your tutorial that you think our readers would like to know?
A. I think that our tutorial is successful for our population and it’s not at the low end of tutorials or at the high end of tutorials but it’s a scalable workable example of a useful instructional tool.

February 2008 PRIMO Site of the Month