2007 April Site of the Month

http://mmiweb.mmi.mcgill.ca/dev/LorieKLODA/HealthLib-Instruction.htm

Authors: Content created by Lorie Kloda and edited by McGill librarians, Life Sciences Library.
Designer: Joanne Hui, McGill Molson Medical Informatics
Institution: McGill University, Montreal, Canada

Interview with: Lorie Kloda
Interviewer: Peggy Noynaert

Description: The goal of the online tutorial, “Searching the Pharmacology Literature,” is to provide the student with the information and resources necessary to support self-directed learning. The content is designed to assist the student in locating, retrieving, evaluating, and using pharmacology information.

Q. What was the motivation to create the tutorial?

A. McGill University’s undergraduate medical school (MD) program was undergoing changes to the curriculum. The changes are, broadly, an emphasis on small group teaching and independent study, and the incorporation of a new professionalism component which will run throughout the four-year program. These changes will hopefully emphasize the utility of the information literacy instruction and the library’s resources for small group and individual work. Additionally, in the past students had complained about the lack of interactivity of the lecture format, compared to other library instruction provided throughout the program. We also wanted a free-standing tutorial which could be accessed anytime. In 2005, the library partnered with an instructional designer to replace a lecture with an online tutorial.

Q. Who do you expect to use this tutorial?

A. Undergraduate students in the MD program are expected to use the tutorial during their course on pharmacology. That is, the tutorial is an integral component of the curriculum, but we don’t take “attendance.” The e-curriculum technology does not allow faculty to verify student use of online resources. This pharmacology class alone is composed of approximately 200 students each year. Potentially, students from the nursing department, as well as medical residents and faculty could benefit from the tutorial.

Q. Did you receive support from staff outside the library?

A. Yes. The McGill Molson Medical Informatics (MMI) Project at McGill University collaborated with the McGill Life Sciences Library. An instructional designer (and medical illustrator) from the MMI designed the tutorial and an instructional technology librarian from the Library developed the content, oversaw the design, and managed the updating and assessment of the tutorial.

Q. Could you please describe in more detail how you worked with the instructional designer? What insights did you gain from the experience of working with the designer?

A. Originally, the project was conceived by the librarian. The librarian corresponded via email with the project manager of the MMI and was paired with the instructional designer. The librarian and designer then met over several occasions to go over the content, which had previously been prepared by the librarian. At these meetings, the librarian showed examples of existing tutorials from other institutions as “best practices” to give the designer a sense of what we were trying to do. Following this, the team went through the content of the tutorial, page by page, and discussed how to best incorporate it into the tutorial. In some cases, this meant merely rendering the text on-screen. In other cases, the librarian wanted an interactive diagram to illustrate the content. The interactive illustrations would be created by the designer, while all the static content (text, image captures) and screencasts would be created by the librarian. Once this was agreed upon, the actual look and feel of the tutorial was discussed, with a lot of attention to the structure and navigational features. The designer then created a “mock-up” or shell of the tutorial, which was viewed and commented on by the librarian. Once the shell was approved, the designer added all the content to the tutorial. The team then met again to go over problems, inconsistencies, and to revise some things that did not function well. Following this, the librarian requested minor changes via email until the final product was ready. Overall, the experience of working with a designer was very rewarding. The most difficult aspect was at the beginning of the process, where it is really important that the designer understand what the tutorial is supposed to be. In a sense, we had to get on the same page. The best way to do this, in my opinion, is to show examples of other tutorials, for all types of library instruction, so that the designer can have a good idea of what it is you want, and what you don’t want.

Q. Approximately how many hours did it take to create this tutorial?

A. Less than 100 hours. Much of the content was already created, since the instruction was previously delivered in a lecture format. (This includes the time of both the librarian and the instructional designer.)

Q. What technologies were used to create this tutorial?

A. The instructional designer used Flash to design the site.

Q. Why was Flash chosen?

A. The decision to use Flash was made by the designer after meeting with the librarian and understanding what was needed. The designer was proficient in designing with the technology.

Q. Did you conduct usability testing for your project?

A. Yes. We first piloted the tutorial on librarians at the university and some acquaintances in the field of medicine to make sure there were no major problems with the tutorial. The testing was conducted by the librarian, and the results were shared with the instructional designer. We then collaborated to solve any problems that were identified. Once the tutorial was made available to students, a link to an online survey was provided. The survey included questions both on the usability and the content of the tutorial.

Q. What challenges, technological and other, did you have to deal with in creating this tutorial?

A. The biggest challenge was deciding where the tutorial would reside on the website. We also wanted to ensure that the tutorial would function properly within the electronic curriculum currently used with undergraduate medical students. This virtual learning environment is different from tools such as WebCT and Blackboard, and we wanted students to be able to use the tutorial and stay within the e-curriculum. The e-curriculum was designed by the Molson Medical Informatics (MMI) Project. It is structured in such a way as to list all the program content, whether it be lecture, small group, or online tutorial. The tutorial appeared chronologically within the unit of study. A link was provided with an explanation of the purpose and scope of the tutorial. The tutorial is also available as a link from the library's website.

Q. How do you assess student performance? What have you seen in the results?

A. Student performance is assessed indirectly through a written research paper which requires searching and referencing the literature. There is also a single multiple choice question included on the students’ exam which assesses one of the learning outcomes. The success rate of this question was lower than in previous years when the lecture was offered. Only 65% (131/201 students) of students answered the question correctly, as compared to 73% (137/184 students) in the previous year when the content was delivered via lecture. This suggests that students may not be using the tutorial, or that they may not be able to decide what is important from all the content.

Q. What impact has the tutorial had on the quality and quantity of resources cited in the student papers? Have the professors noticed any improvement in the quality of student papers?

A. This has yet to be determined.

Q. Please give more information about how you obtained user feedback through the tutorial?

A. There is a link to an online survey. At the end of the tutorial, participants are encouraged to provide feedback using this mechanism. The survey asked questions on both the tutorial’s usability and on the content and had space for additional comments. The feedback on the usability was used to improve the tutorial. The feedback on the content, while not major, is being kept for future improvements. Generally, there was positive feedback on the usefulness of the content, though some students said that the tutorial content duplicated previous workshops. We received about 30 surveys the first year and about 15 the second year.

Q. Do you receive feedback from faculty or other interested parties?

A. We haven’t received any other feedback at this time.

Q. How often is the tutorial reviewed and updated?

A. It is reviewed yearly for major changes such as revising screencasts. It is updated as required for smaller changes, such as broken URLs.

Q. In creating another tutorial, what would you do differently?

A. Ideally, more interactivity would be added – activities, quizzes, etc.

Q. What recommendations or "words of wisdom" would you share with future tutorial creators?

A. It is important to start with a very detailed outline of the content and structure before beginning the design. It is also essential to have all of the content ready, as this can take a lot of time to create, and even to revise. The instructional designer needs to have an idea of the type and quantity of the content in order to design an appropriate “shell.” Once the design is approved, everything moves very quickly!