primo - ala acrl is

March 2006 Site of the Month

Doing Research: An Introduction to the Concepts of Online Searching

Authors:Annie Armstrong and Helen Georgas
Institution: University of Illinois at Chicago

Interviewer: Barbara Petersohn

An animated, interactive tutorial that introduces undergraduate students to some basic concepts of online searching. These concepts include: 1) Using the search term “AND” 2) Identifying keywords 3) Thinking of synonyms and related terms 4) Examining a citation and 5) Putting it all together. Concepts are illustrated interactively through a series of highly visual modules/games. Whenever possible, concepts and instructions are explained with a minimum amount of text. The tutorial is not resource or university specific in any way. Rather, any university (or high-school) student could complete it to gain an understanding of basic information literacy skills.

Q. What was the motivation for creating this tutorial?
A. We found that teaching Boolean logic and the importance of selecting keywords for a topic didn’t always work with our students. Regardless of the various approaches we took to explaining these concepts, students frequently proceeded to enter their topics as phrases in catalogs and indexes. We hoped that by taking advantage of the interactivity of online tutorials, and by engaging students in an activity designed to illustrate these concepts, students might better retain the idea of selecting keywords, thinking of synonyms, and using “AND” to connect keywords. It was also important to us to create a tutorial that wasn’t resource specific. We wanted students to be able to apply these basic research principles to searching any catalog or index.

Q. The content includes concepts that librarians would like all students beginning research to have mastered. How is it used at your school? Is it required for any classes?
A. The tutorial is not required for any library instruction classes. Instead, a few librarians have used it within the classroom during the first session of a two-part, introductory English composition class. When we use it, we usually have students complete the tutorial at the beginning of the first class. Then we transition into searching the catalog by giving them a topic (e.g., “the effects of technology in the workplace”) and asking them to find books on that topic. Ideally, after completing the tutorial, students will select the most important words, and perform a Boolean search. After that, we elaborate on search techniques by discussing such concepts as using “OR,” and using subject headings.

Q. Describe your design team and their responsibilities for content development, software selection, and technology support, etc. Why did you select the software you used?
A. The tutorial emerged as part of an IMLS (Institute of Museum and Library Services) grant. In this grant, Karen Markey, a professor at the University of Michigan’s School of Information, wanted to study the effectiveness of training librarians to create interactive online tutorials using Flash. After that, she wanted to study the effectiveness of these tutorials in imparting research skills to library users. We went through multiple phases of usability testing using undergraduates at our school. As a part of the grant, we received training in instructional design, and using Flash to create an interactive tutorial. While we developed the content of the tutorial, and created storyboards outlining each section, we eventually hired a graduate student to build the tutorial using Flash. The background of the IMLS grant and the results of the study were published in a recent article in portal. (Markey, K. et al. (2005). Testing the Effectiveness of Interactive Multimedia for Library-User Education. portal: Libraries and the Academy, 5, 527-544.)

Q. Did everyone on the team receive training in instructional design and Flash?
A. Both of us (Annie and Helen) received training in Flash, and basic principles of instructional design. We came up with the idea for the tutorial and created the content and storyboard. Originally, we planned to develop the tutorial ourselves using Flash. However, when we realized that this would be too time-consuming, we decided to use grant money to hire a Flash developer--a graduate assistant from the University of Michigan named Michael Spaly.

Q. It seems from your description of the process that you were creating the tutorial by means of a virtual collaboration with your developer at the University of Michigan. How did this work out for you?
A. We started out by mailing our storyboard to him, and we conducted our first "meeting" over the phone, where we went through the storyboard page by page. He would work on segments of the tutorial, and then email us the Flash files. We would review them and respond to him via email with our comments, questions, and requests for changes. So the majority of the communication was via email, supplemented by occasional phone calls. It was time-consuming in terms of having to write out our comments and send them to him, but we felt this would be easier for Michael (so that he would have something to refer to in terms of what we wanted done). Overall, it went well and was a positive experience.

Q. What support did you receive from your library administration or school for developing the tutorial?
A. We were mainly given the time to do it. The funding to develop and test the tutorial came from two different sources. Part of the funding came via the IMLS grant secured by Karen Markey at the University of Michigan, and part of it came from a grant that we received from the University of Illinois at Chicago’s Council for Excellence in Teaching and Learning.

Q. Discuss any other tutorial projects you have planned or completed. Is the “Doing Research” tutorial one of many (or part of a larger group of) tutorials or web-based instruction?
A. It was originally envisioned as part of a larger series of tutorials. We would love to develop more animated and interactive tutorials, but it is very time-consuming to work with Flash. Right now we’re working on a short series of resource-based tutorials using Camtasia and Macromedia Captivate—software that is a little more user-friendly than Flash, in terms of development.

Q. What feedback have you received from students about it? From faculty?
A. We tested the tutorial with thirty undergraduate students, and their feedback was largely positive. They liked the fact that it was animated and interactive. Some of them even called it “fun.” When we use it in classes, some of the students like it and some don’t. But we find that it works in teaching the concepts, so that’s good. Several faculty members have seen it, and their feedback has been really positive as well.

Q. You mention that your tutorial was part of a larger study and you went through usability testing with it. Do you have any advice, like what's involved and how to get started, for librarians wanting to conduct (or have someone conduct) usability testing on their own tutorials or websites?
A. Because this tutorial was part of a larger study by Karen Markey, the questions for the usability study were designed by her.

Our own advice would be to ask a variety of questions that give you both quantitative and qualitative feedback because some of the written (narrative) responses from the students were the most useful in finding out how much they liked the tutorial and whether they found it useful. We were also present during the testing, so we could ask additional questions if we wanted to, and take notes.

The pre-tutorial questionnaire consisted of multiple-choice questions that 1) measured students’ understanding of basic online research concepts, and 2) asked them basic questions about themselves (e.g., gender and previous experience with the library and its resources). The post-tutorial questionnaire consisted of 1) multiple-choice questions that would measure whether the students understood the content of the tutorial, 2) Likert-scale questions that asked students to rate their experience of the tutorial, and 3) written (narrative) response questions that asked students about using the tutorial and similar tutorials in the future.

Q. In the development of the tutorial, did you learn any lessons from the experience that you would pass on to other "librarian/designers"?
A. Our advice is to allow individuals working on the project to do what they do best. As librarians, we created the idea for the tutorial, the content, the storyboard, and oversaw the project, step-by-step. We were not experts in Flash development, so we hired someone else to do that. We offered feedback to the Flash developer throughout the process, and he would change things accordingly.

March 2006 PRIMO Site of the Month