September 2005 Site of the Month
NetTrail, The University of California Santa Cruz's Information Literacy Tutorials
UCSC NetTrail Development team: Ann Hubble (co-chair); Deborah Murphy (co-chair);
Christy Caldwell; Christy Hightower; Ken Lyons; Bryn Kanar
Institution: The University of California Santa Cruz
Interviewer: Courtney Greene
NetTrail is an online resource for UCSC students, a self-instructional tutorial to teach them basic library and research skills. It covers the research process from initial topic selection to citation styles and the issue of plagiarism. Its content is organized into six modules, which contain text, graphics, and interactive modules. Each of the six sections takes about 10-15 minutes to complete, and students can do this on their own time outside of class. The modules are followed by a short self-administered quiz.
NetTrail is designed for students new to the research process, specifically lower-division undergraduates. It is not subject-specific and can be used in a variety of introductory courses. It has been used by instructors teaching across a wide range of disciplines, such as Writing and Computer Sciences.
Q. What was the motivation for creating NetTrail?
A. The University of California held a system-wide conference in 1996 on undergraduate education. This brought together a self-selected group of UC Santa Cruz librarians, computing services staff, and faculty interested in developing a resource to provide incoming undergraduate and transfer students a solid grounding in basic online skills and an introduction to more specialized computer resources available at UCSC. Such a resource would supplement the teaching of these skills by faculty in the classroom.
Q. How long did the development process take?
A. Work on the original "NetTrail: The UCSC Computer Literacy Course" began in fall 1996 and was released in fall of 1997. In 1998, our development team competed for campus-wide instructional technology funds and, based in part on the successful introduction of NetTrail, was awarded a $36,000 grant that has allowed us to revisit deferred items and take development further. During the summer of 1998 some of this money was used to hire an editor to give a consistent voice to all of the NetTrail modules; in 2003 a web designer was retained to aid us in updating our site.
NetTrail remains under continuing development. Constantly evolving to keep pace with the current state of student needs and knowledge, it is a project that is never really finished. A smaller Continuing Development group of UCSC librarians continues work on refining and updating modules as needed.
Q. Who was involved?
A. NetTrail began as a creative collaboration between interested UCSC librarians, faculty, and campus Computing and Technology Services. Those who worked closely with undergraduates and online resources, had been interested in instructional issues, and were willing to develop supporting resources, formed the group.
Library involvement was key in keeping these tutorials current and focusing them increasingly on information literacy rather than specific computer-based skills. Given the library's interests, commitment, and technological expertise, development of NetTrail is now overseen entirely by UCSC librarians.
Q. What factors did you consider as you revised NetTrail over the years?
A. Changes in the nature of library research and technologies have led us to refocus our instruction away from our initial skills-based model to one that addresses how best to find, evaluate, use, and communicate the information a student needs. In a major revision in 2003, the team reviewed and redirected the entire NetTrail site. We looked at everything from content, graphics, and interactivity to the quality of the "voice" of the tutorials. As a result, we have retained a similar feel and flow to our original design, but have incorporated changes in our approach to instruction and innovations in online technology to keep the modules current and fresh.
Individual modules were developed to be used either in sequence or independently as needed. Within those modules we incorporated:
- Text-for introduction and explanation of basic concept.
- Animation-for step-by-step demonstration of more complex concepts and resources, as well as to reduce the reliance on text-heavy pages; some graphical elements have been used for purely aesthetic purposes, to provide continuity between modules and to add an element of whimsy
- Interactivity-to engage the user with hands-on decision-making and application of skills acquired while taking the tutorial
Q. Tell us about the technologies that were used to create NetTrail, and why you chose them. Were there others that you considered?
A. Consulting several well-respected antecedent tutorials such as Colorado State University's The Data Game and the University of Texas System's TILT, it became apparent that a more technologically and graphically engaging version of NetTrail was possible. Our aim was to appeal to the maximum number of users while avoiding any requirement that they have specialized software or hardware installed.
One example of this was the decision to utilize Flash technology. After looking at our local student population, we found it widespread enough that the majority of users would already have the application, as well as being versatile enough to provide the graphical elements -- sequential motion-picture demonstrations and animated accents -- we desired.
Q. What support, if any, did you get to assist you in the creation of NetTrail?
A. The expanded technical skills of the librarians, and especially of the library Web developer who joined the team, gave us a deep and varied level of expertise with much of our work based on sweat equity. Our grant monies were used to hire short-term assistance for editing and specialized application development as needed. On a broader level, Open Public License agreements used by NetTrail and other tutorials have allowed us to share ideas and concepts across the nation. This collegiality has been critical in exploring and developing new modules.
Q. How is NetTrail currently used?
A. Several thousand students have used NetTrail over the last eight years. Initial usage indicated that the library research component of the early version of NetTrail was the most heavily used and this data helped in the decision to move the tutorial towards an information-literacy focus. Usage statistics of completed tutorial quizzes were kept until recently. This indicated usage was highest in those classes where students were required by faculty to complete the tutorial. Although primarily used by first-year Writing students, we found faculty in larger lower-division Chemistry, Computing, Biology, and Literature courses have all used this resource in their classes.
Q. Is it tied to any particular class as a requirement, or how do students and faculty learn about it?
A. It is not subject-specific. It has been used by instructors teaching across a wide range of disciplines, such as Writing and Computer Sciences. We've used a variety of methods to introduce the resource to faculty and students. There are direct links to it on the top library Web pages, as well as links on UCSC faculty-related Web sites. We do regular informational mailings both in print and online, and articles regarding new features have appeared in the campus's Center for Teaching Excellence newsletter. We also actively participate in campus-related instructional technology activities and programs.
We've been promoting NetTrail as an outreach resource, both for incoming students as well as local high school and junior college students interested in doing research at UCSC.
Q. What kind of feedback, either formal or informal, have you received from students or faculty?
A. Sample comments from our user testing indicated that NetTrail enlightened both novice and more experienced students to the fact that:
- Journal articles can not only be searched for, but often downloaded in full text from databases available through the library’s Web site
- There are identifiable differences between popular magazines and scholarly journals
- If they later ran into a research problem, NetTrail is one resource from which concise information could be had
- Users also liked that NetTrail made learning interactive, that it didn’t require completion in one session, and that the graphics and animation gave an informative break from the textual content.
Q. Do you collect any response data from the assessment quizzes?
A. The type of data that we collect has evolved as the tutorial itself has evolved. We had access to a programmer from campus computing staff for the original NetTrail site. It was originally a 'computer literacy' tutorial, with modules covering web browsing, how to use email (remember pine?), usenet, as well as library resources. Faculty would first sign up their class and then statistics were kept on each student who signed up for NetTrail and what modules they took. Faculty had access to the statistics, as did NetTrail team members.
Registration for NetTrail required someone from the computing staff to actually create and register classes one by one. It was a bit of a barrier, though the statistics were extremely interesting. When we (librarians) revised the site into an information literacy site, we did not have easy access to a programmer. In the interest of updating the site quickly, we decided we could revisit using a programmer at a later time. We do have regular stats on number of visitors to the site, just not the fancy ones we had before.
After completing our computer literacy site revision, the NetTrail team wondered just how challenging the quiz would be for our students. We did some user testing and found that it was too easy, so we rewrote many of the questions to make it more challenging. We also wanted to test usability of the whole site, but were especially interested in the interactive modules. We focused some of our testing on the two modules that included interactive portions, but did some additional testing with the entire site as well.
Q. Can you describe the usability testing process?
A. We designed three different user tests:
- To assess the quiz questions, we gave the test to 150 students who had not taken the tutorial. We also asked demographic questions (number of years in school, number of library-provided instruction sessions, etc.).
- We chose two modules for focused usability testing: one that included an interactive piece, and one that included a Flash movie. The module with the Flash movie was among the longest. These modules differed in tone, one being casual and the other more formal. The participant would navigate through the two modules, be encouraged to ‘think aloud’ and be observed by two NetTrail team members who would keep track of start/stop times, thoughts expressed, body language, and links clicked.
- To gauge how long the entire tutorial was, two participants completed the entire tutorial including the quiz, and recorded their start and stop times. They were then asked a series of questions. It took each participant approximately 45 minutes to complete the entire tutorial and quiz.
Q. How do you use the data?
A. It helped us uncover a consistent area of student confusion: using our local OPAC system to search for a journal title when they have a complete citation.
It has also demonstrated a need for a pop-up glossary available throughout all of the modules of the tutorial. By designing the modules to be stand-alone, concepts introduced in one module cannot be assumed in another module. Key terms and concepts can be linked to a small pop-up window that can describe or define them.
Q. What were some of the challenges (technological or other) that you faced?
A. We've instead listed a few recommendations, based on our experience:
- Start user testing earlier than you think you need it! From the beginning of the project, we kept a list of issues that should be evaluated with user testing.
- Make a plan to begin user testing well before rollout of the product.
- Keep in mind that some tutorial components, such as Flash movies or interactive items, can be user-tested early and independent of the rest of the tutorial.
- Work with librarian colleagues, as well as faculty, during the user-assessment phase. Not all librarians were as comfortable with or as interested in technology as those on this development team. Determine the instructional needs and concerns of your colleagues and market to them as well as to faculty.
Q. How has NetTrail contributed to or influenced your library's instructional services?
A. NetTrail has gradually become a key component of our instruction and outreach programs. Developed ahead of the curve, it has taken awhile for colleagues to decide how to incorporate it into their instruction. It's now a featured resource of our instruction program. The library development team has grown over the years as interested and technologically innovative colleagues have joined our group.
Q. What are your future plans for NetTrail?
A. We keep a wish list of development ideas and routinely review these throughout the year. Our current list includes:
- Continue assessment and development
- Conduct effectiveness study to assess knowledge gained
- Add pop-up glossary to accommodate modular structure
- Add more interactive features to replace textual explanations
- Expand by adding targeted tutorials for different audiences (subject-specific, graduate, new faculty, etc.)
- Expand reporting and statistical capabilities
September 2005 PRIMO Site of the Month