Children, Technology, and Instruction: A Case Study of Elementary School Children Using an Online Public Access Catalog (OPAC)

SLMQ  Volume 23, Number 1, Fall 1994

Paul Solomon, Associate Professor, School of Information and Library Science, University of North Carolina at Chapel Hill. At the time of this article, he was Assistant Professor.

Library media specialists are challenged by a wide array of ideas, methods, and tools that are touted for improving services and advancing learning. Integrated language arts, flexible scheduling, and electronic systems for information retrieval, transfer and management are but a few of the approaches the library media specialist is called upon to champion. Yet little is known about how well such approaches fit together and how to fine-tune them separately and in combination to influence learning. This column examines elementary students' use of an online public access catalog (OPAC) within the overall instructional environment of their school in order to open a window on the broader interaction of children, technology, curriculum, instruction, and learning.

The column describes a personal odyssey as well as a research study. Paul Solomon was intrigued by his experiences as a volunteer in his award-winning wife's library media center (Gerry Solomon was named Media Educator of 1989 for the state of Virginia), and he wanted answers to the complex, context bound questions that occurred to him as he worked with students. He was one of the first researchers in the school library media field to use qualitative methods to explore such questions, and his column provides a primer on this methodology as well as important insights into the factors converging around technology that enhance as well as constrain meaningful learning. His observations of some nine hundred OPAC transactions by almost five hundred students, buttressed by interview data and documentary evidence, yielded findings about the nature of and reasons for students' successes and breakdowns (i.e., failures in retrieval) in OPAC use, the strategies the students use to control the OPAC, the influences of search terminology and the interface on their performance, and the changes overtime in their searching behavior Solomon's insights into the contributions of various facets of the school environment to each of these areas are of particular interest to library media specialists.

The adventure began quite by chance. As a volunteer in an elementary library media center, I had the dual tasks of cataloging materials and helping children use the online public access catalog (OPAC). First, I found myself struggling to select appropriate subject headings for the catalog records; this was not easy, as my first thoughts of search terms were usually not in the authority list, and the terms in the list were not usually the ones that occurred to me. Second, as I helped children select terms for their OPAC queries and dealt with other questions pertaining to their use of the OPAC, I began to wonder: How can this work? How can the words that children naturally choose for their OPAC queries match in some way with the OPAC's terminology for subject description so that items appropriate to children's interests and assignments can be retrieved? How can children consistently retrieve information from this system without frequent help?

The answers to these questions have been reported in several forums;(1) when I first began to look for them, however, I found very little previous research on information retrieval systems for children to guide my efforts.

Yet what was available was helpful. An early study by Joyce and Joyce(2) raised the possibility of information systems designed specifically for children. These researchers tested some preliminary design principles relating to the organization of the information contained in a database of the Pueblo culture in the southwestern United States. This important work suggested the significance of classification systems, especially their organization and display, in facilitating or hindering retrieval. Thus, I concluded that both the subject heading system in use and children's search terms were strong candidates for factors that contributed to success and breakdown in children's information retrieval.

Two additional research programs offered further ideas for consideration. Kuhlthau (3) raised the issue of developmental states in information seeking and tied these to articular information needs, suggesting that a child's developmental level might offer a potential explanation for success or breakdown in OPAC use. For instance, were elementary school children developmentally ready to combine concepts in a query or to see possible connections between subject and title searches? Would these children, especially in grades one through three, have trouble employing multiple concepts or shifting from one perspective to another?

Edmonds, Moore, and Balcom (4) compared children's OPAC and card catalog use and reported several unexpected results that deepened and expanded my own questions: not only did they find a 10 percent success rate for searches involving the OPAC versus a 65 percent success rate for those with the card catalog, they also noted that the children preferred the card catalog. As these results were far different from my own experiences and intuitions, I examined their research report more deeply. The card catalog results began to make sense when I fully understood the nature of the test OPAC: it used a touch screen interface that required children to move through at least nine levels of screens before they could satisfy their interest in an author, title, or subject. The interface also required children to have well-developed alphabetization skills; children without these skills were likely to become hopelessly lost in "OPAC space" and give up their searches. Thus, I surmised that the nature of the interface was another potential factor in children's success and breakdown in information retrieval.(5)

Finally, while my interest began with a focus on children using OPACs, it soon became clear from my reading and my experiences that I could not answer my questions without understanding the broader influences of the library media center program, the instructional approaches taken by teachers in the classroom, and the school curriculum. Thus, from an initial concern with the match between child and OPAC, the research concept expanded beyond the information system to focus on children, the library media center, the classroom, and the school, as well as on what these contextual factors contributed to children's successes and breakdowns in their OPAC transactions.

Methodology

Given the interests and questions out. lined above, it was clear that the typical research approach of selecting a small number of variables to specify a model for investigation would not be productive. Rather, I had to choose a method that would support exploration and allow identification and consideration of factors that had not been considered in earlier OPAC studies. In this way, I could uncover unanticipated impacts and influences. The naturalistic approach that I employed is ideal for such purposes because it lets children pursue their own interests as well as those that normally arise during the course of their work. This allows researchers to pursue findings that emerge when schools and other research settings are operating in their natural manner. Using this methodology, I was able to document the students' problems in information use as they arose and to check my tentative solutions against the children's own views.(6) Drawing on the perspective provided by the ethnography of communications,(7) I described and analyzed each OPAC transaction as a communicative event. I chose a detailed case study format, which allowed me to explore a single case in order to identify patterns of behavior that elucidate the phenomena under study, to gain a deep understanding of these phenomena, and to develop a rich description that would expand the understanding of others as well.(8)

Naturalistic research may employ any of the data collection methods that are typically used in social science research. The key characteristic of this approach is that the methods are used in situations that are as "natural" as possible: the research setting is not constructed, manipulated, or controlled. Most naturalistic research involves multiple research methods that allow the researcher to view phenomena from different viewpoints or to check what might be identified through observations against written documentation or comments obtained through interviews. The use of these multiple methods increases the trustworthiness of the findings.(9)

In this study, my methods included observation of children using the OPAC; interviews with children, teachers, the reading specialist, the library media specialist, and volunteers; think-aloud protocols; and documentary evidence. The documentary evidence included physical traces of children's actual use of materials identified through their OPAC sessions (e.g., written reports, charts); the media specialist's logs and other records; teachers' plans; curricular materials; and technical manuals for the OPAC itself. My primary method, observation, involved the detailed recording of events at the OPAC: what query terms were entered, what problems were encountered by children, and if and how these problems were solved. Initially, I had intended the interviews with the students to follow the observations in order to avoid influencing the naturalness of the children's activities. During the work, however, it became clear that the observation and interviewing were more natural for the children when they were connected. In retrospect, it seems evident that questioning is part and parcel of the learning conversations in the school and that children become accustomed to questioning as they perform their assignments and other school activities.(10)

One of the important features of the research design was that it tried to move beyond the single-snapshot view of socio-technical phenomena that is inherent in many studies of the use of technology. Thus, while I did map children's individual OPAC transactions, the very fact that there were many of these transactions and that they were tied to specific children over time gave a longitudinal dimension to the study. Thus, the study design allowed some insights into the dynamics of the situation as well as into changes in children's OPAC use over time. This feature of the study design proved important in understanding possible influences of classroom instructional strategies and library media center policies on children's information retrieval behavior at the OPAC.

I collected data during some thirty-five observation sessions over a full school year. Most sessions took place over a full school day, and the days were randomized to ensure that the observations picked up differences that might occur on different days of the week. In total, 902 OPAC transactions involving approximately five hundred students were observed over the course of the study. I recorded the details of the children's OPAC transactions by hand during each session; afterwards, I transcribed and expanded my handwritten field notes along with tape recordings of my conversations with the children by adding my comments, interpretations, and preliminary analyses. By completing these steps for each observation session before beginning the next session, I could use subsequent sessions to address questions raised in earlier ones and to test and revise or elaborate my preliminary interpretations.

Development of this analytical chain of observations, field note expansions, and "petite" analyses was accompanied by the development of a categorization scheme that I used to code the data. I designed a code for each of the various strategies, problems, and other contributors to children's successes and breakdowns at the OPAC and applied the codes to the entire data set: the expanded notes, the interviews with school staff and volunteers, and a variety of written records. For example, I assigned the code "S-P-EXT" (for "Strategy-Planned-External Support") to all the instances of behaviors, events, and strategies related to students' use of this particular approach. By looking at all the data segments coded "S-PEXT" as a package, I could perform "grand" analyses(11) of the patterns that emerged-that is, I could identify commonalities and anomalies related to such factors as grade level, the impetus for a query, and the passage of time. This entire process was facilitated through the use of information management software that combines the features of word processing, database, and text retrieval systems and that allowed me to retrieve segments by code. For example, if I wanted to look at the incidence of the external support strategy for first-graders over the course of the school year, I could retrieve the appropriate records from the text-base of expanded field notes using the appropriate codes. Retrieving and examining groups of records both for individual codes and for codes in combination constituted the final data analysis that led to the identification of the themes, issues, and other findings noted below.

Additional approaches employed to enhance the trustworthiness of the study's results included discussions with the library media specialist, the reading specialist, and a few of the classroom teachers to share interesting or unusual findings and to get their insights and reactions to my interpretations. Also, I discussed the findings and interpretations with a small group of children to gain further insights into the nature of children's use of technology for information retrieval. (12)

The Situation

In a naturalistic study, consideration of the impact and influence of the context of the research on the phenomena under investigation is an important part of data collection and analysis. Since some understanding of the situation is also important for understanding the findings presented here, key aspects of the research context are highlighted in this section.

The School and Students

The study school was located in a large suburban school district in the Washington, D.C., metropolitan area and had been in operation for one full year at the beginning of the data collection period. The school's population ranged from families in public housing communities who received public assistance (almost 7 percent of the school's student body received free or reduced-cost lunches) to those who lived in large and expensive homes; the majority of students, however, were from middle-class families. The total school enrollment was 872 students from preschool through sixth grade, but only the 679 students in grades one through six used the OPAC. About 8 percent of these OPAC users received special instruction related to identified learning disabilities and, with this added support, were able to function in regular classrooms. These disabilities did not appear to influence OPAC use.(13) About 18 percent of the students were classified as minorities, with about 8 percent Asian, 6 percent African-American, and 3 percent Hispanic. The performance of minority students at the OPAC did not seem to differ from the general behavior patterns of all students.

Curriculum and lnstruction

The school's primary curricular and instructional strategy was the integrated language arts program, which heightens reading and writing within the curriculum by treating these as fundamental to most learning experiences. One element of the program as adopted in the study school involved an emphasis on critical thinking skills and child-oriented, purposeful, self-directed activities. This instructional approach had many effects on the library media center's program and strongly influenced the findings of this study. The traditional basal reading text was not used in favor of trade books and self-selection of library media center materials. As a result, collection development was closely tied to both curriculum and children's interests. Availability of and access to materials became critical, because children were expected to find materials related to their interests and assignments. Thus, computer-based access tools became an essential part of the process of identifying materials.

It is important to note that adoption of integrated language arts and its associated instructional techniques varied by grade. The second- and third-grade teachers were strong adherents to integration, use of trade materials, and children's following their own interests in selecting materials. For instance, children studying animals selected and read materials about animals of their own choice and then wrote and talked about the habitats, foods, or houses of those particular animals.

Most fifth-grade teachers followed a more traditional, subject-centered curriculum. Assignments were oriented to facts and did not emphasize critical thinking and application of facts. Teachers of first, fourth, and sixth grades seemed to take an intermediate position by using both approaches to varying extents. One of the most interesting findings of this study was that the instructional approach taken in the classroom integrated language arts or not-had an influence on children's OPAC behavior over time.

The Library Media Center

With its vaulted ceiling and walls of red brick, a story pit, and a separate classroom, the library media center provided many spaces that accommodated combinations of classes, small groups, and individual students. In addition to a brand new collection, displays of materials, and an array of children's art work, the facility boasted a 60-foot dragon hanging from one highly visible wall and a huge soft sculpture of Mother Goose that seemed to fly across the story pit. Comfortable seating of various types for children of different sizes, activities of all kinds, and varieties of animals (stuffed and otherwise) added to the room's attractiveness.

The library media center maintained both flexible scheduling and an open door. Thus, a teacher might allow individual children to go to the facility, send small groups, or take the whole class to perform research, follow up on some question, complete a fact-finding mission, or find a book to read. Flexible scheduling along with the library media specialist's philosophy of tying information skills instruction to teaching in the classroom were major facilitators of the integrated language arts program. Taken together, they provided a process approach to information skills instruction that was tied to specific information needs at particular points in time.(14) Thus, early in an information search process, topics and topic-finding strategies using information resources might be the emphasis of information skills instruction. Later in the process, strategies for filling in the details of a topic might receive attention.

The introduction of computer-based tools for information access and retrieval was an integral part of the library media center's offerings. For instance, children were introduced to encyclopedias on CD-ROM when looking for facts about people, places, or things; they were introduced to online searching when dealing with subjects that were absent from the library media center's collection or when current information was required. Thus, while the primary tool used for this study was an OPAC, many students were also using other technologies available in the marketplace at the time of the research.

The OPAC

The study OPAC itself was part of an integrated library management software package that included circulation, catalog maintenance, and inventory functions in addition to catalog access. This software ran on a local area network of five workstations: three devoted to public access to the online catalog, one to circulation, and one to catalog management and maintenance.

The initial OPAC screen displayed subject, title, and author selection boxes and instructed users to "type in what you are looking for" and "Press ENTER to display the results found." The challenge for children was to figure out what to enter in which boxes. The use of the keyboard to construct keywords was also a problem for many children. There was no help within the OPAC system to aid children in moving from an interest or need to the correct spelling of an author's name or the correct form of a subject term. Some external aids were provided: a list of commonly used subjects was posted above the OPAC stations, and a notebook containing the subject headings in use and a card with instructions for truncation were available at the workstations.

There were three possible outcomes of a search in this OPAC: (1) a TERM NOT FOUND message; (2) a QUERY ERROR message; or (3) a list of retrieved items that matched the keywords typed into the author, title, and/or subject blocks. The TERM NOT FOUND message indicated that there was no match between the query and the contents of the OPAC's database and could result from a misspelling or the use of an inappropriate subject term as well as from the absence of a subject in the collection. The QUERY ERROR message indicated that the query was misformed, and could result from such minor errors as the use of punctuation or a space at the end of the query.

A user who created a match was presented with a list of retrieved items in a short bibliographic format: author, title, place of publication, publisher, publication date, and call number. Users could either browse through the list by moving from screen to screen or request more information on an item by selecting it. If an item was selected, a user could choose either to (1) look at a more detailed display that provided the short bibliographic format described above, a short summary of contents, and the item's subject headings or (2) note whether the item was checked out, missing, or available.

This set of alternatives structures the user's OPAC actions. Yet each action leads to a potential breakdown. Even a successful match, for example, could be inappropriate, as when the term "bats" retrieved items on bats as sports equipment as well as on bats as an animal. The study OPAC provides no support at any potential breakdown point, suggesting that people using software of this sort either must be trained to overcome a wide range of potential OPAC breakdowns or must fend for themselves. Ideally, the software could offer aid and comfort. Certainly, one outcome of this study is to help recognize the ways in which both user training and information retrieval systems might be advanced.

Findings

The study yielded findings in five areas: (1) the overall success of children using the OPAC and the reasons for their success or breakdown (i.e., failure to retrieve useful information); (2) the strategies that children used to control the OPAC; (3) the contribution of the interface; (4) the correspondence between children's search terms and the subject headings included in the database; and (5) change over time in children's searching behavior. The following discussion emphasizes the impacts of differences in class instruction and assignment on children's information retrieval behavior and its outcomes.

Patterns of Success and Breakdowns

Success. Overall, the success rate for children ran at about 66 percent for the information retrieval transactions observed during the study period. This is at the high end of reports of students' OPAC success. The touch screen/card catalog study mentioned earlier(15) reported success rates of 10 percent and 65 percent, respectively, while Shu-Hsien Chen's study of the online catalog searching behavior of high school students reported successes for author, title, and subject searches of 69 percent, 67 percent, and 60 percent, respectively.(16)

Analysis of the reasons for success in children's OPAC transactions suggested three primary contributing factors: (1) the recognition by some children that they needed help; (2) the use of control strategies to overcome OPAC breakdowns; and (3) the choice of simple, concrete query terms (e.g., "cats," "dogs," "karate") that frequently matched the Library of Congress Subject Headings (LCSH) used in the database.

Children who succeeded because they sought help recognized that they had problems they could not solve themselves. Not surprisingly, requests for help were greatest among first graders, who used assistance to overcome breakdowns in about 18 percent of their transactions. Requests for help were lowest among sixth graders, who rarely asked for assistance; only I percent of their transactions employed an assistance strategy. Students in the intervening grades showed a gradual reduction in their use of assistance strategies to overcome breakdowns.

Children who succeeded because they followed initially unsuccessful strategies with one or more other strategies that ultimately resulted in retrieval success were able to recognize that they had problems, to diagnose possible approaches to overcoming those problems, and to apply these solutions in a productive way. This use of strategies increased roughly with grade level, but there were interesting deviations in both frequency and type of strategy use across the grades that are highlighted in the discussion of search strategies below.

The third contributor to success is related to children's interests and natural language use. Young children, for instance, enjoy animals (e.g., dogs, cats, horses) and sports (e.g., baseball, karate) that translate directly into LCSH. Since their interests tended toward simple, concrete objects that did not have widely used alternative terms, their queries often matched the LCSH and they experienced retrieval success.

Breakdown. Children's attempts to retrieve information failed when they were unable to overcome problems in the retrieval process. Because the study OPAC provided no clues to interpret and fix unwanted results other than the TERM NOT FOUND and QUERY ERROR messages, children had to develop their own approaches for overcoming problems. Reaching a conclusion about what enabled children to overcome some problems and what was characteristic of those problems that children were unable to overcome could not happen through a static, snapshot view of children's OPAC use. Rather, an explanation became evident only late in the course of the study, when the traces of the OPAC transactions of individual children over time and the overall occurrences of breakdowns were placed together to suggest patterns of behavior.

When first graders were introduced to the OPAC, I was able to observe their labor intensively. At the beginning, I saw them struggle with the keyboard in a painfully slow manner. It was only a matter of weeks, however, before they could make the keyboard work for them. This experience, which took place when the school year was about two-thirds complete, suggested a possible pattern of development. Checking the first graders' pattern against the progression of problems for students in all grades, I found a comparable array of skills, rules, and knowledge development. For instance, keyboard problems that were common in the first month of the study became relatively infrequent later on. This phenomenon of learning was present in many streams in the observations: children seemed first to rely on general knowledge-based control strategies, then to develop rules (often by induction), and ultimately to become skillful. For example, with keyboard mastery, children first used the general strategy of systematic search to find a desired key; then developed rules to narrow the search (e.g., the a key is on the left); and, finally, "just knew" where to find a specific key.

This realization that breakdowns could be viewed in terms of kinds of knowledge, rules, and skills led to the general classification presented in table 1. Within each of the general categories of knowledge, rules, and skills, there tended to be a progression of movement from the basic to the more advanced. For instance, fundamental knowledge breakdowns happened when children did not analyze their results to see if they were appropriate to their needs. In other words, children could not correct a problem if they did not realize that one existed. If a query using the term "clothing" returned a TERM NOT FOUND message, a child could accept this result or recognize that there must be some information on clothing and move on. Similarly, if a child realized that a problem occurred, follow-up actions needed to be considered: if the query term did not work, try another one (e.g., "clothes"). For a follow-up action or strategy to work, children needed to have sufficient content knowledge of the field to try various query terms until a match was found: "I tried 'clothing' and 'clothes'; my teacher suggested 'apparel.' I can't think of anything else." It turned out the LCSH in use was "costume."

In short, children moved through a learning progression to overcome those situations in which an initial query did not get them what they wanted. The knowledge, rules, and skills breakout simply focuses attention on the key contributors to the breakdowns and, along with their subcategories, suggests design and instructional solutions as indicated in table 1. These solutions are similar to those presented by Neuman(17) as a result of her study of high school students' use of on-line and CD-ROM databases, suggesting that there may indeed be generic approaches to the problems students face in using electronic information resources.

Why did some children find it easier to develop OPAC skills, rules, and knowledge than others? Tracking the information retrieval behavior of individual children over time suggested that the kind of instruction and the nature of assignments in the classroom had a great deal to do with what materials children were trying to find using the OPAC and the sorts of breakdowns they encountered.

The clearest contrast comes from comparing fifth graders with other students. The fifth graders were given clear-cut assignments, such as "Turn in a written report with the following pieces of information on your state." Their OPAC skills were not challenged with this sort of assignment, as most children could simply enter the name of a state (e.g., Alabama) and get a list of pertinent items. At worst, they might have to correct a misspelling or, with a state such as Washington, search a longer list that included extraneous items. While the success rate for fifth graders was among the highest, the lack of challenge in their searching limited what they discovered about information retrieval. Fifth graders, as a result, were less able to recognize and respond to breakdowns than other students who had more variety in their OPAC experiences. Also, the sophistication of their OPAC strategies was often less impressive than that of children at lower grades.

In contrast, children in other grades, especially second and third graders, were given assignments that required more in-depth knowledge and more creative control actions at the OPAC. For instance, individual second graders were asked to select an animal of their choice, find a book about it, read it, pick out certain pieces of information, and report to the class. Later assignments built on this foundation by encouraging additional reading and comparisons with other animals. Children quickly went through the books directly related to their animals and began to develop strategies that employed broader, narrower, and coordinate terms that would help them identify additional sources. A similar situation happened with a small number of first graders who were each asked to write a report on a planet of their choice. They began by using the catalog to find books on their planets. When they had exhausted those books, they began trying to find other books that mentioned their planets, using such query terms as "solar system" and "planets."

This pattern was repeated again and again in many assignments and carried over to children's use of other resources of the library media center. In turn, the challenge of assignments that encouraged children to build on their interests or investigate subjects in some depth carried over to their information retrieval behavior at the OPAC. They needed to move beyond the simple subject search to discover, learn, and try many strategies to get information on their interests and to overcome OPAC breakdowns. Thus, the school's integrated language arts curriculum and instructional strategies that highlighted critical thinking seemed to carry over to the children's use of technology. The OPAC, however limited in its support capabilities, provided a vehicle that allowed children to discover and learn.

Search Strategies

Children who used strategies learned to take control of the OPAC as they directed and redirected their searches to find the information they needed. It was evident from the think-aloud protocols and interviews with children that some strategies were planned in advance to deal with anticipated breakdowns. The use of external support strategies, which included asking for assistance from others, is a good example of a strategy that was often planned in advance. Other strategies were employed to overcome expected breakdowns and to follow up on insights gained in previous moves. Thus, there was evidence that many children, even young ones, were able to use the OPAC for information retrieval in a critical way. When breakdowns were anticipated, these children weighed possible strategies and selected or developed initial actions to avoid breakdowns. When breakdowns were unanticipated, children analyzed the given situation, posited a cause, selected a strategy that would deal with the cause, and applied the strategy. If the strategy did not work, they modified it or tried another.

  Table 2 profiles the strategies that children learned or discovered in their efforts to take control of the OPAC. Analysis of these strategies and their circumstances of use suggested that the following factors helped to shape this range: instructional approaches and children's interests, the structure of interaction presented by the OPAC's interface, children's natural choices of query terms, and the learning effects of time and experience.

Instructional Approaches and Children's Interests

Important contributors to children's information retrieval behavior are their motivation, purpose, and interest in their topics. During the course of this study, it became clear that some students were more involved in the information retrieval process than others. For some, it was an adventure and challenge that carried them to new understanding of the material they were studying. Their products went beyond copying facts from their readings to point out discrepancies and unanswered questions and, thus, to display their critical thinking skills. These children's experiences also led them to develop more advanced information skills, which they needed in order to pursue their interests in depth. For instance, the group of first graders studying planets soon began to share information about their planets and to propose theories that related the characteristics of the planets to their size and distance from the sun. Their written reports contained ideas and questions about such matters as what it would take to allow people to live on presently uninhabited planets.

For others, that spark of interest was missing. These children found the minimal information they needed to satisfy the assignment but contributed little of their own thinking and did not advance their information skills. Their products were simple lists of facts without interpretation or comparison. For instance, when fifth graders were asked to write reports about the states, their assignment sheet contained a list of pieces of information to be included in each report (e.g., population, area, major cities). Not surprisingly, their actual reports provided "just the facts."

This contrast suggests that differences in instructional approach, particularly with respect to allowing children to pursue interests in some depth, seemed to contribute to differences in students' involvement, learning, and production. These differences, in turn, influenced children's use of the library media center and its resources. When children needed to study some interest in depth, they needed-and wanted-more than one good book on the topic. The concomitant need for depth in the collection, reflecting children's interests and subjects receiving emphasis in the curriculum, extended to the reference collection and to current events sources available either in the immediate collection or through on-line databases.

The OPAC served as an essential finding aid for many of these sorts of resources, and the disparity resulting from differences in instruction was apparent at the OPAC. Children who needed only one good book to get the stipulated facts just did not benefit from the thinking and problem-solving required of those children who were challenged to keep on looking. Those children, once they had found the easily located materials on their topics, were driven to find additional information and to develop strategies to get it. Ultimately, the study OPAC-when coupled with the rich instructional strategy of integrating reading and writing with children's questions and interests-served as a vehicle for helping children find information while helping them build their information seeking skills. We can only wonder what an information retrieval system designed to provide active support or instruction might do to help children find the information they need.

Structure of lnteraction

Human-computer interaction is structured by the user's purposes and expectations in approaching an information system, the interface provided by the system, and the support the interface provides during the course of the interaction. Most children understood that the purpose of the study OPAC is to provide access to the items in the library media center's collection, but from time to time they had other expectations in mind. These expectations suggest a variety of possibilities for OPAC design.

Some children, for example, tried to use the OPAC as a fact finder. This approach sometimes worked incidentally through a child's interpretation of information provided by a title or the interaction of a child's knowledge with the information inherent in the list of retrieved items. This fact-finding behavior suggests that the natural progress of development in OPACs might be the integration of book access with fact access, especially since children must sometimes learn more about a subject in order to search for information about it.

Some children enjoyed exploring the OPAC by testing its limits and trying to figure out what would happen if a certain key were pressed. This exploratory behavior often led to incidental learning, suggesting the possibility of channeling the exploration to more constructive learning. Why not accommodate children's natural curiosity in information systems designed for them?

The study OPAC offered a minimal interface, and this structure had positive benefits for those children who knew what they were seeking. The interface helped them center their attention on the key issues of focus (author, title, or subject) and keyword selection in developing a query, thus easing the demands on children in many search situations. In some situations, however, the subject title-author entry structure was not helpful. For instance, the absence of a browsing capability was a problem for those children who were unsure of the appropriate query term or its form. Similarly, problems inherent in the keyword searching approach were a source of irritation for children. For example, use of the keyword "Virginia" retrieved items with the subject headings "Virginia" and "West Virginia" as well as any other records containing that keyword. While the keyword search feature has many benefits, including the elimination of the need for exact entry of each subject heading that contains the relevant keyword, it would serve children better if it were one of a series of options-keyword, exact match, and perhaps others.

Also problematic was the lack of tailored assistance within the OPAC's structure to help its users identify and recover from problems. A spelling checker would be a great step forward. Cross-references from children's natural language entry terms to LCSH (e.g., "weapons" 3> "arms" and "armor'; "clothes" 3> "costumes") would be another leap. Interfaces that combine direct query entry with browsing are beginning to appear(18) and should allow testing of how well these two modes will work together. Ultimately, of course, there is no reason for an OPAC or any other information retrieval system to issue TERM NOT FOUND or zero hits messages. Whatever the structure of interaction, the interface should help people learn to control the system.

Search Terms

One of the questions that led to this study concerned the match between children's natural selection of search terms and the subject headings in use in the OPAC's database. Direct match of the simple, concrete terms that young children are inclined to use with the LCSH in the OPAC was a major source of success in information retrieval. The lack of match of search terms is a more complex mystery, because simple, concrete terms sometimes did not match the LCSH in use (e.g., children preferred "bikes" and "planes" to "bicycles" and "airplanes"). Similarly, complex, abstract search terms such as "slavery" and "freedom" usually did not match LCSH, but they did occasionally (e.g., "revolution, ' "recycling"). Not surprisingly, there is no clear-cut rule that either human or machine might use to translate directly from children's natural language to LCSH. While a direct mapping from entry term to LCSH might be possible in the form of something like a thesaurus, such a map requires a sense of the words children actually use.

Patterns of similarity and variety in children's search term usage suggest ways of mapping and otherwise organizing the search terms that children use with the subject headings in an OPAC's database. For instance, both children's interests (e.g., dinosaurs, magic, drawing) and topics included in the curriculum (e.g., African-Americans, pollution, Civil War) suggest domains that might receive special attention, perhaps through provision of cross-references or some form of graphical display. Furthermore, a knowledge of the relatively small number of search terms students use would give us a place to start in providing mechanisms within the information system to aid students when they encounter breakdowns: the twenty most frequently used search terms accounted for 24 percent, and the one hundred most frequently used search terms accounted for almost 51 percent, of all search terms employed.

A detailed terminological analysis of the breakdowns associated with children's search terms indicated a diversity of conditions that contribute to these breakdowns and led to some understanding of the strategies to ameliorate them. For instance, some terms suffer from terminological diversity; any one of a variety of synonyms might be used as a search term. Also, the word forms that children use might vary from LCSH (e.g., children tended to use the singular form of "countable" nouns rather than the plural used in LCSH). Some words have multiple meanings (e.g., banks, bats). Some words sound alike (e.g., Wales and whales). Some concepts or objects of interest to children are not covered in LCSH or are not covered with the degree of specificity children require (e.g., types of bicycles, such as BMX). These and other conditions all suggest both user strategies and software design strategies that could enhance children's success at the OPAC.

Learning Effects

The kinds of breakdowns children encountered when they began to use the OPAC changed over the course of the study. For instance, basic breakdowns due to lack of reading skills disappeared quickly, as did most breakdowns that resulted from lack of understanding of what the OPAC was designed to do. Keying and spelling breakdowns also declined substantially as children became mote facile in using the keyboard and began to review what they had typed before entering it. As children overcame these skill hurdles, they moved on to more sophisticated breakdowns that required more thoughtful responses. Breakdowns at the intermediate level, for example, included a lack of understanding that word form influences success and that the output of a search must be monitored and interpreted. At the advanced level, breakdowns occurred when children were unable to construct strategies and develop sufficient knowledge in the subject area to suggest alternative terms (e.g., broader, narrower, coordinate). The point is that over time, children were faced with pushing their knowledge, rules, and skills further to overcome the increasingly complicated breakdowns they encountered.

Most software seems to be designed according to a "one-size-fits-all" model, and the size does not fit children particularly well. Consequently, while the interface exists to help users by structuring their interaction with the software, it may also limit how far those users (especially elementary children) can go in taking advantage of filters or tools that are hidden from view. An understanding of the nature of the learning process, the reasons for success, and the contributors to breakdowns (e.g., the forgetting, backslides, and overskilled learning) can provide insights that ultimately can help develop information retrieval systems that lessen children's struggles to retrieve information and expand their horizons as to the possibilities.

Conclusions

This study centered on a specific OPAC to note the circumstances in which it met students' needs and those in which it did not. Overall, the study led to some insights about learning, technology, and instruction that had not previously been apparent and that support the effectiveness of strong integrated language arts programs, flexible scheduling, and process-based information skills instruction. The study's findings also may provide some understanding of how OPAC designs of today might be advanced and how these advanced OPACs might help children learn.

There are many more stories of children, technology, and instruction to be told. There is a need to understand far more about how policies, instructional strategies, and information resources work together to influence learning. Library media specialists need to use what is known to fine-tune, if appropriate, and to throw out and invent, if fine-tuning is insufficient. Library media specialists also need to bring together the individual stories to expand understanding of what broad combinations of factors influence learning outcomes through library media programs. As library media specialists learn about the tools and resources that are available now and will be available in the future, they also need to reflect on the role of people in the learning process. No tool is anything without the people who make it work.

References and Notes

  1. Paul Solomon, "Children's Information Retrieval Behavior: A Case Analysis of an OPAC," Journal of the American Society for Information Science 44 (Jun. 1993): 245-64; Paul Solomon, "On the Dynamics of Information Systems Use: From Novice to ?" Proceedings of the 55th ASIS Annual Meeting 29 (Oct. 1992):162-70.
  2. Bruce R. Joyce and Elizabeth A. Joyce, "The Creation of Information Systems for Children," Interchange 1 (1970):1-12.
  3. Carol C. Kuhlthau, "Meeting the Information Needs of Children and Young Adults: Basing Library Media Programs on Developmental States," Journal of Youth Services in Libraries 1 (Fall 1988): 51-57.
  4. Leslie Edmonda, Paula Moore, and Kathleen M. Balcom, "The Effectiveness of an Online Catalog," School Library Journal 36(Oct. 1990): 29-32.
  5. A prototype science library catalog for children, which has been under development by Christine Borgman and associates at UCLA, employs a mouse for browsing access. It does not require skills beyond reading and recognition. See, for instance, Virginia A. Walter and Christine L. Borgman, "The Science Library Catalog: A Prototype Information Retrieval System for Children," Journal of Youth Services in Libraries 4 (Winter 1991): 159-66. Kid's Catalog, a product of CARL, Inc., is available commercially as a front end for some OPACs. It allows browsing, direct entry of queries, and interface tailoring. For further information, see Paula Busey and Tom Doerr, "Kid's Catalog: An Information Retrieval System for Children," Journal of Youth Services in Libraries 7 (Winter 1993): 77-84.
  6. Delia Neuman, "Naturalistic Inquiry and Computer-Based Instruction: Rationale, Procedures, and Potential," Educational Technology Research and Development 37 (Fall 1989): 39-51; Delia Neuman, "Learning Disabled Students' Interactions with Commercial Courseward," Educational Technology Research and Development 39 (Spring 1991): 31-49.
  7. Muriel Saville-Troike, The Ethnography of Communication: An Introduction (New York: Blackwell, 1989).
  8. Sharan B. Merriam, Case Study Research in Education: A Qualitative Approach (San Francisco: Jossey-Bass, 1988).
  9. Egon G. Guba, "Criteria for Assessing the Trustworthiness of Naturalistic Inquiries," Educational Communication and Technology Journal 29 (Spring 1981): 75-91.
  10. Hugh Mehan, Learning Lessons (Cambridge, Mass.: Harvard Univ. Pr., 1979).
  11. Tenata Tesch, Qualitative Research: Analysis Types and Software Tools (Philadelphia, Pa.: Falmer Pr., 1990).
  12. Guba, "Criteria for Assessing."
  13. In this school system, both serously disabled and gifted students are channeled to special centers. Therefore, the study school's student population did not include children at either extreme.
  14. Carol C. Kuhlthau, Seeking Meaning: A Process Approach to Library and Information Services (Norwood, N.J.: Ablex, 1993).
  15. Edmonds, Moore, and Balcom, "The Effectiveness of an Online Catalog."
  16. Shu-Hsien Chen, "A Study of High School Students' Online Catalog Searching Behavior," SLMQ (Fall 1993): 33-40.
  17. Delia Neuman, "Designing Databases as Tools for Higher-Level Learning: Insights from Instructional Systems Design," Educational Technology Research and Development 41 (Winter 1993): 25-46.
  18. Busey and Doerr, "Kid's Catalog."