Home Publications College & Research Libraries News Back Issues: 2007 July/August Information literacy-related student behaviors

Information literacy-related student behaviors: Results from the NSSE items

C&RL News, July/August 2007
Vol. 68, No. 7

by Bonnie Gratch-Lindauer

In the November 2005 C&RL News, the Institute for Information Literacy’s College Students Surveys Project Group¹ reported  their activities and progress in developing information literacy-related items to be included on the 2006 National Survey of Student Engagement (NSSE) as experimental items. Ten items were included and administered to 12,044 students at 33 institutions on the 2006 NSSE. As experimental items, the purpose was to test them on a wide array of institutions and students to determine if the questions were good and what lessons may be learned from them.

Frequency data were provided in late summer 2006, and the analysis and correlation findings were released to the Project Group members just in time for the 2007 ALA Midwinter Meeting in Seattle, where they were able to discuss the nine tables of data, and observations made by Robert Gonyea (associate director and project manager at the Indiana University Center for Postsecondary Research) and formulate follow-up questions. After receiving answers to the follow-up questions, the group feels ready to share these findings. First, though, a brief review of the project group’s activities and purpose for those unfamiliar with their work.

Project background
The project grew out of interest from some ACRL members to study the national college student engagement surveys for items related to information literacy. The charge evolved to include an analysis of seven national standardized college student surveys. After initial investigation of the seven surveys, the project group decided to concentrate on one survey—NSSE—examining it in depth. Survey items on NSSE “represent empirically confirmed ‘good practices’ in undergraduate education. That is, they reflect behaviors by students and institutions that are associated with desired outcomes of college.”² At the 2005 ALA Midwinter Meeting the Project Group’s discussion centered on NSSE director George Kuh’s suggestion to focus item development work on student behaviors that contribute to what we define as information literacy, in addition to student interactions with librarians or their library experiences. Since there was plenty of time to prepare the items for the 2006 NSSE, the project group members decided to seek broader input.

This broader input was obtained from a six-month adapted Delphi process to gather evidence from a polling of library and information science educators and practitioners. This resulted in a ranked list of behaviors and activities that was reviewed at the project group’s summer meeting. Members reviewed the findings from the field along with the items then on the 2005 NSSE to determine the final items for submission. The resulting items were reviewed and endorsed by the Executive Committee of the Institute for Information Literacy and the ACRL Executive Board. In mid-August six new items were submitted to the NSSE staff for consideration. After NSSE staff review and revisions, four items were used on the 2006 NSSE for analysis (see sidebar above).

Findings
The findings are very encouraging and overall support modest to high significant positive relationships between the two information literacy scales and eight scales derived from NSSE items, particularly among seniors with gains in practical competence and general education. Of course, none of these findings imply causality. However, as explained by Gonyea:

The correlations between the information literacy scales and the other NSSE measures are as good or nearly as good as other scales on NSSE. What this indicates is that all these behaviors and perceptions go together, as roughly the same students that use the campus library resources actively also report that they are “deep learners,” “collaborative learners,” and so on. The difficult question is to try to understand the unique contributions of engagement with learning information literacy and certain outcomes. In other words, if I were to put deep learning, active and collaborative learning, student-faculty interaction into a regression model and control for them would “active learning in information literacy” still have a strong significant contribution?³

The findings were reported on nine tables, which can only be briefly described here.  The sidebar above lists the information literacy-related items, which can be consulted when specific items are referenced in this article.⁴ Table 1, “Information Literacy Item Frequencies by Class Rank,”  reports the frequency with which first-year students and seniors engaged in various activities. Perhaps as expected, seniors report doing these activities a bit more often than first-year students and also report greater gains related to information literacy.

Project group members were a bit disappointed, however, with the lower percentages for use of online tutorials and noted the need to consider revising the wording of that item, perhaps even consolidating it into the fourth item. A revised item dealing with instruction and online tutorials that would connect more directly with course work might be worded as:

Which of the following have you done or do you plan to do before you graduate from your institution?
a) Participate in a research skills instructional session conducted by a librarian or complete an online research skills tutorial connected to your course work.

Table 2, “Information Literacy Scales—Reliability Statistics and Component Items,” reports the items that were used to construct the two information literacy scales, as well as the reliability statistics for each scale. The Active Learning in Information Literacy scale consists of the first three items listed in Table 1 (see sidebar) and the Institutional Emphasis and Contributions in Information Literacy scale is composed of items six through ten. The items asking about participation in an instructional session and in an online library tutorial were not included in these scales because they did not correlate well with the first three items.

Table 3, “Information Literacy Scales—Descriptive Statistics by Class Rank,” provides the descriptive statistics for each scale, comparing first-year students to seniors. On the Active Learning in Information Literacy scale, where a mean of 2 is equivalent to an average response of “sometimes” and a mean of 3 to “often,” the mean for first year students is 2.3 and for seniors 2.5. On the Institutional Emphasis and Contributions in Information Literacy scale, the mean of 3 for first-year students and 3.1 for seniors is essentially equivalent to an average response of “quite a bit.”

Table 4 shows the basic relationship between the two IL scales and eight other NSSE scales, including four of the five benchmarks of effective educational practice, a “deep learning” scale, and three scales that measure students’ self-reported gains in knowledge, skills, and personal development. The results demonstrate modest to high positive significant correlations, which means “as scores increase in the information literacy scales they also increase in benchmarks and gains.”⁵

The four NSSE benchmarks are: 1) level of academic challenge, 2) active and collaborative learning, 3) student interactions with faculty members; and 4) supportive campus environment.

The Deep Learning scale is composed of twelve items:
• worked on a paper or project that required integrating ideas or information from various sources,

• included diverse perspectives (different races, religions, genders, political beliefs, etc.) in class discussions or writing assignments,

• put together ideas or concepts from different courses when completing assignments or during class discussions,

• discussed ideas from your readings or classes with faculty members outside of class,

• discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.),

• synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships,

• analyzing the basic elements of an idea, experience, or theory, such as examining a particular case or situation in depth and considering its components,

• making judgments about the value of information, arguments, or methods, such as examining how others gathered and interpreted data and assessing the soundness of their conclusions,

• applying theories or concepts to practical problems or in new situations,

• examined the strengths and weaknesses of your own views on a topic or issue,

• tried to better understand someone else’s views by imagining how an issue looks from his or her perspective, and

• learned something that changed the way you understand an issue or concept.

Table 4 shows that the highest positive correlations between the Institutional Emphasis and Contributions in Information Literacy Scale are with the NSSE Gains in General Education scale (.67 for seniors) and the Gains in Practical Competency scale (.63 for seniors). The Gains in Practical Competency scale is composed of five items:

• acquiring job or work-related knowledge and skills,

• working effectively with others,

• using computing and information technology,

• analyzing quantitative problems, and

• solving complex real-world problems. 

The Gains in General Education scale is composed of four items:

• writing clearly and effectively,

• speaking clearly and effectively,

• acquiring a broad general education, and

• thinking critically and analytically.

The Gains in Personal and Social Development scale is composed of seven items:

• developing a personal code of values and ethics,

• understanding yourself,

• understanding people of other racial and ethnic backgrounds,

• voting in local, state, or national elections,

• learning effectively on your own,

• contributing to the welfare of your community, and

• developing a deepened sense of spirituality.

Table 5, “Frequencies by Major for Seniors,” illustrates that some majors seem to have more frequent engagement or higher ratings. For example, the first two items show that arts and humanities majors are more actively engaged with the library, but business majors are less engaged. For the last item, which asks the extent to which their experience at their institution has contributed to their knowledge, skills, and personal development in the ethical use of information sources in academic work (proper citation use, not plagiarizing, etc.), the seniors in the social sciences display the highest percentage of “quite a bit” and “very much” responses.

Table 6, “Information Literacy Item Frequencies by Living Arrangement (First-Year Students Only),” confirms the expectation that students living on campus more often use library resources, and Table 7, “Information Literacy Item Frequencies by Enrollment Status and Class Rank,” reveals that full-time students are more engaged with library services and resources than part-time students. 

Table 8 ,“Information Literacy Item Frequencies by Gender and Class Rank,” shows only slightly higher scores for females, but nothing really substantial. Two interesting exceptions are for seniors. The data for the third item—using Web-based resources—show that females are 12 percent more likely to do this very often. The last item shows a higher percentage of senior females (51 percent marked “very much” compared to 42 percent males), who reported that their experience at their institution contributed to their knowledge, skills, and personal development in the ethical use of information sources in academic work.

Table 9, “Information Literacy Item Frequencies by Race/Ethnic Status and Class Rank,” illustrates that African American and Latino students report higher levels of engagement on the first three items than Whites. One statistic noted in the observations made by Gonyea is that 31 percent of the African-American seniors marked “often” or “very often” to the question about how often they asked a librarian for help (in person, e-mail, chat, etc). Totaling percentages for “often” and “very often” results in 31 percent for African-Americans and 18 percent for Whites, a fairly sizeable difference.

Next steps
What lies ahead for these information literacy and library-use related items? NSSE staff have not yet decided, but Gonyea states that they will certainly keep them on the table when looking at new versions of the NSSE or NSSE modules. He encourages librarians to consider including these items on their local administration of the NSSE. 

He also explained that “due to constraints on the length of the NSSE instrument, it’s unlikely that all the info lit items would be incorporated on the core NSSE instrument. However, every four years or so we plan to do a thorough review of the items . . .” and that “some of the information literacy items could be considered for inclusion on the core NSSE instrument.⁶ 

Another option that NSSE staff are discussing is “the feasibility and utility of a modular approach by which additional survey items, tested and robust, could be selected by institutions and/or consortia to be included with their NSSE administration. This is also a possibility for the information literacy items.”⁷ Perhaps the final option is to do further testing of these items by editing them somewhat and running them again as experimental items in 2008, possibly including a regression analysis of several benchmark scales with the “active learning in information literacy” scale. There may even be an opportunity to work with institutional colleagues. As Gonyea mentioned, “I’m working now with the writing-across-the-curriculum (writing program administrators), who are interested in testing some experimental items in 2008. There may be a connection to your work.”⁸ The project group is interested in hearing your comments.

You may also want to request that these items be included in an upcoming NSSE survey at your institution. A subgroup of community college members from the project team is currently working to identify items for possible inclusion on the Community College Survey of Student Engagement.

Notes 
1. Current Project Group members are Bonnie Gratch Lindauer, chair; Lisa Janicke Hinchliffe; Kwasi Sarkodie-Mensah; Polly Boruff-Jones; Margit Watts; Scottie Cochrane; Ann Roselle; Troy Swanson; Ellen Sutton; and MaryAnn Sheble.

2. “NSSE Facts,” nsse.iub.edu/html/quick_facts.cfm. The NSSE Web site provides a wealth of information and reports, including “Accreditation Toolkits,” which map NSSE items to specific regional accreditation standards.

3. From a February 8, 2007, e-mail with Robert Gonyea.

4. If you would like a copy of the nine tables, request them from Bonnie Gratch Lindauer, bgratch@ccsf.edu.

5. “ILT Summary” attachment to January 17, 2007, e-mail from Gonyea. This article’s “Findings” section is based almost entirely on this document of his comments and observations of the nine tables.

6. February 8, 2007, e-mail with Gonyea.

7. January. 27, 2006, e-mail with Gonyea.

8. February 8, 2007, e-mail with Gonyea.

Bonnie Gratch-Lindauer is coordinator of library instructional services/information competency at the City College of San Francisco, e-mail: bgratch@ccsf.edu

© 2007 Bonnie Gratch-Lindauer