Using Project Outcome with Digital Literacy Programs and Online Training Services to Improve Programming, Inform Resource Investment, and Sustain Partnership
Thomas Crane Public Library (TCPL) uses Project Outcome surveys to measure its impact and ensure that its programs are always improving and meeting the needs of an increasingly diverse community.
- Library Name: Thomas Crane Public Library
- Library Location: Quincy, MA
- Library Size: Medium-large (4 branches, service area: 93,000)
- Immediate Surveys Used: Digital Learning, Education/Lifelong Learning
Data-Driven Changes Strengthen Digital Literacy Program, and Evidence of Patron Outcomes Helps Improve and Sustain an Important Community Partnership
Saheli is a non-profit partner of TCPL that primarily serves South Asian women and families. At the library, Saheli provides digital literacy training to diverse groups of community members. Participants in the training complete Project Outcome surveys either midway through the course or at the end. One example of data-driven change is that after reviewing mid-course survey responses, the Saheli trainer added social media lessons to the training. A Saheli trainer explained, “Quincy was new for us. And we really wanted to… get in touch with the South Asian population… try to find out their needs, and how to incorporate them with the mainstream population... [There are] a lot of families [who] want someplace to go out and connect with other people. When they travel, they tend to travel… in herds, they want to be in places where there are other South Asians they can talk to. But there's always this gap of not being able to connect with other people outside, and... that barrier opens up when you're in a classroom like this, where you're talking about not just academic or work-related stuff… The last year and a half, we did social media based on the surveys… and people started inviting each other to… online groups, and started connecting that way!” The trainer also learned that one participant started her own business using new connections she made through the class.
Saheli made other data-driven improvements to the program, including changing the time of the class, adding lab hours, and clarifying program expectations at the beginning of each class, which supported program retention. “Project Outcome lets us be real about the experience, and tweak it in real time, so we can improve it. Otherwise, we would have no other knowledge, or means to improve,” said a library staff person that works with Saheli. A Saheli trainer said Project Outcome surveys tell her “a lot about what the community needs, what each person's needs are, what else we could bring in, as far as technology is concerned, which is important as technology is changing pretty much every day now. The amazing things about these classes is that people from so many different levels and backgrounds come in, and they sit down together in the class, and have a conversation.”
The positive outcomes reported by Saheli’s digital literacy participants helped TCPL decide to continue working with Saheli and successfully pursue funding for the continued partnership. As explained by a library leader, “It was information from Project Outcome that gave us the confidence to ask the Friends [to fund the program], and to justify asking for that money.”
Evidence of Patron Outcomes from Online Training Services Support Library Decision-Making about Resource Investment
Project Outcome survey results also informed TCPL’s decision to continue their subscription to Lynda, an online training service that patrons can access through the library. TCPL uses Lynda’s metrics to understand how often patrons use the service, but TCPL felt the investment in the subscription also needed to generate meaningful outcomes. Using email addresses Lynda users provided when they registered for a Lynda account, TCPL sent users invitations to take the Project Outcome survey online. The results showed that patrons value the service and it helped them to develop their skills and knowledge across a range of topics.
Factors That Supported Successful Use of Project Outcome
TCPL’s successful use of Project Outcome surveys started with leadership support. The library’s assistant director appreciates the value of customer feedback and outcome assessment, and works to build a culture within the library that does the same. He shared, “I've actually designed a whole bunch of surveys just on my own… [to get] anonymous feedback most often, on different staff trainings we've done. And I've found that to be really valuable and… conducive to the kind of general culture of…‘This is what we've heard, and this how we're going to improve the next step.’ I've found it really useful just to take this framework to apply in a lot of different arenas. And even though it's not strictly Project Outcome, it's certainly inspired by Project Outcome. And I hope that that adds, or creates an environment that is conducive and supportive of doing more outcome-based assessment.” Further, he provides material support for staff use of Project Outcome by creating the surveys, helping to implement them, and discussing survey results. TCPL also attributes success to using a staged approach, which allowed staff to learn from successive survey administrations with several types of programs.
TCPL will use Project Outcome surveys to assess progress on the goals in their new strategic plan during the next five years. One priority in the new strategic plan is to inspire curiosity and lifelong learning, and Project Outcome surveys will enable them to track the percentage of parents and caregivers who say they learned something new that they can share with their children at early literacy programs. A library staff leader noted, “Project Outcome is really just exploding here in the next several years.”
Prepared by ORS Impact, September 2017