Data-Driven Decision Making

To what extent does ALCTS use data-driven strategies in decision-making and planning?

Working Group Members: Andrew Hart, Reeta Sinha, Cheryl Kern-Simirenko

Background: The working group reviewed the 7 Measures of Success Survey results and the comments regarding data-driven strategies. This report focuses on responses that relate to ALCTS using data for organizational decision-making. We note, however, that a couple comments addressed the conduct of research on the areas of librarianship that ALCTS covers.

Areas of Concern Particularly Noted in Comments of Respondents

  • Lack of knowledge re: data collected by ALCTS
  • Lack of awareness whether ALCTS collects any data at all
  • Lack of communication from ALCTS to membership re: data collected
  • Lack of evidence that data is used for ALCTS decision-making
  • Desire to see data re: membership, evaluation of DGs and committee activities, and CE workshops/events

Some 48% of the comments deal with information about ALCTS decision-making and communication/sharing data. Members appear to be as concerned about their access to, and understanding of, this information as they are about data-driven decision-making. It might be beneficial to review these comments in the context of Item 1 (customer service) and Item 4 (dialog and engagement).

Positive Developments in Last Few Years

  • Use of online surveys
  • Implementation of the Strategic Planning database
  • Effective dissemination of ALCTS budget data by Executive Director

Next Steps

  • Inventory data routinely collected by ALCTS and/or ALA
  • Review current analysis and distribution of this data
  • Review/analyze use of data for ALCTS decision-making

In other words, what data is currently collected? Is it analyzed? Is it available to those who could apply it to decision-making? And is it actually used in the decision-making process?

  • Share the above information broadly within ALCTS
  • Determine a set of data to be distributed, at least annually, to the Board, to each division committee, and to each section executive committee

Pending the outcome of these next steps, further work may be warranted to determine needs for additional data collection and to promote the use of data in decision making.    

What might ALCTS do to excel at being data-driven in our decision-making?

(Comments received from December 2007 survey)

Sometimes we do this; sometimes we make decisions based on anecdotal evidence, individual (leader/squeaky wheel) preference. But do we always have time to compile data before making a decision? What data do we routinely collect? (Not much) Maybe we need an annual survey or other data-collection activity. For example, Consumers Union has an annual survey of products, services, etc. and they use this throughout the year to report on various products and services.

I suspect that our ability to do this is hampered by the fact that most of us are volunteers and have limited resources to devote. Would it be useful/possible to look to our teaching and research professionals (e.g. Library School professors, researchers in organizations like OCLC, etc.) for more partnering? I am unaware of how much we, as an ALA division, work to pose effective research questions or hypotheses to the research community, and cultivate them as partners in research endeavors.

My perception is that little in the way of data is collected, and what is collected is not shared -- either with the membership or with leaders. For example, simple counts of who attend programs at annual and midwinter might inform leaders of the topics that are of greatest interest. Compiled evaluations of programs and CE might guide those who develop these events to modify delivery mechanisms to better match member preferences. Few ALCTS leaders seem aware that ALCTS has a license with Survey Monkey -- and might be interested in using it to collect data on specific topics.

Data, in the service of decision making, is a tool. It should be used when and if appropriate---if it's the tool we need. Librarians, in general, are not good at designing useful research that incorporates data in a meaningful way.

I'm not familiar enough with current practice to make an informed comment on this one.

If we're doing, we need to communicate it better. Data needs to be systematically collected and made easily accessible to leaders at all levels. Systematic is a key word here: how often do we collect, who do we collect from, what info do we collect, how do we disaggregate data, when do we use multiple sources, what can we compare longitudinally? We should be using data from membership records, participation in events, surveys of members and non-members, financial records, website logs, etc. We should be looking at data from multiple sources simutaneously; what do members say they want versus what they actually use versus what we can afford.

Although I do believe we strive for continuous improvement, I'm not aware that we do much serious research or data collection to understand where we are and where we need to go. We could improve in this area.

Data? What data?

We do a terrible job of collecting and using data. Most decision-making is driven by opinion, heresay and anecdote.

I am on an ALCTS level Committee and Charles always gives us the data we need to make good decisions

We need more analysis of our members and what areas of ALCTS are their primary interests. Because folks can sign up for a number of sections...it is hard to identify those whose primary focus is a specific section. We need to survey the entire membership on programs, publications, what would be useful to them.

There is no data or metrics on the section's operations. we don't have evaluations or feedback for discussion groups and committee meetings so there is no way to validate their existence.

If data is collected and analyzed, the information needs to be shared and communicated with members and even more importantly, with leaders to improve planning and make planning more based on reality, rather than in desire

Collect data from members and non-members alike through focused surveys; set aside funding for staff who will mine the data and produce reports that are shared with those who need to engage in and respond to planning processes

I'm not aware of much rigorous research going on but maybe I can't find it on the website.

It would be interesting to know more about what types of data are relevant here, in order to help answer the question.

Don't bog down on developing databases for strategic planning purposes

Not apparent what data except budget data that is used in planning at the ALCTS level.

Use surveys and environmental scans to gain insight into best practices when formulating strategies

Good strategic plan in place; execution seems to be following along in decision making and planning

Stop making decisions based on politics

I am aware that Budget & Finance uses financial data in their deliberations. And I assume Membership reviews member numbers. I'm not sure what other data is available to ALCTS and what purpose it would serve. Perhaps the Board could consider this issue. I suspect that the Planning Database will become an important tool—maybe that is the best place to put our efforts.

The rate of cancellation and/or low turn-out for in-person CE suggests workshops are developed around good ideas rather than around knowledge of the market. A streamlined, easy method for doing market surveys (including price tolerance) would better support members AND positively impact the Association's revenue stream.

Populate the planning database!

More support for publications

Fiscally, yes; not sure otherwise

Move away from research studies which take too much time for publication and engage earnestly in more user surveys, feasibility studies, etc.

I don't have any information as to what drives the decision-making in ALCTS other than the need to have a revenue stream to continue functioning as an organization.

Tracking committee participation as well as program an discussion group attendance remains a reliable method of identifying what is happening in the present and mapping out the future. Centralized reporting and timely posting will be critical to continued success.

I do not truly know if we are or are not. it is something we should improve on though

There may be people who know this, but I don't recall seeing much evidence of data, other than financial reports, that is used for strategic and operational planning. Maybe financial data is the main data needed for decisions.

I think ALCTS does a good job in this area, but could be doing more.

ALCTS is working hard on this with the development of the new planning database. Hopefully, it will be a useful tool to assure that action items are thoughtfully identified and acted upon in a timely manner.

First by communicating more and widely with the membership on ALCTS decision-making and planning process.

Very little data available for committee use. Data would be a great help in planning programs, preconf. and workshops also on-line courses

Maybe I'm just not privy to it, but I don't see much data at all.

This kind of survey is a start and I know dollars and attendance at paid events are measures, but probably need to keep working on becoming more data-driven

Gather some before making decisions!