Hot Topics Discussion Group - Web Tools for Library Assessment: Focus on SurveyMonkey

sts

Sunday June 24, 2007 10:30 am-12:30 pm

Capitol Hilton, Federal Room, Washington DC

 

Minutes | Speakers | Handouts and Presentations | Book Recommendations | Other Links

Sponsored by STS Annual Hot Topics Discussion Group and the STS Assessment Committee
Co-chair: Maribeth Slebodnik
Moderator: Annie Zeidman-Karpinski
Recorder: Daria O. Carle
Speakers: Katherine O’Clair Arizona State University, Tempe, AZ
Marilyn Christianson Auburn University, AL
Bryna Coonin East Carolina University, Greenville, NC
Kari Zhe-Heimerman Le Moyne College, Syracuse, NY

   

Minutes

Introduction

  • SurveyMonkey (SM) was contacted but did not offer a speaker.
  • Overview of SurveyMonkey and its use related to assessment.
  • Speakers will discuss projects using SM, its advantages and disadvantages, how info was used.

Speaker: Katherine Clair

  • Used SM for course evaluations, pre- and post-test for instruction, surveys for state library association, faculty needs assessment for electronic journals.
  • Setting up survey the first time was very time consuming, but got easier with use.
  • Used blank survey, but templates are available.
  • Recommend coming up with questions first, then conduct pre-test first to catch mistakes or confusing wording.
  • SM allows different types of questions (e.g., essay, matrix).
  • Basic SM is free and offers privacy, but has less functionality.
  • Subscription SM is customizable, with more download options (e.g., PDF, Excel).
  • Other tools used: Blackboard (problem: identifies students, whereas SM is anonymous), Zoomerang (more expensive)

Speaker: Kari Zhe-Heimerman

  • Began using free SM as result of LibQual.
  • First use of SM for class survey of library summer hours; resulted in change of weekend hours.
  • Second surveyed library users for their favorite book during Library Week. Responses posted on LibraryThing.
  • Analysis of results: simple percentages, response count.
  • Likes: ability to view responses as they came in.
  • Dislikes: limitations of free SM (e.g., 10 question limit; cannot download results to PDF; responses supposedly limited to 100, but got 149 for second survey).

Speaker: Marilyn Christianson

  • See handout (6 pages).
  • Used SM for a variety of types and levels of surveys.
  • Provides SM link in email to potential responders and/or library web page, depending on survey.
  • Another option: TLT Group (also offers free survey to members).
  • New SM upgrade offers flexibility in size of response box for open ended questions; choice of fonts and colors.
  • Exports results to Excel.
  • SM has option to customize, but cautions that it takes more time.
  • Using logos and photos is getting easier with new version.

Speaker: Bryna Coonin

  • See handout (Reasons for Using Tools Like Survey Monkey)
  • Science research not typically survey-based.
  • Discussed other survey tools: Perseus (high end research tool); Zoomerang.
  • Discussed branching and piping ability in survey tools.
  • Likes: not cluttered; create your own pop-ups; filters answers by category
  • Dislikes: no download to Word, clunky, no piping.

Questions and Answers

Q: Does SM provide info about constructing proper questions?
A. No. Look in Social Science Research Methods or similar textbook for examples of survey questions. Check survey design by testing ahead to be sure questions are clear. Test not just questions, but what potential results will be to make sure results will answer the point of the survey.

Q. Is SM anonymous?
A. Free version is anonymous, and new SM upgrade can be set up to be anonymous.

Q. How to keep survey recipients anonymous?
A. Subscription SM will send for a fee; otherwise use blind cc: in regular email.

Q. What about sending reminders to complete the survey?
A. Subscription SM tracks who has responded and sends reminder.

Q. Are there other good resources on survey design?
A. Besides Social Science Research Methods mentioned above, investigate: CRC Handbook of Parametric and Non-Parametric Design? Survey design assistance through teaching excellence group at your institution. Most institutions also have an Institutional Review Board (IRB) or something similar. Check with social sciences faculty who may have expertise in survey design.

Q. Could SM be used as a suggestion box?
A. Yes. There are five ways to export data. Other suggestions: Limit use of comment boxes. Keep comments limited, although there are social sciences programs that quantify comments. Comments can be downloaded separately.

Q. What was steepest part of SM learning curve?
A. Survey design (there are 20 different types). Analysis of results (many options here also). Terminology to define questions.

Q. What future projects using SM do you have planned?
A. Survey large classes to determine best time for scheduling library instruction sessions. Information literacy (pre- and post-test for biology students) Encourage faculty on tenure-track to use as a support tool. Evaluation of library instruction.

Summary

  • Ease of use Templates or not; text; size of boxes; browser compatibility
  • Alternatives Zoomerang; TLT; Perseus; Blackboard
  • Cost and functionality Branding and piping; pop-ups; filtering responses; spell checking
  • Analyzing results Easy to share; put up and take down; downloading options
  • Survey design Harder to create proper question; consult social sciences sources
  • Confidentiality Keep in mind for responses and for keeping track of surveys; evaluations; soliciting comments
  • Limitations of free account Number of questions, replies; downloading options
  • Reminders to complete surveys Email or use SM reminder feature
  • Archiving surveys SM keeps surveys in your account; ability to restore to reuse again
  • Future uses Assess skills learned; evaluation tool; encourage faculty to use SM
  • Don’t overuse! Beware of survey fatigue!
   

Speakers

Katherine O’Clair (Arizona State, life sciences librarian): First used SurveyMonkey 2.5 years ago to survey members of local chapter of SLA. First time creating survey was time-consuming. WYSIWIG, can ask a variety of questions. Arizona State has a paid account ($200/year) used to conduct surveys such as faculty journal needs assessment as well as signups for luncheons, pre-post testing of kinesiology students, etc. So far has used about 20 times. Data downloads as PDF or Excel. Cheapter than Zoomerang.

Kari Zhe-Heimerman (Le Moynes College, Syracuse): See handout. Started using SM as result of LibQual, which was cumbersome and produced ambiguous results. Use free SM account w/o extensive knowledge of survey design. Emailed survey to Physicians Assistant students who complained about library hours and got 39 responses! Also embedded a survey in library website asking visitors to “tell us about your favorite book” and got 149 answers! Shared titles via LibraryThing. Analysis tool is easy to use. Can see responses as they come in.

Marilyn Christianson (Auburn Univ. Libraries, biological sciences, forestry and wildlife sciences, and mathematics liaison): See handout. Used SM to conduct survey at library open house. SM allows for variety of fonts and simple HTML tags to dictate color of text. However pictures are hard to position.

Bryna Coonin (Joyner Library at East Carolina University): See handout. Check if your campus already has SM or other university-wide survey tool. Need to get clearance from your institution for doing research on subjects, sometimes need to get training as well. Raised interesting issues such as wording of survey questions need to differ for American vs. British subjects. Some survey tools (not SM) allow for branching/piping, which can be used to skip irrelevant questions and make the survey look like it was designed just for one individual.

   

Handouts and Presentations

Kari Zhe-Heimerman's  handout (pdf 68kb) and powerpoint of the  presentation (ppt 901kb)

Marilyn Christianson handout in 3 files:  part 1 (pdf 141kb), part 2 (pdf 132kb), part 3 (pdf 231kb)

Bryna Coonin's  handout (pdf 87kb)

   

Book Recommendations

Survey research methods  by Earl Babbie. [8] Main Author: Babbie, Earl R. Edition: 2nd ed. Publisher: Belmont, Calif. : Wadsworth Pub. Co., c1990.

The Practice of Social Research (Hardcover) by Earl R. Babbie Wadsworth Publishing; 11 edition (February 1, 2006)

Handbook of Parametric and Nonparametric Statistical Procedures , Fourth Edition David J. Sheskin Western Connecticut State University, Danbury, USA. ISBN: 9781584888147 ISBN10: 1584888148 Publication Date: 1/19/2007 Number of Pages: 1736 (!)

  • Presents 160 statistical procedures and virtually all theoretical and practical issues relevant to statistics
  • Provides decision tables for selecting the appropriate statistical test
  • Features a standardized organizational format for each test
  • Discusses the use of SPSS in conducting and interpreting multivariate and log-linear analysis
  • Shows when two or more procedures produce equivalent results
  • Uses small sample sizes and integer numbers in all examples, allowing for easy understanding of the computations
   

Other Links

Presentations from the ACRL STS Assessment pre-conference Annual in New Orleans, LA 2006
Lots of useful tips for doing assessment of web pages, focus groups, collections, instruction and more

More on  SurveyMonkey pricing

Assessing the session
And, what you've been waiting for, a  survey for the session using Survey Monkey