Content

Methodo

Design

The task

•          is meaningful, academic

•          relates to internal and external learning standards

•          uses tools of the expert

•          requires problem solving, decision making

•          culminates in a summative assessment based in whole or part, on display, presentation, and sharing of outcomes

The learner

•          uses prior knowledge, experiences

•          applies information to new situations

•          uses divergent, critical thinking

•          engages in a variety of tasks

•          has choices

•          uses ongoing formative assessments that offer opportunities for self-evaluation, peer review, and revision

•          has opportunities to work in groups

The design

��         includes clear expectations and outcomes

��         provides exemplars

��         identifies resources

��         offers assessment tools appropriate for the task

��         includes learners in the development of the assessment

��         includes input from learners and teachers for task evaluation and revision

What is a questionnaire?

A set of questions presented to a group (usually a sample rather than the entire group.) See “How Can I Do It Better Next Time?”

Is a questionnaire the best way for you to collect your data?

Do you have 5–10 questions that you would like to ask 10–20 people?

Do you want to compare how answers are the same or different for subsets in your sample (e.g., gender, age)

Do you want to quantify (count) responses to look for patterns?

Do you want to analyze respondents’ comments to look for patterns?

What kind of sample should you use?

1.Random sampling

a. Decide how many respondents you need for your sample. (Suggestion: 20% of the relevant universe.)   

b. Determine the relevant universe (population.) For example, all high-school students.

2. Select a sample where all candidates have the same chance of being selected. For example, you want to survey students to determine how they feel about the ban on smoking planned for 1999. You might choose 40 people by placing names of all students in a hat and picking 40 names.

3. Stratified random sampling

a. Decide how many respondents you need for your sample. (Suggestion: 20% of the relevant universe.)   

4. Select a sample where you can create subgroups with equal representation.

For example, you want to survey students by grade level to see how they feel about the ban on smoking and whether there are any patterns in the data that show relationships between grade level and attitude toward the smoking ban. You might choose 10 people from each grade level at random from class grade level lists.

5. Systematic sampling

a. Decide how many respondents you need for your sample. (Suggestion: 20% of the relevant universe.)   

b. Make a list of the names in the population, choose one at random, and count down every 4 or 5 names to choose those in the sample. For example, you want to survey all students to determine how they feel about the smoking ban. You might get a list of students, choose the 2nd or 3rd name as the first in the sample, and count down every 4th name until you get the number of names you need in your sample.

6.   Cluster sampling

a. Decide how many respondents you need for your sample. (Suggestion: 20% of the relevant universe.)                   

b. Divide the population into small groups (clusters) and randomly sample the clusters. Include every person or household in each sample cluster. For example, to survey a city, divide it into blocks, randomly sample the blocks, and include every house on each sampled block in the survey.

How do I write the questionnaire items?

1. Determine characteristics of your respondents that you want to identify (e.g., gender, age, nationality, experience). Provide a place at the top of the questionnaire to collect this information or use a Respondent Profile Sheet.

2. Decide if you are using closed or open questions, or both. Closed questions ask the respondent to choose from given answers; open questions ask the respondent to write in their replies.

3. Decide what you want to know. Use your research question and sub-questions. Write your questions.    

4. Decide how respondents will reply to the questions. For example, you might want to use a Likert rating scale where the respondent chooses a number from 1 to 4: 1 means strongly disagree; 2 means disagree; 3 means agree; 4 means strongly agree.

5. Write instructions explaining the rating scale at the top of the questionnaire.

6. Funnel your questions. Start with the general and move to the more specific.

How do I refine my questionnaire?

1. Pilot (test) your questionnaire using at least 3 respondents.

2. Make changes according to the results of the pilot questionnaire. For example, you may want to make a question clearer, add a question, or change your sample if all the respondents answered all the questions the same way. Use “Traps to Avoid” to determine whether any of your questions are faulty. Revise faulty questions.

3. Use Standards for Validity and Reliability to revise questions that do not meet these two standards.

4. Can your questionnaire be used without help? Edit your questionnaire for readability and clarity.

5. DO NOT ADD NEW QUESTIONS AFTER THE PILOT.

Traps to Avoid: What are common errors made when writing questions?

Double question: Do you walk to school or carry your lunch?

Wrong-choice question: Is your hair yellow, purple, green, or blue?

The kitchen-sink question: Please list all the schools you have attended, your teachers, and what grades you got.

The fuzzy-word question: Should middle-aged people live it up?

The cover-the-world question: What do you think of racism?

Jargon questions: Do you feel your cognitive and affective domains have been adequately addressed in school?

Dream questions: What kind of education would you like for your child?

Leading questions: Why are you happy at this school?

Hearsay questions: Do you think students are happy at this school?

Assumptions: Why did you choose this school?

What are the standards for validity and reliability?

Validity: Do your questions measure what you want them to measure?

Do they produce answers that accurately reflect the respondent’s beliefs, experiences, or situation?

Reliability: Do the questions and answers have consistency and reliability?

Consistency: Are the questions and answers consistent and repeatable? Will the same question asked of the same person in similar circumstances produce the                                      same answer? [Ed/Au: should this be “the same question asked of a different person”?]

Repeatability: If the same questions were asked in different ways, at different points, would the answers be the same?

How do I administer the questionnaire?

Get informed consent from each respondent.

You must decide on a method of distributing and collecting the questionnaires that will ensure that you get back as many as possible. Your return rate could be very low if you leave it to chance. Suggestion: if possible, stay with the respondents while they fill out the questionnaire. If this is not possible, send out twice the number of surveys that you need to be returned.

Stay in control of your sample: do not let anyone participate who has not been chosen for the sample.

How do I analyze the data?

a. Quantifying and categorizing (grouping) data

Count the responses for each question or category of questions. For example, if you used the Likert rating scale, count how many people answered 1, 2, 3, and 4 for each question.

Make a table or graph to display the data. Discuss the results in your paper and include the display.

Find the mean (average), the mode (the most frequent answer), or the median (the middle answer) for each question, or for questions you want to discuss more fully in your paper. These can also be displayed in a table.

b. Color coding and categorizing data

Look for patterns or trends (for example, words that have been repeated by several different respondents in open-ended questions) and highlight the words. Use different colored highlighters for different ideas or categories (groups) of data. Use your sub-questions for ideas for categories. Look for relationships between characteristics of the respondents (age, gender, nationality, experience) in the Profile and the responses given.

Discuss the trends or patterns you observed in the data in your paper. Use quotations as the display of data.

1. What suggestions do you have for improving the way the action research project was structured, presented, and implemented? (This question relates to what the MR should change in order to do this better next time.)

2.   If you could do your action research project over again, how would you do it differently? What changes would you make? Why?  (This question relates to what you would change in order to do this better next time.)

3. Would you use the action research methods you learned again? Why or Why not?

4. What other question(s) would you like to explore related to library instruction in future action research studies?

5.   What other areas of your practice would you like to study?

6.   If this action research project helped you to plan and implement instruction, how did it help? If it didn’t help please state why.

7. What was the most difficult aspect(s) of doing action research?

From Director to MR

From MR to Director

Totals

Number of e-mails

15

14

29

Total words

1,434

2,637

4,071

Categories

11 Logistics

3 Strategies

1 Evaluation

5 Logistics

8 Strategies

1 Evaluation

16

11

2

Research Stages

Elementary PRs

(3 SLMSs)

Middle PRs

(2 SLMSs)

High PRs

(2.5 SLMSs; 1 technology integrationist

Task Initiation/

Topic Selection

Developed topic-based curricular topic for unit of inquiry for students (penguins, black history) and on student skills (critical thinking)

13 e-mails/880 words

Developed topic based on what they wanted to know about their teaching: “What do we really need now regarding teaching note taking?”

5 e-mails/789 words

Developed topic based on students’ information-seeking behaviors (plagiarism, poor note taking, under-use of subscription databases, and improving student writing.)

9 e-mails/1,614 words

Stating the Research Question

Developed focus on   what they wanted to know about student learning (higher levels of thinking)

13 e-mails/880 words

Developed focus on student information skills (Which method of note taking will student choose when given a choice?)

5 e-mails/780 words

Developed focus on problems

(plagiarism, poor note-taking skills, under use of subscription databases) and solutions (technology and writing improvement).

10 e-mails/1,110 words

Source

Reasons for Use of Source

Sternberg

Skinner

ERIC

Data Collection

Find test for learning styles

Action research study design

Data collection (control group)

Data collection ideas

Bloom

Piaget

Data Analysis

Taxonomy of thinking skills

Stages of cognitive development

Grant Wiggins

Ross Todd

Kathy Schrock

Ruth Small

Vygotsky

Authentic Learning Task Design

Authentic assessments

Web evaluation

Web evaluation

Evaluation skills

Meta-cognition; social learning

Level/no. of e-mails/word count

Elementary /4/ 205

Middle /3/ 708

High /12/ 458

No. of e-mails/word count

19/1,371

Method of Analysis:

Look for...

What it means

Comparing (Similarities)

How are things alike?

Contrasting (Differences)

How are things different?

Constructing support

What is the support for the argument? What are the limitations of it?

Classifying/Labeling

(Categories)

How can I put things in groups?

What are the rules governing membership in these groups?

How can I name the groups?

Structural Analysis

(Main idea/supporting evidence)

What is the most central idea?

What is the evidence to support what you say about main idea?

Induction

What conclusions, generalizations can you make and what is the support for them?

Deduction

What is the proof that this must be true?

Chronology by time or stage

How can I arrange data in time order or by stages or steps?

Pros/Cons

What are the arguments in favor of and against a point of view?

Causes/Effects

What were the reasons and results of an event?

Problems/Solutions

Why doesn’t something work and how can we make it work?

Procedures/Experimental inquiry

What are the steps involved in doing something?

What do I observe? How can I describe, explain it?

Relationships

(Spatial, Human)

What does one thing have to do with another?

Analyses

How can the subject be divided into significant parts?