Accelerated Reader:

September/October 2007

Once Again, Evidence Lacking

Stephen Krashen

In previous papers (Krashen, 2003, 2005) I concluded that research on Accelerated Reader (henceforth AR) was inconclusive, that AR had not yet been properly evaluated.

My logic was as follows: AR consists of four components: (1) it provides lots of books; (2) it allots lots of time to read – AR recommends one hour per day, far more than is done in sustained silent reading; (3) it includes tests, based on low level facts of the book the child read; (4) it awards prizes, based on the number of points scored on the quizzes.

I argued that we already know that (1) and (2), books and time to read, will result in substantial gains in reading (Krashen, 2004). What we do not know is whether (3), tests, and (4), rewards, are worthwhile. Thus, a proper study should compare children in a full AR program, with elements 1-4, and children who receive only (1) and (2). Unfortunately, this comparison has not been done: Studies compare AR with business as usual, “traditional” language arts programs. We thus have no idea whether elements (3) and (4), the tests and prizes, add anything.

I report here on some current research on AR. This research does not solve the problem of having the proper comparison group, and it includes some data showing that AR results in little or no gains over regular instruction. The studies have attracted attention because one of them is the basis for the What Works Clearinghouse’s apparent endorsement of Accelerated Reading, which the AR parent company, Renaissance Learning, announced both in a press release and on their website (renlearn.com).

The study, Ross, Nunnery, and Goldfeder (2004), consists of two separate sub-studies, one of which was republished in a slightly altered form as Nunnery, Ross, and McDonald (2006). Ross et. al. reported that children in an Accelerated Reader program outperformed comparisons in grades K through 3, but differences between the groups in two studies of children in the upper grades were small or non-existent (table 1), and not statistically significant. Table 1 presents effect sizes, which indicate the strength of a treatment.

Table 1: Effect sizes: Ross et. al.

Grade	Study 1	Study 2
K	0.71
1	0.36
2	0.25
3	0.33	0.36
4	-0.01	0.16
5	0.11	0.09
6	0.14	0.09

Cohen (1988) has recommended the following benchmarks for the interpretation of effect sizes: An effect size of .2 is considered small .6 moderate, and .8 or more considered large.  

Renaissance Learning posted a description of the 2006 paper (study 2 of the 2004 paper) on its website (renlearn.com), claiming that its results support the use of AR. They interpret the effect sizes obtained in the study more positively than the authors of the study do, and far more positively than Cohen’s benchmarks warrant (table 2).

Table 2. Interpretation of effect sizes in study 2

Grade	Effect size	Renlearn	Authors	Cohen
3	0.36	“large”	“strong”	between small and moderate
4	0.16	“moderate”	“small to moderate”	less than small
5,6	.09, .09	“small, positive”	“no effect”	less than small

Cohen’s benchmarks: .2 = small; .5 = medium; .8 large
Authors’ quotes from Nunnery et. at.,. 2006 .

Most readers of their website, unfortunately, will not consult the original paper, nor are most readers aware of how educational researchers typically interpret effect sizes.

In addition to this problem of interpretation of the results, the Ross et. al. study repeats the problems of earlier studies and adds a few more.

The Comparison Groups

Ross, Nunnery, and Goldfeder (2004) do not include descriptions of the treatment received by the comparison groups. In the version published as Nunnery et. al. (2006), which covered a second study of AR in grades 3,4,5, and 6, we are informed that comparisons had “sustained silent reading” but we are not told how much. We are also told that a “commercially available basal reading program” (Ross et al, 2006, p. 7) was used and students participated in “small- and whole-group activities” (p. 7). It is thus doubtful that comparisons put in anywhere near as much time reading as the experimental group did.

(SSR is usually 10 to 20 minutes per day. Ross et. al. report that 80% of the AR teachers said they devoted at least 45 minutes per day to reading, and 95% said reading time was at least 30 minutes).

What is “AR” in Grades K-3?

Ross et. al. (2004) state that most of the reading activity in grades K, 1, 2 and 3 in the AR group was not independent reading: “The focus in the younger grades (K-3) tends to be on time spent reading to (T) and with (W) students; students then transition to an increasing emphasis on independent reading (I)” (p. 10).

The Only Large Effect = Kindergarten (61 children)

In these studies, AR only had a clearly positive effect in grades K through 3, but only had a substantial effect in kindergarten. The kindergarten group was also the smallest of all the groups tested, with only 61 children in the experimental group taking both the pre and posttest.

Incomplete Version of AR

In this study, AR did not include rewards, only tests.

Summary

The Ross et. al. study produced positive results only in the lower grades, where students did little independent reading. The strongest results were in kindergarten, with only 61 children in the AR group.

The study was not the proper comparison needed to test the effectiveness of AR. The treatment did not include rewards, thus, it included only elements (1), (2) and (3), as described earlier. There was no description of the comparison group for the lower grades, and comparisons in the upper grades did much less free reading; thus, element (2), time to read, was not the same for both groups.

Is AR Harmful?

In general, more reading time results in greater gains in literacy (Krashen, 2004). Despite having more time to read, however, AR classes in the upper grades did not do better than comparisons. It is possible that the use of AR tests emphasizing low-level, literal facts, focused students on retaining small details of the books they read in order to get higher scores on tests. This means shallower involvement in reading, less of a chance of a student getting “lost in the book” (Nell, 1988), or entering the “Reading Zone” (Atwell, 2007), the state of mind that readers are in when they are absorbed in a text. This state may be optimal for language acquisition and literacy development (Krashen, 2007). A shorter time reading, but spent in the Reading Zone, may be more effective than more reading outside the Zone.

The Clearinghouse and AR

As noted earlier, the AR website now claims that Accelerated Reader has been given the “green light” by the National What Works Clearinghouse. This is quite an exaggeration.

Unmentioned in the AR website or in its press release is the fact that the Clearinghouse reviewed 35 studies of Accelerated Reader, and dismissed 34 of them as not meeting its standards in methodology. The only one accepted by the Clearinghouse is the first half of the first study in Ross, Nunnery, and Goldfeder (2004). This is, as we have seen, a study in which AR children did little if any independent reading, where strong gains only appeared with 61 children in Kindergarten, and in which no information about the comparison group was provided.

The Clearinghouse did not mention that AR had little or no impact in grades 4,5, and 6, as its goal was only to review research on children in grades K to 3.

What Can We Conclude?

There is still no clear data supporting the test and prize aspects of Accelerated Reader. According to the What Works Clearinghouse , AR software, without books, costs a school $4 per student per year, with a one-time fee of $1,499. The average school library in the US spends about $12 per year on books (calculated from 2004-5 data in Schontz and Farmer, 2007). Schools, in other words, are spending the equivalent of one-third of their book budget on software that has not produced any concrete evidence that it helps children.

Works Cited

Atwell, Nancy. 2007. The Reading Zone. New York: Scholastic.

Cohen, Jacob. 1988. Statistical Power for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Krashen, Stephen. 2003. The (Lack of) Experimental Evidence Supporting the Use of Accelerated Reader. Journal of Children’s Literature 29, no. 2: 9, 16-30.

Krashen, Stephen. 2004. The Power of Reading. Westport, CONN: Libraries Unlimited and Portsmouth, NH: Heinemann.

Krashen, Stephen. 2005. “Accelerated Reader: Evidence Still Lacking.” Knowledge Quest 33, no. 3:48-49.

Krashen, Stephen. (2007). “Free Reading in School: Three Hypotheses.” Paper presented at CELC conference, Singapore, May, 2007.

Nell, Victor. 1998. Lost in a Book. New Haven: Yale University Press.

Nunnery John., Ross, Steven., & McDonald Aaron. (2006). “A Randomized Experimental Evaluation of the Impact of Accelerated Reader/Reading Renaissance Implementation on Reading Achievement in Grades 3 to 6,” Journal of Education for Students Placed at Risk 11, no. 1: 1–18.

Ross, Steven, Nunnery, John, & Goldfeder, Elizabth. (2004). A Randomized Experiment on the Effects of Accelerated Reader/Reading Renaissance in an Urban School District: Preliminary Evaluation Report. Memphis, TN: The University of Memphis, Center for Research in Educational Policy.

Schontz, Marilyn. and Farmer, Leslie. 2007. “The SLJ Spending Survey.” School Library Journal 53, no. 1: 44-51.

---

Stephen Krashen      is professor emeritus at the University of Southern California and is a linguist, educational researcher, and activist. He has has contributed to the fields of second language aquisition , bilingual education and reading.