Hi Tom, 

I want to comment on this because this study was done in a school using EBLI (teaching and testing done by their staff) and reported on my web site.  The 9 year gain was on Word Attack (pseudo words) and is not atypical on this subtest with middle/high school students at my center and in schools using EBLI.  The gains on this subtest is typically about twice as high as the gains on WID.  The gains of these same students in other areas (Word ID, Vocab, and Comprehension) were 2 - 3 1/2 grade level gains for the same group of students.  If a student has little or no knowledge of the structure of the English alphabetic system and few if any phonemic awareness skills, scores at a 2nd - 3rd grade level initially are common (for all ages we see).  So, being able to read pseudo words (typically using some form of the Woodcock-Johnson tests that are used in a lot of schools I work in to qualify students for Special Ed) does not mean they have learned 5000 new words; it just means that they can actually use the English alphabetic code to decode words.  If they can read pseudo words then we know they haven't previously memorized these words (because they've never seen them!).  With Word ID (real words) most students have memorized some or many of the words so it's not as telling of how they manage the code.  Of course, it's also not going to tell us explicitly about their comprehension, just a necessary piece that leads to comprehenison (reading words with perfect or near perfect accuracy).  



While these type of results even continue to surprise me, these students frequently no longer qualify for special ed and go on to do very well in school with no further assistance.  Of course, many are skeptical about the gains and this is something I hear frequently in my trainings.  I encourage trainees to not believe a word I say, go back and use EBLI and be sure to collect data, and see for themselves.  So many times, we've been asked to 'show me your research' and then after they see it 'We don't like the tests that were used' or other resistant remarks.  To me, if the students are reading, excited to read, and having fun reading as well as managing their work in school well and easily, and their behavior and self esteem has done a turn around, that to me is the main goal.  However, I so would love someone to tell me what tests they would like to see be used to show gains.  In my fantasy world, we'd have pre and post PET scans avaiable to show the changes in the brain!  This is really an area of frustration for me and for teachers who are using EBLI and are also getting these type of results.  I think the perception is that this is not possible.  What would you do if you were getting these results?  I would so much appreciate your input.  



I also want to share that these types of gains are often overwhelming to adults (usually 25 and older). They typically don't think they can really learn to read and when they do, and they're moving along quickly, they have significant psychological difficulties.  What will this mean for my life?  Will I have to get a different job?  Will my husband be angry because now I can read and he still can't?  Issues like that.  




Best, 

Nora 



----- Original Message ----- 
From: tsticht at znet.com 
To: learningdisabilities at nifl.gov 
Sent: Wednesday, October 28, 2009 5:13:35 PM GMT -05:00 US/Canada Eastern 
Subject: [LearningDisabilities 4190]  What does a grade level gain mean? 

All: The extraordinary gains reported between pre and post tests following 
just over 5 hours of instruction need to be regarded with a high level of 
scepticism. What could it possibly mean for someone to make over 9 years 
gain in just over 5 hours of instruction? Some research indicates the 
normal, typical child in school in the US learns about 1,000 new words per 
school year. That would mean someone making 9 years gain would have learned 
5,000 new words in less than 6 hours! How could this happen? In fact, how 
could one make a genuine year's gain in 5 hours, when the normal, typical 
child in the US makes just one year's gain in some 1010 hours in school per 
year? 

I once did a study in which we tested a large group of young men on a 
standardized reading test, did nothing, then two weeks later we retested 
those who had scored below the 6th grade level again, then did nothing for 
another two weeks, then retested those still below the 6th grade level a 
third time. This procedure reduced the number scoring below the 6th grade 
by 40 percent! 

If we are actually going to understand what we are accomplishing in adult 
literacy education we need to come up with much better ways to evaluate 
what we are actually accomplishing. 

This need for better assessments is indicated by the finding that while the 
test administrators in the National Asssessment of Literacy declared almost 
half of adults to be at risk for low literacy skills, some 95 percent of 
adults reported they read well or very well. Maybe the adults just didn't 
know how well they actually read, or perhaps the test developers had 
produced a faulty instrument. 

All together, these kinds of considerations cal for caution in the use and 
interpretation of test scores I think. 

Tom Sticht 



---------------------------------------------------- 
National Institute for Literacy 
Learning Disabilities mailing list 
LearningDisabilities at nifl.gov 
To unsubscribe or change your subscription settings, please go to http://www.nifl.gov/mailman/listinfo/learningdisabilities 
Email delivered to ooprc at comcast.net 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.nifl.gov/pipermail/learningdisabilities/attachments/20091031/af8ec123/attachment.html