Michael and Tom:
 
High quality tests usually publish reliability data about the test. I
don't administer the California Achievement Test, but there should be
data about its reliability. Test-retest reliability data can be
confounded in many ways. If the interval between administrations is
short, practice effects can alter the scores. If the interval is too
long, the correlation between the scores can be confounded by an actual
change in what the student has learned. I base my assumptions about the
reliability of a test on the published reliability numbers. I commonly
use the WIAT-II which has high reliability for reading and spelling at
all age groups I test. It does however only have moderate reliability
for writing. If you doubt a test you can look this information up in
books such as The Achievement Test Desk Reference.
 
Thanks,
 
Brant Hayenga
Educational Diagnostician 
Stapleton Elementary/Rio Rancho Middle School
(505) 896-0667 ext. 226 (District Office)
(505) 891-8473 ext. 519 (Stapleton Elementary)
bhayenga at rrps.net
 
 
-----Original Message-----
From: learningdisabilities-bounces at nifl.gov
[mailto:learningdisabilities-bounces at nifl.gov] On Behalf Of
tsticht at znet.com
Sent: Monday, November 02, 2009 1:23 PM
To: learningdisabilities at nifl.gov
Subject: [LearningDisabilities 4217] Re: What does a grade level gain
mean?
 
Michael, Brant, and all: In the repeated testing I mentioned in an
earlier
note we used three alternate forms of the California Achievement Test,
which, you may find of interest, was the forerunner to the TABE test
which
is the most widely used standardized test in ABE. The following note
discusses some more of the strange aspects of a strong commitment to
standardized testing in ABE for national accountability.
 
Tom Sticht
 
Research Note March 13, 2001
 
A Fantasy in Three Themes for Adult Literacy Education
 
Thomas G. Sticht
International Consultant in Adult Education
 
American Heritage Dictionary Definition:
Fantasy. 2. A medley of familiar themes, with variations and interludes.
 
The Conductor enters to much applause, raises the baton calling the
orchestra to attention, the house goes silent, and the concert starts
with
the first of three familiar themes in adult literacy education:
 
 
Competency Based Adult Education. Theeee're Baaack! Just when you
thought
CBAE with all those thousands of separate, discrete "competencies" (the
Comprehensive Adult Student Assessment System (CASAS) has over five
thousand test items in its bank, each of which can be considered a
"competency") was gone, and it was safe to think about constructivist
approaches to teaching and learning to develop rich >bodies of
knowledge,
with critical reflection about interrelationships among thoughts and
ideas,
that could help solve numerous problems of daily life, promote
professionalism in numerous occupations, and provide recreational
reading,
spiritual uplifting and all the rich experiences that depend on literacy
and knowledge, ...you're wrong! Building on thirty years of failing to
get
CBAE ensconced in adult >literacy education, with test data from
California
showing no improvement in learning after over a decade of trying to
switch
programs to CBAE, the powers that be have now shifted from trying to use
CBAE with students and instead are trying to use it with teachers! Visit
the >Pro-Net 2000 (1) site to read the numerous "competencies" that you,
as
a teacher, need to be considered qualified for the work you have been
doing
for decades. Also see the numerous "what adults need to know and be able
to
do" in the Equipped for the Future project of the National Institute for
Literacy.
 
 
 
Unlearning Literacy. Almost thirty years ago (1974) a colleague and I
prepared a paper entitled "The Problem of Negative Gain Scores in the
Evaluation of Reading Programs." (2) The paper ascribed the major
sources
of such scores to three properties of the standardized tests typically
used
in pre- and post-testing: response format, content, and instructions and
we
discussed issues in deleting negative gain scores from the summary
statistics used to evaluate the gain in achievement due to literacy
education. Still, the issue of negative gain in adult literacy education
is
hardly ever discussed. For example, a year 2000 report from MassInc. (3)
reported some 4 percent negative gain scores and 40 percent no change in
scores in ABE and a year 2001 report from the United Kingdom's Basic
Skills
Agency (4) reported over 30 percent negative gain scores with no
discussion
of how these zero and negative gain scores were treated. But if in
reporting to the National Reporting System we interpret positive gain to
mean learning, shouldn't we interpret zero gain to mean no learning and
negative gain to mean unlearning? Can students who have spent time in
programs and learned nothing and those who have had their literacy
unlearned sue for damages?
 
 
Learner Centered. For decades adult educators have emphasized the
importance
of being "learner-centered" . Pro-Net 2000 professional development
competencies also emphasize being learner centered. For instance, under
Competency 12, Indictors of Competency 12.3 states, "Shares with learner
appropriate decision making responsibilities", and 12.4 states, "Links
instruction to learner's needs assessments." Yet in a 1998 survey of
over
270 adult literacy programs across the United States NCSALL researchers
(5)
reported only 10 percent that involved learners in dialogue and decision
making to a meaningful degree. At the level of large-scale literacy
needs
assessments, the International Adult Literacy Survey (IALS) in
commenting
on the fact that so few adults in literacy Level 1 said they read poorly
for their work simply dismissed adult self-assessments of their literacy
skills in a sidebar that states "Most people do not recognize that they
have a literacy problem.." (6). Learner centered needs assessment is
apparently good at the local level but not at the national and
international level. It seems the "experts" know what adults need
without
asking them, even if teachers don't.
 
 
Intermission time. Be back in another thirty years!
 
 
Pro-Net 2000 web site: www.pro-net2000.org
 
Caylor, J . & Sticht, T. (1974, April). The Problem of Negative Gain
Scores
in the Evaluation of Reading Programs. Chicago, IL: paper presented at
the
meeting of the American Educational Research Association.
 
Comings, J., Sum, A. & Uvin, J. (with others) (2000, December).New
Skills
for a New Economy: Adult Education's Key Role in Sustaining Economic
Growth
and Expanding Opportunity, Boston, MA: MassInc.
 
Brooks, G. et al (2001, January). Progress in Adult Literacy: Do
Learners
Learn? London: Basic Skills Agency.
 
Purcell-Gates, V. et al (1998, July) U. S. Adult Literacy Program
Practice:
A Typology Across Dimensions of Life-Contextualized/Decontextualized and
Dialogic/Monologic. NCSALL Reports #2. Cambridge, MA: National Center
for
the Study of Adult Learning and Literacy.
 
Organization for Economic Co-operation and Development (1995). Literacy,
Economy and Society. Paris: OECD (page 101).
Cover
 
 
There is also a discussion of regression effects in ABE and the issue of
negative gain in
 
Testing and Accountability in Adult Literacy Education
 
Focus on Workplace Literacy Resources for Program Design, Assessment,
Testing, & Evaluation
 
Thomas G. Sticht
 
November 1999
 
 
This report can be found using Google.
 
Tom Sticht
 
 
 
 
----------------------------------------------------
National Institute for Literacy
Learning Disabilities mailing list
LearningDisabilities at nifl.gov
To unsubscribe or change your subscription settings, please go to
http://www.nifl.gov/mailman/listinfo/learningdisabilities
Email delivered to bhayenga at rrps.net
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.nifl.gov/pipermail/learningdisabilities/attachments/20091102/10a2336d/attachment.html 
-------------- next part --------------
Please note that RRPS email addresses have changed. Please update your 
address books or distribution lists.