[Assessment 1334] Re: Tests vs. Self Assessments of Literacytsticht at znet.com tsticht at znet.comMon Jun 9 14:12:42 EDT 2008
David: Regarding your questions about adult literacy (L) and numeracy (N) needs, I have posted your questions and my answers below. Question: Why do countries assess literacy (L) and numeracy (N) need? Answer: To find out how many adults have poor LN abilities and who might, therefore, benefit from education to improve their skills. This may lead to policies providing for education provision to meet the need for LN education. Question: What have they learned? Answer: Based on standardized tests of literacy (L) and numeracy (N) they have learned that LN skills are distributed in the adult population from low to high, particularly if the tests are designed to create such a distribution (resembling to some extent the familiar bell curve). They have also learned that most adults do not think they have major problems with literacy and numeracy, though more, primarily immigrants, think they could use better English language skills. They have also learned about a number of correlations with test scores, e.g., unemployment, family background, etc. Question: What is the added value of these assessments to low-literacy or low-numeracy residents? Answer: This is hard to say. In England the assessments added more to those with medium to higher LN skills as the government moved to increase the numbers of adults with educational qualifications, similar to passing one or more of the GED tests in the U.S. and Canada. In the U.S. following the NALS of 1993 and the implementation of the National Reporting System in 1998, then numbers of enrollees in the Adult Education and Literacy System, i.e., the programs funded in part through the Workforce Investment Act, Title 2, fell from over 4 million per year to around 2.6 million in 2005, the last date for which I have seen data. In England latest reports are that adult education in further education courses have dropped by around 1.5 million in the last few years, though this does not exactly focus on those with low LN. Questions: What is the added value of these assessments to their governments and societies? Have the data had an impact on policy (yet)? What has been the "return on investment" for national/international needs assessments of adult literacy and numeracy? Answer: Again, these are difficult to answer questions. The added value to governments and societies has not been reported as far as I know. The data have had an impact on policies for adult LLN provision in Australia and Canada to less degree than in England and New Zealand and has had very little impact on adult LLN policies at the federal level in the U.S., if by policy one refers to added investment in the AELS. The NALS data has had a bit more impact on funding policies in various states I believe. No "return on investment" for assessments has been calculated for the nations I have studied. Question: Why are we measuring need but not demand? Answer: In my 1998 paper on Beyond 2000: future directions for adult education, I provided data showing that "demand" or "need" as indicated by enrollments, had increased by about 100,000 a year from 1966 to reach a level of some 4 million in 1998. I thought this illustrated the value of the AELS and expected that with increased resources the growth would continue up to 5 million or more. This would provide a strong basis for arguing for increased funding for the AELS. But the imposition of the NRS accountability requirements, with heavy demands on testing and data collection, led to a precipitous drop in enrollments of over 1.5 million which persists up to the present. I am all in favor of making a demand-based argument for AELS support and enrollment data have served that purpose in the past. Questions: What percent or number of people see that they have basic skills needs -- and want to pursue classes, tutorials or online learning to address it? Would that be a more useful figure for public policy decisions than needs data? Answer: The percent of people who see that they have basic skills needs and want to pursue LN education IS a type of "needs" data. The question should be: is this a better indicator of needs than the NAAL or similar national standardized tests distributions of scores. I think it would be a more useful figure for public policy decisions than NAAL-type assessments which are subject to many, many arbitrary decisions in their construction of test score distributions and more such decisions in deciding at which level or levels the statements of "need" should fall. Many have used the present NAAL levels of Basic and Below Basic to declare some 93 million adults incapable of meeting their daily needs for work , living, and health despite a complete lack of any basis for defining the levels using these terms and for setting societal incompetence at these levels. Different terms were used to refer to the earlier NALS levels, illustrating the arbitrary nature of these level names, and the NALS used different statistical data for making literacy levels than the NAAL, again illustrating the arbitrary nature of the test development and interpretation. Questions: Some states have begun to count the number of people who want classes but are put on waiting lists. This is one measure of demand. Should these states be supported to improve/refine their data collecting and reporting so we can have confidence in this measure, so it can be used for decision making? Should we be looking at other measures of demand? If so, what? Should we do latent demand studies in adult education? ("If English classes or GED Prep -- were offered on Saturday mornings would you attend?") If so, should we do these studies systematically? Perhaps we should work on addressing the actual demand staring us in the face and worry about measuring latent demand -- and need -- later. Answer: National, state and local needs and wants surveys asking people about their skills needs and wants, using direct questioning and "latent needs" questioning, seem like a good idea to me. There should be some systematic studies along these lines and some repeated local needs/wants assessments from time to time to reach the local folks would be useful. I think that some adult educators should work on addressing the actual local demand for provision right now, while others, perhaps those mostly committed to research, might work on measuring directly stated and latent needs for policy making at state and national levels. I have demonstrated how this might be accomplished using telephone surveys in the U.S. and such surveys have also been used in England and Canada. Tom Sticht
More information about the Assessment mailing list |