Return-Path: <nifl-aalpd@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id h464ILU09001; Tue, 6 May 2003 00:18:21 -0400 (EDT) Date: Tue, 6 May 2003 00:18:21 -0400 (EDT) Message-Id: <3ECCDF24@webmail.utk.edu> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-aalpd@literacy.nifl.gov Originator: nifl-aalpd@literacy.nifl.gov Sender: nifl-aalpd@literacy.nifl.gov Precedence: bulk From: jataylor <jataylor@utk.edu> To: Multiple recipients of list <nifl-aalpd@literacy.nifl.gov> Subject: [NIFL-AALPD:333] from Tom Sticht - ALPD: Evidence-Based? X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Transfer-Encoding: 8bit X-Mailer: WebMail (Hydra) SMTP v3.61.08 Content-Type: text/plain; charset="ISO-8859-1" Status: O Content-Length: 5297 Lines: 94 Hello all - The following message is from Tom Sticht, and due to a tech glitch, it was intended to have been posted last week. Apologies for the delay in getting this message through. Please read on - Jackie ALPD: Faith-Based or Evidence-Based? My interest in finding "scientific, evidence-based" information for the efficacy of adult literacy professional development was earlier stimulated by an action research project that colleagues and I worked on from 1992 to 1998. In a January 1998 final project report entitled Passports to Paradise: The Struggle to Teach and to Learn on the Margins of Adult Education (available from ERIC), colleagues and I reported information about California’s 20 some year effort to implement competency-based adult education (CBAE). A great deal of money and years of time was spent trying to implement CBAE through professional development, preparation of "how to" manuals, conferences, and development of the California Adult Student Assessment System (CASAS-now called the Comprehensive Adult Student Assessment System) to measure improvements in learning gains as CBAE was implemented in more and more programs. However, when our team examined twelve years of the pre and post test data collected state wide from 1985 to 1996 by the CASAS there was no noticeable improvement in gain scores for either ESL or ABE students. For ESL, the gain stayed at from 4 to 5 CASAS scale points and for ABE gain scores for all levels of ABE stayed constant when scores from the same forms of the CASAS tests were used. In this case then, there was no reason to infer that all the professional development had lead to increases in student learning as indexed by the official assessment of learning developed for the express purpose of measuring the gains in learning resulting from increasing the numbers of programs implementing CBAE. Similarly, there was no improvement in the percentage of adults in ABE or ESL retained for 80 to 120 hours of instruction, which was the range of hours required to participate in the CASAS pre and post testing, suggesting that the CBAE approach was not particularly effective in increasing retention. And, somewhat surprisingly, measures of the percentage of adult students in ABE or ESL who reported that they met their goals before leaving the program actually dropped from almost 15 percent in 1987 to 5 percent in 1996, a drop of two-thirds in those who said they met their goals before leaving their programs. This work in which almost two decades were dedicated to professional development to implement CBAE seems to have produced no improvements using the data expressly developed by the state to indicate the positive improvements that were expected as CBAE implementation increased throughout the state. Today, millions of dollars and a great deal of time is devoted to adult literacy professional development, and apparently there will be a pre-COABE session presented by Cristine Smith and colleagues to show teachers and others how to use "evidence-based" research to change their practice in some manner. I have assumed that such change would be expected to improve practice in some way that is demonstrable in some manner, otherwise if there was no way to determine whether the new practice had in fact improved something, why would one bother with it? I don’t believe any of the philosophers Andres Muro mentioned have presented methods to deal with this problem of determining whether something has or has not occurred, nor did Catherine King present any references to work of her own or to any other work demonstrating the efficacy of any particular adult literacy professional development efforts for improving practice, whether assessed using "dialogic" methods which she seems to prefer, or any other methods. And parenthetically, I should note that "scientific" research is expressly based on "dialogic" methods, and that is why peer reviewed journals are important, scientists "dialogue" in print so that others may track the "dialogue" and a public record of the "dialogue" is kept for others to study and evaluate in their own terms. Given the history of the long term attempt to improve practice through professional development in California, with at least some questions of its success suggested by the "evidence-base" the CASAS presented for a dozen years, and given the current emphasis upon "evidence-based" research applied through professional development to the improvement of practice, I am curious to know if anyone has references to "evidence-based" professional development that has actually indicated in some manner that some improvement in practice had occurred as a consequence of the professional development. The closest thing to the CBAE movement of which I am aware today is the EFF project, which has gone on for some seven years with considerable federal support, which is why I have wondered if any evidence for its efficacy over the status quo has been reported. So far, responses to the AAPLD list suggest to me that there isn’t an abundance of such information – if any. Possibly, adult literacy professional development falls more within the present federal government’s "faith-based" initiatives than its "evidence-based" initiatives. : -) Tom Sticht Tsticht@aznet.net
This archive was generated by hypermail 2b30 : Thu Mar 11 2004 - 12:15:10 EST