Return-Path: <nifl-aalpd@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id h48HQoC28506; Thu, 8 May 2003 13:26:50 -0400 (EDT) Date: Thu, 8 May 2003 13:26:50 -0400 (EDT) Message-Id: <5896FADE.141BAE70.0AB94E44@aol.com> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-aalpd@literacy.nifl.gov Originator: nifl-aalpd@literacy.nifl.gov Sender: nifl-aalpd@literacy.nifl.gov Precedence: bulk From: AndresMuro@aol.com To: Multiple recipients of list <nifl-aalpd@literacy.nifl.gov> Subject: [NIFL-AALPD:358] Re: from Tom Sticht - ALPD: Evidence-Based X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 X-Mailer: Atlas Mailer 2.0 Status: O Content-Length: 11600 Lines: 155 Tom, et al: After searching the database, I think that I found the message that you were referring to. To begin with, the people that I am referring to are, by definition, a sociologist, a social theorist and a philosopher, and all three explore the question of how we arrive at determining how things have or have not occurred. They all criticize the positivistic model for being uncritical in claiming to be the only model valuable in determining how things may have or may have not occurred. I explained in a follow up message in more detail, the grounds for their criticism. I copied and pasted my follow up response below. The point that I was trying to make was that traditional scientific research is essentially ineffective in determining complex questions such as the impact of professional development on a program. I spelled the criticism clearly and wondered how you would respond to such criticism. You did not respond to my question. Furthermore, Bourdieu does, in fact, extensively explore ways in which we can determine how things may or may not occur. His problem has to do with the concept of "habitus". Essentially, what Bourdieu claims is that when an observer observes something. He may conclude that what he senses, as a response to an input is the actual outcome to the input. The problem with this is that the observer's sense of what the outcome of the input is, may in fact not be the outcome to the input. It may be an outcome to many other inputs, or simply a misguided interpretation, or non-of the above. Hence, in order for a researcher to be able to observe the outcome of an input, he needs to be familiar with the "habitus" (social context) in which an input and response take place. Only in this fashion, the researcher can get a good sense of what the connection is between inputs and responses may be and the effectiveness of the senses in determining the validity of the observation. This mode of research requires that we step outside of the traditional data-collecting model. Allow me to illustrate in simple terms. Lets suppose that I want to determine if a movie is good and I ask people if they like the movie. Suppose that someone says: Man, that was bad!!! Or the movie was bad ass! What should I conclude? Let's suppose that the respondent feels that there may be an advantage to respond in one way vs another. How do I evaluate this? Assume that someone says that the movie was cool. What does this mean? Suppose that I know that if the movie gets good ratings there will be more $$$$ and I will be able to keep my job? So, in a sense, a researcher needs to spend time in a context becoming familiar with the behaviors of people within that context before she may be able to draw conclusions. The fact that there is no scientific evidence to determine if PD works or not doesn't mean anything in terms of the value of PD. A more valid question from a Bourdieuian perspective would be: Do teachers feel that PD improves their practice? The ultimate point is that the arguments against scientific research are valid. However, in the circles of power, scientific research has more credibility than asking teachers, even if there is no validity to the scientific research. As long as scientists are not critically re-examine their positions, it is pointless to have a discussion. All they have to do is to call other people philosophers in order to invalidate them (in their minds). Andres Quote from previous message. "When I refer to positivistic, for clarification purposes, I mean that research that claims to be able to objectify the topic of study and observe it from a "neutral, objective" perspective. What determines the objectivity of the observer? How can the observer be neutral if he/she is a subject? Of course, ultimately she cannot, so she needs a protocol to conduct the observations that would guarantee his/her objectivity. Assuming that the protocol will ensure the absolute neutrality of the observer, which of course it never does, what guarantees the neutrality of the one who created the protocol to establish the rules for objectivity? In essence, what we need is not just positivistic science to conduct an observation, but also, as Bourdieu suggest, a science that observes the objectivity of the observer who is doing the research, and another one of the objectivity of the protocol that was established for the observer, and one for observing how those who created the protocol, created it in the first place. Ultimately what you will arrive to is that a group of people agreed to establish a paradigm, because they all felt that this was the best method to arrive at evidence in a given situation. There is nothing wrong with this, and in fact I support this. However, this is merely pragmatism, something that I also, support, as you know. In other words, a group of people gets together and decides through dialogue to find a way to resolve a particular problem and agree to this, hence building a community. Of course, the thing that legitimates a given group's method to arrive a truth is simply power. So this is something that needs to be questioned. What chaps my posterior is that, when in power, people begin to claim that theirs, is the only valid way to arrive at truth. Neither rationally, nor empirically they can make that claim, unless, of course, they are philosopher kings :-)" Andres In a message dated 5/7/2003 8:53:31 PM Eastern Standard Time, AndresMuro writes: > >Tom writes: > >"I don’t believe any of the philosophers Andres Muro mentioned > have > presented methods to deal with this problem of determining > whether > something has or has not occurred" > >Tom: > >Wow!!! While I am sure that you are right, I don't know what I said, what you are referring to, or what you are talking about. Could you quote or paraphrase what I may have said? > >Thanks, > >Andres > >In a message dated 5/5/2003 11:17:42 PM Eastern Standard Time, jataylor <jataylor@utk.edu> writes: > >>Hello all - The following message is from Tom Sticht, and due to a tech >>glitch, it was intended to have been posted last week. Apologies for the >>delay in getting this message through. Please read on - Jackie >> >> >>ALPD: Faith-Based or Evidence-Based? >> >>My interest in finding "scientific, evidence-based" information for the >>efficacy of adult literacy professional development was earlier stimulated >>by an action research project that colleagues and I worked on from 1992 to >>1998. >> >>In a January 1998 final project report entitled Passports to Paradise: The >>Struggle to Teach and to Learn on the Margins of Adult Education >>(available from ERIC), colleagues and I reported information about >>California’s 20 some year effort to implement competency-based adult >>education (CBAE). A great deal of money and years of time was spent trying >>to implement CBAE through professional development, preparation of "how >>to" manuals, conferences, and development of the California Adult Student >>Assessment System (CASAS-now called the Comprehensive Adult Student >>Assessment System) to measure improvements in learning gains as CBAE was >>implemented in more and more programs. >> >>However, when our team examined twelve years of the pre and post test data >>collected state wide from 1985 to 1996 by the CASAS there was no >>noticeable improvement in gain scores for either ESL or ABE students. For >>ESL, the gain stayed at from 4 to 5 CASAS scale points and for ABE gain >>scores for all levels of ABE stayed constant when scores from the same >>forms of the CASAS tests were used. In this case then, there was no reason >>to infer that all the professional development had lead to increases in >>student learning as indexed by the official assessment of learning >>developed for the express purpose of measuring the gains in learning >>resulting from increasing the numbers of programs implementing CBAE. >> >>Similarly, there was no improvement in the percentage of adults in ABE or >>ESL retained for 80 to 120 hours of instruction, which was the range of >>hours required to participate in the CASAS pre and post testing, >>suggesting that the CBAE approach was not particularly effective in >>increasing retention. And, somewhat surprisingly, measures of the >>percentage of adult students in ABE or ESL who reported that they met >>their goals before leaving the program actually dropped from almost 15 >>percent in 1987 to 5 percent in 1996, a drop of two-thirds in those who >>said they met their goals before leaving their programs. >> >>This work in which almost two decades were dedicated to professional >>development to implement CBAE seems to have produced no improvements using >>the data expressly developed by the state to indicate the positive >>improvements that were expected as CBAE implementation increased >>throughout the state. >> >>Today, millions of dollars and a great deal of time is devoted to adult >>literacy professional development, and apparently there will be a >>pre-COABE session presented by Cristine Smith and colleagues to show >>teachers and others how to use "evidence-based" research to change their >>practice in some manner. I have assumed that such change would be expected >>to improve practice in some way that is demonstrable in some manner, >>otherwise if there was no way to determine whether the new practice had in >>fact improved something, why would one bother with it? >> >>I don’t believe any of the philosophers Andres Muro mentioned have >>presented methods to deal with this problem of determining whether >>something has or has not occurred, nor did Catherine King present any >>references to work of her own or to any other work demonstrating the >>efficacy of any particular adult literacy professional development efforts >>for improving practice, whether assessed using "dialogic" methods which >>she seems to prefer, or any other methods. And parenthetically, I should >>note that "scientific" research is expressly based on "dialogic" methods, >>and that is why peer reviewed journals are important, scientists >>"dialogue" in print so that others may track the "dialogue" and a public >>record of the "dialogue" is kept for others to study and evaluate in their >>own terms. >> >>Given the history of the long term attempt to improve practice through >>professional development in California, with at least some questions of >>its success suggested by the "evidence-base" the CASAS presented for a >>dozen years, and given the current emphasis upon "evidence-based" research >>applied through professional development to the improvement of practice, I >>am curious to know if anyone has references to "evidence-based" >>professional development that has actually indicated in some manner that >>some improvement in practice had occurred as a consequence of the >>professional development. The closest thing to the CBAE movement of which >>I am aware today is the EFF project, which has gone on for some seven >>years with considerable federal support, which is why I have wondered if >>any evidence for its efficacy over the status quo has been reported. >> >>So far, responses to the AAPLD list suggest to me that there isn’t an >>abundance of such information – if any. Possibly, adult literacy >>professional development falls more within the present federal >>government’s "faith-based" initiatives than its "evidence-based" >>initiatives. : -) >> >>Tom Sticht >>Tsticht@aznet.net >> >> > > >-- >go here: www.geocities.com/andresmuro/art.html > > -- go here: www.geocities.com/andresmuro/art.html
This archive was generated by hypermail 2b30 : Thu Mar 11 2004 - 12:15:10 EST