Return-Path: <nifl-aalpd@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id i0PMbLI29493; Sun, 25 Jan 2004 17:37:22 -0500 (EST) Date: Sun, 25 Jan 2004 17:37:22 -0500 (EST) Message-Id: <7D71C62A.29FA85CC.0AB94E44@aol.com> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-aalpd@literacy.nifl.gov Originator: nifl-aalpd@literacy.nifl.gov Sender: nifl-aalpd@literacy.nifl.gov Precedence: bulk From: AndresMuro@aol.com To: Multiple recipients of list <nifl-aalpd@literacy.nifl.gov> Subject: [NIFL-AALPD:935] Re: examining the effectiveness of professional development X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 X-Mailer: Atlas Mailer 2.0 Status: O Content-Length: 11067 Lines: 189 Hello Jackie! As I said earlier and in my responses to Eileen and Deborah, I believe in the legitimacy of PD and, as Deborah said, Texas has done a lot of work to develop neat PD projects. We ourselves have done extensive PD and so have many others. I currently have a grant to do PD in relationship to health throughout the USA (funded by Pfizer). Elsa Auerbach and Heide Wrigley have published some cool books that can be used to do PD with teachers. However, this is what we have: Neat PD models and ideas that we all intuitively know that the work both to enhance the ability of the teachers, the quality of instruction and the outcomes of instruction. However, your question wasn't about this. Your question was about having data that points to the effectiveness of PD. The answer is "no" and we shouldn't be asking this question. Moreover, we need to articulate the reasons for not asking these questions. Data implies a collected set of values and effectiveness implies a causal relationship. In other words, what you are asking is if as a result of PD can there be a measurable effect (a set of values) that demonstrate that it had the intended outcome. All we can produce is opinions by teachers about the quality of the PD, ie, the presenter knows his stuff, I learned new things, I believe that this will improve my ability to teach, I believe that this will increase retention, completion, satisfaction, etc., a description by teachers of how this will improve outcomes and a narrative of how they perceived that this may have improved outcomes. This is very powerful and very valuable stuff. However, it is all based on subjective perception and intuition. So, in the empirical sense, it does not constitute data that shows the effectiveness of PD. So, what shall we do? First, we need to validate anecdotal evidence and teachers’ subjective perceptions and intuition, as valid instruments to document impact, and refer to the literature, particularly in anthropology, supporting this. Second, we need to articulate why it is very difficult, if not impossible to collect "hard" empirical data on PD's effectiveness. Third, we need to request funding and support to do some empirical studies, understanding that these are costly, require expertise that many of us do not posses, and, because of too many independent variables, they will not be full proof. Regarding Texas regional PD centers, it'll be very cool, and I am hoping that we get one in El Paso. These centers will do neat PD and offer teachers extensive resources for improving practice that they lack. However, I don't think that they will be trying to measure effectiveness in the "hard, empirical" sense. Rather, they will provide extensive resources for teachers. BTE, We invaded Iraq based on intuition and subjective perceptions of a few, rather than "hard data". As a result, we spend a cool qutrizinmegazillion in this, in addition to a lot of lives. However, those that are governing based intuition and opinion expect us to produce hard data in our field. Ironic, isn't it? Andres In a message dated 1/25/2004 11:22:50 AM Eastern Standard Time, jataylor <jataylor@utk.edu> writes: >Hello Deborah, Eileen, Andres and All, >I asked the question "What data collection/analysis is going on in your >program, region and/or state to examine the effectiveness of PD?" because I am >aware that some subscribed to this forum are faced with the same or similar >issues that Deborah outlined below. In the environment of accountability and >scarce PD dollars, many of us are faced with the question - how do we examine >the effectiveness of professional development? > >Do we stay within the frame that PD serves the teacher; therefore we only >measure teacher learning? Or dare we explore the qualitative (as Eileen >noted) and quantitative results of those who benefit (or not) from adult and >family literacy? How is teacher learning tied to student outcomes? By >outcomes I mean a broad interpretation, as I assume funders do care whether or >not the quality of students' lives are changed as a result. > >While I am not advocating that the effectiveness of PD should be measured >based on student outcomes, I do acknowledge the challenge facing some of us in >the days ahead. And I'm sure this challenge will pose many questions along >the way, and shake the very foundation of our beliefs. Perhaps even some of us >will question, as Andres did, the underlying base of this question, and *will* >take actions to change it. > >Are there other states who have recently learned that they may be looking at >measuring the effectiveness of PD based on student outcomes? > >And Deborah, I hope you will keep us posted as to the progress of your new >accountability journey. I feel certain that many will be watching your group >and Texas LEARNS as some of us swim (whether by force or by choice) to climb >on to this boat. > >Thanks! > >Jackie > > > >>===== Original Message From nifl-aalpd@nifl.gov ===== >>I think Jackie asks an important, timely and totally appropriate question. >>Actually, all states receive funds from the federal Adult Education and >>Family Literacy Act appropriation and Section 223 authorizes states to use >>up to 12-1/2% of those grant funds for "state leadership." The preponderant >>use of State Leadership funds in most states is for professional >>development. Texas, for example, where Andres lives, is currently funding >>professional development centers for a total of about $5.1 million over the >>next 18 months. So, data collection to examine the effective use of these >>funds is completely appropriate in this environment of accountability. >>Since NRS does not collect data regarding effective use of these funds, it >>is prudent that states would examine the issue and initiate such data >>collection. >> >>Everyone knows that there is bad professional development and mediocre >>professional development out there, as well as professional development that >>meets the needs of teachers and administrators in continuously improving >>programs for students in adult and family literacy. In El Paso, where >>Andres lives, there has been a significant amount of cutting edge >>professional development provided in regard to project-based learning and >>using technology to improve adult literacy instruction and especially >>instruction for English language learners. Collecting data on the >>effectiveness of professional development in regard to student achievement >>will help policy makers know how to use State Leadership funds to maximize >>these scarce dollars. Texas has new state leadership, known as Texas >>LEARNS, for its AEFLA program that is doing some exciting things and is >>anxious to gather the data that will substantiate effective professional >>development practices. While no professional development data collection >>has been developed yet, the new professional development centers are charged >>with this responsibility. In addition, Texas LEARNS is working to >>incorporate software with the state's adult education data system, known as >>ACES, that will collect data in regard to teachers who will participate in >>the state's nascent credential for adult educators system. >> >>So, Jackie, check back with us and with Texas LEARNS as we begin this >>journey and thanks for posting the question. I hope other programs respond >>to this challenge as well. >> >>Deborah Stedman >>Texas State University >> >> >> >>----- Original Message ----- >>From: <AndresMuro@aol.com> >>To: "Multiple recipients of list" <nifl-aalpd@literacy.nifl.gov> >>Sent: Thursday, January 22, 2004 11:41 AM >>Subject: [NIFL-AALPD:923] Re: examining the effectiveness of professional >>development >> >> >>> Jackie asks: >>> > What data collection/analysis is going on in your program, region, >>and/or >>> > state to examine the effectiveness of professional >>> > development? >>> >>> None in our program, and I personally think that this is a totally silly >>question. Let me explain. If I jump into a pool I will get wet. I know this, >>even though I don't think that there is any research that collected data to >>predict this. >>> >>> Just as I know that if I get into the pool I will get wet, I know that if >>I know more about a topic I will do a better job. Of course, there are >>exceptions to the rule. However, the assumption is that if I study about >>something I will get better at it. Otherwise, nobody will study. >>> >>> Since there are virtually no schools that teach adult literacy I decided >>several years ago that I was going to learn on my own. Eight years ago I >>decided to devote an hour a day, specifically, to study adult literacy >>related stuff. I also have traveled to a lot of conferences and experimented >>with the acquired knowledge. As a result, I know that I am a much better >>literacy provider than I was 8 years ago. Do I have any data to prove this. >>Of course not. My program is doing a lot better. However, even though I know >>that it is partly because of me, I can't isolate my knowledge as an >>independent variable to be studied. Actually, theoretically I could, but it >>would be a silly task that would require an investment in developing a >>research model to be tested. Essentially, I could offer two classes. In one, >>the students would receive instruction w/o any application of the knowledge >>that I now posses. In the other we would do the opposite. We could predict >>that the students that ben! >>> efit from my knowledge would do better. However, there are variables that >>we cannot control such as illness of students, employment status, etc. So, I >>would have to repeat the study a significant amount of times to be able to >>collect valid data. >>> >>> Since no adult education program gets money to do this, how are they going >>to do this. Also, the government insists that the only programs that will >>get funded will have to do this. However, they have not been funding this >>either. Even if they funded this sort of think, does anyone realize that it >>would be completely stupid to do this? >>> >>> Also, bringing Tom Stich into the picture, I 've noticed that he does a >>lot of traveling to do lectures. Why does he do this? Simply to entertain >>people? For the $$$$? Or, does he think that people will be better at their >>jobs if they listen to him? Does he have any hard data to prove this? If >>not, why continue to do this? >>> >>> Andres >>> >>> >>> >>> >>> >>> In a message dated 1/22/2004 10:06:48 AM Eastern Standard Time, >>jataylor@utk.edu writes: >>> >>> > >>> > Colleagues: >>> > >>> > What data collection/analysis is going on in your program, region, >>and/or >>> > state to examine the effectiveness of professional >>> > development? Would you >>> > tell us about it? >>> > >>> > Thanks! >>> > >>> > Jackie >>> > >>> > Jackie Taylor >>> > NIFL-AALPD >>> > List Facilitator >>> > >>> > >>> > >>> > >>> >>> go here: www.geocities.com/andresmuro/art.html > > -- go here: www.geocities.com/andresmuro/art.html
This archive was generated by hypermail 2b30 : Thu Dec 23 2004 - 09:45:36 EST