Return-Path: <nifl-4eff@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id f3OG71g00628; Tue, 24 Apr 2001 12:07:01 -0400 (EDT) Date: Tue, 24 Apr 2001 12:07:01 -0400 (EDT) Message-Id: <3.0.6.32.20010424115916.011e0170@pop.utk.edu> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-4eff@literacy.nifl.gov Originator: nifl-4eff@literacy.nifl.gov Sender: nifl-4eff@literacy.nifl.gov Precedence: bulk From: Brenda Bell <bsbell@utk.edu> To: Multiple recipients of list <nifl-4eff@literacy.nifl.gov> Subject: [NIFL-4EFF:1516] Re: EFF/NRS Connections X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Type: text/plain; charset="us-ascii" X-Mailer: QUALCOMM Windows Eudora Light Version 3.0.6 (32) Status: O Content-Length: 9052 Lines: 183 George -- as we move ahead we are certainly considering the issues you raise, and we will be posting more in the coming weeks. I'm not attempting to reply fully this time to your posting due to time constraints -The EFF assessment team is spread pretty thin at the moment. We'll be in touch when schedules allow. But know that we appreciate your interest and concerns. Brenda Bell At 10:11 AM 4/21/2001 -0400, you wrote: > Thanks Brenda and Kathleen, also for your excellent and thorough >response. > > It is clear to me that the work going into the refinement of the >standards > is extensive and the linkage of the 4 identified dimensions of expertise > (structure of knowledge base, fluency of performance, independence of > performance, range of conditions for performance) to specific levels of >expertise, is important. I gather, then, that as the work progresses it >is the relationship between these levels and the second generation of NRS >standards that is under >discussion between NIFL and OVAE. > > I had originally thought, as you say that the EFF Assessment Framework > focuses on the 16 process-oriented Standards, though in the EFF Voice > article (p. 4) it does say: > > "Our goal is to assure that, in the next generation of the NRS, adult > learning programs and states will be able to report learner progress >from > level to level on all 16 Standards and to report achievement of >measurable > learner outcomes related to the roles of parent/family member, > citizen/community member and worker." > >Of course, EFF is designed to enhance the capacity of students >specifically in those contexts, but if I have it right, the core >*content* of the EFF framework are the proces-oriented standards >(stemming from the generative skills), while the roles represent the >*context* of the framework. When you consider also the rest of the >framework, the 4 purposes, common activities, and knowledge domains, >those are all process-oriented, while the role maps are the most directly >product-based portions of the framework--that is, they result in >specific, potentially measurable outcomes and perhaps most easily >reportable through the behaviorist, linear, objective, measurable, and >standardized model which characterizes the current generation of NRS. > >My position is not that such outcomes shouldn't be reported. The problem >is in how they are reported and the various intents reflecting both the >overt and "hidden" curriculum of the meaning and purpose of adult basic >education implicit within them. (This has been extensively discussed on >the NLA (09-12-99). So, in reading the EFF Voice article and reflecting >both on the current NRS and recent discussion on the WIA/NRS as well as >taking into account the development of EFF, I continue to raise the issue >of how an assessment framework will come down in the second generation of > NRS. In particularly I wonder about the relationship between process >oriented dimensions of assessment reflective of the Standards, even >though a rubric scale is being developed which will have a quantitative >component that could include the more behaviorist "outcomes" that might >apply to the role maps as well, which may be easier to measure in >traditional ways of quantitative reporting to which "the system" is used >to. > >Allow a suggestion since NIFL does have OVAE's ear and the specific >direction of the second generation presumably remains open. It is a >suggestion that John Comings issued some time ago about the importance of >sampling and obtaining multi-measures on fewer students in order to gain >a fuller understanding on how, what, and why students have learned. When >asked about this once, Ron Pugsley said that while sampling was not >disallowed by the NRS it was problematical because of the difficulty of >obtaining accurate sampling information from each participating program. > >I believe this can be resolved which I'll briefly sketch below, but let >me state first, how central I believe it is if we are ever to get off the >quantitative metaphor as the primary we narrate our *story,* instead of, >say, through qualitative measures. Given the importance of that, >combined with what Tom Sticht refers to as the ever present *turbulence* >that not only characterizes the situations that many of our students >experience, but our fieldas a whole, it is more than a pipe dream and >highly futile to expect to obtain "objective, measurable," and >standardized" information on *every* student. I suggest we try to change >things so that we do not have to play that game. > > The following is what I wrote on the NLA over a year ago, also >reproduced as >part of an article that includes several opther authors, in All Write >News, Sept/Oct 2000: > >"At the end of each funding cycle the state office would have to issue a >narrative report on the programs that are funded by federal money. The >report would be based on the narratives that the programs provided that >then would be summarized, synthesized and analyzed by the state offices >of adult education. This would be an ongoing progress report. A broad >range of information would go into the report at both the local and state >level and there would be no need of a "one size fits all" approach." May >1000 flowers bloom. Obviously, certain criteria would be needed, though >the plurality of the system in a given state would also be respected, >(Why should a town run ABE program be judged on the same criteria as a >community-based literacy program?). > > This scenario would include quantitative information, but more as >support (secondary measures) that would amplify or help explain the >primary story told through narrative. In this scenario, sampling, rather >than focusing on quantitative information on each student would be more >pervasive. Given the narrative focus, there would be an emphasis on >development such as: > > a) This is where we've been > > b) This is what we attempted to achieve during this funding cycle > > c) This is what we actually accomplished, including unanticipated >breakthroughs and a whole host of projects and initiatives that flowed >out of our work > > d) These are the problems that we still have-some of which we have the >capacity to improve on, some of which we don't given current resources > > e) This is where we'd like to go for the next funding cycle > >This report could be developed by a representative team from both the >field and the state office with consultative support from a research >institute. The information would flow back to the programs as well as >upward to the federal government and would be a format to stimulate >discussion, analysis, and program development as well as "data" for >national accounting. > >To assume that my suggestion is 'subjective' as opposed to the >'objectivity' provided by a quantitative, measurable, and standardized >format that drives the assumptions of the NRS is to assume that numbers >accurately depict reality rather than interpret it. I do not make that >assumption.'" > >I also believe such an accountability system would provide the field with >a wealth of information and insight currently lacking. Such a system >would be based on qualitative as well as quantitative information, but >not privileging the latter as the "primary" measure. Both qualitative >and standardized information are useful and relevant. It is only the >privileging of one philosophical and research tradition (positivism, >empiricism, behaviorism, and "value-free" research) over others >(phenomenology, critical theory, ethnography, constructivism and >qualitative research) that makes it seem that one is "objective" >and the other is "subjective." > > Something like what I am laying out would allow scope for > > a) measuring outcomes based on the role maps > > b) an evaluation of the more complex and process-oriented Standards > > c) provide general statistical information about programs > > d) provide opportunity for a narrative description of program >development within a fiscal year based on "a-e" as identified above >and/or whatever else > > And it would greatly extend the knowledge base of our field and lay >groundwork for further development. > > As I have argued elsewhere, the three primary points of conflict in our >field are over: > > a) purposes and philosophy of instruction > > b) assessment processes linked to particular purposes and philosophy of >instruction > >c) social policy which grounds a and b > > If any vital and enduring consensus is going to emerge amidst the >contested ground, subtle and sustained mediation of the issues which do >conflict need to take place. > >Perhaps the perspective highlighted above focusing on a >narrative report that *includes* standardized data, but doesn't privilege >it as foundational or primary can help to resolve some of these issues >and help to create a more meaningful system. > > George Demetrion > Literacy Volunteers of Greater Hartford > Gdemetrion@juno.com > Gdemetrion@lvgh.org >
This archive was generated by hypermail 2b30 : Tue Apr 09 2002 - 13:29:53 EDT