Return-Path: <nifl-4eff@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id fAKMxE008344; Tue, 20 Nov 2001 17:59:14 -0500 (EST) Date: Tue, 20 Nov 2001 17:59:14 -0500 (EST) Message-Id: <d1.fafe0f8.292c3955@aol.com> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-4eff@literacy.nifl.gov Originator: nifl-4eff@literacy.nifl.gov Sender: nifl-4eff@literacy.nifl.gov Precedence: bulk From: McGuireP555@aol.com To: Multiple recipients of list <nifl-4eff@literacy.nifl.gov> Subject: [NIFL-4EFF:1930] update on EFF Assessment Field Development X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas X-Mailer: AOL 7.0 for Windows US sub 118 Content-Type: multipart/alternative; boundary="part1_d1.fafe0f8.292c3955_boundary" Status: O Content-Length: 18271 Lines: 317 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Greetings, everyone, and best wishes for a Happy Thanksgiving 2001. This is Peggy McGuire, checking in from the EFF Assessment Consortium team with an update on our field development activities toward building an assessment framework for the EFF Standards. This posting is rather long -- you may want to save it for reading later if you're in a crunch -- but I hope you find it interesting and useful! As you know from earlier 4EFF postings (see messages from Brenda Bell and yours truly, in April, May and June 2001), the 2000-2001 phase of EFF field development was defined by a new partnership with the U.S. Department of Education, Office of Vocational and Adult Education/Department of Adult Education and Literacy (OVAE/DAEL), allowing us to explore how the development of the EFF assessment framework might enrich the next generation of the National Reporting System (NRS) mandated by Title II of the Workforce Investment Act. This Joint EFF/NRS Data Collection Project has had as its purposes 1) to enhance the level descriptors for the skills currently included in the NRS (read, write, speak listen, numeracy) and 2) to allow the measurement and reporting of skill gains on the other 11 EFF Standards. So as some 88 EFF Field Development teacher/researchers from 24 programs in five partner states (Ohio, Oregon, Maine, Tennessee, and Washington) were working hard to develop well-constructed EFF Performance Tasks and document learner performance on those tasks throughout the year, their research efforts were serving several related purposes: · - to build a "continuum of performance" for each of ten EFF Standards: Read with Understanding, Convey Ideas in Writing, Speak so Others can Understand, Listen Actively, Use Math to Solve Problems and Communicate, Solve Problems and Make Decisions, Take Responsibility for Learning, Learn Through Research, Use Information and Communications Technology, Cooperate with Others, and by doing so, · -To identify research-based performance descriptors for each NRS level (both ABE and ESL) that will be useful to teachers in diagnosing learner strengths and needs and in planning instruction, as well as useful to programs and states for reporting progress, and · -To identify a range of performance tasks that "benchmark" transition points from level to level and that will enable programs to report both what learners know and what they can do at each level. During the late spring and summer of 2001, Assessment Consortium staff and Field Assistants reviewed and analyzed the data submitted by teacher/researchers. We met in July to begin the process of constructing tools based on that data for four of the Communications Standards (Read, Write, Speak, Listen), including · - A Performance Data Template for each Standard, on which we attempted to range relevant data excerpts in sequence across a developmental continuum for the dimensions of knowledge base, fluency and independence, and to identify preliminary "transition markers" from one "level" to another where possible; · - A first draft of a Performance Continuum for each standard, built from what we identified as possible transition markers and summary statements of the behaviors/performances described in each Performance Data Template; and · - A "Task View" for each Standard, which illustrated the range (types, and overall rating for complexity, context and knowledge base requirements) of tasks that had been developed by teacher/researchers to elicit the performance data that they had submitted. We brought these tools with us to a final Project meeting for field development participants, held in Portland, Oregon on September 12-15. Despite the extraordinary and painful circumstances we found ourselves in as a result of the events of 9/11, we decided to go on with the meeting and were amazed that over half of the scheduled attendees were able to attend. More than attend -- their focus and thoughtfulness were astonishing, and while we sorely missed those who could not join us, we did a lot of very difficult work and valued the opportunity to do it together (we also danced some, and cried some, and enjoyed some the lovely and welcoming place we were in -- thank you Portland and our Oregon hosts!). We had two specific areas of work to accomplish during the meeting: 1) We reviewed the four draft continua and made recommendations for refinements. Were those summary statements the "right" ones according to our reading of the data? Could we return to the data and find "behavioral anchors" -- specific, literal descriptions of behavior in performance -- to support the summary statements? Did the developmental sequence of summary statements along each dimension make sense? 2) We discussed and evaluated preliminary materials for the other six Standards for which teachers had submitted data, and made recommendations for additional rounds of fieldwork as well as for beginning construction of draft performance continua. We also took time to acknowledge and celebrate the hard work and learning that had happened through the year, to evaluate this round of field research that was coming to an end, and to offer suggestions about the content and process for the next round. And so we come to the 2001-2002 phase of EFF Field Development, in which we are continuing our partnership with OVAE/DAEL in a second year of the EFF/NRS Joint Data Collection Project. Since the September meeting we have further refined the four draft continua based on practitioner feedback and additional data analysis, and we have built a Performance Data Template for Use Math. In early November we brought together in Cincinnati, Ohio the teachers, administrators and State staff from our field development partner states who will be participating in the research this year. This was a great chance to catch folks up who were not able to be in Portland as well as to give everyone the opportunity to see and comment on what we had done since September. But the primary goal of the meeting was to prepare for the upcoming research. In 2001-2002 field development, we hope to · - Collect additional performance data, and develop draft performance continua, for the other six Standards on which we began work last year; · - Begin task development and data collection/analysis on two new Standards, Plan and Resolve Conflict and Negotiate . So our hardworking teacher/researchers chose which Standards they will work on for the next months and spent a good deal of time in the meeting in Standard-specific groups beginning to plan and construct EFF performance tasks for their learners. As these teacher/researchers are generating new data, the Consortium staff will be preparing the fruits of their earlier labors -- the draft continua and a range of performance tasks for Read, Write, Speak, Listen and Use Math -- for validation through a series of expert judging panels to be held in Winter and Spring 2002. Our aim, by the summer of 2002 and as a direct result of the amazing efforts of our field development partners, will be to produce · - Validated level descriptors for the NRS of these 5 EFF Standards; · - Rich definitions of performance to support new assessments for these 5 Standards; and · - Benchmark EFF Assessment Tasks that mark transition from level to level on the NRS for these 5 Standards. Our long-term, "big picture" aim, of course, is to build an Assessment Framework encompassing all of the EFF Standards that truly supports accountability for results that matter to learners, programs and funders. For a nice summary of where we are with that "big picture" -- and where we hope to go -- I invite you to read Sondra Stein's November 10, 2001 message to the NLA discussion list. So this is where we have come so far. I've used a lot of words to get here (I warned you!), but words can hardly express how much I appreciate the efforts of our development partners in this most challenging (often difficult, and almost always exciting) work. What a privilege it is to work with such fine, committed people! And a pleasure to share this update with the many more fine, hardworking EFF-ers on this list. So now, having shared, I'm wondering -- how do folks see (or not) the field development objectives and activities I have described here making a difference in your EFF practice? Where do you see yourselves in this phase of the EFF journey? While you're thinking about how to respond, I'll sign off. All the best to you, and again, Happy Thanksgiving!
This archive was generated by hypermail 2b30 : Tue Apr 09 2002 - 13:30:26 EDT