Return-Path: <nifl-assessment@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id fALFHB000079; Wed, 21 Nov 2001 10:17:11 -0500 (EST) Date: Wed, 21 Nov 2001 10:17:11 -0500 (EST) Message-Id: <001901c1729f$db792240$cd607b83@educ.kent.edu> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-assessment@literacy.nifl.gov Originator: nifl-assessment@literacy.nifl.gov Sender: nifl-assessment@literacy.nifl.gov Precedence: bulk From: "Dianna Baycich" <dbaycich@archon.educ.kent.edu> To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov> Subject: [NIFL-ASSESSMENT:57] Cross post from EFF list: update on EFF Assessment Field Development X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas X-Mailer: Microsoft Outlook Express 5.00.2919.6600 Content-Type: multipart/alternative; Status: O Content-Length: 19861 Lines: 408 This is a multi-part message in MIME format. ------=_NextPart_000_0016_01C17275.F25855A0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hi all, Thought this posting to the EFF list might be interesting in view of our = discussions about standardized vs. authentic assessment. Dianna ----- Original Message -----=20 From: McGuireP555@aol.com=20 To: Multiple recipients of list=20 Sent: Tuesday, November 20, 2001 5:59 PM Subject: [NIFL-4EFF:1930] update on EFF Assessment Field Development=20 Greetings, everyone, and best wishes for a Happy Thanksgiving 2001. This = is Peggy McGuire, checking in from the EFF Assessment Consortium team = with an update on our field development activities toward building an = assessment framework for the EFF Standards. This posting is rather long = -- you may want to save it for reading later if you're in a crunch -- = but I hope you find it interesting and useful! As you know from earlier 4EFF postings (see messages from Brenda Bell = and yours truly, in April, May and June 2001), the 2000-2001 phase of = EFF field development was defined by a new partnership with the U.S. = Department of Education, Office of Vocational and Adult = Education/Department of Adult Education and Literacy (OVAE/DAEL), = allowing us to explore how the development of the EFF assessment = framework might enrich the next generation of the National Reporting = System (NRS) mandated by Title II of the Workforce Investment Act. This = Joint EFF/NRS Data Collection Project has had as its purposes 1) to = enhance the level descriptors for the skills currently included in the = NRS (read, write, speak listen, numeracy) and 2) to allow the = measurement and reporting of skill gains on the other 11 EFF Standards. So as some 88 EFF Field Development teacher/researchers from 24 programs = in five partner states (Ohio, Oregon, Maine, Tennessee, and Washington) = were working hard to develop well-constructed EFF Performance Tasks and = document learner performance on those tasks throughout the year, their = research efforts were serving several related purposes: =B7 - to build a "continuum of performance" for each of ten EFF = Standards: Read with Understanding, Convey Ideas in Writing, Speak so = Others can Understand, Listen Actively, Use Math to Solve Problems and = Communicate, Solve Problems and Make Decisions, Take Responsibility for = Learning, Learn Through Research, Use Information and Communications = Technology, Cooperate with Others, and by doing so,=20 =B7 -To identify research-based performance descriptors for each NRS = level (both ABE and ESL) that will be useful to teachers in diagnosing = learner strengths and needs and in planning instruction, as well as = useful to programs and states for reporting progress, and =B7 -To identify a range of performance tasks that "benchmark" = transition points from level to level and that will enable programs to = report both what learners know and what they can do at each level. During the late spring and summer of 2001, Assessment Consortium staff = and Field Assistants reviewed and analyzed the data submitted by = teacher/researchers. We met in July to begin the process of constructing = tools based on that data for four of the Communications Standards (Read, = Write, Speak, Listen), including =B7 - A Performance Data Template for each Standard, on which we = attempted to range relevant data excerpts in sequence across a = developmental continuum for the dimensions of knowledge base, fluency = and independence, and to identify preliminary "transition markers" from = one "level" to another where possible; =B7 - A first draft of a Performance Continuum for each standard, built = from what we identified as possible transition markers and summary = statements of the behaviors/performances described in each Performance = Data Template; and =B7 - A "Task View" for each Standard, which illustrated the range = (types, and overall rating for complexity, context and knowledge base = requirements) of tasks that had been developed by teacher/researchers to = elicit the performance data that they had submitted. =20 We brought these tools with us to a final Project meeting for field = development participants, held in Portland, Oregon on September 12-15. = Despite the extraordinary and painful circumstances we found ourselves = in as a result of the events of 9/11, we decided to go on with the = meeting and were amazed that over half of the scheduled attendees were = able to attend. More than attend -- their focus and thoughtfulness were = astonishing, and while we sorely missed those who could not join us, we = did a lot of very difficult work and valued the opportunity to do it = together (we also danced some, and cried some, and enjoyed some the = lovely and welcoming place we were in -- thank you Portland and our = Oregon hosts!). We had two specific areas of work to accomplish during = the meeting: 1) We reviewed the four draft continua and made recommendations for = refinements. Were those summary statements the "right" ones according to = our reading of the data? Could we return to the data and find = "behavioral anchors" -- specific, literal descriptions of behavior in = performance -- to support the summary statements? Did the developmental = sequence of summary statements along each dimension make sense? 2) We discussed and evaluated preliminary materials for the other six = Standards for which teachers had submitted data, and made = recommendations for additional rounds of fieldwork as well as for = beginning construction of draft performance continua. We also took time to acknowledge and celebrate the hard work and = learning that had happened through the year, to evaluate this round of = field research that was coming to an end, and to offer suggestions about = the content and process for the next round. And so we come to the 2001-2002 phase of EFF Field Development, in which = we are continuing our partnership with OVAE/DAEL in a second year of the = EFF/NRS Joint Data Collection Project. Since the September meeting we = have further refined the four draft continua based on practitioner = feedback and additional data analysis, and we have built a Performance = Data Template for Use Math. In early November we brought together in = Cincinnati, Ohio the teachers, administrators and State staff from our = field development partner states who will be participating in the = research this year. This was a great chance to catch folks up who were = not able to be in Portland as well as to give everyone the opportunity = to see and comment on what we had done since September. But the primary = goal of the meeting was to prepare for the upcoming research. In = 2001-2002 field development, we hope to =B7 - Collect additional performance data, and develop draft performance = continua, for the other six Standards on which we began work last year; =B7 - Begin task development and data collection/analysis on two new = Standards, Plan and Resolve Conflict and Negotiate . So our hardworking teacher/researchers chose which Standards they will = work on for the next months and spent a good deal of time in the meeting = in Standard-specific groups beginning to plan and construct EFF = performance tasks for their learners. As these teacher/researchers are generating new data, the Consortium = staff will be preparing the fruits of their earlier labors -- the draft = continua and a range of performance tasks for Read, Write, Speak, Listen = and Use Math -- for validation through a series of expert judging panels = to be held in Winter and Spring 2002. Our aim, by the summer of 2002 and = as a direct result of the amazing efforts of our field development = partners, will be to produce =B7 - Validated level descriptors for the NRS of these 5 EFF Standards; =B7 - Rich definitions of performance to support new assessments for = these 5 Standards; and =B7 - Benchmark EFF Assessment Tasks that mark transition from level to = level on the NRS for these 5 Standards. Our long-term, "big picture" aim, of course, is to build an Assessment = Framework encompassing all of the EFF Standards that truly supports = accountability for results that matter to learners, programs and = funders. For a nice summary of where we are with that "big picture" -- = and where we hope to go -- I invite you to read Sondra Stein's November = 10, 2001 message to the NLA discussion list. So this is where we have come so far. I've used a lot of words to get = here (I warned you!), but words can hardly express how much I appreciate = the efforts of our development partners in this most challenging (often = difficult, and almost always exciting) work. What a privilege it is to = work with such fine, committed people! And a pleasure to share this = update with the many more fine, hardworking EFF-ers on this list. So = now, having shared, I'm wondering -- how do folks see (or not) the field = development objectives and activities I have described here making a = difference in your EFF practice? Where do you see yourselves in this = phase of the EFF journey?=20 While you're thinking about how to respond, I'll sign off. All the best = to you, and again, Happy Thanksgiving!=20 ------=_NextPart_000_0016_01C17275.F25855A0 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> <HTML><HEAD> <META content=3D"text/html; charset=3Diso-8859-1" = http-equiv=3DContent-Type> <META content=3D"MSHTML 5.00.2919.6307" name=3DGENERATOR> <STYLE></STYLE> </HEAD> <BODY bgColor=3D#ffffff> <DIV><FONT face=3DArial>Hi all,</FONT></DIV> <DIV><FONT face=3DArial>Thought this posting to the EFF list might be = interesting=20 in view of our discussions about standardized vs. authentic=20 assessment.</FONT></DIV> <DIV><FONT face=3DArial>Dianna</FONT></DIV> <DIV> </DIV> <DIV style=3D"FONT: 10pt arial">----- Original Message -----=20 <DIV style=3D"BACKGROUND: #e4e4e4; font-color: black"><B>From:</B> <A=20 href=3D"mailto:McGuireP555@aol.com"=20 title=3DMcGuireP555@aol.com>McGuireP555@aol.com</A> </DIV> <DIV><B>To:</B> <A href=3D"mailto:nifl-4eff@literacy.nifl.gov"=20 title=3Dnifl-4eff@literacy.nifl.gov>Multiple recipients of list</A> = </DIV> <DIV><B>Sent:</B> Tuesday, November 20, 2001 5:59 PM</DIV> <DIV><B>Subject:</B> [NIFL-4EFF:1930] update on EFF Assessment Field = Development=20 </DIV></DIV> <DIV><BR></DIV><FONT face=3Darial,helvetica><FONT size=3D2>Greetings, = everyone, and=20 best wishes for a Happy Thanksgiving 2001. This is Peggy McGuire, = checking in=20 from the EFF Assessment Consortium team with an update on our field = development=20 activities toward building an assessment framework for the EFF = Standards. This=20 posting is rather long -- you may want to save it for reading later if = you're in=20 a crunch -- but I hope you find it interesting and useful!<BR><BR>As you = know=20 from earlier 4EFF postings (see messages from Brenda Bell and yours = truly, in=20 April, May and June 2001), the 2000-2001 phase of EFF field development = was=20 defined by a new partnership with the U.S. Department of Education, = Office of=20 Vocational and Adult Education/Department of Adult Education and = Literacy=20 (OVAE/DAEL), allowing us to explore how the development of the EFF = assessment=20 framework might enrich the next generation of the National Reporting = System=20 (NRS) mandated by Title II of the Workforce Investment Act. This Joint = EFF/NRS=20 Data Collection Project has had as its purposes 1) to enhance the level=20 descriptors for the skills currently included in the NRS (read, write, = speak=20 listen, numeracy) and 2) to allow the measurement and reporting of = skill=20 gains on the other 11 EFF Standards.<BR><BR>So as some 88 EFF Field = Development=20 teacher/researchers from 24 programs in five partner states (Ohio, = Oregon,=20 Maine, Tennessee, and Washington) were working hard to develop = well-constructed=20 EFF Performance Tasks and document learner performance on those tasks = throughout=20 the year, their research efforts were serving several related = purposes:<BR>=B7 -=20 to build a "continuum of performance" for each of ten EFF Standards: = <I>Read=20 with Understanding, Convey Ideas in Writing, Speak so Others can = Understand,=20 Listen Actively, Use Math to Solve Problems and Communicate, Solve = Problems and=20 Make Decisions, Take Responsibility for Learning, Learn Through = Research, Use=20 Information and Communications Technology, Cooperate with Others</I>, = and by=20 doing so, <BR>=B7 -To identify research-based performance descriptors = for each NRS=20 level (both ABE and ESL) that will be useful to teachers in diagnosing = learner=20 strengths and needs and in planning instruction, as well as useful to = programs=20 and states for reporting progress, and<BR>=B7 -To identify a range of = performance=20 tasks that "benchmark" transition points from level to level and that = will=20 enable programs to report both what learners know and what they can do = at each=20 level.<BR><BR>During the late spring and summer of 2001, Assessment = Consortium=20 staff and Field Assistants reviewed and analyzed the data submitted by=20 teacher/researchers. We met in July to begin the process of constructing = tools=20 based on that data for four of the Communications Standards (<I>Read, = Write,=20 Speak, Listen</I>), including<BR>=B7 - A Performance Data Template for = each=20 Standard, on which we attempted to range relevant data excerpts in = sequence=20 across a developmental continuum for the dimensions of knowledge base, = fluency=20 and independence, and to identify preliminary "transition markers" from = one=20 "level" to another where possible;<BR>=B7 - A first draft of a = Performance=20 Continuum for each standard, built from what we identified as possible=20 transition markers and summary statements of the behaviors/performances=20 described in each Performance Data Template; and<BR>=B7 - A "Task View" = for each=20 Standard, which illustrated the range (types, and overall rating for = complexity,=20 context and knowledge base requirements) of tasks that had been = developed by=20 teacher/researchers to elicit the performance data that they had=20 submitted. <BR><BR>We brought these tools with us to a final = Project=20 meeting for field development participants, held in Portland, Oregon on=20 September 12-15. Despite the extraordinary and painful circumstances we = found=20 ourselves in as a result of the events of 9/11, we decided to go on with = the=20 meeting and were amazed that over half of the scheduled attendees were = able to=20 attend. More than attend -- their focus and thoughtfulness were = astonishing, and=20 while we sorely missed those who could not join us, we did a lot of very = difficult work and valued the opportunity to do it together (we also = danced=20 some, and cried some, and enjoyed some the lovely and welcoming place we = were in=20 -- thank you Portland and our Oregon hosts!). We had two specific areas = of work=20 to accomplish during the meeting:<BR>1) We reviewed the four draft = continua and=20 made recommendations for refinements. Were those summary statements the = "right"=20 ones according to our reading of the data? Could we return to the data = and find=20 "behavioral anchors" -- specific, literal descriptions of behavior in=20 performance -- to support the summary statements? Did the developmental = sequence=20 of summary statements along each dimension make sense?<BR>2) We = discussed and=20 evaluated preliminary materials for the other six Standards for which = teachers=20 had submitted data, and made recommendations for additional rounds of = fieldwork=20 as well as for beginning construction of draft performance = continua.<BR><BR>We=20 also took time to acknowledge and celebrate the hard work and learning = that had=20 happened through the year, to evaluate this round of field research that = was=20 coming to an end, and to offer suggestions about the content and process = for the=20 next round.<BR><BR>And so we come to the 2001-2002 phase of EFF Field=20 Development, in which we are continuing our partnership with OVAE/DAEL = in a=20 second year of the EFF/NRS Joint Data Collection Project. Since the = September=20 meeting we have further refined the four draft continua based on = practitioner=20 feedback and additional data analysis, and we have built a Performance = Data=20 Template for <I>Use Math</I>. In early November we brought together in=20 Cincinnati, Ohio the teachers, administrators and State staff from our = field=20 development partner states who will be participating in the research = this year.=20 This was a great chance to catch folks up who were not able to be in = Portland as=20 well as to give everyone the opportunity to see and comment on what we = had done=20 since September. But the primary goal of the meeting was to prepare for = the=20 upcoming research. In 2001-2002 field development, we hope to<BR>=B7 - = Collect=20 additional performance data, and develop draft performance continua, for = the=20 other six Standards on which we began work last year;<BR>=B7 - Begin = task=20 development and data collection/analysis on two new Standards, <I>Plan = </I>and=20 <I>Resolve Conflict and Negotiate .</I><BR>So our hardworking=20 teacher/researchers chose which Standards they will work on for the next = months=20 and spent a good deal of time in the meeting in Standard-specific groups = beginning to plan and construct EFF performance tasks for their=20 learners.<BR><BR>As these teacher/researchers are generating new data, = the=20 Consortium staff will be preparing the fruits of their earlier labors -- = the=20 draft continua and a range of performance tasks for <I>Read, Write, = Speak,=20 Listen and Use Math </I>-- for validation through a series of expert = judging=20 panels to be held in Winter and Spring 2002. Our aim, by the summer of = 2002 and=20 as a direct result of the amazing efforts of our field development = partners,=20 will be to produce<BR>=B7 - Validated level descriptors for the NRS of = these 5 EFF=20 Standards;<BR>=B7 - Rich definitions of performance to support new = assessments for=20 these 5 Standards; and<BR>=B7 - Benchmark EFF Assessment Tasks that mark = transition from level to level on the NRS for these 5 Standards.<BR>Our=20 long-term, "big picture" aim, of course, is to build an Assessment = Framework=20 encompassing all of the EFF Standards that truly supports accountability = for=20 results that matter to learners, programs and funders. For a nice = summary of=20 where we are with that "big picture" -- and where we hope to go -- I = invite you=20 to read Sondra Stein's November 10, 2001 message to the NLA discussion=20 list.<BR><BR>So this is where we have come so far. I've used a lot of = words to=20 get here (I warned you!), but words can hardly express how much I = appreciate the=20 efforts of our development partners in this most challenging (often = difficult,=20 and almost always exciting) work. What a privilege it is to work with = such fine,=20 committed people! And a pleasure to share this update with the many more = fine,=20 hardworking EFF-ers on this list. So now, having shared, I'm wondering = -- how do=20 folks see (or not) the field development objectives and activities I = have=20 described here making a difference in your EFF practice? Where do you = see=20 yourselves in this phase of the EFF journey? <BR><BR>While you're = thinking about=20 how to respond, I'll sign off. All the best to you, and again, Happy=20 Thanksgiving! <BR></FONT></FONT></BODY></HTML> ------=_NextPart_000_0016_01C17275.F25855A0--
This archive was generated by hypermail 2b30 : Fri Jan 18 2002 - 11:33:24 EST