Return-Path: <nifl-assessment@literacy.nifl.gov>
Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id g6Q12CX06709; Thu, 25 Jul 2002 21:02:12 -0400 (EDT)
Date: Thu, 25 Jul 2002 21:02:12 -0400 (EDT)
Message-Id: <200207260059.g6Q0xO331153@charger.oldcity.dca.net>
Errors-To: listowner@literacy.nifl.gov
Reply-To: nifl-assessment@literacy.nifl.gov
Originator: nifl-assessment@literacy.nifl.gov
Sender: nifl-assessment@literacy.nifl.gov
Precedence: bulk
From: John Sabatini <sabatini@literacy.upenn.edu>
To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov>
Subject: [NIFL-ASSESSMENT:164] RE: norm vs criterion
X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="US-Ascii"
X-Mailer: QuickMail Pro 1.5.4r2 (Mac)
Status: O
Content-Length: 5768
Lines: 109
Reply to: RE: [NIFL-ASSESSMENT:163] RE: norm vs criterion
True, more has probably been written about portfolio's as formative tools. However, there is a considerable history of their use as summative, accountability tools which continues to today.
Both Vermont and Kentucky have used Portfolio assessments statewide for accountability purposes. The studies concerning Vermont (Koretz and colleagues) are classics. You can look them up on ERIC, keywords portfolio assessments and Vermont.
A key issue with portfolios as summative tools and most performance assessments for that matters has been that expert scorers are needed and they need to be reliable - interrater reliability. It's just an expansion of the issue in essay grading on AP or other writing exams. As more automatized scoring of key features (often quite complex, human-expert-like scoring I might add) in performances are perfected, you'll see more and more simulation based and portfolio/performance tasks used in summative tests. Think of current National Board teacher portfolios or accreditation for architects, doctors, and other professionals. In those cases, the trend is away from the traditional multiple-choice response standardized, norm referenced test.
Despite your kind words, I'll keep this one curt.
Best,
John
John P. Sabatini
Dianna Baycich wrote:
>Hi,
>First off, your posts are not overly wordy but rather good explanations that
>are helping me gain a better understanding!
>What you said makes sense. My interpretation is that portfolios are
>formative and standardized tests summative - more or less anyway. Do you
>know if there is any imformation about the predictive validity of
>portfolios? Any studies? Or is it mostly anecdotal evidence?
>Dianna B.
>
>-----Original Message-----
>From: nifl-assessment@nifl.gov [mailto:nifl-assessment@nifl.gov]On
>Behalf Of John Sabatini
>Sent: Wednesday, July 24, 2002 10:04 PM
>To: Multiple recipients of list
>Subject: [NIFL-ASSESSMENT:162] RE: norm vs criterion
>
>
> Reply to: RE: [NIFL-ASSESSMENT:161] RE: norm vs criterion
>Hi again,
>
>I agree with the principle that a range or 'porfolio' of assessments is a
>sound strategy to meet the various goals or purposes of assessments
>(diagnosis, placement, formative instructional guide, selection,
>achievement, evaluation). And that is the key, knowing the goal or purpose
>before choosing an assessment type.
>
>I'm all for portfolio assessment, but I wouldn't jump to the conclusion that
>performance on any particular portfolio task generalizes to more than
>performance on an almost identical real world task. A portfolio task may
>only tell one how well an individual will do on other portfolio tasks. Is
>writing a letter to the gas company the same as writing a letter to the
>principal of your child's school, the same as writing a memo to one's boss?
>And what is portfolio assessment of reading ability? What would make it more
>or less authentic or real world? (I might add that not all tasks I've seen
>called portfolio are authentic/real world outside of a classroom context.)
>
>Predicting how any one will perform on a range of real world tasks is a
>tricky business for all assessments including portfolios. Before one can
>make the claim that a portfolio performance generalizes to other real world
>tasks, one has to conduct validity studies, observing how well people who do
>well on a portfolio assessment perform on similar tasks as they arise in
>real world situations. Predictive or concurrant validity studies of
>standardized tests typically check to see whether high scorers on a test are
>also the ones who perform well in the settings that demand those abilities,
>for example, that most successful college level students outperform
>non-successful or high school or middle school students on a test predicting
>college performance. Such validity studies don't guarantee that the test
>results tell us real world ability, but they provide the kind of evidence we
>should demand of any assessment that claims to predict performance in
>settings outside the assessmen!
>t situation.
>
>Portfolio assessments can inform instruction better than most standardized
>tests, since one often gets performance evidence that the learner and
>instructor can reflect on and uses as a guide to improving performance on
>the next try. That assumes that the portfolio is being used for the purpose
>of aiding instructon, not for outcome assessment.
>
>That was neither an endorsement of or against portfolio or standardized
>tests, just another overly wordy response to a post.
>
>Best,
>
>John
>
>
>
>
>RFC822 header
>-----------------------------------
>
> Return-Path: <nifl-assessment@literacy.nifl.gov>
> Received: from literacy.nifl.gov ([192.188.111.2])
> by mustang.oldcity.dca.net (8.11.6/8.9.3/DCANET) with ESMTP id g6PGMpY08213;
> Thu, 25 Jul 2002 12:22:51 -0400
> Received: from literacy (localhost [127.0.0.1])
> by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id g6PFkhX29424;
> Thu, 25 Jul 2002 11:46:43 -0400 (EDT)
> Date: Thu, 25 Jul 2002 11:46:43 -0400 (EDT)
> Message-Id: <NGBBJDHKPMGKJEBFMDKOAEKACDAA.dbaycich@literacy.kent.edu>
> Errors-To: listowner@nifl.gov
> Reply-To: nifl-assessment@nifl.gov
> Originator: nifl-assessment@literacy.nifl.gov
> Sender: nifl-assessment@nifl.gov
> Precedence: bulk
> From: "Dianna Baycich" <dbaycich@archon.educ.kent.edu>
> To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov>
> Subject: [NIFL-ASSESSMENT:163] RE: norm vs criterion
> X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
> X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0)
> Content-Transfer-Encoding: 7bit
> Content-Type: text/plain;
> MIME-Version: 1.0
>
This archive was generated by hypermail 2b30 : Fri Jan 17 2003 - 14:46:25 EST