Return-Path: <nifl-assessment@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id i3B1REm28624; Sat, 10 Apr 2004 21:27:14 -0400 (EDT) Date: Sat, 10 Apr 2004 21:27:14 -0400 (EDT) Message-Id: <D95995DA-8B56-11D8-9394-00039381D39E@theworld.com> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-assessment@literacy.nifl.gov Originator: nifl-assessment@literacy.nifl.gov Sender: nifl-assessment@literacy.nifl.gov Precedence: bulk From: David Rosen <DJRosen@theworld.com> To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov> Subject: [NIFL-ASSESSMENT:498] Re: why "valid and reliable"? X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas X-Mailer: Apple Mail (2.552) Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed Status: O Content-Length: 2072 Lines: 43 Eileen, On Saturday, April 10, 2004, at 05:09 PM, Eileen Eckert wrote: > How can we measure what we say we're measuring, and not something > else, without separating or isolating what we say we're measuring from > everything else? Can you give a specific example of how to achieve a > valid assessment? And address reliability as well? In vocational education, for example, sometimes there are both basic skills (reading, writing, math and others) and specific job-related skills needed to perform a task or a complex set of tasks. It is common to assess these skills, not by a paper and pencil multiple-choice test, but by providing a set of tools and materials (typical of those needed to perform a particular job task or set of tasks) a set of measurable outcomes and acceptable standards of performance, and sometimes a set of conditions under which the task will need to be performed. The tools, materials and conditions are chosen to make the task as close to what one would need in an on-the-job context. Attention is paid to putting the task -- and the measure of performance -- in context, not to isolate it. The degree of validity of the assessment depends not only on the measure chosen, but also on getting the assessment materials, tools and conditions right. It's often complex and never perfect, but often it is right, it accurately predicts performance in real life contexts, and when that happens, over time there is a consensus that the assessment measures what it says it measures, that it is valid. If the same tools, materials, conditions and measures are used each time, without variation, it will also be reliable. Perhaps someone else could give an example from adult basic education or ESOL, perhaps using authentic assessment in writing. How are validity and reliability dealt with, for example, in writing portfolio assessment? Incidentally, Eileen, although others may wish to, I cannot pursue this further now as I will be away without access to e-mail for awhile. David J. Rosen djrosen@comcast.net
This archive was generated by hypermail 2b30 : Thu Dec 23 2004 - 09:46:14 EST