Return-Path: <nifl-assessment@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id j0GDjRn16077; Sun, 16 Jan 2005 08:45:27 -0500 (EST) Date: Sun, 16 Jan 2005 08:45:27 -0500 (EST) Message-Id: <4677BB4A.553EA46C.0004C68E@aol.com> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-assessment@literacy.nifl.gov Originator: nifl-assessment@literacy.nifl.gov Sender: nifl-assessment@literacy.nifl.gov Precedence: bulk From: HthKar@aol.com To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov> Subject: [NIFL-ASSESSMENT:859] RE: [NIFL-ASSESSMENT:848] Re: Skills Discussion fr X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=iso-8859-1 X-Mailer: Atlas Mailer 2.0 Status: O Content-Length: 3462 Lines: 31 Here's another interesting reference http://www.nfer.ac.uk/research/papers/DiagnosticAssess.doc I liked this as it was the first example I had seen of ACTUAL pieces of work that somebody has assessed using a particular tool against the standards. What is said to me initially was that if you are an ESOL student you get a much higher grade than if you are not, because my understanding is that some tutors are using a different rubric and working on the assumption that if there are 'any' mistakes then you have to fail a student at that level. So you can compare the rubric used in these worked examples, where for each bullet point (see previous contributions of mine on these) a student can be emerging, consolidating, established. For the rubric used in the assessment tool to which my previous email leads, one has simply to pass or fail students on each bullet point; ANY mistake in the - wait for it - 'skill' leads to a fail, and then an overall grade is awarded on the basis of a mark out of something. The IT ones seem to be set up so that you type a number into the machine and it prints out a pre-prepared statement. Probably this will tell you to go back and look at the work to see which bullets the student failed on, since obviously two students could obtain the same mark with a different profile of passes or fails. I recently had a long conversation with somebody who thought that once you had a curriculum with a long list of bullet points in it assessment against this was unproblematic. I was surprised how difficult I found it to explain why this was not the case. I took a deep breath and said, well to put it very simply, some markers are stricter than others. In some areas in this country, we go through standardistation procedures where we discuss particular pieces of work and come to an agreement about which level they should be put at. This is said to increase inter-rater reliability. But, this new set of assessments is being produced in an educational context which has believed that once you have specified standards precisely, assessment against them is unproblematic. But it's not. I have recently done some informal cross checking between results on one set of new tools and some others. It is possible for our new initial assessment to say a lad is at a particular level one week and then the lad to get a pass mark at a test three levels up a few weeks later. It is possible for a lad to seem to be awful at spelling given lists of incorrect and correct spellings and asked to pick out the correct, but when asked to write correct spellings a week or so later using the lists from the diagnostics, as opposed to those from the initial, he may come out several levels higher. And sometimes he may correctly spell to dictation words that he could not pick out from the confusing list including incorrectly spelled words. May be the lists of words confused him. You could say well in that case his spelling isn't really secure, but which of the two tasks seems a more valid predictor of his chance of spelling the word in any real life situation in which he might be required to do it?? And now some people are using lists of incorrect spellings and correct spellings to teach spelling. ... this is an unforeseen piece of washback/backwash. Regards K NB ... and no, I do not have simple perfect answers to all these difficulties..... But you asked for some links, so now you have had two.
This archive was generated by hypermail 2b30 : Mon Oct 31 2005 - 09:48:44 EST