Return-Path: <nifl-4eff@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id h8GKMST05700; Tue, 16 Sep 2003 16:22:28 -0400 (EDT) Date: Tue, 16 Sep 2003 16:22:28 -0400 (EDT) Message-Id: <15b.24a47a77.2c98c9ce@aol.com> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-4eff@literacy.nifl.gov Originator: nifl-4eff@literacy.nifl.gov Sender: nifl-4eff@literacy.nifl.gov Precedence: bulk From: MWPotts2001@aol.com To: Multiple recipients of list <nifl-4eff@literacy.nifl.gov> Subject: [NIFL-4EFF:2547] Re: Assessment X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Transfer-Encoding: 8bit X-Mailer: AOL 5.0 for Mac sub 39 Content-Type: text/plain; charset="UTF-8" Status: O Content-Length: 5526 Lines: 185 I am posting this for Tom Sticht at his request. Meta Potts, Moderator 4-EFF List In a recent message posted on the NIFL EFF discussion list, Brenda Bell, Associate Director, Center for Literacy Studies responded to an earlier posted message and said: "I'm assuming that by '3 different types of assessment" you are referring to three separate RWU assessment tasks. However, there is another 'set of 3' that is important in understanding how assessment works in standards-based instruction: the 3 different times that we need assessment (rather than just different types of assessment). We need assessment before instruction (to help us decide what to teach and at what level). We need assessment during instruction (to monitor progress and guide further learning and instruction). Finally, we need assessment after instruction (to show learning outcomes, results). We use different forms of assessment for these different times, but all of these can and should be aligned with the standard in a standards-based instructional system. For the third purpose (assessing learning results), we need a standardized approach to assessment (to meet National Reporting System guidelines)." In work to develop a job-related reading program for the U.S. Army, our team used all three types of assessment that Brenda mentions. First, we used the USAFI Reading Test (adapted from the Metropolitan Achievement Test) which was mandated by the Army to measure reading in grade levels. This permitted us to determine the levels of reading of personnel and to select for the program those personnel reading at or below the 6th grade level. This corresponds to Brenda’s assessment before instruction to decide if the person should be assigned to the program or not. Importantly, we assessed personnel three times before deciding that they were truly reading below the 6th grade level and should be assigned to the literacy program. This multiple assessment reduced the number of personnel reading below the 6th grade level by some 40 percent! This results from the fact that most standardized tests that measure a wide range of skills are not very reliable at the lowest ends of the test. When assigned to the program, students were assessed both on their general reading using the USAFI test and on specially developed job-related reading task tests. These assessments were made of materials and tasks sampled from across several Army jobs and normed in terms of the grade level scale of the USAFI test. These tests were constructed response type tests to reduce errors of guessing that can occur with multiple choice tests. They were designed to measure pre and post course gains in job-related reading. Importantly, these job-related reading tests measured the generalization of job-related reading training from a specific job field, say cook, to reading in several job fields. Once in the literacy program we had a curriculum strand made-up of six modules of instruction for 1. reading tables of contents, 2. Reading indexes, 3. Locating information in the body of texts, 4. Following procedural directions, 5. Reading tables and graphs and 6. Completing forms used on the job. Each of these modules of instruction had pre and post tests to determine if students needed to take the instruction (pre test) and when the student mastered the instruction (post test). These correspond to Brenda’s assessment during instruction to monitor progress and guide further learning and instruction. Finally, at the end of the six week course, we administered different forms of both the USAFI general reading test and the job-related test to measure gain in general literacy and job literacy. This corresponded to Brenda’s call for assessment after instruction to show learning outcomes, results. Using this series of three tests we measured general literacy gain (USAFI), which is the most difficult to achieve in a brief period of time because it requires that learners make the types of scores that those who normed the general literacy test made after one or more years of instruction and practice outside of school. Still, the job-related program made as much or more gain in general literacy as did programs of general literacy that the Army had in place. The job-related tests that were normed in grade levels measured generalization within a more narrow domain of reading that that of the general literacy test, but broader than the module pre and post tests which measured learning in the more restricted domain of the specific job-related reading modules for cooks, clerks, and other jobs. Significantly, the gain on the job-related reading task tests in grade levels was 3 to 5 times that of the gain in general literacy. This indicates that it is important to measure more closely the domain of instruction that you are providing in order to more validly assess learning than is the usual case with nationally normed, standardized tests. For more information about the Army literacy program and its assessment system see my Functional Context Education (FCE) Workshop Resource Notebook available for free pdf downloading from http://www.nald.ca/fulltext/context/cover.htm (or www.nald.ca under Full Text Documents searched by S for Sticht.) The notebook presents theory of cognition and literacy, and it provides information about the Army program and the three types of assessment. Tom Sticht Tsticht@aznet.net
This archive was generated by hypermail 2b30 : Thu Mar 11 2004 - 12:14:45 EST