Return-Path: <nifl-assessment@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id iANF1uQ26556; Tue, 23 Nov 2004 10:01:56 -0500 (EST) Date: Tue, 23 Nov 2004 10:01:56 -0500 (EST) Message-Id: <681A95205B5ACB4AAD697401486AE7120471B2@hal9000.lvgh.prv> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-assessment@literacy.nifl.gov Originator: nifl-assessment@literacy.nifl.gov Sender: nifl-assessment@literacy.nifl.gov Precedence: bulk From: "George Demetrion" <george.demetrion@lvgh.org> To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov> Subject: [NIFL-ASSESSMENT:754] FW: Re: The problem situation (reading X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Transfer-Encoding: 8bit Content-Type: text/plain; Status: O Content-Length: 7011 Lines: 35 Karen, First, thanks for that very insightful reflection! I had taken a scan through that British document and note that in places it states that the standards are not meant to uniformly apply to each student, but were guidelines. I'm paraphrasing as I don't have the document with me. Perhaps that's something that should be tossed back to the assessment specialists. No doubt in the quest for scientific (and also political) legitimacy, the assessment specialists are seeking as close to exactness as possible-hence, the tight correlations between the standards and assessment measurements based on staying within the parameters of the levels. The critique that you have of the levels parallels those of Tom Sticht. See, in particular his article, "The International Adult Literacy Survey: How Well Does It Represent the Abilities of Adults? In the Canadian Journal for the Study of Adult Education, Vol 15 # 2, November, 2001, 19-36. The scientific-oriented philosopher Karl Popper critiqued any view of science that sought a greater exactitude than what was feasible for any study under investigation. What he sought was rigorous analysis of relevant data and theories drawn upon to explain a given phenomenon and identifying a problem worthy of solution. When it comes to an understanding of adults learning to read and to draw on what they are learning for various purposes related to their lives outside the program, complexity may be more of the norm in which any valid scientific study would need to grapple with. We might have to give up a too easy allure of "objectivity" which a mathematical metaphor purports to provide for more discerning portraits of adult literacy learning. To get beyond ethnography, individual and collective portraits can be compared to each other which a scientific approach to the subject would demand. Generalities may be plausible, but they have to be earned through the crucible of well focused research through a broad community of informed and/or inquisitive researchers. I conclude with Popper's 6-step model of scientific research, something worthy of our consideration for research on adult literacy: (1) A clear exposition of the problem-, or if the problem may be assumed to be well known, a clear reference to it and to an exposition of it...[including any clarifications of] the always shifting problem situation (italics in original). (2) A more detailed survey of the relevant hypotheses bearing on the problem (and of the experiments bearing on the hypotheses, indicating the degree to which these are able to contribute to the appraisal of the hypotheses). (3) A more specific statement of the hypothesis (or hypotheses) which the author intends to propose, or to discuss, or to test experimentally. (4) A description of the experiments and the results. (5) An evaluation: whether the problem situation has changed; and if so, how. (6) Suggestions for further work arising from the work reported (Popper, Realism and the Aim of Science, 1956/1983, pp. 50-51). In short, the imperative is to be as exacting as the data and problem situation allows, and not one iota less or more so. The goal on this interpretation is not so much prediction, but understanding, illumination, and, where appropriate, application within the context of the contingencies of field experience. No doubt, we have a long way to go, on either side of the Atlantic before such a complex model of adult literacy research informs normative policy orientations and models of assessment and evaluation. Such a shift will require a politics that is willing to support adult literacy well before anything remotely representing "all" the data is in. This is a profound problem that does not negate the importance of continuing the quest for such understandings of adult literacy education as Popper's model, among others opens up. For if we surrender this, we surrender the legitimacy of the field to articulate its own issues on its own terms. Such a sacrifice is worthy of a counting of the costs as is the price to be paid for continuing the struggle. George Demetrion ----- Original Message ----- From: HthKar@aol.com Sent: Tuesday, November 23, 2004 4:31 AM To: Multiple recipients of list Subject: [NIFL-ASSESSMENT:752] Re: The problem situation (reading However, though it may look from our standards as if solutions have been found to squaring all sorts of circles, I do not think that in practice this is the case. A number of assessment 'instruments' has been devised based on the standards OR curriculum (it is important to realise that these are not the same, having been written by different sets of people for different agencies) and I think that there is room to doubt whether the results of assessments based upon these tools would correlate. For example, it could be that a person who has passed a summative test which operationalises the standards in one way would come out as being at a lower level when assessed using the 'screening instrument'. Further, I think that much of the final 'curriculum' is, and this is not intended as a criticism of the work of those who completed it, no doubt under severe pressures of time etc, format driven. I personally question for example a progression ladder that says that learners should be PREVENTED from writing complex sentences till they have mastered compound ones on the grounds that all sorts of things which native speakers would naturally say are much harder to say if not impossible without subordinating conjunctions Example I had something to eat because I was hungry. You would have to make sure that they said I was hungry and I had something to eat, leaving the 'causation' bit to be implied. Further, how do you explain to a learner who is deemed by virtue of being at a certain level incapable of grasping the nature of a complex sentence that they are not supposed tobe attempting them. ALso, the thing is underpinned by as far as I can see 'mastery' expectations, ie onl! y one or two minor errors are allowed. We are expected to teach something small and precise till it is learned, and this fits in with the idea of small steps for learners who find things difficult, but there is a risk of wholly distorting the nature ofthe subject. I should also point out that the same standards (though not the same curriculum) are used both for native speakers of ENglish as a first language and for what we call 'ESOL' speakers. I am not sure that the same basic progression ladders are appropriate for these learners. In fact I very much doubt it, but if I went for more training in teaching ESOL in an attempt to find out I would be on a hiding to nothing since all the training now is in terms of applying the framework not asking what the empirical or theoretical basis of it is. You get trained to map bits of language to the curriculum for audit purposes, basically. Some tools are used both with native speakers and with ESOL learners and this too I would question.
This archive was generated by hypermail 2b30 : Thu Dec 23 2004 - 09:46:24 EST