Authors: April Zenisky, Lisa Keller, Stephen G. Sireci

Institutional affiliation/Agency/Program: Center for Educational Assessment, UMass Amherst

Publication date or date of most recent update: Spring 2004

Number of pages: 6

Website URL: http://sabes.org/resources/publications/adventures/vol16/16zenisky.htm

Type of product: training product/description of practice

Key words: Standardized tests, standardization, validity, reliability, scale scores, interpreting test results

Targets: Instructors, Administrators

Skill levels of instructional materials (lessons, curricula):
ABE Beginning Literacy (NRS Educational Functioning Level 1)
ABE Beginning Basic Education (NRS Educational Functioning Level 2)
ABE Intermediate Low (NRS Educational Functioning Level 3)
ABE Intermediate High (NRS Educational Functioning Level 4)
ASE Low (NRS Educational Functioning Level 5)
ASE High (NRS Educational Functioning Level 6)
ESL Literacy (NRS Educational Functioning Level 1)
Low Beginning ESL (NRS Educational Functioning Level 2)
High Beginning ESL (NRS Educational Functioning Level 3)
Low Intermediate ESL (NRS Educational Functioning Level 4)
High Intermediate ESL (NRS Educational Functioning Level 5)
Advanced ESL Literacy (NRS Educational Functioning Level 6)

Training required: The product itself does not require training, but can itself be used in training for practitioners interested in constructing or administering tests.

Abstract: The purpose of the article is to equip readers with a basic understanding of what goes into standardized test making and what test scores purport to show about learners' skills and abilities. The authors note that "any constructive use of this knowledge, whether it be better instruction or better policies" start with accurate knowledge of the subject matter. The article reviews important components of testing including validity and reliability, scale scores, and interpreting test results.

What the experts say: As the title suggests, this article provides a basic overview of standardized tests. The "interpreting test results" snapshots highlight the important issues.

This article provides a good overview of the basics but I think it could use a few additional pieces to be more relevant to the adult education community:

1. A reference to the NRS descriptors and levels would provide a very direct connection to the uniform framework (i.e. the NRS) that is being used nationally to evaluate local/state/national performance.
2. Though IRT can be confusing/technical, I think some brief overview of its fundamental concepts (e.g. the ability of the test taker is independent of the test content) may be important for teachers to understand. It can help to explain why a performance scale is needed, why scale scores and not raw scores should be used, and why scale scores are comparable across test forms.
3. The paper talks of standard error of measurement (SEM) but it fails to take the next step and address the important issue of selecting the "test difficulty level" that best matches the learner's ability. SEMs are typically smallest in the middle of fixed-form tests. A chart that displays the different tests and their wide ranging difficulty levels could visually show the relationship among (i) the overall scale, (ii) the ranges of the test, (iii) the SEMs, and (iv) the NRS levels. For an example of what is used in the CASAS system [the system I am most familiar with], please see
   https://www.casas.org/home/index.cfm?fuseaction=home.viewFile&MapID=1050.
4. A related issue is norm versus criterion-referenced measurement and also why grade-levels or GLEs are really not appropriate for discussing adult learner performance.
5. Computer Adaptive Testing is another piece that I think would be useful to include here. IRT allows for adaptive testing and we are seeing more of it. I think adult education teachers should definitely be aware of it.

I read the Basic Primer piece by Zenisky,Keller, and Sireci. This piece is very good and is exactly as advertised, a very basic primer. I think it is something that every adult ed teacher should read - even though it really sticks to the bare minimum in explaining reliability, validity, and interpretation of standardized test scores. Still, I think that most adult ed practitioners (and teachers in general) really could use a review of these assessment basics. It wouldn't hurt to add some references if teachers want to learn more - starting with the 1999 Standards for Educational and Psychological Testing.