Mary, 

 

Where did the valid score place the excellent student who was doing poorly?
Since we work from the scale score and not the number correct I do not know
how to interpret the doing so poorly.  When working with teachers and
students, we try to place an emphasis on the where the student falls on the
scale or NRS level.  In training with teachers, we always stress that
students are not a single level but students have range of strengths and
needs and we discuss about how to present this information to the student.  

 

I agree that a missed question does not tell us why the student was not able
to correctly answer the question; however,  by taking "multiple measures" of
student performance over time with a variety of formats tells me if the
student has the concept and if the student can transfer what is known to
different contexts. 

 

In the example cited below, in most exercises that use maps the X is also
marked with the words such as "you are here."  It would seem that the
student using "X marks the spot" and not "X you are here" would indicate a
reading/literacy problem.  As a teacher I would watch that student read
directions and try to perform tasks.  Does the student ask others to help
explain the task? Does the student ask me, the teacher, for help?  Observing
the student's behavior and patterns of work is part of the assessment
process, and from your other posts I think that is what you seem to do in
your program with such a variety of learners at vastly different levels. 

 

The last point you make about application forms when the student does not
intend to seek employment presents a challenge that we always face.  If not
application for work, all ESL students are faced without filling out forms
and as  a teacher I want to have students learn to "transfer" knowledge so I
think I might use your quote "learn to read and then read to learn."  That
is I would use the application form for work as a transition to other forms
and so filling out forms is the concept that I am teaching as well as
helping students understand that there are many forms that they will have to
fill out in English and forms have certain questions and vocabulary in
common.  That way there is a relationship between the assessment and what is
taught.  The context of the question is not as important as the ability to
read and fill forms.  I explain to students that the employment form for
them is not important but I would try and brainstorm with them when they
might need to help someone with an employment form.  

 

Since you mention using CASAS life skills test you might want to look at the
new tests CASAS has developed such as the Life and Work. If your state has
other approved assessments for ESL then you might look at those assessments
to see if you think it is a better match for your curriculum.  I have found
that CASAS Literacy tests and level A reading tests are very sensitive
instruments in tracking lower level student learning gains.

 

Dan Wann 

Professional Development Consultant  

Indiana Adult Education Professional Development Project 

 

dlwann at comcast.net  

 

 

 

  _____  

From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov] On
Behalf Of Mary Beheler
Sent: Tuesday, April 17, 2007 5:58 PM
To: The Assessment Discussion List
Subject: [Assessment 737] Data quality and usefulness

 

Does GIGO apply to the data all of us, big and small, are gathering?

 

The information gleaned from the CASAS Life Skills (or any other) assessment
tool can be useful for spotting problems and successes, but because of the
*multiple layers of skills involved in answering any one question* the
specific question(s) missed must be looked at very carefully. But, getting
the student feedback that makes looking carefully possible gets into
problems of assessment confidentiality.

 

Example: A excellent student may understand *everything* else about a
certain CASAS map question, but if he or she has heard the expression "X
marks the spot," and assumes X means the goal, not the beginning, this
question will be nonsense. I don't think the check-off sheet of demonstrated
skills specifies, "Knows that on *this* map X means, "Start here."  If the
student's tutor or I can't discuss a missed question with the student, how
will I discover that?

 

How do we know that the question a student answers or misses actually
assesses the skill the assessment manual tells us it does? 

 

I was scoring an answer sheet and was dismayed at how poorly a student was
doing, when I noticed I wasusing the math answers, not reading. I switched
to the correct set, and the student made even a worse score! Both scores
were in the "valid" range, too! How much confidence should I place in that
assessment?

 

Having the questions in a booklet and marking the answers to multiple choice
questions on a separate sheet may be a skill even somewhat advanced adult
students do not have.Because our student workbooks don't use multiple choice
questions, we have actually created lists of number-letter pairs to see if a
student can mark the letter in the appropriate column of each numbered row
on a separate answer sheet. That's all. (Did that after a strong level 2
student marked the answer sheet by page number, not question number, with
answers to 2 and 3 questions marked in the same row.)

 

Has anyone made an effort to see if the lower level literacy students want
to learn what the CASAS or other accepted NRS assessments wants us to teach?
Lots of our students are on SSI. They don't see the point in learning about
employment applications. That often means any question about employment
forms isn't important enough to take seriously, even if the ones about other
forms are.

 

Colleges have sense enough to make all freshman year classes pretty generic,
and leave the "major" study to later years. Is it useful for assessment of
beginning literacy to get so specific so soon? Whatever happened to "Learn
to read; then read to learn"?

 

We used to use the quick and unintimidating SORT-R for student assessment.
Even on that simple test almost all men missed word "dainty," no matter what
their reading level.  Does anyone know if any specific question(s) on the
assessments used for gathering NRS data is answered incorrectly by most
students at any one level? Or if a significant number of students in the
Laubach series misses different questions than students in Challenger or
Voyager, or another series?

 

Mary G. Beheler
Tri-State Literacy
455 Ninth Street
Huntington, WV 25701
304 528-5700, ext 156 

-----Original Message-----
From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov]On
Behalf Of Marie Cora
Sent: Monday, April 16, 2007 1:29 PM
To: Assessment at nifl.gov
Subject: [Assessment 714] Just joining us? Here's what you need to know...

Hi folks,

 

A number of subscribers have just joined us and so I would like to give them
the necessary info for joining our discussion.  Please post your questions
and share your experiences now!

 

View the archives at:
http://www.nifl.gov/pipermail/assessment/2007/date.html to get up to date
with the current conversation.

 

See suggested resources at:
http://www.nifl.gov/lincs/discussions/assessment/07program_impr.html
(Scroll to the bottom!!)

See more resources at:  

For the 3 power points from New York State/LAC, click on these links:

http://www.nifl.gov/lincs/discussions/list_docs/ReportCardRubric.ppt

http://www.nifl.gov/lincs/discussions/list_docs/RollingOutReportCard.ppt

http://www.nifl.gov/lincs/discussions/list_docs/DevelopingDisseminatingRepor
tCards.ppt

Here are your prompts: add your voice!:

* Do you use data in your program? What type? How? What have been the
results? 

* What information (data) would you like to track and why? 

* What data would you like to learn how to use? 

Thanks!!

 

Marie Cora

 

 

 

Marie Cora

 <mailto:marie.cora at hotspurpartners.com> marie.cora at hotspurpartners.com

NIFL Assessment Discussion List Moderator

http://www.nifl.gov/mailman/listinfo/assessment

Coordinator, LINCS Assessment Special Collection

http://literacy.kent.edu/Midwest/assessment/

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.nifl.gov/pipermail/assessment/attachments/20070417/344f3b37/attachment.html