MessageYes, WV would "consider our mitigating circumstances" if we had
missed a few of our goals. But we would also get a warning about, "Money is
tight, and if you don't make your goals...."

We have made our goals, but sometimes completing the nagging and pleading it
takes to get the one last student in to post test (and succeed) has been
frighteningly close to the fiscal year deadline. We had a group of 5. Three
post-tested: 2 "improved a level" and 1 did not (even though he had actually
gained more CASAS points than the others). Therefore, our NRS score at that
level was only 40%. We needed something above 50%. We had to get another
student assessed!

One missing student had moved hundreds of miles away. The other had a new
job providing him many over-time hours. He finally came in on the last
possible day to post test and fell across the FFL line with a 1 or 2 point
gain. Our improvement rate for that level suddenly jumped from 40 to 60%!
That's silly (IMHO).

To complicate things a bit, other funders, such as United Way, look at these
statistics as well. Explaining levels and how some students can improve
quite a bit and not be a "success" is hard to do without sounding like we
are just whining. Fortunately, they have allowed us to use any 5 point CASAS
gain as a measure of success instead of using the NRS brackets. And if one
student comes up 20 points, we get 4 United Way credits! (On the NRS report
it counts only one gain in his entry level, though he may have crossed more
FFL lines.

Doesn't using using the different level brackets, instead of total points
gained by each individual, distort the results, even in larger groups? Why
are they used?

Since the statistics for small groups can be changed so drastically by even
one individual, why not pool the results from groups like ours? (Call the
pool "The Long Tail" if you must:
http://en.wikipedia.org/wiki/The_Long_Tail) The pool might give researchers
an idea of whether classrooms or one-on-one tutoring is most effective at
the lowest literacy levels.

So, if you have any influence, please try to persuade NRS to describe the
goals for small groups in terms that make more sense for our situation.
Right now we are like little kids clomping around in Mom's high heels. It is
hard to work that way, and very far from useful.

All of which is not to say that individual assessments are not quite helpful
for for spotting individual problems and successes. They do help.

Mary G. Beheler
Tri-State Literacy
455 Ninth Street
Huntington, WV 25701
304 528-5700, ext 156
  -----Original Message-----
  From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov]On
Behalf Of Rosemary Matt
  Sent: Tuesday, April 17, 2007 7:39 AM
  To: The Assessment Discussion List
  Subject: [Assessment 724] Re: What about us little guys
usingvolunteersforone-on-one tutoring?


  Good Morning Everyone!

  I am sorry to be so late joining the group.  I am one of the panelists,
Rosemary Matt.  Just recently, I accepted the position of NRS Liaison for
New York State so monitoring data and providing technical assistance to
programs in need is now my entire focus.  We also have a large contingent of
programs that provide service through a volunteer network in a one to one
tutoring arrangement.  Your concerns Mary regarding the inability for
programs such as these to meet performance benchmarks is shared by New York
programs as well.  Our state department has thoroughly considered the
population these folks serve and consider that to be a mitigating
circumstance when assessing their performance.  The value of these
organizations serving some of our lowest skilled readers is well known and
appreciated.  In a state the size of New York it is possible to absorb the
lack of educational gain increments from these agencies as they are balanced
by other programs serving students for whom gain is eminent.

  At the same time however we have worked closely with these programs and
their statewide leadership team to provide technical assistance in the area
of assessment.  As they learn more about the strategies and nuances that
evolve around the NRS accountability system, they are better able to show
whatever gain is possible from their students.

  As some of you are aware, New York also utilizes the program level Report
Card.  We attended the training two years ago that was provided by Larry and
his staff at AIR.  I would strongly recommend this training to any state
considering this accountability tool for programs.  We have made incredible
advances in terms of identifying high performing programs and targeting
those in need of technical assistance through our Report Card Rubric.  Marie
has posted three power points that I offer in training built around this
rubric.  To further support our volunteer programs, our state department has
chosen to rank these programs among themselves providing a homogeneous
category specific to their needs.  They are not measured against the cohort
of traditional adult education programs.

  Another strategy we have recently embarked upon is through our statewide
data system, we have introduced Collaboration Metrics.  Many students
working first with these volunteer programs while they are at minimal skill
levels will eventually move into traditional programming and continue to
succeed through the educational levels.  To ensure the volunteer programs
remain tied to the student's success, they are informed of the students
progress through the data system and can subsequently report on that gain as
well.

  These few methods of support have been well received by our volunteer
affiliates.  Hope they may give you and your state some thoughts for the
future.
  Rosemary



  Rosemary I. Matt
  NRS Liaison for NYS
  Literacy Assistance Center
  12 Meadowbrook Drive
  New Hartford, NY 13413
  315.798.1026


----------------------------------------------------------------------------
--
  From: assessment-bounces at nifl.gov on behalf of Sandy Strunk
  Sent: Mon 4/16/2007 5:58 PM
  To: The Assessment Discussion List
  Subject: [Assessment 723] Re: What about us little guys using
volunteersforone-on-one tutoring?


  Mary,

  I think you raise some valid concerns. When you’re working with a small
pool of students like this, aggregate statistics can be rather meaningless.
I would think the most helpful data for you would be individual diagnostic
reading assessment and progress monitoring data. Are you familiar with the
Adult Reading Component study and the work related to using reading
profiles? You might want to check out http://www.nifl.gov/readingprofiles/.
I’m wondering if the component reading assessments wouldn’t go a long way
toward “focusing” the reading instruction you offer based on each learner’s
profile. That doesn’t get you off the hook for NRS reporting, but it does
provide a mechanism for meeting the highly individual needs of your
learners. Just a thought.



  Sandy Strunk

  Program Director for Community Education

  Lancaster-Lebanon Intermediate Unit 13

  1020 New Holland Avenue

  Lancaster, PA 17601

  (717) 606-1873


----------------------------------------------------------------------------
--

  From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov] On
Behalf Of Mary Beheler
  Sent: Monday, April 16, 2007 5:03 PM
  To: The Assessment Discussion List
  Subject: [Assessment 721] Re: What about us little guys using volunteers
forone-on-one tutoring?



  We tutor adults. No children.

  Almost all our students are at Begining Literacy to High Intermediate ABE
level. Almost no high or even low adult secondary. At the secondary level we
only get the students that can't (or won't) tolerate study in a regular ABE
classroom. ESL instruction is done by a different organization, with paid
teachers.



  We net 20 to 25 students with more than 12 hours of study each year. We
are so small that sometimes an entire FFL will have only one student in it.
When that happens the only question is, "How many advanced a level: 0 or
100%?"



  We deal with students on a highly individualized basis. One may need to
learn to read again after having a stroke or a fever. Another may have
taught himself to sight read at a very high level, but neglected to teach
himself any spelling or writing skills. A high school graduate may not have
learned even his ABCs, for whatever reason. One or two students a year might
have an employment or higher education goal. (Then WV can't verify it, if
the student works or studies out of state.)  I can safely say that no two
students have been alike in the nearly ten years that I have been here.



  I genuinely *like* statistics and know they can be very useful, and don't
mind gathering data to be put in a bigger pool if what comes back is
helpful. However, if a level has only 3 students, is the data even
"statistically significant" if just 2 of them are available for both pre and
post assessment? 2 of 4?



  Some things are better seen by microscopes and others by telescopes. Right
now neither NRS nor CASAS seems especially useful at a local level. Maybe
all I need is to find out how to focus them. Maybe they should be just
trashed. They may cost more to use than they return in terms of time and
money and *stress*, on us and especially on our students.



  I'm from West Virginia, not Missouri, but, "Show me!" (Please.)



  Mary G. Beheler
  Tri-State Literacy
  455 Ninth Street
  Huntington, WV 25701
  304 528-5700, ext 156

    -----Original Message-----
    From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov]On
Behalf Of Marie Cora
    Sent: Monday, April 16, 2007 1:29 PM
    To: Assessment at nifl.gov
    Subject: [Assessment 714] Just joining us? Here's what you need to
know...

    Hi folks,



    A number of subscribers have just joined us and so I would like to give
them the necessary info for joining our discussion.  Please post your
questions and share your experiences now!



    View the archives at:
http://www.nifl.gov/pipermail/assessment/2007/date.html to get up to date
with the current conversation.



    See suggested resources at:
http://www.nifl.gov/lincs/discussions/assessment/07program_impr.html
(Scroll to the bottom!!)

    See more resources at:

    For the 3 power points from New York State/LAC, click on these links:

    http://www.nifl.gov/lincs/discussions/list_docs/ReportCardRubric.ppt

    http://www.nifl.gov/lincs/discussions/list_docs/RollingOutReportCard.ppt

    http://www.nifl.gov/lincs/discussions/list_docs/DevelopingDisseminatingR
eportCards.ppt

    Here are your prompts: add your voice!:

    * Do you use data in your program? What type? How? What have been the
results?

    * What information (data) would you like to track and why?

    * What data would you like to learn how to use?

    Thanks!!



    Marie Cora







    Marie Cora

    marie.cora at hotspurpartners.com

    NIFL Assessment Discussion List Moderator

    http://www.nifl.gov/mailman/listinfo/assessment

    Coordinator, LINCS Assessment Special Collection

    http://literacy.kent.edu/Midwest/assessment/




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.nifl.gov/pipermail/assessment/attachments/20070417/f9620dfb/attachment.html