Hello Tom:

I think I see the problem here.  Thank you for your explanation of the 
pragmatics of the situation--of using non-cognitive/cognitive as a 
distinction.  But bear with me here a moment:

If I understand correctly, you say:

"ABE programs frequently produce outcomes that are not captured by the 
cognitive tests of reading or mathematics."

And so, (if I am correct) you are equating those uncaptured outcomes with 
what you mean by non-cognitive--because the said reading and mathematics 
tests are referred to as "cognitive tests" which do not ask the kinds of 
questions that would capture such outcomes.

Then you say:  "These non-cognitive test outcomes" (presumably produced by 
current ABE program testing) "are nonetheless likely to indicate returns to 
investment (ROI) that are not usually cited when studies of the economic 
value of ABE are reported. Hence, ABE is generally undervalued in such 
studies."  So that test results are separated in this way?

(I want to be as brief as I can on this.)  With that brevity in mind, I ask: 
Wouldn't it follow, then, that referring to this "other" data collected 
through APA as non-cognitive (besides all of the other reasons we have 
explored here not to use the term non-cognitive) actually feeds into the 
presumption that such data is not worthwhile capturing as having effect on 
ROI?

As also cognitive (which it is) but "other," (remote, affective, deeper 
learning, identity-development, etc.,)--that WE KNOW greatly influences more 
proximate learning, e.g., math and reading skills--wouldn't it be 
better-argued and make more sense (to policy-makers, etc.) in ROI decisions 
(with the more "surface" learning that you say they draw from now) TO 
INCLUDE the more comprehensive human development that BOTH kinds of learning 
promote and reveal?

The whole idea of non-cognitive speaks to some fuzzy arena that no one can 
really understand, though we know it's there in all of us.  "It cannot be 
counted (like math), so it doesn't count."  So we're still going on that, 
even though .....?

As an example, I just ran across some research in a student paper that 
speaks to student motivation.  Now this is in K-12, but it relates here 
also--as example.   My student (the teacher) is researching why her fourth 
grade students are not motivated to learn.  Her focus is on how her 
colleagues praise their students.  (1) they praise their basic intelligence 
or (2) they praise their efforts.  She has discovered (through a 
well-developed literature review) that many students' motivation (or lack 
of) is directly tied to how they understand their own intelligence--either 
(1) already given, or (2) tied to their efforts. It seems her experience 
with many teachers is that they praise the former, and ignore the latter. 
So briefly, what her students presently believe, or have LEARNED, whether 
true or not, influences their more topical learning (math, etc., or surface, 
proximate, other-object oriented). But as learned, such motivation is tied 
to a cognitional event, and not a non-cognitional event.

In cognitional terms, in this case of motivation, what her students LEARNED 
earlier (cognated) about their own intelligence, either directly or 
indirectly and regardless of WHEN they learned it, became their foundational 
assumption about it.  That assumption, in turn, set the stage for their 
attitude about learning other more proximate material--it is motivational 
and it is learned (cognitional).  It's not that we cannot locate personality 
traits that we are born with, but that the division between what we are born 
with, what is learned, and how what we are born with is or is not developed, 
is way out of place with such applied language--such as non-cognitional to 
such cognition-affected issues.

In view of that kind of layered cognitional process of (affective, or remote 
and ranging, self-subject-development as distinct from, but as always 
underpinning, more proximate learning) it seems really short-sighted NOT to 
include both kinds of cognitive events and their influences in ROI--if we 
are thinking about anything but very short-term learning outcomes as 
distinct from more comprehensive human development--not to mention our views 
of what education is.

Lastly, you say:  "These are some of the reasons, along with implications 
for instruction, for focusing on cognitive and non-cognitive education in my 
workshop on Adult Literacy: A Focus On Cognitive and Non-Cognitive Skills 
and Behavior With Children's Picture Books By Leo Lionni. I would appreciate 
any discussion on the Assessment list which could advance the use of 
non-cognitive assessments that could bolster the ROI argument for ABE."

My argument is that using that term is not going to bolster the argument for 
(rightly) focusing on such content.  Rather, it will only help solidify the 
sense (in the minds of policy-makers) that this content is not worthwhile 
pursuing--it's fuzzy and "touchy-feely," and if its non-cognitive, how can 
we **know** it or influence it in education--which is about learning?

We need to educate the policy-makers to a better and more comprehensive 
notion of what education means and how focusing funding on short-term ROI 
not only steals the golden eggs (uses tax-payer money), but can kill the 
goose along the way (short-term support commonly yields short-term gains), 
which your (Tom's) own research shows, if I remember correctly, often has 
little or no staying power.

If you've read this far, thank you for listening,


Catherine King
Adjunct Instructor
Department of Education
National University
San Diego/Huntington Beach, CA

----- Original Message ----- 
From: <tsticht at znet.com>
To: <assessment at nifl.gov>
Sent: Wednesday, October 07, 2009 1:53 PM
Subject: [Assessment 2193] ROI and Non-Cognitive Measures of Outcomes in ABE


Colleagues: A large part of my interest in non-cognitive measures in ABE has
been based on the many surveys of adults, both qualitative and quantitative,
which have indicated that ABE programs frequently produce outcomes that are
not captured by the cognitive tests of reading or mathematics. These
non-cognitive test outcomes are nonetheless likely to indicate returns to
investment (ROI) that are not usually cited when studies of the economic
value of ABE are reported. Hence, ABE is generally undervalued in such
studies.

For instance, in many studies adults report that they feel more
self-confident after attending an ABE program. I have read that  frequently
adults who lack self-confidence will enter therapy at an average of about
$150 dollars per hour. If they attend these sessions for just to hours it
would cost $300  to have their self-confidence improved. That is more than
the average federal funding per enrollee in ABE in the WIA, Title 2, Adult
Education and Family Literacy Act as presently funded. Self-confidence is
widely considered as a non-cognitive, affective aspect of a person's
psychological make-up and it appears to improve for free in ABE.

Another outcome I have found in research in ABE is that frequently when
adults enter ABE they later report that this has a positive influence on
their own and their children's attitudes about school. For instance, in one
research project with Wider Opportunities for Women, the adult students, all
mothers on welfare,  reported at the end of their program that 40 percent of
their children "liked school better." These types of changes are similar to
those reported in Head Start and other types of pre-school programs, which
typically cost in excess of $7000 per child. They occur often in ABE
programs for free. But this type of non-cognitive, "likes school better"
motivational outcome is generally not assessed in ABE programs, so the
programs do not get credit for this important outcome in computing ROI to
ABE.

These are some of the reasons, along with implications for instruction, for
focusing on cognitive and non-cognitive education in my workshop on Adult
Literacy: A Focus On Cognitive and Non-Cognitive Skills  and Behavior With
Children's Picture Books By Leo Lionni. I would appreciate any discussion
on the Assessment list which could advance the use of non-cognitive
assessments that could bolster the ROI argument for ABE.

Tom Sticht


-------------------------------
National Institute for Literacy
Assessment mailing list
Assessment at nifl.gov
To unsubscribe or change your subscription settings, please go to 
http://www.nifl.gov/mailman/listinfo/assessment
Email delivered to cb.king at verizon.net