Return-Path: <nifl-assessment@literacy.nifl.gov>
Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id i37N7qm24672; Wed, 7 Apr 2004 19:07:52 -0400 (EDT)
Date: Wed, 7 Apr 2004 19:07:52 -0400 (EDT)
Message-Id: <40748799.3050308@webster.edu>
Errors-To: listowner@literacy.nifl.gov
Reply-To: nifl-assessment@literacy.nifl.gov
Originator: nifl-assessment@literacy.nifl.gov
Sender: nifl-assessment@literacy.nifl.gov
Precedence: bulk
From: Jeri Levesque <levesqjr@webster.edu>
To: Multiple recipients of list <nifl-assessment@literacy.nifl.gov>
Subject: [NIFL-ASSESSMENT:480] 400+
X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Status: O
Content-Length: 13802
Lines: 241
Dear List,
Please excuse the first posting, my server was having a bad day and
after two Message Rejected notices I sent the rough draft rather than
the edited version. That's really bad on a literacy discussion list!
Mea Culpa,
Jeri Levesque
Laura,
I am the evaluation coordinator for Missouri's state literacy resource
center and local Even Start family literacy programs. I'm answering
Jane's call for evaluators to respond to your posting.
The current GPRA direction is aimed at performance outcomes and program
impacts. The evaluation model currently in vogue is the logic model.
The model comfortably accommodates performance outcomes. Your concern
about putting things in writing is valid in the current funding
environment. We're all aware that federal education programs that can't
demonstrate measured outcomes, as opposed to program activities
(outputs) are in jeopardy. One example is the Even Start program that is
in danger of being zero budgeted because while it consistently produced
outputs it did not produced enough outcomes to suit the administration.
At the same time, coming up with objective performance assessments to
document program success is trying.
Your topic; "I am currently trying to evaluate the success of a peer
education curriculum designed to motivate women to get breast and
cervical screenings" is one I am fairly familiar with as I did a series
of presentations for our BCC project here in Missouri. My target was to
change practitioners' knowledge and attitude about health literacy and
to improve their communication skills (outcomes centered on skills and
behaviors). I could do simple post session evaluations about content
knowledge but did not follow through to actually observe these folks in
practice.
Marsha Drew Hohn did some amazing work in Lowell, Mass in a family
literacy program, on the same subject. She published her work as a NIFL
Fellowship Report, I think you can order it from ed.gov/pubs to see what
measures she used.
The outcomes you want to consider are women's knowledge, attitudes,
skills, and behaviors associated with BCS. Is there a correlation
between knowing how important it is to do screening and then actually do
monthly BSE's and get regular mams and paps? That's the real connection
you're trying to document. As with all evaluation design, once you pose
your key questions about what happens because of your work, assessment
measures will start to fall in place. I'm sure the experts on this list
have plenty of suggestions.
Raising awareness (an knowledge and attitude outcome) is important but
getting changes in behaviors is challenging and in your practice, more
significant. There are many women who know monthly self breast exams are
important but they just don't get around to doing them. You want to
connect knowledge with action, or as you put it, "behavior changes."
Perhaps working with a clinic you could make a simple tool to track
women who attend your peer taught sessions and have them demonstrate to
the health provider a self exam (a skill outcome is doing a breast exam
correctly and actually going for a training is a behavior outcome). If
the clinic is willing to keep a confidential record for you that
includes number of correct demonstrations and number of learners who
have a check up, they could also keep records of early detection of
suspicious lumps and actual breast tumors. This would be a powerful
impact from your program.
You might move ahead by reading Marsha's outstanding report on Project
HEAL (it is online at NIFL but it's a pretty clunky set of PDFs) and
then giving her a call. Another suggestion is to go to the Health
Literacy conference in Canada this spring. Contact lchiarelli@cpha.ca
<mailto:lchiarelli@cpha.ca> for more information.
I've also included some web sites to help you to consider different
measures and evaluation designs.
As a pink ribbon wearing survivor, I would like to apply any suggestions
for your work to our efforts here in Missouri.
Jeri Levesque, Ed.D.
Associate Professor, Director, Webster Institute for Literacy
Webster University
St. Louis, MO
1. American Evaluation Association
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=1&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D57f5faff626e8334%26clickedItemRank%3D3%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.eval.org%252F%26invocationType%3D-%26fromPage%3DNSCPIndex2%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.eval.org%2F>
Devoted to the application and exploration of evaluation in all
its forms. ... Click here to go directly to the Evaluation 2004
website. ...
http://www.eval.org/
2. Evaluation Criteria from "The Good, The Bad & The Ugly: or, Why
...
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=1&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D57f5faff626e8334%26clickedItemRank%3D6%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Flib.nmsu.edu%252Finstruction%252Fevalcrit.html%26invocationType%3D-%26fromPage%3DNSCPIndex2%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Flib.nmsu.edu%2Finstruction%2Fevalcrit.html>
A easy to use guide for web evaluation. Lists evaluation criteria
with links to actual pages that illustrate each point. ...
Evaluation Criteria. ...
http://lib.nmsu.edu/instruction/evalcrit.html
3. evalwebs
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=2&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc9838eb%26clickedItemRank%3D13%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fgsociology.icaap.org%252Fmethods%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fgsociology.icaap.org%2Fmethods%2F>
... Resources for Methods in Evaluation and Social Research. This
page lists FREE resources for methods in evaluation and social
research. ...
http://gsociology.icaap.org/methods/
4. Resource Selection and Information Evaluation
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clickedItemRank%3D23%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Falexia.lis.uiuc.edu%252F%257Ejanicke%252FEvaluate.html%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Falexia.lis.uiuc.edu%2F%7Ejanicke%2FEvaluate.html>
Resource Selection and Information Evaluation. ... Resource
selection and information evaluation involves answering three main
questions. ...
http://alexia.lis.uiuc.edu/~janicke/Evaluate.html
5. WWW Virtual Library: Evaluation
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clickedItemRank%3D27%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.policy-evaluation.org%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.policy-evaluation.org%2F>
The World Wide Web Virtual Library: Evaluation. The Social
Policy-Evaluation area of the the World-Wide Web Virtual Library.
Links ...
http://www.policy-evaluation.org/
6. The Evaluation Center
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clickedItemRank%3D28%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.wmich.edu%252Fevalctr%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.wmich.edu%2Fevalctr%2F>
The Evaluation Center: Provides information about Center projects,
publications, staff, history, and support services, including the
Directory of Evaluators ...
http://www.wmich.edu/evalctr/
7. Practical Assessment, Research and Evaluation
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=3&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c5510%26clickedItemRank%3D29%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fpareonline.net%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fpareonline.net%2F>
... on-line journal provides education professionals access to
refereed articles that can have a positive impact on assessment,
research, evaluation, and teaching ...
http://pareonline.net/
8. Online Evaluation Resource Library
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=4&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc984211%26clickedItemRank%3D37%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Foerl.sri.com%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Foerl.sri.com%2F>
home, : Online Evaluation Resource Library. ... Welcome to OERL,
the Online Evaluation Resource Library. ... OERL has been serving
the evaluation community for 4 years. ...
http://oerl.sri.com/
9. Basic Guide to Program Evaluation
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=5&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc9849b7%26clickedItemRank%3D46%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.mapnp.org%252Flibrary%252Fevaluatn%252Ffnl_eval.htm%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.mapnp.org%2Flibrary%2Fevaluatn%2Ffnl_eval.htm>
Basic Guide to Program Evaluation Written by Carter McNamara, MBA,
PhD | Applies to nonprofits and for-profits unless noted
First-Timers | Library Catalog ...
http://www.mapnp.org/library/evaluatn/fnl_eval.htm
10. Rubrics
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=6&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c6acd%26clickedItemRank%3D52%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.ncsu.edu%252Fmidlink%252Fho.html%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.ncsu.edu%2Fmidlink%2Fho.html>
... Rubrics and Evaluation Resources. ** NOTE: Download ... you.
The form also provides for self-evaluation, as well as
teacher-evaluation. You ...
http://www.ncsu.edu/midlink/ho.html
11. evaluation cookbook contents
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=7&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc98555e%26clickedItemRank%3D63%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.icbl.hw.ac.uk%252Fltdi%252Fcookbook%252Fcontents.html%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.icbl.hw.ac.uk%2Fltdi%2Fcookbook%2Fcontents.html>
A practical guide to evaluation methods for lecturers, for
browsing online. A publication from LTDI. to ... INITIATIVE,
Evaluation Cookbook. ...
http://www.icbl.hw.ac.uk/ltdi/cookbook/contents.html
12. MEASURE Evaluation - Home Page
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=7&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D59bf6f74fc98555e%26clickedItemRank%3D64%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.cpc.unc.edu%252Fmeasure%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.cpc.unc.edu%2Fmeasure%2F>
Monitoring and Evaluation to Assess and Use Results. ...
http://www.cpc.unc.edu/measure/
13. User-Friendly Handbook for Project Evaluation:
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=8&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D57f5faff626eb9b2%26clickedItemRank%3D79%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.ehr.nsf.gov%252Frec%252Fprograms%252Fevaluation%252Fhandbook%252F%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.ehr.nsf.gov%2Frec%2Fprograms%2Fevaluation%2Fhandbook%2F>
User-Friendly Handbook for Project Evaluation: ... Table of
Contents & Introduction to the User-Friendly Handbook for Project
Evaluation: ...
http://www.ehr.nsf.gov/rec/programs/evaluation/handbook/
14. The Evaluation Exchange - at the Harvard Family Research Project
...
<http://search.netscape.com/ns/boomframe.jsp?query=evaluation&page=10&offset=0&result_url=redir%3Fsrc%3Dwebsearch%26requestId%3D4f4e6e365c7f9d%26clickedItemRank%3D92%26userQuery%3Devaluation%26clickedItemURN%3Dhttp%253A%252F%252Fwww.gse.harvard.edu%252Fhfrp%252Feval.html%26invocationType%3Dnext%26fromPage%3DNSCPNextPrev%26amp%3BampTest%3D1&remove_url=http%3A%2F%2Fwww.gse.harvard.edu%2Fhfrp%2Feval.html>
Harvard Family Research Project's free periodical, The Evaluation
Exchange, provides a forum for knowledge-sharing among evaluators,
policymakers, and ...
http://www.gse.harvard.edu/hfrp/eval.html
McREL and others have free publications (print) on evaluation - I have a
bunch of other resources that I'd be happy to share.
North Central Regional Education Lab Evaluation Reports
http://www.ncrel.org/re/mdade/
Schwerdtfeger, Jane wrote:
> Laura, your topic sounds really interesting. It would be a great
> topic to
> explore, and maybe we could get some evaluators or psychometricians who
> focus on educational assessment to comment on the discussion too.
>
> Jane Schwerdtfeger
>
>
>
>
This archive was generated by hypermail 2b30 : Thu Dec 23 2004 - 09:46:14 EST