As Tom points out, there are several ways to measure the literacy skill 
levels of people living in a community. The phone survey is inexpensive and 
probably provides sufficient information. The NALS/NAAL test is the most 
expensive approach but does have the value of an internationally accepted 
standard.  A community might be able to lower cost of a NALS/NAAL 
assessment by following Tom's example of finding a local academic and 
graduate students to collect and analyze the data. Some of the standard 
tests of reading components seem to correlate with the NALS/NAAL scores, as 
well, and they offer an easier, and probably less expensive way to assess 
literacy skills.

Any of these approaches could measure change over time, but a positive 
change could not be attributed to the community literacy program, unless 
the the skills of a similar community that did not implement a community 
literacy program were also assessed over the same time period. This is only 
one of several hurtles that would have to be overcome to reliably assess 
the impact of a community literacy program by measuring changes in literacy 
skills. I can think of ways in which a focused research project might look 
at impact in this way, but I can't think of a way for each community to 
measure impact literacy skills in a reliable way.

However, any community literacy program could set goals and then measure 
whether or not those goals were met. The goals could be changes in literacy 
skills and practices, changes in behaviors that are considered supportive 
of literacy development(reading to children for example), knowledge of 
reading development (the information in Partnership for Reading booklet for 
parents for example), knowledge and attitudes of community leaders, use of 
the library, etc. Most of these could be measured through phone interviews 
or easily collected data.  With multiple measures of change, any weakness 
in one measure (assessing skills, for example) would be countered by the 
breadth of measures.

If community literacy were to become a movement, with many communities 
around the country taking the same basic approach and employing the same 
assessment of change, a positive outcome would be compelling. This would 
have the added advantage of providing a common framework in which 
communities could learn from each other's experience.

John Comings, Director
National Center for the Study of Adult Learning and Literacy
Harvard Graduate School of Education
7 Appian Way
Cambridge MA 02138
(617) 496-0516, voice
(617) 495-4811, fax
(617) 335-9839, mobile
john_comings at harvard.edu
http://ncsall.gse.harvard.edu