ARCHIVED - Summative Evaluation of the Community Action Program for Children : 2004-2009

 

EXECUTIVE SUMMARY

Overview

The Community Action Program for Children (CAPC) is a Government of Canada initiative – an investment by the Public Health Agency of Canada (PHAC) in children’s health and well-being.

CAPC funds community-based groups and coalitions to develop and deliver comprehensive, culturally appropriate prevention and early intervention programs that promote the health and social development of children (0-6 years). In particular, CAPC engages and responds to the needs of vulnerable children and their families – those facing conditions of risk, least likely to participate in mainstream programming, or to have ready access to health services.

Focussed on the broader determinants of health, CAPC aims to alleviate health inequities, while striving to strengthen skills and capabilities of communities, parents and care-givers to take action on the health and well-being of their children and families.

The established objectives of CAPC are to:

  • Improve the health and social development of children and their families;
  • Increase partnerships and collaboration;
  • Increase the number of effective community resources and programs;
  • Increase recognition and support for communities at-risk, their needs, interests and rights;
  • Increase empowerment and knowledge of families and communities; and
  • Increase accessibility to culturally-appropriate and linguistically-sensitive programming.

Evaluation Context

Ongoing evaluation has been an integral aspect of program improvement and accountability for CAPC since its inception. This summative evaluation provides a comprehensive assessment of the program between the years 2004 and 2009 and identifies program weaknesses and strengths in order to inform ongoing program improvement. This summative evaluation will also fulfill an accountability requirement of the program’s Results-based Management and Accountability Framework (RMAF). CAPC’s summative evaluation focuses on seven RMAF questions, listed below, that address the issues of program relevance, impact and cost-effectiveness as outlined in the 2001 Treasury Board evaluation policy.

  1. To what extent do threats to children’s health persist?
  2. Is CAPC in line with current government objectives on children?
  3. To what extent is the program reaching families living in conditions of risk?
  4. How has the CAPC program contributed to healthy child development?
  5. To what extent is the program implementing a population health approach?
  6. To what extent has the program operated in a cost-efficient manner?
  7. To what extent has the CAPC program resulted in cost-savings / cost-avoidance to the health, social and justice system within Canada?

Methodology

The CAPC program design is comprehensive and complex. It is based on sound theory informed by extensive research in the areas of health promotion, child development, early intervention, and public health. This well-elaborated theory underpinning the program provides a detailed description of how program activities are expected to lead to program outcomes.

Suitably, the methodology used for this Summative Evaluation is a meta-evaluation, meaning multiple lines of evidence were assessed to tell the CAPC performance story.

First, these included a review, synthesis, and analysis of data from numerous national and regional evaluations and survey reports in order to present and triangulate observations and evidence of a relationship between CAPC activities, outputs, and outcomes. Second, to further substantiate the evidence underlying these linkages, a review of relevant research literature was completed. This literature review followed the program theory to explore evidence of linkages between CAPC activities and potential long-term impacts on the health and well-being of children and their families living in conditions of risk.

Program Design

CAPC is designed to address the risks that place the development of some groups of children at a disadvantage. The theoretical basis for CAPC design is drawn from a body of rich research literature on the risks to healthy child development.

The central tenet of the program is that the health and development of an at-risk child can be protected from the effects of conditions of risk by providing funding for early interventions that address the needs of the whole family. Evidence shows that such interventions produce greater family stability, and improve the child’s context for development. Furthermore, the underlying philosophy of CAPC is that community-based agencies are in the best position to allocate resources effectively because they are respected and well-placed within their communities to know the unique conditions of at-risk children and families among them.

Key Findings and Conclusions

Relevance

The evaluation reveals that CAPC remains relevant as a contribution to the strategic outcomes of both PHAC and the Government of Canada. Many threats to child and family health persist among various population groups in Canada. CAPC remains a viable mechanism in the Public Health Agency’s effort to reduce health disparities and strengthen public health, thereby contributing to more “healthy Canadians.” CAPC is also one of the means by which the Government of Canada fulfills its international commitments to the United Nations Convention on the Rights of the Child and the Millennium Development Goals.

Impact

Reaching Vulnerable Populations

A high proportion of families reached by CAPC are living in conditions of risk. Moreover, the majority of the parents and caregivers reached by CAPC are affected by multiple risks.

A review of 2008 CAPC participant data indicated that 54% of CAPC participants had total family incomes below the Low-Income Cut-Off (LICO) in their community, 24% were single parents, 26% had not graduated from high school and more than 13% self-identified as Aboriginal. Of those Aboriginal CAPC participants, 80% had incomes below the LICO, 52% had not completed high school and 48% were single parents.

Findings have also shown that CAPC reaches a high proportion of at-risk populations and that the proportion of vulnerable individuals represented among CAPC participants exceeds the rate at which these populations are represented in the general population. This evidence of successfully engaging a population with these characteristics is a critical finding in assessing the effectiveness of CAPC. Research has shown that early intervention is important for children growing up under such conditions as they often have higher rates of poor health, developmental difficulties, social, cognitive and behavioural problems.

Contributing to Healthy Child Development

An analysis of qualitative data collected from CAPC project participants and staff, indicates that CAPC has a positive impact on the health and social development of children.

The most frequently reported outcomes include:

  • Improved health and social development for children (48%)
  • Parental self-improvement and increased parenting knowledge and skills (50%); and
  • Enhanced community capacity (42%).

The evidence of CAPC impacts for children includes:

  • Increased social knowledge and competence;
  • Increased language and cognitive development; and
  • Improvements in behaviour and attention problems.

The evidence of CAPC impacts for parents includes:

  • improved parenting knowledge and skills;
  • Increased social support;
  • Enhanced parental empowerment;
  • Improved development of life skills; and
  • Enhanced knowledge and use of community resources. The evidence of CAPC impacts for community capacity includes:
  • Increased networking with other organizations; and
  • Increased opportunities for participant involvement.

These early impacts become more powerful when coupled with the research literature, which shows that such early childhood development initiatives can contribute in the long term to children’s general development, school readiness, educational performance and employment prospects later in life.

Implementing the Population Health Approach

CAPC projects are rooted in a population health approach to the promotion of public health. Projects demonstrate a high degree of partnership and collaboration with a wide variety of organizations. In a typical year, CAPC projects reported more than 6,600 different partnerships with an average of 17 partners per CAPC project. These partnerships are integral to the availability and quality of services provided to children and families living in conditions of risk. CAPC has a strong commitment to providing opportunities for participant involvement, which has had a positive impact on participants’ self-development. The population health approach, by targeting determinants of health, is consistent with a global emphasis on the need to increase health equity. In their efforts to acknowledge and respond to these determinants, CAPC projects have evolved a wide range of intervention strategies and partnership models relevant to their particular communities.

Cost-effectiveness

Economic modelling of CAPC demonstrates the potential cost savings and overall program impacts on the health, social and justice systems in Canada. An assessment of leading health economic literature illustrated that the potential cost savings of CAPC on select health outcomes such as: grade retention; special education; high school drop-out; community sentencing; institutional sentencing; obesity rate; and maternal depression far outweigh investment in the program.

The overall annual cost per participant for CAPC was $1,473, falling within the mid-range of the World Bank’s cost per participant in similar early childhood development programming. While affordability is an important criterion in deciding which program approach to adopt, the cheapest program is not always the best in fostering healthy child development and the most expensive programs do not always produce the best quality service. When the opportunities to develop certain abilities are missed early in life, later remediation is less effective or more costly.

National performance measurement data also illustrated that CAPC projects leverage more than $48 million in additional resources in a typical year including additional funding, volunteerism and in-kind resources from provincial/territorial, municipal and community partners.

Recommendations

Several recommendations flow from the findings of this report and will help to guide decisions on future program and evaluation planning. In particular, it is recommended that:

  1. CAPC continue monitoring threats to child health so as to inform implementation and to enable the program to adapt and respond to emerging health priorities.
  2. CAPC continue data collection to support planning, performance reporting and evaluation.
  3. CAPC undertake further study to identify most effective delivery models within the overall program design including:

    1. Review and identify core program objectives;
    2. Review and identify a typology of interventions and services; and
    3. Develop a national outcome evaluation strategy.
  4. Results of this Summative evaluation, including key findings, conclusions, and limitations of the analysis, be considered in the context of long-term decisions regarding future PHAC investments in child health.
  5. Findings on the economic benefits of early intervention and adequacy of current CAPC funding levels be considered in the context of long-term decisions regarding future PHAC investments in child health.
  6. Any future evaluation of CAPC be positioned to meet the mandate of the 2009 Treasury Board Policy on Evaluation.

Page details

Date modified: