Page 3: Evaluation of the Aboriginal Head Start in Urban and Northern Communities Program at the Public Health Agency of Canada

Executive summary

This report presents the findings of an evaluation of the relevance and performance of the Aboriginal Head Start in Urban and Northern Communities program at the Public Health Agency of Canada. It was conducted by Evaluation Services, Public Health Agency.

Evaluation purpose and methodology

The purpose of the current evaluation is to assess the relevance and performance of the Aboriginal Head Start in Urban and Northern Communities (AHSUNC) program. The AHSUNC program was last evaluated by the Centre for Health Promotion in 2005-06. The Financial Administration Act^{Footnote 11} and the Treasury Board of Canada Secretariat’s Policy on Transfer Payments require departments to conduct a review every five years of their grants and contributions programs, and the 2009 Treasury Board of Canada Secretariat’s Policy on Evaluation requires comprehensive evaluation coverage of all program spending on a five-year cycle.

Evaluation Services analyzed information from multiple sources including key internal documents, case studies, interviews and literature reviews.

The AHSUNC program

The AHSUNC program is an early childhood development initiative for Aboriginal preschool children and families living off-reserve. Through contribution funding provided by the Public Health Agency, Aboriginal community organizations design and deliver holistic programs to enhance the spiritual, emotional, physical and social well-being of Aboriginal children under six and their families. The program targets three distinct groups of Aboriginal people living in urban and northern communities: First Nations living off-reserve, Métis and Inuit. The AHSUNC program is delivered through the Public Health Agency’s Regional Operations through contribution agreements with project recipients (commonly referred to as “sponsors”). These are typically Aboriginal organizations.

The AHSUNC program began in 1994-95 and was initially designed as a four-year pilot program. It was renewed as an ongoing initiative in 1999-2000, with annual A-base funding of $24 million. Since then, there has been one ongoing enhancement provided in 2002-03 and two additional time-limited strategic allocations (monetary increases): $5M annually was approved in 2005-06 and again in 2010-11 which sunsets in 2015-16. In total, the AHSUNC program receives an annual allocation of approximately $41.3 million per year.

The AHSUNC projects are typically a centre-based preschool program for three to five-year-old children, running three to four half-days per week, nine months per year, between September and June.

The key features of the AHSUNC program are as follows.

• It is intended to reach vulnerable populations of Aboriginal children most at risk for poor early childhood development outcomes.
• It targets not only children, but also their families and communities.
• There is a strong emphasis on parental involvement in the program.
• Aboriginal culture and language are woven into the design and delivery of the program. The program is based on a holistic model that recognizes the multi-dimensional aspects of well-being for Aboriginal children.
• To the greatest extent possible, project staff are hired from within the Aboriginal community.
• The program is based on an empowerment model through which local ownership and decision making are encouraged and fostered.

The Public Health Agency, Health Promotion and Chronic Disease Prevention Branch, Centre for Health Promotion is responsible for AHSUNC programming and budgetary authority, as well as for ongoing monitoring of accountability and financial administration.

Each recipient administers one or more AHSUNC projects, called a site. As of 2011-12, the Public Health Agency has 94 contribution agreements with 91 recipients (three recipients hold two separate contribution agreements), operating a total of 128 AHSUNC project sites. The reach to preschool children has been fairly consistent over the last five years, ranging from 4,530 to 4,831 children per year.

Evaluation conclusions and recommendations

The analysis of the information gathered for this evaluation resulted in findings and conclusions about relevance and performance (summarized in Figure 1), which in turn led to six recommendations.

Conclusions

Relevance

There is a continued and growing need for the Aboriginal Head Start in Urban and Northern Communities program. There are approximately 47,910 Aboriginal children aged three to five years in Canada. The AHSUNC program typically reaches 4,640 children per year aged zero to six (86 per cent of whom are between the ages of three to five). Population growth rates are higher for Aboriginal people than for the general Canadian population, and a shift towards urban settings has increased the need for early childhood development programming off-reserve.

It is appropriate for the federal government and the Public Health Agency to administer the AHSUNC program. While provinces and territories have the primary responsibility for early childhood development, the Government of Canada plays a supporting role and can invest in an area of shared jurisdiction if there is a demand to address an issue of national scope, fill gaps for a vulnerable population and/or complement provincial or territorial directions. The AHSUNC program meets all of these criteria.

Performance

This program is highly regarded and appreciated by all groups involved in its delivery (communities, children, parents, project and Public Health Agency staff at all levels). The program has had a positive effect on school readiness, specifically in improving children’s language, social, motor and academic skills. Performance results have also demonstrated effectiveness in improving cultural literacy and enhanced exposure to Aboriginal languages and cultures. The program also has positive effects on health promoting behaviours such as children's access to daily physical activity as well as determinants of health, such as access to health services. Although the program's impact on parents and communities have not been consistently tracked, there is evidence that some project sites have become known as the community 'hub', creating a sense of community for Aboriginal children and their families.

However, there are areas for improvement. There are gaps related to knowledge development and exchange and a need for more coordination and collaboration with other federal departments, provinces and territories as well as other stakeholders. In addition, the program has the potential to reach more children.

The evaluation found that the program has not been subject to a comprehensive review of its design and delivery. Moreover, different sources indicated that it is timely to review the advisory function that supports the implementation and governance of this program as it has not been critically assessed since the program’s inception; the program could benefit from renewal of this function when discussing its strategic direction.

The program has benefited from the dedication of many committed personnel. Interview findings indicate that, although staff turnover is a challenge in many project sites, there are many dedicated individuals who have remained with the program from the start and have been pillars in the implementation of AHSUNC.

Recommendations

As mentioned above, this program has been in existence for more than fifteen years. It is therefore timely to discuss its future direction and develop strategies to address issues that could improve performance. The findings and conclusions point to the following six recommendations.

Reach

It appears that the program would benefit from a systematic review of AHSUNC site locations. While some project sites have waiting lists for enrolment, others are not consistently at full capacity, suggesting that program reach could potentially be improved.

A variety of delivery models are currently used across the country in support of the AHSUNC program. Some models seem to be more conducive to partnership development, resource leveraging and increased reach. Opportunities exist to increase partnerships and leverage resources through some of these delivery model designs.

Coordination and collaboration

The landscape of federal and provincial/territorial investments in early childhood development programming and research has shifted considerably since AHSUNC began. To align with this evolving context to capitalize on knowledge advances, as well as contribute AHSUNC knowledge to influence other programs, this evaluation identified the need to increase coordination and collaboration at a national level.

National coordination among stakeholders in Aboriginal early childhood education was identified as a strategic gap. Filling this gap could increase the program’s potential to achieve several intended outcomes, and perhaps most importantly, could ease Aboriginal children’s transitions into mainstream education systems.

Program outcomes related to knowledge development and exchange, support to parents and families as well as collaboration with early childhood development programs have not been systematically assessed. There is a need to explore the conditions that promote desired outcomes (e.g. what contexts and factors promote parental involvement and what impact does this have on child and family outcomes). As a result, there is a lack of information about what works for whom and under what conditions — in other words, knowledge that would deepen our understanding of the full potential and impact of this program.

Children’s long-term outcomes

Pilot studies on children’s long-term outcomes have been conducted in some provinces. A national study has not taken place. To inform program improvement and subsequent program evaluations, data on long-term outcomes for children could be helpful. However, longitudinal studies are resource intensive, therefore consideration must be given to the feasibility of such an approach.

Performance measurement

There is a need to develop an approach to performance measurement that considers all of the program’s intended intermediate and long-term outcomes, so that data collection efforts can be streamlined and periodic studies can be introduced to deepen the understanding of the full potential of this program.

This program has invested effort in implementing performance studies that have demonstrated that AHSUNC has a positive impact on participating children’s social, motor, language and academic skills as well as cultural literacy.

Extensive performance data have also been collected on community-level activities, processes and reach, however this information is not being disseminated to stakeholders in a timely manner and has not always been collected using a common framework.

Advisory function that supports the governance structure

The National Aboriginal Head Start Council was originally formed to provide advice to the Public Health Agency. However, internal and external interviewees expressed concerns with the current role of the Council. In addition, the current composition of the Council raises concerns about potential conflicts of interest as the members of the Council are direct recipients of project funds. Although no actual instances of conflict of interest were observed, there was no evidence of mitigating strategies in place for this potential.

National Strategic Fund

Over the last five years, implementation of the National Strategic Fund has been challenging. The evaluation found that the design of the National Strategic Fund may not be conducive to the selection of innovative initiatives that are national in scope aimed at improving programming for AHSUNC children and families. This time-limited funding was renewed recently for the next five years, until 2014-15.