ARCHIVED - Population Health Fund Evaluation 2008 Final Report

 

3. EVALUATION FINDINGS

3.1 Activities and outputs

3.1.1 Program Leadership

Program Leadership refers to the activities undertaken to build and maintain community and governmental support for the PHA. Because the PHF began in 1997, a lot of this work was done earlier in the evolution of this program. Thus the work in this area reviewed in this report may not be as extensive as it has been in the past.

a. Outputs: Networks, reports, training, and presentations

Networks

Alberta listed six provincial networks: Topics included health, food, social, and evaluation. The Manitoba/Saskatchewan region reported eleven networks. Four related to rural issues. Two addressed seniors’ issues, and two related to food security. Other networks focussed on community collaboration, poverty, and summer school planning. Quebec reported thirteen intersectoral networks: Four focussed on project selection and funding, and three on sustainable development. Six networks established by Quebec PHF projects included emergency, culture, food security, and/or elder isolation. The Atlantic region reported participating in two national networks (evaluation and obesity) and Atlantic networks on seniors health promotion, policy, mental health, injury prevention, summer institute, and rural health and seniors’ issues and elders abuse networks in Newfoundland and Labrador. The Northern Secretariat reported no involvement in intersectoral networks.

Reports

Alberta identified two regional evaluations, one summarizing projects from 2002 to 2005, and one on a common framework and indicators. Quebec identified a report of short- and medium-term results of healthy and sustainable communities projects, and a report on sustainable development, health determinants and public health analysis of three projects. The Atlantic Region identified five reports in 2005 - one on seniors funding, one a snapshot of the PHAC, a discussion paper, a report on inequity and chronic disease, and an evaluation report of projects ending in 2004 and 2005. They also identified an environmental scan related to injection drug use in 2006. In 2007, five reports were identified, one environmental scan related to mental health, one on healthy food, one on public health and the environment, and two related to sustainable development and health. In 2008, they identified a retrospective review covering 2001 to 2008, and one for the food security working group. Manitoba/Saskatchewan did not identify any reports during this time period. The Northern Secretariat identified only project specific reports.

Training

Alberta reported (a) seven teleconferences with PHF projects and (b) two face-to-face meetings with PHF projects. Saskatchewan reported a Health Promotion Summer School. Manitoba hosted three sessions on project evaluation and the population health approach. In Quebec there were two daylong forums presenting PHF funded projects, and a training and networking day organized by the Réseau québécois des jardins collectifs and Maison Quartier. In 2005 a provincial workshop was held in each of the four Atlantic Provinces to promote “Legacy: The Life Stage Approach and Seniors Funding in Atlantic Canada.” One English and one French Think Tank (TT) introduced the concepts of Sustainable Development and the PHF in 2007. Both TTs had representatives form all four Atlantic provinces. Five workshops on were held in 2008 on “Population Health Fund Atlantic Initiative: A Retrospective Review (2001-2008).” The Northern Secretariat reported no training events.

Presentations

Alberta reported at a PHAC Research forum for federal government employees in 2006. Saskatchewan and Manitoba reported presenting information to boards, community groups, and committees. Quebec reported three presentations in 2007 and two in 2008. All of these related to the development of healthy and sustainable communities. The Atlantic Region reported 20 presentations on topics including the life stage approach, seniors’ issues and funding, a PHF retrospective, and an environmental scan of mental health and mental illness in Atlantic Canada. No presentations were identified by the Northern Secretariat.

3.1.2 Delivery Management

The activities of consultants in delivery management include managing the solicitation process and providing support to recipients and potential recipients during proposal development and project implementation.

a. Activities

Although we do not have data regarding the activities of program consultants in delivery management, interview respondents were asked about some aspects of the delivery management processes. Respondents reported hearing about the availability of funding through several channels: a solicitation letter (21%), the Health Canada (or PHAC) website (18%), a Health Canada or (PHAC) consultant (21%), or another agency or organization (23%). Two reported they knew about it from previous experience, one of whom had formerly been a consultant at PHAC.

The 44 interview respondents were also asked to rate various aspects of the delivery management process. Almost all of them rated the application guidelines as either very clear (51%) or reasonably clear (36%). Most reported that the support received from PHAC consultants during proposal development was excellent (66%) or good (14%). Two rated the support as fair. Seven said they didn’t have information about support during proposal development. Respondents were even more positive about the support offered by consultants during project implementation, with 84% rating it as excellent, and 9% as good. One rated implementation support as fair, and one didn’t know.

(We) learned so much from the consultant. (She) lives and breathes for population that they were very helpful and ten health. (She) was very flexible... a great source of help....The support provided by (her) was phenomenal.

Il a eu une grande appréciation des échanges. Une bonne communication et un bon support.

Respondents described the consultants as knowledgeable and able to offer good advice (12 respondents), ten respondents commented that they were very helpful and ten commented positively on their availability. Seven appreciated the direct involvement of consultants either in site visits or events, and five descriv\bed them as supportive. Two mentioned flexibility/receptivity as a positive. The few negative comments included two that consultants were too rigid or restrictive, one that the consultant should have offered more guidance, and one commented positively on the regional consultants but said the Ottawa consultants didn't understand.

Respondents were also asked about any difficulties they may have had with the PHF processes. Most difficulties (identified by 12 respondents) revolved around the reporting requirements, both financial and quarterly reporting or paper work in general. On the other hand, four respondents commented positively on the reporting requirements. Five respondents identified difficulties with aspects of time lines around proposal development, funding approval or reporting. Four mentioned cash flow problems. Other difficulties appeared to be unique to specific projects.

Respondents were asked for suggestions to improve the operation of the PHF. The most common suggestions (12 respondents) was to offer longer term funding; seven respondents suggested fostering connections among project recipients for mutual learning; and three suggested that PHAC could provide more support with sustainability and dissemination.

b. Outputs

The outputs of delivery management activities are grants and contribution agreements. Among the 116 projects completed during this time period, 33 were grants and 83 were contribution agreements. All but one of the National-Directed projects were grants, whereas National-Solicited projects were all contribution agreements. Among Regional projects, 18% were grants.

The mean duration of projects was 25.5 months, and the mean allocation was $228,297. Contribution agreements had a significantly longer mean duration than grants (30 months vs. 14 months, t =-9.3, df=111, p<.001). There was no significant difference in mean allocation between grants and contribution agreements.

The total duration and allocation of projects at the Regional and National levels are presented in Table 3.1. There are significant differences among the groups in mean duration (F=21.98, df=2,113, p<.001) and mean allocation (F=23.45, df=2,113, p<.001).

Table 3.1 - Duration and Total Allocation by Region
Region Mean Duration of Project Mean Total Allocation
Regional 26 months $150,716
National-Solicited 35 months $312,217
National-Directed 13 months $394,688
Total 26 months $228,297

In addition to these 116 completed projects, 29 projects that received funds through the period from October 2005 to the present are not yet completed. These include seven National-Directed projects, two National-Solicited projects, and 20 Regional projects. Since we have only partial file review data for these ongoing projects, their data are included only where appropriate.

The file review was used to classify projects with respect to their stated objectives. Projects had a mean of 2.5 objectives. Almost three-quarters of the projects (74%) had program development as an objective; 60% had an objective of policy development; 59% were implementing a community-based intervention or program, and 22% were taking action to influence or change policy. Other objectives were mentioned by 40% of projects, most frequently knowledge exchange/transfer (22%), or partnership/collaboration (24%). National projects, solicited and directed, were significantly more likely to have program development as a goal (95%) than regional projects (61%) (X2= 17.59, df = 2, p<.001). Regional projects were more likely to have an objective of taking action to influence or change policy (30%) than national projects (9%) (X2 = 6.98, df=2, p<.05).

Most project sponsors were from the health (35%) or social service (28%) sectors. Other sectors were research/policy institutes (10%), education (9%), cultural (4%), recreation (3%), and environmental (3%). One sponsor was from the justice/law enforcement sector. National-directed projects were significantly more likely to have sponsors from the health sector (81%) than either national-solicited (33%) or regional (23%) projects. Regional projects were more likely to have sponsors from the social service sector (41%) than national-solicited projects (17%)projects. None of the national-directed projects had a sponsor from the social service sector (X2=29.64,df=4,p<.001).

About half of the sponsors were community not-for-profit organizations (52%). Other sponsors were educational institutions (13%), coalitions (12%), professional or membership associations (12%) or provincial/territorial governments or government agencies (5%). Six (5%) were international organizations, and one described itself as an advocacy organization. There were significant differences among the regional groupings (X2=14.33, df=2, p<.01). National-solicited projects were more likely to fund community not-for-profit sponsors (75%) compared to regional (51%) or national-directed (29%) projects. National-directed projects more likely had sponsors other than community not-for-profit or educational institutions (62%) compared to regional (37%) or national-solicited (8%) projects.

About half the projects targeted specific life stages: children and adolescence (16%), early to mid-adulthood (9%) or later life (23%). National-directed projects were significantly less likely to target a specific life stage (14%) than national-solicited (71%) or regional (51%) projects (X2=14.78, df=2, p<.001).

Projects were roughly evenly split between those that targeted community members directly (57%) and those that targeted intermediaries (43%). Again, there was a significant difference between national-directed projects of which 95% targeted intermediaries, and national-solicited (38% intermediaries) and regional (30% intermediaries) projects (X2=28.88, df=2, p<.001). Health professionals were the most common intermediaries, representing 58% of the projects that targeted intermediaries. Other common intermediaries included policy and decision makers (48%), community organizations and agencies (40%), and researchers (28%). A few targeted educators (8%).

Ten projects were funded that targeted official language minority communities, either English-speaking communities in Quebec or French speaking communities in the rest of Canada. Three of these are ongoing National-Directed projects, one targeting anglophones in Quebec and two francophones in other parts of Canada. Two are English projects in Quebec, three are French projects in Atlantic Canada, one is a French project in Manitoba and one is a French project in BC.

In line with the equity issues related to population health, about two-thirds of the projects (64%) targeted a vulnerable population. Of these, 22% targeted at-risk seniors, 15% at- risk youth, 18% ethnic minorities or immigrants, and 12% low-income populations. Other vulnerable populations addressed were aboriginal groups (6 projects), people with disabilities (4 projects), those with specific health risks or risk behaviours (6 projects), and groups described as marginalized (6 projects).

The file reviews identified which health determinants were being addressed by projects. This information was available in 111 of the files. Social support networks, the social environment and personal health practices and coping skills were the three most frequently addressed determinants (over 60% each). Income and social support, education, the physical environment and gender were addressed by between 40% and 60% of projects. The other determinants were addressed less frequently, but all were addressed by at least some projects. The mean number of determinants addressed by projects was 5.1. The details of determinants addressed are displayed in Table 3.2.

There were some significant differences among the regional groupings in which determinants were addressed. National-directed projects addressed education less frequently (29%) than the national-solicited (50%) or regional (63%) projects (X2=6.49, df=2, p<.05). National-solicited projects were less likely to address employment/working conditions (8%) than national-directed (29%) or regional (37%) projects (X2=7.10, df=2, p<.05).

TABLE 3.2 - Determinants addressed by projects
Determinant No. Projects addressing determinant % of projects addressing determinant
*Significant differences in the proportions among the regional groupings
Income and Social Status 46 41.4%
Social Support Networks 77 69.4%
EducationTable 3.2 - Footnote 1* 61 55.0%
Employment/Working conditionsTable 3.2 - Footnote 1* 33 29.7%
Social Environment 72 64.9%
Physical Environment 47 42.3%
Personal Health Practices/Coping Skills 75 67.6%
Healthy Child Development 26 23.4%
Culture 33 29.7%
Health Services 52 46.8%
Gender 34 30.6%
Biology and Genetic Endowment 13 11.7%

Page details

Date modified: