Chronic Diseases and Injuries in Canada

Volume 33 · Number 1 · December 2012

Utilization of the Canadian Incidence Study of Reported Child Abuse and Neglect by child welfare agencies in Ontario

L. Tonmyr, MSW, PhD (1); S. M. Jack, RN, PhD (2); S. Brooks, BA (3); G. Williams, MSc (1); A. Campeau, MA (1); P. Dudding, MM, MSW, RSW (4)

https://doi.org/10.24095/hpcdp.33.1.04

This article has been peer reviewed.

Author references:

  1. Injury and Child Maltreatment Section, Health Surveillance and Epidemiology Division, Public Health Agency of Canada, Ottawa, Ontario, Canada
  2. School of Nursing, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
  3. Research Support Services, School of Nursing, Faculty of Health Sciences, McMaster University, Hamilton, Ontario, Canada
  4. Child Welfare League of Canada/ La Ligue pour le bien-être de l'enfance du Canada (CWLC/LBEC), Ottawa, Ontario, Canada

Correspondence: Lil Tonmyr, Injury and Child Maltreatment Section, Health Surveillance and Epidemiology Division, Centre for Chronic Disease Prevention and Control, Public Health Agency of Canada, 200 Eglantine Driveway, Tunney's Pasture, A.L. 1910C Ottawa, ON K1A 0K9; Tel.: 613-954-3339; Email: lil.tonmyr@phac-aspc.gc.ca

Abstract

Introduction: The purpose of this study was to analyze how child maltreatment surveillance data from the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) is used by senior child welfare decision makers.

Methods: This triangulation mixed-methods study included quantitative and qualitative methods to facilitate an in-depth exploration from multiple perspectives. We interviewed Ontario child welfare decision makers to measure utilization of the CIS in policy development.

Results: The majority of respondents were aware of the CIS data. Decision makers reported using these data to determine resource allocation, understand reported maltreatment trends and validate findings at their own agencies. Urban agencies used the data more than did rural agencies.

Conclusion: This study is the first to triangulate data to understand and improve utilization of child maltreatment surveillance data. The study participants indicated considerable appreciation of the data and also provided ideas for improvements across the surveillance cycle.

Keywords: child maltreatment, surveillance, Canadian Incidence Study of Reported Child Abuse and Neglect, data utilization, policy development

Introduction

The Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) is one of the Public Health Agency of Canada's (PHAC) national health surveillance programs. Since 1998, data have been collected from child welfare agencies every five years. The CIS captures data on child maltreatment (exposure to intimate partner violence, neglect, emotional maltreatment, physical and sexual abuse), the extent of its harm, the source of the allegation, short-term investigation outcomes, child and family characteristics and functioning issues.Endnote 1 The CIS captures information at the national level, but some provinces and territories collect additional data to obtain estimates specific to their jurisdiction. For example, provincial data in Ontario have been collected through the Ontario Incidence Study of Reported Child Abuse and Neglect (OIS), the antecedent of the CIS, since 1993.

Surveillance data are collected to support decision makers in setting priorities and allocating resources in policy development. The data should be able to identify at-risk populations, monitor trends, detect emerging issues and notice changes in professional practice.Endnote 2 The components of the surveillance cycle are the collection, analyses, interpretation and dissemination of data. Feedback is then solicited from the field to improve subsequent cycles. CIS data have been analyzed to produce surveillance reports, articles, book chapters and fact sheets. These publications illustrate how CIS surveillance data inform child welfare practice and policy, for example, by providing educational material about child welfare to students in high schools, universities and continuing education programs;Endnote 3 supporting the implementation of differential response in some jurisdictions;Endnote 3 contributing to the United Nations' understanding of child neglect;Endnote 4 requesting augmented fundingEndnote 5 and enhancing maltreatment prevention by First Nations agencies. However, no CIS surveillance evaluation on aspects such as flexibility, system accessibility and stability has been published to date.

Provincial and local child welfare decision makers are a target audience for the CIS findings. They can influence and adapt programs, policies and practices by responding to emerging trends and issues highlighted by surveillance data. While there is an emerging field of science within child welfare focused on evidence-informed decision-makingEndnote 6 and the importance of integrating research evidence into practice and policy,Endnote 7 no attention has been paid to exploring how decision makers perceive and use surveillance data, a very specific form of research evidence. We identified only one study from First Nations agencies regarding the utility of surveillance data.Endnote 8 However, no clear conclusions could be drawn due to the small sample size. The authors speculated that the remoteness of the locations impeded opportunities for decision makers to foster networks with researchers and/or participate in conferences and meetings. Challenges in accessing evidence are an identified barrier to utilizing child-welfare research in Australia, Ireland and Ontario.Endnote 9–11 A Quebec study showed that relative position within an organization mattered in terms of research utilization.Endnote 12

This analysis is a component of a larger mixed-methods study.Endnote 11 Our study focuses on Ontario child welfare decision makers' perceptions and use of the results from the CIS/OIS. We define utilization of research, and specifically surveillance data, as the transfer and uptake of research-based knowledge into policy and practice. The objectives of this paper are to:

  • examine Ontario child-welfare agency decision makers' awareness and perceptions of the CIS;
  • describe the CIS dissemination methods that decision makers prefer;
  • explore how the CIS is utilized for child welfare policy and practice in Ontario; and
  • identify strategies for improving aspects of the surveillance cycle (data collection, data analysis, dissemination and feedback from the field).

Methods

This mixed methods study included both quantitative and qualitative methods to facilitate in-depth research from a number of perspectives.Endnote 13 We administered a quantitative survey to senior Ontario child-welfare decision makers to measure research utilization in policy development. This component covered the first and third objectives of our study (awareness/perceptions of the CIS; preferred CIS dissemination methods). The qualitative component of this analysis used case-study methodologyEndnote 14 to explore how child-welfare decision makers used the CIS/OIS public health surveillance data and to identify what influence and impact surveillance findings have had on child-welfare policy. The qualitative study covered all four objectives.

Only the three most senior decision makers in each agency (executive directors, services directors and supervisors/managers/ other positions) were eligible to participate in both the quantitative and qualitative components.

The Hamilton Health Sciences/McMaster Faculty of Health Sciences Research Ethics Board and the Ontario Association of Children's Aid Societies (OACAS) reviewed and approved the project.

Qualitative case study

We used an embedded, multiple-case approachEndnote 14 to guide this case study of child-welfare agencies delivering services to populations in urban centres, mixed rural/small urban centres, or remote communities. It was determined that we would reach theme saturation by selecting 13 agencies that provided services to geographically or culturally unique populations. Nine consented to participate and four declined due to time constraints. From each of these nine participating agencies, the three most senior decision makers were invited to participate in two semi-structured, in-depth qualitative interviews. We conducted 21 interviews in-person and six by telephone (due to schedule conflicts or remote location of agencies) between March and September 2007. The focus of the first interview, lasting 60 to 90 minutes, was to explore the individual, organizational and system-level influences on the interviewees' ability to utilize research evidence in decision-making. We also asked about their awareness and utilization of CIS/ OIS data (questionnaire available upon request). Six to nine months after this initial interview, we conducted telephone interviews (n = 19) lasting on average 45 minutes. This interview allowed us to verify our interpretation of the data and confirmed the validity of concepts that arose throughout all the interviews. The remaining 8 participants did not complete a second interview either because they had left the agency or because they could not be contacted despite numerous attempts.

All participants completed a demographic questionnaire. We kept field notes on topics, observations and researchers' responses to events.Endnote 15

Qualitative data analysis

We conducted the data analysis and collection concurrently to identify themes requiring further exploration. Content analysis principles guided the examination of each transcript. Two investigators independently reviewed and coded each transcript. This double-coding and peer examination promoted consistency of emerging qualitative findings. Once the core themes from each interview were identified, we used a constant comparative processEndnote 15 to contrast findings across contexts and identify research utilization concepts and factors influencing the research uptake process.

Quantitative data

In the second phase of the study, aiming for a census we contacted all 53 child-welfare agencies in Ontario to participate in the study. Of these, 41 agreed to participate (77% participation rate). Of the 123 eligible senior decision makers from the participating agencies, we interviewed 98 (80% participation rate) using a questionnaire developed for decision makers in public health.Endnote 16,Endnote 17 We added questions about the CIS/OIS to this tool, totalling 53 questions and lasting 30 to 45 minutes. One researcher conducted telephone interviews (between December 2007 and September 2008) and completed each questionnaire using a customized module in Microsoft Access 2007 (Redmond, WA, United States).

Quantitative data analysis

We performed the statistical analyses in two stages. First, we analyzed univariate and bivariate relationships. We conducted a significance test (Fisher exact test) for each variable by the respondent's position and gender and the agency's location. Next, we ran a linear regression to model participants' satisfaction with the CIS/OIS. Correlates were selected from the variables described in Table 1 using a backward selection approach (α =.10). Due to the similarities between CIS and OIS findings, we only report information about the CIS. We used PROC FREQ and PROC GLM in SAS/STATH software, version 9.1 for Windows (SAS Institute Inc., Cary, NC) for the analyses.

Table 1 Description of the CIS/OIS questions/variables used in the quantitative survey
Variable Measure

Abbreviations: CIS, Canadian Incidence Study of Reported Child Abuse and Neglect; OIS, Ontario Incidence Study of Reported Child Abuse and Neglect.


aUrban and mixed locations were collapsed due to the small number of respondents and empirical similarities.
Organizational characteristics of agency
Type Children's Aid Society or First Nations Agency
Location Urban/mixedTable 1 - Footnote a or rural
Individual characteristics
Sex Male or female
Current position Executive director, director (services or quality assurance) or supervisor/manager/other
Education (highest degree achieved) Secondary, post-secondary or other
Participant's perception of research evidence
Direct supervisor expects the participant to use research evidence for planning Likert scale: 1 (low) to 7 (high)
Research evidence is consistently included in program planning Likert scale:1 (low) to 7 (high)
Relevance of research literature to the participant's work Likert scale:1 (low) to 7 (high)
Participant's perception of the CIS/OIS
Has ever seen the CIS Yes/no
Has ever seen the OIS Yes/no
Has used the CIS within past year to make policy/program decisions Yes/no
Has used the OIS within past year to make policy/program decisions Yes/no
Organization has made policy/program decisions related to child abuse and neglect in the past year Yes/no
CIS relevance to the participant's field Likert scale: 1 (low) to 7 (high)
CIS ease of use Likert scale: 1 (low) to 7 (high)
OIS relevance to the participant's field Likert scale: 1 (low) to 7 (high)
OIS ease of use Likert scale: 1 (low) to 7 (high)
Extent to which CIS data was considered in the decision-making process in the past year Likert scale: 1 (low) to 7 (high)
Extent to which CIS data influenced that decision Likert scale: 1 (low) to 7 (high)
Extent to which incorporating CIS data in the decision-making process leads to concrete changes in policies/programs Likert scale: 1 (low) to 7 (high)
Extent to which incorporating CIS data in the decision-making process confirmed current policies/programs related to the decision Likert scale: 1 (low) to 7 (high)
The participant's general satisfaction with CIS Likert scale: 1 (low) to 7 (high)

About half (55%) of the agencies in the sample had a formal university affiliation. Almost 85% of the respondents held a graduate degree and had extensive experience in child welfare.

Sample characteristics

Table 2 describes the demographics for both parts of the study. For the quantitative phase, 84 of the respondents worked in an urban/mixed agency and 14 in a rural agency. Of the respondents, 55 were women and 43 were men; 36 were executive directors; 32 were service directors; and 30 worked as supervisors/ managers or in other positions.

Results

The following qualitative results correspond to all four study objectives and the quantitative results to the first and third objectives.

Qualitative results
Awareness and perception of the CIS

The majority of respondents (84%) were aware of the CIS, having learned about it through, for example, the distribution of reports, participation in meetings or conferences, postings of CIS findings (e.g. by the OACAS or the Child Welfare League of Canada), and presentations and involvement in the data collection by the agency. Most respondents acknowledged that the report was circulated among senior decision makers within the agency but did not always reach the front-line workers.

Table 2 Characteristics of the respondents in the qualitative (initial interview) and quantitative surveys
Variable Qualitative (initial
interview)
(n = 27)
Quantitative
(n = 98)
n % n %

aNot asked in quantitative survey.
Location Urban/mixed 18 67 84 86
Rural 9 33 14 14
Position Executive Director 9 33 36 37
Services Director 9 33 32 33
Supervisor/manager/other positions 9 33 30 31
Experience Average years in child welfare 19 70 21 21
Average years in current agencyTable 2 - Footnote a 14 52
Average years in current position 7 26 7 7
Education Bachelor's degree 4 15 6 6
Master's degree or higher 20 74 85 87
College/diploma/other 3 11 7 7
Sex Female 18 67 55 56
Male 9 33 43 44
Total 27 98

Decision makers from urban/mixed sites were knowledgeable about the content of the CIS and could identify examples of data collected. Those from rural agencies, with the exception of respondents who had participated in CIS data collection, were not familiar with the CIS. One rural respondent identified how their agency's participation in the CIS fostered an investment in the findings:

[…] we were one of the first rural participators... So we've paid attention to the outcome of that research because we see it personally, we're engaged in it, so it was important for us to review and think about the outcomes of that.

Description of preferred CIS dissemination methods

Respondents emphasized the importance of the CIS using many different ways to disseminate information and the value of frequent communication. They confirmed that surveillance reports should be available in hard and electronic formats. In addition, they considered essential the inclusion of interpretations of findings in report summaries because they were too busy to read lengthy reports. The majority disclosed that they primarily read the executive summary and/or fact sheets and used the full report only to obtain further information on specific topics. Respondents also stressed the value of face-to-face presentations to agency staff by someone knowledgeable about CIS findings, especially in rural agencies. One respondent said:

The information was right there, it was visual, and they were talking to people who live this everyday. And so there was time left in the presentation for a discussion—why do you think that would be, does that resonate with you guys or not, does it seem like it's right off the wall? And there was lots of time for conversation and for exchange of ideas and really kind of connecting with the information.

Respondents reflected positively on discussions with and presentations by CIS researchers and agency staff who were knowledgeable about CIS and could interpret the data. They saw it as an opportunity to increase learning and engagement with the data.

An additional theme emerged around the frequency with which CIS reports were disseminated. The respondents preferred receiving data more frequently than the present rate of every five years, current data being most relevant to policy development. Decision makers are regularly required to prioritize where to invest their limited resources; thus up-to-date information on current and emerging trends (such as the decline in substantiated sexual abuse) is perceived as helpful. Some respondents mentioned advantages to regularly receiving ''short CIS summaries'' via portals frequently accessed by staff. One respondent suggested:

I think people don't have time to read anymore. Here's a great idea, do like a one liner like every week… Send me a highlight from this posted on OACAS, wherever… So in the weekly news I can say ok this is the finding I saw this week. So it's that constant messaging, changing it up every time so you are not saying the same thing…

Other suggestions included having fact sheets and summaries with embedded links to background documents or related research. Yet another idea was to link CIS results to practical interventions in order to demonstrate a concrete utility of findings.

Utilization of the CIS

Many of the decision makers acknowledged that the CIS data, presented at a national level, provided the ''big picture'' and was very relevant for provincial policy development. At the agency level, decision makers found the CIS data most useful in (1) identifying emerging child maltreatment trends so that agency policies or programs could be adjusted; (2) providing a benchmark for their local statistics and insights into issues; and (3) confirming local observations and hypotheses about child maltreatment trends. For example, in 2005 the CIS confirmed an increase in reports of children exposed to intimate partner violence, a decline in reported sexual abuse and that neglect is common.Endnote 1 These data influenced some organizations to re-examine their resource allocation to families exposed to intimate partner violence or to restructure their sexual abuse prevention and treatment programs. As one decision maker explained:

I think [the CIS is] a very good effort to iron out those wrinkles and to give the people who are making policy and who are doing the work related to this phenomenon information about trends and incidence so the amount of different kinds of abuse and the nature of that abuse and how it's changing over time, and some of the implications of that. One of the strongest ones I think you know—there's upswings in physical abuse and downswings in sexual abuse over the years and we're always challenged to figure out why exactly those things are happening, whether it's because we're being more effective, less effective, whatever.

Some respondents mentioned that the CIS informed their policies indirectly by creating knowledge that could influence day-to-day decision-making. Others were less optimistic about their ability to influence policy and claimed that policy changes only happen through the provincial Ministry of Children and Youth Services. Rural participants were the least convinced that the findings did—or would—impact their agencies' policy making.

Improving various aspects of the surveillance cycle

Participants from the qualitative phase provided insights into areas where the CIS could be improved. The majority of the respondents felt that the surveillance report was comprehensive and required no changes. Some others suggested that further data could be collected and analyzed to be predictive of future trends, rather than just reporting incidence. Yet others suggested that the results needed to be interpreted and contextualized in terms of Ministry mandates. Some rural respondents sought agency staff involvement in developing the CIS questionnaire to ensure that the information was relevant. Others suggested using longitudinal data collection and linkages to existing data sources such as Looking after Children.Footnote * Respondents also asked if the CIS could find a way to track the outcomes of investigations and to determine the effectiveness of child welfare interventions. Others proposed providing provincial comparators or demographic characteristics, such as immigration/citizenship status, race and gender. Comparisons of the findings to the community child population were also requested as well as the collection and analysis of Aboriginal data.

One of the barriers to using CIS findings was that it differs from data collection for planning mandated by the Ministry of Children and Youth Services. Definitions and content used in the CIS differ from Ministry systems, which impedes comparison. For instance, while agencies count the final decision for short- and long-term placements regardless of maltreatment type, the CIS captures short-term placements and reason for the investigation.

Respondents provided suggestions for specific analyses of the data, such as detailed information about types of maltreatment in general, and neglect and exposure to intimate partner violence specifically, considering their high prevalence in the CIS and their co-occurrence with other maltreatment. They also requested data about the effectiveness of placement, in particular, kinship. Other areas for exploration included the relationship between poverty and the need for child welfare intervention, and parents' and children's mental health and/or addictions.

Quantitative findings
Awareness/perception and utilization of the CIS

Overall, 96% of respondents were aware of the CIS. We found significant differences between urban/mixed and rural agencies on awareness of the CIS, relevance of the research literature to the participant's work, as well as ease of use of CIS data and general satisfaction (Table 3). However, differences by position and gender were not significant in our analysis and therefore are not shown.

Table 3 Descriptive statistics from the quantitative survey
Total Respondents (N = 98) Urban/mixed (n = 84) Rural (n = 14) p valueTable 3 - Footnote b
RespondedTable 3 - Footnote a, % Median RespondedTable 3 - Footnote a, % Median

Abbreviations: CIS, Canadian Incidence Study of Reported Child Abuse and Neglect; OIS, Ontario Incidence Study of Reported Child Abuse and Neglect.


aMay not add to 100% due to rounding.
bFisher exact test.
cThe Likert scale ranges from 1 (low) to 7 (high).
Dichotomous response
Has ever seen the CIS
Yes 98 86 < .05
No 2 14
Missing/not applicable 0 0
Has ever seen the OIS
Yes 95 86
No 2 0
Missing/not applicable 2 14
Has used the CIS within last year to make policy/program decisions
Yes 33 14
No 62 71
Missing/not applicable 5 14
Has used the OIS within last year to make policy/program decisions
Yes 40 21
No 55 64
Missing/not applicable 5 14
Organization has made policy/program decisions related to child abuse and neglect in the past year
Yes 98 86
No 1 14
Missing 1 0
Likert scaleTable 3 - Footnote c
Direct supervisor expects the participant to use research evidence for planning
95 5.00 93 4.00 < .001
Research evidence is consistently included in program planning
99 4.00 100 4.00
Relevance of research literature to the participant's work
99 6.00 100 5.00 < .05
CIS relevance to the participant's field
93 6.00 64 5.00 < .05
CIS ease of use
  87 6.00 57 4.50 < .05
OIS relevance to the participant's field
90 7.00 64 6.00
OIS ease of use
  85 6.00 57 4.50 < .01
Extent to which CIS data was considered in the decision-making process in the past year
85 2.00 79 1.00
Extent to which CIS data influenced that decision
83 2.00 79 1.00
Extent to which the incorporation of CIS data in the decision-making process lead to concrete changes in policies/programs
81 1.00 93 1.00
Extent to which the incorporation of CIS data in the decision-making process confirmed current policies/ programs related to the decision
81 1.00 93 1.00
The participant's general satisfaction with CIS
87 6.00 57 5.00 < .05

Figure 1 shows the distribution of overall satisfaction with the CIS data and the difference between urban/mixed and rural agencies. On a scale of 1 to 7, the most common score was 6 in all types of agencies. However, rural agencies had more response variability.

Figure 1 Distribution of the overall satisfaction with the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) by location of respondents of the quantitative survey

Figure 1 Distribution of the overall satisfaction with the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) by location of respondents of the quantitative survey
[Figure 1, Text Equivalent]

Chronic Diseases and Injuries in Canada - Volume 33, no. 1, December 2012

Figure 1 shows the distribution of overall satisfaction with the CIS data and the difference between urban/mixed and rural agencies. On a scale of 1 to 7, the most common score was 6 in all types of agencies. However, rural agencies had more response variability.

Table 4 shows the regression results for individual and agency characteristics that are associated with satisfaction with and relevance of the CIS. Respondents who gave a higher score to the relevance of research literature to their work also gave a higher score to the relevance of the CIS. The CIS relevance score was lower for rural agencies. Overall CIS satisfaction was associated with CIS ease of use and urban/mixed location. Rural agencies scored CIS satisfaction lower than did urban/mixed agencies.

Table 4 Results of multivariate analyses on the Canadian Incidence Study of Reported Child Abuse and Neglect (CIS) perception and satisfaction
Model Covariates ß se (ß) t p
Dependent variable

Abbreviations: CIS, Canadian Incidence Study of Reported Child Abuse and Neglect; OIS, Ontario Incidence Study of Reported Child Abuse and Neglect.


aMay not add to 100% due to rounding.
bFisher exact test.
cThe Likert scale ranges from 1 (low) to 7 (high).
CIS relevanceTable 4 - Footnote a Intercept 4.14 0.70 5.95 < .01
Location (rural) -0.86 0.44 -1.95 .05
Relevance of research literature to participant's workTable 4 - Footnote b 0.31 0.12 2.69 < .01
Overall satisfaction with the CISTable 4 - Footnote c Intercept 3.17 0.57 5.54 < .01
Location (rural) -0.70 0.34 -2.06 .04
Child welfare experience (years) 0.02 0.01 1.94 .05
CIS ease of useTable 4 - Footnote b 0.36 0.09 4.01 < .01

 

Discussion

The results of our study show that the majority of respondents were aware of the CIS. Study findings reached them through websites, conferences and researchers' visits to the agencies. Not surprisingly, those agencies that participated in data collection and/or attended researchers' presentations had a better grasp of CIS content. CIS data were considered useful although respondents had suggestions for improvements. Our data suggested that respondents from urban/mixed locations are more knowledgeable about the CIS than those from rural areas. Cost is a prohibitive factor in data collection from rural agencies as is the ability to present them with research findings individually.

Among the interviewed decision makers, the CIS met its surveillance objectives in that respondents confirmed its utility in identifying at-risk populations, monitoring trends, detecting emerging issues and directing changes in practice. The respondents mentioned that it was especially useful to monitor maltreatment trends and confirm their local observations. It seems counterintuitive that while respondents were generally satisfied with the CIS and considered it highly relevant, only a minority used CIS data in decision-making. One possible explanation is that our data collection tools did not successfully capture direct use at the agency level. It is also possible that the CIS may be more useful at the Ministerial level.

It is notable that many of the critiques and suggested changes are outside the scope of CIS surveillance. This indicates that some respondents do not recognise the goal and limitations of surveillance data. For instance, the CIS is designed to inform development of interventions through risk factor identification, but this is not intervention data per se. It is important that the scope of surveillance data be clarified for users. The additional data requested by respondents (i.e. demographics) already exist in the CIS program (except immigration status), which suggests these respondents were unfamiliar with the CIS. Many suggestions to expand the CIS scope were likely derived from the paucity of Canadian child maltreatment data. The CIS has provided an important platform to promote further data collection efforts. However, there is a need to expand research within child welfare.

Many of the gaps identified by respondents are being addressed by PHAC and its partners. For example, respondents in various agencies mentioned the need for more data on Aboriginal people. Aboriginal agency participation has increased with each cycle,Endnote 18 and several analyses have focused on Aboriginal children and their families.Endnote 19–22 Also, the CIS has started to be used in conjunction with other data sets to obtain a more complete depiction of child maltreatment. For instance, one researcher investigated the reported decline in substantiated sexual abuse comparing CIS and Quebec data.Endnote 23

Also promising is that analyses have been conducted on several of the issues suggested by the respondents, for example, youth substance abuse,Endnote 24 anxiety/depression in adolescents,Endnote 25 neglectEndnote 26 and exposure to domestic violence.Endnote 27 In a recent review of the CIS, the authors identified 37 manuscripts based on original analyses published in peer-reviewed journals.Endnote 28 However, several issues remain unanalyzed, and a process needs to be developed to inform decision makers about existing CIS analyses.

Interview respondents identified conferences as dissemination channels for CIS findings; however, earlier research has questioned the effectiveness of these for health care professionals.Endnote 29 Possibly conferences are viewed more positively by those working in child welfare than in health care. Other dissemination methods such as websites and presentations at the agencies were also mentioned. Rural agencies valued in-person presentations more than did urban/mixed agencies. The respondents felt that on-site presentations created an opportunity to clarify findings and allowed all staff members, not just management, to be present. We cannot say which means of dissemination were the most effective as participants were not asked to rank information sources.

Dissemination plans have been developed for each CIS cycle.Endnote 30 These plans have stressed the importance of developing multiple strategies for different audiences and targeting products specifically for these audiences. The next dissemination plan should incorporate key findings from this study. The respondents valued fact sheets, but they also found that the surveillance reports were an important resource. The utility would increase for them if the CIS were indexed. They also concluded that both hard and electronic copies of CIS-related materials were useful.

Collaboration is important in improving research use in decision making.Endnote 31 This idea has been emphasized since the CIS inception. For this, PHAC has established committees with representatives from the various Canadian Ministries. PHAC has also hosted several fora for the exchange of ideas about improvements to the CIS.Endnote 32,Endnote 33 The findings from our regression models are not surprising. Estabrooks et al.Endnote 34 showed that predicting research utilization based on education had mixed results. Among professionals, those in managerial/leadership roles in health care fields similar to the child welfare field consistently demonstrated more research utilization.Endnote 12,Endnote 34 Our findings of the underutilization of research are consistent with the child welfare literature; lack of access to the research has been suggested as an explanation.Endnote 8–11 Gender and age are included as control variables in many studies; however, they were not significant in this study, and thus were not included in our models. Additionally, since these variables are not modifiable, the focus has been on other individual characteristics.

Implications

Although some suggestions for improvements have been addressed, there are others to consider. Several respondents felt that the CIS needed to be conducted more frequently to increase utility. A cost-benefit analysis needs to be conducted; service data have a shorter shelf-life than population-based data, since changes in practice influence what constitutes maltreatment. For instance, expansion of reporting laws to include exposure to intimate partner violence in Minnesota created an influx of new cases.Endnote 35 Other respondents asked for longitudinal data, to better understand children's situation when placed outside the home. Intervention data were also requested. Surveillance systems should be flexible to meet the needs of the users so the feasibility of including other requested information in the CIS needs to be explored. Most importantly, disseminating efforts should target rural areas.

Strengths and limitations

This study has many strong points: we used both qualitative and quantitative methods for data collection and analysis to promote overall data credibility; we used an embedded, multiple-case approach to guide the case study (three case studies were conducted at the same agency providing different perspectives); and we interviewed professionals at both urban/mixed and rural agencies across Ontario.

However, the findings should be interpreted within the limitations of the study. We do not know if these findings are generalizable outside Ontario. Moreover, the small sample size may have precluded the detection of differences in responses. We can only speculate how the results were influenced by attrition between the first and the second qualitative interview. The 19 who participated in the second interview agreed that the research team had accurately interpreted the experiences they had described in the first interview; however, it is impossible to predict if the eight who did not participate would have concurred.

Conclusion

The CIS, as part of Canada's child health surveillance program, provides valuable and important data on a highly vulnerable population who face risk factors with potential lifelong consequences. There is a growing recognition of the significance of these data in influencing practice, policy and program development at all levels. This triangulation study was the first to analyze the utilization of maltreatment surveillance data among decision makers. It identified a high appreciation of the CIS and provided ideas for improvements in all aspects of the surveillance cycle.

Acknowledgements

We would like to thank the research project participants from Ontario child welfare agencies who shared their time, knowledge and expertise with the members of the study team. We appreciate the warm welcome that we received at each participating agency.

This study was funded through the Ontario Centre of Excellence for Child and Youth Mental Health at the Children's Hospital of Eastern Ontario. We also want to extend our thanks to Jasminka Draca, Pascal Roberge, Dr. Anne-Marie Ugnat and Dr. Catherine McCourt at the Public Health Agency of Canada and Karen Levine for their support.

Dr. Susan Jack is supported through the Institute of Human Development, Child and Youth Health, Reproduction and Child Health New Investigator Personnel Award from the Canadian Institutes of Health Research.

References


Page details

Date modified: