4.0 Data Analysis

4.1 ANALYTICAL FRAMEWORK

A participatory evaluation is based on the perspectives, opinions, and interests of those being evaluated. Both the design of the evaluation and the development of the key questions were based on the ideas of CAPC parents, volunteers and board members, and staff from across the Atlantic provinces. Their knowledge and expertise also were used in the analysis.

4.1.1 Strategies for Data Analysis

It was considered essential to remain true to the participants’ descriptions and interpretations of their experiences. This was done by using direct quotes throughout the results section, which allowed the participants’ own words to reflect the reality of their experiences. In addition, the analyses were structured to respond to the regional evaluation framework and questions that were developed through consultations with CAPC stakeholders.

4.1.2 Coding of the Data

Qualitative data gathered through interviews and focus groups was coded to facilitate classification, organization, and analysis. The codes provided a way to group similar responses within interviews and across interviews (for example, discussions about satisfaction with experiences in the program or how children had been affected by their participation in the program). All information relating to a specific question could then be considered in describing and analyzing results.

The first step in coding the data was to develop an initial list of codes that were linked directly to the interview questions and to the regional evaluation framework. This preliminary list of codes was then pretested by the Central Analysis Team (made up of the Regional Evaluation Team with support from other research consultants) using transcripts of interviews with participants. Interviewers were also consulted for their insights into the data they had collected. Based on this, the code list was modified to include codes that covered information that had not been directly asked for but had been volunteered during interviews. However, no codes were collapsed until all interviews had been completed and coded.

During provincial meetings, this list of codes was used by members of the Provincial Project Evaluation Teams (PPET) to analyze transcripts of participant interviews. They evaluated the codes and examined the way they had been defined. PPET members were asked whether the code list captured the essence of what they believed was revealed by the data. New codes that were suggested were added, and members were urged to ask questions whenever they had any difficulty with the coding. This allowed for collective decisions about the definitions of codes. During these meetings, most transcripts were coded by more than one reviewer.

Following all of the PPET meetings, the interviews were again coded to reflect the decisions made by the PPET members. The Central Analysis Team developed similar code lists for the staff interviews and the focus groups. When possible, the same code words were used when addressing the same information in different interview sources.

All transcripts (interviews and focus groups) were coded a final time by the Central Analysis Team to ensure validity of coding and to prepare for entry of the data into the Ethnograph software program: the software package used to manipulate the data and organize the data by code. At this stage of the analysis, extensive reading of the transcripts revealed that the original level of specificity had not been necessary, so several codes were collapsed. Codes that were not distinct or that were sub-categories of another code were regrouped so that information that had been categorized under two different codes was now combined under only one. Although categorized differently, all data was still considered in the analysis.

The data was printed by code using the Ethnograph, a qualitative software program, to reorganize it for analysis. To address each question in the regional evaluation framework, the codes linked to each question were carefully considered, and themes or patterns that emerged from the responses were identified. These themes provided the response to the regional evaluation questions. The tables and other information presented in PartIII were developed based on this material.

 

[Previous] [Table of Contents] [Next]