3.0 Data Collection

The approach to data collection taken for the Atlantic CAPC Regional Evaluation was primarily qualitative to learn how people experienced CAPC and to use their own words as an important data source to describe the impact of the program on their lives. Qualitative methods used to gather information were in-depth interviews and focus groups. Quantitative data from the National Evaluation was to be incorporated as complementary information where appropriate.

3.0.1 Interviews

Participant² Interviews
Four participants were selected from each of the 40 projects for a possible sample of 160. Of these, 151 participants were interviewed. Participants were selected to provide different perspectives on CAPC participation. Each type of participant had a viewpoint that was essential to understanding the process of being part of CAPC.

The participants interviewed included

• Form E Participants. Two of the participants selected had to have completed National Evaluation Form E to allow for a direct linkage between the regional and national evaluations.

• Long-term Participant. The third participant selected from each project was an individual who had participated in the project for at least one year. Long-term participants could provide a different perspective on the impact that CAPC has had on their lives because of their extended involvement with the project. If a project had been in operation for less than a year, a long-term participant was one who had been involved since the project’s initiation.

• Former Participants. A fourth participant was selected from those who had completed National Evaluation Form E but had since stopped participating in the project. Interviewing former participants provided information about why people stopped participating -- what barriers might have existed, whether needs had been met, etc. In the event that all people who had completed Form E were still participating, participants were randomly selected from those who had completed Form D but had since stopped participating.

The proposed sample size exceeded the 20-50 interviews needed to permit rich description of the overall program and its participants (Swanson, 1986) and provided enough numbers in the subgroups to allow independent consideration of categories of participants (e.g, long-term vs. new participants).

Recruitment
Participants were selected with the assistance of the project coordinators, who provided lists of names of participants who met the inclusion criteria for each subgroup. The definition of participation was determined to some extent by the staff of the centre but focused more on consistency rather than necessarily high frequency of attendance.

Individuals who had completed Form E but were no longer participating in the project were listed as potential interviewees but as former participants. No restrictions were placed on the length of time they had participated. However, participants who had moved away from the area or who had stopped participating because their child had turned seven were not included in the sample.

Staff also provided lists of those parents who were considered long-term participants. They could be those involved in a number of ways in the project (e.g, program volunteers or board member) but could not be paid staff.

Most of the 151 interviewees were mothers. The sample included three fathers and a few grandmothers and day care providers who brought the children they cared for to the centre.

Number of Form E participants:		74
Number of long-term participants:		42
Number of former participants:		35

Reasons for the discrepancy (9) between the targeted sample size (160) and the actual number of 151 interviewees included unavailability of interviewees during the few days that the interviewer was in the area, technological failures (recording of interviews unsuccessful), or interviewees not actually meeting the sample criteria.

Staff Interviews
A total of 50 interviews, at least one per project, were conducted with staff members. Project coordinators were interviewed at all projects except one, where the coordinator was ill, so another staff member completed the interview. For multi-site projects in Nova Scotia, more than one individual was interviewed to fully represent all aspects of the project and to address some specific areas such as the outreach activities of the project. In Prince Edward Island, additional staff members were interviewed at two projects because in one instance, coordinator responsibilities were equally shared, and in the other, the coordinator had changed immediately before data collection. Staff interviews were conducted by provincial evaluation coordinators, who had been hired by the contractor.

Group Interviews with the Joint Management Committees and Program Advisory Committees Interviews with the Joint Management Committee/Program Advisory Committee (JMC/PAC) from each province provided an additional perspective on the development of the projects. The committee was presented with a series of questions and asked to come to a consensus for responses. These interviews were also conducted by provincial evaluation coordinators.

3.0.2 Recording Interviews

Interviews were audio-taped to preserve information exactly as stated. If a participant did not wish to have the interview taped, the interviewer respected this choice and relied solely on notes. Most interviewees consented to be taped. (Sixteen interviews were not taped.) Table 1 shows the number of interviews conducted in each province.

TABLE 1: Interviews with Parents³ and Staff in Each Province

3.0.3 Focus groups

Participant Focus Groups
Participant focus groups were conducted for all 40 projects. Capitalizing on the benefits of the group process, these focus groups were designed to supplement the information being gathered by the interview method and to increase our understanding of how the projects affected the lives of parents. The information from 39 focus groups was considered as part of the analysis. One group was not included due to significant deviation from the standard introduction by an interviewer, a factor that might have influenced participant responses.

Board/Volunteer and Community Agencies/Organizations Focus Groups
Approximately 50% of the projects were selected for participation in focus groups with board/volunteers and community organizations. As part of the selection process, projects were categorized according to criteria that reflect the variety and scope of projects in the Atlantic region. The criteria were province, rural/urban, multi-site/single-site, Francophone, Aboriginal, and Black.

Once categorized, projects were randomly selected from among those meeting the criteria for each group. Selected projects participated in both the board/volunteers and community organizations focus groups so that the data could be triangulated. Twenty sites were selected; project coordinator sat these sites were asked to provide lists of community partners, current board members and volunteers.

Board/Volunteer Focus Groups
One focus group, composed of a group of board members (referring to both parents and community resource people who donate their time to serve) and parent/non-parent volunteers, was conducted for each of 19 projects. The format of the focus group included both small and large group work. This format was chosen so that each group (parents and resource people) could articulate its unique perspective on the development of the project and also have the opportunity to assess the project as viewed by the other group. Focus groups were conducted by the provincial evaluation coordinators and the interviewers from each province. The groups included some with participation from parents only and others with a combination of the parents and community resource people.

Community Agencies/Organizations Focus Groups
Representatives from agencies and organizations associated with 19 of the projects were contacted and invited to participate in a focus group. One group was not conducted due to a misunderstanding about whether that site had been included in the pilot testing. These focus groups were designed to gain an understanding of how the project was viewed by the community, what types of partnerships were established, and how these partnerships were working. Focus group sessions provided an opportunity for organizations to become aware of their interdependence and to consider how their collective efforts could benefit the communities in which they worked. The agencies/organizations contacted included those currently working with CAPC. During the group session, representatives were asked to identify other agencies/organizations (not currently involved) that would be suitable partners to further assess community linkages and to establish ways to build the network. The range of community partners included representatives from civic groups, local schools, churches, government agencies or departments, individuals from the community, and other community organizations.

3.0.4 Recording Focus Groups

Focus groups were audio-taped to preserve information exactly as stated. If a participant did not wish to have the focus group taped, the facilitator respected this choice and relied solely on notes.(Only two focus groups were not taped.)

Table 2 shows the number of focus groups conducted in each province.

TABLE 2 : Numbers of Focus Groups and Participants

3.0.5 Children’s Drawings

One picture drawn by a child from each project has been included in this final report. The child’s story about what the picture contains is also included. One picture was randomly selected from each site for inclusion in the report, providing each child with an equal chance to have his/her artwork included.

3.1 DATA COLLECTION INSTRUMENTS

3.1.1 Interview Guides

The interview guides were developed in collaboration with CAPC key stakeholders and the Project Evaluation Team (see Appendix IV). Standardized guides using open-ended questions were used to allow personal stories to emerge. Questions to guide interviews were developed and sequenced before the interviewing began to reduce variation among interviewers and to ensure that core topics,namely those addressing the evaluation questions, were covered in every interview. Interviewees, however, still had an opportunity to state their experience in their own words, to elaborate on topics, and to take the interview in unanticipated directions.

Guides were developed the same way for staff interviews and the focus groups. When possible, questions on the different data collection tools "mirrored" one another, thus triangulating the data and providing perspectives on the same theme from different data sources (e.g., parents, staff, community members, board members, and members of the Joint Management/Program Advisory Committees (JMC/PAC). The same process of development was used for the focus group guides. Questions were written in a style appropriate for a group process.

The guides were extensively pilot tested and revised. Revisions, in most cases, reflected a reduction in the number of questions and a reordering of the sequence of questions. Revisions to the parent interview guide and the guide for the board volunteer focus groups were revised to such an extent that they were retested before formal data collection began.

3.1.2 National Evaluation Forms

Information from the National Evaluation Forms A to E (described in the Glossary) was used to further describe the same topics covered by the interviews and focus groups. In particular, information from the forms provided statistics regarding program development and information for participant profiles.

3.1.3 Children’s Drawings

Children from each project were asked to draw a picture showing their experiences of the project. A parent or guardian then asked the child, "Tell me about the picture." Each project developed its own process for generating the drawings, but at all sites, the children described their picture to a parent or staff member to get a more complete and more natural response than may have been elicited by a stranger (i.e., an evaluator). Each project developed its own process to select the picture that would be used in the Atlantic CAPC Regional Evaluation.

The children’s descriptions of their drawings were written down and attached to the pictures. Children participated only with a parent’s permission, and no psychological interpretation of the pictures was done.

 

[Previous] [Table of Contents] [Next]