T. Elton-Marshall, PhD (1,2); S. T. Leatherdale, PhD (2); S. R. Manske, EdD (1); K. Wong, MSc (1); R. Ahmed, PhD (1,3); R. Burkhalter, MMath (1)
This article has been peer reviewed.
Correspondence: Scott T. Leatherdale, School of Public Health and Health Systems, Faculty of Applied Health Sciences, University of Waterloo, 200 University Ave. West, Waterloo ON N2L 3G1; Tel.: (519) 888-4567 ext 37812; Fax: (519) 886-6424; Email: firstname.lastname@example.org
This paper describes the survey development, design and data collection protocol for the 2008/2009 Youth Smoking Survey (YSS) and the changes to the YSS survey and protocols across the 5 survey cycles (1994, 2002, 2004/2005, 2006/2007, 2008/2009). Canada’s Youth Smoking Survey is a nationally representative school-based survey of students (grades 6 to 12 in 2008/2009) from randomly sampled public and private schools in the ten provinces. The main objective of the YSS is to provide benchmark data on national smoking prevalence rates for youth. Key features of the 2008/2009 YSS include consistent measures across survey cycles, a survey team of researchers and non-governmental organizations, a link to school and student level measures, provision of tailored feedback reports to schools and publicly available datasets.
Keywords: youth, smoking behaviour, Canadian Youth Smoking Survey, survey cycles, questionnaires
Nationally representative surveys of youth smoking behaviour are necessary to understand the social, regulatory, educational and commercial factors that influence smoking; to provide evidence for tobacco control policies and programs; and to monitor tobacco consumption in Canada.*
A pan-Canadian consortium of university and non-governmental organizations implemented the 2008/2009 YSS. Members of the Youth Health Team at the Propel Centre for Population Health Impact at the University of Waterloo (Ontario) provided central leadership, while members from the other nine provinces provided leadership in their respective provinces. Members developed survey content during teleconferences. Those who could not participate in the scheduled meetings were asked to provide input prior to the teleconference. This approach allowed provincial stakeholders and the federal government to ensure the survey content included measures relevant to each jurisdiction. Content meetings ensured that core items (those required to compute smoking prevalence rates and derive other key, comparable variables) were retained. Questions added to the existing survey were those deemed higher in priority particularly if they were relevant to active policy agendas. The consortium made consensus decisions about which questions to include in the survey after discussing the merit of all survey questions during team teleconferences.
Each iteration of the YSS allows for a few new items; however, for every addition, about the same number is removed to keep the questionnaire the same length. Those items that tend to appear every other cycle are considered “periodic.” Some items, known as “deleted items,” are phased out completely if the issue/question is no longer relevant. While consistent content permits monitoring of trends over time, introducing new items permits identifying new trends that need to be monitored (see the 2008 YSS user guide for a list of survey items by cycle).
Top of Page
Core measures. To be consistent, “core” survey measures remain the same across all survey years. These include the measures used to define the smoking status of each respondent according to Health Canada definitions, measures of key prevention indicators such as susceptibility to future smoking, age of initiation and amount smoked, and key demographic variables. The core outcomes measured in the YSS are susceptibility to future smoking among never smokers and smoking status. The validated algorithm of Pierce et al. was used to measure susceptibility to future smoking among never smokers (those who have not smoked even a few puffs of a cigarette).PUMF), the de-identified dataset available to researchers. If a question could be skipped within the structure of the questionnaire, it was coded as 96 or 996 or 9996 within the PUMF dataset. For example, a smoker would still be asked questions about susceptibility to smoking but the responses for those questions would be coded as a “valid skip” and would be excluded from the analyses associated with smoking susceptibility.
Starting with the 2004/2005 cycle, the YSS used the School Health Action, Planning and Evaluation System (SHAPES) for school-based data collection. Thus each participating school received a school-specific feedback report and executive summary within 10 weeks of data collection. This report provides customized information including smoking rates and other behavioural (e.g. time spent reading) and environmental information (e.g. smoking on school property) specific to the school. As a supplement to the YSS, information about the school environment (programs, policies and the built environment) was also collected.†
The target population for the YSS consisted of all young Canadian residents attending private and publicly funded schools in the 10 Canadian provinces. Those residing in the Yukon, Nunavut and Northwest Territories and those living in institutions or on First Nations reserves were not included in the sampling frame. Young persons who were attending special schools (e.g. schools for visually and hearing-impaired) or schools located on military bases were also excluded from the sampling frame.
The YSS team at the Propel Centre obtained a comprehensive list of all schools in each province via provincial Department of Education websites. The sampling for the YSS was based on a stratified multistage design. Sampling was stratified according to health region smoking rate and type of school (elementary or secondary). In Stage 1, the Canadian Community Health Survey (CCHS) was used to calculate the smoking rate among 15 to 19 year olds for each health region. The school lists obtained from the provincial Departments of Education for each of the 10 provinces included enrolment data by grade for each school. Using this list, the total eligible grade enrolment in a health region was used as a weight to compute the median smoking rate for each province. Each school’s six-digit postal code was used to identify the health region in which it was located. Schools were then categorized as “low” or “high” smoking rate stratum based on the smoking rate in their health region compared to the median (where greater than or equal to the median was categorized as “high”).
In Stage 2, schools were stratified into elementary or secondary school strata (calculated based on whether there was a higher enrolment of students in grades 6 to 8 or 9 to 12). Elementary and secondary schools were sampled on a 2:1 ratio due to the smaller enrolment sizes of the elementary schools. Schools were also over-sampled in each province based on the provincial school recruitment rate from the 2004/2005 and 2006/2007 YSS cycles.
In Ontario, the design of the 2008/2009 cycle included a third health region stratum, Greater Toronto Area (GTA). The GTA health region stratum acknowledged the size of the GTA and the importance of being able to capture schools from the GTA even if there were refusals from the larger school boards in the city of Toronto.
Lastly, sampling of private schools was based on a simple random sample of private schools in each province. The number of schools originally selected was roughly proportional to the number of students enrolled in private schools in that province as compared to the total in public schools. The sampling design is constructed to provide a representative sample of youth in all provinces in Canada.
In the 2008/2009 cycle, the school board response rate was 84% (the number of school boards that agreed to participate/the number of school boards that were approached); the school level response rate was 59% (the number of schools that agreed to participate/the number of schools that were approached); and the student level response rate was 73.2% (based on the number of completed surveys/the number of eligible students; students who were absent during the data collection were counted as a non-response).
In all provinces, YSS site coordinators contacted school boards prior to approaching schools. Private schools were approached directly because there is no governing board to review research requests for these schools. School boards were typically contacted via a formal board-specific application or a standard board recruitment package that included a school invitation letter, a project brochure, a sample student survey, sample parent information and permission materials, and a template school feedback report.‡ Provincial site coordinators made follow-up calls to the school board to answer any questions and, ideally, obtain board permission to recruit schools. Once a school board was successfully recruited, the schools within that school board were approached via a school recruitment package and follow-up phone calls. The contents of the school recruitment packages were the same for both boards and schools. Only when the school had agreed to participate in the YSS was the survey implemented with eligible students in that school.
Within each participating school, all students in the eligible survey grades (6 to 12) were requested to complete the survey. Active parental permission was required by the school or board for 62% of grade 6 to 8 classes (n = 913) and 19% of grade 9 to 12 classes (n = 372). Students in eligible classrooms took home information letters describing survey details. Active permission protocols required signed parental and child permission forms for the child to receive and complete a survey. In 81% of secondary school classes (n = 1631), passive permission protocols were used to reduce the burden on schools and improve response rates. In this procedure, the school mailed an information letter home to parents that detailed survey procedures, and asked parents to call a toll-free number or inform the school if they did not want their child to participate. Students whose parents objected were put on a “no permission” list and did not receive a survey on the day of data collection. All other students received a survey to complete. Regardless of whether parents provided permission, students were able to decline participation on the day of data collection.
Provincial site coordinators worked with a school contact to arrange data collection at each school. On the day of data collection, teachers administered the survey using standardized protocols during a designated class period. To ensure confidentiality and therefore encourage honest responses, teachers were asked to avoid circulating among the students. Students were also required to place their completed survey in an envelope and seal this envelope before it was collected by a student in the classroom. When parents as well as students were surveyed, active consent was required, and a tear-off sheet with the student’s name was attached to the front of the survey. Students removed the tear-off sheet. A serial code on both the tear-off sheet and the student survey enabled linkage for survey cycles that included a parent interview to be linked to the student responses. The information containing the student’s identification and responses were removed from all public datasets and only those directly related to the research had access to any identifying information. On average, the survey took 30 to 40 minutes to complete. A data collector was on site at the school throughout the data collection period and available to answer respondent questions and collect the completed student surveys.
‡ For sample documents, e.g. surveys, feedback report, etc., see www.yss.uwaterloo.ca/recruitment.
Surveys were machine scanned using Optical Mark Reading (OMR) technology. Quality control measures (e.g. visual scanning, OMR scanning twice to find discrepancies) were used to ensure accuracy of the scanned data. An online survey implementation system (OSIS) permitted central management of recruitment, implementation, analysis and feedback processes.
Survey weights were created to “weight” the data to be representative of the general population of Canadian youth in school. The survey weights were developed in two stages. In the first stage, a weight (W1) was created to account for the school selection within health region and school strata. A second weight (W2) was then calculated to adjust for student non-response. The weights were then calibrated to the provincial gender and grade distribution so that the total of the survey weights by gender, grade and province would equal the actual enrolments in those groups. Finally, bootstrap weights for each province (to estimate sampling error) were generated.
The protocols described were used to implement the 2008/2009 YSS cycle. One of the strengths of the YSS has been its consistent protocols, which allow comparisons over cycles. However, there have been slight modifications to the sampling and protocols in each cycle based on experience in previous cycles; these modifications were made to improve student recruitment and survey completion rates, and to reduce the burden on participating schools. The following section describes some of the significant differences in the YSS over the various cycles.
Changes to survey administration. One of the most significant changes to the YSS occurred in 2004/2005 when the survey administration shifted from Statistics Canada to the University of Waterloo. In 1994 and 2002, the YSS content was developed by Health Canada’s Office of Tobacco Control and data were collected by Statistics Canada. As previously noted, the University of Waterloo’s Propel Centre for Population Impact (formerly the Population Health Research Group and the Centre for Behavioural Research and Program Evaluation) has provided central leadership since 2004/2005.
Changes to the survey. Table 1 summarizes the differences in the survey over time. Until 2006/2007, the sample included grades 5 to 9 only. In 1994, all students in grades 5 to 9 responded to the same survey. In 2002 and 2004/2005, students in grades 7 to 9 answered additional questions about alcohol and drug use. In 2006/2007, students in grades 7 to 12 were randomly assigned to receive one of two versions of the survey. While the majority of the questions were the same in both versions, including those that related to alcohol and drug use, some different questions were added to each (e.g. in one version there were questions about smoking on school property whereas another version had questions about beliefs about the harmful effects of smoking). All other students (grades 5 and 6) received a survey with no questions on alcohol and drug use. Because there were two different versions of the survey in this cycle, there were two survey weights calculated for this dataset and two User Guides to facilitate use of the dataset.
|Survey cycle||Survey dates||Target population, grades||Sample size
|Changes to the survey protocol|
|1994||Sep–Nov 1994||5–9||14 270|
|2002||Oct–Dec 2002||5–9||19 018||Students in grades 7–9 answered additional questions about alcohol and drug use|
|Feb–Jun 2005||5–9||29 243||Adoption of SHAPES (School Health Action Planning and Evaluation System)
Computer-generated feedback reports delivered to schools
Surveys machine-scanned using Optical Mark Read (OMR) technology
|Nov 2006–Jun 2007||5–12||71 003||
Addition of grades 10–12
Students in grades 7–12 completed either Module B1 or B2:
In New Brunswick, data were collected to support the Healthy New Brunswick en santé project (data on smoking using YSS, healthy eating, physical activity, and mental fitness)
Census of schools in New Brunswick
In New Brunswick, 50% of students in grades 5–6 completed the YSS Module A, 25% of students completed a Physical Activity Module and 25% completed a Healthy Eating Module. Within each class in grades 7–12, 25% of students completed the YSS Module B1, 25% of students completed the YSS Module B2, 25% of students completed a Physical Activity Module and 25% of students completed a Healthy Eating Module
|Dec 2008–Jun 2009||6–12||51 922||
Grade 5 students no longer included in the survey
The student survey data were collected using two instruments:
Collaboration in PEI with the Comprehensive School Health Research Group supporting the SHAPES-PEI project, which collected data on smoking (YSS), healthy eating, physical activity and mental fitness. Among grade 5 students, 50% completed a Healthy Eating Module and 50% completed a Physical Activity Module. Among grade 6 students, 50% completed the YSS Module A, 25% completed the Healthy Eating module and 25% completed the Physical Activity module. In grades 7–12 in each school, 50% of the students completed YSS module B and 50% completed the SHAPES module (all questions)
In 2008/2009, grade 5 students were no longer included in the survey, primarily because of the low prevalence of smoking among students in this grade and the challenges of having students in this grade to complete the survey in the time allotted. Grade 6 students completed the survey without the alcohol and drug use questions whereas those in grades 7 to 12 completed a survey that included alcohol and drug use questions.
Collaboration. Whenever possible, YSS data collection was coordinated with other data collections taking place at the same time. In 2006/2007, YSS collaborated with the University of New Brunswick’s Health & Education Research Group (HERG) and with the Comprehensive School Health Research Group in Prince Edward Island to implement their provincial surveys in 2008/2009 (NB Wellness Survey and SHAPES-PEI, respectively). Both initiatives collected data on smoking (YSS), healthy eating (HE), physical activity (PA), and mental fitness (MF) from students in grades 5 to 12 (grades 6 to 12 for NB Wellness with the exception of YSS-sampled schools, which were grades 5 to 12). The data included a census of eligible schools in the respective provinces. The YSS dataset does not include any data collected from the NB Wellness or SHAPES-PEI additional modules, but the dataset does include the additional students who responded to the YSS. The data collection procedures therefore varied slightly for NB and PEI. Table 1 summarizes these differences in data collection.
In 1994, the sample design consisted of a two-stage stratified clustered design in which schools were the primary sampling units and classes were the secondary units. There were two levels of stratification. Each province was the main stratum and there was an implicit stratification by grade. The school sample was selected systematically with probability proportional to school size (the total number of students for each grade). Classes within schools were randomly selected and all students in a selected class were included in the final sample.
In 2002, the sample design featured three levels of stratification. Each province was the main stratum and there was an implicit stratification by grade. Schools were also stratified by census metropolitan area (CMA) versus non-CMA, with additional strata in Quebec (Montréal) and Ontario (Toronto). The sample was then selected in each stratum independently, meaning that some schools could be selected more than once for different grades. Classes were randomly selected from the schools that were recruited.
In 2004/2005, the sampling was conducted in two stages. In stage 1, school boards were sampled within each province. The Canadian Community Health Survey (CCHS) was used to estimate the current smoking rate at the level of health region. Estimated adult smoking rates were calculated for each school board and the school boards were ranked and categorized as “upper stratum” or “lower stratum.” In stage 2, schools were sampled from the list of selected school boards. School boards were selected based on their adult smoking rate. Within each selected school board, schools were stratified into two strata: senior strata (students in senior elementary or high school grades) or junior strata (students in a school with grades 5, 6, 5–6, 5–7, and 6–7). Where possible, there was an over-selection of junior stratum schools. All eligible grades within a school were selected to participate, rather than just a random selection of classes within a school.
The sample design in 2006/2007 was the same as the design described for the 2008/2009 survey cycle with a few small exceptions. The smoking rate calculated for the province and health region was based on adult smoking rates, and there was no separate stratum for the Greater Toronto Area. Again, all classes in eligible grades in selected schools were surveyed.
The Youth Smoking Survey is a nationally representative school-based survey of youth in Canada. The YSS was designed to provide both national (excluding Yukon, Northwest Territories and Nunavut) and provincial estimates of smoking prevalence, as well as surveillance of tobacco-related knowledge, attitudes and behaviours of young people in Canada. However, the YSS is more than a surveillance tool. It was designed to assess and help develop public education programs and policies for tobacco control. With the integration of the YSS with the SHAPES, the YSS is even more capable of integrating tobacco control policy and practice and monitoring the effectiveness of tobacco control strategies through the school-specific feedback reports.
There are several unique features of the 2008/2009 YSS when compared to other Canadian surveys:
The YSS has been used to guide tobacco control policies and programs nationally. For instance, the 2008/2009 YSS data were instrumental in prompting the federal government to amend the Tobacco Act in 2009 as part of Bill C-32 to prohibit the use of flavour additives in cigars and cigarillos.§http://www.propel.uwaterloo.ca/index.cfm?section=28&page=377
The authors would like to thank the Propel Centre for Population Health Impact for providing support for this project. Dr. Leatherdale is a Cancer Care Ontario Research Chair in Population Studies. The 2008/2009 Youth Smoking Survey is a product of a pan-Canadian capacity-building project that includes Canadian researchers from all provinces and provides training opportunities for university students at all levels. Production of this paper has been made possible through a financial contribution from Health Canada. The views expressed herein do not necessarily represent the views of Health Canada. The authors recognize the key role that Murray Kaiserman, now retired Director in Health Canada's Tobacco Control Programme and the Controlled Substances and Tobacco Directorate, has played in the initiation and continued funding of the YSS.