T. Elton-Marshall, PhD (1,2); S. T. Leatherdale, PhD (2); S. R. Manske, EdD (1); K. Wong, MSc (1); R. Ahmed, PhD (1,3); R. Burkhalter, MMath (1)
This article has been peer reviewed.
Correspondence: Scott T. Leatherdale, School of Public Health and Health Systems, Faculty of Applied Health Sciences, University of Waterloo, 200 University Ave. West, Waterloo ON N2L 3G1; Tel.: (519) 888-4567 ext 37812; Fax: (519) 886-6424; Email: firstname.lastname@example.org
This paper describes the survey development, design and data collection protocol for the 2008/2009 Youth Smoking Survey (YSS) and the changes to the YSS survey and protocols across the 5 survey cycles (1994, 2002, 2004/2005, 2006/2007, 2008/2009). Canada’s Youth Smoking Survey is a nationally representative school-based survey of students (grades 6 to 12 in 2008/2009) from randomly sampled public and private schools in the ten provinces. The main objective of the YSS is to provide benchmark data on national smoking prevalence rates for youth. Key features of the 2008/2009 YSS include consistent measures across survey cycles, a survey team of researchers and non-governmental organizations, a link to school and student level measures, provision of tailored feedback reports to schools and publicly available datasets.
Keywords: youth, smoking behaviour, Canadian Youth Smoking Survey, survey cycles, questionnaires
Nationally representative surveys of youth smoking behaviour are necessary to understand the social, regulatory, educational and commercial factors that influence smoking; to provide evidence for tobacco control policies and programs; and to monitor tobacco consumption in Canada.1 The Youth Smoking Survey (YSS) is the only school-based national survey of youth smoking in Canada. The YSS is a cross-sectional classroom-based survey of a representative sample of schools in the 10 Canadian provinces. When first administered in 1994, it was the largest and most comprehensive survey on youth smoking behaviour since 1979 for students in grades 5 to 9. To date, five survey cycles have been conducted (1994, 2002, 2004/2005, 2006/2007, 2008/2009) to monitor changes over time. In 2006/2007, the YSS survey was extended beyond grade 9 to include all other grades of secondary school students (i.e. grades 10 to 12 in most provinces and in Quebec, Secondaire IV to V). The population coverage for YSS 2008/2009 was similar to the YSS 2006/2007 except that grade 5 students were excluded due to the very low smoking rate in this age group.
The YSS is undertaken with the cooperation, support and funding of the Controlled Substances and Tobacco Directorate, Health Canada. The research team is pan-Canadian, interdisciplinary, and from university and non-governmental organizations across the country. The main objective of the YSS is to provide comparable benchmark data on national prevalence rates for youth every two years to guide policy and practice decisions. In addition, it provides a unique opportunity to advance our knowledge of the psychosocial correlates of smoking behaviour, including initiation and cessation. It can help examine individual differences in the influence of tobacco marketing, purchasing controls and other policy initiatives. The YSS offers a detailed snapshot of how youth buy or get cigarettes and of smoking behaviours, and the effects of continued tobacco marketing. This information is critical to assessing the need for increased legislative controls on tobacco and bolstering public support for these policy options. Interventions directed at children and youth are easy for legislators and the populace to support and often support tobacco use reduction in adults as well. Without this type of monitoring, we cannot gauge the effectiveness of our prevention efforts.
This paper describes the survey development, design, and data collection protocol for the 2008/2009 YSS and highlights changes to this cycle relative to the previous four. Additional information on the design, measures and protocols of this and previous cycles of the YSS are available online.*
A pan-Canadian consortium of university and non-governmental organizations implemented the 2008/2009 YSS. Members of the Youth Health Team at the Propel Centre for Population Health Impact at the University of Waterloo (Ontario) provided central leadership, while members from the other nine provinces provided leadership in their respective provinces. Members developed survey content during teleconferences. Those who could not participate in the scheduled meetings were asked to provide input prior to the teleconference. This approach allowed provincial stakeholders and the federal government to ensure the survey content included measures relevant to each jurisdiction. Content meetings ensured that core items (those required to compute smoking prevalence rates and derive other key, comparable variables) were retained. Questions added to the existing survey were those deemed higher in priority particularly if they were relevant to active policy agendas. The consortium made consensus decisions about which questions to include in the survey after discussing the merit of all survey questions during team teleconferences.
Each iteration of the YSS allows for a few new items; however, for every addition, about the same number is removed to keep the questionnaire the same length. Those items that tend to appear every other cycle are considered “periodic.” Some items, known as “deleted items,” are phased out completely if the issue/question is no longer relevant. While consistent content permits monitoring of trends over time, introducing new items permits identifying new trends that need to be monitored (see the 2008 YSS user guide for a list of survey items by cycle).2
Several key considerations guided the development of content for the 2008/2009 YSS:
Prior to implementation, the survey questionnaire was pilot tested (in both French and English). During the two-hour pilot-testing sessions, students representing smokers and non-smokers from all grades completed the questionnaire independently and were encouraged to write comments/questions while doing so. Respondents then participated in a 75-minute focus group discussion in their first language led by a moderator using a pre-developed survey guide. The moderator explored students’ comprehension of the survey questions (with particular focus on all new questions), the logic and order of the questions, and overall flow of the questionnaire. The objectives of the pilot-testing sessions were to: (1) assess the length of time required to complete the survey; (2) probe students’ comprehension of the survey questions (with particular focus on all new questions); and (3) test the logic and order of the questions, including overall flow of the survey instrument. Changes to the survey were based on the feedback obtained in these sessions. Health Canada and the implementation team jointly decided on questionnaire revisions based on these pilot results.
Many of the items that have been used in other youth smoking surveys (e.g. Global Youth Tobacco Survey,3 Ontario Student Drug Use and Health Survey4) have been found to be reliable (e.g. current alcohol, marijuana, and tobacco use questions in the Youth Risk Behaviour Survey)5,6 and have been validated in other studies (e.g. assessing attitudes towards smoking, smoking intentions).7
All protocols and materials, including the final survey instrument, received ethics approval from the University of Waterloo Office of Research Ethics and local institutional review boards where required (e.g. in some cases, from two additional levels: the provincial host institution and the school board).
Core measures. To be consistent, “core” survey measures remain the same across all survey years. These include the measures used to define the smoking status of each respondent according to Health Canada definitions, measures of key prevention indicators such as susceptibility to future smoking, age of initiation and amount smoked, and key demographic variables. The core outcomes measured in the YSS are susceptibility to future smoking among never smokers and smoking status. The validated algorithm of Pierce et al. was used to measure susceptibility to future smoking among never smokers (those who have not smoked even a few puffs of a cigarette).8 Susceptibility was determined from responses on a 4-point Likert scale to the following questions: “Do you think in the future you might try smoking cigarettes?”; “If one of your best friends was to offer you a cigarette, would you smoke it?” and “At any time during the next year do you think you will smoke a cigarette?” Never smokers who answered “definitely not” to all three questions were considered non-susceptible; they were considered susceptible to future smoking if they responded positively to at least one of the questions.
Smoking status was determined by asking respondents if they had ever tried a cigarette (even just a few puffs), if they had ever smoked a whole cigarette, if they had ever smoked 100 or more whole cigarettes in their lifetime, and on how many of the last 30 days they had smoked one or more cigarettes. Consistent with Health Canada’s operational definitions of smoking status for the YSS,9 respondents were then grouped into the following eight categories: daily smoker (smoked at least 100 cigarettes and currently smokes cigarettes every day); occasional smoker (smoked at least 100 cigarettes and currently smokes cigarettes but not every day); former smoker (smoked at least 100 cigarettes but had not smoked in the last 30 days); experimental smoker (smoked in the last 30 days but had not smoked at least 100 cigarettes); past experimental smoker (had smoked a whole cigarette but had not smoked in the last 30 days and had not smoked at least 100 cigarettes); puffer (had tried smoking but has not smoked a whole cigarette) and never tried (never tried a cigarette, not even a few puffs).
Non-core questions. Non-core questions provided information on such issues as where and how youth obtained cigarettes, exposure to second-hand smoke, awareness of health risks due to smoking, and attitudes and beliefs and related health behaviours. Answers to these questions help understand smoking behaviour and uptake among youth, as well as other associated behaviours (e.g. watching television, playing video games). (See Appendix A of the 2008 microdata file to see a comprehensive list of questions and the survey cycles in which these questions appeared).2
Skip patterns. The youth questionnaire was intentionally designed with no respondent-use skip patterns to avoid identifying smokers by rate of survey completion during the classroom session. Thus all smoking behaviour items included a response option such as “I do not smoke.” However, due to the logical flow of the questions, a number of questions were extraneous based on the answer to a previous question. In these cases, a skip pattern was imposed within the operational definitions for appropriate measures within the public use metafile (PUMF), the de-identified dataset available to researchers. If a question could be skipped within the structure of the questionnaire, it was coded as 96 or 996 or 9996 within the PUMF dataset. For example, a smoker would still be asked questions about susceptibility to smoking but the responses for those questions would be coded as a “valid skip” and would be excluded from the analyses associated with smoking susceptibility.
Starting with the 2004/2005 cycle, the YSS used the School Health Action, Planning and Evaluation System (SHAPES) for school-based data collection. Thus each participating school received a school-specific feedback report and executive summary within 10 weeks of data collection. This report provides customized information including smoking rates and other behavioural (e.g. time spent reading) and environmental information (e.g. smoking on school property) specific to the school. As a supplement to the YSS, information about the school environment (programs, policies and the built environment) was also collected.†
The target population for the YSS consisted of all young Canadian residents attending private and publicly funded schools in the 10 Canadian provinces. Those residing in the Yukon, Nunavut and Northwest Territories and those living in institutions or on First Nations reserves were not included in the sampling frame. Young persons who were attending special schools (e.g. schools for visually and hearing-impaired) or schools located on military bases were also excluded from the sampling frame.
The YSS team at the Propel Centre obtained a comprehensive list of all schools in each province via provincial Department of Education websites. The sampling for the YSS was based on a stratified multistage design. Sampling was stratified according to health region smoking rate and type of school (elementary or secondary). In Stage 1, the Canadian Community Health Survey (CCHS) was used to calculate the smoking rate among 15 to 19 year olds for each health region. The school lists obtained from the provincial Departments of Education for each of the 10 provinces included enrolment data by grade for each school. Using this list, the total eligible grade enrolment in a health region was used as a weight to compute the median smoking rate for each province. Each school’s six-digit postal code was used to identify the health region in which it was located. Schools were then categorized as “low” or “high” smoking rate stratum based on the smoking rate in their health region compared to the median (where greater than or equal to the median was categorized as “high”).
In Stage 2, schools were stratified into elementary or secondary school strata (calculated based on whether there was a higher enrolment of students in grades 6 to 8 or 9 to 12). Elementary and secondary schools were sampled on a 2:1 ratio due to the smaller enrolment sizes of the elementary schools. Schools were also over-sampled in each province based on the provincial school recruitment rate from the 2004/2005 and 2006/2007 YSS cycles.
In Ontario, the design of the 2008/2009 cycle included a third health region stratum, Greater Toronto Area (GTA). The GTA health region stratum acknowledged the size of the GTA and the importance of being able to capture schools from the GTA even if there were refusals from the larger school boards in the city of Toronto.
Lastly, sampling of private schools was based on a simple random sample of private schools in each province. The number of schools originally selected was roughly proportional to the number of students enrolled in private schools in that province as compared to the total in public schools. The sampling design is constructed to provide a representative sample of youth in all provinces in Canada.
In the 2008/2009 cycle, the school board response rate was 84% (the number of school boards that agreed to participate/the number of school boards that were approached); the school level response rate was 59% (the number of schools that agreed to participate/the number of schools that were approached); and the student level response rate was 73.2% (based on the number of completed surveys/the number of eligible students; students who were absent during the data collection were counted as a non-response).
In all provinces, YSS site coordinators contacted school boards prior to approaching schools. Private schools were approached directly because there is no governing board to review research requests for these schools. School boards were typically contacted via a formal board-specific application or a standard board recruitment package that included a school invitation letter, a project brochure, a sample student survey, sample parent information and permission materials, and a template school feedback report.‡ Provincial site coordinators made follow-up calls to the school board to answer any questions and, ideally, obtain board permission to recruit schools. Once a school board was successfully recruited, the schools within that school board were approached via a school recruitment package and follow-up phone calls. The contents of the school recruitment packages were the same for both boards and schools. Only when the school had agreed to participate in the YSS was the survey implemented with eligible students in that school.
Within each participating school, all students in the eligible survey grades (6 to 12) were requested to complete the survey. Active parental permission was required by the school or board for 62% of grade 6 to 8 classes (n = 913) and 19% of grade 9 to 12 classes (n = 372). Students in eligible classrooms took home information letters describing survey details. Active permission protocols required signed parental and child permission forms for the child to receive and complete a survey. In 81% of secondary school classes (n = 1631), passive permission protocols were used to reduce the burden on schools and improve response rates. In this procedure, the school mailed an information letter home to parents that detailed survey procedures, and asked parents to call a toll-free number or inform the school if they did not want their child to participate. Students whose parents objected were put on a “no permission” list and did not receive a survey on the day of data collection. All other students received a survey to complete. Regardless of whether parents provided permission, students were able to decline participation on the day of data collection.
Provincial site coordinators worked with a school contact to arrange data collection at each school. On the day of data collection, teachers administered the survey using standardized protocols during a designated class period. To ensure confidentiality and therefore encourage honest responses, teachers were asked to avoid circulating among the students. Students were also required to place their completed survey in an envelope and seal this envelope before it was collected by a student in the classroom. When parents as well as students were surveyed, active consent was required, and a tear-off sheet with the student’s name was attached to the front of the survey. Students removed the tear-off sheet. A serial code on both the tear-off sheet and the student survey enabled linkage for survey cycles that included a parent interview to be linked to the student responses. The information containing the student’s identification and responses were removed from all public datasets and only those directly related to the research had access to any identifying information. On average, the survey took 30 to 40 minutes to complete. A data collector was on site at the school throughout the data collection period and available to answer respondent questions and collect the completed student surveys.
‡ For sample documents, e.g. surveys, feedback report, etc., see www.yss.uwaterloo.ca/recruitment.
Surveys were machine scanned using Optical Mark Reading (OMR) technology. Quality control measures (e.g. visual scanning, OMR scanning twice to find discrepancies) were used to ensure accuracy of the scanned data. An online survey implementation system (OSIS) permitted central management of recruitment, implementation, analysis and feedback processes.
Survey weights were created to “weight” the data to be representative of the general population of Canadian youth in school. The survey weights were developed in two stages. In the first stage, a weight (W1) was created to account for the school selection within health region and school strata. A second weight (W2) was then calculated to adjust for student non-response. The weights were then calibrated to the provincial gender and grade distribution so that the total of the survey weights by gender, grade and province would equal the actual enrolments in those groups. Finally, bootstrap weights for each province (to estimate sampling error) were generated.
The protocols described were used to implement the 2008/2009 YSS cycle. One of the strengths of the YSS has been its consistent protocols, which allow comparisons over cycles. However, there have been slight modifications to the sampling and protocols in each cycle based on experience in previous cycles; these modifications were made to improve student recruitment and survey completion rates, and to reduce the burden on participating schools. The following section describes some of the significant differences in the YSS over the various cycles.
Changes to survey administration. One of the most significant changes to the YSS occurred in 2004/2005 when the survey administration shifted from Statistics Canada to the University of Waterloo. In 1994 and 2002, the YSS content was developed by Health Canada’s Office of Tobacco Control and data were collected by Statistics Canada. As previously noted, the University of Waterloo’s Propel Centre for Population Impact (formerly the Population Health Research Group and the Centre for Behavioural Research and Program Evaluation) has provided central leadership since 2004/2005.
Changes to the survey. Table 1 summarizes the differences in the survey over time. Until 2006/2007, the sample included grades 5 to 9 only. In 1994, all students in grades 5 to 9 responded to the same survey. In 2002 and 2004/2005, students in grades 7 to 9 answered additional questions about alcohol and drug use. In 2006/2007, students in grades 7 to 12 were randomly assigned to receive one of two versions of the survey. While the majority of the questions were the same in both versions, including those that related to alcohol and drug use, some different questions were added to each (e.g. in one version there were questions about smoking on school property whereas another version had questions about beliefs about the harmful effects of smoking). All other students (grades 5 and 6) received a survey with no questions on alcohol and drug use. Because there were two different versions of the survey in this cycle, there were two survey weights calculated for this dataset and two User Guides to facilitate use of the dataset.
|Survey cycle||Survey dates||Target population, grades||Sample size
|Changes to the survey protocol|
|1994||Sep–Nov 1994||5–9||14 270|
|2002||Oct–Dec 2002||5–9||19 018||Students in grades 7–9 answered additional questions about alcohol and drug use|
|Feb–Jun 2005||5–9||29 243||Adoption of SHAPES (School Health Action Planning and Evaluation System)
Computer-generated feedback reports delivered to schools
Surveys machine-scanned using Optical Mark Read (OMR) technology
|Nov 2006–Jun 2007||5–12||71 003||
Addition of grades 10–12
Students in grades 7–12 completed either Module B1 or B2:
In New Brunswick, data were collected to support the Healthy New Brunswick en santé project (data on smoking using YSS, healthy eating, physical activity, and mental fitness)
Census of schools in New Brunswick
In New Brunswick, 50% of students in grades 5–6 completed the YSS Module A, 25% of students completed a Physical Activity Module and 25% completed a Healthy Eating Module. Within each class in grades 7–12, 25% of students completed the YSS Module B1, 25% of students completed the YSS Module B2, 25% of students completed a Physical Activity Module and 25% of students completed a Healthy Eating Module
|Dec 2008–Jun 2009||6–12||51 922||
Grade 5 students no longer included in the survey
The student survey data were collected using two instruments:
Collaboration in PEI with the Comprehensive School Health Research Group supporting the SHAPES-PEI project, which collected data on smoking (YSS), healthy eating, physical activity and mental fitness. Among grade 5 students, 50% completed a Healthy Eating Module and 50% completed a Physical Activity Module. Among grade 6 students, 50% completed the YSS Module A, 25% completed the Healthy Eating module and 25% completed the Physical Activity module. In grades 7–12 in each school, 50% of the students completed YSS module B and 50% completed the SHAPES module (all questions)
In 2008/2009, grade 5 students were no longer included in the survey, primarily because of the low prevalence of smoking among students in this grade and the challenges of having students in this grade to complete the survey in the time allotted. Grade 6 students completed the survey without the alcohol and drug use questions whereas those in grades 7 to 12 completed a survey that included alcohol and drug use questions.
Collaboration. Whenever possible, YSS data collection was coordinated with other data collections taking place at the same time. In 2006/2007, YSS collaborated with the University of New Brunswick’s Health & Education Research Group (HERG) and with the Comprehensive School Health Research Group in Prince Edward Island to implement their provincial surveys in 2008/2009 (NB Wellness Survey and SHAPES-PEI, respectively). Both initiatives collected data on smoking (YSS), healthy eating (HE), physical activity (PA), and mental fitness (MF) from students in grades 5 to 12 (grades 6 to 12 for NB Wellness with the exception of YSS-sampled schools, which were grades 5 to 12). The data included a census of eligible schools in the respective provinces. The YSS dataset does not include any data collected from the NB Wellness or SHAPES-PEI additional modules, but the dataset does include the additional students who responded to the YSS. The data collection procedures therefore varied slightly for NB and PEI. Table 1 summarizes these differences in data collection.
In 1994, the sample design consisted of a two-stage stratified clustered design in which schools were the primary sampling units and classes were the secondary units. There were two levels of stratification. Each province was the main stratum and there was an implicit stratification by grade. The school sample was selected systematically with probability proportional to school size (the total number of students for each grade). Classes within schools were randomly selected and all students in a selected class were included in the final sample.
In 2002, the sample design featured three levels of stratification. Each province was the main stratum and there was an implicit stratification by grade. Schools were also stratified by census metropolitan area (CMA) versus non-CMA, with additional strata in Quebec (Montréal) and Ontario (Toronto). The sample was then selected in each stratum independently, meaning that some schools could be selected more than once for different grades. Classes were randomly selected from the schools that were recruited.
In 2004/2005, the sampling was conducted in two stages. In stage 1, school boards were sampled within each province. The Canadian Community Health Survey (CCHS) was used to estimate the current smoking rate at the level of health region. Estimated adult smoking rates were calculated for each school board and the school boards were ranked and categorized as “upper stratum” or “lower stratum.” In stage 2, schools were sampled from the list of selected school boards. School boards were selected based on their adult smoking rate. Within each selected school board, schools were stratified into two strata: senior strata (students in senior elementary or high school grades) or junior strata (students in a school with grades 5, 6, 5–6, 5–7, and 6–7). Where possible, there was an over-selection of junior stratum schools. All eligible grades within a school were selected to participate, rather than just a random selection of classes within a school.
The sample design in 2006/2007 was the same as the design described for the 2008/2009 survey cycle with a few small exceptions. The smoking rate calculated for the province and health region was based on adult smoking rates, and there was no separate stratum for the Greater Toronto Area. Again, all classes in eligible grades in selected schools were surveyed.
The Youth Smoking Survey is a nationally representative school-based survey of youth in Canada. The YSS was designed to provide both national (excluding Yukon, Northwest Territories and Nunavut) and provincial estimates of smoking prevalence, as well as surveillance of tobacco-related knowledge, attitudes and behaviours of young people in Canada. However, the YSS is more than a surveillance tool. It was designed to assess and help develop public education programs and policies for tobacco control. With the integration of the YSS with the SHAPES, the YSS is even more capable of integrating tobacco control policy and practice and monitoring the effectiveness of tobacco control strategies through the school-specific feedback reports.
There are several unique features of the 2008/2009 YSS when compared to other Canadian surveys:
The YSS has been used to guide tobacco control policies and programs nationally. For instance, the 2008/2009 YSS data were instrumental in prompting the federal government to amend the Tobacco Act in 2009 as part of Bill C-32 to prohibit the use of flavour additives in cigars and cigarillos.11 YSS data have also been used to inform provincial tobacco control policies and strategies. For instance, during the 2010 renewal of the Smoke-Free Ontario Strategy, YSS data played a key role in informing the policy recommendations in Chapter 5 of the new guide for comprehensive tobacco control in Ontario.12 The YSS has also been used by researchers to understand tobacco use among youth in Canada and to identify and inform future tobacco control priorities including tobacco use among off-reserve Aboriginal youth in Canada,13 contraband cigarettes use,14 bidi and hookah use,15 alcohol and illicit substance use,16-19 cigarette brand preferences and price,20 taxation,21 second-hand smoke exposure,22,23 cigarette access,24 school policies and smoking,25 socialization towards smoking26,27 and smoking among adolescent girls.28
The YSS has expanded to collect relevant information on other risk behaviours (physical activity, obesity, healthy eating). This data will be used to make future policy and programming decisions regarding other health policies in addition to tobacco control. The 2010/2011 cycle of the YSS is currently in progress, and we hope that researchers and policymakers will continue to use this important dataset to understand tobacco use and other risk behaviours among youth in Canada.
The authors would like to thank the Propel Centre for Population Health Impact for providing support for this project. Dr. Leatherdale is a Cancer Care Ontario Research Chair in Population Studies. The 2008/2009 Youth Smoking Survey is a product of a pan-Canadian capacity-building project that includes Canadian researchers from all provinces and provides training opportunities for university students at all levels. Production of this paper has been made possible through a financial contribution from Health Canada. The views expressed herein do not necessarily represent the views of Health Canada. The authors recognize the key role that Murray Kaiserman, now retired Director in Health Canada's Tobacco Control Programme and the Controlled Substances and Tobacco Directorate, has played in the initiation and continued funding of the YSS.