Quality Determinants of Organized Breast Cancer Screening Programs

Special Projects

[Previous] [Table of Contents] [Next]

9.1 Quality Assurance Survey

Fulfilling a request by the National Committee of the Canadian Breast Cancer Screening Initiative, the subset Working Group on Quality Determinants of Organized Breast Cancer Screening Programs developed the Quality Assurance Survey as a means of assessing current practices across Canada. The first wave of the survey was carried out in 1998, the second in 2002, with 11 provinces and territories participating in both surveys. It is evident that there had been a notable advance in organized screening program practices, even within the short interval between the two sets of results. For instance, all organized breast cancer screening programs are now established and continue to grow at a rapid rate. Yet it is likely that there will always remain some areas needing further improvement.

Quality assurance for screening programs is mainly implemented on a provincial/ territorial basis. In 1996, six programs had at least one designated individual responsible for data quality assurance; nine programs now have assigned individuals. Programs maintain documentation related to screening as needed, including a standard set of relevant definitions, data collection procedure manuals, data coding manuals/data dictionary, and protocols for quality assurance in data collection. These are reviewed on an annual or biennial basis. The quality of data for surveillance purposes is improving with time and better technology. All but one province use a combination of automated and manual processes to perform data quality tests, including checks for missing values, duplicate records, valid ranges of values, intra-record agreement, coding procedures, and checks against medical records.

On a screening facility level many components are collected in the clinical database, including client registration, risk factor information, recruitment information, history of breast procedure, the results of clinical examination and screening tests, and information on further assessments. Also, information on detected cancers, treatment, and recurrence is also collected, but not as widely across the country. The number of programs that amalgamate regional clinical database information on a provincial or territorial basis varies, which may simply be a reflection of program organization. For instance, one program has a system that automatically combines information into both a local and provincial database; another program maintains a provincial database that is accessible at a screening facility level. In most cases, programs have designated an individual to analyze these data and to produce program results on a monthly or annual basis.

With this information, over time we will be better able to assess program impact on breast cancer incidence and mortality. Surveillance data on case fatality, number of deaths prevented, life-years gained, quality of life, and the overall cost-effectiveness of screening can help gauge the effectiveness of screening. The use of these measures can only come with time - years of data need to be available for analysis in order to assess outcomes. In the 1998 survey, only one province looked at one of these factors (cost-effectiveness), and four programs were planning to assess one or more. As seen in the 2002 survey, one program now assesses three of the measures, and five programs are eventually planning to do so.

Since the 1998 survey, several programs have noted some new directions in quality assurance practices. One program has been working with provincial partners to develop a province-wide screening program that collects data from both the organized screening program and other service providers. With cooperation between partners, data for the province will be more comprehensive and can be further refined. Another program has improved its quality management support to screening centres through the creation of a full-time managerial position to provide quality management and resources.

9.1.1 A client service approach

In 1998, only four programs had official policies in place to ensure that women were fully informed about the screening examination procedure, although no detail was requested in the survey regarding the type of policy or how clients were prepared in the other seven programs. In 2002, 10 programs have established policies to ensure that full information is provided to clients. For the remaining program, the policies are not developed on a province-wide basis but are applied at a regional level. Programs use a variety of media to relay this information; most frequently this includes showing a video, providing a pamphlet, or communicating orally.

All programs have procedures in place to maintain the confidentiality of patient data (in both hard copy and electronic format); in 1998, only nine programs had this in place. Government regulations to protect the privacy of individuals, written policies on data confidentiality, and written procedures specifying proper handling of patient data all have a strong influence on the protection of confidentiality. Other areas that improve confidentiality include the provision of orientation and training in data confidentiality issues for program staff, and defining an approval process for the release of patient data. As well, three programs have a data confidentiality committee or officer on hand.

As in 1998, client satisfaction is monitored by 10 of the responding programs - four do so routinely, and six do so sporadically. Client satisfaction is monitored through the provision of a "comments" section on forms, the provision of a telephone number and contact information, or the circulation of "client satisfaction" questionnaires.

9.1.2 Recruitment

Most provinces still quote the standard recruitment target of 70% of women aged 50-69 years within 10 years of program establishment. Currently, eight programs have dedicated personnel responsible for outreach and recruitment activities; 10 programs conduct these activities when capacity permits. Targets are always difficult to attain, but breast screening programs across Canada are aided by an ability to identify women for recruitment purposes. With the available information, programs claim 75% to 100% accuracy (with a mean of 91%) in identifying eligible women for screening (i.e. completeness of registration, address accuracy). Previously, it ranged from 80% to 97%, with a mean of 93%. Most programs have and continue to use data from medical insurance/service plans to identify women, although motor vehicle licensing lists, community group membership lists, and provincial client registries are being used to a lesser extent. If multiple sources are used, duplicate entries are dealt with through the use of matching software or through manual resolution.

Within the period between the two surveys, the number of existing facilities grew from 169 to 236, with some facilities holding multiple mammography units. In addition, the number of mobile units in use continues to grow; now 15 mobile units exist to serve Canadians. Despite this, capacity is currently an issue for these established programs. Although some programs are able to serve most of their target population, with current capacity levels others can only serve 20%.

While mass recruitment approaches exist, women are mainly recruited through the issuing of personal invitations by mail. Several checks are made before issuing invitations. The majority of provinces check recruitment data against breast cancer registration, program attendance records, and death registration. The timing and frequency of these checks are varied, although all programs usually ensure that they are done when individuals are first identified and when invitations are issued. If there is no response from potential first time screening clients, five programs out of eight send second invitations, usually within 2 to 14 weeks. Most programs send first screen invitations biennially until a response is received; two programs limit the number of attempted first screen invitations to two.

9.1.3 Retention

All programs issue recalls for rescreening when applicable, although one program only does this some of the time. When there is no response, a recall reminder is issued by at least seven of the programs, sent within 2 to 14 weeks, but varying by program. If there is still no response, further follow-up is taken by five of these seven programs through the use of telephone calls or reminders in subsequent years. Similar to first screen invitations, checks are performed using breast cancer registration, program appointment records, and death registration information.

As in 1998, seven programs monitor reasons for nonattendance at their breast cancer screening program. Two programs do so routinely, whereas the other five do this sporadically.

9.1.4 Quality of mammography

According to the provincial and territorial programs, in 1998 CAR accreditation was mandatory in seven programs while being voluntary in the remaining four, and in 2002 it is mandatory in eight programs while remaining voluntary in three. In addition to CAR accreditation, six programs participate in provincial/regional formal accreditation processes. These processes are administered by various authorities (i.e. health district or facility), usually renewed every 3 years, and oversee a varied combination of two or more of the following areas: technical (equipment and image quality), certification of education and experience of technologists and radiologists, communication of results, follow-up of abnormal results (diagnostic and treatment), information systems, client care, or environmental management.

Site visits were an important part of quality control in 1998, 10 programs receiving visits. In 2002, seven programs conducted periodic site visits to the mammography facilities, covering such aspects as equipment, personnel training, recruitment of target population, communication of results, client satisfaction, follow-up of abnormal results (diagnosis, treatment), and information systems. These aspects are mandatory or voluntary, varying by program, and most are done on an annual basis, once again varying by program. As in 1998, results are given to the screening program as well as to the facility that was inspected. In addition to image audits performed for formal accreditation, for five programs it is a mandatory requirement that facilities periodically submit films for internal image audit.

For most programs, X-ray machines, image receptor systems, and processor equipment are calibrated for dedicated mammography use. Each program provided detailed information on quality control tests performed on a variety of equipment. On the whole, programs are comparable in the frequency with which tests are performed as well as in the individuals who perform the tests (i.e. technologists, radiologists). The quality control guidelines most commonly followed are those specified by CAR.

Across Canada, the two most frequently listed sources of accreditation for radiologists are the Royal College of Physicians and Surgeons and CAR. Program requirements for radiologists include specialized training in mammography, success in standard tests, and documentation of 15 to 40 hours of continuing medical education (CME) credits in mammography every 3 years. Nine programs list a minimum number of mammograms to be read by radiologists. These minimums range from 480 (three programs), to 3,000 (three programs). For three programs, this minimum is required, whereas it is only recommended in two other programs. Compared with the 1998 survey, there is now less variation among programs and a move towards higher minimums.

Technologists are required to be certified (usually with CAMRT as the registered source) and are required to have completed Mammography I and II of the CAMRT quality control program. One program allows technologists to read screening mammograms as a first reader, although it is unofficially done in two other programs as well. The required training to do this includes an intensive mammography interpretation course such as that offered by Dr. Tabar. Programs list additional criteria for technologists, including preferred experience of a minimum of 1 to 2 years, initial training lasting between 35 hours and 6 weeks, and 15 hours of CME credits in mammography every 3 years. Five programs list a minimum number of screening mammograms to be performed by technologists. Minimums range from 1,000 to approximately 2,600 examinations a year. In general, these requirements are higher than they were in 1998.

Initial orientation or training is provided in all programs; training in one program varies across its regional sites. Training for clerical staff lasts from 1 day to 4 weeks. For radiologists or interpreters, eight programs provide initial training, which lasts from 1 day to 2 weeks. For technologists, initial training lasts from 1 day to 4 weeks. Additional staff, such as technologists who interpret, physicists, and examiners, are given initial training as well. Seven programs also offer continuing education to clerical staff, usually recommended when needed. Eight programs offer continuing education opportunities to radiologists (interpreters); nine programs offer this to technologists. Continuing education for radiologists and technologists is usually guided by CME credit requirements. Two programs provide continuing education to technologists who interpret, three programs offer this to physicists, and one offers it to examiners.

9.1.5 Screening examination procedures

An important policy addresses the need for pain minimization during a mammogram. Only one additional program has newly created this policy since the 1998 survey, bringing the total to 7 programs with pain minimization policies. In contrast, all programs provide policies and procedures on proper positioning during the screening examination. This is an improvement from 1998, when only eight programs had these policies. However, as in 1998, only six programs currently require that feedback on the quality of positioning be given by a supervisor.

All but one program monitor the percentage of films retaken for technical reasons. The standard for repeat rates for technical reasons varied from a maximum of 2% to 5%, the majority of programs listing 5% as their maximum. Seven programs employ double reading of films; in 1998, eight programs employed this process. The process within a program remained similar from 1998 to 2002; however, it differs among programs. The percentage of films that are double read in each program varies from 5% to 100% of all films. In general, most programs use random selection for this process, and to a lesser degree films may be chosen by the radiologist, or can be a combination of random and problem cases. The tendency is to have the second reader blind to the first, while providing the results of the second reading back to the first reader. Disagreements between the two readers are resolved by soliciting advice from a third reader (blinded or nonblinded), arriving at a consensus, or keeping the most anomalous result.

Rates have been developed to monitor the progress of screening programs. Six programs have specified standards on the percentage of mammograms found to be abnormal. General (overall) standards range from 7% to 10% of all mammograms. More specifically, for first screens, standards range from 7% to 15% of all screens, and for subsequent screens, 5% to 7%. Ranges are consistent with what they were in 1998, but only five programs had set standards at that time. There are no set standards regarding the percentage of women with abnormal mammograms undergoing fine needle aspiration (FNAs); only one program monitors the rate of unsatisfactory FNAs. On the whole, programs currently do not monitor the percentage of women with abnormal mammograms undergoing core or surgical biopsies. Five programs now have standards for breast cancer detection rates, whereas in 1998 four programs did. Overall standards for breast cancer detection are generally not reported, but for first screens, rates range from 5% to 8%, and for subsequent screens standard rates range from 2% to 5%. These ranges are wider - yet higher - than those set in 1998. Two programs that currently do not set their own standards still compare their rates against international standards.

9.1.6 Reporting

In all but one program, standardized forms are used to report the interpretation of screening mammograms. For the one program without standardized forms, the required elements to be captured are specified, but the actual forms may vary from region to region.

9.1.7 Communicating results

Currently, nine programs provide screening results to both the patient and physician, regardless of whether the results are normal or abnormal. In all cases, programs try to ensure that the physician is always the first to receive the news. In the two remaining programs, the communication of results to the patient is controlled. For one program, neither normal nor abnormal results are given to the patient but are given to the patient's physician. For the other program, the patient is provided only normal results, whereas the physician receives normal or abnormal results.

During the process of communicating results, seven programs monitor the timeliness of reporting normal results, five programs monitor the receipt of abnormal results, and eight programs monitor the timeliness of reporting abnormal results. Only two programs monitor the receipt of normal results.

9.1.8 Diagnostic workup

All programs now have a mechanism in place to monitor follow-up investigation of an abnormal screening result. In 1998, only eight programs were able to monitor this. Previously, a time limit for investigation was given by only three programs, but five programs now do so. The time limit varies by program in detail, length, and in the manner in which adherence is monitored. Only one program currently has guidelines detailing the clinical pathway and time interval between procedures for assessing screen-detected abnormalities.

Many programs have procedures in place to help expedite the diagnostic process. These include tracking progress through the family doctor; using direct referral from screening program to diagnostic facility with prior consent of the family doctor; and arranging appointments directly with the diagnostic facility. As in 1998, five programs provide assessment services for screen-detected abnormalities at one or more of their screening facilities. The methods of referral to assessment services vary across programs, but most use direct referral or referral by the family doctor upon receipt of the screening report. Services provided include clinical breast examination, diagnostic mammography, ultrasound, surgical consultation, fine needle aspiration, cytology service, core biopsy, or biopsy. Three programs provide all of these, one provides six of them, and one provides two of the services.

Procedures are in place for eight programs to review the diagnostic workup of cases with abnormal screening results. Five programs review all abnormal screens, and three review a selected number of the abnormal screens. These reviews are mainly performed by radiologists and/or technologists but can also be done by other members of the medical team (including resident medical staff) or at radiology quality assurance meetings. Eight screening programs currently link to cancer registries although the amount, flexibility, timeliness, and frequency of information exchange vary; three programs do this manually, and for five programs the linkages are computerized. The approximate time lag is between 2 and 12 months, with a mode of 6 months. The comprehensiveness of breast cancer information from this source ranges from 94% upward. Work is still needed to improve data flows and two-way communication of the full range of breast cancer diagnostic information, including staging data, between registries and screening programs. Seven programs have active access to data on survival and cause of death from additional sources, including vital statistics or contact with the patient, patient's family, or family physician.

Generally, data on lesions are collected from standardized data forms completed by pathologists, reports, hospital records, and data from the cancer registry. Programs differ in the way data are accessed - actively, passively, or both. More specifically, nine programs obtain cytology data on aspirated lesions, although one only does this at the facility level. Cytology data most often come from cytology or pathology data from the cancer registry; in 1998, data were most often collected from cytology reports. Pathology data on biopsied lesions (needle core biopsies and open incisional biopsies) for both benign and malignant cases are collected by all programs except one (which collects data for malignant cases only). Pathology data were and are still obtained most often from pathology reports. The type and number of variables collected from these sources used to be inconsistent; now, almost all programs have access to many of the elements regarding biopsies obtained for preoperative diagnosis.

A wide variety of tumour, histological, and nodal information is being collected on excised breast cancers for invasive and in situ carcinomas. Tumour size is based mostly on microscopic measurements by the pathologist, although sometimes it can be obtained from gross pathological measurement, mammographic measurement, or clinical assessment before or at the time of surgery. Nodal involvement is assessed on the basis of pathological (microscopic) evaluation, although sometimes it can be based on palpation.

In 1998 as well as today, eight programs ascertain nonprogram detected breast cancers in previous program participants. Data are obtained from the cancer registry, with supplemental information from linkages with the death registry, from relatives of the client, or from pathology reports. These cancers are all subject to radiological review, but the review protocol for interval cancers is still very different across the programs. Three programs have cases reviewed retrospectively by radiologists; four programs review cases through blind mixes at varying ratios; one program does both retrospective reviews and blind mixes.

In 1998, seven programs could ascertain cancers in the general, nonscreened population; now eight programs can do so. Sources for these data are mixed, although the majority of the information is obtained through the cancer registry. At this time, programs tend not to know whether these data are comparable to data on screen-detected cancers. Only one program adds general population cancers to the program database, but half of the programs have started to analyze these data.

9.2 Pan-Canadian Study

The Working Group recognizes the need to provide minimum reading volume guidelines for organized breast cancer screening programs in Canada. A study on the subject has been performed in the Screening Mammography Program of British Columbia, but with limited data available for analysis in the low volume range¹. Organized breast screening programs elsewhere in Canada could offer radiologist performance data over a wide range of annual reading volumes.

A new project was presented by Dr Nancy Wadden and accepted by medical directors of Canadian organized screening programs at the meeting of the Data Management Committee in May 2002. Financial support was approved by Health Canada in October 2002.

The purpose of the study is to determine the relation between annual screening volume and radiologists' performance in Canada. Abnormal interpretation ratios and cancer detection ratios will be used to summarize the performance. All the ratios will be adjusted for the age and screening history of the screened women (first versus subsequent screening exam). Different aspects influencing radiologist performance (e.g. years of experience, cancer detection rates) will be analyzed as well.

9.3 Reference

1. Kan L, Olivotto IA, Warren Burhenne LJ, Sickles EA, Coldman AJ. Standardized abnormal interpretation and cancer detection ratios to assess reading volume and reader performance in a breast screening program. Radiology 2000;215:563-567.

[Previous] [Table of Contents] [Next]