Framework and Tools for Evaluating Health Surveillance Systems

• Framework and Tools for Evaluating Health Surveillance Systems, March 2004, ( 756 KB)
  

TABLE OF CONTENTS

• Acknowledgements
• Executive Summary
• Background and Purpose
• Managing Surveillance Systems
  • Health Surveillance
  • Evaluation
  • Evaluation and Surveillance Systems
  • Results-based Management
• Evaluation Framework for Health Surveillance
  • Step 1 - Establishing the Context
  • Step 2 - Identifying Evaluation Questions
  • Step 3 - Data Collection / Management
  • Step 4 - Findings
    
  • Step 5 - Reviewing the Report
  • Step 6 - Follow-up
• Summary
• Appendix A - Example of Evaluation Planning Documents
• Appendix B -
  Surveillance System Specification Sheet (html)
• Surveillance System Specification Sheet( 35 KB)
• Appendix C - Glossary of Surveillance Evaluation Framework Terms
• Appendix D -
  Methods Worksheet (Example 1) (html)
• Methods Worksheet (Example 1)( 34 KB)
• Appendix E -
  Methods Worksheet (Example 2) ((html)
  Methods Worksheet (Example 2) ( 25 KB)
• Appendix F -
  Guidelines for Reviewing Evaluation Reports(html)
  Guidelines for Reviewing Evaluation Reports( 25 KB)
• Appendix G - CIPARS Logic Model
• Appendix H - Surveillance System Evaluation Reference Materials

Executive Summary

The Framework and Tools for Evaluating Health Surveillance Systems is designed to help managers of health surveillance systems identify and document issues relating to the rationale, implementation and effectiveness of their health surveillance systems. The Framework and tools provide standard approaches that managers can apply in their efforts to identify current practices and to enhance the ability of surveillance to provide relevant information for the review of public health objectives.

The framework outlines six steps in evaluating health surveillance.

Step 1 Establishing the context of the surveillance system
Step 2 Developing evaluation questions
Step 3 Designing the process for data collection and management
Step 4 Collating and presenting the findings
Step 5 Reviewing an evaluation report
Step 6 Following up on the use of findings

These steps outline a process for systematically reviewing the purpose, design, management and operational characteristics of a system within the context of its program. As noted by Klaucke , the strength of an evaluation depends on the evaluator's ability to assess a system's characteristics with respect to its objectives.

The ease of implementation and degree of success of an evaluation is closely linked to the maturity of results-based management practices. Results-Based Management and Accountability Frameworks (RMAFs) help managers ensure that:

• Clear roles and responsibilities among partners are described;
  
• Resources are tied to expected outcomes;
  
• Appropriate performance measures are determined for ongoing adjustments;
  
• Evaluation is identified and planned within the overall program life-cycle; and
  
• Adequate reporting of outcomes is ensured.

Background and Purpose

Surveillance is to public health as accounting is to a commercial enterprise. Both track the "life-blood" of the flow of information in support of crucial decisions that impact on the lives of many citizens.

The Framework and Tools for Evaluating Health Surveillance Systems provides managers of health surveillance systems with a standard approach for assessing:

• The quality of the information that their systems produce;
  
• The effectiveness of their systems in supporting the objectives of the programs that they serve;
  
• The effectiveness of their systems in supporting informed decision-making; and
  
• The efficiency of their systems.

This document has been developed in response to the emerging application of results-based management practices within Health Canada, an Auditor General's report focusing on the practices of measuring and reporting of performance, and the desire for systemic change in the wake of the recent SARS crisis in Canada. Health Canada mandated its Population and Public Health Branch (PPHB) to initiate the development of an evaluation framework for enhancing the performance of surveillance systems.

In January 2003, the Centre for Surveillance Coordination undertook this project with the guidance of the PPHB Health Surveillance Coordinating Committee, whose membership represents the branch's Centres, Directorates and Laboratories.