Quality of Life Measurement Among Persons with Chronic Mental Illness: A Critique of Measures and Methods

Next | Previous | Table of Contents

4. Central Issues Around Quality of Life Evaluation

Program and policy decision-makers are faced with quite a task when trying to draw a comparison among various outcome indicators gathered with non-standard and disease-specific instruments used with differing psychiatric populations treated in programs employing different service delivery models and reporting a wide variety of comorbidity and illness acuity factors. An overarching question is, how to be fair and even-handed in the absence of a universally applicable gold standard for evaluating QoL? Complicating matters, when such standards are proposed, programs may identify themselves as an exception to the rule and ask for special consideration. Indeed, the mental health care delivery system is founded on a long and complex history of largely uncoordinated initiatives. While such differences contribute to the rich diversity of services and approaches, ideologies of uniqueness do not easily accommodate aspirations towards a system-wide coordinated planning of health services. One must remember, however, that coordinated approaches are a necessary component of efficient and continuous service delivery and in spite of programmatic differences, over the next few years trans-program comparisons and decisions will be made. These decisions will rely heavily on a utilitarian model that seeks the best good for the most people at the lowest cost. This approach will not necessarily be welcomed by those who face increasing service demands.

Academics and clinicians throughout the mental health system continue to advocate for an equitable treatment of mental health services compared to other sectors of health (Lamb et al., 1993). It is noted repeatedly in the literature, however, that the more disenfranchised members of society typically receive only secondary or discriminatory consideration (Dossa, 1989; Lamb et al., 1993). Compounding the problem, those individuals who are only occasional users of mental health services, and do not fall into the category of the chronically mentally ill, are reluctant to act as advocates for the services they have received, due to the stigma associated with mental health problems. This apparent lack of public support further contributes to an underestimation of the importance of mental health services and the erosion of its funding base. Issues of representation and lobbying cannot help but influence the way in which findings from evaluative studies are contextualized within the broader picture of health care policy formulation and service planning. In recent years, relatives and informal caregivers of consumers of mental health services have made concerted efforts to reverse this trend (Katsching & Konieczna, 1989).

4.1 Defining Productivity Units: Service versus Therapeutic Effect

Issues surrounding the quantification of service units inevitably arise in attempts to balance costs against benefits and to improve the efficiency and effectiveness of service delivery systems. There are two general approaches to defining productivity units: defining units of effect and defining units of service. Administrative data on service units are widely gathered throughout the health care system. Service units are fairly easy to define; typically they are the number of time-weighted procedures completed with particular patients. These measures, however, tell us little about the therapeutic effectiveness of such activities unless the activities themselves are either based on tried and true standards for treatment or empirically linked with health outcome indicators. Units of effect, on the other hand, can be thought of as the change in health status or health-related quality of life which is thought to occur as a result of a specific intervention. Unfortunately, statements made about these units of effect often lack the definitiveness needed to justify funding decisions. Measurement of aspects of a patient's health status prior to and following treatment is a quick and sometimes informative approach, although in its simplicity one can never really be sure about the reason for a set of observed changes.

The difficulty in demonstrating treatment effects among the chronically ill is compounded by the fact that very effective programs often show no change over time, since the program's effectiveness is manifest by the prevention of relapse (or prevention of negative change). Alternately, within illness populations with a cycling prognosis, assessment at intake will almost always be worse than discharge simply due to the immediacy of presenting concerns and temporal changes in the illness. Other cyclic and periodic effects also require consideration. For example, teasing out the effects of medications on neuropsychological performance from the effects of psychosocial and skills based treatment is virtually impossible using simply constructed evaluation designs (Awad & Hogan, 1994; Buchanan, Holstein & Breier, 1994) and determination of cause and effect relationships between treatments and outcomes is best handled using tried and true scientific methods of subject randomization, a carefully matched control group, or longitudinal baseline studies.

The question then remains as to how QoL measures might usefully inform the processes of treatment, monitoring/planning and scientific investigation. Binner (1993) suggests that therapeutic practice and outcome assessment need to be aligned with one another so that the goals of treatment become the specific dimensions on which the treatment is evaluated. Within the context of this discussion, this implies that service evaluators use therapeutic strategies and/or treatment goals which coincide with dimensions on the selected QoL measures. Alternatively one might advocate the use of QoL instruments, such as the QoL Index for Mental Health (see XVI in Chapter 5), which use clinician and patient ratings of changes with respect to goals which are defined over the course of treatment. Potentially, such a solution has broad implications for the way in which services are conceptualized and delivered.

4.2 Population Health Status

Reduction in the disability, distress and/or discomfort of patients is particularly important to health outcomes which involve chronic disease. Past work on the conceptual basis of disability and distress lays the theoretical basis for computation of QoL outcome units such as Quality Adjusted Life Years (QALY) (Rabin et al., 1993). The QALY is based on the principle that a year of poor health is of lower ‘utility' or value than a year of life with good health quality. QALY units, which are gained as the beneficial effect of treatment on disease conditions, are balanced against the costs of providing that treatment resulting in a cost-benefit analysis (Gudex & Kind, 1988; Lubeck, 1991). While intuitively appealing, there are several concerns about the validity of such an approach to program funding.

Despite their apparent soundness, attempts to link QALY units to treatment effects raise a series of concerns about the rigour, sensitivity and validity of the methodologies. Rosser (Rosser & Kind, 1978), a pioneer of QALY methodologies in the UK, states that this method was not intended to provide indicators of sufficient sensitivity for detection of treatment effects during clinical trials. This information raises doubt as to the suitability of global QoL measure as an effectiveness indicator of specific treatment approaches. It may be possible, however, to increase the sensitivity of QALY methods to treatment effects by adding disease-specific components and strengthening evaluation designs with experimental and/or quasi-experimental methods. Yet when treatments occur over a long period of time and therapeutic gains are small or primarily preventive (as is the case for chronic mental disability), the disentanglement of treatment and temporal effects remains a very difficult undertaking. One is left with the feeling that global health measures might best serve a more limited role, as flags which point clinical service providers and planners towards areas of patient need.

Further complicating QALY comparisons across programs are difficulties with the model itself. It is implicitly assumed that QALY units are uniform across diagnostic groups and equally graded between QoL levels within diagnostic groups. There is evidence, however, that "normal" persons' perceptions of the disability and distress associated with mental disorders is greater than with other physical disease states (Rabin et al., 1993; Wilkinson, Williams, Krekorian, McLees & Falloon, 1992). When questioned as to the reasons for the lower desirability (i.e., utility) ratings associated with these disorders, non-mentally ill persons reported chronicity, untreatability, impaired ability to think clearly, social stigma and the loss of personal control as the major factors contributing to particularly poor life quality of persons with mental illness. These authors concluded that low global QoL scores for persons with one type of illness may have a different utility rating (based on public consensus) than the same QoL score among patients with a different illness.

Disease chronicity is another factor which can influence individuals' evaluation of the impact of illness on their lives. Acute changes or alterations in health-related life experience are typically experienced (and reported) more intensely than are the effects of disease states with which the patient has been coping for some time. The reported impact of illness may be lessened due to the moderating effects of perceived control and ability to cope, which in turn comes as a result of experience (Tope, Ahles & Silberfarb, 1993). Unfortunately, the literature has not adequately addressed the impact which chronicity and perceptions of control have on QoL ratings provided by patients with mental illness.

Another problem with QALY's based on self-report measures is that the very act of reporting is likely to be affected by aspects of the mental illness, including impaired judgement, optimistic self-other judgements based on an impoverished comparison group and the diminishment of expectations which comes about through long-term dependence. Certain aspects of life quality which may receive low endorsement by persons with schizophrenia (e.g., medication compliance), if not checked, may jeopardize both their physical and mental well-being. These observations have serious implications when one considers the current trend towards increased consumer involvement and responsiveness to consumer demands. The central role which advocacy plays in the delivery of mental health services should not be relinquished too quickly. At the same time, ways should be found to preserve individuals' basic needs for self-determination in light of the "paternalism" (however justifiable or apparently justifiable under certain circumstances) present in some treatment sectors (Munetz, Birnbaum & Wyzik, 1993).

Thus while QALY methods have intrinsic value, it should be remembered that the delay between treatment and effect, the costs and expertise needed for adequate rigour, weak sensitivity of global measures, the difficulty equating the meaning of specific treatment effects across patient groups, and the impact of mental impairment on subjects' self-report ratings persist as some of the most formidable barriers to linking funding with comprehensive program evaluation initiatives.

4.3 Economics, Policy and Planning

Outcome or evidence based reimbursement models are quite simply outlines of methods to control the cost of health care to society while maximizing the health status of individuals within society. The Health Resource Allocation Strategy laid out by Patrick and Erickson (1993) elegantly outlines an approach which has found endorsement in both Europe and America. Although the rational basis is quite straightforward (namely, if we know the amount of resources being spent for any treatment outcome, then we are able to prioritize cost effective services) several substantial issues stand in the way of linking outcome measurement to program funding. Juxtaposed are two of the most formidable concerns, both of which have been addressed throughout this document: the high material and personnel costs associated with definitive research/evaluation projects and also the inherent problems associated with the use of less rigorous indicators of outcome (i.e., the "straw in the wind" approach). The closing of several large scale programs due to fiscal constraints presents us with some sobering information when considering the role of evaluation in funding allocation strategies for mental health care systems (McPheeters, 1984).

The expense of any evaluation should not be under emphasized, particularly the implementation of system-wide initiatives (Health Canada, 1994b). For the most part, economies of scale do not apply to the labour-intensive activities associated with study design, implementation and analysis. It may be more efficient if a few research-focused centres were relied upon to conduct rigorous examination of treatment effectiveness and to formulate treatment guidelines (cf., Sullivan, Wells & Leake, 1992). Subsequently, the practices of other programs could be asked to conform to these guidelines (subject to continuing staff training and program audits). This approach is likely to be a more cost effective means of ensuring program quality than requiring programs to conduct numerous poorly designed studies which lack the rigour to make definitive statements about treatment impacts. The trade-off between clinical efficiency and scholarly evaluation activities must be given a realistic consideration.

Automated data collection and entry systems are solutions which are often raised by those faced with the high cost of evaluation. Several large scale public health initiatives are underway which utilize an automated approach to QoL measurement. The Behavioural Risk Factor Surveillance System implemented by the Centres for Disease Control and Prevention in Atlanta, Georgia utilizes interactive voice response (i.e., IVR) over the telephone lines to monitor the self-perceived physical and mental health of patients (Hennessy, Moriarty, Zack, Scherr & Brackbill, 1994). It is important to note, however, that this project is a health monitoring system and is not intended for the purposes of reimbursement or program evaluation. Other technologies which automate data gathering and entry can cut cost substantially during the implementation and data collection phases of research and evaluation studies. Two examples are the Clinical Outcomes and Resource Measure (Marks, 1995), and the Survey Administrator System (UTI, 1996). Reduction in collection and entry costs, however, does not address the need for some expertise in research design, psychometrics and statistical analysis when conducting quality evaluation projects.

One last methodological challenge, which also frustrates attempts to link outcomes to funding in a global health delivery model, is the inability to estimate costs to multiple systems over large portions of patients' lives (e.g., episodes of care). The notion of episodes of care is gaining popularity as health care planners begin to consider the costs and needs of individuals over the lifespan. This change in perspective is influenced by several factors. There is a growing appreciation of the importance of developmental and systemic factors in health promotion and illness prevention. This awareness has contributed to our understanding that the effects of our health care planning and intervention need to extend beyond the time-limited political term of holders of the public purse. Reinforcing the move towards systemic planning, there has been a dramatic improvement in informatics tools over the last years (e.g., relational database technologies) which are capable of handling and storing large longitudinal datasets. A lifespan approach to service delivery is particularly important when addressing the needs of the chronically mentally ill, since the very reasons for acute relapse are often related to long-term factors such as instability in both formal and informal support networks and in medication compliance. The staggering costs associated with acute care admissions might be partially reduced through planned continuity of care over the lifespan.

This portion of the document has implicitly focused on the need for therapeutic resources for persons with mental disorders. From this perspective, planners are placed in a position of determining what they are able to give to these individuals: this could inadvertently relegate consumers of mental health services to the role of "passive patient". The consequences of playing this role have been described by Lord (1989) as feelings of powerlessness, alienation, loss of self-esteem and loss of dignity.

Perhaps a more appropriate question for evaluators to ask is "What are the needs of these individuals and their informal caregivers in relation to integration into their communities and how can services support the contribution which these individuals make to their communities?". When approached from this perspective, patients' abilities, skills and strengths are addressed in the planning and evaluation of mental health services. Clients who have found a place of responsibility within their community may possess a greater sense of self-respect, importance and life quality (Mercier & King, 1994). They are also likely to contribute in a supportive role to those in need around them (Arns & Linney, 1995). Further, such involvement could be health-promoting in the sense of sustaining mental health and related psychosocial resources and preventing acute psychiatric relapse. It is anticipated that support dollars which are channelled into patients' self-sufficiency could be recovered through both the reduction of external forms of intervention and improved QoL through meaningful vocation. This would seem to be a suitable area for further research into quality of life for persons with mental disorders, its determinants and sequelae.

Next | Previous | Table of Contents