[Previous] [Contents] [Next]

A Focus on Quality

4 Clinical audit

Introduction

Clinical audit is perhaps the best known example of the wider group of clinical effectiveness activities. It may be defined as:

'…the systematic and critical analysis of the quality of clinical care. This includes the procedures used for diagnosis and treatment, the associated use of resources and the effect of care on the outcome and quality of life for the patient(1).

The primary function of clinical audit is to improve patient care by informing healthcare professionals' understanding of their clinical practice. This is usually achieved by setting standards, measuring current performance against those standards, identifying shortfalls and putting in place any necessary action. As standards change, re-audit will become necessary.

Clinical audit is a multi-professional activity that has its origins in medical audit, which was uni-professional and initially involved only doctors. The Clinical Resource and Audit Group was set up in 1989 following publication of the Scottish Working Paper 2 Implementation of Medical Audit, which outlined the fundamental principles of medical audit and set out how it would be introduced into the NHS in Scotland. In 1993, audit entered its second stage evolving from uni-professional audit into multi-professional clinical audit.

CRAG has provided a focus for clinical audit work both at national and at local level, issuing guidance to Trusts and Health Boards to support the local application of clinical audit and, subsequently, clinical effectiveness. CRAG does not audit services itself. It sponsors a range or individuals and organisations (such as Royal Colleges, lead Trusts and voluntary agencies) to carry out the work on its behalf.

It is important to distinguish between 'clinical audit' and 'clinical research'. In essence, clinical audit '…aims to establish the extent to which actual clinical practice compares with best clinical practice. Clinical research aims to establish what is the best clinical practice.' (2). Although the two elements are different, it is clearly the case that the one might lead to the other. The outcome of a piece of clinical audit may inform the need for clinical research, and the findings of a piece of clinical research may establish the need to audit clinical practice.

Clinical audit should be seen as a continuous process by which clinical practice and patient care can be improved.

History in Scotland

A brief outline of the history of clinical audit in Scotland is in 'Origins of CRAG’ in Section 1. A summary of the key stages in the development of audit includes:

1985

Setting up of the Transfer of Resources Group to examine clinical work by identifying good practice from a clinical and economic standpoint, and disseminating these standards.

   

1987

First meeting of Clinical Resource Use Group (CRUG) that evolved from the Transfer of Resources Group with an extended membership including chief area medical officers (CAMOs) and clinicians. This was later broadened to include a nurse and a general manager.

   

1989

Setting up of the Clinical Resource and Audit Group (CRAG). This followed from the White Paper, Working for Patients (1989), and the recognition that medical audit could provide an ideal mechanism to secure change. Membership of CRAG was based on that of CRUG with the addition of a postgraduate dean, a regional adviser in general practice and a second nurse. The original remit of CRAG introduced elements of medical audit while maintaining a focus on the effective use of resources.

   
 

Those parts of the remit that related specifically to audit were to:

 

  • determine national audit strategy, identify and disseminate good audit practice
  • co-ordinate audit practice at national level
  • monitor audit training
  • receive and scrutinise annual reports of the Area Audit Committees
  • consider the need for national and regional studies, external peer reviews, auditing Area Audit Committees
  • contribute to the formation of a national register of audit projects.
   
 

There was a major drive to implement medical, nursing, dental and pharmaceutical audit and later audit in other professions. CRAG set up four uni-professional audit subcommittees that remained effective until 1992:

   
 

  • Medical Audit Subcommittee (CRAG-MAS) - December 1989
  • Nursing Audit Subcommittee (CRAG-NAS) - October 1990
  • Dental Audit Subcommittee (CRAG-DAS) - May 1991
  • Pharmaceutical Audit Subcommittee (CRAG-PhAS) - June 1991.
   

1990

National Projects Committee (NPC) was set up to stimulate and manage a programme of national audit projects.

 

 

1993

The National Projects Committee was reconstituted to make the membership more widely representative of the range of interests in the NHSiS. By 1995, over 120 national audit projects had been funded by NPC (which became the Clinical Effectiveness Programmes Sub-group in 1998).

   
 

In May, CRAG's publication of The Thomson Report - The Interface Between Clinical Audit and Management — led to the development of local audit. The report set out principles for the use of clinical audit, and the roles and responsibilities of health professionals and managers involved in audit. It supported the move to multi-professional clinical audit and recommended the establishment of Area Clinical Audit Committees (ACACs) to oversee the development of audit within Health Boards.

   
 

Formation of the Clinical Audit Subcommittee (CRAG-CAS), an amalgamation of CRAG's four uni-professional audit subcommittees.

   

1994

The Scottish Clinical Audit Resource Centre (SCARC) was set up by CRAG (and based at the University of Glasgow) as a clearing house for information on audit in Scotland. SCARC's remit included education, information and library services, research and development, and support. Details of all local audit projects in Scotland were maintained on a database maintained by SCARC, but funding was discontinued in April 1999.

   

1995

Development of the Strategic Framework for Clinical Audit in Scotland by CRAG-CAS. The framework document set out to consolidate existing guidance and to complement Boards' local audit strategies. The Implementation Sub-Group replaced CRAG-CAS in 1998.

   

1998

Internal review of CRAG leading to the establishment of the current structure encompassing:

   
 

Clinical Effectiveness Strategy Group (CESG) - set up to provide guidance on the strategic direction of the clinical effectiveness agenda and to improve the co-ordination of the different bodies involved in clinical effectiveness.

   
 

Clinical Effectiveness Programmes Subgroup (CEPS) - CRAG's main funding committee with responsibility for developing and supporting new clinical effectiveness programmes and projects in addition to the existing portfolio of national audit projects.

   
 

CRAG Implementation Subgroup (CIS) — set up to support the NHSiS in taking forward the clinical effectiveness agenda and promote the output of work sponsored by CRAG, such as clinical guidelines or the findings of national audits.

 

Funding of clinical audit work

Specific funding for audit was first made available in 1990/91. In the 5 years to March 1995, 26m (including 2m of capital) had been allocated to clinical audit. Two thirds of these funds were allocated to Health Boards for local audit. From 1994/95, funding for audit at local level was included in the general allocations to Health Boards.

Since 1995/96, approximately 2.7m has been allocated each year to support CRAG's work programme. In 1998/99 and 1999/2000, about 60% of the budget was allocated to fund clinical effectiveness projects. Another 30% went to fund clinical guidelines and the CRAG Implementation Subgroup. The remainder is used to support CRAG's other work (eg publications, conferences and committee expenses). The administrative costs of the Secretariat are centrally funded.

Activity levels and range of projects

The range and scope of projects funded through the NPC and more recently through CEPS is extensive, covering a wide range of specialties and disciplines. However, this represents only a small part of the total work that is currently being undertaken in Scotland, with the major part being carried out at local level.

In the early days of audit, the priority was to stimulate interest and involvement in audit. As a result projects funded by NPC tended to be ad-hoc, reflecting the interests and needs of the clinicians involved. Since CEPS was set up, the focus has changed towards supporting a number of commissioned programmes concentrating on the national priorities.

These programmes support a range of activity in cancer, CHD and stroke, mental health and children’s services. Although, the proposal is to target the majority of funding (currently around 80%) towards these priority areas, CEPS will continue to consider spontaneous applications from all professional groups in Scotland engaged in delivering healthcare services.

Individual projects range greatly in size and scope. Some are tightly focussed on a particular aspect of treatment, for example the Scottish national audit of ECT. Others cover a range of activity under one heading, for example, the Scottish Programme for Clinical Effectiveness in Reproductive Health (SPCERH) and the Scottish Programme for Improving Clinical Effectiveness in Primary Care (SPICE PC).

One of the four complementary roles of SPCERH is to carry out work in the areas of audit and guidelines. SPCERH currently administers the Confidential Enquiry into Maternal Deaths (CEMD) and the Scottish Stillbirth and Infant Death Survey (SSBID), both on behalf of the Chief Medical Officer. It also has a commitment to initiate one new topic-based audit each year. Topics covered to date are an Audit of Pregnancies in Diabetic Women, an Audit of Maternity Services in Scotland and the Scottish Audit of the Prevention and Management of Emergencies in Labour.

SPICE PC, led by the Royal College of General Practitioners Scottish Council, aims to assist clinicians in providing effective care and encourage quality improvement in primary care.

In year one, criteria related to quality were developed in seven topics:

These identify a baseline measurement from which effective evaluation can be achieved. For further information about SPICE PC, see Section 10.

Current and recently completed projects

Current and recently completed projects can be grouped together in nine broad areas (the number of projects in each area is shown in brackets). Details of current and past projects are available on the CRAG website.

In addition, a range of new projects in mental health and children’s services is about to begin.

Examples of projects

Cancer - Scottish Audit of Gastric Oesophageal Cancer

Aims to:

  • identify variations in clinical practice in the investigation and treatment of gastric and oesophageal cancer in Scotland
  • identify good and inappropriate practice based on clinical outcomes and identify possible reasons for these
  • identify areas which require further investigation.

Expected outcomes are the:

  • provision of a population based picture of the management of upper gastro-intestinal cancer in Scotland on which future developments in practice can be based
  • start to an ongoing audit of oesophageal and gastric cancer in Scotland to ensure high quality management of these tumour types
  • provision of a mechanism via which purchasers can ensure that quality of treatment of upper gastro-intestinal cancer is monitored.

 

Children’s Services - Early Detection of Surgical Outcome in Cleft Lip and Palate Subjects in Scotland

Aims to:

  • assess the outcome of surgery in children with cleft lip and/or cleft palate
  • estimate the potential need for osteotomy surgery in late adolescence by studying standardised models of the child’s face taken at age 5 against a recently developed and validated index for the detection of surgical outcome in cleft lip and palate
  • improve compliance in the cleft treatment centres in Scotland with the recording of models of the child’s face at age 5 in line with national and international recommendations.
  • Models at age 5 will be those currently available to The Scottish Association for Cleft Lip and Palate (SCALP).

 

Coronary Heart Disease and Stroke - Scottish Audit of Carotid Endarterectomy

Aims to:

  • determine the appropriateness of the management of patients undergoing carotid endarterectomy, in terms of patient selection for surgery, whether patients receive preoperative neurological assessment and appropriate investigations, and the time delays from referral to surgery
  • compare hospital outcomes in terms of cerebro and cardiovascular events, local complications and deaths
  • determine and compare risk adjusted in hospital outcomes
  • determine crude and risk adjusted longer term outcomes in terms of re-operation and death through linkage with SMR1 and GRO data

 

Diabetes - Clinical Evaluation of Diabetes Care: Use of Innovative IT

Aims to:

  • develop a dynamic and user friendly district diabetes information system

In Tayside and Forth Valley, the grant holders will integrate the hospital-based diabetes systems with dedicated general practice information transfer systems using optical character reader (OCR) technology.

In Lanarkshire, the hospital-based Lanarkshire system will be integrated with general practices using a mixture of paper proformas and electronic data capture to create an efficient register and recall system.

The project will audit the components of locally derived diabetes data set (SIGN 25) before and after the implementation of general practice information transfer system, with comparison between the three regions.

 

Mental Health - National Audit of Electroconvulsive Therapy

Aims were to:

  • describe the population receiving ECT in Scotland in terms of age, social class and ethnic origin
  • determine which diagnostic groups receive ECT
  • describe the practice of ECT in terms of frequency, number of treatments and equipment used
  • audit the effectiveness of ECT in the clinical settings utilising standardised outcome measures
  • audit ECT in respect of legal and clinical guidelines

 

Primary Care — Scottish Leg Ulcer Project

Aim is to:

  • improve leg ulcer care in Scotland
  • It is a randomised control trial designed to compare the benefits of SIGN guidelines with SIGN guidelines reinforced by a formal structured training programme.

 

Renal - Improving the Management of End-Stage Renal Disease

Aims are to:

  • evaluate patient survival and the ability to achieve recommended standards in a one year cohort of new RRT patients in Scotland
  • study the influence of co-morbidity on the achievement of these targets
  • Standards that have an influence on survival and quality of life will be identified and the implementation of changes in practice to achieve them will be discussed with all nephrologists in Scotland. The standards will then be re-measured, in the light of changes made, completing the quality cycle.

 

Reproductive Health - Scottish Programme for Clinical Effectiveness in Reproductive Health

Aims to:

  • promote the delivery of clinically effective and evidence-based care in routine clinical practice to ensure ongoing improvements in reproductive health
  • For further details see above.

 

Miscellaneous - Development of Quality Assurance and Audit Systems of ICU in Scotland

Aims to:

  • further develop QA and audit of intensive care units in Scotland
  • make the programme suitable for incorporation into overall NHSiS quality assurance arrangements, with data collection based on a minimum core dataset from the CSA's Information and Statistics Division
  • The objectives and action plan to undertake this work are set out in the SICS Audit Group paper dated February 1999 and will form the basis for monitoring progress.

A number of new projects in the fields of coronary heart disease and stroke, osteoporosis, mental health and children’s services will come on stream during the next few months.

Local and national examples

One of CRAG's roles is to review the development of local audit and to do this, each local area is asked to produce an annual report. As audit developed to become a strand of clinical effectiveness, Health Boards were asked to extend their audit activity reports to become clinical effectiveness reports. CRAG also visits a range of Health Board areas to assess progress against clinical effectiveness goals which are set annually.

Clinical effectiveness reports from different Health Board areas in Scotland vary widely, reflecting the different structures and systems in place and local challenges. Reports generally include an assessment of performance against CRAG goals and a summary of (largely) audit work undertaken in the area during the previous 12 months.

In October 1999, CIS issued new goals for the NHSiS (Figure 1), taking account of the impact of clinical governance and changes in the support available for clinical quality improvement.

Goals for 1999/00 onwards take a more strategic view of clinical effectiveness, setting the required direction of travel but leaving operational details to individual Health Boards and Trusts. These new goals were distributed with accompanying guidance notes to the NHS under MEL (1999) 76. The goals provide a template to guide and monitor the development of clinical effectiveness in Scotland.

CIS also considered whether the previous approach to clinical effectiveness visits was appropriate given the reconfiguration of Trusts, the setting up of the Clinical Standards Board for Scotland, and the new focus on clinical governance.

CRAG recognised the benefits of wide consultation with the service over its needs for support in clinical effectiveness, and for following up issues identified in Clinical Effectiveness Reports. It also provided an opportunity to reinforce the new strategic direction of CRAG in general and the clinical effectiveness goals specifically.

Although visits require significant input from both the visiting team and the people being visited, CIS decided that the visits were a valuable opportunity to listen and learn about work at local level, and visited all 15 Health Boards during 1999/00.

A report based on this series of visits is available on the CRAG website. Although the visits highlighted a number of issues that need to be addressed, there was evidence of good and interesting work on clinical effectiveness from every Health Board area.

Figure 1

Goals for clinical effectiveness

These goals are relevant to the whole of the NHS in Scotland. They should be interpreted as applying to primary care, community services, acute hospitals, Health Boards and centrally funded services. To avoid repetition, the goals refer only to 'Trusts'.

  • Trusts should have an explicit strategy for clinical effectiveness, which should be part of a broader quality and clinical governance strategy. Organisational arrangements and mechanisms for the systematic monitoring and improvement of the quality of clinical care should be in place.
  • Trusts should have an appropriate infrastructure to support clinical audit and clinical effectiveness, and be able to provide evidence of:

a) systems to monitor clinical effectiveness activity (including clinical audit)

b) mechanisms to assess and implement relevant clinical guidelines

c) systems to disseminate relevant information

d) an IM&T strategy that supports clinical effectiveness

  • Trusts should foster a culture in which clinical effectiveness is integral to all clinical care. Developing clinical effectiveness skills should be central to continuing professional education and development, and part of a multidisciplinary systematic approach to continuous quality improvement.
  • Clinical effectiveness activities should support priority setting and reflect:

a) national priority areas identified in Priorities and Planning Guidance

b) local priorities identified in the Health Improvement Plan

  • Trusts should make sure that all operational sub-units have identified programmes of clinical effectiveness activity.
  • Trusts should promote clinical effectiveness activities that cross boundaries and support collaboration within and between Primary Care Trusts, Acute Trusts, emerging managed clinical networks and other agencies.
  • Trusts should be able to demonstrate an increase in public/patient participation in:

a) service planning and standard setting

b) monitoring the quality of care

  • Trusts should be able to demonstrate that cost effectiveness issues are being addressed alongside clinical effectiveness.
  • Clinical effectiveness should be a prominent feature of the HIP/TIP process - informing commissioning and underpinning service development.
  • Trusts should be able to demonstrate that clinical effectiveness activities are:

a) informing clinical governance

b) leading to changes in practice and improvements in standards of care

c) providing best value

September 1999

 

National audits

One of CRAG's roles is to nurture and develop national systems to audit care. A number of these audits provide a detailed 'snapshot' of a service, allowing problems to be identified and improvements made. In some cases, a decision is taken to maintain the audit over the long term. Four of the audits which were judged to be of national importance were transferred from CRAG to ISD:

Scottish Hip Fracture Audit

  • set up in 1993/94 — four centres CRAG funded
  • locally funded expansion in 1994-99
  • all now locally funded
  • covers 18 out of 25 orthopaedic units
  • audit nurses employed to collect standard data sets and conduct follow-up
  • documents hip fracture care in terms of case mix, surgical procedures and complications
  • outcomes including mobility, dependency, residential status and mortality
  • promotes and evaluates service developments
  • links with standardisation of audit of hip fracture in Europe

Feedback is provided:

  • six-monthly, in the form of reports to centres
  • via ad-hoc reports on specific issues
  • in reports to CRAG (four to date)

 

Scottish Audit of Surgical Mortality

  • set up in 1994
  • administered from offices in Aberdeen, Dundee, Edinburgh and Glasgow
  • covers all surgical specialties, except thoracic, cardiac and obstetric (covered by UK-wide national mortality audits) and almost all consultant surgeons and anaesthetists in Scotland
  • identifies all deaths which occur under the care of a surgeon, whether or not there has been an operation
  • approximately 4,500 deaths are identified annually

Feedback is provided:

  • to individual consultants, on cases they have dealt with where adverse factors in management have been identified
  • to all consultants and trainees (anonymised, collated case note assessments received at intervals)
  • at hospital or specialty level on request, comparing the selected area with the total data set. This is an area of activity currently being expanded and is seen as a service to Trusts in support of their Clinical Governance responsibilities
  • in an annual report highlighting important lessons

 

Scottish Trauma Audit Group

  • set up 1991
  • seven centrally funded staff and 25 local co-ordinators funded by health boards
  • covers all injured patients admitted to hospital for three days or more or who die
  • approximately 7,500 new cases added to database each year
  • excludes patients over 65 with an isolated fracture of the neck of femur and/or pubic ramus
  • covers Scottish ambulance service, blood transfusion service, general/orthopaedic/vascular/ cardiothoracic/neuro surgery, intensive care, anaesthetics, radiology and forensic medicine

Feedback is provided:

  • daily to medical and nursing staff
  • monthly feedback of national standards to A&E consultants
  • six-monthly routine analysis to local medical directors
  • annually to each Director of Public Health
  • three times a year to meetings of regional multi-specialty groups
  • national conference every 18 months
  • via ad-hoc reports to CMO

 

Scottish Renal Registry

  • collects, collates, analyses and reports on data relevant to improving renal services for patients on renal replacement therapy
  • covers all centres and all patients receiving renal replacement therapy for chronic renal failure
  • data extracted from the primary clinical record
  • patients followed up until death
  • based at Royal Infirmary, Glasgow

Feedback is provided:

  • to each renal unit showing their performance in relation to the whole country, other renal units and targets, where available
  • anonymised results discussed at annual Scottish Renal Association meeting

 

Annual symposia

Since 1990, CRAG has hosted an annual symposium on clinical audit (and more recently clinical effectiveness) to provide feedback to the Health Service about work being carried out in Scotland.

In the early years, the symposia featured a broad and sometimes eclectic range of examples of work. Presentations included Barium enema audit (1993), Neuroleptic audit in learning disability (1994), Audit of prophylaxis against venous thromboembolism (1996), Play preparation obviating the need for general anaesthesia in children having MRI scans (1997), Time Delay in Fast Track MI — A Nurse Led Audit (1998).

As audit has became more firmly established and the annual meetings increasingly popular, the events have been extended to include themed parallel sessions to allow delegates to hear a series of presentations around specific topics. In 1999 these included cancer, coronary heart disease, mental health and primary care.

The CRAG meetings have provided an opportunity for people working in audit to present details of their work to a national audience. These symposia have become an important feature of the annual calendar of the NHS, allowing an exchange of information and experiences among a diverse group of people. Approximately 400 people from a wide range of geographical and specialty areas attended the 10th CRAG symposium in December 1999

Exit strategies

Initially, the focus of CRAG's work was to encourage the uptake of audit and evidence-based medicine. But as mentioned in Section 1, although audit was considered essential it was often '…difficult to demonstrate direct benefit for patients or action following audit and other related activities.'(3)

To address this, greater emphasis is now being placed on developing 'exit strategies' for all CRAG-funded projects. Essentially, an exit strategy is the means by which the project findings are systematically disseminated, put into practice, incorporated into service provision or followed up (possibly through re-audit).

Clearly, if a piece of work has shown that, for example, following a set of guidelines on management of patients with a specific condition improves patient outcomes, then the following of these guidelines should become standard practice. This will not happen unless a proactive approach is taken to passing on and incorporating the knowledge that has been gained.

All projects funded by CEPS must address their proposed exit strategies at the outset to be considered for funding. Although it is recognised that the exact nature of the exit strategy may evolve over the course of the project, it is important to consider possible alternatives early on and to plan ahead. The publication of a report is unlikely to impact on clinical practice unless there is some form of positive interaction with those at whom the report is aimed.

The future

The need to measure and evaluate clinical activity against standards, and to revise those standards and re-evaluate the care provided in the face of new evidence, treatments and techniques will ensure that clinical audit retains a central role in the wider arena of clinical effectiveness.

Sharing information on audit activities, whether carried out locally or nationally, is important to allow people in or across professional groups to learn from each other. CRAG will continue to hold national meetings, including an annual symposium, to provide the opportunity for people involved in clinical audit and wider clinical effectiveness issues to meet and discuss their work.

 

Key summary

  • the primary aim of clinical audit is to improve patient care
  • clinical audit aims to assess how actual clinical practice compares with best clinical practice, measuring and monitoring performance against standards
  • clinical audit is an intrinsic part of the wider clinical effectiveness agenda
  • clinical audit should increasingly become an accepted part of everyday work for all health care professionals

 

Ms B Cant
Senior Programme Manager
CRAG Secretariat
Scottish Executive Health Department

 

 

References

1. The Interface Between Clinical Audit and Management — A report of a Working Group set up by the Clinical Resource and Audit Group. The Scottish Office, 1993: p9.

2. The Interface Between Clinical Audit and Management — A report of a Working Group set up by the Clinical Resource and Audit Group. The Scottish Office, 1993: p44.

3. A Focus on Quality Report — Section 1 — Clinical Resource and Audit Group

[Previous] [Contents] [Next]