Скачать 41.89 Kb.
Taking DREEM Forward
Background and summary of experience with REE/DREEM so far
and recommendations – 20th June 2006
Author(s) Piers Allott, Michael Clark and Mike Slade
The purpose of this paper is to outline development work and future thoughts about how to measure the recovery orientation of mental health services. To do this we begin by placing the work into a broader context of recent development of mental health policy and services, and the place of recovery in this.
The need for valid measures of recovery and the recovery orientation of services is seen as a vital step in the next phase of the national move to recovery based mental health services. The DREEM tool is seen to hold promise for assessing the recovery oriented nature of services to support research in this field and critical reflection for local service improvement. Pilot work using DREEM is described, along with the insights and lessons learnt from this. The paper concludes with thoughts on what needs to happen next to fully develop a valid measure based on the DREEM tool.
The concept of recovery from mental illness is one that has grown out of the experiences of people in recovery from mental illnesses and from a better understanding of the longitudinal nature of diagnosed mental disorders. The Vermont Longitudinal Studies (Harding et al.1987 and Harrison et al. 2001) have demonstrated that significant numbers of people diagnosed with psychosis do recover over time and supported the lived experience of people diagnosed with mental illness encouraging and enabling them to identify self-management approaches for their experiences. Self-management has now begun to be accepted by traditional psychiatry and mental health services as an evidence based practice (http://www.mentalhealth.samhsa.gov/cmhs/communitysupport/toolkits/illness/).
The United Kingdom recovery movement began in the mid-1990s. A key development was support from what was the West Midlands Regional Health Authority for a series of ‘Directional Papers’ that articulated the direction of future mental health services. This was followed by support for three pieces of work: a literature review (that eventually became Allott et al. 2002), and the delivery of a narrative research project as well as developing training capacity in Wellness Recovery Action Planning (WRAP – www.mentalhealthrecovery.com). The latter fired the imaginations of many people who used services (Cullotty, T. 2005 and Williamson, J. 2005), whilst the literature review informed professionals and academics. At the same time there were a few similar narrative research projects as well as some people beginning to publicly tell their stories (May, Taking a Stand, Radio 4, 6th Feb 2001) and speak about recovery.
The appointment of a ‘Fellow for Recovery’ by the National Institute for Mental Health in England (NIMHE) was an important recognition of the growing importance of the concept of recovery. The post was intended to stimulate interest in and knowledge about recovery from mental illness and act as a ‘pipeline’ programme within NIMHE and for the development of services. This led to a number of significant developments over the three years of the post, including:
Recovery and the International Context
Recovery is now adopted as national policy in most of the developed countries around the world. The State of Wisconsin, origin of the mental health components of the NHS Plan in England, adopted recovery in 1997 and the Federal Government adopted recovery in 2005 (New Freedom Commission on Mental Health (2005) Achieving the Promise: Transforming Mental Health Care in America, Rockville, MD: U.S. Department of Health and Human Services.). New Zealand adopted recovery in 1998 (Mental Health Commission (1998) Blueprint for Mental Health Services in New Zealand. Wellington: Mental Health Commission.) and has produced the greatest number of recovery focussed materials outside of the US. More recently Scotland have invested in recovery through the establishment of the Scottish Recovery Network (www.scottishrecovery.net) including reasonably significant investment in recovery research. The Republic of Ireland has recently published its ‘Vision for a Recovery Model in Irish Mental Health Services’ in December 2005 ( Mental Health Commission (2005) A Vision for a Recovery Model in Irish Mental Health Services, MHC: Dublin.
The need for measures of recovery
Whilst recovery has moved up policy and service user agendas, research into recovery from mental illness is still sparse. In particular, measures of the recovery orientation and effectiveness (in relation to recovery, rather than traditional outcome measures) of services have not been fully developed and tested. Resource and methodological challenges are the main issues to be addressed here. Measures need to exhibit the following features:
There is some guidance available in England about what mental health recovery entails and what constitutes a recovery-oriented mental health system (NIMHE Guiding Statement on Recovery 2005). However very few research tools have been designed to help services assess their recovery-orientation, either from the perspective of the personal recovery of the people they serve, or from the perspective of the services and supports provided. The Recovery Enhancing Environment Measure (REE) was designed to help fill this knowledge gap and the Developing Recovery Enhancing Environments Measure (DREEM) is its English edition, edited for England, with permission.
Development of REE and DREEM
REE is an instrument driven by the assessment of people themselves of their own state, and their preferences, needs and desires, and assessments concerning the assistance provided by the helping system that support and uphold recovery.
The items in REE were developed based upon:
The instrument is intended to provide an assessment of the service user’s own recovery state, his or her preferences, needs and desires, and a user driven assessment of the recovery assistance provided by services and support systems.
The REE measure was pre-tested and refined based on feedback from people who used services in the Kansas ‘Consumers as Providers’ training programme and other people who used services served by a Kansas Community Support day treatment programme. Development of the instrument benefited from the extensive input of two established researchers (Patricia E. Deegan, a consumer leader, and Allan Press, a statistician and measure designer). The REE then underwent two large field tests, one in Kansas (Ridgway et al. 2003) and one in Massachusetts (N=500+), and was psychometrically tested and revised before being finalized. Analyses completed indicated that REE is psychometrically sound with results indicating more than adequate reliability. Results were presented showing that various scales and subscales can differentiate between high and low performing services and provide evidence that the instrument has criterion-related validity. Additional analyses were also performed that show that the various scales and subscales show the expected interrelationships (Ridgway et al. 2004). It was found to have high face validity, good internal consistency (indicated by alpha coefficients of greater than 0.7 for all sub-scales), and adequate criterion validity (in discriminating between services).
The REE is designed for use in relation to US mental health services, so inevitably refers to non-UK structures and concepts. Its cross-cultural validity is therefore limited. DREEM is a version of REE edited to be suitable for use in the UK.
The DREEM questionnaire asks about individuals’ view of themselves and their recovery, the services they are receiving with respect to their recovery and specific issues of individual needs. It is designed to either be self administered or with assistance of a person not involved in providing that individual with psychiatric care. It has 166 items organised in seven components-
DREEM has been piloted by four mental health services in England. The results of these pilot studies are now reported.
Pilots of DREEM
DREEM has been piloted, with data collection and some reporting, in four English sites. The following is a summary of each pilot, followed by a discussion of collective points from the sites.
A number of people who had all been using mental health services for at least one year agreed to complete the DREEM, either individually or through an interview with a member of the Network. Despite some apprehensions about the tool (such as length and some options on the multiple choice questions) they all fully completed it, providing preliminary evidence of feasibility. Some people found it hard to keep their responses focused on a specific local mental health service/team and sometimes replied about the whole (local) system. People who were keen to provide their views welcomed the opportunity to elaborate on their views in the open-ended questions.
At the time of writing this summary, the data for the Sefton pilot had not been analysed. Those who undertook the pilot concluded that the DREEM tool provided a good ‘dashboard’ summary of local services and issues needed to be addressed to make them more supportive of recovery, indicating face validity. The team recommended that the tool be developed to provide more focused data, which can be summarised in indicators to help individuals and organisations plan local service development.
In the Derbyshire pilot, 12 questionnaires were completed by residents of a local residential facility for people with long term mental health difficulties. Respondents were supported in completing the questionnaires by an independent, external facilitator. Given the small sample size, the data from the Derbyshire pilot has not been aggregated, but reported as individual case studies of responses. Respondents responded “strongly agree” or “agree” to between 78 and 95 per cent of the questions on the DREEM tool. It is not clear whether or not this is a facet of the people included, the environment they were in, the manner of doing the data collection, or of the tool itself. The team undertaking the study did, though, question some psychometric attributes of the tool i.e. the fact that all questions are so worded that a positive view is always associated with the answers “strongly agree” or “agree”, and that the format means that these answers are always to the left of the scale. The team did find some early, tentative findings about a group of questions on the tool which may indicate their validity, reliability and usefulness.
The team concluded that in its present form the questionnaire is long and so would require huge numbers of subjects to validate it. They felt that the nature of the intended client population means that assistance is necessary to support completion – quite probably influencing responses. On the other hand, they felt some of the more tentative findings can possibly be exploited as indications of how it might be either reduced in volume or used to support a more investigative, qualitative analysis of the intended phenomena.
A team in Exeter used the DREEM tool as a focus for staff and service development in the local in-patient service. Service users assisted residents of the service (n=10) to complete three components (1, 3 and 7 in the list above) of the tool.
The results from service users were analysed and used as part of a staff development day, at which staff (n=26) also completed DREEM and had their results presented to them. The combined results were used to inform discussion on service development. The results of the data analysis demonstrated that:
An action plan for service development was developed in collaboration with staff and residents through a further series of meetings and is in the process of being implemented. The results have also been shared with the local Rehabilitation and Recovery Service.
The quantitative analysis of responses to this data has been useful to the local services, but is based on small numbers to fully test the validity and reliability of DREEM. However, specific factors were rated by residents as being particularly important to their recovery, including a sense of identity, self-managing one’s own illness, a sense of control and a feeling of hope. These could be focal issues for revising DREEM. The qualitative responses also provided local focus for development initiatives, and provide insights into what may be key aspects to include in a revised DREEM.
The team acknowledge that the results of the pilot need to be treated with caution as the DREEM has not been validated, and the impact of daily fluctuations in service user attitudes towards services is unknown as no level of test-retest reliability is given. Further, the tool is not designed to be completed by staff, which required them to interpret some questions. The team also offered insights into using DREEM, including concern about how long it is (hence they only used 3 of 7 sections), a need to ensure the tool as a whole, and its sub-sections, are clearly focused on the things they are trying to measure, being mindful that how the tool is administered might be important and that this is an intervention in itself, the need to review the language of the tool and any preconceptions behind questions, and provide more guidance on how to analyse the tool and interpret meanings. They conclude with the following list of current strengths and weaknesses of DREEM:
A client satisfaction survey was conducted using DREEM of the recovery enhancing working practices at a resource centre. The complete DREEM was used as a tool to establish the extent to which the services were seen by clients to be supporting their potential for recovery, and the aspects of the service, if any, which required changing to support them better.
Fifty-five clients from the ward (n=20) or community (n=35) were invited to take part in a survey. All invited persons had active mental health symptoms which affected their social functioning and lives generally. All potential respondents were offered the chance to either complete the questionnaire as part of an interview, or to complete and return it themselves.
Responses were received from the wards (n=7) and the community (n=7). (The fact that another survey was sent out at the same time as this work is likely to have adversely affected the response rate. The length of DREEM was felt to have potentially reduced response too.) Most of the respondents were male, between 46 and 55 years of age, White English, and mainly unemployed. Most were thinking about recovery but did not feel they were actively involved in the recovery process.
All 24 elements of the recovery scale were rated as important by the majority of respondents, but some variation in replies was identified. The majority of respondents felt the service was meeting their needs in terms of most of the recovery elements, but again variation in responses was identified. For five recovery elements, for example, only a minority of respondents felt their needs were being supported. Only 14 percent of respondents felt they were directing their own treatment, a factor rated ‘important’ for recovery by 93 percent of respondents.
On the basis of the survey, tentative insights were developed into the relationships between the staff and clients and possible means of improving the service.
The pilots of DREEM have provided helpful insights into its organisation and application. Very preliminary evidence has been established in relation to several important psychometric properties: face validity, construct validity, and feasibility. Issues highlighted for further development include the length, the focus, and the difficulties in using the same instrument for multiple uses.
It is recommended that a staged development process now be undertaken as set out in the appendix.
Allott, P., Loganathan, L. and Fulford, K.W.M. (2002). Discovering hope for recovery: a review of a selection of recovery literature, implications for practice and systems change in Lurie, S., McCubbin, M., & Dallaire, B. (Eds.). International innovations in community mental health [special issue]. Canadian Journal of Community Mental Health, 21(3).
Cullotty, T. (2005) A project evaluating trainee’s experiences of implementing a recovery focused tool into mental health services in South Tyneside – unpublished paper
Harding, C. M., Brooks, G. W., Ashikaga, T., Strauss, J. S., & Breier, A. (1987). The Vermont longitudinal study of persons with severe mental illness: I. Methodology, study sample, and overall status 32 years later. American Journal of Psychiatry, 144, 727-735.
Harrison, G., Hopper, K., Craig, T., Laska, E., Siegel, C., Wanderling, J., Dube, K. C., Ganev, K., Giel, R., An Der Heiden, W., Holmberg, S. K., Janca, A., Lee, P. W. H., León, C. A., Malhotra, S., Marsella, A. J., Nakane, Y., Sartorius, N., Shen, Y., Skoda, C., Thara, R., Tsirkin, S. J., Varma, V. K., Walsh, D., and Wiersma, D. Recovery from psychotic illness: a 15- and 25-year international follow-up study Br. J. Psychiatry, Jun 2001; 178: 506 – 517
Ridgway, P., Press, A., Ratzlaff, S., Davidson, L., and Rapp, C. (2003) Report on Field Testing the Recovery Enhancing Environment Measure (REE) University of Kansas School of Social Welfare, Office of Mental HealthResearch & Training.
Ridgway, P., Press, A., Anderson, D. and Deegan, P. (2004) The Recovery Enhancing Environment Measure (REE): Report of a Massachusetts Field Test. A project of Pat Deegan, Ph.D & Associates, LLC, Byfield, MA
Williamson, J. (2005) A users and providers perspective of the implementation of WRAP plans in Stoke on Trent (unpublished report)
On the basis of the existing scientific work in the US and pilot work in the UK, the next phase of development is justified. This involves two stages:
Stage 1: Measure amendment
In the light of the pilot studies, three draft versions of DREEM will be developed: User-DREEM, Staff-DREEM and Service-DREEM. The aim of User-DREEM will be to identify the patient’s perspective on their current level of recovery. The purpose of Staff-DREEM will be for staff to assess the extent to which they are working in a recovery-focussed way with patients on their caseload. The aim of Service-DREEM will be to assess the extent to which the service as a whole is promoting recovery. For each measure, relevant items from the existing DREEM will be extracted, with appropriate item-reduction techniques employed. Three approaches to shortening the questionnaires will be used; First, statistical data reduction techniques based on the available pilot data; Second, user-led consultation with patients and their representatives and Third, consultation with experts in questionnaire design.
Stage 1 will cost approximately £65,000 (£4,000 statistical consultancy, £45,000 for RA1B.6 researcher for 1 year (including on-costs and overheads) to undertake user consultation, £2,000 data management, 5 x £1,000 expert consultation, £4,000 project supervision, £5,000 contingencies)
Stage 2: Evaluation
The three measures will then be tested in a formal reliability and validity trial, using standard approaches to establishing adequacy of psychometric properties. This will include multi-site testing of the measures in routine use, augmented with questionnaire surveys to establish content validity.
Stage 2 will cost approximately £160,000 (£135,000 2 x RA1B.6 for 18 months (inc. on-costs and overheads), £6,000 project supervision, £3,000 statistical support, £2,000 data management, £6,000 service user involvement, £8,000 contingencies)
Although presented as separate stages, it is recommended that they be commissioned as a single piece of work.
The study could be co-ordinated by Dr Mike Slade, Head, Section of Community Psychiatry (PRiSM), Health Service Research Department, Institute of Psychiatry. Dr Slade has extensive experience in development of standardised measures, and is Borough Lead Psychologist for Recovery in Croydon.
|Taking forward the Mayor’s Transport Strategy Accessibility Implementation Plan||Piers Anthony Alien Plot The author wishes to thank Alan Riggs for developing this volume|
|Summary List of Recommendations and Pending Policy Options||1891 Introduction by Frederick Engels On the 20th Anniversary of the Paris Commune|
|I. Introduction and Background||Introduction and Background|
|1. 0 introduction 1 Background to the programme||I am delighted to have this opportunity of speaking on this topic. I warmly welcome the interest that ippr is taking in the impact of faith on public policy, on identity and the public realm, and I am grateful to Professor Michael Kenny in particular who invited me to contribute to this series|
|Introduction and Historical Background of the Eighth Century 5||Keywords Interactive experience, Emotional design, Mobile eBooks. Introduction|