Foreword




Скачать 434.56 Kb.
НазваниеForeword
страница1/12
Дата27.10.2012
Размер434.56 Kb.
ТипДокументы
  1   2   3   4   5   6   7   8   9   ...   12





Decisive Moment’


The independent review of the Best Value Audit process


Final report to the Accounts Commission and Audit Scotland


May 2007

Clive Grace, Sandra Nutley, James Downe and Steve Martin

Foreword



Over the last two years, some two thirds of local authorities in Scotland have been reviewed through the process of Best Value audits. Some of these audits have reported good or very good performance against the statutory Best Value criteria; others have been less favourable. Some councils have welcomed the auditors’ findings, but others have contested them vigorously. For all concerned - in Audit Scotland, which undertakes the audits, as well as in the councils themselves - the Best Value audit process has been a demanding and challenging one, and has represented a significant investment of resource in terms of staff and councillors’ time, as well as money.


In the tradition of ‘physician heal thyself’, it is now appropriate for the Best Value audit approach to be subject to independent scrutiny and review in order to evaluate its strengths and weaknesses to date, and make recommendations about its future course and development. This is the purpose of this report. The study was commissioned as an independent evaluation from the Universities of Cardiff and Edinburgh. It was led by Dr Clive Grace and Professor Sandra Nutley, with Dr James Downe and Professor Steve Martin completing the research team.


It has been a fascinating and rewarding study to undertake, and as a research team we have learned a lot ourselves, as well as hopefully contributing to knowledge and understanding about the Best Value audit approach. We would like to thank all those who gave so generously of their time and views in interviews and by responding to the survey we carried out. Special thanks go to Carol Calder, our key contact at Audit Scotland, who was immensely helpful throughout the work.


The overwhelming impression from all the material we collected was of a local government ‘family’ who were all strongly committed to improving local authority services and community leadership - most importantly the councils themselves, but also COSLA, the Improvement Service for Scotland and SOLACE, as well as Audit Scotland, the Scottish Executive, and other stakeholders. This is undoubtedly a decisive moment for local public services in Scotland with the changes ushered in by the new electoral arrangements, with the arrival of a new administration in the Scottish Executive, with local authority performance indicators being revised, and with the Crerar Review of scrutiny due to report in the near future. We hope that our report will help to contribute to this momentum for change and strengthen the shared endeavours of all stakeholders to secure improved services and governance.


May 2007


Contents

page

Foreword 2


Acknowledgements and Authors 4


Summary and Recommendations 5


1.0 Introduction 10



  1. Background and Policy Context 14




  1. Assessment of the Best Value Audit Process 20




  1. Key Findings 28




  1. The Way Forward 51


Annex 1 The Research Team 60

Annex 2 Interview and case study topic guide 63

Annex 3 Survey questions 69

Annex 4 Bibliography 72

Acknowledgments


We acknowledge with gratitude the assistance of the many national stakeholders and officers and councillors in local authorities who agreed to be interviewed in the course of the study. We are also grateful to the officers who completed our survey, and we want to place on record our appreciation of the invaluable help and guidance that we have received from Audit Scotland.


The views expressed are however our own and do not necessarily reflect those of Audit Scotland, the Accounts Commission or the case study councils.


The authors

Clive Grace is Honorary Research Fellow in the Centre for Local & Regional Government Research, Cardiff University


Sandra Nutley is Professor of Public Management at the University of Edinburgh and Director of the Research Unit for Research Utilisation


James Downe is Senior Research Associate in the Centre for Local & Regional Government Research, Cardiff Business School


Steve Martin is Professor of Public Policy and Management and Director of the Centre for Local & Regional Government Research at Cardiff University

Summary and Recommendations


S1. This independent review of the Best Value audit (BVA) process was commissioned by the Accounts Commission as part of its wider review of the progress of the audits of Best Value and Community Planning. The study was undertaken by a team from the Centre for Local & Regional Government Research at Cardiff Business School and the School of Management at Edinburgh University. The overall objectives of the independent review were to:

  • Evaluate the effectiveness of the Best Value audit approach to date in making judgements about Best Value and promoting improvement in council performance;

  • Identify relevant knowledge and experience from the research literature and from practice in the rest of the UK; and,

  • Make recommendations for improvement of the BVA process, with a view to ensuring that the audits are proportionate and focused on service performance and the experience of citizens.


S2. The approach we took was to consider six main areas of investigation:

  • How well are current Best Value audit methods working?

  • How effective are Best Value audit reports?

  • What are the views of local authorities and other stakeholders on the role of the Accounts Commission?

  • What impact are Best Value audits having on councils?

  • What impact have audits had on citizens, service users and other stakeholders?

  • How should the Best Value audit approach develop in future, both in itself and in relation to other existing and potential new methodologies of providing assurance and supporting improvement and in the wider context of public services reform in Scotland?


We undertook our investigations between mid December 2006 and the end of March 2007.


S3. Best Value and Community Planning were introduced as statutory duties for local councils by the Local Government in Scotland Act 2003 which requires them to:

  • Secure Best Value (defined as achieving continuous improvement in the performance of functions); and

  • Initiate, facilitate and maintain Community Planning to ensure organisations work together to provide better public services and engage local communities in making decisions that affect them.


S4. In developing its BVA approach, the Accounts Commission (and Audit Scotland) drew on its prior experience of the Local Government Management Arrangements audits. The key principles which underpinned the resulting BVA process were:

  • Responsiveness to local context;

  • No league tables or scores;

  • Corporate arrangements; and

  • Continuous improvement.


S5. Key features of the way in which BVAs are currently undertaken include audit by a central specialist team and a cyclical audit. The first Best Value audit report was published in 2004. Reports on 18 of the 32 councils had been published by 31 March 2007.


S6. Audits have taken place against the backdrop of a number of very important wider policy developments including the Public Service Reform agenda. The Scottish Executive's Public Services Reform document sets out an approach which seeks to drive up quality and encourage innovation and to ensure that services are increasingly user focused and personalised.


S7. Our overall assessment is that the BVA approach is now established, has gained credibility, and in broad terms has been effective. The best test of this is that even those who have significant criticisms of the BVA approach almost invariably argue that it needs to be fine tuned rather than abandoned. It is also warranted by comparison with the broadly equivalent methodologies in Wales and in England as reflected in the WPI and CPA.


S8. There are some strong similarities between the BVA approach and both of these regimes. BVA, the WPI and CPA draw on the same underlying rationale i.e. the assumption that continuous improvement in a council’s functions and services is only possible over the medium term if the corporate processes are working well. This model of improvement has particular relevance as councils are increasingly required to address complex ‘wicked’ issues which cross traditional service and organisational boundaries.


S9. Quality of BVA reports - There has been an overall improvement in the quality of the reports as the BVA process has developed and matured. We compared published BVA reports with what we took to be ‘best practice’ examples of CPA reports (WPI reports are not public documents) and found no fundamental differences in quality.


S10. Professionalism of audit teams - One of the most important aspects of the BVA approach has been the degree of expertise and professionalism of the audit teams which have carried out the work. There is convincing evidence from a number of sources that the perceived credibility of teams has varied. Many of the weaknesses appear to us to have been in the early phases of the development of BVA. It is plain from Audit Scotland's own evidence to us that they have recognised the issue and have invested, and continue to invest, considerable efforts to strengthen the Best Value audit teams across all of these dimensions. The changes to the Best Value audit teams are fundamental to the success and development of BVA and the contribution it is capable of making.


S11. Clarity of method - Because the audit teams’ theory of improvement has not been made explicit, there has been a lack of clarity about the criteria that they are using to judge councils. This is an important area for attention because it fuels concern about the transparency of the process and the issues on which judgments are based.


S12. In our view a future round of the BVAs will therefore need to be based on a more explicit understanding of how improvement happens and will need to ensure that the:

  • Framework for assessing the forces and factors which actually contribute to a council’s ‘equation for change and improvement’ is aligned with the theory and model of improvement employed by Audit Scotland;

  • Self-assessment process by councils in turn tunes into that framework; and

  • Best Value audit process then provides a challenge to and test of an authority’s self assessment by a team with the skills and experience required to make independent judgements which command confidence and credibility.



S13. Consistency - The issue of perceived inconsistency arises in a number of forms. The first is that councils that have gone through the process later have experienced it as more demanding and challenging than those which went through it earlier. The second ‘inconsistency’ relates to judgements made between councils. The third perceived ‘inconsistency’ that we have found is between the BVA report and other audit reports which a council has received previously. The fourth ‘inconsistency’ is that some councils report that there has been a significant difference between the verbal feedback given to them by audit teams at the end of their work on-site and the written report which has followed (the latter is seen as having been more critical) and then in some cases a further gap between the judgements made by Audit Scotland, and those reached by the Accounts Commission.


S14. The issue of ‘impact’ - It is clear that BVAs have had significant positive impacts on councils, but that their effects have been uneven. Nearly all councils have benefited from the process of self-assessment and the conscious and systematic review of their own processes and performance that it has entailed.


S15. There is no evidence that BVAs have had a direct impact on service users. Given the nature of the process and the way in which it has been conducted, this is to be expected, but this needs to be made clearer to councils and others. The impact on councils’ partners has also been negligible because to date BVAs have focused on councils’ corporate capacity rather than on community planning. But this is clearly an area that demands more attention in the future.


S16. Despite the improvements which have been made as the BVA process has developed the evidence that we have gathered points to a number of issues which still need to be tackled in addition to those that we have highlighted above. These include the strong managerial focus of BVA reports; relatively little attention to community planning; insufficient linkage between the Best Value audit work and the work of other inspectorates, and the annual audit process; the phrase Best Value Audit has caused confusion; limitations in the use and interpretation of performance data and of the statutory performance indicators and in the extent to which the BVA reports deal with questions of probity and propriety and with the ‘joining up’ agenda; and, BVA reports have limited value in terms of public reporting.


S17. The way forward - Our recommendations are informed by ‘two faces of improvement’:

  • The first ‘face of improvement’ is improvement in council performance. This may be stimulated by a range of factors including internal forces for change, external scrutiny, sanctions and incentives, and public expectations; and

  • The second ‘face of improvement’ is the change which takes place in the methods and approach to external scrutiny in response to improvements in councils and/or as a result of learning and development by audit bodies.


S18. Our recommendations address three key areas:

  • Developing the purpose and the method of BVA in light of experience to date;

  • Developing and adapting BVAs in the light of changes taking place in councils; and

  • Developing BVAs in the light of developments in the overall strategy for reform and regulation of public services.


S19. Developing the purpose and method of BVAs - In our view, the future development of the purpose and method of BVAs needs to:

  • Build on the baselines provided by the first round of BVAs;

  • Clarify the purpose, scope and theory of improvement associated with BVAs;

  • Enhance the credibility and capacity of BVA teams;

  • Strengthen the links between BVA and other forms of external scrutiny; and,

  • Improve reporting, publication and communication of the findings of BVAs.


S20. The BVA approach is now sufficiently well established and well regarded that the first round of audits can be seen as providing credible baselines against which to measure future progress by councils. There are a number of ways in which we believe that the Accounts Commission and Audit Scotland need to clarify the purpose, scope and methods of BVAs. In our view the credibility and capacity of the BVA team needs to be developed as a matter of priority. This should involve putting in place a Development Plan for the Team as a whole and supplementing the existing team with senior ‘associate’ peers drawn from right across the UK. There is ample evidence from councils and other evidence that BVAs and other forms of audit and inspection are not currently sufficiently co-ordinated. There are a number of ways in which this issue could be addressed including improved co-ordination of the timetabling of inspections and better sharing of information between inspectorates; the introduction of relationship managers or a lead agency to be the first point of contact with councils on behalf of all inspections; and the merger of some or all of the inspectorates that oversee councils. There is a need in future to Improve the links between BVAs and the annual audit process, and improve the links between BVAs and other inspections. In our view there are a number of areas in which reporting of BVAs might be improved including removing the anomaly around publication, increasing public involvement in BVAs, improving reporting to the public, and a more active engagement of the media.


S21. Developing and adapting BVAs in the light of changes taking place in councils - In our view the future development of BVAs should involve:

  • A more proportionate and flexible approach;

  • An increased emphasis on self assessment;

  • A greater emphasis on corporate outcomes;

  • An increased emphasis on community leadership; and

  • Better use of benchmarks and existing good practice.


S22. Developing BVAs in the light of developments in the overall strategy for reform and regulation of public services - There are at least four important developments that we believe could help to shape the context within which future BVAs are implemented:

  • Making better use of the resources within and available to the ‘local government family’ itself;

  • The findings of the Crerar Review;

  • A restatement of key regulatory principles; and

  • The establishment of a task force to help produce and develop future guidance relating to BVAs.

Section 1: Introduction


1. This independent review of the Best Value audit (BVA) process was commissioned by the Accounts Commission as part of its wider review of the progress of the audits of Best Value and Community Planning. The study was undertaken by a team from the Centre for Local & Regional Government Research at Cardiff Business School and the School of Management at Edinburgh University. It is intended to complement the public consultation which has been undertaken by the Accounts Commission and which involved submissions from a wide range of interested parties. The consultation and the independent review between them will help to inform the Commission’s consideration of how Best Value audits should develop in the future.


2. The Commission considered that the time was right to review the BVA approach and any key changes that might be needed in the second round of audits once all 32 councils have completed the first round. A key objective for the Commission was to move on from current systems and processes to achieve a more rounded assessment of service delivery and outcomes which takes more account of the views and experience of citizens and service users.


3. The overall objectives of the independent review were to:

  • Evaluate the effectiveness of the Best Value audit approach to date in making judgements about Best Value and promoting improvement in council performance;

  • Identify relevant knowledge and experience from the research literature and from practice in the rest of the UK; and,

  • Make recommendations for improvement of the BVA process, with a view to ensuring the audits are proportionate and focused on service performance and the experience of citizens.


4. In reaching our conclusions we were asked by the Commission to take account of the:

  • Commitment to self assessment;

  • Importance of the local government democratic mandate;

  • Importance of local priorities;

  • Effectiveness of corporate governance of councils;

  • Need to minimise the burden and maximise the impact of Best Value audits by building on existing approaches and adopting a selective, risk based approach; and

  • Need for effective dialogue between councils and the Commission to agree improvement plans.

5. Experience from BVAs to date had suggested that the methodology might need to become more flexible, risk-based and proportionate, in order to recognise the differing levels of progress being made by councils, and their capacity to sustain continuous improvement. Other issues that we were particularly asked to consider included: the perception that there were differences between the Commission’s findings and the conclusions contained in the Controller of Audit’s reports; whether or not to introduce some form of scoring or rating of councils; the potential use of peers in the audit process; and ways of enabling more opportunity for the views of service users to be reflected in the audit process.


6. We were asked to address a number of key areas for inquiry but were not required to confine our analysis only to these questions if other important issues presented themselves during the course of our work or if our own experience and analysis pointed to other questions. In the event, during the course of the review we introduced a number of different ways of thinking about the BVA process, and these are reflected in our recommendations.


7. The review was based on in-depth interviews with key national stakeholders and senior officers and councillors in a range of different councils which had contrasting experiences of BVAs. As academic researchers we were able to bring independence to the study combined with substantial experience of policy relevant research and an in-depth knowledge of inspection of local government across the UK. The Commission and Audit Scotland made it very clear that there was a genuine willingness on their part to hear contrary, challenging and uncomfortable views if the evidence called for that.


8. The approach we took was to consider six main areas of investigation:


  • How well are current Best Value audit methods working?

  • How effective are Best Value audit reports?

  • What are the views of local authorities and other stakeholders on the role of the Accounts Commission?

  • What impact are Best Value audits having on councils?

  • What impact have audits had on citizens, service users and other stakeholders?

  • How should the Best Value audit approach develop in future, both in itself and in relation to other existing and potential new methodologies of providing assurance and supporting improvement and in the wider context of public services reform in Scotland?


9. We addressed these questions by gathering evidence from eight main sources:

  • A detailed analysis of the BVA reports that had been completed by the time of the study.

  • An analysis of approaches to audit and inspection in other parts of the UK.

  • An analysis of research and policy reports, papers and other documents in Scotland and elsewhere in the UK.

  • In-depth semi-structured interviews in seven councils.

  • In-depth semi-structured interviews with senior staff in the Commission.

  • In-depth semi-structured interviews with a range of key stakeholders at the national level.

  • A workshop with Best Value audit staff.

  • A survey of a range of corporate and service officers in all authorities which had been the subject of Best Value audits by the time of our study.


We undertook our investigations between mid December 2006 and the end of March 2007.

Section 2: Background and Policy Context


10. Best Value and Community Planning were introduced as statutory duties for local councils by the Local Government in Scotland Act 2003 which requires them to:

  • Secure Best Value (defined as achieving continuous improvement in the performance of functions); and

  • Initiate, facilitate and maintain Community Planning to ensure organisations work together to provide better public services and engage local communities in making decisions that affect them.


11. Prior to the 2003 Act, the duty of Best Value existed on a voluntary basis, having been introduced in 1999/2000 as a quid pro quo for a moratorium on compulsory competitive tendering. The 2003 Act and the supporting statutory guidance and advisory notes1 built on the practices that had been developed since 2000, and significantly extended the powers of the Accounts Commission and the Controller of Audit's reporting duties to include the duty of Best Value and Community Planning.


12. In developing its BVA approach, the Accounts Commission (and Audit Scotland) drew on its prior experience of the Local Government Management Arrangements audits. It also sought to learn from the experience of the equivalent systems in England and Wales (Comprehensive Performance Assessment (CPA) and the Wales Programme for Improvement (WPI) respectively). It was keen to ensure that the Best Value audits provided an overall assessment of an authority’s corporate capacity but, at the same time, it was keen to avoid a ‘rules-based assessment’ or one that ended up with an overall label or score for performance. This stance was primarily one of principle but there was also a pragmatic recognition that the CPA approach could not have been transferred to Scotland at the time, even if it had been deemed desirable, because of the lack of requisite performance data.


13. The key principles which underpinned the resulting BVA process were:


  • Responsiveness to local context – defining the goals that local authorities should aim for but allowing them discretion on the methods and routes they use to achieve these, taking into account the needs of local communities.

  • No league tables or scores – the provision of a descriptive account of each council’s strengths and weaknesses, focusing on areas where improvement is most needed.

  • Corporate arrangements – a focus on a council's corporate arrangements and how well these support Best Value. This includes examining how elected members meet their strategic and scrutiny responsibilities and how a council’s corporate management team fulfils its strategic role.

  • Continuous improvement – examining how councils can demonstrate they are achieving continuous improvement in performance.


14. Key features of the way in which BVAs are currently undertaken include:

  • Audit by a central specialist team within Audit Scotland, with local auditor input;

  • A cyclical audit, which aims to cover all councils over a period of three years;

  • A selective and tailored audit – which responds to the identified strengths and weaknesses of each council (which means that the BVA report on each council may cover different areas in detail and, as a result, the reports are not precisely comparable); and

  • Using and building on existing scrutiny information, including inspectorate reports and the annual work of the local external auditor.


15. Box 1 provides an overview of the key features of current BVAs.

  1   2   3   4   5   6   7   8   9   ...   12

Похожие:

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Foreword iconForeword

Разместите кнопку на своём сайте:
Библиотека


База данных защищена авторским правом ©lib.znate.ru 2014
обратиться к администрации
Библиотека
Главная страница