I've mentioned my role as Chair of Southwark's Housing Scrutiny committee a couple of times on this blog. Since taking this on in May last year our biggest piece of work has been an investigation into the statistics used to measure performance in Southwark's housing repairs service. Sounds technical, remote and boring doesn't it? I will do my best to convince you otherwise.Back in July 2010, the committee had to decide which housing issues we wanted to look into. Councillors on the committee (from all parties) had just been through an election campaign in which we'd knocked on lots and lots of doors. Among Southwark's tenants, by far the biggest concern was the quality of the contracted-out repairs service.
But there was a problem. All of the statistics used to measure the quality of the service showed not just good, but brilliant levels of performance. 99.9% of appointments made by contractors were kept. Satisfaction levels hovered around the 94% mark. 90%+ of repairs were completed on time. Even more striking was the fact that, based on these stats, Southwark had won a national award for its repairs service from Inside Housing magazine, describing Southwark's repairs service as "the best in London".
Faced with these contradictory messages, we decided to try and do something different. Instead of doing a general scrutiny of the repairs service, we chose to investigate how the stats were gathered and calculated. Even the most cynical of the committee members could not have imagined what we would find. The stats were not just flawed, but rotten to the core. Just a few of the things we discovered were:
- most of the stats come from the contractor operatives themselves via mobile PDA devices. This means they completely control the information for each job they had been asked to do.
- concrete evidence of mis-reporting of the completion of work and attendance at appointments by contractors, found during a "case-tracking" process.
- satisfaction levels are measured with a survey which excludes anyone who has an ongoing problem with a repair.
- Satisfaction is measured with a five number rating system which is rarely explained to tenants doing a phone survey, and which is weighted in favour of higher levels of satisfaction.
- the results from a new (and reliably conducted) survey which show dramatically lower levels of satisfaction (69% compared to 88.4%), appointments kept (78.9% compared to 99.9%) than are shown in the official stats.
- contractors themselves admit their performance doesn't live up to anything like levels shown in the stats. One representative said: “I don’t recognise the performance of my operatives in these statistics.”
The report, which is still in draft form, concludes: “Southwark’s housing repairs Key Performance Indicators currently provide little or no insight into the actual quality of service being provided by SBS and Morrisons” It goes on to make constructive proposals for the creation of a system that can be relied upon, including:-
- A new texting system that will enable customers to verify if a call is complete via their mobile phone
- A satisfaction survey which accurately measures real levels of satisfaction with the repairs service
- a recommendation that the council makes sure it is no longer paying multiple times for single repairs. This will save the council money.
Through this scrutiny we found that the stats which have so often been wheeled out to prove how good Southwark’s repairs service is, are inaccurate and unreliable. For tenants this has meant unattended appointments and incomplete repairs going unnoticed. I hope this draft report will serve as a catalyst for improving the repairs service which too often lets down tenants in Southwark.
The draft version of the report can be found here from page 7 onwards: http://moderngov.southwarksites.com/mgConvert2Pdf.aspx?ID=4339&T=9
This will now be subject to some minor changes following last Tueday's meeting and will then go before the full Overview and Scrutiny Committee. The sub-committee is optimistic that the report will eventually go before Southwark Cabinet and the recommendations will be implemented.
- a recommendation that the council makes sure it is no longer paying multiple times for single repairs. This will save the council money.
Through this scrutiny we found that the stats which have so often been wheeled out to prove how good Southwark’s repairs service is, are inaccurate and unreliable. For tenants this has meant unattended appointments and incomplete repairs going unnoticed. I hope this draft report will serve as a catalyst for improving the repairs service which too often lets down tenants in Southwark.
The draft version of the report can be found here from page 7 onwards: http://moderngov.southwarksites.com/mgConvert2Pdf.aspx?ID=4339&T=9
This will now be subject to some minor changes following last Tueday's meeting and will then go before the full Overview and Scrutiny Committee. The sub-committee is optimistic that the report will eventually go before Southwark Cabinet and the recommendations will be implemented.
Well done Gavin. Any estimate of the cost of repairs paid for several times as in "a recommendation that the council makes sure it is not paying multiple times for single repairs"? Not possible to get the money back, I suppose. How about changing the contractors who do the repairs? (or don't do them).
ReplyDelete