Managing transactional services is a complex task and can be quite bewildering with many systems and measurements involved. However, because we have digital tools and technologies, we can now pull together data from lots of different sources and inspire action through powerful visualisations. Adam Bailin from the Performance and Delivery Unit presents the Alpha of the Digital Performance Framework which intends to fulfil these aspirations.
Hi, my name is Adam. I work in the Performance & Delivery Unit in GDS.
We’ve been talking to some of the people in government who work on big, complex transactional services - people who work in organisations like Department for Work and Pensions (DWP), HM Revenue and Customs (HMRC), Driving and Vehicle Licensing Agency (DVLA), UK Border Agency (UKBA) and the Identity and Passport Service (IPS).
We wanted to get an understanding of how departmental and agency staff currently measure and manage the performance of their services, and whether we could help the product managers who are responsible for those transactions. They are busy people and the last thing they need is folks in the Cabinet Office asking them to fill out another form, but we’ve found a real appetite for guidance about how digital technology can be used to used to improve service performance.
We’ve also found a lots of appetite among the product managers within GDS for measuring and monitoring the products that we’re working on, to help reinforce the data driven design of service and platforms like GOV.UK
So we’ve drafted a checklist.
Checklists can help in all sorts of circumstances and can be particularly helpful when managing large complex systems. I can highly recommend Atul Gawande’s book ‘The Checklist Manifesto’ for amazing examples of checklists being used in aviation, construction and medicine.
Our checklist is called the Digital Performance Framework. In truth it’s a bit more than a checklist. It’s a step by step guide to digital performance management with examples and links to further reading. There are seven steps:
-
Understand user needs & business objectives:
Who uses performance information in your organisation? What are they interested in?
-
Decide what to measure and how to measure it:
What measures indicate how well the service is performing? Where will the information come from?
-
Install and configure platforms:
What software needs to be in place to measure and report the performance information you’re interested in?
-
Establish a baseline and benchmarks:
What are your current performance levels? How does that compare with other similar services?
-
Collect and aggregate data:
Can you easily combine performance information from different sources? How will you store the data?
-
Analyse and visualise data:
Is performance information visible to the right people? Are they acting upon it?
-
Monitor, iterate, and improve:
Have you tested performance improvements and measured which work best?
This is far from a finished product, but in the spirit of releasing early, we’re sharing a first draft now. Please let us know what you think. Is it useful? How can it be improved? Is there anything missing? Is it too short or too long? Is it just plain wrong?
We plan to release a second iteration of the framework in about a month (in HTML), and we’d love to incorporate ideas from the suggestions and comments we get whether you’re a member of staff, a user, or just someone who is interested in the performance of public services. The best way to give feedback is via email or to simply comment below.
9 comments
Comment by Policy, delivery, Government digital service posted on
[...] service for Office of Public Guardian (Lasting Powers of Attorney) with Ministry of Justice Jun: Digital Performance Framework and digital actions throughout the Civil Service Reform Plan, plus we joined OIX to develop the [...]
Comment by Digital Transformation in 2013: The strategy is delivery. Again. | Government Digital Service posted on
[...] service for Office of Public Guardian (Lasting Powers of Attorney) with Ministry of Justice Jun: Digital Performance Framework and digital throughout the Civil Service Reform Plan (PDF) plus we joined OIX Jul: Launched the [...]
Comment by Paul Scott posted on
I’d like to congratulate the team on an excellent piece of work. The checklist and framework document are concise, comprehensive and easy to follow.
The performance framework provides exactly the kind of structured approach we’d recommend. There are two areas we’d like to recommend further work on:
1. adopting performance metrics which capture customer feedback are vital for continuous performance improvement, but even more vital is ensuring the feedback is acted on in a closed loop process. NPS is an example of how this works, but there are plenty of other measures that work this way. We can explain in more detail if required.
2. Ensuring closed loop customer satisfaction processes work requires another key success factor: culture. If there isn’t a culture that encourages and embraces customer feedback in a wholly positive way, it’s almost impossible to improve performance based on customer feedback. This may be slightly outside the remit of the performance framework, but it is nonetheless a critical success factor.
The team has already identified the value of benchmarking and again, this is an areas our organisation has had considerable experience in over the past 30 years. We’d be keen to share this with you when the timing is right.
Comment by Andrew McNair posted on
Hi Adam,
as Head of Benchmarking and author of the annual Global Benchmarking Report, I'd be very happy to assist in providing a regional and/or sector specific cut of relevant customer management data (and guidance on key metrics) in support of section 4 Establish a Baseline and Benchmarks.
In the interim, i'd ask be happy to provide all group members with a free complimentary digital copy of the 2011 Global Benchmark Report (value of GBP 1500). I'm sure this would provide excellent scene setting and increase awareness levels on the value benchmarking can have on any operational improvement strategy, particularly as one endeavours to establish the 'as is' and 'to be' start and end points of each item contributing towards strategic objectives.
Please feel free to drop me a line at andrew.mcnair@dimensiondata.com and we can discuss your requirements.
Best regards,
Andrew McNair
Comment by Link roundup | Kind of Digital posted on
[...] Introducing the Digital Performance Framework – Alpha release | Government Digital Service – "Managing transactional services is a complex task and can be quite bewildering with many systems and measurements involved. However, because we have digital tools and technologies, we can now pull together data from lots of different sources and inspire action through powerful visualisations." [...]
Comment by Jonathan Joyce posted on
Thanks Adam - a useful top-level checklist I'd say.
I think there would definitely be an appetite for you to provide an overview of what you have done on GOV.UK against each of your checklist items.
Do you use goals and funnels to examine the use of your transactional apps? Do you measure social interactions and look at the spread of information beyond your sites?
Comment by adambailin posted on
Thanks Jonathan.
We're using the checklist internally against all of our products, including GOV.UK, and we're trying to embed it early on in the design process. Inevitably, some products are further developed and have completed more of the checkpoints than others. But even those at an early stage have started - see the example in 'Install and configure platforms' of using goal funnels on the 'register to vote' app.
I'm not an expert on measuring social interactions but we are making use of social media tools to engage on the performance framework: this blog, Twitter, bit.ly. Did you have any particular measures or tools in mind?
Comment by Naran Hirani posted on
Hi Adam,
I think the framework is very good and should prove very useful especially due to the plentiful examples.
The one comment I have for your consideration is that, to my mind, we should measure services end-to-end as experienced by the service users. I don’t think I spotted anything mentioning this notion in the current framework. Apologies in advance, if it was mentioned and I failed to spot it.
I am particularly interested in this area so would love to discuss this with you further.
Hope this helps.
KRs
Naran
Comment by adambailin posted on
Hi Naran. That's a really good point and certainly something we support, although we've not explicitly mentioned it in the framework. I'll make sure it's looked at for the next version.