Title Measuring satisfaction with GOV.UK
Workshop Workshop 2014
Year 2014

Purpose of the study: Government Digital Service (GDS) requires that all UK government services measure user satisfaction with experience as one of four key performance indicators. We wanted to explore the best way to ask about user experience, both in terms of questions to ask and the way to ask them.
Approach and preliminary findings: We run frequent usability tests, so we used them as opportunities to test the most widely-used user experience questionnaire (Brooke, 1996) and questions from SUPR-Q, a similar instrument that includes credibility and loyalty (Oliver, 2010). Neither instrument tested well, so we created a bespoke questionnaire. We expect to try different approaches in the next few months to explore how these work compared to the existing single question. The approaches that are currently being discussed or explored are:

  •  A permanent banner that offers a link to a survey to everyone
  •  A permanent question at the end of each transaction that offers a link to a survey
  •  A pop-up request to respond that is offered to a sample of visitors
  •  The questions presented in an omnibus survey offered to a market research panel.
  •  Email invitations to an online survey offered to a probability sample.

Value: GDS is committed to doing as much as possible in public. The scores from these measures are prominently displayed on the Performance Platform and are used to assess departmental performance (and quite probably, the performance of service managers as individuals). It is important to find an easily-administered, reliable measure that fits with the GDS Design Principles. This creates challenges for questionnaire development.
Research limitations: We are not survey methodologists; we are user researchers who are working within government constraints, and within the remit of Lane Fox (2010). This creates the opportunity to try things quickly, but not in a way that would be thought of as an academic experiment. We are keen to learn how we can improve the ways in which we are collecting and interpreting this data.