With the number of ministerial departments, public bodies and agencies on Inside Government increasing each month and features being released or adjusted daily, it is important to gauge the success of each iteration. That way, we can make sure that it continues to support existing and new users’ needs.
At the end of December, I wrote about the usability research we have been conducting on Inside Government since its launch. Here’s an update on our latest round of lab-based testing.
Qualitative insight
Our lab-based testing provides qualitative data. The testing combines 11 moderated lab-based and 14 unmoderated ‘in-home’ sessions conducted by an independent research agency.
Participants in the January sessions had a professional interest in the Ministry of Defence (MoD) and the Department of Energy & Climate Change (DECC) and were long-standing, frequent users of those departmental sites. They were, therefore, well-placed to give us honest, informed appraisals of what was now working better since transition and what wasn't.
Positive feedback
Awareness of GOV.UK was very strong, with many participants mentioning it even before they had been told it was the site they would be testing. They reacted positively to the new departmental sections, especially the consistent look and feel, which they felt was appropriate for a range of audiences
Participants felt that the department homepages looked good, and provided access to useful information, while the inline links offered useful cues for further journeys through the site.
After previous rounds of testing, we'd tweaked and repositioned the filters on index pages. Participants now found it easier to locate detailed policy information.
As soon as they landed on a policy page, participants were keen to find the latest information. Following on from that feedback we've ensured that this content is far more visible.
Topic pages also performed well, bringing together content on a particular subject from a number of departments. Providing this ‘horizontal’ browsing route across multiple organisations can be tricky but we were heartened to find out that users really appreciated this approach.
Constructive criticism
Positive feedback is great but we're more interested in finding out if and at what point, the site falls short of users’ expectations. These sessions showed us which sections of GOV.UK needed more work.
We know that search is the main method for navigating around GOV.UK, but these sessions demonstrated that the search functionality available within the site did not perform as well as expected. This caused some frustration among our participants.
This is now a priority for us and we're working on it. There’s a multidisciplinary group tasked with rapidly improving the search. Immediately after the lab testing, the Inside Government team used the information from the sessions to improve the relevance and timeliness of results in the site's internal search.
The top navigation menu was less used than we thought. Once users did see it, some expected that it would only apply to specific departments rather than the whole of government. We are addressing this and we'll retest soon.
Navigating back to departmental pages also proved challenging for participants when they found themselves within the main GOV.UK site. This ‘cross-product’ proposition will form the basis of a much larger piece of research, but right now we're looking at highlighting the ‘route home’ for those who want to base their use of Inside Government around a departmental homepage.
Large ‘advert-like’ images on department pages fared less well with participants, who criticised the use of these sites as ‘marketing vehicles’. We know from our research that users like appropriate imagery on the department sites, and we want to encourage departments to make choices that will enhance, engage and add value to their pages. With editorial guidelines already in place, these user insights will help to guide publishers on what works best.
Where do we go from here?
It may be uncomfortable to hear and see people critique the site, yet without this insight we wouldn't know what to improve. That is why we take these findings seriously and iterate the designs and functions. We then re-test.
We have another round of qualitative testing scheduled for April. We'll look at the issues we've discussed in this post and we'll test new features including topical events and location-based sites.
Before then we will have the results from another round of testing and data analysis on how Inside Government is being used. We’ll keep you posted.
7 comments
Comment by Nick Holmes posted on
"We know that search is the main method for navigating around GOV.UK, but these sessions demonstrated that the search functionality available within the site did not perform as well as expected. This caused some frustration among our participants."
That's somewhat of an understatement I suspect. I am completely baffled by how results are ranked as they appear to me completely random. Could you explain?
Comment by angelacollinsrees posted on
Hi Nick,
Thanks for your comment. Within the Inside Government results tab there is some boosting applied which ranks recent content above older content. This is something we introduced recently and tuning it is an ongoing process. We'd be happy to take a look at any specific search terms you think are returning 'random' results. We are actively working on search improvements and you can expect significant improvements in the coming weeks and months.
Comment by Nick Holmes (@nickholmes) posted on
Angela
Thanks for the reply. I think expectations will be different between public and pro users. For the public user, keyword relevance will be of prime importance and the expectation is "like Google"! For Inside Gov pro users, a heavier recency weighting is probably good.
But just about every result set baffles me.
Try a search for “legal aid”. The first several General results are all relevant (though as one can't see the keywords in context it's difficult for me to assess the relevance to me personally). But looking a bit further down the list we find a page on “AIDS and driving” which has no relevance whatever to legal aid and the page doesn't even include the word "legal" or any related concept.
Comment by angelacollinsrees posted on
Nick, I’m sorry to hear that you find the set of results baffling.
In your example, 'AIDS and driving' appears because 'aids' can also mean the plural of 'aid'. The top results contain the phrase 'legal aid', while later results match 'legal' and/or 'aid'. In most cases, users should find what they need in the first few results.
Not everyone searches for an exact phrase, so the partial matches help to cater for longer search terms where only some of the words may be relevant.
Comment by angelacollinsrees posted on
Hi Birgit,
Thank you for expressing an interest in giving us feedback.
We typically use independent research agencies who specialise in recruitment for user research. However, if you would like to send us feedback directly, you can do this by selecting the feedback link (https://www.gov.uk/feedback/contact) on our site.
We've already made some changes, so we would welcome your thoughts.
Comment by Marking one year since the Inside Government beta | Government Digital Service posted on
[...] our most recent round of usability testing, for example, many participants came to the sessions already aware of the project. Several of them [...]
Comment by Birgit Schmidt posted on
I'm really interested in giving user feedback on the site from a new user perspective, how is it possible to get involved in your next round of testing?