e-petitions – the Second Week
We launched the ability to sign petitions on the e-petitions site at noon on Thursday 4th August.
And it is fair to say that it has been far more popular than we ever expected.
The site has had over 2 million visitors since launch, over 12 million page views, 12,000 petitions raised, 700,000 signatures collected and the first petition hit 100,000 signatures (now over 200,000) in 5 days.
At peak we have seen over 12,000 hits per minute.
To put these figures in context, this is the level of traffic we would expect for the entire of Directgov.
At times the site has struggled to keep up with the demand on it. We have been working hard to increase the performance and tackle the bottlenecks and I must thank colleagues on the single domain team for their support which has been invaluable and really helped us out. There will be cake.
The Technical Issues
For those of you who are interested in the technical detail these are some of the technical issues we ran into and some of the learning from them:
- Sending emails out was too slow and was blocking the web servers from responding quickly.
- The firewall configuration was limiting the number of connections and affecting the ability of systems to intercommunicate.
- The search indexing was putting too much load on the search server, slowing down all search queries and therefore the site as a whole.
- We did not have enough monitoring at key points in the system.
- Scaling up the number of servers and the bandwidth took too long.
So learning from the experience:
- We are adding technical architect capacity to the team to review configuration at key stages and as part of live operation.
- We will back this up with standard components and tools for common functions such as email and firewall configuration.
- We are implementing additional monitoring tools to provide a range of consistent ways of monitoring all our systems and services.
- We are putting in place processes to ensure that we can rapidly redeploy resources to support systems under load, including making sure that we have all the necessary access to systems and tools, and can bring inhouse resources to bear in a managed way.
For now I would just like to thank everyone who has used the site, all those who have provided feedback (and we read every piece we get) and apologise to those who have had difficulty in accessing the site over the last week.
I hope we have addressed those issues but if you do have any problems or questions then please let me know.
Photo credit: Aaron Landry