The Better Regulation website is a system created to catalogue and search for formal and legal documents, and to notify about changes introduced in these documents. It is used by large companies and organisations in the UK and Ireland.
Purpose and challenges
The aim of the project was to build a system allowing for easy browsing and managing thousands of documents, as well as searching and immediate sending many emails notifying the clients about changes in the documents or the scope of documents they observe.
We have encountered several significant challenges that we had to face during our work. These were:
The client wanted to move from the old system to the new one as soon as possible. On our part, this involved migration of a huge amount of data simultaneously.
The whole development was organised in such a way as not to interfere with the simultaneous page editing. Immediately after the migration and the creation of basic administrative functionality, the client efficiently moved to the new system, while the development continued. We were creating and implementing new functionalities almost every day, in such a way as not to risk any delay of the entire team of editors working in the system.
In order to provide users with the best experience when using the website, the Better Regulation team has continued to iterate, change and increase the requirements. Our task was to remain flexible in relation to the client's expectations and to quickly adapt to their new needs and priorities.
The client's requirements went far beyond what the Drupal's core and the available modules can offer. We have managed to overcome the challenge by extending the functionalities and the potential of Drupal to a very high degree. In addition, we have divided large functionalities into microsystems.
Large data resources require non-standard server solutions. In order to make using the website pleasant and failure-free, we had to develop a scalable production and testing environment. We have employed the AWS infrastructure for this.
it allows us not only to change the version or replace any technology used on the website but also gives the ability to test the architecture locally (on the developer's computer).
The project has more than 25 containers used for, among others: generating and queuing the sending of a very large number of emails, searching for invalid links and files in the subpages' content, regenerating data in order to create HTML, PDF and DOC files.
content searching is one of the most important functionalities on the website. Solr allows us to dynamically adapt the search engine to the client's requirements.
notifying the users in real time about new records, using WebSockets.
an infrastructure allowing for sending of a very large number of messages with notifications and newsletters.
automatic tests are necessary for such a large project. Before every implementation, we are able to check whether our changes have not introduced new errors on the website.
they monitor the performance of the application, which allows us to quickly find problems with performance and response immediately.
it monitors the parameters of our servers, such as the amount of free space or the use of CPU/RAM.
In order to ensure the proper pace, agility and response to changes, we have employed a continuous approach to the development process supported by Git, Jenkins and Fabric. Every functionality and change was created within an independent branch, separately tested and verified by the QA team on a dedicated copy of the production environment. Portions of changes were automatically collected and presented to the client for testing. Upon approval, the changes were implemented for production. This process could be repeated often. Sometimes only a few hours have passed from the initial idea to the implementation into production, without bearing a significant risk.
In order to ensure no regression in the code, automatic tests were used.
Implementation and support
When the website was made available to users, we were present at every step to ensure that everything goes smoothly. The first few days are always key to making sure there are no hidden errors or other problems.
Support is carried out in a weekly sprint. After the publication of new functionalities in the test environment and approval by the client, the changes are implemented once a week in the evening.
Thanks to the process described above, since 2015 we have been able to react quickly to new requirements and changes resulting from observing the behaviour of the website users.
I could not rate Droptica highly enough. They have genuinely helped our dream service become a reality.