Better Regulation

or a project full of challenges

i

The Better Regulation website is a system created to catalogue and search for formal and legal documents, and to notify about changes introduced in these documents. It is used by large companies and organisations in the UK and Ireland.

better-regulation
better-regulation

Purpose and challenges

The aim of the project was to build a system allowing for easy browsing and managing thousands of documents, as well as searching and immediate sending many emails notifying the clients about changes in the documents or the scope of documents they observe.

We have encountered several significant challenges that we had to face during our work. These were:

Migration

The client wanted to move from the old system to the new one as soon as possible. On our part, this involved migration of a huge amount of data simultaneously.

The whole development was organised in such a way as not to interfere with the simultaneous page editing. Immediately after the migration and the creation of basic administrative functionality, the client efficiently moved to the new system, while the development continued. We were creating and implementing new functionalities almost every day, in such a way as not to risk any delay of the entire team of editors working in the system.

227 000
Number of migrated records
70
Number of tables
22 GB
Size of the base
12 h
Duration of the migration
New iterations

In order to provide users with the best experience when using the website, the Better Regulation team has continued to iterate, change and increase the requirements. Our task was to remain flexible in relation to the client's expectations and to quickly adapt to their new needs and priorities.

Drupal extension

The client's requirements went far beyond what the Drupal's core and the available modules can offer. We have managed to overcome the challenge by extending the functionalities and the potential of Drupal to a very high degree. In addition, we have divided large functionalities into microsystems.

Infrastructure

Large data resources require non-standard server solutions. In order to make using the website pleasant and failure-free, we had to develop a scalable production and testing environment. We have employed the AWS infrastructure for this.

Few functionalities

Searching
An extremely complex search, including logical operators and special characters. The user can save the search parameters.
Notifications
Extensive email notifications: new results in the searches saved by the user, changes to the documents added to favourites, reviews based on the user's preferences. Mails sent at a specific hour of the day.
Printing
Printing functionalities tailored to the needs, including printing to various formats (HTML, PDF and DOC) and the ability to print selected parts of documents.
Authorisations
Very extensive user rights that depend on the selected modules and countries.
Mass editing of documents - Views Bulk Operation
It allows for editing many documents simultaneously. When there's a large amount of data, it makes the editors' job easier.
500 000
Number of records
218
Number of fields
25
Number of content types
Results exporting
Exporting the search results to an Excel and Word file.
Crawler
A bot scanning the documents for invalid links and the like.
Marketing tools
Enabling loading documents with limited resources for non-logged persons, in order to promote the website.

Technologies

Docker

it allows us not only to change the version or replace any technology used on the website but also gives the ability to test the architecture locally (on the developer's computer).

The project has more than 25 containers used for, among others: generating and queuing the sending of a very large number of emails, searching for invalid links and files in the subpages' content, regenerating data in order to create HTML, PDF and DOC files.

Solr

content searching is one of the most important functionalities on the website. Solr allows us to dynamically adapt the search engine to the client's requirements.

Node.js

notifying the users in real time about new records, using WebSockets.

RabbitMQ

an infrastructure allowing for sending of a very large number of messages with notifications and newsletters.

Codeception and Selenium for tests

automatic tests are necessary for such a large project. Before every implementation, we are able to check whether our changes have not introduced new errors on the website.

New Relic and Blackfire

they monitor the performance of the application, which allows us to quickly find problems with performance and response immediately.

Zabbix

it monitors the parameters of our servers, such as the amount of free space or the use of CPU/RAM.

Development process

In order to ensure the proper pace, agility and response to changes, we have employed a continuous approach to the development process supported by Git, Jenkins and Fabric. Every functionality and change was created within an independent branch, separately tested and verified by the QA team on a dedicated copy of the production environment. Portions of changes were automatically collected and presented to the client for testing. Upon approval, the changes were implemented for production. This process could be repeated often. Sometimes only a few hours have passed from the initial idea to the implementation into production, without bearing a significant risk.

In order to ensure no regression in the code, automatic tests were used.

Implementation and support

When the website was made available to users, we were present at every step to ensure that everything goes smoothly. The first few days are always key to making sure there are no hidden errors or other problems.

Support is carried out in a weekly sprint. After the publication of new functionalities in the test environment and approval by the client, the changes are implemented once a week in the evening.

Summary

Thanks to the process described above, since 2015 we have been able to react quickly to new requirements and changes resulting from observing the behaviour of the website users.

3 developers and a tester
Team
14 000 h
Total hours since the beginning of the project
350 h
Average monthly working time

Client’s opinion

I could not rate Droptica highly enough. They have genuinely helped our dream service become a reality.

Need a team of Drupal and PHP web development experts?

Contact us now!