The LUSID platform is formed of individual services that are operated by individual teams. Each team operates a continuous integration and continuous delivery (CICD) pipeline that sees changes go from a developer’s machine to production.
FINBOURNE performs many production deployments daily, ranging from infrastructure to telemetry and supporting services through to the applications in use by customers. LUSID can ship up to five times a day, with each change containing multiple commits or library changes.
LUSID has over sixty thousand tests that run during the testing and release cycle to ensure that a consistently excellent experience is delivered to customers. The SDLC for releasing change at FINBOURNE is designed to ensure that breaking API, functional changes or performance degradations are caught pre-release, requiring developers to advertise a change or to revert. All components in use within the LUSID platform must adhere to a minimum level of test coverage and pass a series of static analysis checks.
During the development of LUSID we discovered that customers wanted to be able to stage changes within a preproduction LUSID environment, to ensure tests pass before a change reaches production environments and to prevent disruption to end users.
To achieve this FINBOURNE created the release rings. This is a mechanism that allows two different versions of the LUSID API to run within a single tenant environment. One of these is the fast ring, the most up-to-date version of the code. The second is the slow ring, which is a version of the code that trails by seven days.
At the end of every day the most recent change deployed to production becomes the slow ring release for seven days’ time.
The LUSID web app (Web UI)
The LUSID web app is part of the slow ring to ensure that day-to-day changes are released in a more controlled and predictable manner. While FINBOURNE makes every endeavour to ensure that possible issues with features, functionality or performance do not get released into production, we understand that some clients may wish to mitigate this risk by making use of the slow ring.
It is the case that issues can be discovered within the slow ring; sometimes these issues require a code change. If a code change must be made it may take seven days for the fix to reach the slow ring.
Shared infrastructure
The LUSID platform shares some infrastructure and microservices between the fast and slow rings. For example, the underlying database is shared although data might be partitioned into different logical databases on a single physical server. Other resources are shared, such as:
- Microservices not included in the mesh; this includes Drive, Scheduler, Identity and Access.
- AWS and Azure resources, such as compute, databases, storage and IP addresses.
- Configurations that impact the routing of traffic to either the fast or slow ring. This means that the configuration for the routing exists on the fast ring.
How to use the rings
Typically, we find that customers wish to run a development environment on the fast ring and a production environment on the slow ring. This enables flexibility to integrate new features, whilst allowing time to review changes before moving to production.