Staging data changes for review and approval

By default, anyone with suitable access control permissions to change an entity can commit such a change to LUSID immediately.

You can configure LUSID to stage changes for approval instead. A staged change request is only committed when the last in a list of deciding users approves. If the change request is rejected by any deciding user in the list, the update is rejected and the entity is not changed.

Note that ‘change’ in this context means either:

Creating an entity belonging to one of the following types: Portfolio, Instrument, DataType, PropertyDefinition or an instance of a custom entity type. Other LUSID entity types coming soon.
Updating one of these entities; that is, editing its core data fields or adding, editing or removing properties and, where applicable, identifiers.
‘Deleting’ one of these entities. Note that since LUSID is bitemporal, no entity is actually deleted, but rather marked as removed from the ‘as at’ timestamp of the delete operation.

Consider a scenario in which two users with different permissions co-operate to create a portfolio in LUSID (click to enlarge):

Note the following:

You can operate the staging service using APIs, but the LUSID web app offers the best experience.
If an API request changes more than one entity, each is staged (or not) on its own merits. So if you batch upsert multiple instruments, some may be staged for approval while others are committed straight away.

Example: Staging change requests for portfolios

Let's imagine we want to stage the following change requests for approval:

A request to delete a portfolio.
A request to create a portfolio in a particular scope by a user with a particular role.
A request to change a particular property value.

In addition, we want to specify that only LUSID administrators can be deciding users.

All other portfolio change requests in the scenario can be committed immediately, without approval.

Step 1: Creating a staging rule set

The first step is to create a staging rule set for portfolios. Note the following:

Only one staging rule set for portfolios can exist in LUSID at a time.
The rule set can contain any number of staging rules specifying match criteria. These rules are ordered, and for any change request the first matching rule applies.
If a change request does not match any rule, it is not staged and therefore committed straight away.

We could call the CreateStagingRuleSet API and specify staging rules at the same time, but the simplest method is to use the LUSID web app to create an empty rule set first:

Navigate to the Staging Service > Rule Sets dashboard.
Click the Create rule set button.
Make sure Portfolio is selected from the Entity type dropdown, and specify a suitable name and description.
Click the Create button to add a rule set for portfolios to the list:

Step 2: Creating a staging rule for portfolio delete requests

We could call the UpdateStagingRuleSet API to add rules to an empty rule set, but the simplest method is to use the LUSID web app:

Click on the Portfolio link (highlighted above) to navigate to Staging Service > Rule Sets > Portfolio (if you are not transitioned to this dashboard automatically).
Click the Create rule button to create the first rule.
At a minimum, specify a Rule ID to uniquely identify the rule in the set, for example 001.

As a demonstration, we can now click the Create button to accept the remaining default dialog settings and create a generic rule that stages any portfolio change request by any user for approval by a single deciding user, who can be any LUSID user except the staging user:

We can call the GetStagingRuleSet API for Portfolio to examine the JSON definition of this generic rule:

{
  "entityType": "Portfolio",
  "stagingRuleSetId": "1672b0a7-55c3-4630-a567-8a7d57bc7cf9",
  "displayName": "Portfolios",
  "description": "Staging rule set for portfolios",
  "rules": [
    {
      "ruleId": "001",
      "description": "",
      "status": "Active",
      "matchCriteria": {
        "actionIn": [
          "Create",
          "Update",
          "Delete"
        ],
        "entityAttributes": "True"
      },
      "approvalCriteria": {
        "requiredApprovals": 1,
        "stagingUserCanDecide": false
      }
    }
  ],
  ...
}

To narrow the scope of this rule to stage just delete requests for approval by LUSID administrators instead:

Click the Edit button at the end of the list to edit the rule:
In the Actions area, uncheck Create and Update to leave just Delete:
In the Approver area, click Select role and choose lusid-administrator. You can increase the No. of approvals required, but note if this exceeds the number of LUSID administrators then change requests will never be approved:

We can call the GetStagingRuleSet API again to see how the JSON definition of the staging rule set has changed:

{
  "entityType": "Portfolio",
  "stagingRuleSetId": "1672b0a7-55c3-4630-a567-8a7d57bc7cf9",
  "displayName": "Portfolios",
  "description": "Staging rule set for portfolios",
  "rules": [
    {
      "ruleId": "001",
      "description": "",
      "status": "Active",
      "matchCriteria": {
        "actionIn": [
          "Delete"
        ],
        "entityAttributes": "True"
      },
      "approvalCriteria": {
        "requiredApprovals": 1,
        "decidingUser": "id.scope eq 'LUSID_SYSTEM' and id.code eq 'lusid-administrator'",
        "stagingUserCanDecide": false
      }
    }
  ],
  ...
}

Step 3: Creating a staging rule for portfolio create requests

In the LUSID web app:

Click the Create rule button to create the second rule.
Specify a Rule ID that is different to the first rule, for example 002.
In the Actions area, uncheck Update and Delete to leave just Create:
In the Portfolio matching criteria area, specify values for any number of fields and/or properties that the change request must match in order for it to be staged. If the request does not match all the values, the rule does not apply. In our scenario, we’ll click Select fields to choose the Scope field and then enter an appropriate value, for example US-Equities. This means that any change request for a portfolio not in this scope will be committed straight away:
In the Requestor area, click Select role and choose an appropriate role, for example us-portfolio-manager. This means that any change request by a user without this role will be committed straight away, even if it is in the US-Equities scope:
In the Approver area, click Select role, choose lusid-administrator and then specify a suitable No. of approvals:

Note: If you use the UpdateStagingRuleSet API to add the second rule then note that it overwrites existing rules, so include the first rule in the request as well, in the correct order.

We can call the GetStagingRuleSet API to see how the JSON definition of the staging rule set rule has changed:

{
  "entityType": "Portfolio",
  "stagingRuleSetId": "1672b0a7-55c3-4630-a567-8a7d57bc7cf9",
  "displayName": "Portfolios",
  "description": "Staging rule set for portfolios",
  "rules": [
    {
      "ruleId": "001",
      "description": "",
      "status": "Active",
      "matchCriteria": {
        "actionIn": [
          "Delete"
        ],
        "entityAttributes": "True"
      },
      "approvalCriteria": {
        "requiredApprovals": 1,
        "decidingUser": "id.scope eq 'LUSID_SYSTEM' and id.code eq 'lusid-administrator'",
        "stagingUserCanDecide": false
      }
    },
    {
      "ruleId": "002",
      "description": "",
      "status": "Active",
      "matchCriteria": {
        "actionIn": [
          "Create"
        ],
        "requestingUser": "id.scope eq 'default' and id.code eq 'us-portfolio-manager'",
        "entityAttributes": "id.scope eq 'US-Equities'"
      },
      "approvalCriteria": {
        "requiredApprovals": 1,
        "decidingUser": "id.scope eq 'LUSID_SYSTEM' and id.code eq 'lusid-administrator'",
        "stagingUserCanDecide": false
      }
    }
  ],
  ...
}

Step 4: Creating a staging rule for portfolio update requests

In the LUSID web app:

Click the Create rule button to create the third rule.
Specify a Rule ID that is different to the other rules, for example 003.
In the Actions area, uncheck Create and Delete to leave just Update:
In the Portfolio changes to stage area, specify one or more fields and/or properties to monitor the change request for. If at least one of these fields/properties changes value, the request is staged. If none change value, the rule does not apply. In our scenario, we’ll click Select properties and then choose a Portfolio/Staging/ManagerName custom property, for example:
In the Approver area, click Select role, choose lusid-administrator and a suitable No. of approvals:

We can call the GetStagingRuleSet API to see how the JSON definition of the staging rule set rule has changed:

{
  "entityType": "Portfolio",
  "stagingRuleSetId": "1672b0a7-55c3-4630-a567-8a7d57bc7cf9",
  "displayName": "Portfolios",
  "description": "Staging rule set for portfolios",
  "rules": [
    {
      "ruleId": "001",
      "description": "",
      "status": "Active",
      "matchCriteria": {
        "actionIn": [
          "Delete"
        ],
        "entityAttributes": "True"
      },
      "approvalCriteria": {
        "requiredApprovals": 1,
        "decidingUser": "id.scope eq 'LUSID_SYSTEM' and id.code eq 'lusid-administrator'",
        "stagingUserCanDecide": false
      }
    },
    {
      "ruleId": "002",
      "description": "",
      "status": "Active",
      "matchCriteria": {
        "actionIn": [
          "Create"
        ],
        "requestingUser": "id.scope eq 'default' and id.code eq 'us-portfolio-manager'",
        "entityAttributes": "id.scope eq 'US-Equities'"
      },
      "approvalCriteria": {
        "requiredApprovals": 1,
        "decidingUser": "id.scope eq 'LUSID_SYSTEM' and id.code eq 'lusid-administrator'",
        "stagingUserCanDecide": false
      }
    },
    {
      "ruleId": "003",
      "description": "",
      "status": "Active",
      "matchCriteria": {
        "actionIn": [
          "Update"
        ],
        "entityAttributes": "",
        "changedAttributeNameIn": [
          "properties[Portfolio/Staging/ManagerName]"
        ]
      },
      "approvalCriteria": {
        "requiredApprovals": 1,
        "decidingUser": "id.scope eq 'LUSID_SYSTEM' and id.code eq 'lusid-administrator'",
        "stagingUserCanDecide": false
      }
    }
  ],
  ...
}

Staging users: Understanding when change requests are staged

A staging user is informed that a change request is staged, but not when it is approved or rejected.

For example, if you are using the Data Management > Portfolios dashboard in the LUSID web app to create a new portfolio then you see the following notification:

If you are using the CreatePortfolio API then you can examine the stagedModifications area of the 201 Created response. If countPending is greater than 0 then the change request has been staged (note the count may increment if another user creates a portfolio at the same time as you):

{
  "id": {
    "scope": "US-Equities",
    "code": "Tracker"
  },
  "type": "Transaction",
  "displayName": "US equities portfolio",
  "created": "2024-08-05T12:00:00.0000000+00:00",
  "stagedModifications": {
    "countPending": 1,
    "hrefPending": "https://acmecorp.lusid.com/api/api/stagedmodifications/?asAt=2024-10-03T10%3A02%3A56.9497150%2B00%3A00&filter=entityUniqueId%20eq%20%27dde14522-ef5c-44eb-ad28-f4dba9c445ec%20and%20status%20eq%20%27Pending%27",
    "idsPreviewed": [
      "35b5d105-d498-41bd-b2d6-f091090bd7f8"
    ]
  },
  "isDerived": false,
  "baseCurrency": "USD",
  "relationships": [],
  "instrumentScopes": [],
  "accountingMethod": "Default",
  "amortisationMethod": "NoAmortisation",
  "transactionTypeScope": "default",
  "cashGainLossCalculationDate": "Default",
  "instrumentEventConfiguration": {
    "transactionTemplateScopes": [
      "default"
    ]
  },
}

Deciding users: Approving or rejecting change requests

A deciding user can approve or reject a staged change request at any time, and must give a reason for the decison.

All deciding users must approve a change request in order for it to be committed to LUSID. Only one rejection is required to dismiss the request, in which the entity is not changed.

Note that deciding users are not currently notified that a change request is pending and a decision required. Each deciding user is responsible for monitoring the list of change requests periodically.

To do this, you could call the ListStagedModifications API with a LUSID filter of stagingRule.currentUserCanDecide eq true to list all change requests you must decide upon, then the ListRequestedChanges API with the ID of a particular request to examine details, and then the AddDecision API to either Approve or Reject it.

However, the simplest method is to use the LUSID web app:

Navigate to the Staging Service > Staged Modifications dashboard to list all change requests:
Click the Catch up button to cycle through each change request you must make a decision on in turn. You must Add Comment before you can click the Reject or Approve buttons.

Note that an entity in LUSID may have changed between the time a staging user posts a change request and the time you approve it. By default, the Actual change button is enabled to show the difference between data in the change request and data in LUSID now. You can click the Intended change button to see the difference between data in the change request and data in LUSID at the time the staging user posted it: