Setting up and monitoring an integration

You can set up, run, and manage your LUSID integrations via Integrations > Dashboard in the LUSID web app. Read more about our Partnerships and Integrations.

Setting up an integration

To set up an integration instance:

1. Contact us to enable access to your chosen integration. Note most integrations require you to have a third party licence to ingest data.

2. Navigate to Integrations > Dashboard and select Create instance.
   

3. Choose an integration from the list.

4. Follow the prompts in the Integration settings to configure your integration. The configurable settings can vary across integrations; if you require assistance, contact us.
   Note the following:

   • You can select the View imported fields button and inspect the Source Fields and Transformation Description columns to see how LUSID maps your data.
      

   • You can select the Edit imported properties button to specify additional properties you wish to add or remove from your data import. You cannot remove mandatory properties.
     

   • You may edit your integration at any time. The changes will take effect from the next run.

Running an integration

Your scheduled integration instances run automatically, but you can also choose to run an instance using one of the following methods:

curl -X POST 'https://<your-domain>.lusid.com/horizon/api/integrations/instances/<your-integration-instance-ID>/execute'
  -H 'Content-Type: application/json' 
  -H 'Authorization: Bearer <your-API-access-token>'

select * from Integration.Execute where InstanceId ='<your-integration-instance-ID>';

Monitoring integration runs

Monitoring the latest run

Once set up, you can monitor the most recent run of an integration via Integrations > Dashboard.

The Last run time and Last run result columns show you when the integration last ran, and whether it completed successfully.

Viewing and troubleshooting all runs

You can view further details for runs of all your integrations via Integration > Runs.

Note

You can also monitor your integration runs using dedicated Luminesce providers. Read more.

The dashboard displays information on each run, such as the number of data attributes:

• Mapped

• Loaded

• Skipped

• Failed

• Missing

Every run has a unique Run ID that you can click on to view in-depth information that can assist troubleshooting, including:

• The progress of the import at various stages

• If an integration supplies data via a file, which file LUSID has used

• At what stage in the process an error occurred

• Why LUSID skipped certain items

You can filter on the Ref Run ID column to see all automatic retries that LUSID attempts for a given initial integration run.

Note

To see the Ref Run ID column on your dashboard, you may need to reorder your columns.

Subscribing to IntegrationRunEnded system events

In addition to monitoring integration runs via the LUSID web app, you can subscribe to the IntegrationRunEnded system event. This allows you to trigger automated processes or receive notifications when an integration instance run completes.

LUSID emits an IntegrationRunEnded event regardless of whether the run succeeds or fails.

You can view the complete event schema, including the fields you can filter on, by calling the GetEventType API.

Example

The following example contains the request body you might send to subscribe to IntegrationRunEnded events for a particular integration instance by filtering with Body.InstanceId:

{
  "id": {
    "scope": "IntegrationRunEnded",
    "code": "BloombergDLPS"
  },
  "displayName": "Bloomberg DLPS Complete",
  "description": "Subscription for Bloomberg DLPS integration completion events",
  "status": "Active",
  "matchingPattern": {
    "eventType": "IntegrationRunEnded",
    "filter": "Body.instanceId eq '44d096c6-39d2-43db-952c-f5f65e30a74d'"
  },
  "useAsAuth": "00uji4ve5haBc1Hlf2p7"
}

Once you have a subscription, you can create:

• A notification to specify how you want to be informed

• An event handler in the Workflow Service that creates tasks

You can use mustache templates to include event fields dynamically in your notification messages.

Read more on system events and notifications.