Configuring the DataScope Select integration schedule to import EOD pricing data

The DataScope Select integration runs a schedule daily in your LUSID domain to provide you with the latest market data for EOD pricing. All quotes are upserted to the LUSID Quote Store in the scope LUSID-Refinitiv-DataScopeSelect.

On Day 1, you must configure the instruments you want to import data for, and configure the schedule to import EOD pricing data for at least one stock exchange.
Subsequently, you can reconfigure the instruments and schedule at any time to import data for different instruments, different stock exchanges, or create more schedules to cover different regions.

Note: As a prerequisite to this article, you must be subscribed to the DataScope Select integration, with your DataScope Select account connected to LUSID via the Configuration Store. Instruments must also be decorated with the TNC_ExchangeCode property from the Terms and Conditions schedule.

Step 1: Configuring the instruments you want to import EOD pricing data for

First, ensure Terms and Conditions reference data (which includes a TNC_ExchangeCode property value) has already been imported for any instrument you want to import EOD pricing data for. See how to do this.

To retrieve DataScope Select EOD pricing data for an instrument, you must configure the instrument with the property Instrument/LUSIDConfig-Refinitiv-DataScopeSelect/RefreshEodPrice and set the value to true. If an instrument does not have this property, or if its value is set to false, no pricing data will be retrieved for the instrument.

You can add one or more properties to many instruments simultaneously using the LUSID web app. To do this:

Sign in to the LUSID web app.
From the top left menu, select Data Management > Instruments:
Use the checkboxes to select all the instruments you want to add properties to. Then click the More icon and select Edit.
Select the property with Scope LUSIDConfig-Refinitiv-DataScopeSelect and Code RefreshEodPrice, then click Done:
Set the value of the property to true and then click Run. Note you can update the property value to false at any time to no longer retrieve data for a particular instrument.

Alternatively, there are many other ways to add properties to your instruments if you prefer. Read more on adding a property to an entity.

Step 2: Configuring the DataScope Select EOD pricing schedule arguments

Once you have added the RefreshEodPrice property to the instruments you want to retrieve data for, you can configure the schedule to import EOD pricing data for various exchange codes, such as LSE, NYSE or ASX. We recommend setting up multiple schedules, one schedule per region, to ensure the daily data import is done at an appropriate time.

To do this:

Sign in to the LUSID web app using the credentials of a LUSID administrator.
From the top left menu, select Jobs & Scheduling > Jobs:
Select Menu on the job with Scope LUSID-Refinitiv-DataScopeSelect and Code datascopeselect-quotes and then Create a schedule:
Give the schedule a Scope and Code that together uniquely identify it, along with a user-friendly Schedule name and Description:
Select Next to reach the Arguments tab. Click Edit on the --exchange-code optional argument and provide a Value of one or more exchange codes as a comma-delimited list. This list determines which data is imported into your LUSID domain when the schedule next runs; for example, you can set the argument to LSE,AEX,PAR to request the schedule imports data for instruments labelled with one of the three specified exchange codes:

Note data is only imported for instruments that have the properties Instrument/LUSID-Refinitiv-DataScopeSelect/RefreshEodPrice set to true and Instrument/LUSID-Refinitiv-DataScopeSelect/TNC_ExchangeCode populated with a value of the relevant exchange code (see previous section).
Select Next to reach the Triggers tab. Click Add trigger and select Daily to provide a UTC time for the schedule to run on days the exchanges are open.
We recommend choosing a time that's at least an hour after the EOD pricing data is released by the DataScope Select API to ensure validated end of day prices are retrieved for that day. You can find out what time data becomes available for each exchange by querying the Refinitiv.DataScopeSelect.FileCode Luminesce provider and consulting the FirstFileReleaseTimeGmt and SecondFileReleaseTimeGmt columns. See how to query the Refinitiv.DataScopeSelect.FileCode provider.

Note: See your DataScope Select User Guide for more information on data availability times.
Click Save to finish creating the schedule. You can navigate to Jobs & Scheduling > Schedules to view your new schedule. If you leave the schedule unpaused, the daily data import will commence on the next occurrence of your selected time trigger.
Note: All quotes are upserted to the LUSID-Refinitiv-DataScopeSelect scope in the LUSID Quote Store and stored against each of the instrument identifiers found on the instrument.
Repeat steps 1-7 to create one schedule for each region you want to retrieve EOD pricing data for.

Important: A close price of 0.0 is recorded if the schedule runs on a non-trading day and the checkbox Exclude Null Codes For Universal Close Price is not selected in your DataScope preferences dashboard. You can tick this checkbox to ensure null close prices are not included in the upserted quotes.

Once the schedule has run and upserted quotes to the Quote Store…

…you can configure a recipe to use this market data when performing valuations of your holdings.

Step 3: Enabling instrument enrichment for instruments created after Day 1

For instruments created after the initial data load, you can enable a subscription that automatically configures each new instrument created in the default scope with the properties specified in step 1.

To enable enrichment for new instruments after the initial data load has taken place:

Sign in to the LUSID web app using the credentials of a LUSID administrator.
From the top left menu, select Jobs & Scheduling > Subscription Management:
Locate the subscription with scope LUSID-Horizon and code Instrument-Enrichment and select Unpause.

Once the subscription is unpaused, any new instrument created in the default scope will be automatically configured with the required properties for DataScope Select data to be retrieved the next time the schedule runs.

Monitoring and troubleshooting

You can check the status and configuration of the latest EOD pricing run by navigating to Jobs & Scheduling > Data Feed Monitoring from the top left menu in the LUSID web app: