Recipes: Locating market data

A recipe has a market section that you must populate so LUSID can locate and retrieve suitable market data for each type of instrument you wish to value:

• For simple exchange-traded instruments such as equities, you are likely to require market prices held in the LUSID Quote Store. Note you can override market prices in some circumstances using holding-level data such as unit cost or last traded price.

• For complex exchange-traded or OTC instruments, you are likely to require interest rate, discounting or credit curves and/or volatility surfaces held in the LUSID Complex Market Data Store.

• For any instrument (including a cash instrument) held in a different currency to the portfolio currency, you are likely to require FX rates held in the LUSID Quote Store.

Note: You can call APIs to check the quality/quantity of market data required for the valuation operation you are trying to perform. See how to do this.

By default, a recipe must be able to locate market data that is valid ~ one day prior to each date in the valuation schedule. The precise validity period is determined by the valuation time. For example, if you are valuing a portfolio containing a single equity on:

• 7 March 2022, the time defaults to 00:00:00 and the recipe must be able to locate at least one market price for that equity (and an FX rate, if applicable) effective 6 March 00:00:00 -> 7 March 00:00:00. Prices effective 5 March or earlier, or 7 March later than midnight are not valid.

• 7 March 2022 17:00:00, the recipe must be able to locate at least one market price effective 6 March 00:00:00 -> 7 March 17:00:00.

• 7 March 2022 23:59:59, the recipe must be able to locate at least one market price effective 6 March 00:00:00 -> 7 March 23:59:59.

In other words, there is an automatic look back, or ‘grace’, period of between 24 and 48 hours. You can specify an explicit quoteInterval to look back further than this if you wish.

Understanding market data rules

For each category of market data (that is, market price, FX rate, interest rate curve and so on) you are required to locate you should specify a market data rule in the marketRules array and/or groupedMarketRules array of the market section of your recipe. Consider the following example, of three market data rules in the marketRules array intended for a portfolio valuation on 7 March 2022:

Note the following:

• Each market data rule is self-contained and must contain all the information required to locate a particular category of market data in an appropriate store.

• Market data rules are processed in order. The first matching rule found for a category of market data is used (as determined by its key).

• You can provide multiple rules for the same key in order to provide fallback logic. For example, you could have two rules for Quote.Figi.* that are identical except the first has a supplier of DataScope and the second a supplier of Bloomberg. Prices from Refinitiv would be preferred, but the recipe would fall back to Bloomberg if no Refinitiv prices could be found.

• You can provide more sophisticated fallback behavior by specifying market data rules in the groupedMarketRules array in addition to the marketRules array. More information.

• You can perform operations such as taking the highest or latest price from a group of providers, or even synthesising an average price, by specifying market data rules in the groupedMarketRules array. More information.

• You can specify ultimate fallback behaviour for market prices (not FX rates or complex market data) by using pricing data stored at the portfolio level, such as unit cost or last traded price. More information.

Summary of fields in a market data rule

For information on all the fields and allowed values in a market data rule, examine the API reference (expand the configurationRecipe/market/marketRules section):

The following table provides a summary. Note in particular that:

• The key field defines the category of market data to locate.

• Given the category, all the other fields define the rule's criteria for retrieving appropriate data.

More information on the key field

The following table explains the key field in a market data rule in more detail:

Resolving market data errors

When you call the GetValuation API, you may see a MarketResolverFailure similar to the one below if one or more market data rules fails to locate suitable market data in an appropriate store. The first step is to make sure field values in the market data rule match those in market data objects according to the table above. If they do, you can try setting the allowPartiallySuccessfulEvaluation pricing model option to True to relax validation and call the API again, though of course missing market data will not be used in calculations:

{"name":"MarketResolverFailure","errorDetails":[{"id":"Failed to resolve market data item 
[Provider: Edi, PriceSource: , InstrumentId: BBG000C05BD1, InstrumentIdType: Figi, QuoteType: Price, Field: mid].
","detail":"Attempted to resolve with supplier [Edi] and sourceSystem [Lusid] in scope [recipe1] for effective date 
[07/03/2022 00:00:00], quoteInterval [None] and with as-date [2022-07-05T13:35:54.6077760+00:00], but failed 
because failed to find quote in store."}…]",…}

Grouping market data rules and performing preference operations

You can specify market data rules in the groupedMarketRules array as well as, or instead of, the marketRules array:

• You might specify market data rules in both arrays in order to provide sophisticated fallback behavior from a preferred supplier of quotes to a group of secondary suppliers.

• You might specify market data rules in just the groupedMarketRules array to perform an operation such as taking the most recent or highest price from a group of suppliers, or even synthesising an average price.

You can of course combine these use cases to perform operations on a group of secondary suppliers.

Consider the following example, of a market section with one market data rule for prices in the marketRules array and three inside a single group in the groupedMarketRules array, intended for a portfolio valuation on 7 March 2022. Note the operation on the group of rules is FirstLatest:

Note the following:

• A recipe should have at least one market data rule in order to be functional. It can be in the marketRules array, or on its own in a group in the groupedMarketRules array.

• You can have any number of market data rules in the marketRules array, and any number of rules in any number of groups in the groupedMarketRules array.

• LUSID reads the marketRules array first, then the groupedMarketRules array.

• Within the marketRules array, the order of market data rules is significant.

• Within the groupedMarketRules array, the order of groups is significant. The order of market data rules within a group is significant in some circumstances, depending on the operation being performed.

• Within the groupedMarketRules array, all market data rules must have the same key and quoteType. One exception is for quoteType with the FirstLatest operation, where you can mix the Price and DirtyPrice quote types.

• LUSID uses the first matching rule found for a particular category of market data.

The following marketDataKeyRuleGroupOperations are available:

Example: Synthesising an average market data item

You can request that LUSID retrieves market data items from all the rules in a group and synthesises a new market data item representing an average:

• Choose the AverageOfAllQuotes operation if you want the request to fail if a single rule in the group fails to retrieve a market data item.

• Choose the AverageOfQuotesFound operation to ignore failures unless all rules fail to retrieve items.

For example, imagine you have upserted to the LUSID Quote Store the following three market prices for a particular stock:

...and created a recipe with the following three market data rules in a group with the AverageOfAllQuotes operation:

"market": {
  "groupedMarketRules": [
    {
      "marketDataKeyRuleGroupOperation": "AverageOfAllQuotes",
      "marketRules": [
        {
          "key": "Quote.RIC.*",
          "supplier": "DataScope",
          "dataScope": "RefinitivPrices",
          "quoteType": "Price",
          "field": "open",
        },
        {
          "key": "Quote.RIC.*",
          "supplier": "Lusid",
          "dataScope": "LusidPrices",
          "quoteType": "Price",
          "field": "bid",
        },
        {
          "key": "Quote.RIC.*",
          "supplier": "Bloomberg",
          "dataScope": "BBGPrices",
          "quoteType": "Price",
          "field": "ask",
        },
      ]
    }
  ]
},
...

Assuming all three original market prices can be located, LUSID synthesises the following average price:

Note a synthesised market data item is not upserted to the Quote Store, so it cannot be retrieved by an API such as ListQuotesForScope. However, it is available in the manifest for a valuation operation.

Overriding market prices or falling back to holding-level pricing data

If the valuation operation you want to perform requires a market price (not an FX rate or complex market data), you can optionally specify override and/or fallback rules to use holding-level pricing data such as unit cost or last traded price:

• You might specify an override rule to use holding-level data in preference to a market price stored in the Quote Store.

• You might specify a fallback rule to use holding-level data if a market price cannot be located in the Quote Store.

Rules can be global or specific; that is, apply to every type of instrument or just to a particular type or asset class. Note if you specify a global override rule and no specific overrides then LUSID never attempts to locate market prices in the Quote Store.

Note: Holding-level pricing data may differ between portfolios, so if you are valuing multiple portfolios this may mean they are valued differently, even with positions in the same underlying instruments.

To do this, specify a holdingPricingInfo object in the pricing object of a recipe (not the market object); see the API reference for more information:

The following pricing data is available for each holding in any portfolio being valued:

Example: Specifying override rules

In the following example, if the valuation operation requires market prices:

• Instruments of type ComplexBond are valued using UnitAmortisedCost.

• Instruments of type Equity trading in euros are valued using LastTradedPrice.

• Instruments of every other type are valued using prices located in the Quote Store. Since no global overrideField is provided,  a market data error occurs if no such prices can be found:

"pricing": {
  "holdingPricingInfo": {
    "overrideField": "None",
    "specificOverrides": [
      {
        "dependencySourceFilter": {
          "instrumentType": "ComplexBond"
        },
        "field": "UnitAmortisedCost"
      },
      {
        "dependencySourceFilter": {
          "instrumentType": "Equity",
          "domCcy": "EUR"
        },
        "field": "LastTradedPrice"
      }
    ]
  },
  ...
}

Example: Specifying fallback rules

In the following example, if the valuation operation requires market prices that cannot be located in the Quote Store:

• Instruments of type Bond are valued using LastTradedPrice.

• Instruments of every other type are valued using UnitCost:

"pricing": {
  "holdingPricingInfo": {
    "fallbackField": "UnitCost",
    "specificFallbacks": [
      {
        "dependencySourceFilter": {
          "instrumentType": "Bond"
        },
        "field": "LastTradedPrice"
      }
    ]
  },
  ...
}