You can retrieve data from LUSID using API endpoints designed to return one, some, or all instances of a particular entity type.
Note: This article describes filtering entities based on their own attributes. For information on filtering entities based on attributes of related entities, see this article. For filtering performance, see this article.
For example, to retrieve instruments from the LUSID security master, you might call:
- The ListInstruments API to retrieve all instruments.
- The GetInstruments API to retrieve multiple instruments with particular identifiers, for example FIGI
- The GetInstrument API to retrieve a single instrument with a particular identifier, for example FIGI
You can apply a filter to most API endpoints that retrieve more than one entity to restrict and qualify the data retrieved. For example, the following call to
ListInstruments applies a filter expression of
name startswith 'App' to retrieve just those instruments with 'App…'-like names:
curl -X GET "https://<your-domain>.lusid.com/api/api/instruments?filter=name%20startswith%20%27App%27" -H "Authorization: Bearer <your-API-access-token>"
To see whether an API endpoint supports filtering, examine its LUSID API documentation and look for the
filter query parameter:
Note that filtering in LUSID:
- Is case-insensitive.
- Must enclose string values in single straight quote marks (the
- Supports a set of operators such as
startswith; see Appendix A.
- Does not return null values or values that are not set (except when filtering using the
- Has some known limitations; see Appendix B.
Exploring filtering through an example dataset
Different types of entity have different attributes you can filter on:
|Attribute||Entity type||Example||More information|
|Built-in data fields and nested fields||Any||Click here|
|Custom data fields||Custom entity||Click here|
|SHKs (not shown in picture)||Portfolio||Click here|
|Relationships (not shown in picture)||Instrument, Portfolio, Portfolio group, Person, Legal entity, Custom entity||See this article.|
You can filter on an identifier using the syntax
identifiers['<identifier>'] <operator> <value>, for example
identifiers['Isin'] eq 'GB0007980591'.
<identifier> is case sensitive, so
identifiers['Isin'] is not the same as
identifiers['ISIN']. Note the identifier value must be enclosed in single quote marks.
You can filter on a field using the syntax
<field> <operator> <value>, for example
name eq 'BP'.
You can filter on a nested field using dot notation, for example
instrumentDefinition.instrumentType eq 'Equity'.
The names of fields you can filter on are available from Swagger in the response to a request to get an entity of that type. For example, the schema for a response to the
GetInstrument API looks like this:
Filterable fields are underlined in red. Note also the JSON storage format of the field (string, enum, array, dictionary and so on); not all operators are available for every storage format.
You can filter custom entities on a custom data field using the syntax
fields['<field>'] <operator> <value>, for example
fields['lastCheck'] gt 2021-12-01.
Note: The field value must be enclosed in single quote marks.
You can filter on a single-value property using the syntax
properties[<key>] <operator> <value>, for example
properties[Instrument/Ibor/Industry] eq 'Energy'.
<key> is case sensitive, so
properties[Instrument/Ibor/Industry] is not the same as
properties[Instrument/IBOR/Industry]. The 3-stage key value should NOT be enclosed in single quote marks.
You can filter on a multi-value property using the syntax
properties[<key>] <match> (~ <operator> <value>), where
any to retrieve entities with partially matching values or
all to retrieve just entities with exactly matching values. See the examples below for more information.
|To retrieve instruments with an Industry property…||Assumes Industry is a…||Apply the filter expression…|
|...where the property value starts with ‘Energy’||Single-value perpetual or time-variant property|
|...where any property value starts with ‘Energy’||Multi-value perpetual or time-variant property|
|...where all property values start with ‘Energy’||Multi-value perpetual or time-variant property|
Note: For multi-value properties, using the
eq operator in the standard way (for example
properties[Instrument/Ibor/Industry] eq 'Energy') acts as a shortcut for
properties[Industry/Ibor/Industry] any (~ eq 'Energy').
You can filter portfolios on a sub-holding key using the syntax
subholdingkeys[<key>] <operator> <value>, for example
subholdingkeys[Transaction/Demo/Strategy] eq 'Income'. Note this is not shown in the example dataset above.
<key> is case sensitive, so
subholdingkeys[Transaction/Demo/Strategy] is not the same as
subholdingkeys[Transaction/DEMO/Strategy]. The 3-stage key value should NOT be enclosed in single quote marks.
Note: For any operator in the table below, you can precede a filter expression with the
not operator to reverse the operation.
not displayName in 'MyPortfolio', 'MyOtherPortfolio'.
|Operator||Meaning||Description||Filter expression examples|
|Equals||Returns a record if the value exactly matches the specified value.|
|Not equals||Returns a record if the value does not match the specified value.|
|Greater than||Returns a record if the value is greater than the specified value.|
|Greater than or equal to||Returns a record if the value is greater than or equal to the specified value.|
|Less than||Returns a record if the value is less than the specified value.|
|Less than or equal to||Returns a record if the value is less than or equal to the specified value.|
|Returns a record if the value starts with the specified value.|
|Returns a record if the value matches any of the specified values.|
|Returns a record if the value is not in any of the specified values.|
|Returns a record if the value is set.|
|Returns a record if the value is not set.|
|Logical AND operator||Connects two or more filter expressions and returns a record if all resolve to true.|
|Logical OR operator||Connects two or more filter expressions and returns a record if one resolves to true.|
|Note you can combine the |
|If used with a filter expression, returns a record if any of the values in a set match. If used without, returns a record if there are any values. For use with multi-value properties and relationships only.|
|Returns a record if all of the values in a set match a filter expression. For use with multi-value properties and relationships only.|
|Returns a record if there are no values in a set. For use with multi-value properties and relationships only.|
The operators you can use depend on the JSON storage format of the attribute you want to filter on:
|Storage format||Example attributes||Supported operators|
|Not supported||Not supported|
|Dictionary||properties, identifiers (depends on entity type)||Depends on storage format of values|
|Array/List||properties, identifiers (depends on entity type), relationships||Depends on storage format of values|
Note the following:
- Derived properties are currently not filterable in LUSID.
- Filtering on attributes with a storage format of
datetimecutlabelis not supported.
- For multi-value properties, only values with 1024 characters or less are included in a filter result set in order to keep the system performant.
The LUSID API returns user-friendly messages that can be used to troubleshoot filtering errors:
|Error name||Error code||Example description||Troubleshooting|
|There was a problem with the syntax of the provided filter. The problematic token was: 'equal'||Check the syntax supplied in the string is correct. In this case the operator |
|One or more failures occurred. Cannot apply 'operator' to a 'instrument' and a 'string'||Operator used in the filter syntax is not supported for the storage format. Please use the operators as described in Appendix A.|
|One or more failures occurred. The field 'fieldName' referred to in the supplied filter is not valid or is not available for filtering||Field being used in the filtering is not currently supported. Check if |
|Timed out when attempting to filter records||LUSID allows a maximum time of 30 seconds for a query with a filter to run. Try improving the filter expression by adding additional parameters or contact Technical Support to discuss your use case.|
|Property not defined||Make sure that the 3-stage property key is correct and uses the property syntax described above. Please note that the property key is case sensitive.|