How do I add records to a dataset?

You can add any number of records to a dataset in accordance with the schema in its relational dataset definition (RDD).

Methods

LUSID API

LUSID web app

Luminesce

Call the BatchUpsertRelationalData API and specify the scope and code of the RDD in the URL, for example:

curl -X POST 'https://<your-domain>.lusid.com/api/api/relationaldatasets/MyDatasets/Addresses/$batchUpsert'
  -H 'Content-Type: application/json-patch+json' 
  -H 'Authorization: Bearer <your-API-access-token>'
  -d '{
  "record1": {
    "dataPointDataSeries": {
      "seriesScope": "MyScope",
      "applicableEntity": {
        "entityType": "LegalEntity",
        "entityScope": null,
        "identifierScope": "MyIdentifiers",
        "identifierType": "Banks",
        "identifierValue": "X-12345",
        "subEntityId": null
      },
      "seriesIdentifiers": {
        "AddressType": "Registered"
      }
    },
    "valueFields": {
      "Street": "1 Avenue Road",
      "City": "AcmeTown"
    },
    "metaDataFields": {
      "Email": "a@b.com"
    },
    "effectiveAt": "2025-01-01T00:00:00.0000000+00:00",
  },
  "record2": {
    "dataPointDataSeries": {
      "seriesScope": "MyScope",
      "applicableEntity": {
        "entityType": "Person",
        "entityScope": null,
        "identifierScope": "MyIdentifiers",
        "identifierType": "Managers",
        "identifierValue": "Doe-165344",
        "subEntityId": null
      },
      "seriesIdentifiers": {
        "AddressType": "Correspondence"
      }
    },
    "valueFields": {
      "Street": "3 Station Blvd",
      "City": "ToyTown"
    },
    "metaDataFields": {
      "Email": "johndoe@acme.com"
    },
    "effectiveAt": "2024-06-15T00:00:00.0000000+00:00",
  }'

Coming soon

Request fields

This section supplements the API documentation and on-screen help text in the LUSID web app.

You can upsert 5000 records in a single request. Each record:

Must have a dataPointSeries comprising components that together form the primary key. See below for more information.
Must have a valueFields containing key/value pairs for all the mandatory Value fields in the RDD, where the key is the fieldName. Since at least one Value field is always mandatory, this object cannot be empty.
Must have a metaDataFields containing key/value pairs for all the mandatory Metadata fields in the RDD, where the key is the fieldName. If there are no Metadata fields, or all are optional, you can omit this object.
Must have an effectiveAt specifying the datetime (or cut label) at which the record becomes valid in LUSID from a business perspective.

Constructing the primary key

A record is considered unique if all the values in its dataPointSeries are unique. More information.

This object consists of the following fields:

A seriesScope. Note we recommend giving every record in the dataset the same seriesScope providing a dataset has no need for entitlement boundaries. However, if you want to partition a dataset and give users specific access rights using LUSID’s IAM system, consider differentiating the seriesScope of privileged records.
An applicableEntity that is one of the allowed entity types specified in the RDD. If you want the record to apply to all entities of that type, specify just the entityType field. If you want to restrict the record to just one or some entities of that type, populate as many of the following fields as applicable:
- entityScope to identify the scope of a particular Portfolio, Instrument, InvestorRecord or InvestmentAccount. For Transaction, this is the scope of the parent portfolio. Omit this field for Person, LegalEntity or a custom entity, which are not scoped entities.
- identifierScope to reference part of a property identifier for a particular Person, LegalEntity, InvestorRecord, InvestmentAccount or custom entity. Omit this field for Portfolio, Instrument and Transaction, which do not have property identifiers.
- identifierType to reference part of a property identifier for a particular Person, LegalEntity, InvestorRecord, InvestmentAccount or custom entity. For Instrument, optionally set this field to an instrument identifier, for example LusidInstrumentId or Figi. For Portfolio and Transaction, specify a fixed value of code.
- identifierValue to reference a property identifier value for a particular Person, LegalEntity, InvestorRecord, InvestmentAccount or custom entity. For Instrument, optionally set this field to an instrument identifier value, for example LUID_0000GDHD or BBG000C05BD1. For Portfolio, specify the code; for Transaction, specify the code of the parent portfolio.
- subEntityId to optionally identify a particular Transaction in a portfolio.
A seriesIdentifiers containing key/value pairs for all the mandatory SeriesIdentifier fields in the RDD, where the key is the fieldName. If there are no SeriesIdentifier fields, or all are optional, you can omit this object.

The following table summarises applicableEntity fields for allowed entity types:

`entityType`	`Portfolio`	`Instrument`	`Transaction`	`Person` or `LegalEntity`	`InvestorRecord` or `InvestmentAccount`
`entityScope`	<scope>	<scope>	<scope>		<scope>
`identifierScope`				<idTypeScope>	<idTypeScope>
`identifierType`	`code`	<instrumentIdType>	`code`	<idTypeCode>	<idTypeCode>
`identifierValue`	<code>	<instrumentId>	<code>	<code>	<code>
`subEntityId`			<transactionId>
Example	Record applies to a specific portfolio: `"applicableEntity": { "entityType": "Portfolio", "entityScope": "Growth", "identifierScope": null, "identifierType": "code", "identifierValue": "UK-Equities", "subEntityId": null }`	Record applies to all instruments in a specific instrument scope with ISIN identifiers: `"applicableEntity": { "entityType": "Instrument", "entityScope": "FixedIncome", "identifierScope": null, "identifierType": "Isin", "identifierValue": null, "subEntityId": null }`	Record applies to a specific transaction in a portfolio: `"applicableEntity": { "entityType": "Transaction", "entityScope": "Growth", "identifierScope": null, "identifierType": "code", "identifierValue": "UK-Equities", "subEntityId": "Txn001" }`	Record applies to a specific person: `"applicableEntity": { "entityType": "Person", "entityScope": null, "identifierScope": "MyIdentifiers", "identifierType": "Managers", "identifierValue": "Doe-165344", "subEntityId": null }`	Record applies to a specific investor record: `"applicableEntity": { "entityType": "InvestorRecord", "entityScope": "EMEA", "identifierScope": "MyIdentifiers", "identifierType": "External", "identifierValue": "123-ABCDEF", "subEntityId": null }`

Subsequent updates

You can update existing records by calling the same BatchUpsertRelationalData API. If all the values in the request match for the following fields in the primary key, an existing data series is updated:

The seriesScope
All the fields in the applicableEntity object
All the key/value pairs in the SeriesIdentifiers object
The effectiveAt.

If all match but the effectiveAt is different, a new time-variant data point is added to an existing data series. You can retrieve all the data points in a data series by calling the QueryRelationalData API with queryMethod set to TimeSeries.

Note: If effectiveAt is a cut label and the only change is to a different cut label, then a new data point is added if this resolves to a different datetime. Otherwise, an existing record is updated.

If any value in the primary key (other than effectiveAt) does not match, a new data series is created.

In summary: LUSID updates an existing data series if you change a Valueand/or Metadata field. If you change any other field, LUSID creates a new data point, whether this is a time-variant addition to an existing data series, or an entirely new data series.

Deleting records

Coming soon