What is bitemporal data?

This is a way of storing information where each piece of data is captured with two datetime values:

effectiveAt: the datetime the piece of data is considered ‘valid’, in a business context.
asAt: the datetime the data was physically recorded in LUSID.

The general principle is explained in this Wikipedia article, this FINBOURNE blog post, and also by this third-party blogger.

Consider a live price quote for a stock, for example:

Vodafone Group PLC (VOD) @ 20 Aug 2018 10:02 UTC = 176.14 GBX

The effectiveAt for this quote is 2018-08-20 10:02:00 UTC. UTC is Universal Time Coordinated, and represents time at longitude 0°.

The asAt would be the datetime the quote was actually saved into the system. This datetime may be slightly later than the effectiveAt, as there may have been a delay in the quote being published or uploaded into LUSID.

Whenever any data is retrieved from LUSID, the user must supply datetimes for both effectiveAt and asAt (if omitted, the default is 'now', or latest). These datetimes are then used to filter the universe of data, to locate the most appropriate value.

The primary benefit of storing data bitemporally is that it allows all data changes to be recorded and audited. Consider if this quote had been saved with the wrong value. Using the bitemporal model, we are able to correct the value retrospectively. The corrected piece of data would have the same effectiveAt as before, but the asAt datetime on the updated record would correspond to the datetime the correction was made in LUSID. Since LUSID stores both versions of the piece of data, you are able to retrieve the latest (i.e. corrected) value using an asAt of 'now', or retrieve the original value using the asAt from the original record.

Understanding perpetual and time-variant data items

In LUSID:

A perpetual data item has a value that is considered permanent, for example a person's place of birth. It's still possible to correct wrongly-entered data and roll back the asAt timeline to see a history of values, but it's not meaningful to roll back the effectiveAt timeline; for a given asAt datetime, the value is the same from the beginning to the end of time. Perpetual data items are considered monotemporal in LUSID.
A time-variant data item has a value that can vary over time, for example a person's address. When you enter such a value, you specify an effectiveFrom datetime; you can either specify an explicit effectiveUntil datetime, or LUSID can apply one for you if/when you supersede the value (that is, enter one with a later datetime). For a time-variant data item, it's possible to both:
- Correct wrongly-entered data and roll back the asAt timeline to see a history of values.
- Roll back the effectiveAt timeline and see a history of a time-series of values.
Time-variant data items are considered fully bitemporal in LUSID. For an examination of perpetual vs time-variant properties, see this article.

Entity type (see a full list)	Entity (core)	Identifier(s)	Data fields	Properties	Explanation
Portfolios and related
Portfolios (transaction, reference and derived)	Time-variant	Time-variant	Time-variant	Perpetual, Time-variant
Transactions (sub-entity of portfolio)	Time-variant	Time-variant	Perpetual	Perpetual	A transaction represents an effective-dated portfolio event.
Holdings (sub-entity of portfolio)	Time-variant	N/A	Perpetual	Perpetual	A holding is a point-in-time concept. Its fields and properties evolve with the effective date upon which portfolio holdings are built.
Portfolio groups	Time-variant	Time-variant	Time-variant	Perpetual, Time-variant
Corporate action sources	Perpetual	Perpetual	Perpetual	N/A	A corporate action source is a container for (effective-dated) instrument events.
Instrument events (corporate actions)	Perpetual	Perpetual	Perpetual	Perpetual	An instrument event is an effective-dated concept. Other parts of LUSID can process them in effective date order.
Tax rule sets	Time-variant	Time-variant	Time-variant	N/A
Instruments and related
Instruments	Perpetual	Perpetual, Time-variant	Perpetual	Perpetual, Time-variant	Instrument economic definition describes, among other things, the effective dated instrument lifecycle events.
Conventions	Perpetual	Perpetual	Perpetual	N/A
Counterparty agreements	Perpetual	Perpetual	Perpetual	N/A
Credit support annexes	Perpetual	Perpetual	Perpetual	N/A
Calendars	Perpetual	Perpetual	Perpetual	Perpetual, Time-variant	A calendar is a container for effective dates.
Order lifecycle
Allocations	Perpetual	Perpetual	Perpetual	Perpetual	The entity represents a concept with a single (effective) date. Other parts of LUSID can process this entity in effective date order.
Blocks	Perpetual	Perpetual	Perpetual	Perpetual	The entity represents a container for orders, each of which has an effective date.
Executions	Perpetual	Perpetual	Perpetual	Perpetual	The entity represents a concept with an effective date range. Other parts of LUSID can process this entity in effective date order.
Orders	Perpetual	Perpetual	Perpetual	Perpetual	The entity represents a concept with a single (effective) date. Other parts of LUSID can process this entity in effective date order.
Order instructions	Perpetual	Perpetual	Perpetual	Perpetual
Packages	Perpetual	Perpetual	Perpetual	Perpetual
Participations	Perpetual	Perpetual	Perpetual	Perpetual
Placements	Perpetual	Perpetual	Perpetual	Perpetual	The entity represents a concept with an effective date range. Other parts of LUSID can process this entity in effective date order.
Others
Legal entities	Perpetual	Perpetual	Perpetual	Perpetual, Time-variant
Persons	Perpetual	Perpetual	Perpetual	Perpetual, Time-variant
Custom entities	Perpetual	Perpetual	Perpetual, Time-variant	N/A	You can choose which fields are perpetual and which are time-variant when defining the custom entity
Relationships	Time-variant	N/A	N/A	N/A
Market and related data
Complex Market Data	Time-variant	Time-variant	Perpetual	N/A	Each piece of complex market data represents a point in effective datetime. The effective date forms part of the data's identifier.
Quotes	Time-variant	Time-variant	Perpetual	N/A	Each quote represents a point in effective datetime. The effective date forms part of the quote's identifier.
Structured Result Data	Time-variant	Time-variant	Perpetual	N/A	Each piece of structured result data represents a point in effective datetime. The effective date forms part of the data's identifier.
Types and configuration
Transaction types	Perpetual	Perpetual	Perpetual	Perpetual
Recipes	Perpetual	Perpetual	Perpetual	N/A
Cut labels	Perpetual	Perpetual	Perpetual	N/A
Custom entity types	Perpetual	Perpetual	Perpetual	N/A
Data types	Perpetual	Perpetual	Perpetual	N/A
Property Types	Perpetual	Perpetual	Perpetual	N/A
Relationships	Perpetual	Perpetual	Perpetual	N/A
Sequences	Perpetual	Perpetual	Perpetual	N/A