How do I run several queries together as a multi-query?

Prev Next

A multi-query lets you build a dataset in named steps: each query can read the results of the ones above it, independent queries run at the same time, and you can save the whole set, and all its results, in one go. It's the tidiest way to turn a long script into something you can rerun and share.

Prerequisites

Method

Step 1: Write the queries

At the prompt, start each query with a header line, --- @Name. Everything up to the next header is that query's SQL.

--- @Fields (internal)
select TableName, FieldName, DataType
from Sys.Field
where TableName like 'Lusid.Instrument%'

--- @ByProvider
select TableName, count(*) as Fields
from @Fields
group by TableName

--- @ByDataType
select DataType, count(*) as Fields
from @Fields
group by DataType

--- @Instruments
select LusidInstrumentId, DisplayName
from Lusid.Instrument
where DisplayName like :name:'%Bond%'
limit 20

In this example:

  • @ByProvider and @ByDataType both read @Fields, so they wait for it, then run at the same time.

  • @Instruments reads nothing, so it starts straight away.

  • (internal) marks @Fields as a step that only feeds the others. It runs, but has no results of its own to show or save.

  • :name:'%Bond%' is a scalar parameter with a default value. LumiCode asks for its value when the set runs.

As you write, Tab offers the queries above as @Name wherever a table can go, and LumiCode underlines any problem with the set on its line.

Step 2: Run the set

Press Enter. The first time you run a set, LumiCode asks:

  1. How many queries run at once. The default is one at a time.

  2. How long each query may take.

  3. How long the results are kept.

  4. A name for the set.

To change these answers later, use /set queryset.

LumiCode then asks for any scalar parameter values, once for the whole set, and runs it. Each result is reported under its query's name.

If a query fails, the queries that read it are skipped, and the rest still run.

Step 3: Explore the results

/rows, /stats, /chart and /group take a query's name first. For example:

/rows ByProvider
/chart ByDataType bar

Without a name, LumiCode asks which result you mean. /feedback shows which query read which, followed by each query's log.

Step 4: Save the set and its results

Type /save. After a set has run, you can save:

  • All the results as one file: a workbook for Excel, a database for SQLite, or otherwise a zip with one file per result. Save it to your computer or to Drive.

  • Each result as its own file.

  • The set itself, into LUSID Workspaces or as a .lumiq.json file. /run opens it again later, with its settings.

Rules for writing a multi-query

  • Names start with a letter or _, contain only letters, digits and _, are at most 31 characters, and must be unique (case is ignored).

  • A query can only read queries above it.

  • A set can hold at most 25 queries.

  • To include a Saved Query, write --- @Name = {visibility}/{workspace}/items/{group}/{name}, with nothing under that line.

  • Not allowed: text above the first header, an empty query, and a \ separator line, because the headers already split the queries.

  • Lineage isn't available for queries run in a set.

In LumiCode, /help sets shows these rules at any time.