REST API Tutorial 5: On Demand market depth extraction

 

Last update Dec 2023
Environment Any
Language Any HTTP is supported
Compilers None
Prerequisites DSS login, internet access, REST API Tutorial 3
Source code Below

Tutorial purpose

Please follow the REST API tutorial 3, which explains the workflow, before going through this one.

This tutorial explains how to:

  • Retrieve the available field list from the server (optional).
  • Request market depth (top of the order book) data from the server, using an on demand request.
  • Check the request status.
  • Retrieve the data.

 

Table of contents

 

Market depth views

Market depth is a representation of the order book, i.e. open buy and sell orders, with the best prices.

When requesting Tick History market depth you can select different data representations, using a parameter called View. Here are the possible values:

 

    	
            

<EnumType Name="TickHistoryMarketDepthViewOptions">

<Member Name="RawMarketByPrice" Value="0"/>

<Member Name="RawMarketByOrder" Value="1"/>

<Member Name="RawMarketMaker" Value="2"/>

<Member Name="LegacyLevel2" Value="3"/>

<Member Name="NormalizedLL2" Value="4"/>

</EnumType>

These views deliver the data in different formats.

Most views deliver data in a raw format, where each record is a set of lines, the first defining the nature of the record (for example an update), the next lines defining the fields that changed, and their new value. Example for an update:

    	
            

CARR.PA,Market By Price,2016-09-29T09:00:00.079808027Z,+2,Raw,UPDATE,UNSPECIFIED,,,,6508,,31360,,0

,,,,Summary,,,,,,,,,3

,,,,FID,4148,,TIMACT_MS,32400031,

,,,,FID,6952,,TIMACT_MSP,95,

,,,,FID,17,,ACTIV_DATE,2016-09-29,

,,,,MapEntry,,UPDATE,,,,,,23.370A,7

,,,,FID,3427,,ORDER_PRC,23.37,

,,,,FID,3428,,ORDER_SIDE,2,ASK

,,,,FID,3430,,NO_ORD,2,

,,,,FID,4356,,ACC_SIZE,736,

,,,,FID,6527,,LV_TIM_MS,32400031,

,,,,FID,6529,,LV_DATE,2016-09-29,

,,,,FID,3886,,ORDER_TONE,2,

Raw data requires more involved code to manage the data, which requires caching to build a complete view of the order book.

The NormalizedLL2 view, which is illustrated in the rest of this tutorial below, is the most common and easy to use view, because it does not use raw data. With this view you can select the main output fields from this list: "Ask Price", "Ask Size", "Bid Price", "Bid Size", "Number of Buyers", "Number of Sellers". You can also define how many levels of depth you require. The maximum available level depends on the exchange, so you might receive less than what you requested; many exchanges deliver 10 levels, none go beyond 25.

Note that apart from the maximum depth level, the availability of data depends on exchanges. For coverage information, refer to the Market Depth sheet in the latest Data Coverage Guide, which you can find in the documentation under section Data.

The API Reference Tree lists the API call options; go here and in the drop down for the 1st parameter (ExtractionRequest) select value TickHistoryMarketDepthExtractionRequest.

 

Get available field list for market depth data - HTTP request

Field selection only applies to a request that uses the NormalizedLL2 view.

This step is optional. If you do not know what content field names are available, you can request a list of those available.

URL:

The available field set depends on the type of data you want to request. The data type must therefore be specified in the request, which is done by setting a report template type in the URL. As we want market depth data we set a value of TickHistoryMarketDepth. The preceding and following tutorials show other possibilities.

 

    	
            https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/GetValidContentFieldTypes(ReportTemplateType=DataScope.Select.Api.Extractions.ReportTemplates.ReportTemplateTypes'TickHistoryMarketDepth')
        
        
    

Method:          GET

Headers:

Note: for all requests we need a user token, set in the header. The token was retrieved in Tutorial 1.

    	
            

Prefer: respond-async

Content-Type: application/json

Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

 

Get available field list for market depth data - HTTP response

If the token is valid, this is the response we get:

Status:                        200 OK

Relevant headers:

    	
            Content-Type: application/json; charset=utf-8
        
        
    

Body:

Here is the beginning of the response:

    	
            

{

"@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#ContentFieldTypes",

  "value": [

    {

      "Code": "THM.Ask Price",

      "Name": "Ask Price",

      "Description": "Best Ask Price at level x",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "THM.Ask Size",

      "Name": "Ask Size",

      "Description": "Total size of all market makers at Ask level x",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "THM.Bid Price",

      "Name": "Bid Price",

      "Description": "Best Bid Price at level x",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "THM.Bid Size",

      "Name": "Bid Size",

      "Description": "Total size of all market makers at Bid level x",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "THM.Domain",

      "Name": "Domain",

      "Description": "Internal field to support instrument preview",

      "FormatType": "Text",

      "FieldGroup": " "

    },

This goes on with all the other available fields. Here is the last part:

    	
            

    {

      "Code": "THM.Number of Buyers",

      "Name": "Number of Buyers",

      "Description": "Number of market makers at Bid level x",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "THM.Number of Sellers",

      "Name": "Number of Sellers",

      "Description": "Number of market makers at Ask level x",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "THM.Sample Data",

      "Name": "Sample Data",

      "Description": "Sample data. For internal use only.",

      "FormatType": "Text",

      "FieldGroup": " "

    }

  ]

}

The result contains the field code, name, a description, field type (number, text, date) and group.

Choose the field names you want. In the next step we will make a request for market depth data, using these data fields.

 

Request market depth data - HTTP request

This is an On Demand extraction request.

URL:

    	
            https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRaw
        
        
    

Method:          POST

Headers:

    	
            

Prefer: respond-async

Content-Type: application/json

Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

Body:

The body of the request must mention it is an extraction request. It contains several parts:

  • The type of extraction: as we want market depth data we set a value of TickHistoryMarketDepthExtractionRequest. The preceding and following tutorials show other possibilities.
  • The list of field names: these were determined in the previous step of this tutorial. This parameter only applies to a request using view NormalizedLL2.
  • The list of instrument identifiers, each one with its type. Below we define one instrument using a RIC.
  • The conditions: they include among others the date range for the request. One parameter should only be included if the request uses the NormalizedLL2 view: the number of depth levels. This defines an upper limit on what you want to receive. Note that the number of available depth levels depends on the exchange. It is often 10, the maximum being 25. If the number you set is higher than what can be delivered by a specific exchange, you will receive empty fields for the levels that are not available.
    	
            

{

  "ExtractionRequest": {

    "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryMarketDepthExtractionRequest",

    "ContentFieldNames": [

      "Ask Price",

      "Ask Size",

      "Bid Price",

      "Bid Size",

      "Domain",

      "History End",

      "History Start",

      "Instrument ID",

      "Instrument ID Type",

      "Number of Buyers",

      "Number of Sellers",

      "Sample Data"

    ],

    "IdentifierList": {

      "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  

      "InstrumentIdentifiers": [

        { "Identifier": "CARR.PA", "IdentifierType": "Ric" }

      ]

    },

    "Condition": {

      "View": "NormalizedLL2",

      "NumberOfLevels": 10,

      "MessageTimeStampIn": "GmtUtc",

      "ReportDateRangeType": "Range",

      "QueryStartDate": "2016-09-29T09:00:00.000Z",

      "QueryEndDate": "2016-09-29T12:00:00.000Z",

      "DisplaySourceRIC": true

    }

  }

}

 

Request market depth data - HTTP response

The possible responses are described in detail the REST API tutorial 3.

In a nutshell, the most likely outcome is a response with an HTTP status of 202 Accepted, and the header will contain a location URL. The next step is to check the request status by polling the location URL regularly until it returns a 200 OK.

If the request is for a small amount of data, the response could have an HTTP status of 200 OK, and the body will contain a jobId and Notes. If the Notes contain the message: Processing completed successfully, we can proceed further. We skip the step where we check the request status, and go directly to the last step, which is to retrieve the data using the jobId.

Other HTTP status codes can be encountered, follow this link for a full list with detailed explanations. It is strongly recommended that your code handle all possible status codes.

 

Check request status - HTTP request

Skip this step if the previous step returned an HTTP status of 200 OK.

If the previous step returned an HTTP status of 202 Accepted, this step must be executed, and repeated in a polling loop until it returns an HTTP status of 200 OK.

URL:

This is the location URL, taken from the 202 response header received in the previous step.

    	
            Location: https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x057c17fba31b2f86')
        
        
    

Method:          GET

Headers:

    	
            

Prefer: respond-async

Content-Type: application/json

Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

 

Check request status - HTTP response

If you receive an HTTP status 202 Accepted response (the same as in the previous step), it means the request has not yet completed. You must wait a bit and check the request status again.

If you receive an HTTP status 200 OK response, the body will contain a jobId and Notes. If the Notes contain the message: Processing completed successfully, we can proceed further. We go to the last step, which is to retrieve the data using the jobId.

 

Retrieve data - HTTP request

It is mandatory to have received a 200 OK response with a JobID from a previous step before proceeding with this last step.

URL:

Note the jobId, used as parameter in the URL.

    	
            https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/RawExtractionResults('0x057c17fba31b2f86')/$value
        
        
    

Method:          GET

Headers:

    	
            

Prefer: respond-async

Content-Type: Accept-Encoding: gzip, deflate

Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

 

Retrieve data - HTTP response

We should get a response of this type:

Status:                        200 OK

Relevant headers:

    	
            

Content-Encoding: gzip

Content-Type: text/plain

Body:

Here is the beginning of the response content, which for the above query contains more than 90000 lines:

    	
            

#RIC,Domain,Date-Time,GMT Offset,Type,L1-BidPrice,L1-BidSize,L1-BuyNo,L1-AskPrice,L1-AskSize,L1-SellNo,L2-BidPrice,L2-BidSize,L2-BuyNo,L2-AskPrice,L2-AskSize,L2-SellNo,L3-BidPrice,L3-BidSize,L3-BuyNo,L3-AskPrice,L3-AskSize,L3-SellNo,L4-BidPrice,L4-BidSize,L4-BuyNo,L4-AskPrice,L4-AskSize,L4-SellNo,L5-BidPrice,L5-BidSize,L5-BuyNo,L5-AskPrice,L5-AskSize,L5-SellNo,L6-BidPrice,L6-BidSize,L6-BuyNo,L6-AskPrice,L6-AskSize,L6-SellNo,L7-BidPrice,L7-BidSize,L7-BuyNo,L7-AskPrice,L7-AskSize,L7-SellNo,L8-BidPrice,L8-BidSize,L8-BuyNo,L8-AskPrice,L8-AskSize,L8-SellNo,L9-BidPrice,L9-BidSize,L9-BuyNo,L9-AskPrice,L9-AskSize,L9-SellNo,L10-BidPrice,L10-BidSize,L10-BuyNo,L10-AskPrice,L10-AskSize,L10-SellNo

CARR.PA,Market Price,2016-09-29T07:00:00.051359443Z,+2,Normalized LL2,,8363,12,,10294,15,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6404,5,23.1,75,2,23.22,528,2,23.16,675,1,23.21,367,1,23.2,2397,2

CARR.PA,Market Price,2016-09-29T07:00:00.523391632Z,+2,Normalized LL2,,8363,12,,10362,16,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6404,5,23.1,75,2,23.22,528,2,23.16,675,1,23.21,367,1,23.2,2397,2

CARR.PA,Market Price,2016-09-29T07:00:02.611495107Z,+2,Normalized LL2,,8363,12,,10362,16,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6404,5,23.1,75,2,23.22,160,1,23.16,675,1,23.21,367,1,23.2,2397,2

CARR.PA,Market Price,2016-09-29T07:00:03.339541963Z,+2,Normalized LL2,,9319,13,,10362,16,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6404,5,23.1,75,2,23.22,160,1,23.16,675,1,23.21,367,1,23.2,2397,2

CARR.PA,Market Price,2016-09-29T07:00:03.419546754Z,+2,Normalized LL2,,9319,13,,7840,15,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6404,5,23.1,75,2,23.22,160,1,23.16,675,1,23.21,367,1,23.2,2397,2

CARR.PA,Market Price,2016-09-29T07:00:03.419546754Z,+2,Normalized LL2,,9319,13,,12567,16,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6404,5,23.1,75,2,23.22,160,1,23.16,675,1,23.21,367,1,23.2,2397,2

CARR.PA,Market Price,2016-09-29T07:00:04.203596232Z,+2,Normalized LL2,,9319,13,,12567,16,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6985,6,23.1,75,2,23.22,160,1,23.16,675,1,23.21,367,1,23.2,2397,2

CARR.PA,Market Price,2016-09-29T07:00:06.743755267Z,+2,Normalized LL2,,11278,14,,12567,16,24.42,949,2,20.85,1593,4,24.19,130,1,21.66,644,2,24.08,231,1,22.56,566,1,23.8,1087,1,22.57,483,1,23.27,1634,1,22.68,6,1,23.26,3040,1,23.09,1900,1,23.25,6985,6,23.1,75,2,23.22,160,1,23.16,675,1,23.21,367,1,23.2,2397,2

Here is the end of the response content:

    	
            

CARR.PA,Market Price,2016-09-29T09:59:59.786673497Z,+2,Normalized LL2,23.305,287,1,23.31,1550,5,23.3,1442,7,23.315,2859,9,23.295,3241,13,23.32,3277,12,23.29,3271,12,23.325,3517,13,23.285,4355,11,23.33,4403,12,23.28,3975,12,23.335,3257,10,23.275,1755,8,23.34,3413,10,23.27,11535,9,23.345,4210,13,23.265,1479,7,23.35,13383,11,23.26,3454,7,23.355,1739,8

CARR.PA,Market Price,2016-09-29T09:59:59.786673497Z,+2,Normalized LL2,23.305,287,1,23.31,1850,6,23.3,1442,7,23.315,2859,9,23.295,3241,13,23.32,3277,12,23.29,3271,12,23.325,3517,13,23.285,4355,11,23.33,4403,12,23.28,3975,12,23.335,3257,10,23.275,1755,8,23.34,3413,10,23.27,11535,9,23.345,4210,13,23.265,1479,7,23.35,13383,11,23.26,3454,7,23.355,1739,8

CARR.PA,Market Price,2016-09-29T09:59:59.786673497Z,+2,Normalized LL2,23.305,287,1,23.31,2182,7,23.3,1442,7,23.315,2859,9,23.295,3241,13,23.32,3277,12,23.29,3271,12,23.325,3517,13,23.285,4355,11,23.33,4403,12,23.28,3975,12,23.335,3257,10,23.275,1755,8,23.34,3413,10,23.27,11535,9,23.345,4210,13,23.265,1479,7,23.35,13383,11,23.26,3454,7,23.355,1739,8

CARR.PA,Market Price,2016-09-29T09:59:59.786673497Z,+2,Normalized LL2,23.305,287,1,23.31,2378,8,23.3,1442,7,23.315,2859,9,23.295,3241,13,23.32,3277,12,23.29,3271,12,23.325,3517,13,23.285,4355,11,23.33,4403,12,23.28,3975,12,23.335,3257,10,23.275,1755,8,23.34,3413,10,23.27,11535,9,23.345,4210,13,23.265,1479,7,23.35,13383,11,23.26,3454,7,23.355,1739,8

CARR.PA,Market Price,2016-09-29T09:59:59.786673497Z,+2,Normalized LL2,23.305,287,1,23.31,2378,8,23.3,1192,6,23.315,2859,9,23.295,3241,13,23.32,3277,12,23.29,3271,12,23.325,3517,13,23.285,4355,11,23.33,4403,12,23.28,3975,12,23.335,3257,10,23.275,1755,8,23.34,3413,10,23.27,11535,9,23.345,4210,13,23.265,1479,7,23.35,13383,11,23.26,3454,7,23.355,1739,8