REST API Tutorial 3: On Demand intraday extraction, embargo
Last update Nov  2023
Environment Any
Language Any HTTP is supported
Compilers None
Prerequisites DSS login, internet access
Source code Below

Tutorial purpose

This tutorial explains how to:

  • Retrieve the available field list from the DSS server. This is only required if you do not know the exact names of the data fields you want to retrieve.
  • Retrieve intraday data from the DSS server, using an on demand request. Intraday data is data from the current day.
  • Handle embargoes on intraday datai.e. when you are not permissioned for intraday data. Embargoes are a complex topic. To understand embargoes, and how DSS handles them, study the entire .Net SDK Tutorial 4, in particular (but not only) this section.

Table of contents

Get available field list for intraday - HTTP request

If you do not know what field lists are available, you can request a list of those available.

The report template type must be specified in the path. As we want intraday data we set a value of IntradayPricing. The preceding and following tutorials show other possibilities.

Note: for all requests we need a user token. This was retrieved in Tutorial 1.

URL:  

    	
            
https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/GetValidContentFieldTypes(ReportTemplateType=DataScope.Select.Api.Extractions.ReportTemplates.ReportTemplateTypes'IntradayPricing')

Method:          GET

Headers:

    	
            

Prefer: respond-async

Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

Get available field list for intraday - HTTP response

If the token is valid, this is the response we get:

Status:                        200 OK

Relevant headers:

    	
            
Content-Type: application/json; charset=utf-8

Body:

There are more than 500 values in the response. Here is the beginning of the response:

    	
            

{

  "@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#ContentFieldTypes",

  "value": [

    {

      "Code": "IDP.30-Day Implied Volatility - Call",

      "Name": "30-Day Implied Volatility - Call",

      "Description": "Volatility implied by the market price of a call option contract based on a theoretical pricing model on a 60 days period",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "IDP.30-Day Implied Volatility - Put",

      "Name": "30-Day Implied Volatility - Put",

      "Description": "Volatility implied by the market price of a put option contract based on a theoretical pricing model on a 60 days period",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "IDP.60-Day Implied Volatility - Call",

      "Name": "60-Day Implied Volatility - Call",

      "Description": "Volatility implied by the market price of a call option contract based on a theoretical pricing model on a 60 days period",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "IDP.60-Day Implied Volatility - Put",

      "Name": "60-Day Implied Volatility - Put",

      "Description": "Volatility implied by the market price of a put option contract based on a theoretical pricing model on a 60 days period",

      "FormatType": "Number",

      "FieldGroup": " "

    },

    {

      "Code": "IDP.90-Day Implied Volatility - Call",

      "Name": "90-Day Implied Volatility - Call",

      "Description": "Volatility implied by the market price of a call option contract based on a theoretical pricing model on a 90 days period",

      "FormatType": "Number",

      "FieldGroup": " "

    },

This goes on with all other available fields. Here is the last part:

    	
            

        {

            "Code": "IDP.Yield Net Change",

            "Name": "Yield Net Change",

            "Description": "Net change of the current real-time yield from the historic closing yield.",

            "FormatType": "Number",

            "FieldGroup": " "

        },

        {

            "Code": "IDP.Yield to Maturity",

            "Name": "Yield to Maturity",

            "Description": "Yield taking into account the price discount or premium over face value.",

            "FormatType": "Number",

            "FieldGroup": " "

        },

        {

            "Code": "IDP.Yield Type",

            "Name": "Yield Type",

            "Description": "Yield type field describing the type of yields held in RT_YIELD_n or SEC_YLD_n",

            "FormatType": "Text",

            "FieldGroup": " "

        },

        {

            "Code": "IDP.Z Spread",

            "Name": "Z Spread",

            "Description": "Constant spread in basis points that will make the price equal to the present value of the cash flows when added to each relevant point of the yield curve.",

            "FormatType": "Number",

            "FieldGroup": " "

        }

    ]

}

The result contains the field code, name, a description, field type (number, text, date) and group.  Use this to choose the field names you want. In the next step we will make a request for data, using some data fields we chose.

Get intraday data - HTTP request

Like the previous tutorial, this is an On Demand extraction request. The path is the same, and the body of the request follows the same structure.

The body of the request must mention it is an extraction request. It contains several parts:

  • The type of extraction: as we want intraday data we set a value of IntradayPricingExtractionRequest. The preceding and following tutorials show other possibilities.
  • The list of field names: these were determined in the previous step of this tutorial.
  • The list of instrument identifiers, each one with its type. Similar to the previous tutorial, below we define one instrument using a CUSIP code and another one using a RIC.
  • The conditions: in this example we select ScalableCurrency.

Note: for all requests we need to include a user token in the header of our request. This was retrieved in Tutorial 1.

​URL:   

    	
            
https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractWithNotes

Method:          POST

Headers:

    	
            

Prefer: respond-async

Content-Type: application/json

Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

Body:

    	
            

{

  "ExtractionRequest": {

    "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest",

    "ContentFieldNames": [

      "RIC",

      "Ask Price",

      "Asset Type",

      "Bid Price",

      "Currency Code",

      "Exchange Code",

      "High Price",

      "Instrument ID",

      "Instrument ID Type",

      "Low Price",

      "Open Price",

      "Previous Close Date",

      "Previous Close Price",

      "Security Description",

      "Settlement Price",

      "Trade Date",

      "User Defined Identifier",

      "Volume"

    ],

    "IdentifierList": {

      "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  

      "InstrumentIdentifiers": [

      { "Identifier": "438516AC0", "IdentifierType": "Cusip" },

      { "Identifier": "IBM.N", "IdentifierType": "Ric" }

      ]

    },

    "Condition": { "ScalableCurrency": true }

  }

}

Get intraday data - HTTP response

No embargo

If the token is valid, if there is no timeout, and we are permissioned for intraday data for all instrument identifiers in the list, the data will be delivered.

This is the response we get:

Status:                        200 OK

Relevant headers:

    	
            
Content-Type: application/json; charset=utf-8

Body:

    	
            

{

    "@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#DataScope.Select.Api.Extractions.ExtractionRequests.ExtractionResult",

    "Contents": [

        {

            "IdentifierType": "Cusip",

            "Identifier": "438516AC0",

            "RIC": null,

            "Ask Price": null,

            "Asset Type": "CORP",

            "Bid Price": null,

            "Currency Code": "USD",

            "Exchange Code": "EJV",

            "High Price": null,

            "Instrument ID": "438516AC0",

            "Instrument ID Type": "CSP",

            "Low Price": null,

            "Open Price": null,

            "Previous Close Date": null,

            "Previous Close Price": null,

            "Security Description": "HON    9.500 06/01/16 MATd",

            "Settlement Price": null,

            "Trade Date": null,

            "Volume": null

        },

        {

            "IdentifierType": "Ric",

            "Identifier": "IBM.N",

            "RIC": "IBM.N",

            "Ask Price": 0,

            "Asset Type": "EQTY",

            "Bid Price": 0,

            "Currency Code": "USD",

            "Exchange Code": "NYS",

            "High Price": 139.72,

            "Instrument ID": "IBM.N",

            "Instrument ID Type": "RIC",

            "Low Price": 137.24,

            "Open Price": 137.56,

            "Previous Close Date": "2022-05-26",

            "Previous Close Price": 136.89,

            "Security Description": "INTERNATIONAL BUSINESS MACHINES ORD",

            "Settlement Price": null,

            "Trade Date": "2022-05-27",

            "Volume": 1244799

        }

    ],

    "Notes": [

        "Extraction Services Version 16.0.43633 (806c08a4ae8f), Built May  9 2022 17:21:07\r\nProcessing started at 05/30/2022 08:34:07.\r\nUser ID: 9008895\r\nExtraction ID: 2000000402708670\r\nCorrelation ID: CiD/9008895/AAAAAA.08083386c0ddf0a5/RA\r\nSchedule: 0x08083386c0edf0a5 (ID = 0x0000000000000000)\r\nInput List (2 items):  (ID = 0x08083386c0edf0a5) Created: 05/30/2022 08:34:07 Last Modified: 05/30/2022 08:34:07\r\nReport Template (23 fields): _OnD_0x08083386c0edf0a5 (ID = 0x08083386c0fdf0a5) Created: 05/30/2022 08:34:03 Last Modified: 05/30/2022 08:34:03\r\nSchedule dispatched via message queue (0x08083386c0edf0a5)\r\nSchedule Time: 05/30/2022 08:34:05\r\nWARNING: No Pricing for instrument (CSP,438516AC0,EJV), segment 'G' due to instrument not being traded.\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 05/30/2022 08:34:07, it was scheduled to snap at 05/30/2022 08:34:05.\r\nProcessing completed successfully at 05/30/2022 08:34:07, taking 0.457 Secs.\r\nExtraction finished at 05/30/2022 07:34:07 UTC, with servers: x09q27, QSDHA1 (0.0 secs), QSHC18 (0.2 secs)\r\nNo embargo required for this report.\r\nUsage Summary for User 9008895, Client 65508, Template Type Intraday Pricing\r\nBase Usage\r\n        Instrument                          Instrument                   Terms          Price\r\n  Count Type                                Subtype                      Source         Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n      1 Corporate                                                        N/A            N/A\r\n      1 Equities                                                         N/A            N/A\r\n-------\r\n      2 Total instruments charged.\r\n      0 Instruments with no reported data.\r\n=======\r\n      2 Instruments in the input list.\r\nNo Evaluated Pricing Service complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",

        "Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"

    ]

}

Notes on null prices:

  • The first instrument is an expired bond, that is why there is no pricing data.
  • Usually one makes intraday requests during market open. But if an intraday request is done outside of market open, then some prices will be null.

 

Embargo

The permissioning of the account used to create this tutorial does not include real-time for the German exchange Xetra, so by adding to our request an instrument from that exchange (like ALVG.DE) we were able to generate an embargo.

In the case where an embargo applies to at least one instrument identifier, this is the response we get:

Status:                        202 Accepted

Relevant headers:

    	
            
Location: https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractResult(ExtractionId='0x053b22a3986b5821')

Body:                          Response does not contain any data.

You must wait at least for the duration of the embargo before sending another HTTP GET request to retrieve the result, using the location URL that was delivered in the HTTP 202 Response header. In this example the embargo duration is 15 minutes for Xetra, but this value could be different, depending on which exchange(s) were embargoed.

Note that, as of version 13.0, it is possible to limit an extraction to only non-embargoed data. To exclude embargoed instruments from the resulting extraction, use the following condition:

    	
            
"OnlyNonEmbargoedData": true

Eventual embargoed instruments will be identified in the corresponding notes file.

 

Get intraday data, subsequent request after an embargo - HTTP request

After waiting for the duration of the embargo, we can request our data.

This is how we resend our request. Note the path, taken from the 202 response header:

URL:  

    	
            
https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractResult(ExtractionId='0x053b22a3986b5821')

Method:          GET

Headers:

    	
            

Prefer: respond-async

Content-Type: application/json

Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061

If you send this request before the end of the embargo, the response will be the same as for the original request.

At the end of the embargo period, the data will be made available. You can request the data, you should get the results.

If the request is for a lot of instruments, or the DSS server is under heavy load, the extraction might take longer and you might have to wait a bit more and try again later. The more data is requested, the longer it can take for it to be available.