| Last update | Nov 2023 |
| Environment | Any |
| Language | Any HTTP is supported |
| Compilers | None |
| Prerequisites | DSS login, internet access |
| Source code | Below |
Tutorial purpose
This tutorial explains how to:
- Retrieve the available field list from the DSS server. This is only required if you do not know the exact names of the data fields you want to retrieve.
- Retrieve EoD (End of Day) data from the DSS server, using an on demand request.
- Handle the case where the request times out before results are available.
Table of contents
Get available field list for End of Day - HTTP request
If you do not know what field lists are available, you can request a list of those available.
The report template type must be specified in the path. As we want End of Day data we set a value of EndOfDayPricing. The following tutorials show other possibilities.
Note: for all requests we need a user token. This was retrieved in Tutorial 1.
URL:
https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/GetValidContentFieldTypes(ReportTemplateType=DataScope.Select.Api.Extractions.ReportTemplates.ReportTemplateTypes'EndOfDayPricing')
Method: GET
Headers:
Prefer: respond-async
Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061
Content-Type: application/json; charset=utf-8
Body:
There are more than 700 values in the response. Here is the beginning of the response:
{
"@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#ContentFieldTypes",
"value": [
{
"Code": "EOD.3 Month High",
"Name": "3 Month High",
"Description": "Instrument's highest price over the last 3 months",
"FormatType": "Number",
"FieldGroup": " "
},
{
"Code": "EOD.3 Month Low",
"Name": "3 Month Low",
"Description": "Instrument's lowest price over the last 3 months",
"FormatType": "Number",
"FieldGroup": " "
},
{
"Code": "EOD.3 Month Percent Change",
"Name": "3 Month Percent Change",
"Description": "Percentage change in price over the last 3 months",
"FormatType": "Number",
"FieldGroup": " "
},
{
"Code": "EOD.52 Week High",
"Name": "52 Week High",
"Description": "Fund's 52-week high price represented in the fund's default currency",
"FormatType": "Number",
"FieldGroup": " "
},
{
"Code": "EOD.52 Week Low",
"Name": "52 Week Low",
"Description": "Fund's 52-week low price represented in the fund's default currency",
"FormatType": "Number",
"FieldGroup": " "
},
This goes on with all the other available fields. Here is the last part:
{
"Code": "EOD.Z Spread",
"Name": "Z Spread",
"Description": "Constant spread in basis points that will make the price equal to the present value of the cash flows when added to each relevant point of the yield curve",
"FormatType": "Number",
"FieldGroup": " "
},
{
"Code": "EOD.Zero Curve",
"Name": "Zero Curve",
"Description": "Name of Zero Curve used in OTC Pricing",
"FormatType": "Text",
"FieldGroup": " "
},
{
"Code": "EOD.ZPage",
"Name": "ZPage",
"Description": "Unique system-assigned four-digit alphanumeric identifier for Eurobonds",
"FormatType": "Text",
"FieldGroup": " "
}
]
}
The result contains the field code, name, a description, field type (number, text, date) and group. Use this to choose the field names you want. In the next step we will make a request for data, using some data fields we chose.
Get End of Day data - HTTP request
The body of the request must mention it is an extraction request. It contains several parts:
- The type of extraction: as we want End of Day data we set a value of EndOfDayPricingExtractionRequest. The following tutorials show other possibilities.
- The list of field names: these were determined in the previous step of this tutorial.
- The list of instrument identifiers, each one with its type (Cusip, Isin, Ric, etc., more than 30 are available). In the example below we define one instrument using a CUSIP code (Committee on Uniform Securities Identification Procedures, an identification system for north American and Canadian stocks) and another one using a RIC. More information on them can be found by browsing in the API Reference Tree or searching in the help in the DSS web GUI:
- The conditions.
Notes:
- Timeouts: any request can timeout. This depends on the amount of requested data and the DSS servers load, and the nature of the data (intraday data can be embargoed, more on this topic in the next tutorial). A timeout can also be manually set for the request, using the wait parameter in the Prefer header. The example below sets a timeout of 5 seconds. The default is 30 seconds. Apart for testing purposes, it is very strongly recommended not to set this manually !
- Whenever a request times out, the DSS server will answer with a status 202 Accepted, and deliver a URL that can be used to retrieve the requested data later on. The format of that request will be different, see the last part of this tutorial. The mechanism is allways the same, so what you learn in this tutorial will apply to all the other tutorials, and to any data request where a timeout occurs.
- For all requests we need a user token. This was retrieved in Tutorial 1.
URL:
https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractWithNotes
Method: POST
Headers:
Prefer: respond-async, wait=5
Content-Type: application/json
Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061
Body:
{
"ExtractionRequest": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.EndOfDayPricingExtractionRequest",
"ContentFieldNames": [
"Ask Price",
"Asset Category",
"Asset Category Description",
"Asset ID",
"Asset Status",
"Asset Status Description",
"Asset SubType",
"Asset SubType Description",
"Asset Type",
"Asset Type Description",
"Bid Price",
"Block Trades",
"CIN Code",
"Common Code",
"Currency Code",
"Currency Code Description",
"Currency Code Scaled",
"Currency Code Scaled Description",
"CUSIP",
"Exchange Code",
"Exchange Description",
"Exercise Style",
"Expiration Date",
"File Code",
"High Price",
"Instrument ID",
"Instrument ID Type",
"ISIN",
"Issuer OrgID",
"Lot Size",
"Low Price",
"Market MIC",
"MIC",
"Mid Price",
"Net Asset Value",
"Number of Price Moves",
"Offer Price",
"Official Close Price",
"Open Price",
"Previous Close Price",
"Put Call Indicator",
"Quote ID",
"RCP ID",
"RIC",
"Security Description",
"SEDOL",
"Settlement Date",
"Settlement Price",
"SICC",
"Sicovam",
"Strike Price",
"Ticker",
"Trade Date",
"Trading Status",
"Trading Symbol",
"Turnover",
"Underlying RIC",
"Universal Ask Price",
"Universal Bid Ask Date",
"Universal Bid Price",
"Universal Close Price",
"Usage Instrument SubType",
"Usage Instrument Type",
"Valoren",
"Volume",
"VWAP Price",
"Wertpapier"
],
"IdentifierList": {
"@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",
"InstrumentIdentifiers": [
{ "Identifier": "438516AC0", "IdentifierType": "Cusip" },
{ "Identifier": "IBM.N", "IdentifierType": "Ric" }
]
},
"Condition": null
}
}
Content-Type: application/json; charset=utf-8
Body:
{
"@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#DataScope.Select.Api.Extractions.ExtractionRequests.ExtractionResult",
"Contents": [
{
"IdentifierType": "Cusip",
"Identifier": "438516AC0",
"Ask Price": null,
"Asset Category": null,
"Asset Category Description": null,
"Asset ID": "0x0000190000004b52",
"Asset Status": "MAT",
"Asset Status Description": "Expired/Matured",
"Asset SubType": "CIND",
"Asset SubType Description": "Industrials",
"Asset Type": "CORP",
"Asset Type Description": "Corporate",
"Bid Price": null,
"Block Trades": null,
"CIN Code": null,
"Common Code": "038610325",
"Currency Code": "USD",
"Currency Code Description": "U.S. Dollar",
"Currency Code Scaled": "USD",
"Currency Code Scaled Description": "U.S. Dollar",
"CUSIP": "438516AC0",
"Exchange Code": "EJV",
"Exchange Description": "REFINITIV PRICING SERVICE",
"Exercise Style": null,
"Expiration Date": null,
"File Code": null,
"High Price": null,
"Instrument ID": "438516AC0",
"Instrument ID Type": "CSP",
"ISIN": "US438516AC05",
"Issuer OrgID": "18181",
"Lot Size": null,
"Low Price": null,
"Market MIC": null,
"MIC": null,
"Mid Price": null,
"Net Asset Value": null,
"Number of Price Moves": null,
"Offer Price": null,
"Official Close Price": null,
"Open Price": null,
"Previous Close Price": null,
"Put Call Indicator": null,
"Quote ID": "0x000405000002c5c6",
"RCP ID": "300018181",
"RIC": null,
"Security Description": "HON 9.500 06/01/16 MATd",
"SEDOL": null,
"Settlement Date": null,
"Settlement Price": null,
"SICC": null,
"Sicovam": null,
"Strike Price": null,
"Ticker": "HON",
"Trade Date": null,
"Trading Status": 0,
"Trading Symbol": null,
"Turnover": null,
"Underlying RIC": null,
"Universal Ask Price": null,
"Universal Bid Ask Date": null,
"Universal Bid Price": null,
"Universal Close Price": null,
"Usage Instrument SubType": null,
"Usage Instrument Type": "Corporate",
"Valoren": null,
"Volume": null,
"VWAP Price": null,
"Wertpapier": "A0T020"
},
{
"IdentifierType": "Ric",
"Identifier": "IBM.N",
"Ask Price": 139.27,
"Asset Category": "ORD",
"Asset Category Description": "Ordinary",
"Asset ID": "0x0003dd001379d460",
"Asset Status": "ISS",
"Asset Status Description": "Issued",
"Asset SubType": "ODSH",
"Asset SubType Description": "Ordinary shares",
"Asset Type": "EQTY",
"Asset Type Description": "Equities",
"Bid Price": 139.26,
"Block Trades": 3,
"CIN Code": null,
"Common Code": null,
"Currency Code": "USD",
"Currency Code Description": "U.S. Dollar",
"Currency Code Scaled": "USD",
"Currency Code Scaled Description": "U.S. Dollar",
"CUSIP": "459200101",
"Exchange Code": "NYS",
"Exchange Description": "New York Stock Exchange",
"Exercise Style": null,
"Expiration Date": null,
"File Code": "77",
"High Price": 139.72,
"Instrument ID": "IBM.N",
"Instrument ID Type": "RIC",
"ISIN": "US4592001014",
"Issuer OrgID": "18228",
"Lot Size": null,
"Low Price": 137.24,
"Market MIC": "XNYS",
"MIC": "XNYS",
"Mid Price": 139.265,
"Net Asset Value": null,
"Number of Price Moves": 7845,
"Offer Price": null,
"Official Close Price": null,
"Open Price": 137.56,
"Previous Close Price": 136.89,
"Put Call Indicator": null,
"Quote ID": "0x0003dc004a024fd8",
"RCP ID": "300018228",
"RIC": "IBM.N",
"Security Description": "INTERNATIONAL BUSINESS MACHINES ORD",
"SEDOL": "2005973",
"Settlement Date": null,
"Settlement Price": null,
"SICC": null,
"Sicovam": null,
"Strike Price": null,
"Ticker": "IBM",
"Trade Date": "2022-05-27",
"Trading Status": 1,
"Trading Symbol": "IBM",
"Turnover": 172950512,
"Underlying RIC": null,
"Universal Ask Price": 139.27,
"Universal Bid Ask Date": "2022-05-27",
"Universal Bid Price": 139.26,
"Universal Close Price": 139.27,
"Usage Instrument SubType": null,
"Usage Instrument Type": "Equities",
"Valoren": "941800",
"Volume": 1244799,
"VWAP Price": 138.9385,
"Wertpapier": "851399"
}
],
"Notes": [
"Extraction Services Version 16.0.43633 (806c08a4ae8f), Built May 9 2022 17:21:13\r\nHoliday Rollover of Universal Close Price waived.\r\nProcessing started at 05/30/2022 08:29:25.\r\nUser ID: 9008895\r\nExtraction ID: 592932800\r\nCorrelation ID: CiD/9008895/AAAAAA.080825b00afdf067/RA/EXT.592932800\r\nSchedule: _OnD_0x080825b00b0df067 (ID = 0x080825b0108df067)\r\nInput List (2 items): _OnD_0x080825b00b0df067 (ID = 080825b00b8df067) Created: 05/30/2022 08:29:23 Last Modified: 05/30/2022 08:29:24\r\nSchedule Time: 05/30/2022 08:29:24\r\nReport Template (73 fields): _OnD_0x080825b00b0df067 (ID = 0x080825b00b1df067) Created: 05/30/2022 08:29:23 Last Modified: 05/30/2022 08:29:23\r\n(CSP,438516AC0,EJV) is inactive.\r\nColumn 'CIN Code' suppressed for lack of 'CIN Code' permission.\r\nProcessing completed successfully at 05/30/2022 08:29:26, taking 0.901 Secs.\r\nExtraction finished at 05/30/2022 07:29:26 UTC, with servers: x08i05, QSDHA1 (0.0 secs), QSHC12 (0.5 secs)\r\nUsage Summary for User 9008895, Client 65508, Template Type EOD Pricing\r\nBase Usage\r\n Instrument Instrument Terms Price\r\n Count Type Subtype Source Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n 1 Corporate N/A N/A\r\n 1 Equities N/A N/A\r\n-------\r\n 2 Total instruments charged.\r\n 0 Instruments with no reported data.\r\n=======\r\n 2 Instruments in the input list.\r\nNo Evaluated Pricing Service complex usage to report -- 2 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",
"Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"
]
}
Note: the first instrument is an expired bond, that is why there is no pricing data.
The extraction notes are very useful. This text file contains details of the extraction, including information on instrument expansion, applied embargoes, warnings, errors, quota messages, etc. It is strongly recommended to analyze this text file to detect issues.
Request accepted, but with a timeout
If the token is valid, but there was a timeout, this is the response we get:
Status: 202 Accepted
Relevant headers:
Location: https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractResult(ExtractionId='0x053b21c6eadb5821')
Body: Response does not contain any data.
You will have to send another HTTP GET request to retrieve the result, using the location URL that was delivered in the HTTP 202 Response header.
Testing tip: to test the case of a timeout, set a timeout of 1 second, this will probably generate a timeout allowing you to test your timeout handling code. Again, do not leave this in production, as it is strongly recommended not to change the default value.
https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractResult(ExtractionId='0x053b21c6eadb5821')
Method: GET
Headers:
Prefer: respond-async
Content-Type: application/json
Authorization: Token F0ABE9A3FFF2E02E10AE2765ED872C59B8CC3B40EBB61B30E295E71DE31C254B8648DB9434C2DF9299FDC668AA123501F322D99D45C8B93438063C912BC936C7B87062B0CF812138863F5D836A7B31A32DCA67EF07B3B50B2FC4978DF6F76784FDF35FCB523A8430DA93613BC5730CDC310D4D241718F9FC3F2E55465A24957CC287BDEC79046B31AD642606275AEAD76318CB221BD843348E1483670DA13968D8A242AAFCF9E13E23240C905AE46DED9EDCA9BB316B4C5C767B18DB2EA7ADD100817ADF059D01394BC6375BECAF6138C25DBA57577F0061
If the data is available, you will get the results. Otherwise you will have to try again later. The more data is requested, the longer it can take for it to be available.
Once you have retrieved the results, they are removed from the server, attempting to retrieve the data a second time will result in an error.