Testing

LSEG Tick History REST API with CURL

Zoya Farberov
Developer Advocate Developer Advocate

Introduction

In this article we will go over testing LSEG Tick History API extraction workflow with curl.  Using on demand TickHistoryRawExtractionRequest as an example, we will:

  1. Authenticate to obtain a valid token
  2. Submit on-demand extraction request
  3. Ascertain that the extraction request processing has been completed
  4. Obtain the extraction result

A specific version of curl is not required to do this type of testing.

Another on-demand request can be substituted in the workflow, in place of TickHistoryRawExtractionRequest, and tested with curl.

Authentication - Obtaining A Valid Token

    	
            curl -i -X POST -k -H "Content-Type: application/json" -H "Prefer: respond-async"  https://selectapi.datascope.refinitiv.com/RestApi/v1/Authentication/RequestToken -H 'X-Direct-Download: True' -x MYVALIDPROXY:MYPROXYPORT -d @creds.txt
        
        
    

with creds.txt file stored in the run directory and containing:

    	
            

{

 "Credentials": { 

      "Username": "MYVALIDDSSUSERID", 

      "Password": "MYVALIDDSSPASSWORD"

 }

}

The result from this call should contain a valid token, that is good for 24 hours of use with other requests, similar to

    	
            

HTTP/1.1 200 OK

Cache-Control: no-cache

Pragma: no-cache

Content-Type: application/json; charset=utf-8

Expires: -1

Server: Microsoft-IIS/10.0

BeginRequestTime: 22:21:25.9504696

BeginRequestDate: 2021-08-23

X-Request-Execution-Correlation-Id: CiD/0/AAAAAA.07ae7673cf8d0f48/RA

X-App-Id: Custom.RestApi

X-App-Version: 15.0.155.64

CPUUtilization: 14.20399

RequestsPerSec: 4.75508

W3WP-PrivateBytes: 7196

Date: Mon, 23 Aug 2021 22:21:25 GMT

Content-Length: 405

 

{"@odata.context":"https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#Edm.String","value":"_YuBtkhCDqU8h3S7amxPa4pDtdLKcuefiYq0ab9s23RPfXmi2c7By2ZZZA1VcnoerhtUfsSQnyBlmAN0sDyB-iXSDxm8rxefnv21c7x_tDncPB_Cnp22SZoLhiXyaqXCXmuZcuA1kX3zsyHTGQo_CM5cDzhWUdpbVeiLKpIsXfrsj47kfCTOem1w_RjXvW2G_fsdot7XDt_-krdXGF0L-mpkLtdPJPXWPoBkoDGwWAhygraUpslQy-QhUucEX5Ra8ov5f7Hhu1ZYq4LhKxHZ51aILypJy3pWefVHXqcp21Ts"}

Submit On-Demand Extraction Request

We use the valid token obtained in the previous step to submit TickHistoryRawExtractionRequest

 

    	
            curl -i -X POST -k -H "Content-Type: application/json" -H "Prefer: respond-async" "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRaw" -H "Authorization: Token MYVALIDTOKEN" -H 'Content-Type: application/json' -H 'X-Direct-Download: True' -x MYVALIDPROXY:MYPROXYPORT -d @requestBody.txt
        
        
    

with file requestBody.txt stored in the run folder and containing:

    	
            

  "ExtractionRequest": { 

    "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.TickHistoryRawExtractionRequest", 

    "IdentifierList": { 

      "@odata.type": "#DataScope.Select.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  

      "InstrumentIdentifiers": [{

        "Identifier": "CARR.PA",

        "IdentifierType": "Ric"

      }]

    },

    "Condition": {

      "MessageTimeStampIn": "GmtUtc",

      "ReportDateRangeType": "Range",

      "QueryStartDate": "2016-09-29T12:00:00.000Z",

      "QueryEndDate": "2016-09-29T12:10:00.000Z",

      "ExtractBy": "Ric",

      "SortBy": "SingleByRic",

      "DomainCode": "MarketPrice",

      "DisplaySourceRIC": true

    }

  }

}

In result we look for two pieces for information, status and ExtractionID:

    	
            

HTTP/1.1 202 Accepted

Cache-Control: no-cache

Pragma: no-cache

Expires: -1

Location: https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x07ae64718b9d0f41')

Server: Microsoft-IIS/10.0

BeginRequestTime: 21:30:25.3627291

BeginRequestDate: 2021-08-23

Status: InProgress

Progress:

X-Request-Execution-Correlation-Id: CiD/19945/AAAAAA.07ae64718bad0f41/RA

X-App-Id: Custom.RestApi

X-App-Version: 15.0.155.64

CPUUtilization: 26.47529

RequestsPerSec: 0

W3WP-PrivateBytes: 7120

Date: Mon, 23 Aug 2021 21:30:55 GMT

Content-Length: 0

The status code of 202 means that request is valid and is accepted for processing, the status code of 200 means that request is valid, and request processing is completed.  Any other status means that request should be examined as it is not valid in some aspect of the other.

The extraction Id will be used in the next steps to retrieve the results, as well as examine the status of the request until it's processing is completed, resulting in status of 200.

 Checking Status of the Extraction Using Extraction Id

This step is required if the result from the previous step is not completed = 200 but is accepted = 202.

Next we use the extraction Id just obtained to check on the status of the request, waiting in between the checks, until the status that is returned is completed = 200  

    	
            curl -i -X GET -k -H "Content-Type: application/json" -H "Prefer: respond-async" "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/ExtractRawResult(ExtractionId='0x07ae64718b9d0f41')" -H "Authorization: Token MYVALIDTOKEN" -H 'Content-Type: application/json' -H 'X-Direct-Download: True' -x MYPROXY:MYPROXYPORT
        
        
    

the result being the status of the request:

    	
            

HTTP/1.1 200 OK

Cache-Control: no-cache

Pragma: no-cache

Content-Type: application/json; charset=utf-8

Expires: -1

Server: Microsoft-IIS/10.0

BeginRequestTime: 21:57:10.4224282

BeginRequestDate: 2021-08-23

X-Request-Execution-Correlation-Id: CiD/19945/AAAAAA.07ae85c312ed0f67/RA

X-App-Id: Custom.RestApi

X-App-Version: 15.0.155.64

CPUUtilization: 17.59716

RequestsPerSec: 7.619745

W3WP-PrivateBytes: 1569704

Date: Mon, 23 Aug 2021 21:57:10 GMT

Content-Length: 1414

Obtaining a Ready Extraction Result

Once the result is ready we can GET the result and store it into a file

    	
            curl -X GET "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/RawExtractionResults('0x07ae64718b9d0f41')/$value" -H "Authorization: Token MYVALIDTOKEN" -H 'Content-Type: application/gzip' -H 'X-Direct-Download: True' -x MYVALIDPROXY:MYPROXYPORT -v -N -o "tst.csv.gz"
        
        
    

We can next unzip the contents of file tst.csv.gz:

    	
            

#RIC,Domain,Date-Time,GMT Offset,Type,MsgClass/FID number,UpdateType/Action,FID Name,FID Value,FID Enum String,PE Code,Template Number,Key/Msg Sequence Number,Alias Underlying RIC,Number of FIDs

CARR.PA,Market Price,2016-09-29T12:00:00.942730204Z,+2,Raw,UPDATE,UNSPECIFIED,,,,8132,,2800,,13

,,,,FID,4981,,TRTN,0.89,

,,,,FID,4982,,TRTN_3MT,5.85,

,,,,FID,5615,,YR_TRTN,-5.23,

,,,,FID,5619,,YTD_TRTN,-7.95,

,,,,FID,9257,,ASY_VA_DT,2016-09-29,

,,,,FID,9959,,TRTN_1W,-0.59,

,,,,FID,9960,,TRTN_1M,4.97,

,,,,FID,9961,,TRTN_2Y,1.34,

,,,,FID,9962,,TRTN_3Y,-0.39,

,,,,FID,9963,,TRTN_4Y,60.61,

,,,,FID,9964,,TRTN_5Y,52.58,

,,,,FID,9965,,MTD_TRTN,2.29,

,,,,FID,9966,,QTD_TRTN,3.29,

CARR.PA,Market Price,2016-09-29T12:00:03.161984291Z,+2,Raw,UPDATE,QUOTE,,,,8132,,2816,,14

,,,,FID,3866,,AUC_BID,,

,,,,FID,3862,,AUC_BIDSIZ,,

,,,,FID,22,,BID,23.24,

,,,,FID,6579,,BID_COND_N,0,

,,,,FID,30,,BIDSIZE,2752,

Another, very popular approach, that is used to test LSEG Tick History and Datascope Select REST APIs is tool Postman.  Detailed info on testing with Postman complete with starter example deck can be found in REST API Tutorials mentioned in References section

References