In order to reproduce the sample below, you will need to have valid credentials for Refinitiv Tick History. The source code in this article was simplified for illustration purposes, you can find the full project here
With the changing technology and world that is becoming more and more automated, there is more competition for cutting-edge technology and an industry-leading compliance and surveillance system which will get you comprehensive data to help your internal teams, front office traders find opportunities. Our Tick History product offering and scenarios will help ensure:
- Trading’s reliability to the data source at 1 single point.
- Provide broad, fast coverage.
- Offer historic data which is a critical reference in the risk management process
This will help you build a more efficient, scalable, and intelligent system to identify trading opportunities and enhance their competitive advantage.
Our tick history offering will provide you with real-time pricing data from more than 500 trading venues and third-party contributors, understand risk positions, ensuring compliance and trade surveillance around data usage.
- Rapid access to the data through integrated platforms of API / UI.
- Easy access to accurate information, from global data exchanges.
- On-demand access to normalized, global, cross-asset data via a single connection
- Head of Digital
- Head of Investment Product
- Buy-side Head of Trading / Trader
- Head of Data Science
- Head of Quantitative Research
- Portfolio Manager
- Head of Commodity Trading
- Head of Trading
The Use Case can be any of the below that suits your portfolio:
- Quantitative research & Back testing.
- Market surveillance.
- Regulatory compliance.
The Refinitiv tick history
Refinitiv Tick History provides superior coverage of complete, timely, and global nanosecond tick data. This enables you to download the data you need when you need it, via a single interface.
Coverage goes back as far as 1996 with a standardized naming convention based on RIC symbology that helps lower your data management cost.
Captured from real-time data feeds, Tick History offers global tick data since 1996 across all asset classes, covering both OTC and exchange-traded instruments from 500+ exchanges and third-party contributor data, the majority available two hours after markets close.
Tick History product provides you with the below templates:
- Pricing Data
- End of Day Pricing
- Tick History
- Time and Sales (L1)
- Market depth
- Intraday summaries (bars)
- Raw (MP, MBO, MBP)
- Reference Data
- Terms and Conditions
- Corporate Actions
- Standard Events
Refinitiv Tick History Custom Extractions - Data Content Guide
This document identifies the data available from Tick History Custom Extractions offering. Fields are organized by report template. It can be downloaded here (login required) at Support materials > User Guides & Manuals > Refinitiv Tick History Custom Extractions - Data Content Guide
- A list of portfolios with instruments / RICs as instrument lists will be created/updated.
- Timeframe/duration of the extractions for historical tick-by-tick data is selected/identified.
- We are going to use a CSV file as input, which will consist of two columns in your instrument list. Nevertheless, historical search options based on the names, etc. can be used by date ranges.
- The “Tick History Time and Sales” template is being used in the extractions which provides the option to customize the fields based on Auction, Correction, Quotes, and Trades.
Sample instrument/file code to extract based on the entire market for HK considered as below:
List of fields used:
- Auction - Ask Price
- Auction - Bid Price
- Auction - Price
- Correction - Ask Price
- Correction - Bid Price
- Correction - Original Price
- Correction - Price
- Quote - Ask Price
- Quote - Bench Price
- Quote - Bid Price
- Quote - Fair Price
- Quote - Far Clearing Price
- Quote - Freight Price
- Quote - Invoice Price
- Quote - Lower Limit Price
- Quote - Mid Price
- Quote - Near Clearing Price
- Quote - Price
- Quote - Reference Price
- Quote - Theoretical Price
- Quote - Theoretical Price Ask
- Quote - Theoretical Price Bid
- Quote - Theoretical Price Mid
- Quote - Upper Limit Price
- Settlement Price - Date
- Settlement Price - Price
- Trade - Ask Price
- Trade - Average Price
- Trade - Bench Price
- Trade - Bid Price
- Trade - Freight Price
- Trade - Indicative Auction Price
- Trade - Lower Limit Price
- Trade - Mid Price
- Trade - Odd-Lot Trade Price
- Trade - Original Price
- Trade - Price
- Trade - Trade Price Currency
- Trade - Upper Limit Price
- Auction - Volume
- Correction - Accumulated Volume
- Correction - Original Accumulated Volume
- Correction - Original Volume
- Correction - Volume
- Quote - Volume
- Trade - Accumulated Volume
- Trade - Advancing Volume
- Trade - Declining Volume
- Trade - Exchange For Physical Volume
- Trade - Exchange For Swaps Volume
- Trade - Fair Value Volume
- Trade - Indicative Auction Volume
- Trade - Odd-Lot Trade Volume
- Trade - Original Volume
- Trade - Total Buy Volume
- Trade - Total Sell Volume
- Trade - Total Volume
- Trade - Unchanged Volume
- Trade - Volume
The Description for these fields can be referenced in the data content guide.
We are going to use the following Tick History templates: Tick History Time and Sales
Establish a connection to Tick History via the API and using the input instrument to extract the fields from Tick History Time and Sales within the Tick History product.
In order to start working with the Refinitiv Tick History (RTH) - REST API, you can download the RTH REST Tutorials Postman collection in the RTH Downloads page, extract it, and import to Postman tool on your local machine, then you can start exploring the REST API with the collection and environment. In this article, we're using the Python code to execute the API call (You can also download the RTH Python Code Samples in the RTH Downloads page as well)
For more information on the overview, quick start guide, documentation, downloads, and tutorials, please have a look at Refinitiv Tick History (RTH) - REST API Page
As a preparation step, we set these parameters and import necessary libraries
- filePath: Location to save the download file
- fileNameRoot: Root of the name for the downloaded file
- CREDENTIALS_FILE: The name of file that contains RTH credential, we're going to read the credential from this file instead of putting it in the code directly
- useAws: Set the this parameter to
- False: to download from RTH servers
- True: to download from Amazon Web Services cloud (recommended, it is faster)
Here's an example of the credential file (credential.ini)
username = YOUR_RTH_USERNAME
password = YOUR_RTH_PASSWORD
Then import necessary libraries
import pandas as pd
Firstly, we need to establish a connection to the Refinitiv Tick History host to get the token using the credential read from the credential file. (The code to read credential from file can be found in the full code provided at the beginning of this article)
#Step 1: token request
requestUrl = "https://selectapi.datascope.refinitiv.com/RestApi/v1/Authentication/RequestToken"
r1 = requests.post(requestUrl, json=requestBody,headers=requestHeaders)
if r1.status_code == 200 :
jsonResponse = json.loads(r1.text.encode('ascii', 'ignore'))
token = jsonResponse["value"]
print ('Authentication token (valid 24 hours):')
print ('An error occurred, error status code: ' + r1.status_code)
Extracting associated instrument
Next, let us use the TickHistoryTimeAndSalesExtractionRequest template to extract instrument data by sending an on demand extraction request using the received token
#Step 2: send an on demand extraction request using the received token
"Authorization": "token " + token
"Auction - Ask Price",
"Auction - Bid Price",
"Auction - Price",
"Correction - Ask Price",
"Correction - Bid Price",
"Correction - Original Price",
"Correction - Price",
"Quote - Ask Price",
"Quote - Bench Price",
"Quote - Bid Price",
"Quote - Fair Price",
"Quote - Far Clearing Price",
"Quote - Freight Price",
"Quote - Invoice Price",
"Quote - Lower Limit Price",
"Quote - Mid Price",
"Quote - Near Clearing Price",
"Quote - Price",
"Quote - Reference Price",
"Quote - Theoretical Price",
"Quote - Theoretical Price Ask",
"Quote - Theoretical Price Bid",
"Quote - Theoretical Price Mid",
"Quote - Upper Limit Price",
"Settlement Price - Date",
"Settlement Price - Price",
"Trade - Ask Price",
"Trade - Average Price",
"Trade - Bench Price",
"Trade - Bid Price",
"Trade - Freight Price",
"Trade - Indicative Auction Price",
"Trade - Lower Limit Price",
"Trade - Mid Price",
"Trade - Odd-Lot Trade Price",
"Trade - Original Price",
"Trade - Price",
"Trade - Trade Price Currency",
"Trade - Upper Limit Price",
"Auction - Volume",
"Correction - Accumulated Volume",
"Correction - Original Accumulated Volume",
"Correction - Original Volume",
"Correction - Volume",
"Quote - Volume",
"Trade - Accumulated Volume",
"Trade - Advancing Volume",
"Trade - Declining Volume",
"Trade - Exchange For Physical Volume",
"Trade - Exchange For Swaps Volume",
"Trade - Fair Value Volume",
"Trade - Indicative Auction Volume",
"Trade - Odd-Lot Trade Volume",
"Trade - Original Volume",
"Trade - Total Buy Volume",
"Trade - Total Sell Volume",
"Trade - Total Volume",
"Trade - Unchanged Volume",
"Trade - Volume"
r2 = requests.post(requestUrl, json=requestBody,headers=requestHeaders)
#Display the HTTP status of the response
#Initial response status (after approximately 30 seconds wait) is usually 202
status_code = r2.status_code
print ("HTTP status of the response: " + str(status_code))
If required, poll the status of the request using the received location URL and once the request has completed, retrieve the jobId and extraction notes.
- As long as the status of the request is 202, the extraction is not finished, we must wait, and poll the status until it is no longer 202
- If status is 202, display the location url we received, and will use to poll the status of the extraction request
- When the status of the request is 200 the extraction is complete, we retrieve and display the jobId and the extraction notes (it is recommended to analyse their content))
- If instead of a status 200 we receive a different status, there was an error
#Step 4: get the extraction results, using the received jobId.
#We also save the compressed data to disk, as a GZIP.
#We only display a few lines of the data.
requestUrl = "https://selectapi.datascope.refinitiv.com/RestApi/v1/Extractions/RawExtractionResults" + "('" + jobId + "')" + "/$value"
#AWS requires an additional header: X-Direct-Download
"Authorization": "token " + token
"Authorization": "token " + token
r4 = requests.get(requestUrl,headers=requestHeaders,stream=True)
#Ensure we do not automatically decompress the data on the fly:
r4.raw.decode_content = False
print ('Content response headers (AWS server): type: ' + r4.headers["Content-Type"] + '\n')
#AWS does not set header Content-Encoding="gzip".
print ('Content response headers (TRTH server): type: ' + r4.headers["Content-Type"] + ' - encoding: ' + r4.headers["Content-Encoding"] + '\n')
#Next 2 lines display some of the compressed data, but if you uncomment them save to file fails
#print ('20 bytes of compressed data:')
print ('Saving compressed data to file:' + fileName + ' ... please be patient')
fileName = filePath + '/' + fileNameRoot + ".csv.gz"
chunk_size = 1024
rr = r4.raw
with open(fileName, 'wb') as fd:
shutil.copyfileobj(rr, fd, chunk_size)
print ('Finished saving compressed data to file:' + fileName + '\n')
#Now let us read and decompress the file we just created.
#For the demo we limit the treatment to a few lines:
maxLines = 10
print ('Read data from file, and decompress at most ' + str(maxLines) + ' lines of it:')
uncompressedData = ""
count = 0
with gzip.open(fileName, 'rb') as fd:
for line in fd:
dataLine = line.decode("utf-8")
#Do something with the data:
uncompressedData = uncompressedData + dataLine
count += 1
if count >= maxLines:
#Note: variable uncompressedData stores all the data.
#This is not a good practice, that can lead to issues with large data sets.
#We only use it here as a convenience for the next step of the demo, to keep the code very simple.
#In production one would handle the data line by line (as we do with the screen display)
Finally, display the results in the console:
#Step 5 (cosmetic): formating the response received in step 4 or 5 using a panda dataframe
from io import StringIO
import pandas as pd
TimeAndSales = pd.read_csv(StringIO(uncompressedData))
Automating similar routines with Tick History is quick and simple, offering you greater flexibility, and is more efficient than the manual use of Tick History via the browser.