Tutorial Source Code |
Examples referenced: |
Last Update | July 2021 |
Interpreter | Python 3.7.x or greater |
Pre-requisites | Familiarity with Python and a basic understanding of Jupyter Notebook. If accessing content from:
The majority of the examples are built to work within Jupyter Notebook. Ensure this package is installed. |
Whether accessing from the desktop or directly to the cloud, the Refinitiv Data Libraries provide ease-of-use interfaces to retrieve content defined within standard web-based APIs. Built on top of the request-response paradigm, the Refinitiv Data Platform (RDP) defines a number of discrete services providing well-defined endpoints supporting reference, pricing, and analytics data through its Endpoint Request interfaces.
Using the Refinitiv Data Libraries, the following tutorial will outline sample code that provides access to a number of RDP endpoints, supporting different mechanisms to request for data, whether specifying a basic URL, applying query parameters to a URL, or defining a request body via the HTTP POST. The tutorial will cover multiple data services to demonstrate specific features of the library. The latter part of the tutorial will demonstrate a few examples of the interfaces.
Note: In most cases, users of the Refinitiv Data Libraries will utilize interfaces defined within higher layers, such as those defined within the Content layer to retrieve data within the platform. Higher-level interfaces such as the HistoricalPricing, Search, News understand the nature of the specific services and thus define intuitive methods to not only request for data but to extract details within responses. The purpose of this exercise is to bridge the gap where higher-level interfaces do not support specific endpoints, whether they are new, or just not as popular as others.
At the heart of the Endpoint Request interfaces, is the data endpoint URL. For example, the following data services are represented by their endpoint URLs:
When creating a request to retrieve data from an endpoint, the first step is to define the endpoint URL within the interfaces. For example:
// RDP Symbology lookup endpoint URL
lookup_url = 'https://api.refinitiv.com/discovery/symbology/v1/lookup'
When defining a request, depending on the service, you may need to define additional properties to be included within your request. The libraries support a number of interfaces to define the properties of your request as well as the ability to execute the request to retrieve the content.
Depending on the service and how you choose to prepare your endpoint URL, the interfaces support the ability to specify additional properties to be included within your request. The following request features are available:
The following simple code segments demonstrate the above mechanisms:
# ESG endpoint
esg_url = 'https://api.refinitiv.com/data/environmental-social-governance/v2/views/basic'
# Define a request, specifying the company identifier via a query parameter
request_definition = rd.delivery.endpoint_request.Definition(
url = esg_url,
method = rd.delivery.endpoint_request.RequestMethod.GET,
query_parameters = {"universe": "IBM.N"}
)
Path Properties
# Example request with Path parameter - News Story
# Specify our Endpoint URL
story_url = 'https://api.refinitiv.com/data/news/v1/stories/{storyId}'
# Specify our path parameter value
story_id = "urn:newsml:reuters.com:20210805:nZON003MH6:1"
request_definition = rd.delivery.endpoint_request.Definition(
url = story_url,
method = rd.delivery.endpoint_request.RequestMethod.GET,
path_parameters = {"storyId": story_id}
)
# Example request with Path + Header parameter - News Story
# Specify our Endpoint URL
story_url = 'https://api.refinitiv.com/data/news/v1/stories/{storyId}'
# Specify our path parameter value
story_id = "urn:newsml:reuters.com:20210330:nHKS9WjjsZ:2"
request_definition = rd.delivery.endpoint_request.Definition(
url = story_url,
method = rd.delivery.endpoint_request.RequestMethod.GET,
path_parameters = {"storyId": story_id},
header_parameters = {"accept": "text/html"}
)
# Example request with Body Parameter - Symbology Lookup
lookup_url = 'https://api.refinitiv.com/discovery/symbology/v1/lookup'
request_definition = rd.delivery.endpoint_request.Definition(
url = lookup_url,
method = rd.delivery.endpoint_request.RequestMethod.POST,
body_parameters = {
"from": [
{
"identifierTypes": [
"RIC"
],
"values": [
"MSFT.O",
"IBM.N"
]
}
],
"to": [
{
"identifierTypes": [
"LEI",
"ISIN",
"ExchangeTicker"
]
}
],
"type": "auto"
}
)
The following code segment defines the mechanism to request and process data:
...
# Based on the endpoint request, fetch the data...
response = request_definition.get_data()
if response.data.raw:
display(response.data.raw)
else:
print("Response does not contain raw data")
The response from the platform contains basic details such as the raw data object, in JSON format, as well as the success and details of the HTTP response details.
The above example represents a simple request to retrieve the data synchronously from the platform. While this workflow may be suitable for simple applications or when demonstrating functionality, multi-threaded applications, including those that are GUI/Windows-based, will require asynchronous behavior. For example:
...
request_definition_1 = rd.delivery.endpoint_request.Definition(
url = esg_url,
method = rd.delivery.endpoint_request.RequestMethod.GET,
query_parameters = {"universe": "IBM.N"}
)
request_definition_2 = rd.delivery.endpoint_request.Definition(
url = esg_url,
method = rd.delivery.endpoint_request.RequestMethod.GET,
query_parameters = {"universe": "BT.L"}
)
response = request_definition_1.get_data_async()
tasks = asyncio.gather(
request_definition_1.get_data_async(),
request_definition_2.get_data_async()
)
asyncio.get_event_loop().run_until_complete(tasks)
ibm,bt = tasks._result
When using async calls, we should check the response's is_success flag to ensure it worked. If it failed, the http_status should provide a reason for the failure.
# Check if async requests were successful before displaying
def display_reponse(response):
print(response)
print("\nReponse received for", response.closure)
if response.is_success:
display(response.data.df)
else:
print(response.http_status)
display_reponse(ibm)
display_reponse(bt)
# Basic Endpoint retrieval - Directly code RIC into the request
request_definition = rd.delivery.endpoint_request.Definition(
url = "/data/historical-pricing/v1/views/events/VOD.L"
)
# Get Data and Display
response = request_definition.get_data()
print(response.data.raw[0])
# As above, but limit to 5 datapoints
request_definition = rd.delivery.endpoint_request.Definition(
url = "/data/historical-pricing/v1/views/events/VOD.L",
query_parameters = {"count":"5"}
)
In the second example, we utilize a Query Parameter (count), to limit the number of data points returned. In both examples, we rely on default parameters, specifically the fields returned in the response. The Historical Endpoint service provides a number of parameters, including the specification of fields to be returned. When no fields are specified, all will be returned.
Here is an example of what the output will look - starting with some properties, followed by the data column headings and finally the data points (truncated):
The example displays the raw data from the platform. In this example, we're showing a portion of the data, specifically some metadata and header information related to the fields, plus data arrays (truncated) appearing below.
Note: The specification of the Historical Pricing endpoint hard-codes the company specifier. In reality, users will likely utilize the Path Parameter specification to populate the company identifier, but this was intentionally done to demonstrate flexibility and how basic the syntax is to retrieve content from the platform.
For example, the above example could have also been done as:
# Using the Path Parameter instead
request_definition = rd.delivery.endpoint_request.Definition(
url = "/data/historical-pricing/v1/views/events/{universe}",
method = rd.delivery.endpoint_request.RequestMethod.GET,
path_parameters = {"universe": "VOD.L"}
)
In this example, we'll demonstrate the ability to POST a request by applying a JSON object that outlines parameters required by the Symbology Lookup endpoint defined within RDP. In this first code segment, we'll define the endpoint.
lookup_url = 'https://api.refinitiv.com/discovery/symbology/v1/lookup'
Using our endpoint URL, we create our endpoint definition including the JSON that will be applied to the body of the request. In this specific example, we wish to convert a couple of RICs to multiple identifiers, specifically 'ISIN', 'LEI', and the 'ExchangeTicker'.
# RICs to multiple identifiers
request_definition = rd.delivery.endpoint_request.Definition(
url = lookup_url,
method = rd.delivery.endpoint_request.RequestMethod.POST,
body_parameters = {
"from": [
{
"identifierTypes": [
"RIC"
],
"values": [
"MSFT.O",
"IBM.N"
]
}
],
"to": [
{
"identifierTypes": [
"LEI",
"ISIN",
"ExchangeTicker"
]
}
],
"type": "auto"
}
)
response = request_definition.get_data()
response.data.raw
The raw output from the above looks like:
{'data': [
{'input': [{'value': 'IBM.N', 'identifierType': 'RIC'}],
'output': [
{'value': 'US4592001014', 'identifierType': 'ISIN'},
{'value': 'IBM', 'identifierType': 'ExchangeTicker'},
{'value': 'VGRQXHF3J8VDLUA7XE92', 'identifierType': 'LEI'}
]},
{'input': [{'value': 'MSFT.O', 'identifierType': 'RIC'}],
'output': [{'value': 'MSFT', 'identifierType': 'ExchangeTicker'},
{'value': 'INR2EJN1ERAN0W5ZP974', 'identifierType': 'LEI'},
{'value': 'US5949181045', 'identifierType': 'ISIN'}]}
],
'requestId': '502e90bc-7f49-448c-a3ec-8a38ea1f9611',
'effectiveAt': '2021-08-06T11:27:24.924Z',
'messages': []
}
The details outlined provide a good cross-section of the interfaces available to retrieve data defined within RESTful endpoints available within the data platform. As mentioned in the introduction, the endpoint requests presented here will be extremely useful for those services that are not available within higher layers of the libraries. There are many content-specific interfaces, such as Historical Pricing, Symbology, News, ESG, Search available that will further simplify usage to retrieve data. Refer to the Content tutorials within this API for more details.
Additional Examples
There are several other Endpoint related examples on Github in the Tutorials > Delivery > Endpoint folder - as well as in sub-folders of the higher level Examples folder.