Refinitiv Data Platform Libraries - An Introduction
Last Update Sep 2021

Table of contents

Summary

The following guide introduces the Refinitiv Data Platform Libraries, a suite of ease-of-use interfaces providing access to streaming and non-streaming data services offered within the Refinitiv Data Platform (RDP).  targeting both financial coders and professional developers, the libraries are designed to serve single user desktop access, direct access to cloud-based services or through deployed streaming services within the enterprise.  Depending on the level of service and data requirements, developers can easily choose how to access data services using the appropriate access channel defined within the library.  The following diagram presents the different channels of access available to the developer:

The RDP Libraries are designed as a set of uniform interfaces, spanning multiple programming languages, allowing the developer a consistent way to access content. Accessing content within the platform is supported through multiple Access Channels such as:

The design of the API is normalized to provide a consistent interface, irregardless how to access content. For example, applications built to access streaming content from the Eikon desktop can seamlessly switch to another supporting access channel (RDP or Refinitiv Real-Time) with minimal code changes. The goal is to provide the developer an intuitive interface that defines consistent APIs through multiple channels when accessing data services within the platform.

The RDP Libraries define a logical stack of interfaces designed to foster the adoption of our platform by both financial coders and professional developers to programmatically access financial content. The set of interfaces have been designed to serve different use cases from simple scripting applications to more complex object-oriented designs for the enterprise. They cover a large range of normal performance use cases.

Note: High performance scenarios – that represent a smaller part of customer use cases – are addressed by the Refinitiv Real-Time SDK (C++, Java) and its APIs (EMA, ETA).

Overview

As the Refinitiv Data Platform matures and grows, it will become increasingly apparent the need to provide seamless access to our financial content both today and years to come. The platform utilizes modern web standards enabling the design of a set of consistent Web interfaces allowing the developer community ease of use libraries to access both streaming (WebSockets and Alerts) and non-streaming (HTTP REST) content.

In this document, we will outline the basic structure and design of the API, where and when to use the stack of available interfaces, and the decision to select the most appropriate interface when accessing data within platform.

The Refinitiv Data Platform (RDP) Libraries

As previous outlined, the developer has the choice to access the platform from specific access channels ranging from the desktop (Eikon), via high-performance on-premise (Refinitiv Real-Time) deployments to cloud-based services (RTO). Depending on the specific access channel, developers should not have to learn and use different APIs. The same API and libraries should work seamlessly across the platform, using the same paradigms, experience the same functional behavior and receive the same data irregardless of how they choose to access content.

Depending on the application goals, what skill level and development experience is available, different types of developers will want to use this API with a large range of different use case scenarios. These scenarios can be as basic as getting the current values of an instrument via a simple function call or can be as sophisticated as being alerted in real-time when values of an instrument change within the market. These libraries will address these various needs by providing several abstraction layers and operation models.

API design - Abstraction Layers

The abstraction layers target different needs as well as different types of developers.  Depending on the skill and comfort level of the developer, the API offers general abstraction (Core & Delivery) layers specific to the delivery mode of the content to more business-specific (Content & Function) layers defining interfaces more suited to the nature of the data resource.  But even the lower layer (Core & Delivery) provides an intuitive and easy to use interface.

All these layers can be used together in the same application.

Core & Delivery

Core & Delivery layer corresponds to the different delivery modes of the Refinitiv Data Platform:

  • Request (HTTP Request/Response)
  • Stream (WebSockets)
  • Queue (Alerts)
  • Files (bulk)

For each of these delivery modes, defined classes can be used to easily retrieve data from the platform in raw JSON format.

The classes defined in the Core & Delivery layer are not dependent on any specific data resource, meaning that you can use these interfaces to explicitly specify your own resource, so long as that resource is available from the selected access channel. For example, when accessing data from Eikon, both stream (WebSockets) and request (HTTP) resources are supported. Whereas, accessing data through deployed streaming services (Refinitiv Real-Time), only stream (WebSockets) resources are available.

Designed as the lowest abstraction layer, the Core & Delivery targets developers who need specific features that are not offered by other higher level abstraction layers (Content & Function). This layer targets the professional developer but can also be used by financial coders with good programming skills.

The following interfaces are offered as part of this layer:

Sessions

Depending on how you intend to access the platform (via Eikon, directly to RTO, or Refinitiv Real-Time), access requires authentication and connection semantics in order to communicate and retrieve content. The RDP Libraries provide this managed access through an interface called a Session. The session is the interface to a specific access channel and is responsible for defining authentication details, managing connection resources, and implementing the necessary protocol to manage the life cycle of the session. The following logical sessions have been defined within the API:

  • DesktopSession (Eikon)
  • PlatformSession (RTO)
  • DeployedPlatformSession (Refinitiv Real-Time)

The sessions correspond to each of the access channels available and provide unique properties to support the specific requirements for that interface. Once the session has been established, developers are free to select the following data services available for each channel:

Stream

The Stream API is the streaming interface delivering real-time content over WebSockets. As part of the Delivery layer, a stream provides a simple, yet flexible design to access streaming content delivered using Event-driven callbacks defined within your application.

A Stream allows you to subscribe to items of the different supporting models (MarketPrice, MarketByPrice, Machine Readable News (MRN), etc) available via a streaming connection to the platform. With the current design, a Stream is meant to be used with the Event Driven operation mode.

Request

The Request API represents the interface to retrieve non-streaming data over HTTP REST. The interface is designed specifically to work with API end-points within the platform offering content such as company and fundamental information, historical and time-series data, News headlines and stories, etc.

Sent either synchronously or asynchronously, the API supports requests to all available HTTP end points within the platform. Responses are received in the JSON format.

Queue

The Queue API defines a mechanism delivering streaming alerts such as News headlines and research data based on a subscription criteria.  The API provides the ability to access realtime alerts via queues available within a specific cloud service. 

Refer to the Message services delivery mechanism in RDP article for a detailed understanding of this capability.

File

The File API is a mechanism to deliver bulk or very large payloads of data defined within files, such as end of day pricing for a whole venue.

Content

The Content layer refers to logical market data objects, largely representing financial items like level 1 market data prices and quotes,  Order Books, News, Historical Pricing, Company Research data and so on. These objects are built on top of the Delivery abstraction layer and provide value-add capabilities to manage and access the content within the interface. For example, the Pricing interface is a thin layer on top of data services offering realtime market data. In addition, interfaces such as Historical Pricing and Environmental & Social Governance (ESG) are available to allow intuitive access to request and process results.  While the native back-end data format supports JSON structured messages, the Content layer conveniently prepares data messages in a user-friendly format specific to the programming language.  For example, Pandas Dataframes for Python, JSON in TypeScript/JavaScript and .NET.

The Content layer can easily be used by both professional developers and financial coders. It provides great flexibility for familiar and commonly used financial data models.

Function

The Function layer defines interfaces more suited to scripting languages such as Python.  The goal of these interfaces is to allow researchers, data scientists or the casual developer to rapidly prototype solutions within interactive environments such as Jupyter Notebooks.   The Function layer provides simplistic data access in a single function call such as retrieving News Headlines and Stories, retrieving historical intraday / interday data etc.  Because this layer is built on top of the Content layer, it will benefit from convenient abstractions such as preparing data formats suitable for the specific programming language.

The Function layer can easily be used by both professional developers and financial coders. It provides simple and familiar access to popular scripting langauges and preparing data results useful for the particlar langauge.

Operation Models

The abstraction layers outlined above provide several operation models that meet the needs of a large range of use cases, from the simplest to the more advanced. These models are synchronous, asynchronous and event-driven.

Synchronous

Method/function calls are blocking. The result related to the call is available as soon as the method/function returns control to the calling thread.  Synchronous calls are extremely useful when prototyping within scripting environments such as Jupyter Notebooks.

Asyncronous

 

Method/function calls are non-blocking and return control to the calling thread almost immediately, before the result of the call is available. The Asynchronous model allows the calling thread to perform other operations in parallel to the task requested by the method/function call. When done with the other operations, the calling thread can wait for the requested result to be available (Eg: async/await paradigm).  Applications that require more sophisticated interfaces, such as GUI-based applications, can benefit from asynchronous calls.

Event-drive

With this model, the application registers callback methods or lambda expressions that are called when the requested results are available. Mostly applicable to streaming data, the Event driven model can be mixed with the Synchronous model and the Asynchronous model.

When mixed with the Synchronous model, some of the callback methods or lambda expressions are called before the method/function returns control to the calling thread. When mixed with the Asynchronous model, callback methods or lambda expressions are generally called after the method/function returns control to the calling thread.  Event-driven applications are quite common when building applications that work with streaming data.

What about threading models?

The threading model (a.k.a. Operation model in the EMA jargon) is imposed by the library and cannot be changed programmatically. That is: events are dispatched by the library. This is a design decision which eliminates the need of dispatch event loops, simplifying some situations where the Synchronous operation model is mixed with the Event driven model.

The drawback of this design decision is that it imposes the responsibility of protecting application resources within multiple threads. The callback methods and lambda expressions are always called from a background thread owned by the library. This means that developers using the Event driven operation model must take care of the thread safety of their callbacks. This is a situation we already have with the Enterprise Message API (EMA) and its default threading model (operation mode) that is the “API operation mode” (background thread). Fortunately, this is less of a problem here because the Event driven model is the more advanced operation model of the library. Developers who are not comfortable with multi-threading can use more basic operation models like the Synchronous model or utilize some of the higher level interfaces offered within the Content and Function layers.

Additional Resources