Migrating an application from Elektron Connect to Refinitiv Real-Time Optimized
Gurpreet Bal
Platform Application Developer Platform Application Developer

Refinitiv is changing the way our customers can access real-time market data. For the customers who are on Elektron Connect, that legacy product is now at the End Of Life and the strategic replacement is Refinitiv Real-Time – Optimized (RTO), Refinitiv's cloud based conflated real time feed.

To help the customers with this migrations from old product to the new one, we are capturing all the relevant links to the documentation, tutorials, downloads and more here in one page. This includes how to modify/redevelop your application to connect to the new service, how to rebuild using Refinitiv’s latest API’s and how to manage when you are unable to move away from your legacy API code.

Note: This is a living document and will be updated on a regular basis. Please visit often for updated information.

Summary of migration options

The existing Electron Connect service will be end of life in September 2021. The Refinitiv Real-time Optimized (RTO) feed in the cloud, is a strategic replacement and the old applications are expected to migrate to this new service.

All the recent Refinitiv APIs are able to natively take advantage of cloud support and have built in functionality to access and manager cloud credentials. These API's are Refinitiv Real-Time SDK (RTSDK), formerly called Electron Message API (EMA) and Electron Transport API (ETA), and the web-socket API. All flavors and programming language versions of these API's have the cloud support built into them in the latest release.

Older API's, namely Robust Foundation API (RFA), and legacy APIs like SFC/SSL API's do not have this functionality. For the users of these API's, the option is to either rewrite their application using one of the strategic SDK, or use an intermediate product called Legacy Protocol Converter (LPC). LPC is a standalone application, which runs on a Linux system or a virtual machine or a docker image, and appears as a local feed provider to the application. Using this approach involves setting up the LPC application and configuring it to use cloud credentials and the local service and user accounts. LPC therefore becomes the bridge between the user application and the cloud.

Here is the summary of migration options:

Existing application Migration Option 1 (Preferred) Migration Option 2
RFA API Java or C++ Rewrite the application using the latest RTSDK Java or C++ or Websockets API Use LPC
Legacy API like SFC Rewrite the application using the latest RTSDK Java or C++ or Websockets API Use LPC
RFA .NET Migrate the application to use Refinitiv Data Library for .NET, or use Websockets API Use LPC
EMA Java or C++ Modify the application to source data from cloud  

These options are described in detail below -

1. Rewrite the application using the latest RTSDK Java or C++ or Python

For users not wishing to deal with the complexities of installing and maintaining the LPC, and to align their code-base to a recent strategic SDK, it is recommended that they rewrite the application using the latest RT-SDK for Java or C++. Typically, a production application is structured in a way that market data module is an isolated class/classes. It is often easy to just rewrite that module using a different SDK, while keeping the rest of the application.

Some of the advantages of the newer RT-SDK are:

  • Better performance
  • Significantly easier to write code
  • Cloud native support
  • Easier configuration and logging management

Migration steps

  1. Understand your existing application structure. Typically, an application has market data modules in isolated classes. Upon scanning the source code for the legacy application, a developer will try to identify these modules which pertain to the retrieval of market data. This is often much easier, since all legacy API's from Refinitiv were also provided as SDK's and often the user has to search and identify the namespace/classes associated with legacy SDK. Once identified and isolated, it is often easy to rewrite that module using a different SDK, while keeping the rest of the application structure untouched. Following diagram shows this process:

  1. Read a consolidated migration document to understand high level changes in various API capabilities - Choose a real time API

  2. To understand the RT-SDK and get a quick introduction, start with a

  3. Follow up by the step-by-step tutorials.

  4. RT-SDK is open source; to view the source code, visit the repository on the github.

Python and websockets support

Refinitiv Real-time Optimized can also provide data using industry standard websockets protocol. This opens the door to using Python, R, and any other programming language with websockets support. This was not possible with legacy Electron Connect system. If the users deem, that their existing application can take advantage of rapid scripting language like Python, then they can also pick this route for application migration. Here are a few video introductions to the websockets API:

RTO Websockets API Introduction Videos

The following flowchart shows how this service works:

Imp: There are some recent changes in the URL structure of the endpoints provided by service discovery. They are now better aligned with the AWS region. Please read about this change in the Product Notification Change. 

This sequence process is described in the video below along with demonstration:

To learn how to use RTO Cloud with Python, see

2. Modify an existing EMA application

If the current application is using EMA API (Java or C++), then it is trivial to migrate it to the RTO cloud. The latest versions of RTSDK (formerly EMA API) has built in support for RTO cloud connection. The users are encouraged to either -

Easiest mechanism for an EMA application to connect to RTO Cloud, is to employ the configuration to drive the cloud connection. Here is how a typical EMA application connects to the EC:

    	
            
// connect to local EZD which is listening to port 14002


consumer  = EmaFactory.createOmmConsumer(config.host("localhost:14002").username("user"));


ReqMsg reqMsg = EmaFactory.createReqMsg();


// request data from service name: ELEKTRON_DD


consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name("IBM.N"), appClient);

Now, modify the source code, so that now application reads the EMAConfig.xml file. In the configuration file, we define a Consumer block which has following channel entry of connection type RSSL_ENCRYPTED:

    	
            
<Channel>


  <Name value="Channel_4"/>


  <ChannelType value="ChannelType::RSSL_ENCRYPTED"/>


  <CompressionType value="CompressionType::None"/>


  <GuaranteedOutputBuffers value="5000"/>


  <!-- EMA discovers a host and a port from RDP service discovery for the specified location


   when both of them are not set and the session management is enable. -->


  <Location value="us-east"/>


  <EnableSessionManagement value="1"/>


  <ObjectName value=""/>


</Channel>

Now, the source code of EMA application can be modified so that it uses this particular Consumer entry:

    	
            
// the userName/password are the RTO machine ID and the password


consumer  = EmaFactory.createOmmConsumer(config.consumerName("Consumer_4").username(userName).password(password));


ReqMsg reqMsg = EmaFactory.createReqMsg();


// request data from service name: ELEKTRON_DD


consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name("IBM.N"), appClient);

This will instruct the EMA application to automatically use the service discovery and connect to the us-east region.

Above shown changes will suffice for most of the applications migrating from EC to RTO. There may be advanced use cases, in which an application might need to control the connection endpoint programmatically. Please see the Query Service Discovery Sample example on how to do this.

See the EMA Configuration Guide for all configuration options and a complete example of EMAConfig.xml.

3. Migrate the application to use Refinitiv Data library

Refinitiv is working to provide a Refinitiv Data library, which is capable of sourcing data from RTO. This library will be available in three programming languages - Python, .NET and TypeScript. The .NET and TypeScript version of this library is open sourced and community supported. For an existing user of RFA.NET, it might be easier to consider re-writing the application using Refinitiv Data library for .NET. Here are the resources for Refinitiv Data library -

4. Use LPC

The Refinitiv Legacy Protocol Converter (LPC) provides a network connectivity to RTO infrastructures in cloud using near zero footprint datafeed delivery method. Client applications connect to the LPC to route traffic over the Internet. LPC uses the RSSL messaging protocol via TLS over TCP/IP. Cloud-provided data sources require token based authentication for access. LPC will retrieve and manage tokens on behalf of the connecting applications.

LPC has following features and limitations:

  • Source Sink Library (SSL) version 4.0 is supported
  • Supports a watchlist size of up to 50K
  • There can be up to 5 concurrent applications connected to RTO
  • LPC can stream up to 100k RICs
  • Only supported on Linux OS
  • Windows install is supported through Docker toolbox (deprecated)

Following diagram shows how LPC (shown as protocol converter) will fit into the picture and replace Electron Zero Daemon (EZD):

LPC provides support for the existing Refinitiv Wire Format (RWF) and Marketfeed (MF) applications. Please contact your Refinitiv account manager to get access to LPC software. Following resources can help you get started:

Following video guides have been created to help developers visualize the LPC install process.

Operational considerations

While LPC is designed to mimic Electron Connect (EC) in functionality as much as possible, there are a few differences in behavior which an application should be aware of.

Token Refresh Failure

The RTO environment operates on the concept of access and refresh tokens. The access token expire every 300 seconds and need to be refreshed periodically. If a new access token is not provided to an open stream, the remote server can close this stream. This behavior is different from EZD, which did not require recertification of credentials after a period of time. An application user will have to be aware that if there are issues with token refresh, after the initial login, that their application will receive a CLOSE message on the LOGIN domain, and all the subscriptions would stop. These stopped subscriptions will not resume automatically, upon renewal of token.

Different application will react differently to this issue. There are some that might outright terminate, and others which might log an error message in the log files. The application developer should consider how this particular scenario should be handled in an application. It could be a combination of application and/or log monitor, which detects that application is not receiving any updates, or all open subscriptions have been closed. Upon detection of this condition, the application should be terminated and restarted. The future version of LPC (estimated 2022), using the RTO-CIAM certificate based access will not be affected by this issue.

FAQ

Q: Will our application need to be modified to use the LPC or is it a drop in replacement.

A: LPC can be configured to behave like an existing EZD service (with above mentioned exception), so the existing application will not need to change.

Q: Does LPC installation require changes to the network firewall rules.

A: Yes. The LPC will connect to the AWS data center and the host-names will be different from the ones EZD was connecting to. Hence it will require new firewall rules. Refer to LPC installation manual for details.

Q: Is LPC natively supported on Windows OS.

A: No, LPC is supported in Windows through Docker Toolbox and virtualization software like VirtualBox and VMWare only. Users can also run it on a Linux instance in an AWS.

Q: Docker toolbox is deprecated, what are my other options.

A: On a windows machine, you could install a Linux Virtual machine and install LPC on it.

Q: For LPC install on Linux OS, do we need a dedicated Linux machine.

A: No, a Virtual machine can be used as well. This VM can even reside on the same server the application is running from.

Q: Can we have multiple LPC installations.

A: Yes, you can run multiple instances of LPC on multiple docker containers if required.

Q: How does LPC manage which region it can connect to.

A: The LPC configuration file specifies, the local service name and the region it should connect to. In the absence of region, it defaults to us-east.

Q: Can we configure LPC to connect to a particular endpoint.

A: Yes. LPC can be configured to not use service discovery and connect to a particular endpoint.

Q: What monitoring tools are supported with LPC.

A: LPC can be monitored by running LPCmon tool on the Linux instance.

Q: LPC is not listening on the SSL/RSSL port.

A: Check the LPC log files for errors - the most common cause is incorrect user permissions. Error message may look like Scopes required to access the resource: [trapi.streaming.pricing.read]. Missing scopes: [trapi.streaming.pricing.read]","status":"Forbidden"}}. Contact your Refinitiv Account manager to get this fixed.

 

Support: For questions regarding the use of LPC, or to get support with installation, configuration or operation; please contact Refinitiv Support.