Whilst, Refinitiv offers several current APIs and SDKs for working with Real-Time streaming data - many of our users have a considerable deployment of applications developed using our legacy APIs.
For many if not most of these applications, the original developers have moved on and the maintenance of these applications is the responsibility of someone who may have never coded with the underlying API or SDK.
For these applications, there will come a time, where the current maintainers will be asked to rewrite/migrate these legacy API applications to a current Refinitiv API. Based on my experience of working with our users, this is often a cause of considerable anxiety for the developer - particularly if they have never used the legacy APIs and are suddenly faced with the task of learning two APIs - the legacy one and the newer alternative.
However, I have usually found the developer does not need to learn the legacy API in much detail. When helping developers in this situation, I find the following is usually sufficient to enable them to understand the legacy application code enough to migrate it to the newer API.
- Initially, a high-level overview outlining the key features of the legacy API helps the developer put things into context and gain a reasonable understanding of what the application is doing.
- I then follow this up with a walkthrough of key code snippets of an example application or their own codebase - highlighting key items such as configuration, initialisation, item subscription and so on.
In this document, I will attempt to document step 2 of the above. Whilst much of this information can be gleaned from the API's existing documentation, the information below would typically be spread out across several sections and/or documents. I will be focusing on the SFC Java (also sometimes referred to as JSFC) and SFC C++ versions.
If you have not already done so, I would recommend reading the SFC Overview document - a link to which is in the right-hand sidebar.
SFC to EMA Migration Notes
Key Functionality – Refer to Notes for details:
I will begin with a High level view of key features and functionality support across SFC and EMA - so that you can ensure any required SFC functionality or feature is available in EMA.
Feature/Functionality |
Enterprise Message API |
SFC |
Performance |
High |
Intermediate |
Data Format |
OMM/RWF – Binary |
MarketFeed – String based |
Languages |
Java, C++ |
Java, C++, COM |
Native Cloud support |
Yes |
No |
Event-driven |
Yes |
Yes |
Message-Based |
Yes |
No |
Object-based |
No |
Yes |
Configuration |
Programmatic + File based |
Programmatic + File based |
TimeSeries |
No |
Yes |
Full Depth Order Book |
Yes |
No |
Chain handling |
*Helper Class |
Yes |
Ripple Fields |
*No - Manual implementation | Yes |
Snapshot |
Yes |
Yes |
Page based data |
Yes |
Yes |
Custom Domain Models |
Yes |
No |
Views (Field Filtering) |
Yes |
No |
Contributions to RTDS |
Yes |
Yes |
Contribution to Cloud |
Yes |
No |
Thread Safe and Aware |
Yes |
No |
Multi Consumer support |
Yes |
No |
Publisher/Provider |
Interactive + Non-Interactive |
Managed / Unmanaged |
TIC/TIB/SASS support |
No |
Yes |
ANSI Page support |
Yes |
Yes |
Notes:
Performance – EMA offers a considerable improvement in performance over SFC
· optimised API design
· use of Binary Encoded OMM data
· SFC’s internal caching mechanism negatively impacts performance
Data Format – OMM uses the highly optimised binary encoded Refinitiv Wire format and allows richer Data model representation e.g. Full Depth Order book, Yield Curves, and other custom data domains.
Native Cloud Support – EMA can consume data from Cloud or deployed infrastructure. Easily switch between deployed and cloud data source (typically config change / few lines of code)
EMA uses Messages for data requests, responses, and contributions – Message payload can contain data as well as stream state information.
Configuration – EMA defaults most Config parameters to (documented) common values which can then be optionally overridden programmatically. A config file containing multiple named configuration sets (e.g. Dev, QA, Production) can also be used - with the ability to specify a default config set. The developer can choose a non-default config by specifying the name of the required config.
Whilst EMA does not provide a TimeSeries decoder, Refinitiv offers alternative Historical data sets which can be accessed via an easy to use RESTful interface.
Ripple Fields – we provide EMA example code demonstrating how to implement field rippling
Chain handling –a Chain Helper class for EMA Java is available on GitHub.
Page based data – handling Page based data is simpler in SFC, however, there is EMA Page handling guidance on the Developer Portal with example code
Custom Domain Models – using OMM allows the publication and consumption of rich custom data formats within your organisation
Field filtering is possible using the Views feature of EMA – to optimise bandwidth usage and decoding of the payload.
Contribution to Cloud - EMA allows the contribution of data to the cloud based Refinitiv Contribution Channel – eliminating the need for deployed contribution servers/gateways.
Multi Consumer support - EMA allows a single application to easily consume data from multiple servers + sources and to horizontally scale the processing of larger watchlists on multi core CPUs.
Examples of code comparison
In the following section, I will cover a couple of common coding scenarios – comparing C++ and Java versions of SFC with EMA.
SFC C++ - Configuration / Initialisation
For an SFC C++ subscriber application using an SSL type connection you need to perform the minimum configuration to connect to a server:
· ensure the data dictionary files (appendix_a & enumtype.def) are present in the require location (e.g. \var\triarch)
· ensure the ports used by SFC are defined in your OS services file (e.g. \windows\system32\drivers\etc or services/etc) – for most installations, this would be ‘s_name_1 8101/tcp triarch_sink’ i.e. define 8101 with an alias of triarch_sink
· ensure the sslapi.cnf file is populated and present in the required location ((e.g. \var\triarch) containing as a minimum something like:
*ipcRoute: triarch_sink hostname – where hostname is the IP address/name of ADS server.
· An example.cnf file is present in the working directory – with application-specific configuration
In your application code, you then need to typically
· create and populate an RTRXFileDb with the configuration
· apply the configuration parameters
· create an RTRSSLRTRecordService instance
· create an RTRDefaultLogger
· create an instance of a Record callback Client for each instrument (see later for more details)
The above typically requires around 10 lines of code some key lines of which are shown below:
configDb = new RTRXFileDb(configpath);
RTRConfig:: setConfigDb(*configDb);
logger= new RTRDefaultLogger(appId, "logger");
service = new RTRSSLRTRecordService(appId, servicename,"")
RecordUpdateClient client(*service, symbol);
RTRSelectNotifier::run();
EMA C++ configuration/initialisation
At is most basic, for an EMA C++ consumer example the initialisation process is as follows:
AppClient client;
OmmConsumer consumer( OmmConsumerConfig().host( "ads1:14002" ).username( "user" ) );
The above two lines of code pretty much replace all the above SFC configuration file requirements + code (and more) - the 1st line creates an instance of the callback client, and the 2nd line results in the following actions:
· Create an OmmConsumer instance (our main subscription interface with EMA)
· if a config file does NOT exist, create a config using default values (or load the default config settings from the config file if one exists)
· override the default server details with ‘ads1’ on port 14002
· attempt a connection to the server
· send a login request for ‘user’ to the server
· on a successful login response, send a source directory request to the server
· request + download dictionary from the server (or load local if configured in the config file
For further details on EMA Configuration please refer to Enterprise Message API (EMA) - Configuration Overview on the Refinitiv Developer Portal
SFC Java vs EMA Java initialisation
Although class names/methods may vary somewhat, SFC Java (aka JSFC) requires similar initialisation steps as SFC C++ with some minor differences e.g. the need to create a Session and then obtain an RTRecordService from the session.
EMA Java initialisation is very similar to EMA C++:
AppClient appClient = new AppClient();
OmmConsumerConfig config = EmaFactory.createOmmConsumerConfig();
consumer = EmaFactory.createOmmConsumer(config.host("ads1:14002")
.username("user"));
As you will note, the key difference is the use of the EmaFactory to create the OMMConsumerConfig and OmmConsumer instances.
SFC Java - Subscribe to an instrument and display the received data
For SFC, the most basic subscription example requires the following steps to be implemented.
· implement an RTRecordClient(+usually an RTFieldClient) with a constructor, destructor and various event handlers for Status, initial image, stale, not stale, inactive and update complete
· instantiate the above record client providing the name of the single instrument and field(s)
· if you are consuming multiple instruments, you will need to instantiate multiple record clients
The RTRecordClient implementation will need to provide the following:
· a constructor or initialisation method which will
o specify the field names you wish to request
o obtain an RTRecord from the RTRecordService
o specify the RTRecordClient instance as the event handler for the record
o check if the record is active and whether it has data or not
o call the appropriate handlers based on active and data state
For the most basic SFC Java example which displays a single field, the above implementation requires 40-50 lines of code – much of which is shown below:
public void initRecord()
{
String[] fName = { _fieldName };
_record = _service.rtRecord(_symbol, fName);
_record.addClient(this);
if (_record.active())
{
if(!_record.hasData())
processRecordStale(_record);
else
processRecordSync(_record);
}
else
processRecordInactive(_record);
}
public void printQuote()
{
if (_field != null)
System.out.println(_record.symbol() + ": " +
_field.name() + "= " +
_field.string() );
else
System.out.println("Field: " + _fieldName +
" is not available for " +
_record.symbol() );
}
public void processFieldUpdate(RTField field)
{
System.out.print("processFieldUpdate called: ");
printQuote();
}
public void processRecordStale(RTRecord rec)
{
System.out.println("processRecordStale called: stale data");
System.out.println(" Stale: " + rec.text() );
}
public void processRecordSync(RTRecord rec)
{
System.out.print("processRecordSync called: ");
_field = rec.fieldByName( _fieldName );
if (_field != null)
{
_field.addClient(this);
printQuote();
}
}
public void processRecordInactive(RTRecord rec)
{
System.out.println("processRecordInactive called: Record inactiviated!");
System.out.println("!!Inactive: " + rec.text() );
}
public void processRecordNotStale(RTRecord rec)
{
System.out.print("processRecordNotStale called: ");
printQuote();
}
public void processResyncComplete(RTRecord rec)
{
System.out.print("processResyncComplete called: " + rec.text());
_field = rec.fieldByName( _fieldName );
if (_field != null)
{
if (!_field.hasClient(this))
_field.addClient(this);
printQuote();
}
if (!rec.stale())
processRecordNotStale(rec);
}
EMA Java – Subscribe to an instrument and display the received data
Implementing similar basic functionality - displaying all payload fields - in EMA Java can be achieved using the following:
class AppClient implements OmmConsumerClient
{
public void onRefreshMsg(RefreshMsg refreshMsg, OmmConsumerEvent event)
{
System.out.println(refreshMsg);
}
public void onUpdateMsg(UpdateMsg updateMsg, OmmConsumerEvent event)
{
System.out.println(updateMsg);
}
public void onStatusMsg(StatusMsg statusMsg, OmmConsumerEvent event)
{
System.out.println(statusMsg);
}
}
...
// Using the previously created OmmConsumer and AppClient...
ReqMsg reqMsg = EmaFactory.createReqMsg();
consumer.registerClient(reqMsg.serviceName("ELEKTRON_DD").name("IBM.N"), appClient);
Firstly, we implement a basic Client class to handle the events and display any data or status messages we receive.
To make the actual subscription request, we create a Request Msg, set the service name and RIC Code and then register the request with our previously obtained OmmConsumer and pass in a reference to the above callback client.
Note that for more than one record, you do not need to instantiate multiple OmmConsumerClient – unlike the SFC implementation, which requires an RTRecordClient derived instance for each record that you subscribe to. A single OmmConsumerClient derived instance can be used to process multiple records.
SFC C++ vs EMA C++ - Subscribe to an instrument
Although class names/methods may vary slightly, SFC C++ requires very similar code implementation as SFC Java.
EMA C++ instrument subscription and displaying data is very similar to EMA Java:
public void onRefreshMsg(RefreshMsg refreshMsg, OmmConsumerEvent event)
{
cout << refreshMsg << endl; // defaults to refreshMsg.toString()
}
public void onUpdateMsg(UpdateMsg updateMsg, OmmConsumerEvent event)
{
cout << updateMsg << endl; // defaults to updateMsg.toString()
}
void AppClient::onStatusMsg( const StatusMsg& statusMsg, const OmmConsumerEvent& )
{
cout << statusMsg << endl; // defaults to statusMsg.toString()
}
...
// Using the previously created OmmConsumer and AppClient...
consumer.registerClient( ReqMsg().serviceName( "ELEKTRON_DD" ).name( "BT.L" ), client );
Posting / Inserting data using SFC and EMA
Other than subscribing to / consuming data, the other most common use for SFC and EMA is to contribute (sometimes mistakenly referred to as publish) data – either to an internal data source or externally to Refinitiv or other vendors.
From an API perspective, the action required for contributing some data is referred to as sending an Insert (SFC) or a Post (EMA).
Inserting using SFC
Implementing basic Insert functionality in SFC requires, pretty much the same Initialisation and Configuration steps as described earlier for a Subscriber – except that we create an RTRSSLInsertService rather than an RTRSSLRTRecordService.
We also need to provide a client class that implements an RTRInsertServiceClient and RTRInsertClient – with callback handlers for things like Service sync, state, info and Insert succeed and failed.
Looking at the most basic SFC SimpleInsert examples in Java and C++, the source files implementing the above, contain 100+ lines of code - so I won't be pasting that here!
Posting using EMA
The initialisation and configuration steps for Posting with EMA are almost identical to the Consumer example shown earlier e.g. for EMA Java:
AppClient appClient = new AppClient();
consumer = EmaFactory.createOmmConsumer(EmaFactory.createOmmConsumerConfig()
.host("ads1:14002").username("user"));
ReqMsg reqMsg = EmaFactory.createReqMsg();
consumer.registerClient(reqMsg.domainType( EmaRdm.MMT_LOGIN ), appClient, consumer);
The key difference from the Consumer code is that we are registering the callback client for the Login request, so we can post the data for all items on the Login stream – rather than having to establish a stream for each item, we wish to Post data for.
In the RefreshMsg callback handler we then check for a valid Login response and post the data using the Login stream:
public void onRefreshMsg(RefreshMsg refreshMsg, OmmConsumerEvent event)
{
if ( refreshMsg.domainType() == EmaRdm.MMT_LOGIN &&
refreshMsg.state().streamState() == OmmState.StreamState.OPEN &&
refreshMsg.state().dataState() == OmmState.DataState.OK )
{
PostMsg postMsg = EmaFactory.createPostMsg();
RefreshMsg nestedRefreshMsg = EmaFactory.createRefreshMsg();
FieldList nestedFieldList = EmaFactory.createFieldList();
nestedFieldList.add(EmaFactory.createFieldEntry().real(22, 34, OmmReal.MagnitudeType.EXPONENT_POS_1));
nestedFieldList.add(EmaFactory.createFieldEntry().real(25, 35, OmmReal.MagnitudeType.EXPONENT_POS_1));
nestedFieldList.add(EmaFactory.createFieldEntry().time(18, 11, 29, 30));
nestedFieldList.add(EmaFactory.createFieldEntry().enumValue(37, 3));
nestedRefreshMsg.payload(nestedFieldList ).complete(true);
((OmmConsumer)event.closure()).submit(
postMsg.postId( postId++).serviceName( "NI_PUB" ).name( "TEST" )
.solicitAck( true ).complete(true)
.payload(nestedRefreshMsg), event.handle() );
}
}
The above snippet posts data to 2 numeric fields, a date field and an enumerated field. Note that we are using the (stream) handle from the Login event to Submit the message.
We also need to provide an additional callback to handle any acknowledgement message received from the server:
public void onAckMsg(AckMsg ackMsg, OmmConsumerEvent event)
{
System.out.println("Received AckMsg. Handle: " + event.handle() + " Closure: " + event.closure())
}
As you will note, the code required to Post data using EMA is relatively straightforward – even though the post messages contain the more efficient binary encoded data – rather than the inefficient string-based format used by SFC.
The above method of Posting data on the Login stream is referred to as Off-Stream posting. If you need to consume the instruments you are posting to, EMA also offers On-Stream posting. The key difference being that with On-Stream you request each instrument and use the (stream) handle for each instrument to submit the Post.
Code demonstrating both On-Stream and Off-Stream Posting are included in the 300_Series folder of the Training examples provided with the EMA C++ and Java SDK packages.
Closing Comments
Whilst the idea of migrating an application from SFC to EMA may initially seem daunting; the ease of learning, understanding, and coding of EMA – as illustrated above, should help allay much of the concern.
In addition to this, for most client applications I have come across – the classes used for consuming Market Data are abstracted or isolated from the business logic classes which form the core of the implementation. Therefore, it should be possible to replace the consumer classes using a newer API, without affecting the rest of the codebase.
If you have not already done so, I recommend you read in conjunction the linked Articles (see the upper right-hand side).
