About this article
The intended audiences of this article are software developers/engineers interested in or are currently developing a provider application to publish Symbol List. The article explains how to leverage Enterprise Message API to write a Non-Interactive Provider application that publishes Symbol List.
The Enterprise Message API (EMA - formerly known as Elektron Message API) is a data-neutral, multi-threaded, ease-of-use API providing access to OMM and RWF data. As part of the Refinitiv Real-Time SDK (RTSDK - formerly known as ESDK), the Enterprise Message API allows applications to consume and provide OMM data at the message level of the API stack. The message level is set on top of the transport level which is handled by the Enterprise Transport API (ETA - formerly known as Elektron Transport API/UPA). To understand EMA better e.g. concept, functionality, and capabilities, please refer to API Concepts Guide
Symbol List Overview
The Symbol List is a domain model defined by Refinitiv to provide access to a set of symbol names, typically from an index, service, or cache. Data of this domain must contain symbol names and optionally can contain additional cross-reference information such as permission information, name type, or other venue-specific content.
Basic scenario of Symbol List is that a consumer wants to open multiple items but doesn’t know their names, the consumer can first issue a request using a Symbol List. If there is a provider exists that can resolve the symbol list name into a set of item names, the provider sends back only symbol names back.
The Symbol List domain is a Refinitiv Domain Model (RDM) defined by Refinitiv to be used by all internal, and third-party, service providers and consumers. Clients that want to achieve maximum interoperability with these applications should utilize the Reuters Domain Models. The definition of RDMs is described in the RDM Usage Guide document. Therefore, applications can still interoperate as long as they conform to the item type models.
Like other domain models, the response message of Symbol List can be Refresh, Update, and Status.
- The Refresh message sends a list of item names to a consumer. Symbol List refresh can be sent in multiple parts that large symbol list information can be split into smaller and, more manageable pieces.
- The Update message adds or removes items from the list including update cross-reference information.
- The Status message conveys state change information associated with an item stream.
The content of Symbol List response is encoded as a Map, with each symbol represented by a map entry and where the symbol name is the entry key. The key’s type is ASCII string which should contain only characters that are valid in ASCII specification. An entry’s payload is optional, but when present the payload is a FieldList that contains additional cross-reference information such as permission information, name type, or other venue-specific content. As the Map Entry’s payload is optional, the entry’s payload can be empty.
Below is the sample structure of symbol list that an entry’s payload contains cross-reference information. MapEntry’s payload is a FieldList that contains the following information:
- PROV_SYMB (3422): Original symbol provided by the exchange
- PROD_PERM (1): Permission information
Each entry has an associated action which informs Consumer of how to apply the information contained in the entry. For Provider, the entry action helps to manage change on the list of symbols. Below are the possible actions for Map Entry.
- ADD - Indicates that the consumer should add the entry. An add action typically occurs when an entry is initially provided.
- UPDATE - Indicates that the consumer should update any previously stored or displayed information with the contents of this entry. An update action typically occurs when an entry has already been added and changes to the contents need to be conveyed.
- DELETE - Indicates that the consumer should remove any stored or displayed information associated with the entry. No map entry payload is included when the action is DELETE.
Please note that the order in which the Map Entries will be delivered within Map is not guaranteed. The order may be changed after the data pass through components such as ADH/ADS. If order of entry is necessary, additional Field ID (i.e. RANK_POS) can be added to Field List of Map Entry’s payload so Consumer can use the field for Entry sorting.
The map can contain additional Summary data which is the same type as Map Entry’s payload. The Summary data contains common information shared between Entries. Provider optionally can add shared information of Entry or Symbol List information in the Summary data.
ฺTo better understand the SymbolList structure, below are illustrations for sample Refresh and Update messages.
Publisher / Provider Concepts
If you would like to expose a specific data service to consumer applications in this article is Symbol List data, building a Provider application should be considered. Provider applications are able to connect to a Refinitiv Real-Time Distribution System (ADH/ADS servers) and to leverage its real-time and streaming distribution system to publish data. This is an efficient solution to make your own set of capabilities (e.g. content, workflow, etc.) available to your consumer applications.
Provider applications can be either interactive or non-interactive. Anyway, this article focuses on Non-Interactive Providers only.
- Interactive Providers (IP) receive requests from the infrastructure and must respond interactively to these requests. Via these requests, the infrastructure may ask for information about the provided services and capabilities of the provider. The infrastructure also forwards consumer requests demanding for specific data items. For all these requests, the Interactive Provider must respond interactively by either providing the requested information or by rejecting the demand.
- Non-Interactive Providers (NIP) are not solicited by infrastructure requests but instead, they publish information (e.g. Available services, data items…) following their own timing. In this model ADH server cache the published information and serves it to the demanding consumer applications as shown in the NIP Feature Diagram below:
1. Therefore, NIP can publish Symbol List into the ADH cache without needing to handle/wait for requests from any consumer applications. Whenever a consumer application sends a request for Symbol List to Advanced Distribution Server(ADS) , ADS forwards the request to ADH. Finally, ADH provides Symbol List kept in the cache which is received from NIP previously to the consumer application.
Solution
Prerequisites
Download an Refinitiv Real-Time SDK Package
2. Declare the service sent by the NIP in ADH/ADS servers configuration files. This process must be executed by your Market Data administration team. For information, please refer to ADH Configuration for NI Providers which explains the steps involved in this process. If Market Data administration team requires any assistance regarding configuring ADH/ADS servers, they can submit the query to Refinitiv Real-Time support team directly via In this article, we will use the service named NI_PUB which is one of the default service configured in EmaConfig.xml shipped with Real-Time SDK Package.
3. Put EmaConfig.xml shipped with Real-Time SDK package in the working directory. Then, modify the service named NI_PUB configured in the file to support Symbol List capabilities as an example below:
<NiProvider>
<Name value="Provider_1"/>
…
<Directory value="Directory_1"/>
…
<Directory>
<Name value="Directory_1"/>
…
<Service>
<Name value="NI_PUB"/>
…
<Capabilities>
…
<CapabilitiesEntry value="MMT_SYMBOL_LIST"/>
</Capabilities>
…
Building a Non-Interactive Provider to publish Symbol List
The general process of building NIP to publish Symbol List can be summarized in the following steps:
To make a simple NIP and focus on publishing Symbol List, we will let EMA sends automatically a source directory info configured in the EmaConfig.xml file as shown in the 3rd step. Alternatively, this can be performed programmatically by the application itself as demonstrated in ex300_MP_Streaming in Real-Time SDK Java Packages or 300_MP_Streaming in Real-Time SDK C++ Packages.
According to the step 4th, to publish a Symbol List Refresh or Update message properly, you should follow the guideline below:
- Initially, Provider publishes Refresh messages which contain all Symbol names in MapEntry’s key. All MapEntry’s action is ADD. Provider optionally can split a list of the symbol to be sent in multiple Refresh responses.
- In case that symbols are removed from the list, provider publishes Update message which contains removed symbols in MapEntrys. The Entry’s action is DELETE. The MapEntry’s payload needs to be empty.
- In case that symbols are added to the list, provider publishes Update message which contains added symbols in MapEntrys. The Entry’s action is ADD.
- In case that cross-reference information of symbols is changed, provider publishes Update message which contains updated symbols in MapEntrys. The Entry’s action is UPDATE. The Entry’s payload contains updated FieldList.
Source Code
This section is to explain how to develop a NIP application using EMA Java and C++ to publish Symbol List. Our example application contains 2 classes:
- SymbolListMapEntry class. This is a utility class. It represents an entry in the map of Symbol List. Each entry consists of Action, PROD_PERM and PROV_SYMB field as shown in the figure below:
- NiProvider class. It is the main class of this application. It publishes Symbol List Refresh and Update messages to ADH. It uses SymbolListMapEntry class to help to create the Symbol List Map.
The overview steps to develop the application are listed below:
- Create SymbolListMapEntry class in SymbolListMapEntry.java/.h/.cpp file. Add Action, PROD_PERM and PROV_SYMB field of entry and the methods to get the fields. Then, create the methods which performs Add, Delete and Update action for the entry as an example shown below:
EMA Java
import com.refinitiv.ema.access.MapEntry;
//The class is to represent an entry in the map of Symbol List.
public class SymbolListMapEntry {
//Action can be UPDATE(1), ADD(2) or DELETE(3)
private int action;
//PROD_PERM which field id is 1 and type is UINT64
private int prodPerm;
//PROV_SYMB which field id is 3422 and type is RMTES_STRING
private String provSymb;
private static final int DEFAULT_PROD_PERM=3056;
int getAction() {
return action;
}
int getProdPerm() {
return prodPerm;
}
String getProSymb() {
return provSymb;
}
static int getDefaultProdPerm() {
return DEFAULT_PROD_PERM;
}
//For ADD action
//Set PROD_PERM and PROV_SYMB according to the input values
//If the input PROD_PERM is less than zero set to default(3056)
//Set Action to be ADD
void performAddAction(int newpp,String newps) {
if(newpp<=0)
newpp = DEFAULT_PROD_PERM;
prodPerm = newpp;
provSymb = newps;
action = MapEntry.MapAction.ADD;//2
}
//For UPDATE action
//Set PROD_PERM according to the input value
//If the input PROD_PERM is less than zero set to default(3056)
//Set Action to be UPDATE
void performUpdateAction(int newpp) {
if(newpp<=0)
newpp = DEFAULT_PROD_PERM;
prodPerm = newpp;
action = MapEntry.MapAction.UPDATE;//1
}
//For DELETE action
//Set Action to be DELETE
void performDeleteAction() {
action = MapEntry.MapAction.DELETE;//3
}
}
- EMA C++
SymbolListMapEntry.h
#pragma once
#include <string>
#include <Ema.h>
//The class is to represent an entry in the map of Symbol List.
class SymbolListMapEntry {
public:
inline int getAction() {
return action;
}
inline int getProdPerm() {
return prodPerm;
}
inline std::string getProSymb() {
return provSymb;
}
inline static int getDefaultProdPerm() {
return DEFAULT_PROD_PERM;
}
void performAddAction(int newpp, std::string newps);
void performUpdateAction(int newpp);
void performDeleteAction();
private:
//Action can be UPDATE(1), ADD(2) or DELETE(3)
int action;
//PROD_PERM which field id is 1 and type is UINT64
int prodPerm;
//PROV_SYMB which field id is 3422 and type is RMTES_STRING
std::string provSymb;
static const int DEFAULT_PROD_PERM = 3056;
};
SymbolListMapEntry.cpp
#include "SymbolListMapEntry.h"
using namespace refinitiv::ema::access;
//For ADD action
//Set PROD_PERM and PROV_SYMB according to the input values
//If the input PROD_PERM is less than zero set to default(3056)
//Set Action to be ADD
void SymbolListMapEntry::performAddAction(int newpp, std::string newps) {
if (newpp <= 0)
newpp = DEFAULT_PROD_PERM;
prodPerm = newpp;
provSymb = newps;
action = MapEntry::MapAction::AddEnum;//2
}
//For UPDATE action
//Set PROD_PERM according to the input value
//If the input PROD_PERM is less than zero set to default(3056)
//Set Action to be UPDATE
void SymbolListMapEntry::performUpdateAction(int newpp) {
if (newpp <= 0)
newpp = DEFAULT_PROD_PERM;
prodPerm = newpp;
action = MapEntry::MapAction::UpdateEnum;//1
}
//For DELETE action
//Set Action to be DELETE
void SymbolListMapEntry::performDeleteAction() {
action = MapEntry::MapAction::DeleteEnum;//3
}
2. Create a main class named NiProvider in a new file named NiProvider.java/.h/.cpp. The class declares and initializes/uninitializes the variables used to create Symbol List map as shown below.
- EMA Java
public class NiProvider {
//The maximum number of entries in the Symbol List
int MAX_ENTRIES=150;
//The minimum number of entries in the Symbol List
int MIN_ENTRIES=20;
//The maximum chance of the DELETE action is 10
//It the random number is 1-10, perform DELETE action
int deleteMaxChance = 10;
//The maximum chance of the UPDATE action is 40
//It the random number is 11-40, perform UPDATE action
int updateMaxChance = 40;
//The maximum chance of the ADD action is 100
//It the random number is 41-100, perform ADD action
int addMaxChance = 100;
//The List of SymbolListMapEntry represents all entries in the Symbol List from both Refresh and Update messages
List<SymbolListMapEntry> symbolListMap;
//The List of SymbolListMapEntry represents the entries in an update message
List<SymbolListMapEntry> updateMap;
//The Set keeps the PROV_SYMB/Key in the current Symbol List
//It is used to verify if a new generated PROV_SYMB/Key is duplicated with any one in the Set
HashSet<String> currentKeys;
//The Random class is used to generate PROD_PERM and PROV_SYMB/Key at random
//It is used to choose an Action, an entry which will be updated or deleted at random
Random randomInt;
//Initialize the NiProvider's variables
public NiProvider() {
symbolListMap = new ArrayList<SymbolListMapEntry>(MAX_ENTRIES);
updateMap = new ArrayList<SymbolListMapEntry>(5);
currentKeys = new HashSet<String>(MAX_ENTRIES);
randomInt = new Random();
}
}
- EMA C++
NiProvider.h
#pragma once
#include <vector>
#include <unordered_set>
#include "SymbolListMapEntry.h"
#include <time.h>
#ifdef WIN32
#include <windows.h>
#else
#include <sys/time.h>
#endif
#include "Ema.h"
void sleep(int millisecs)
{
#if defined WIN32
::Sleep((DWORD)(millisecs));
#else
struct timespec sleeptime;
sleeptime.tv_sec = millisecs / 1000;
sleeptime.tv_nsec = (millisecs % 1000) * 1000000;
nanosleep(&sleeptime, 0);
#endif
}
class NiProvider {
public:
//Initialize the NiProvider's variables
NiProvider();
//Uninitialize the NiProvider's variables
~NiProvider();
SymbolListMapEntry* generateNewEntry();
void generateUpdatedEntries();
//The List of SymbolListMapEntry represents the entries in an update message
std::vector<SymbolListMapEntry*>* updateMap;
private:
//The maximum number of entries in the Symbol List
static const int MAX_ENTRIES = 150;
//The minimum number of entries in the Symbol List
static const int MIN_ENTRIES = 20;
//The maximum chance of the DELETE action is 10
//It the random number is 1-10, perform DELETE action
static const int deleteMaxChance = 10;
//The maximum chance of the UPDATE action is 40
//It the random number is 11-40, perform UPDATE action
static const int updateMaxChance = 40;
//The maximum chance of the ADD action is 100
//It the random number is 41-100, perform ADD action
static const int addMaxChance = 100;
//The List of SymbolListMapEntry represents all entries in the Symbol List from both Refresh and Update messages
std::vector<SymbolListMapEntry*>* symbolListMap;
//The Set keeps the PROV_SYMB/Key in the current Symbol List
//It is used to verify if a new generated PROV_SYMB/Key is duplicated with any one in the Set
std::unordered_set<std::string>* currentKeys;
};
NiProvider.cpp
#include "NiProvider.h"
#include <iostream>
#include <cstring>
using namespace refinitiv::ema::access;
using namespacerefinitiv::ema::rdm;
using namespace std;
NiProvider::NiProvider() {
symbolListMap = new std::vector<SymbolListMapEntry*>();
updateMap = new std::vector<SymbolListMapEntry*>();
currentKeys = new std::unordered_set<std::string>();
}
NiProvider::~NiProvider() {
for (vector<SymbolListMapEntry*>::iterator it = symbolListMap->begin(); it != symbolListMap->end(); it++)
{
delete *it;
}
delete symbolListMap;
delete updateMap;
delete currentKeys;
}
3. Create a utility method in NiProvider class named generateNewEntry(). The method creates a new entry which is SymbolListMapEntry instance. The entry contains a random symbol, some fields and ADD action. The entry is added to the Symbol List map later.
- EMA Java
//Generate a new SymbolListMapEntry which represents an entry.
public SymbolListMapEntry generateNewEntry()
{
//Initialize a new SymbolListMapEntry
SymbolListMapEntry newEntry = new SymbolListMapEntry();
//The variable to keep PROV_SYMB/Key
String tmpKey;
//PROV_SYMB pattern is [A-Z][A-Z][A-Z].[A-Z]
//Generate a PROV_SYMB/Key till it is not the same as any current PROV_SYMB/Key
do {
tmpKey = String.valueOf(new char[] { (char)(randomInt.nextInt(26) + 65), (char)(randomInt.nextInt(26) + 65), (char)(randomInt.nextInt(26) + 65), '.',(char)(randomInt.nextInt(26) + 65) });
} while(currentKeys.contains(tmpKey));
//Set the SymbolListMapEntry with ADD Action
newEntry.performAddAction(SymbolListMapEntry.getDefaultProdPerm(),tmpKey);
//Add the SymbolListMapEntry in the Symbol List
symbolListMap.add(newEntry);
//Add the PROV_SYMB/Key of the SymbolListMapEntry in the PROV_SYMB/Key Set
currentKeys.add(newEntry.getProSymb());
return newEntry;
}
- EMA C++
NiProvider.h
#include <time.h>
class NiProvider {
public:
SymbolListMapEntry* generateNewEntry();
NiProvider.cpp
//Generate a new SymbolListMapEntry which represents an entry.
SymbolListMapEntry* NiProvider::generateNewEntry()
{
//Initialize a new SymbolListMapEntry
SymbolListMapEntry* newEntry = new SymbolListMapEntry();
//The variable to keep PROV_SYMB/Key
std::string tmpKey;
srand(time(NULL));
//PROV_SYMB pattern is [A-Z][A-Z][A-Z].[A-Z]
//Generate a PROV_SYMB/Key till it is not the same as any current PROV_SYMB/Key
do {
tmpKey.clear();
tmpKey += (char)(rand() % 26 + 65);
tmpKey += (char)(rand() % 26 + 65);
tmpKey += (char)(rand() % 26 + 65);
tmpKey += '.';
tmpKey += (char)(rand() % 26 + 65);
} while (currentKeys->count(tmpKey));
//Set the SymbolListMapEntry with ADD Action
newEntry->performAddAction(SymbolListMapEntry::getDefaultProdPerm(), tmpKey);
//Add the SymbolListMapEntry in the Symbol List
symbolListMap->push_back(newEntry);
//Add the PROV_SYMB/Key of the SymbolListMapEntry in the PROV_SYMB/Key Set
currentKeys->insert(newEntry->getProSymb());
return newEntry;
}
4. Create a utility method in NiProvider class named generateUpdatedEntries(). The method creates a Symbol List Map which will be added to the payload of an Update message. The map contains random entries which have random actions i.e. ADD, UPDATE or DELETE.
- EMA Java
//Generate the List of new entries which will be added to the next update message
public void generateUpdatedEntries()
{
//Clear the list of entries which have been added to the latest update message
updateMap.clear();
//Create the PROV_SYMB/Key Set of the update message
HashSet<String> updateEntries = new HashSet<String>();
//Generate a new ProdPerm which is 1000-5000 at random
int newProdPerm = randomInt.nextInt(4000) + 1000;
//The list contains maximum 5 entries, for each one do:
for (int i = 0; i < 5; i++)
{
//Declare SymbolListMapEntry to keep a new entry
SymbolListMapEntry updateEntry = null;
//Select an Action at random
int action = randomInt.nextInt(100);
//If the number of entries in the Symbol List is less than the minimum(20)
//or the random Action is ADD
if (symbolListMap.size() < MIN_ENTRIES || action > updateMaxChance)
{
//If the number of entries in the Symbol List is less than maximum(150)
if(symbolListMap.size() < MAX_ENTRIES) {
//Perform ADD action by generating a new entry with ADD action
updateEntry = generateNewEntry();
}
//If the number of entries in the Symbol List equal or is more than the maximum(150),
//try to delete an existing entry in the Symbol List
else
{
//Select the deleted entry in the Symbol List at random
int index = randomInt.nextInt(symbolListMap.size());
updateEntry = symbolListMap.get(index);
//If the PROV_SYMB/Key of the selected entry is duplicated with
//any PROV_SYMB/Key in the update Set, skip to create the next entry
if (updateEntries.contains(updateEntry.getProSymb()))
{
continue;
}
//Otherwise, perform DELETE action on the selected entry
updateEntry.performDeleteAction();
//Remove the deleted entry from the Symbol List
symbolListMap.remove(updateEntry);
//Remove the PROV_SYMB/Key of the deleted entry from the PROV_SYMB/Key Set
currentKeys.remove(updateEntry.getProSymb());
}
}
//If the random action is UPDATE
else if (action > deleteMaxChance)
{
//Select the updated entry in the Symbol List at random
int index = randomInt.nextInt(symbolListMap.size());
updateEntry = symbolListMap.get(index);
//If the PROV_SYMB/Key of the selected entry is duplicated with
//any PROV_SYMB/Key in the update Set, skip to create the next entry
if (updateEntries.contains(updateEntry.getProSymb()))
{
continue;
}
//Otherwise, perform UPDATE action on the selected entry
updateEntry.performUpdateAction(newProdPerm);
}
//If the random action is DELETE
else
{
//If the number of entries in the Symbol List equal or less than minimum(20)
if (symbolListMap.size() <= MIN_ENTRIES) {
//Perform ADD action by generating a new entry with ADD action
updateEntry = generateNewEntry();
}
else
{ //If the number of entries in the Symbol List is more than the minimum(20),
//try to delete an existing entry in the Symbol List
//Select the deleted entry in the Symbol List at random
int index = randomInt.nextInt(symbolListMap.size());
updateEntry = symbolListMap.get(index);
//If the PROV_SYMB/Key of the selected entry is duplicated with
//any PROV_SYMB/Key in the update Set, skip to create the next entry
if (updateEntries.contains(updateEntry.getProSymb()))
{
continue;
}
//Otherwise, perform DELETE action on the selected entry
updateEntry.performDeleteAction();
//Remove the deleted entry from the Symbol List
symbolListMap.remove(updateEntry);
//Remove the PROV_SYMB/Key of the deleted entry from the PROV_SYMB/Key Set
currentKeys.remove(updateEntry.getProSymb());
}
}
//Add the PROV_SYMB/Key in the Set of the new update message.
updateEntries.add(updateEntry.getProSymb());
//Add the entry in the List of the new update message.
updateMap.add(updateEntry);
}
}
- EMA C++
NiProvider.h
class NiProvider {
public:
void generateUpdatedEntries();
NiProvider.cpp
//Generate the List of new entries which will be added to the next update message
void NiProvider::generateUpdatedEntries()
{
//Create the PROV_SYMB/Key Set of the update message
std::unordered_set<std::string>* updateEntries = new std::unordered_set<std::string>();
//Generate a new ProdPerm which is 1000-5000 at random
int newProdPerm = rand() % 4000 + 1000;
//The list contains maximum 5 entries, for each one do:
for (int i = 0; i < 5; i++)
{
//Declare SymbolListMapEntry to keep a new entry
SymbolListMapEntry* updateEntry = 0;
//Select an Action at random
int action = rand() % 100;
//If the number of entries in the Symbol List is less than the minimum(20)
//or the random Action is ADD
if (symbolListMap->size() < MIN_ENTRIES || action > updateMaxChance)
{
//If the number of entries in the Symbol List is less than maximum(150)
if (symbolListMap->size() < MAX_ENTRIES) {
//Perform ADD action by generating a new entry with ADD action
updateEntry = generateNewEntry();
}
//If the number of entries in the Symbol List equal or is more than the maximum(150),
//try to delete an existing entry in the Symbol List
else
{
//Select the deleted entry in the Symbol List at random
int index = rand() % symbolListMap->size();
updateEntry = (*symbolListMap)[index];
//If the PROV_SYMB/Key of the selected entry is duplicated with
//any PROV_SYMB/Key in the update Set, skip to create the next entry
if (updateEntries->count(updateEntry->getProSymb()))
{
continue;
}
//Otherwise, perform DELETE action on the selected entry
updateEntry->performDeleteAction();
//Remove the deleted entry from the Symbol List
symbolListMap->erase(symbolListMap->begin() + index);
//Remove the PROV_SYMB/Key of the deleted entry from the PROV_SYMB/Key Set
currentKeys->erase(updateEntry->getProSymb());
}
}
//If the random action is UPDATE
else if (action > deleteMaxChance)
{
//Select the updated entry in the Symbol List at random
int index = rand() % symbolListMap->size();
updateEntry = (*symbolListMap)[index];
//If the PROV_SYMB/Key of the selected entry is duplicated with
//any PROV_SYMB/Key in the update Set, skip to create the next entry
if (updateEntries->count(updateEntry->getProSymb()))
{
continue;
}
//Otherwise, perform UPDATE action on the selected entry
updateEntry->performUpdateAction(newProdPerm);
}
//If the random action is DELETE
else
{
//If the number of entries in the Symbol List equal or less than minimum(20)
if (symbolListMap->size() <= MIN_ENTRIES) {
//Perform ADD action by generating a new entry with ADD action
updateEntry = generateNewEntry();
}
else
{ //If the number of entries in the Symbol List is more than the minimum(20),
//try to delete an existing entry in the Symbol List
//Select the deleted entry in the Symbol List at random
int index = rand() % symbolListMap->size();
updateEntry = (*symbolListMap)[index];
//If the PROV_SYMB/Key of the selected entry is duplicated with
//any PROV_SYMB/Key in the update Set, skip to create the next entry
if (updateEntries->count(updateEntry->getProSymb()))
{
continue;
}
//Otherwise, perform DELETE action on the selected entry
updateEntry->performDeleteAction();
//Remove the deleted entry from the Symbol List
symbolListMap->erase(symbolListMap->begin() + index);
//Remove the PROV_SYMB/Key of the deleted entry from the PROV_SYMB/Key Set
currentKeys->erase(updateEntry->getProSymb());
}
}
//Add the PROV_SYMB/Key in the Set of the new update message.
updateEntries->insert(updateEntry->getProSymb());
//Add the entry in the List of the new update message.
updateMap->push_back(updateEntry);
}
}
5. Add the main function in NiProvider class to start EMA Non-Interactive Provider and publish Symbol List as explained below:
- Create and run EMA Non-Interactive Provider, OmmProvider class, which connects and logs in to ADH.
- Create a Symbol List Map using generateNewEntry() created in the 3rd step. Then, put the map to the payload of a Refresh message and publish the message to ADH using EMA method named OmmProvider.submit(..).
- Create a Symbol List map for an Update message using generateUpdatedEntries() created in the 4th step. Then, put the map into a payload of an Update message and publish the message to ADH using EMA method named OmmProvider.submit(..).
- Send one Update message every 3 seconds till 20 Update messages then log out from ADH and shut down.
The complete source code of the main function is shown below:
- EMA Java
public static void main(String[] args)
{
OmmProvider provider = null;
try
{
//Use the default NiProvider's configuration in EmaConfig.xml found in the working directory
OmmNiProviderConfig config = EmaFactory.createOmmNiProviderConfig();
//change "adhserver:14003" to your ADH server IP/name and its port
//change "user" to NI Provider's user which logs in to ADH
provider = EmaFactory.createOmmProvider(config.host("adhserver:14003").username("user"));
System.out.println("Non-Interactive Provider starts.");
//The handle which identifies item stream on which to send the Refresh message and Update messages of a Symbol List.
long itemHandle = 5;
//Create an object of NiProvider which provides the methods to create/update and manage Symbol List.
NiProvider niprov = new NiProvider();
//Create a map which is encoded content of Symbol List. The map contains entries.
Map map = EmaFactory.createMap();
//Create a field list which is encoded content of an entries.
FieldList fieldList = EmaFactory.createFieldList();
//Encode a map contains 25 entries
for( int i = 0; i < 25; i++ )
{
//generate SymbolListMapEntry instance consisting of action(ADD), PROD_PERM field and PROV_SYMB field
SymbolListMapEntry newEntry = niprov.generateNewEntry();
//set PROD_PERM field
fieldList.add(EmaFactory.createFieldEntry().uintValue(1, newEntry.getProdPerm()));
//set PROV_SYMB field
fieldList.add(EmaFactory.createFieldEntry().rmtes(3422, ByteBuffer.wrap(newEntry.getProSymb().getBytes())));
//set PROV_SYMB as a Key, set Action and fields to an entry. Then, add the entry to the map.
map.add(EmaFactory.createMapEntry().keyAscii(newEntry.getProSymb(), newEntry.getAction(), fieldList));
fieldList.clear();
}
System.out.println("Non-Interactive Provider is publishing a Symbol List Refresh message.");
//Publish a single complete Refresh message of the Symbol List named 0#NEWSBL to the service named NI_PUB.
//The payload of the Refresh message is the map created previously.
provider.submit( EmaFactory.createRefreshMsg().domainType(EmaRdm.MMT_SYMBOL_LIST).serviceName("NI_PUB").name("0#NEWSBL").state(OmmState.StreamState.OPEN, OmmState.DataState.OK, OmmState.StatusCode.NONE, "UnSolicited Refresh Completed").payload(map).complete(true), itemHandle);
Thread.sleep(1000);
map.clear();
System.out.println("Non-Interactive Provider starts publishing Symbol List Update messages.");
//Send one Update message every 3 seconds till 20 Update messages
for( int i = 0; i < 20; i++ )
{
//Generate the List of new entries
niprov.generateUpdatedEntries();
Iterator<SymbolListMapEntry> it = niprov.updateMap.iterator();
//for each entry in the List
while(it.hasNext())
{
SymbolListMapEntry anEntry = it.next();
//An entry which action is ADD or UPDATE contains PROD_PERM and PROV_SYMB field.
if (anEntry.getAction()!=MapEntry.MapAction.DELETE)
{
//set PROD_PERM field
fieldList.add(EmaFactory.createFieldEntry().uintValue(1, anEntry.getProdPerm()));
//set PROV_SYMB field
fieldList.add(EmaFactory.createFieldEntry().rmtes(3422, ByteBuffer.wrap(anEntry.getProSymb().getBytes())));
}
//set PROV_SYMB as a Key, set Action and fields to an entry. Then, add the entry to the map.
map.add(EmaFactory.createMapEntry().keyAscii(anEntry.getProSymb(),anEntry.getAction(),fieldList));
fieldList.clear();
}
//Publish an Update message of the Symbol List named 0#NEWSBL to the service named NI_PUB.
//The payload of the Update message is the map created previously.
provider.submit( EmaFactory.createUpdateMsg().serviceName("NI_PUB").name("0#NEWSBL").domainType(EmaRdm.MMT_SYMBOL_LIST).payload( map ), itemHandle );
map.clear();
Thread.sleep(3000);
}
System.out.println("Non-Interactive Provider published all Symbol List messages. It exits.");
}
catch (InterruptedException | OmmException excp)
{
System.out.println(excp.getMessage());
}
finally
{
if (provider != null) provider.uninitialize();
}
}
- EMA C++
NiProvider.cpp
int main(int argc, char* argv[])
{
try
{
//Use the default NiProvider's configuration in EmaConfig.xml found in the working directory
//change "adhserver:14003" to your ADH server IP/name and its port
//change "user" to NI Provider's user which logs in to ADH
OmmProvider provider(OmmNiProviderConfig().host("192.168.27.46:14003").username("user"));
cout<< "Non-Interactive Provider starts." << endl;
//The handle which identifies item stream on which to send the Refresh message and Update messages of a Symbol List.
long itemHandle = 5;
//Create an object of NiProvider which provides the methods to create/update and manage Symbol List.
NiProvider* niprov = new NiProvider();
//Create a map which is encoded content of Symbol List. The map contains entries.
Map map;
//Create a field list which is encoded content of an entries.
FieldList fieldList;
//Encode a map contains 25 entries
for (int i = 0; i < 25; i++)
{
//generate SymbolListMapEntry instance consisting of action(ADD), PROD_PERM field and PROV_SYMB field
SymbolListMapEntry* newEntry = niprov->generateNewEntry();
string prodSymb = newEntry->getProSymb();
//set PROD_PERM field
fieldList.addUInt(1, newEntry->getProdPerm());
//set PROV_SYMB field
char *cstr = new char[prodSymb.length() + 1];
strcpy(cstr, prodSymb.c_str());
fieldList.addRmtes(3422, EmaBuffer(cstr, sizeof(cstr)));
fieldList.complete();
//set PROV_SYMB as a Key, set Action and fields to an entry. Then, add the entry to the map.
map.addKeyAscii(EmaString(cstr), static_cast<MapEntry::MapAction>(newEntry->getAction()), fieldList);
delete[] cstr;
fieldList.clear();
}
map.complete();
cout<<"Non-Interactive Provider is publishing a Symbol List Refresh message."<<endl;
//Publish a single complete Refresh message of the Symbol List named 0#NEWSBL to the service named NI_PUB.
//The payload of the Refresh message is the map created previously.
provider.submit(RefreshMsg().domainType(MMT_SYMBOL_LIST).serviceName("API_BRICKBAR_NI").name("0#NEWSBL").state(OmmState::StreamState::OpenEnum, OmmState::DataState::OkEnum, OmmState::StatusCode::NoneEnum, "UnSolicited Refresh Completed").payload(map).complete(true), itemHandle);
Sleep(1000);
map.clear();
cout<< "Non-Interactive Provider starts publishing Symbol List Update messages." << endl;
//Send one Update message every 3 seconds till 20 Update messages
for (int i = 0; i < 20; i++)
{
//Generate the List of new entries
niprov->generateUpdatedEntries();
for (vector<SymbolListMapEntry*>::iterator it = niprov->updateMap->begin(); it != niprov->updateMap->end();it++)
{
SymbolListMapEntry* anEntry = *it;
//An entry which action is ADD or UPDATE contains PROD_PERM and PROV_SYMB field.
string prodSymb = anEntry->getProSymb();
char *cstr = new char[prodSymb.length() + 1];
strcpy(cstr, prodSymb.c_str());
if (anEntry->getAction() != MapEntry::MapAction::DeleteEnum)
{
//set PROD_PERM field
fieldList.addUInt(1, anEntry->getProdPerm());
//set PROV_SYMB field
fieldList.addRmtes(3422, EmaBuffer(cstr, sizeof(cstr)));
}
fieldList.complete();
//set PROV_SYMB as a Key, set Action and fields to an entry. Then, add the entry to the map.
map.addKeyAscii(EmaString(cstr), static_cast<MapEntry::MapAction>(anEntry->getAction()), fieldList);
delete[] cstr;
fieldList.clear();
}
map.complete();
//Publish an Update message of the Symbol List named 0#NEWSBL to the service named NI_PUB.
//The payload of the Update message is the map created previously.
provider.submit(UpdateMsg().serviceName("API_BRICKBAR_NI").name("0#NEWSBL").domainType(MMT_SYMBOL_LIST).payload(map), itemHandle);
map.clear();
//Clear the list of entries which have been added to the latest update message
for (int i = 0; i < niprov->updateMap->size(); i++)
{
if ((*niprov->updateMap)[i]->getAction() == (int)MapEntry::DeleteEnum)
delete (*niprov->updateMap)[i];
}
niprov->updateMap->clear();
sleep(3000);
}
cout << "Non-Interactive Provider published all Symbol List messages. It exits." << endl;
}
catch (const OmmException& excp)
{
cout << excp << endl;
}
return 0;
}
You can download the complete Java and C++ application source code explained above at Refinitiv-API-Samples/Article.EMA.JavaCpp.SymbolListNIP GitHub page.
Build and Run the application
EMA Java
EMA, ETA, and EMA's dependency library files are required to build and run the EMA Java NIP application. Fortunately, all files are shipped with the Real-Time SDK Java package. Hence, you just set the CLASSPATH to all of them before building and running the application.
The following example command lines show how to set the CLASSPATH, build and run the application on Windows:
- Set the classpath to EMA, ETA, and EMA's dependency library files. For example:
set CLASSPATH=C:\RTSDK-2.0.1.L1.java.rrg\Java\Ema\Libs\ema-3.6.1.0.jar;C:\RTSDK-2.0.1.L1.java.rrg\Java\Eta\Libs\eta-3.6.1.0.jar;C:\RTSDK-2.0.1.L1.java.rrg\Java\Eta\Libs\etaValueAdd-3.6.1.0.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Eta\Libs\jdacsetalib.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Ema\Libs\apache\commons-collections-3.2.2.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Ema\Libs\apache\commons-configuration-1.10.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Ema\Libs\apache\commons-lang-2.6.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Ema\Libs\apache\commons-logging-1.2.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Eta\Libs\SLF4J\slf4j-1.7.12\slf4j-api-1.7.12.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Eta\Libs\SLF4J\slf4j-1.7.12\slf4j-jdk14-1.7.12.jar;C:\RTSDK-2.0.1.L1.java.rrg\RTSDK-BinaryPack\Java\Eta\Libs\xpp3-1.1.4c.jar
2. Compile the application:
javac com\refinitiv\ema\examples\training\niprovider\example__SymbolList__Streaming\*.java
3. Run the application. Before you run it, include the path containing NIP application class files generated from the step 2nd to the CLASSPATH.
set CLASSPATH=.;%CLASSPATH%
Then, you can run the application using the following command line:
java com.refinitiv.ema.examples.training.niprovider.example__SymbolList__Streaming.NiProvider
EMA C++
EMA library (i.e. libema) and dependency libraries (i.e. librssl, librsslVA) are required to build and run the EMA C++ NIP application. The library is provided in the Real-Time SDK C++ package. Hence, you can just set the project or makefile to load the library. Please note that to build Windows application with EMA static library, the “/D __EMA_STATIC_BUILD__” option is required.
In Linux package, EMA shared libraries are provided with its package version appended to the end. For example, libema.so.3.6.1.0. The API provides a helpful script that will create soft links for the appropriate library names. You need to run the script (i.e. LinuxSoLink) in the package before running the EMA NIP application. For more information, please see the “2.3.1 Shared Libraries” section of README file in the EMA Linux package.
Anyway, you can refer to the Visual Studio project provided with the example source code. To use the Visual Studio project, the "EMAInstall" variable needs to be added in Windows Environment variable to point to the EMA installation path.
For the dynamic library, you will need to set PATH or LD_LIBRARY_PATH before running the application.
Ema\Libs\<OS>\<Build Option>\Shared
Eta\Libs\<OS>\<Build Option>\Shared
You can run the application using the command line:
NiProvider.exe
The Example Result
When you run the NIP application publishing Symbol List, you should see the messages like below:
May 31, 2018 4:22:44 PM com.refinitiv.ema.access.ChannelCallbackClient reactorChannelEventCallback
INFO: loggerMsg
ClientName: ChannelCallbackClient
Severity: Info
Text: Received ChannelUp event on channel Channel_3
Instance Name Provider_1_1
Component Version adh3.0.6.L1.solaris.tis.rrg 64-bit
loggerMsgEnd
Non-Interactive Provider starts.
Non-Interactive Provider is publishing a Symbol List Refresh message.
Non-Interactive Provider starts publishing Symbol List Update messages.
Non-Interactive Provider published all Symbol List messages. It exits.
While the NIP application is running, you can run the EMA Java Consumer named ex270_SymbolList or EMA C++ Consumer named 270_SymbolList to verify the NIP's service and its published Symbol List. The Consumer application must connect to ADS which connecting to ADH which the NIP publishes Symbol List. According to the given source code, the Consumer application needs to request to the service named NI_PUB with the Symbol list named 0#NEWSBL
The example result shown in ex270_SymbolList:
May 31, 2018 4:55:28 PM com.refinitiv.ema.access.ChannelCallbackClient reactorChannelEventCallback
INFO: loggerMsg
ClientName: ChannelCallbackClient
Severity: Info
Text: Received ChannelUp event on channel Channel_1
Instance Name Consumer_1_1
Component Version ads3.0.6.L1.solaris.tis.rrg 64-bit
loggerMsgEnd
Item Name: 0#NEWSBL
Service Name: NI_PUB
Item State: Open / Ok / None / 'UnSolicited Refresh Completed'
Name Action
MVO.T Add
PROD_PERM
3056
PROV_SYMB
MVO.T
BRT.S Add
PROD_PERM
3056
PROV_SYMB
BRT.S
OPB.P Add
PROD_PERM
3056
PROV_SYMB
OPB.P
EVK.A Add
PROD_PERM
3056
PROV_SYMB
EVK.A
GMV.A Add
PROD_PERM
3056
PROV_SYMB
GMV.A
Troubleshooting
Q: When compiling the NIP EMA Java application, it shows "error: package [package name] does not exist"
com\refinitiv\ema\examples\training\niprovider\example__SymbolList__Streaming\NiProvider.java:10: error: package com.refinitiv.ema.access does not exist import com.refinitiv.ema.access.EmaFactory;
A: the CLASSPATH is not set or set to the wrong path. See step 1 in EMA Java under the section Build and Run the application.
Q: When running the NIP EMA Java application, it shows "Error: Could not find or load main class [class]"
Error: Could not find or load main class com.refinitiv.ema.examples.training.niprovider.example__SymbolList__Streaming.NiProvider
A: the path containing NIP EMA Java application class files are not included in the CLASSPATH or set to the wrong path or the run [class] may be wrong. See step 3 in EMA Java under the section Build and Run the application.
Q: When running the application, it shows "Error - exceeded initialization timeout (5 s)" and "login failed (timed out after waiting 45000 milliseconds) for [server]:[port])"
Jun 08, 2018 9:38:25 AM com.refinitiv.ema.access.ChannelCallbackClient reactorChannelEventCallback
WARNING: loggerMsg
ClientName: ChannelCallbackClient
Severity: Warning
Text: Received ChannelDownReconnecting event on channel Channel_3
RsslReactor Channel is null
Error Id 0
Internal sysError 0
Error Location Reactor.processWorkerEvent
Error text Error - exceeded initialization timeout (5 s)
loggerMsgEnd
Jun 08, 2018 9:38:33 AM com.refinitiv.ema.access.OmmBaseImpl handleLoginReqTimeout
SEVERE: loggerMsg
ClientName: Provider_1_1
Severity: Error
Text: login failed (timed out after waiting 45000 milliseconds) for 192.168.27.99:14003)
loggerMsgEnd
login failed (timed out after waiting 45000 milliseconds) for 192.168.27.99:14003)
A: the possible causes of the problem are:
- The host or port set in the application source code is wrong or the host running ADH goes down. Please contact Market Data administrator to verify the correct host/IP of ADH and if the host goes up.
- There is the network or firewall problem that causes the NIP application cannot connect to the server. Please contact your network administrator to verify this.
Q: When running the application, it shows "Error initializing channel: errorId=-1 text=Connection refused: no further information" and "login failed (timed out after waiting 45000 milliseconds) for [host]:[port])"
Jun 08, 2018 9:47:22 AM com.refinitiv.ema.access.ChannelCallbackClient reactorChannelEventCallback
WARNING: loggerMsg
ClientName: ChannelCallbackClient
Severity: Warning
Text: Received ChannelDownReconnecting event on channel Channel_3
RsslReactor Channel is null
Error Id 0
Internal sysError 0
Error Location Reactor.processWorkerEvent
Error text Error initializing channel: errorId=-1 text=Connection refused: no further information
loggerMsgEnd
Jun 08, 2018 9:47:26 AM com.refinitiv.ema.access.OmmBaseImpl handleLoginReqTimeout
SEVERE: loggerMsg
ClientName: Provider_1_1
Severity: Error
Text: login failed (timed out after waiting 45000 milliseconds) for 192.168.27.11:14003)
loggerMsgEnd
login failed (timed out after waiting 45000 milliseconds) for 192.168.27.11:14003)
A: the possible causes of the problem are:
- ADH goes down or the host:port set in the application source code is wrong. Please contact Market Data administrator to verify the correct host/IP of ADH and if ADH goes up.
- The firewall blocks the messages between the NIP application and ADH. Please contact your network administrator to verify and fix this.
Q: When running the application, it shows "Login rejected. Non-consumer attempting login."
Jun 08, 2018 9:50:52 AM com.refinitiv.ema.access.LoginCallbackClient rdmLoginMsgCallback
SEVERE: loggerMsg
ClientName: LoginCallbackClient
Severity: Error
Text: RDMLogin stream was closed with status message
username user
usernameType 1
State: Closed/Suspect/Usage error - text: "Login rejected. Non-consumer attempting login."
loggerMsgEnd
A: The application connects to ADS(the default port is 14002) instead of ADH(the default port is 14003). ADS is for consumer applications but it does not for NIP applications. Please change the port in the source code to be 14003. If the NIP application still cannot connect to ADH, please contact Market Data administrator who can provide you with the host/port of ADH.
Q: NIP application can connect and publish to ADH, but the consumer application does not receive data and shows "Service name of [service_name] is not found."
Item Name: 0#NEWSBL
Service Name: NI_PUB
Item State: Closed / Suspect / None / 'Service name of 'NI_PUB' is not found.'
A: It means the service provided by NIP application(in the example application is NI_PUB) does not match with the service defined in the ADH configurations. Please contact your Market Data administrator who can help you to verify this and configure ADH and ADS servers for your NIP service.
Summary
After finishing this article, you will understand more about Symbol List, NIP application and how to develop a NIP application using EMA to publish symbol list. Also, you can modify this application to serve your requirements using EMA easier and faster including solving the common problems. If you do not have ADH and ADS servers, please contact Refinitiv Account team for the process and details. If you/your Market Data administrator requires any ADH and ADS servers assistance, You can contact the Refinitiv Real-Time Distrubtion support team directly by submitting your query to Get Support of MyRefinitiv.
For further details, please check out the following resources:
