Scala Overview

Scala was developed specifically with the aim of being a better language than Java. Scala is a modern multi-paradigm programming language designed to express common programming patterns in a concise, elegant, and type-safe way. It smoothly integrates features of object-oriented and functional languages. Scala is designed to interoperate well with the popular Java Runtime Environment (JRE). In particular, the interaction with the mainstream object-oriented Java programming language is as smooth as possible. Hence, you can write a Scala application to call any Java interfaces e.g. EMA Java. Scala syntax is also closed to the Java language so the learning curve for the Java developers is minimized. Therefore, you can develop EMA Java application in Scala within minimum time to create a better Java application e.g. extremely compact and easier to write. For more details of Scala including Documentation, Download, Online Courses, please refer to Scala page.

Non-Interactive Providers (NIP) Overview

If you would like to expose a specific data service to consumer applications, building a Provider application should be considered. Provider applications are able to connect to a ADH server to leverage its real-time and streaming distribution system to publish data. This is an efficient solution to make your own set of capabilities (e.g. content, workflow, etc.) available to your consumer applications.

Non-Interactive Providers are not solicited by infrastructure requests but instead, they publish information (e.g. Available services, data items…) following their own timing. In this model Real-Time Distribution System caches the published information and serves it to the demanding consumer applications as shown in the NIP Feature Diagram below:

Therefore, NIP can publish level1 and level 2 data into the ADH cache without needing to handle/wait for requests from any consumer applications. Whenever a consumer application sends a request for data to Advanced Distribution Server (ADS) which is a component of Real-Time Distribution System,  ADS forwards the request to ADH. Finally, ADH provides data kept in the cache which is received from NIP previously to the consumer application. For more details about Non-Interactive Provider, please refer to API Concepts Guide

Connection Failover

To use this feature, you just specify a list of failover servers via the ChannelSet configuration parameter. This parameter specifies an ordered list of Channel to which NIP will attempt to connect, one at a time, if the previous one fails to connect. When the NIP is disconnected from the current ADH e.g. ADH goes down or the network problem occurs, EMA will automatically retry to connect and log in to the next ADH specified in the ChannelSet.

Recovering to Publish Data

Unlike Connection Failover which is performed by EMA automatically, the NIP needs to perform this process by itself after EMA recovers the connection to another ADH successfully. This is illustrated in ex360_MP_ConnectionRecovery shipped with Real-Time SDK Java Package as well.  We will explain this process in details later in the article.

Prerequisites

1. Download the latest Real-Time SDK Java Package
2. Declare the service sent by the NIP in  ADH configuration files of both Real-Time Distribution sites for connection failover feature. This process must be executed by your Market Data administrator team. For information, please refer to ADH Configuration for NI Providers which explains the steps involved in this process. If Market Dat administration team requires any assistance regarding configuring ADH/ADS server, they can submit the query to Real-Time support team directly via LSEG Support. In this article, we will use the service named NI_PUB which is one of the default service configured in EmaConfig.xml shipped with Real-Time SDK Package. 
3. In this article, I will use the default configuration for NIP which is Provider_1 node in EmaConfig.xml file shipped with Real-Time SDK package. Provider_1 node consists of all ADH server hosts and ports where NIP will connect to and source directory info which EMA will publish to ADH automatically. To make EMA reads the file, put the configuration file in the working directory. Then, modify it to:
   • Under NiProvider named Provider_1, replace <Channel value="Channel_3"/> with <ChannelSet value="Channel_3, Channel_6"/> to set EMA performs connection failover according to a list of failover servers. For example, when ADH specified in Channel_3 goes down, EMA tries to connect to ADH specified in Channel_6.
   • Modify Host and Port of Channel_3 used by the default Provider_1 to be your primary ADH
   • Copy Channel_3 and rename it to be Channel_6, modify Host and Port to be your secondary ADH
   • By default, Service NI_PUB supports the capabilities: 6(Market Price) and MMT_MARKET_BY_ORDER. Hence, add the remaining level 2 data which are MMT_MARKET_BY_PRICE and 9(Market Maker)

The EMA configuration should be like the below:

In the example configuration above, EMA tries to connect to the first channel, Channel_3 (ADH on IP 192.168.27.11), specified in the ChannelSet. Whenever EMA fails to connect to Channel_3, it will try to connect to the next Channel, Chanel_6 (ADH on IP 192.168.27.12).

 The Non-Interactive Provider Sequence Diagram

The figures below are the NIP sequence diagram which shows the interaction between NIP and ADH. The diagram also covers all tasks performed by NIP which we will develop as mentioned in the Introduction section. 

A. The normal situation without any failures

For more details of permission data (Dacs Lock) mentioned in the optional step 5^th, please refer to DACS Lock Developers Guide

B. Failure situation that makes NIP disconnects e.g. ADH goes down, the network problem occurs.

Note: A green arrow and orange arrow represents NIP task and ADH task respectively while the gray arrow represents the EMA task. 

To make a simple NIP and focus on the features and the tasks, we let EMA sends automatically a source directory info configured in the EmaConfig.xml file. Alternatively, this can be performed programmatically by the application itself as demonstrated in ex300_MP_Streaming shipped with Real-Time SDK Java Packages  

Source Code

This section is to explain how to develop a NIP application using EMA Java in Scala language according to The Non-Interactive Provider Sequence Diagram section. Our example application consists of 2 categories:

• Data classes which are MarketPrice, MarketByPrice, MarketByOrder, and MarketMaker. Each class contains all the relevant information regarding an item of each domain type. They are used as the source of canned data for the NIP. Each class has the methods to generate the content of the payload contained in a refresh and an update message according to its domain type i.e. Market Price, Market By Order, Market By Price and Market Maker
• NiProvider object and AppClient class. NiProvider is the main class of this application. It publishes data generated by the methods in the data classes, MarketPrice, MarketByPrice, MarketByOrder and MarketMaker to ADH according to the input domainType application’s parameter. NiProvider object performs all tasks shown in The Non-Interactive Provider Sequence Diagram section(green arrows). AppClient class receives and updates the login/connection events to update the ADH connection status if it is ready to be published.  The status is used by NiProvider object.

The overview steps to develop the application are listed below:

1. Create MarketPrice class in MarketPrice.scala file to generate Market Price data as an example shown below: 

2. Create MarketByOrder class in MarketByOrder.scala file to generate Market By Order data as an example shown below: 

3. Create MarketByPrice class in MarketByPrice.scala file to generate Market By Price data as an example shown below: 

4. Create MarketMaker class in MarketMaker.scala  file to generate Market Maker data as an example shown below:

5. Create NiProvider.scala  file which contains following:

               A.NiProvider object contains the main (args:Array[String]) which extracts the following program parameters from the command line:

• -nipNode <NiProviderName>. The Name of NiProvider node in EmaConfig.xml. The default is Provider_1. The node contains the IP and port of ADH server(s) where the application connects to.
• -service <service_name>. The Service name. The default is NI_PUB.
• -user <name>. The Name of the application user. The default is user.
• -domainType <domain_type>. The Domain Type of the published item. The default is MARKET_PRICE. The valid Domain Types supported by this application are MARKET_PRICE, MARKET_BY_ORDER, MARKET_BY_PRICE, MARKET_MAKER.
• -itemName <a RIC>. The published RIC name. The default is DUMMY.N.
• -updateTimes <number_updates>. The number of updates to be published. The default is 10 updates.
• -updateInterval <update_interval_in_sec>. The Update interval in seconds. The default is 5 seconds.
• -refreshPE <a PE>. a PE for refresh messages. This requires dacsServiceId. The default is no PE.
• -dacsServiceId <a service id>. The serviceID to generate DACS lock for the PE. The default is no serviceID.

Then, the object performs the tasks according to  The Non-Interactive Provider Sequence Diagram section (green arrows). It calls the method in AppClient class to get the current ADH connection status. The snipped source code in the main (args:Array[String]) method:

B.AppClient class implementing OmmProviderClient which contains the callback methods to receive and process any messages e.g. refresh via onRefreshMsg(..) method, status via onStatusMsg(..) method. AppClient checks if an event informs the ADH connection is ready to be published or not and updates the connection status. The class exposes a method to get the status used by NiProvider object. The snipped source code in AppClient class:

You can download the complete NIP Java application source code explained above at LSEG-API-Samples/Article.EMA.Scala.NonInteractiveProvider GitHub page.

Build and Run the application

EMA, ETA, and EMA's dependency library files are required to build and run the EMA Java NIP application. Fortunately, all files are shipped with the Real-Time SDK Java package. Hence, you just set the CLASSPATH to all of them before building and running the application.

The following example command lines show how to set the CLASSPATH, build and run the application on Windows: 

1. Set the classpath to EMA, ETA, and EMA's dependency library files. For example:

2. Compile the application using scalac command: 

3. Run the application:

• Include the path containing the Consumer class files generated from step 2 to the classpath:

• Run the application using scala command. For example:

The command above is to log in with the user named pimchayaProv to publish the default domain type (Market Price) with the item named NIP.MP to the default service named NI_PUB. Specify a refresh message contains the permission data consisting of PE 351 and the service Id 4811, Hence, the field named PROD_PERM(PERMISSION field which field id is 1) with value 351(PE) is added. Moreover, the application publishes in the default rate which is 10 updates every 5 seconds.

The Example Result

When you run the NIP application to publish Market Price, you should see the messages like below:

While the NIP application is running, you can run the EMA Java Consumer named ex100_MP_Streaming to verify the NIP's service and its Market Price data. The Consumer application must connect to ADS (one component of Real-Time Distribution System) connecting to ADH which the NIP connect to. According to the example command line, you need to modify Consumer.java to request to the service named NI_PUB and the item named NIP.MP. The example result shown in ex100_MP_Streaming:

Troubleshooting

• The NIP application shows "<user>, unknown to system."

Resolution: The application logs in with the user who does not have the permission to access ADH. Please contact your infrastructure administrator or the Account team to ask for a valid username.

• The NIP application shows“Attempt to submit initial RefreshMsg with a service name of NI_PUA that was not included in the SourceDirectory. Dropping this RefreshMsg.”

Resolution:  The service specified in the command line running NIP does not match with the service’s name of <Directory> node which the application publishes to ADH.

For example, In EMAConfig.xml

Hence, you should use -service NI_PUB in the command line running the application.

• The Consumer application shows "Capability not supported" 

Resolution: The consumer application requests an item from the NIP service which does not support the domain type of the item. In the example above, the service named NI_PUB does not support Market Maker domain type of the item named NIP.MM. Please make sure that <CapabilitiesEntry value="9"/> which is Market Maker has been added in NI_PUB service defined in EMAConfig.xml read by NIP.

• The Consumer application shows "Service name of <service> is not found."

Resolution: The consumer application requests an item from the wrong service (NI_PUBX); NIP does not provide this service name. Please recheck the service name is correctly typed in with your infrastructure administrator or the Account team.

• The NI application shows“OmmNiProviderConfigImpl::providerName parameter [Provider_1] is an non-existent niprovider name

Resolution: EMA cannot find Provider_1 configuration node because EMAConfig.xml containing find Provider_1 configuration node is not in the application run directory. Please put EMAConfig.xml in the application run directory.

Summary

After finishing this article, you will understand more about EMA, NIP application and how to develop a NIP application using EMA to publish data in Scala language. Also, you can modify this application to serve your requirements using EMA easier and faster. Moreover, the article includes the common problems and how to solve them. If you do not have Real-Time Distribution System (ADH and ADS), please contact the Account team for the process and details. If you/your Market Data administrator requires any ADH and ADS assistance, you can contact the Real-Time support team directly by submitting your query to LSEG Support.