ETA Tutorial 4 - Retrieving and decoding level 1 content

Download tutorial source code

Click here to download

Last update Dec 2020
Environment Windows, Linux
Compilers JDK 1.7 or greater
Prerequisite ETA Tutorial 3 - Establishing a session with a Provider

Introduction​

The goal of this tutorial is to extend Tutorial 3 to request and retrieve level 1 (MarketPrice) data from the OMM-based data Provider. To support level 1 content, new class modules have been added to this tutorial to enforce a modular environment allowing easier management, and support for additional functionality.

Description

In this tutorial, we introduce two new classes that will both play a role in decoding the level 1 data responses from the OMM-based data Provider.

  • basicDictionaryHandler
    Data messages received from the OMM-based Provider are packaged as RWF (Reuters Wire Format) messages. The RWF is a low-level, compact protocol that requires specific decoding in order to be properly interpreted and used within an application. To support this decoding, an application is expected to load a dictionary of field definitions and utilize these definitions, via ETA method calls, to decode RWF messages.
  • basicMsgHandler
    The RDM (Reuters Domain Messages) defines multiple message models that are typically used within a Consumer application. Within this tutorial, we will be requesting for the level 1 MarketPrice message model which contains a well-defined message structure. Because our RDM models utilize well-defined structures such as FieldLists, Maps, etc., our handler only needs to understand the structure to properly process the message.

Implementation Overview - Preparing, requesting, and parsing level 1 data

Prior to requesting for level 1 data, we must ensure we can properly decode the RWF messages provided by the OMM-based Provider by first loading a dictionary. As previously mentioned, the dictionary represents the field definitions. Once loaded, we can safely interpret and decode the fields of data we receive upon market data updates from the Provider.

Dictionary

A Consumer application has two choices to load a dictionary. Either choice has its advantages but once a dictionary is loaded, we can achieve the functionality of decoding RWF messages.  The two choices are:

  • Loading the dictionary from local files
  • Downloading the dictionary from the OMM-based Provider

Refer to the Load or Download Necessary Dictionary Information section defined within the Developers Guide for more information. 

The following code snippet outlines the logic to handle both mechanisms of loading a dictionary:

    	
            

private void init()

{

    ...

 

    // Determine where we are getting our dictionary from - local or server

    consumerRole.dictionaryDownloadMode(dictionaryHandler.loadLocalDictionary() ? DictionaryDownloadModes.NONE :

                                                                       DictionaryDownloadModes.FIRST_AVAILABLE);

 

    ...

 

}

From the above code segment, we instruct the reactor to determine how we want to load the data dictionary.  

Loading the dictionary from local files

The above method call loadLocalDictionary will return a value of true if the dictionary files are available within the local file system.  If so, this method will load the dictionary files and the enumeration value NONE is assigned to the dictionaryDownloadMode which instructs the reactor to not request the dictionary from the Provider. This logic will be demonstrated within ETA Tutorial 5 - Retrieving and decoding level 2 content.

Note: the tutorial will be looking for the local dictionary files within the current directory where you run the application.

Downloading the dictionary from the Provider

The above method call loadLocalDictionary will return a value of false if the dictionary files are not available within the local file system. As a result, the enumeration value FIRST_AVAILABLE is assigned to the dictionaryDownloadMode which instructs the reactor to request the dictionary from the Provider. This logic will be demonstrated in this tutorial.

Note: The dictionary download procedure is mostly handled via the ETA Reactor. In this case, the dictionary download is completely dependent on the success of the Directory response we utilized in ETA Tutorial 3 - Establishing a session with a Provider.  The Directory response contains the details of the dictionary details sent down to the Provider.

As outlined above, the decision to download or load locally is driven by the loadLocalDictionary method defined within basicDictionaryHandler.java.

    	
            

// The names of the dictionary files we expect if we want to load locally (file system) 

private final String FIELD_DICTIONARY_FILE_NAME = "RDMFieldDictionary"; 

private final String ENUM_TABLE_FILE_NAME = "enumtype.def"; 

 

// Dictionary of fields 

private DataDictionary dictionary = CodecFactory.createDataDictionary(); 

 

public boolean loadLocalDictionary() 

    boolean fieldDictionaryLoadedFromFile = false, enumTypeDictionaryLoadedFromFile = false; 

    dictionary.clear(); 

    if (dictionary.loadFieldDictionary(FIELD_DICTIONARY_FILE_NAME, error) < 0) 

    { 

        System.out.println("Unable to load field dictionary. Will attempt to download from provider.\n\tText: " 

                                                        + error.text()); 

    } 

    else 

    { 

        fieldDictionaryLoadedFromFile = true; 

    } 

 

    if (dictionary.loadEnumTypeDictionary(ENUM_TABLE_FILE_NAME, error) < 0) 

    { 

        System.out.println("Unable to load enum dictionary. Will attempt to download from provider.\n\tText: " 

                                                        + error.text()); 

    else

    { 

        enumTypeDictionaryLoadedFromFile = true; } 

        if ( !fieldDictionaryLoadedFromFile || !enumTypeDictionaryLoadedFromFile ) 

           dictionary.clear(); 

        else 

           System.out.println("\nSuccessfully loaded local dictionaries"); 

        return( fieldDictionaryLoadedFromFile && enumTypeDictionaryLoadedFromFile ); 

    }

There are 2 local files that make up the dictionary, i.e.

  • RDMFieldDictionary
    Contains all the known fields, types and sizes required to decode.
  • enumtype.def
    Contains enumeration definitions for those fields with an Enum type defined withing the RDMFieldDictionary file.

If the local dictionary files are available, the ETA methods loadFieldDictionary and loadEnumTypeDictionary will perform the action of loading the dictionary. If no local dictionaries are available, we return no success, or false, and within the calling method, we instruct the ETA Reactor to attempt to download from the OMM-based Provider.

In the event where we instruct the reactor to fetch the dictionary files from the Provider, this will result in dictionary events.  These events define the details for the complete dictionary downloaded from the Provider.  Upon a dictionary event, we handle the event processing within the following callback:

    	
            

@Override 

public int rdmDictionaryMsgCallback(RDMDictionaryMsgEvent event) 

    // Process the dictionary details 

    return( dictionaryHandler.processDictionaryResponse(event) );

}

The callback above instructs a dictionary processing action via the processDictionaryResponse method. Although we could have housed the entire processing within the basicConsumer.java module, the intention was to offload the functionality into the separate class basicDictionaryHandler for purposes of modularity.

While the processing details are important, the codebase and the decoding of dictionary definition elements are beyond the scope of this tutorial. The basic processing was derived from the ValueAdd example consumer which will act as a good reference for additional processing requirements and explanation.

Level 1 data - MarketPrice

Once we have established our dictionary, we are ready to request for data, in this example for instrument code BB.TO (Blackberry on the Toronto exchange). In the first tutorial, we had setup our series of event callbacks based on the ConsumerCallback interface requirements. Now that we plan on requesting for data, we can put in place the processing logic required to handle data events within these callbacks. At startup, the ValueAdd components will run through a cycle to determine when we are ready to request for data from our Provider. Once the consumer role successfully connects, establishes a session and loads a dictionary, we will be presented with a CHANNEL_READY event within our reactorChannelEventCallback:

    	
            

// Service name

private static final String serviceName = "ELEKTRON_AD";

 

// Item name

private static final List itemNames = Arrays.asList("BB.TO");

 

// Domain Type

private static final int domainType = DomainTypes.MARKET_PRICE;

 

...

 

@Override

public int reactorChannelEventCallback(ReactorChannelEvent event)

{

    switch(event.eventType())

    {

        case ReactorChannelEventTypes.CHANNEL_UP:

        {

            ...

}

 

        case ReactorChannelEventTypes.CHANNEL_READY:

        {

            ...

 

            // The Reactor has successfully completed the main session establishment with our provider

            if ( directoryHandler.isRequestedServiceAvailable() )

            {

                if ( msgHandler.sendItemRequest(event.reactorChannel(), itemNames,

                                                       directoryHandler.getService(),

                                      domainType, errorInfo) != ReactorCallbackReturnCodes.SUCCESS )

                {

                    System.out.println(errorInfo.error().text());

                }

            }

            else

                System.out.println("Unable to request data.  Service: " + serviceName + " is unavailble");

 

            break;

        }

        ...

    }

}

Within the existing callback reactorChannelEventCallback, we are using the CHANNEL_READY event to trigger the action of requesting for our data. The request of data in our example is to open a subscription or specify a streaming request. A streaming request will result in an initial image (REFRESH) and optionally followed by 1 or more update (UPDATE) messages, depending on the time of day and the current market conditions. Based on the REFRESH event, the initial image contains the latest up-to-date values known for the specified item. The updates represent only those fields that result from changes within the market.

To issue our streaming request, we are calling the newly defined method sendItemRequest. Defined within the file basicMsgHandler.java, this method is designed as a generic item request function that requires 3 key pieces of data, i.e.

  • itemNames
    A list of data items to request. Eg: BB.TO (Blackberry on the Toronto exchange)
  • serviceInfo
    The serviceInfo corresponds to a specific entity contained within the Directory Service received from the Provider.  While the entity contains many service attributes such as the name and ID, it is the ETA libraries that require this ID on all subsequent requests to that service.  The serviceInfo name was utilized to identify which service, ELEKTRON_AD, we are interested in.
  • domainType
    The RDM domain type. In this tutorial we request for the level 1 data MARKET_PRICE but future tutorials will utilize additional models. The RDM domain types supported by the Provider were provided within the Directory response we covered within Tutorial 3.

To process our message response:

    	
            

@Override

public int defaultMsgCallback(ReactorMsgEvent event)

{

    return( msgHandler.generalMsgProcessing(event, dictionaryHandler.getDictionary()) );

}

The basicMsgHandler.java file contains many processing functions to manage message encoding, decoding and parsing. In the above code segment, we are executing the generalMsgProcessing method which is designed to handle the decoding of many message types. As mentioned in the Description section above, the message processing within this application intends to keep processing generic based on the structure of the messages, i.e. FieldLists, Maps, etc. as opposed to the model type of the message, i.e. MarketPrice, Symbol List, MarketByPrice, etc. In doing so, we can simplify the processing and reduce the codebase to process multiple message models. The following function outlines the general structure:

    	
            

int processRDMResponse(Msg msg, DecodeIterator dIter, DataDictionary dictionary)

{

    int retval = CodecReturnCodes.SUCCESS;

 

    ...

 

    switch (msg.msgClass())

    {

        case MsgClasses.REFRESH:

        case MsgClasses.UPDATE:

        {

            switch (msg.containerType())

            {

                case DataTypes.FIELD_LIST:

                    retval = decodeFieldList(msg, dIter, dictionary);

                    break;

                default:

                    output.append("Unable to process RDM msg with container type: " + msg.containerType());

                    break;

            }

            break;

 

            ...

 

        }

 

        ...

 

        return( retval );

    }

}

In the code segment above, our level 1 Market Price data is delivered within a FIELD_LIST message structure. As such, we walk through the list and extract the elements and decode the fields, using our loaded dictionary. The fields are simply echoed to the display to verify processing. As we encounter new message structures, we include that logic in the above to expand the processing. While this processing is important, it is beyond the scope of this tutorial. The basic processing was derived from the ValueAdd example consumer which will act as a good reference for additional processing requirements.

Setup and Configuration

Refer to Setup and Configuration section within the ETA Tutorial 2 - Establishing a connection to a Provider prior to building and running this tutorial.

Build and run

Refer to the Build and run section within the ETA Tutorial 1 - Creating a starter consumer application for instructions to successfully execute this tutorial.

Eg (Windows):

> buildConsumer 4
> runConsumer 4

In our execution, we are downloading the dictionaries from the OMM-based Provider. You can see the receipt of the dictionaries; our request to subscribe to the item BB.TO and the corresponding response.

Note: We issued a streaming request which will result in an initial refresh followed by updates showing change based on market events.

    	
            

Running tutorial 4...

Enterprise Transport API (ETA), Java Edition, LibraryVersionInfo

        productVersion: 3.6.0.0

        productInternalVersion: etaj3.6.0.L1.all.rrg

        productDate: Fri Oct 16 12:25:01 CDT 2020

basicConsumer initializing...

Unable to load field dictionary.  Will attempt to download from provider.

        Text: Can't open file: RDMFieldDictionary

Unable to load enum dictionary.  Will attempt to download from provider.

        Text: Can't open file: enumtype.def

 

Connection up!  Registering our connection within our event loop

 

Received Login Refresh for Username: U8007876

Received serviceName: DDN_SSL. State:           StateFilter:

                        ServiceState: 1

                        AcceptingRequests: 1

 

Received serviceName: ELEKTRON_AD. State:               StateFilter:

                        ServiceState: 1

                        AcceptingRequests: 1

 

Found service of Interest: ELEKTRON_AD.  Using serviceId: 356

 

Received serviceName: DDN_RDF. State:           StateFilter:

                        ServiceState: 1

                        AcceptingRequests: 1

 

Received serviceName: EZD_RSSL. State:          StateFilter:

                        ServiceState: 1

                        AcceptingRequests: 1

 

Received serviceName: EZD_SSL. State:           StateFilter:

                        ServiceState: 1

                        AcceptingRequests: 1

 

Received Dictionary Response: RWFFld

Field Dictionary complete.

Received Dictionary StatusMsg

Received Dictionary Response: RWFEnum

EnumType Dictionary complete.

Received Dictionary StatusMsg

Channel is ready. Subscribing to item(s): [BB.TO]

 

BB.TO

DOMAIN: MARKET_PRICE

REFRESH: State: Open/Ok/None - text: "All is well"

        1/PROD_PERM: 84

        2/RDNDISPLAY: 65

        3/DSPLY_NAME: BLACKBERRY LTD

        4/RDN_EXCHID: TOR(10)

        6/TRDPRC_1: 8.755

        7/TRDPRC_2: 8.75

        8/TRDPRC_3: 8.75

        9/TRDPRC_4: 8.75

        10/TRDPRC_5: 8.75

        11/NETCHNG_1: -0.115

        12/HIGH_1: 8.95

        13/LOW_1: 8.74

        14/PRCTCK_1: Þ(1)

        15/CURRENCY: CAD(124)

        16/TRADE_DATE: 14 JUN 2016

        18/TRDTIM_1: 17:59:00:000:000:000

        19/OPEN_PRC: 8.87

        21/HST_CLOSE: 8.87

        22/BID: 8.75

        23/BID_1: 8.75

        24/BID_2: 8.75

        25/ASK: 8.76

        26/ASK_1: 8.76

        27/ASK_2: 8.76

        28/NEWS: YYYY

        29/NEWS_TIME: 13:40:52:000:000:000

        30/BIDSIZE: 12.0

        31/ASKSIZE: 215.0

        32/ACVOL_1: 589640.0

        34/EARNINGS: -1.30553

        35/YIELD: 

        36/PERATIO: 

        ...

        32743/ACVOL_UNS: 589640

 

 

------------------------------------------------------------------------------------

DOMAIN: MARKET_PRICE

UPDATE: QUOTE

        3887/SEQNUM_QT: 443816.0

        1025/QUOTIM: 17:59:16:000:000:000

        22/BID: 8.75

        25/ASK: 8.76

        30/BIDSIZE: 12.0

        31/ASKSIZE: 213.0

 

 

------------------------------------------------------------------------------------

DOMAIN: MARKET_PRICE

UPDATE: QUOTE

        3887/SEQNUM_QT: 443817.0

        1025/QUOTIM: 17:59:16:000:000:000

        22/BID: 8.75

        25/ASK: 8.76

        30/BIDSIZE: 12.0

        31/ASKSIZE: 210.0

 

 

------------------------------------------------------------------------------------

References

For more information, refer to the ETA Java Development Guides.