Last Updated: June 2024
Knowledge Prerequisite – Must have a basic understanding of what Real-Time Streaming data is.
LSEG offers several choices when it comes to consuming Real-Time streaming data – from a lowest-level ultra high-performance Transport level API to a highest-level helper library in Python & C# (and soon TypeScript).
Using the decision tree below in conjunction with the notes contained herein, should help you narrow down the choice of the most likely candidate(s) to meet your requirement.
Primary Determining Factor
Based on my experience of working with a variety of client developers, the primary deciding factor is usually one of the following:
- Performance requirement - the implementation requires high/very high performance e.g. the ability to consume several (hundreds of) thousands of streaming updates a second - or lowest possible latency for algorithmic trading
- Programming language – the development team are restricted in the choice of language(s) they can code in
Performance
As you can see from the diagram, the factor does not necessarily have to be mutually exclusive e.g. if you want the highest performance and can code in C#, Java, C++, or C then all paths lead you to the Enterprise APIs that form the Real-Time SDKs (RTSDKs).
Conversely, for a specialized use case like algorithm trading (which demands the lowest possible latency), the path will lead you to the only choice of the Enterprise Transport API (ETA) which
- Is Open source, highest performance, lowest latency, and the foundation of the Real-Time SDK.
- Allows applications to achieve the highest throughput, lowest latency, low memory utilization, and low CPU utilization
- Available in C, Java, and C# flavors.
Language choice
If outright performance is not the overriding factor, then you have several choices available in terms of language and API/Library choice.
If you can code in Java or C++ or C#, the Enterprise Message API(EMA) is probably the best choice as an:
- Open source, data-neutral, multi-threaded, ease-of-use API (a wrapper around ETA)
- Offering a High Level of performance suitable for 95% of usage scenarios
However, if your language of choice is Python, TypeScript, or .NET then you have the following options:
- WebSocket API – raw standard WebSocket-based interface to streaming data
- LSEG Data Library (RD Library) – high-level ease-of-use wrapper for the Websocket API
If you are wondering why you may use the Data Library over the raw WebSocket; the library takes care of various administrative tasks including login, authentication, and connection management.
NOTES regarding RD Library:
- Whilst alpha versions of Python and .NET RDP Library are still available, they have been redesigned and renamed to RD Library
- The production-ready version of RD Library is now available in Python and .NET
- Beta versions of RD Library TypeScript is now available
For any other languages that support standard WebSocket creation and JSON data parsing, you can use the raw WebSocket API mentioned above.
Whilst using the WebSocket API requires the developer to handle things like login, connection handling & item recovery, etc - we do provide basic examples in several languages demonstrating these techniques.
RTSDK or WebSocket
If you are Java, C/C++, or C# developer, the Enterprise Message API (EMA) is the easiest and recommend API for connecting and consuming data to the Real-Time Optimized (RTO) and the Real-Time Distribution System (RTDS).
The EMA has high-level and easy-to-learn API interfaces supports file/programmatic configurations, and provides session and subscription management for the applications while still having decent performance for most use cases. The RSSL connection is propriety TCP-based connection that is designed for data distribution.
The WebSocket API is designed for easy integration and supports various programming languages. The protocol is based on the open standard WebSocket connection and JSON message format. However, the RSSL connection provides a better performance for data distribution (please see more detail on the Data Format section below).
Performance comparison with legacy APIs
API |
Thread Safety |
Throughput |
Latency |
Memory Footprint |
Transport API |
Safe and Aware |
Very High |
Lowest |
Lowest |
Message API |
Safe and Aware |
High |
Low |
Low |
RFA |
Safe and Aware |
High |
Low |
Medium |
| SFC C++ SFC Java (JSFC) |
None |
Medium |
High |
Medium-High |
WebSocket API |
* |
* |
* |
* |
* performance achievable with WebSocket API would depend considerably on the programming language, WebSocket library used, etc.
The Real-Time SDKs Performance Test Summary article is available if you are interested in a performance difference between ETA and EMA (The RTSDK C# is not included in the test report yet).
Data Formats
Both the Enterprise Transport and Message API use our Open Message Model (OMM) at the application level and the binary Refinitiv Wire Format (RWF) at the transport level.
OMM provides data building blocks, a bit like ‘Lego’ but for data and you can use the ‘Lego’ to build rich data models to represent things like Full-depth order books or Yield curve data. This contrasts with the flat ‘Field-Value’ pair representation used by our legacy APIs.
RWF uses binary encoding to optimize network distribution – reduced message sizes lead to an increase in total throughput and a decrease in overall latency. Again, this contrasts with the delimited ASCII-based representation used by the legacy MarketFeed format.
The Websocket API and the RDP Library use the human-readable JSON format for outgoing requests and incoming response messages.
Connectivity
API/Library |
Deployed Server (RTDS) |
Cloud |
Desktop |
Transport API |
X |
X |
|
Message API |
X |
X |
|
RD Library |
X |
X |
X |
WebSocket API |
X |
X |
In terms of connectivity, all the above APIs/libraries will allow you to consume streaming data from our cloud-based Real-time services (such as Real-Time Optimized) and your deployed Real-Time Distribution System (formerly known as TREP).
Additionally, the RD library will also allow you to connect to an instance of the Eikon or Workspace application running on the same PC.
Data Contribution
If you wish to implement perform Contributions (also known as Inserting/Posting) in your application, the following choices are currently available:
API |
Deployed Server (ADS) |
Cloud |
Transport API |
X |
X |
Message API |
X |
X |
WebSocket API |
X |
X |
All three of the above APIs will allow you to contribute data via your deployed ADS server or directly & securely over the internet to the Contribution Channel.
Data Publication
API |
Interactive Provider (Managed Publisher) |
Non-Interactive Provider |
Transport API |
X |
X |
Message API |
X |
X |
WebSocket API |
X* |
*Coming soon
Next Steps / Additional Reading
Once you have narrowed down your choice of API - please refer to the additional articles and links to tutorials etc on the upper right-hand side of the article.
If you are concerned about the performance characteristics and whether a particular API will meet your requirements, the best option is to run some performance tests in your own environment using your own typical data universe - to get an indication of what you can expect in terms of real-life performance.
The RTSDK includes source code for performance test utilities - for both ETA and EMA located in the PerfTools folder of each API. Using these examples as a basis for any testing you wish to perform should make any such task easier. The Real-Time SDKs Performance Test Summary article is available if you are interested in a performance difference between ETA and EMA (The RTSDK C# is not included in the test report yet).
For the WebSocket API, you may find my Python Websocket TestClient helpful example - as a starting point for any test script you wish to create. There is also a similar example for the RDP Python library.
Testing with Docker Images / AWS EC2
You will also note from the above links panel that we have made available Docker Images for RTSDK and Websocket API. In addition to this, we also provide an AWS AMI containing the RTSDKs and Websocket API. Hopefully, this should make it easier for you to experiment with the various APIs.
Conclusion
As the article outlines, we have a range of APIs that can be used for real-time streaming data and the exact one you choose will depend on various factors - which I hope I have covered in this article. However, if you do need further guidance, feel free to post your query/concerns on our Developer Forum and/or reach out to your LSEG Account representative to arrange a further discussion.
