Author:
Introduction
Refinitiv Data Platform (RDP) is our cloud-enabled, open platform, that brings together content, analytics and proprietary, customer and third-party data distribution and management technology. It provides simple web-based API access to a broad range of content. Therefore, any programming languages which support web-based API can connect and consume data from RDP.
This article demonstrates how to develop a Java application to consume news headlines and stories from RDP Alerts API. The RDP Alerts API allows users to setup subscriptions for different types of content, such as news, and research. The full code is available in GitHub. Retrieving News Headlines or Stories from Refinitiv Data Platform Alerts Endpoints with Java
Delivery mechanisms
The data from an RDP service can be delivered in variety of mechanisms, depending on content set, such as Request/Response, Alerts/Messages-Services, Bulk, and Streaming. The RDP Alerts API uses the Alerts/Messages-Services delivery mechanism to delivery updates (alerts) to an application based on a subscription criteria. The Alerts/Messages-Services uses the Amazon SQS to deliver asynchronous updates (alerts) to a subscribed application.
The following Java libraries are used to retrieve data from the RDP Alerts/Messages-Services delivery mechanism.
| Package | Maven Repository |
| software.amazon.awssdk.sqs | https://mvnrepository.com/artifact/software.amazon.awssdk/sqs |
| com.konghq.unirest-java | https://mvnrepository.com/artifact/com.konghq/unirest-java |
The sqs is the AWS Java SDK for Amazon SQS module used for communicating with Amazon Simple Queue Service and the unirest-java package is the simplified, lightweight HTTP client library.
Steps to subscribe to alerts
This section demonstrates steps to develop a Java application to subscribe to news headlines and stories from RDP alerts API. The application implements the workflow mentioned in the Understanding the Alerts delivery mechanism in Refinitiv Data Platform article. It is developed with JDK 11.
There are six steps to consume news headlines or stories from RDP alerts API.
1. Login
Login is the first step for all RDP API requests. RDP entitlement check is based on OAuth 2.0 specification which requires the following parameters.
| Parameter | Definition |
| Username | The username required to access RDP request-response API (typically email) The username required to access RDP request-response API (typically email) |
| Password | The password for username above -- used to get access token |
| ClietnID | A unique ID defined for an application making the request |
A successful authentication response from a server contains an access token and refresh token. The access token is used to invoke other REST data API calls. The refresh token is used to get the next access token when the current access token is expired.
The following Java code shows how to use the unirest-java library to get an access token.
import kong.unirest.*;
...
public String GetToken(String username, String password, String clientId) {
try {
HttpResponse<JsonNode> response = Unirest.post("https://api.refinitiv.com/auth/oauth2/v1/token")
.header("Content-Type", "application/x-www-form-urlencoded")
.field("username", username)
.field("password", password)
.field("grant_type", "password")
.field("scope", "trapi")
.field("takeExclusiveSignOnControl", "true")
.field("client_id", clientId)
.asJson();
return response.getBody().getObject().getString("access_token");
} catch (Exception ex) {
System.out.println(ex.toString());
System.exit(1);
return null;
}
}
This function accepts a username, password, and client Id and then send them to the authentication endpoint. If the credential is invalid, the function will return an access token. Otherwise, the application will print an exception and then exits.
2. Alerts subscription
The RDP Alerts API allows users to setup subscriptions for news headlines, news stories, and researches. The code below demonstrates how to subscribe to news headlines or news stories by using the unirest-java library.
import kong.unirest.*;
...
public JsonNode SubscribeNewsStories(String token) {
try {
HttpResponse < JsonNode > response = Unirest.post("https://api.refinitiv.com/alerts/v1/news-stories-subscriptions")
.header("content-type", "application/json")
.header("Authorization", "Bearer " + token)
.body("{\"transport\":{\"transportType\":\"AWS-SQS\"}}")
.asJson();
return response.getBody();
} catch (UnirestException ex) {
System.out.println(ex.toString());
System.exit(1);
return null;
}
}
public JsonNode SubscribeNewsHeadlines(String token) {
try {
HttpResponse < JsonNode > response = Unirest.post("https://api.refinitiv.com/alerts/v1/news-headlines-subscriptions")
.header("content-type", "application/json")
.header("Authorization", "Bearer " + token)
.body("{\"transport\":{\"transportType\":\"AWS-SQS\"},\"filter\":{\"operator\":\"and\",\"operands\":[{\"type\":\"language\",\"value\":\"L:en\"}],\"type\":\"operator\"}}")
.asJson();
return response.getBody();
} catch (UnirestException ex) {
System.out.println(ex.toString());
System.exit(1);
return null;
}
}
The SubscribeNewsHeadlines method subscribes to the new headlines while the SubscribeNewsStories method subscribes to the new stories. The methods accept the access token retrieved from the first step and then send the news subscriptions to the news alert endpoints. The methods return responses in JSON format which contains a subscription Id, an Amazon Simple Queue Service (SQS) endpoint, and a cryptography key required to decrypt news content. A subscription Id is used to delete a subscription. If the request fails, the application will print an exception and then exits.
{
"transportInfo": {
"endpoint": "https://sqs.us-east-1.amazonaws.com/642157181326/sqs-edsalerts-main-prod-usersqs-873a0d7...",
"transportType": "AWS-SQS",
"cryptographyKey": "NeJv2OTkISQ1W…."
},
"creationTimestamp": "2023-12-08T08:34:59.813Z",
"subscriptionID": "873a0d78-ff50-4511…"
}
3. Get the credentials to a cloud queue
To access the Amazon Simple Queue Service (SQS), an application needs to get credentials from RDP which is the owner of the queue. The code below demonstrates how to get credentials to access the Amazon Simple Queue Service (SQS).
import kong.unirest.*;
...
public JsonNode GetCloudCredential(String token, String endpoint) {
try {
HttpResponse < JsonNode > response = Unirest.get("https://api.refinitiv.com/auth/cloud-credentials/v1/?endpoint=" + endpoint)
.header("Authorization", "Bearer " + token)
.asJson();
return response.getBody();
} catch (UnirestException ex) {
System.out.println(ex.toString());
System.exit(1);
return null;
}
}
The GetCloudCredential method accepts two parameters. The first one is the access token retrieved from the first step and the second one is the Amazon Simple Queue Service (SQS) endpoint retrieved from the second step. Then, it passes the SQS endpoint to the cloud credentials endpoint to get the cloud credentials. If the request is successful, this method will return responses in JSON format which contains the credentials to access the Amazon Simple Queue Service (SQS), such as access key ID, secret key, and a session token. Otherwise, the application will print an exception and then exits.
{
"endpoint": "https://sqs.us-east-1.amazonaws.com/642157181326/sqs-edsalerts-main-prod-usersqs-019ef77...",
"credentials": {
"accessKeyId": "ASIAZLA…",
"secretKey": "M8P6fSffq7W…",
"sessionToken": "IQoJb3JpZ2luX2VjEFIaCXVzLWVhc3QtM…
"expiration": "2023-12-08T11:00:23.000Z"
}
}
4. Polling and retrieving messages
At this point, the application has the SQS endpoint, credentials to access the SQS, and cryptography key to decrypt the content. The code below demonstrates how to retrieve messages from the Amazon Simple Queue Service (SQS).
import kong.unirest.json.JSONObject;
import java.util.List;
import software.amazon.awssdk.auth.credentials.*;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.sqs.SqsClient;
import software.amazon.awssdk.services.sqs.model.ReceiveMessageRequest;
import software.amazon.awssdk.http.urlconnection.UrlConnectionHttpClient;
import software.amazon.awssdk.services.sqs.model.Message;
...
public void RetrieveMessage(String accesssKeyId, String secretKey, String sessionToken, String endpoint, String cryptographyKey) {
AwsCredentials credentials = AwsSessionCredentials.create(accesssKeyId, secretKey, sessionToken);
SqsClient sqsClient = SqsClient.builder()
.region(Region.US_EAST_1)
.credentialsProvider(() - > credentials)
.build();
System.out.println(endpoint);
// Receive messages from the queue
for (int i = 1; i <= 10; i++) {
ReceiveMessageRequest receiveRequest = ReceiveMessageRequest.builder()
.queueUrl(endpoint)
.maxNumberOfMessages(10)
.waitTimeSeconds(20)
.build();
List < Message > messages = sqsClient.receiveMessage(receiveRequest).messages();
for (Message m: messages) {
try {
String s = new String(Decrypt(cryptographyKey, m.body()), StandardCharsets.UTF_8);
JSONObject jsonObj = new JSONObject(s);
String timestamp = jsonObj.getString("sourceTimestamp");
String headline = jsonObj.getJSONObject("payload").
getJSONObject("newsMessage").
getJSONObject("itemSet").
getJSONArray("newsItem").getJSONObject(0).
getJSONObject("itemMeta").
getJSONArray("title").getJSONObject(0).
getString("$");
System.out.println(timestamp + ": " + headline);
} catch (Exception ex) {
System.out.println(ex.toString());
System.exit(1);
}
}
}
The RetrieveMessage method accepts the SQS credentials, SQS endpoint and cryptography key. This method uses classes in the software.amazon.awssdk.sqs package to connect to the Amazon Simple Queue Service (SQS) and receive messages from the queue. Then, it calls the Decrypt method to decrypt the messages with the cryptography key. If the method fails, the application will print an exception and then exits.
5. Decrypting messages
The retrieved messages are encrypted by the AES256 with GCM algorithm. To decrypt the messages, it requires the cryptography key retrieved from the second step. For more information regarding how the messages are encrypted, please refer to the Understanding the Alerts delivery mechanism in Refinitiv Data Platform article.
The following code uses the javax.crypto.Cipher package to decrypt the messages.
import java.util.Base64;
import javax.crypto.Cipher;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.SecretKeySpec;
...
public byte[] Decrypt(String key, String source) throws Exception {
int GCM_AAD_LENGTH = 16;
int GCM_TAG_LENGTH = 16;
int GCM_NONCE_LENGTH = 12;
byte[] decodedKey = Base64.getDecoder().decode(key);
byte[] decodedSource = Base64.getDecoder().decode(source);
byte[] aad = new byte[GCM_AAD_LENGTH];
System.arraycopy(decodedSource, 0, aad, 0, GCM_AAD_LENGTH);
byte[] nonce = new byte[GCM_NONCE_LENGTH];
System.arraycopy(aad, GCM_AAD_LENGTH - GCM_NONCE_LENGTH, nonce, 0, GCM_NONCE_LENGTH);
byte[] tag = new byte[GCM_TAG_LENGTH];
System.arraycopy(decodedSource, decodedSource.length - GCM_TAG_LENGTH, tag, 0, GCM_TAG_LENGTH);
byte[] encMessage = new byte[decodedSource.length - GCM_AAD_LENGTH];
System.arraycopy(decodedSource, GCM_AAD_LENGTH, encMessage, 0, encMessage.length);
SecretKeySpec secretKey = new SecretKeySpec(decodedKey, "AES");
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec gcmParams = new GCMParameterSpec(GCM_TAG_LENGTH * 8, nonce);
cipher.init(Cipher.DECRYPT_MODE, secretKey, gcmParams);
cipher.updateAAD(aad);
byte[] decMessage = cipher.doFinal(encMessage);
return decMessage;
}
The Decrypt method accepts the cryptography key and the encrypted message. First, it converts the encrypted message and cryptography key from base64 strings to byte arrays. Then, it extracts AAD, NONCE, and TAG from the encrypted message. Next, it imports the cryptography key to the AES256 with GCM algorithm. After that, it uses the Cipher. doFinal method to decrypt the message. Finally, it returns the decrypted message to the caller. The message is in JSON format.
6. Unsubscribe
After retrieving the data, the application can unsubscribe a news subscription by using the subscription ID retrieved from the second step. The code below demonstrates how to unsubscribe news subscriptions.
import kong.unirest.*;
...
public void UnSubscribeNewsStories(String token, String subscriptionID) {
try {
HttpResponse < String > response = Unirest.delete("https://api.refinitiv.com/alerts/v1/news-stories-subscriptions?subscriptionID=" + subscriptionID)
.header("Authorization", "Bearer " + token)
.asString();
} catch (UnirestException ex) {
System.out.println(ex.toString());
System.exit(1);
}
}
public void UnSubscribeNewsHeadlines(String token, String subscriptionID) {
Unirest.setTimeouts(0, 0);
try {
HttpResponse < String > response = Unirest.delete("https://api.refinitiv.com/alerts/v1/news-headlines-subscriptions?subscriptionID=" + subscriptionID)
.header("Authorization", "Bearer " + token)
.asString();
} catch (UnirestException ex) {
System.out.println(ex.toString());
System.exit(1);
}
}
The UnSubscribeNewsHeadlines method unsubscribes a new headlines subscription while the UnSubscribeNewsStories method subscribes to the new stories. The methods accept the access token retrieved from the first step and the subscription ID retrieved from the second step. Then it sends the un-subscription requests to the news alert endpoints. If the request fails, the application will print an exception and then exits.
Application
This section demonstrates how to use the methods from the previous steps to subscribe and retrieve news headlines. The snippet code is shown below.
public static void main(String[] args) {
String username = "<username>";
String password = "<password>";
String clientId = "<client Id>";
System.out.println("News Headlines Subscription ...");
Alerts app = new Alerts();
System.out.println("1. Login ...");
String token = app.GetToken(username, password, clientId);
System.out.println("Token: " + token);
System.out.println("\n2. Subscribe news headlines ...");
JsonNode newsResponse = app.SubscribeNewsHeadlines(token);
System.out.println("Endpoint: " + newsResponse.getObject().getJSONObject("transportInfo").getString("endpoint"));
String endPoint = newsResponse.getObject().getJSONObject("transportInfo").getString("endpoint");
String cryptographyKey = newsResponse.getObject().getJSONObject("transportInfo").getString("cryptographyKey");
String subscriptionID = newsResponse.getObject().getString("subscriptionID");
System.out.println("\n3. Get cloud credentials ...");
JsonNode cloudCredResponse = app.GetCloudCredential(token, endPoint);
System.out.println(cloudCredResponse.toString());
String accessKeyId = cloudCredResponse.getObject().getJSONObject("credentials").getString("accessKeyId");
String secretKey = cloudCredResponse.getObject().getJSONObject("credentials").getString("secretKey");
String sessionToken = cloudCredResponse.getObject().getJSONObject("credentials").getString("sessionToken");
String cloudEndPoint = cloudCredResponse.getObject().getString("endpoint");
System.out.println("Credentials:");
System.out.println("\taccessKeyId: " + accessKeyId);
System.out.println("\tsecretKey: " + secretKey);
System.out.println("\tsessionToken: " + sessionToken);
System.out.println("\tendpoint: " + cloudEndPoint);
System.out.println("\n4. Retrieve messages ...");
System.out.println("5. Decrypt messages ...");
app.RetrieveMessage(accessKeyId, secretKey, sessionToken, cloudEndPoint, cryptographyKey);
System.out.println("\n6. Unsubscribe ... ");
app.UnSubscribeNewsHeadlines(token, subscriptionID);
}
First, the username, password, and application ID must be specified in the code. The code subscribes to English language news headlines. Then it displays the timestamps and titles of news headlines on the screen.
The full code is available in GitHub.
The output after running the code looks like the following.
News Headlines Subscription ...
1. Login ...
Token: eyJ0eXAiOiJhdCtqd3QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImJlcGpHV0dkOW44WU9VQ1NwX3M3SXlRMmlKMFk…
2. Subscribe news headlines ...
Endpoint: https://sqs.us-east-1.amazonaws.com/642157181326/sqs-edsalerts-main-prod-usersqs-a818b7cb...0e0daf
3. Get cloud credentials ...
{"endpoint":"https://sqs.us-east-1.amazonaws.com/642157181326/sqs-edsalerts-main-prod-usersqs-a818b7cb-8db1-43e4-b280-d4cb670e0daf","credentials":{"accessKeyId":"ASIAZ…TU","secretKey":"7YzmPpI…/ysV","sessionToken":" 2ao…YWD3IyHmuUQcs8DHpqd=","expiration":"2023-12-12T12:46:30.000Z"}}
Credentials:
accessKeyId: ASIAZL…F443TU
secretKey: 7YzmPpI…/ysV
sessionToken: IQoJb3JpZ2lu---qek4/g==
endpoint: https://sqs.us-east-1.amazonaws.com/642157181326/sqs-edsalerts-main-prod-usersqs-...e0daf
4. Retrieve messages ...
5. Decrypt messages ...
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
https://sqs.us-east-1.amazonaws.com/642157181326/sqs-edsalerts-main-prod-usersqs-...0daf
2023-12-12T11:46:29.103Z: Rwanda sets July election date for presidential and parliamentary polls
2023-12-12T11:46:35.833Z: DWP - Plan to move government roles out of Westminster brought forward and new headquarters unveiled
2023-12-12T11:46:28.690Z: DJ BSE: BEML Ltd. - Announcement Under Regulation 30 (LODR)-Change In Directorate
2023-12-12T11:46:28.690Z: DJ BSE: BEML Ltd. - Announcement Under Regulation 30 (LODR)-Change In Directorate
2023-12-12T11:46:29.103Z: Rwanda sets July election date for presidential and parliamentary polls
2023-12-12T11:46:29.150Z: *DJ Lazard Nov Preliminary Assets Under Management Totaled $235.9B >LAZ
2023-12-12T11:46:32.533Z: HYLD: [DAILY INFORMA] 12/12/23
2023-12-12T11:46:32.533Z: HYLD: [DAILY INFORMA] 12/12/23
2023-12-12T11:46:33.374Z: BondSpot SA - 2023-12-12 Suspension from trading bonds issued by GMINA MIASTO ELBL?G on the regulated market organized by BondSpot.
2023-12-12T11:46:34.182Z: BondSpot SA - 2023-12-12 Suspension of trading in the alternative trading system on Catalyst in series B bearer bonds of GRENEVIA S.A.
2023-12-12T11:46:28.614Z: REG - Edinburgh Inv. Trust - Net Asset Value(s)
2023-12-12T11:46:34.182Z: BondSpot SA - 2023-12-12 Suspension of trading in the alternative trading system on Catalyst in series B bearer bonds of GRENEVIA S.A.
2023-12-12T11:46:33.374Z: BondSpot SA - 2023-12-12 Suspension from trading bonds issued by GMINA MIASTO ELBL?G on the regulated market organized by BondSpot.
2023-12-12T11:46:36.947Z: China iron ore: Prices fluctuate
2023-12-12T11:46:39.071Z: DJ UBS hires JPMorgan veteran Rueger to co-head equity capital markets -- Financial News
2023-12-12T11:46:39.071Z: DJ UBS hires JPMorgan veteran Rueger to co-head equity capital markets -- Financial News
2023-12-12T11:46:39.182Z: PFIZER INC - WILL SPLIT ITS NON-ONCOLOGY COMMERCIAL ORGANIZATION INTO TWO MORE FOCUSED BUSINESS DIVISIONS
2023-12-12T11:46:40.262Z: China bunkers: Prices lack clear direction
2023-12-12T11:46:39.782Z: ESCORTS-Loss/Duplicate-Share Certificate-XBRL
6. Unsubscribe ...
Summary
This article demonstrates how to develop a Java application to authenticate and subscribe to news headlines or stories from RDP Alerts services. It uses the software.amazon.awssdk.sqs, com.konghq.unirest-java, and javax.crypto.Cipher packages to retrieve the encrypted content from SQS and decrypt the content, respectively. The workflow used in the article is descriptively explained in the Understanding the Alerts delivery mechanism in Refinitiv Data Platform article.
