Article

Using Refinitiv Data Platform Alerts with C#

May 29,2019

Jirapongse Phuriphanvichai

Using Refinitiv Data Platform Alerts with C#

Introduction

Refinitiv Data Platform (RDP) is our cloud-enabled, open platform, that brings together content, analytics and proprietary, customer and third-party data distribution and management technology. It provides simple web-based API access to a broad range of content. Therefore, any programming languages which support web-based API can connect and consume data from RDP.

This article demonstrates how to use C# to consume news headlines from RDP Alerts API. The RDP Alerts API allows users to setup subscriptions for different types of content, such as news, and research. The full code is available in GitHub.

AutoRest

The RDP classed used in the article are generated by an AutoRest tool which is OpenAPI (f.k.a Swagger) Specification code generator. It supports C#, PowerShell, Go, Java, Node.js, TypeScript, Python, Ruby and PHP. You can follow steps in How to use AutoRest with RDP to generate C#, Java, and Python code for RDP APIs.

The RDP packages used in this article are also available in the NuGet gallery, as shown below. You may use the newer version so of RDS APIs.

NuGet Package	RDP API
Refinitiv.EDP.AutoRest.Auth.OAuth2	/auth/oauth2/beta1
Refinitiv.EDP.AutoRest.Auth.CloudCredentials	/auth/cloud-credentials/v1
Refinitiv.EDP.AutoRest.Alerts	/alerts/v1

These NuGet packages support .NET Standard 1.6 and 2.0.

Steps to subscribe to alerts

This section demonstrates steps to develop a C# application to subscribe to news headlines from RDP alerts API. The application implements the workflow mentioned in the Understanding the Alerts delivery mechanism in Refinitiv Data Platform article. It is developed with Visual Studio 2017 with .NET Core 2.0 and uses the following NuGet packages:

Refinitiv.EDP.AutoRest.Auth.OAuth2
Refinitiv.EDP.AutoRest.Alerts
Refinitiv.EDP.AutoRest.Auth.CloudCredentials
AWSSDK.SQS
NSec.Cryptography

Although the generated C# classes support both synchronous and asynchronous methods, the example in this article uses synchronous methods.

There are six steps to consume news headlines from RDP alerts API.

1. Login

Login is the first step for all RDP API requests. RDP entitlement check is based on OAuth 2.0 specification which requires the following parameters.

Parameter	Definition
Username	The username required to access RDP request-response API (typically email)
Password	The password for username above -- used to get access token
ClientID	A unique ID defined for an application making the request. Users can generate/manage their application ID's here

A successful authentication response from a server contains the access token and refresh tokens The access token is used to invoke other REST data API calls. The refresh token is used to get the next access token when the current access token is expired.

The following code shows how to use the EDSAuthentication class in the Refinitiv.EDP.AutoRest.Auth.OAuth2 package to get an access token.

    	
            using Refinitiv.EDP.AutoRest.Auth.OAuth2;
using Refinitiv.EDP.AutoRest.Auth.OAuth2.Models;
using AuthError = Refinitiv.EDP.AutoRest.Auth.OAuth2.Models.Error;
...
...
 
        public Tokenresponse Login(
            string username,
            string password,
            string clientid,
            out string error)
        {
            EDSAuthentication eds = new EDSAuthentication();            
 
            var response = eds.PostToken(
                "password",
                username,
                password,
                null,
                "trapi", 
                null,
                clientid,
                "true");
 
            if (response is Tokenresponse)
            {
                Tokenresponse tokenResp = (Tokenresponse)response;               
                error = null;
                return (Tokenresponse)response;
 
            }
            else if (response is AuthError)
            {
                error = ((AuthError)response).ErrorDescription;
                return null;
            }
            else
            {
                error = "Unknown Type";
                return null;
            }
 
        }

This method accepts username, password, and client ID as parameters. It uses the synchronous PostToken method of the EDSAuthentication class to get an access token. Then, if the authentication process is successful, it will return the Tokenresponse object containing an access token. Otherwise, it populates the error variable with the error text and returns null.

2. Alerts subscription

The RDP Alerts API allows users to setup subscriptions for news headlines, news stories, and researches. This step demonstrates how to use the SubscriptionstocontentalertsAPI class in the Refinitiv.EDP.AutoRest.Alerts package to subscribe to news headlines.

    	
            using Refinitiv.EDP.AutoRest.Alerts;
using Refinitiv.EDP.AutoRest.Alerts.Models;
using AlertsError = Refinitiv.EDP.AutoRest.Alerts.Models.Error;
...
 
        public NewsSubscriptionDetails SubscribeNewsHeadlines(
            Tokenresponse token, 
            JObject newsFilter, 
            out string error)
        {
            TokenCredentials cred = new TokenCredentials(token.AccessToken);
 
            SubscriptionstocontentalertsAPI alerts = new SubscriptionstocontentalertsAPI(cred);
 
            var response = alerts.PostNewsHeadlinesSubscriptions(new NewNewsSubscription { Filter = newsFilter });
 
 
            if (response is NewsSubscriptionDetails)
            {
                error = null;
                return (NewsSubscriptionDetails)response;
            }
            else if(response is AlertsError)
            {                
                error = ((AlertsError)response).ErrorProperty.Message;
                return null;
            }
            else
            {
                error = "Unknown Type";
                return null;
            }
        }

This method accepts the Tokenresponse object from the first step and JSON object which contains conditions used to filter the retrieved news headlines. First, it uses the access token in the Tokenresponse object to create a TokenCredentials object. Then, the TokenCredentials object is used to create an instance of SubscriptionstocontentalertsAPI. Next, it passes the news-headlines filters to the PostNewsHeadlinesSubscriptions method to subscribe to news headlines which match the filters. If the request is successful, it returns the NewsSubscriptionDetails object. Otherwise, it populates the error text and then returns null.

The NewsSubscriptionDetails object contains the Amazon Simple Queue Service (SQS) endpoint, and cryptography key required to decrypt the content.

3. Get the credentials to a cloud queue

To access Amazon Simple Queue Service (SQS), applications need to get credentials from RDP which is the owner of the queue. In this step, the CloudCredentialsAPI class in the Refinitiv.EDP.AutoRest.Auth.CloudCredentials package is used to request credentials for SQS resources.

    	
            using Refinitiv.EDP.AutoRest.Auth.CloudCredentials;
using Refinitiv.EDP.AutoRest.Auth.CloudCredentials.Models;
using CloudCredentialsError = Refinitiv.EDP.AutoRest.Auth.CloudCredentials.Models.Error;
...
 
        public CredentialDetails GetCloudCredentials(
            Tokenresponse token, 
            NewsSubscriptionDetails newsSub, 
            out string error)
        {
            TokenCredentials cred = new TokenCredentials(token.AccessToken);
            CloudCredentialsAPI cloudCredential = new CloudCredentialsAPI(cred);
 
            var response = cloudCredential.Get(newsSub.TransportInfo.Endpoint);
 
            if (response is CredentialDetails)
            {
                error = null;
                return (CredentialDetails)response;
 
            }
            else if (response is CloudCredentialsError)
            {
                error = ((CloudCredentialsError)response).ErrorProperty.Message;
                return null;
            }
            else
            {
                error = "Unknown Type";
                return null;
            }
        }

This method accepts the Tokenresponse object from the first step and the NewsSubscriptionDetails object from the second step. First, it uses the access token in the Tokenresponse object to create a TokenCredentials object. Then, the TokenCredentials object is used to create an instance of CloudCredentialsAPI. Next, it passes the SQS endpoint in the NewsSubscriptionDetails object to the Get method to get the cloud credentials. If the request is successful, it returns the CredentialDetails object. Otherwise, it populates the error text and then returns null.

The CredentialDetails object contains the credentials to access the Amazon Simple Queue Service (SQS), such as access key ID, secret key, and a session token.

4. Polling and retrieving messages

This step uses the CredentialDetails object from the previous step and classes from the AWSSDK.SQS packages to retrieve messages from the Amazon Simple Queue Service (SQS).

    	
            using Amazon.SQS;
using Amazon.SQS.Model;
using Amazon.Runtime;
...
        public List<string> RetrieveMessages(
            CredentialDetails credDetails,
            int numberOfMessages
            )
        {
            var sqsConfig = new AmazonSQSConfig();
            List<string> headlineList = new List<string>();
 
            Uri endPointUri = new Uri(credDetails.Endpoint);
            sqsConfig.ServiceURL = $"{endPointUri.Scheme}://{endPointUri.Host}";
 
            var awsCredentials = new SessionAWSCredentials(
                credDetails.Credentials.AccessKeyId,
                credDetails.Credentials.SecretKey,
                credDetails.Credentials.SessionToken);
 
 
            var sqsClient = new AmazonSQSClient(
                awsCredentials,
                sqsConfig);
 
 
            var receiveMessageRequest = new ReceiveMessageRequest {
                QueueUrl = credDetails.Endpoint,
                MaxNumberOfMessages = 10
            };
 
            ReceiveMessageResponse receiveMessageResponse = null;
            while (receiveMessageResponse == null || headlineList.Count < numberOfMessages)
            {
                receiveMessageResponse = sqsClient.ReceiveMessageAsync(receiveMessageRequest).
                  GetAwaiter().
                  GetResult();
 
                foreach(var message in receiveMessageResponse.Messages)
                {                    
                    if (headlineList.Count < numberOfMessages)
                    {
                        headlineList.Add(message.Body);
                    }
                    DeleteMessageRequest deleteMessageRequest = new DeleteMessageRequest();
 
                    deleteMessageRequest.QueueUrl = credDetails.Endpoint;
                    deleteMessageRequest.ReceiptHandle = message.ReceiptHandle;
                    DeleteMessageResponse response =
                            sqsClient.DeleteMessageAsync(deleteMessageRequest).GetAwaiter().GetResult();
                }
            }        
 
            return headlineList;         
        }

This method accepts two parameters which are the CredentialDetails object to connect to the Amazon Simple Queue Service (SQS) and the number of messages retrieved from the SQS. This method calls the AmazonSQSClient.ReceiveMessageAsync method to receive messages and returns the list of messages to the caller. Each retrieved message is also deleted from the SQS by the AmazonSQSClient.DeleteMessageAsync method.

5. Decrypting messages

The messages in the queue are encrypted by the AES256 with GCM algorithm. To encrypt the messages, it requires the cryptography key in the NewsSubscriptionDetails object from the second step. For more information regarding how the messages are encrypted, please refer to the Understanding the Alerts delivery mechanism in Refinitiv Data Platform article.

The following code uses the NSec.Cryptography package to decrypt the messages.

    	
            using System.Text;
using NSec.Cryptography;
...
        public string DecryptMessage(
            string encryptedText, 
            string cryptographyKey)
        {
            int ivSize = 12;            
            int aadSize = 16;
 
            var cryptographyKeyByte = Convert.FromBase64String(cryptographyKey);
            var encryptedByte = Convert.FromBase64String(encryptedText);
            var aes = new Aes256Gcm();
 
            var aad = new byte[aadSize];
            Array.Copy(encryptedByte, 0, aad, 0, aadSize);
 
            var iv = new byte[ivSize];
            Array.Copy(aad, 4, iv, 0, ivSize);
 
            var key = Key.Import(aes, 
                cryptographyKeyByte, 
                KeyBlobFormat.RawSymmetricKey);
 
            var encryptedByteNoAad = new byte[encryptedByte.Length - aadSize];
            Array.Copy(encryptedByte, aadSize, encryptedByteNoAad, 0, encryptedByteNoAad.Length);
 
            byte[] decryptedByte;
            aes.Decrypt(key, new Nonce(iv, 0), aad, encryptedByteNoAad, out decryptedByte);
 
            return Encoding.UTF8.GetString(decryptedByte);
        }

This method accepts the encrypted message and the cryptography key. First, it converts the encrypted message and cryptography key from base64 strings to byte arrays. Then, it extracts ADD and IV from the encrypted message. Next, it imports the cryptography key to the AES256 with GCM algorithm. After that, it uses the Aes256Gcm.Decrypt method to decrypt the message. Finally, it returns the decrypted string to the caller. The string is in JSON format.

6. Unsubscribe

After retrieving the data, the application can unsubscribe news headlines by calling the SubscriptionstocontentalertsAPI.DeleteNewsHeadlinesSubscriptions method with the subscription ID which is in the NewsSubscriptionDetails object retrieved from the second step.

    	
                    public AlertsError Unsubscribe(
            Tokenresponse token,
            NewsSubscriptionDetails newsSubscriptionDetails
           )
        {
            TokenCredentials cred = new TokenCredentials(token.AccessToken);
            SubscriptionstocontentalertsAPI alerts = new SubscriptionstocontentalertsAPI(cred);
            return alerts.DeleteNewsHeadlinesSubscriptions(newsSubscriptionDetails.SubscriptionID);           
        }

This method accepts two parameters which are Tokenresponse and NewsSubscriptionDetails. It uses the AccessToken from Tokenresponse to create a TokenCredentials for the SubscriptionstocontentalertsAPI. Then, it uses the subscription ID in the NewsSubscriptionDetails object to delete the subscription.

Application

This section demonstrates how to use the methods from the previous steps to subscribe and retrieve news headlines. The snippet code is shown below.

    	
                   public void run()
        {
            string error;
            string username = "<username>";
            string password = "<password>";
            string appId = "<application id>";
            var tokenResponse = Login(
                username,
                password,
                appId,
                out error);
 
            var newsSub = SubscribeNewsHeadlines(
                tokenResponse,
                JObject.FromObject(
                    new {  language = "en"    }
                    ), 
                out error
                );
 
            var cloudCred = GetCloudCredentials(tokenResponse, newsSub, out error);
 
            var messages = RetrieveMessages(cloudCred, 10);
 
            foreach (var message in messages)
            {         
                var headline = DecryptMessage(message, newsSub.TransportInfo.CryptographyKey);
                var obj = JObject.Parse(headline);
 
                if (obj["payload"]["newsMessage"]["itemSet"]["newsItem"][0]["itemMeta"]["title"][0] != null)
                {
                    Console.WriteLine($"{obj["contentReceiveTimestamp"]}: {obj["payload"]["newsMessage"]["itemSet"]["newsItem"][0]["itemMeta"]["title"][0]["$"]}" );
                }
            }
 
            var alertError = Unsubscribe(tokenResponse, newsSub);
        }

First, the username, password, and application ID must be specified in the code. The code subscribes to English language news headlines. It retrieves ten news headlines and then displays the timestamps and titles of news headlines on the screen.

The full code is available in GitHub.

The output after running the code looks like the following.

    	
            News Headlines Subscription ...
1. Login ...
         Access token: eyJ0eXAiOiJKV1Qixxx
 
2. Subscribe news headlines ...
         Endpoint URL: https://sqs.us-east-1.amazonaws.com/64xxx
 
3. Get cloud credentials  ...
         Access Key ID: Axxx
 
4. Retrieve messages ...
 
5. Decrypt messages [10]  ...
5/29/2019 3:39:25 AM: OMX Copenhagen 20 intraday: the upside prevails as long as 993.75 is support
5/29/2019 3:39:25 AM: OMX Copenhagen 20 intraday: the upside prevails as long as 993.75 is support
5/29/2019 3:39:28 AM: REFILE-Huawei asks U.S. court to declare defence bill &apos;unconstitutional&apos;
5/29/2019 3:39:29 AM: EXCLUSIVE-Hong Kong judges see risks in proposed extradition changes
5/29/2019 3:39:31 AM: EXCLUSIVE-Hong Kong judges see risks in proposed extradition changes
5/29/2019 3:39:25 AM: *TOP NEWS* Greater China
5/29/2019 3:39:27 AM: REFILE-Huawei asks U.S. court to declare defence bill &apos;unconstitutional&apos;
5/29/2019 3:39:28 AM: ANALYSIS-Huawei ban puts S.Korea in a familiar place - caught between the U.S. and China
5/29/2019 3:39:32 AM: OBX Stock Index intraday: short term rebound towards 816.56
5/29/2019 3:39:34 AM: OMXS30 index intraday: short term rebound
 
6. Unsubscribe ...

Summary

This article demonstrates how to use classes generated by AutoRest to authenticate and subscribe to news headlines from RDP Alerts services. It uses the AWSSDK.SQS and NSec.Cryptography packages to retrieve the encrypted content from SQS and decrypt the content, respectively. The workflow used in the article is descriptively explained in the Understanding the Alerts delivery mechanism in Refinitiv Data Platform article.