Introduction
Intelligent Tagging REST API can be used to automatically analyze your input text. And then identify and tag mentions (text strings) of things like companies, people, deals, geographical locations, industries, physical assets, organizations, products, events, etc. based on a list of predefined metadata types. Moreover, it can assign tags such as topic tags, social tags, industry tags, and slugline tags that describe what the input document is about as a whole.
There are some use cases that users want to extract the tags(e.g., social tags) and metadata like the Company and Person entity with the relevance and confidence score from news content that came with the MRN Story Real-Time data and MRN achieved(custom solution provided by real-time datafeed). Then it can feed the output from Intelligent Tagging API to their sentiment analysis system. This article will describe steps to use the REST API to analyze the input text from the MRN news story retrieved from the local ADS server or Refinitiv Real-Time – Optimized. We will create a .NET Core sample application that is using RDP.NET API to communicating and retrieving MRN News Story from the server. And then, extract the news body from the MRN JSON message and pass it to the Intelligent Tagging API to identify tags and relevant metadata.
Prerequisites
- Please read the Intelligent Tagging quick start guide before reading this article. You need to access the link from the quick start guide to register for a free account and create your API key. A key is required for the API call.
- Please read this article. You need to understand how to use the RDP.NET library to retrieve MRN Story data.
- You need Refinitiv Real-Time – Optimized account or local ADS server(version 3.2 or higher) with a DACS User with permission to request MRN STORY data.
- You need to install .NET Core SDK—highly recommended version 3.0 or higher versions.
- Visual Studio Code/Text editor or Visual Studio 2017/2019 to open solution/project and modify codes.
Creating a .NET Core application
There are three main steps in our sample application.
1) Requesting MRN Story data from Refinitiv Real-Time – Optimized or ADS server via the WebSocket connection. These steps will use RDP.NET SDK, and it provides a callback method that returns the MRN JSON data.
2) Using Intelligent Tagging REST API with the MRN news content.
3) Parsing a SocialTag and metadata from the response message provided by the REST API and print the output to console.
Step 1 Requesting MRN Story data
Below is a sample code to use MachineReadableNews for retrieving and handling MRN Story update.
In the sample app, we wish to extract the news content which contains the English language code. Therefore, we need to filter the message by checking the language code in the JSON message.
Below is a sample of JSON message for the MRN Story, which contains English language code "en."
Step 2 Using Intelligent Tagging REST API with the news content
The RDP.NET library returns the MRN Story data as the JObject in the callback function. Therefore, we need to create a StoryData class to hold the value of the MRN Story update. Then the application will call method JObject.ToObject() to deserialize the JSON data to StoryData object. Below is the StoryData class, which created based on the structure of the MRN Story JSON message.
The application uses the following codes to deserialize the JSON data from the instance of JObject to StoryData.
Then the application can get the news content from mrnObj.Body and pass it to HTTP request content of the REST API.
Calling Intelligent Tagging API
The API call to tag content is made via a simple HTTP REST interface. Below are the endpoint and HTTP Request message and required parameters we need to set in the request.
- HTTP Method: POST
- HTTP Request Headers:
- HTTP Request Content: Text from a news content
We will use the above headers in our sample app. You can find additional headers with the details about the value you can set from the Input Headers document. Please refer to the online documentation for the endpoint list in case there might be a change of endpoint for your package type.
Below is a sample of C# codes to call the REST API tag content from the MRN Story body. We set the output format to "application/json" because we need the output in JSON format. Then we can use the JSON.NET library to parse specific metadata such as social tag, company, person, and scores from the JSON output.
The following JSON message is sample output from the Intelligent Tagging API call.
The whole JSON message consists of a list of JSON objects for tag an entity like social tag, Person and Company entity. Please note that the result in the output depends on the input text and type of account you are using.
The application can use JSON.NET to parse the object and check the key "_typeGroup" to identify the type of the object. Then the application can get specific property such as the confidence score, importance level from the key in a JSON object. You can find available properties for each type of tag metadata from Intelligent Tagging Document
Step 3 Parsing a SocialTag and metadata from the response message
The application can get tag metadata from the Intelligent Tagging JSON output, by using JSON.NET to convert the JSON plain text to JObject by calling JObject.Parse(jsonoutput) method. And then, you can access the value of social tag and entities using the following snippet of codes.
It will show the output like the following sample.
Anyway, our sample will create an interface named IMetaData, which is a based interface to create a more specific class for a particular tag and entity. Then application just needs to check the value of the property named "_typeGroup" to identify what kind of class needs to use for deserializing from the JSON object.
Below is an implementation of the IMetaData interface.
And the following class is the SocialTagElement class and BaseElement class, which sharing a generic property with the other class. SocialTagElement class was created for holding SocialTag data. We will add a more specific property, such as an importance score and original value to the class.
The sample app also provides IntelligentTaggingParser class for parsing the JSON object. It will keep the tag metadata object in the C# Dictionary class, where the Dictionary's key is a hash key defined by Intelligent Tagging API.
The class also provide property to get a list of tag and entities. The below snippet of codes uses Linq to query the list of the social tag from the Dictionary object.
Please note that the sample app currently supports only some of a known tag or entity such as Language, Version, SocialTag, Person, and Company entity. Anyway, you can modify the app to add more entity classes to get a more specific object from the JSON message.
Use IntelligentTaggingParser in the sample application.
The application will create a HttpClient object and pass it to GetIntelligentTagging, which is a utility function. It will return JObject, which you can pass it to IntelligentTaggingParser class.
Sample codes to get JObject from Intelligent Tagging REST API
The sample app will pass the JObject to the DumpTagging method to parse the data and print its values to console output.
Build and Run the sample app
You can download the example from GitHub.
The sample application is a .NET Core console application so that you can build and run it on a platform supported by .NET Core SDK.
Please modify Program.cs and add your username and password in either region TREPCredential or RDPUserCredential. The default mode will use the RDP account, and you can change useRDP to false if you want to test with a local ADS Server.
You can open a solution file MRNIntelligentTagging.sln on Visual Studio 2017 or 2019 and then build or publish(menu Build->Publish MRNIntelligentTagging) the console application.
If you do not have Visual Studio, you can install the .NET Core SDK on your OS, and you may follow the following steps to build the application.
1) Run the Windows command line or using the terminal on macOS or Linux. Change the folder to the repository from GitHub. You should see MRNIntelligentTagging.sln in that folder. Then change the folder to the MRNIntelligentTagging folder, which is the primary app project folder.
2) Make sure that you are running with .NET Core 3 or higher version. Just check by running dotnet --version.
3) Run below dotnet publish command where "-c release" is for the release build, and "release_build" is the name of the output folder. Note that the below command is for Windows x64 build.
You should see folder release_build with an executable file MRNIntelligentTagging.exe and required DLLs under the release_build folder.
To build it on Linux or macOS, you can change win-x64 to another OS, please find the list from the rid-catalog page.
Sample Output
Below is a sample output when running the sample app. It will show a news content followed by the tag and entity from the JSON output.
Note that you can set redirectOutputToFile in Program.cs to true if you want to redirect the console output to file named Output.txt instead.
Summary
This article explains how to use Intelligent Tagging REST API getting tag metadata from a news content returned by MRN Story data. It also provides a sample application to demonstrate the API usage and provide a sample output from the REST API and how to extract the relevance tag and entity from the JSON output. The sample app uses RDP.NET to retrieve the MRN STORY from Refinitiv Real-Time – Optimized and local ADS server. The use case can be a solution for the client who needs to implement standard tagging across their news repository, using PermID. Intelligent Tagging API also offers a socialTags with relevance scoring for the clients. The client can feed the output provided by the API to the external system for sentiment analysis.
References
- Intelligent Tagging Quick Start Guide
- Intelligent Tagging Case Studies
- Intelligent Tagging FAQ
- REFINITIV DATA PLATFORM LIBRARIES - AN INTRODUCTION
- The Refinitiv Data Platform Libraries for .NET (RDP.NET)
- The Introduction to RDP Libraries document
- Refinitiv DataPlatform for .NET Nuget
- Refinitiv DataPlatform Content Nuget
- Websocket API
- MRN DATA MODELS Guide
- Newtonsoft JSON.NET document