Use Case Two

Refinitiv Data Platform News - Middle Office Compliance Use Cases

Zoya Farberov
Developer Advocate Developer Advocate

Introduction to Use Case

How a user can make a RDP API Request/Response call for news:

  • Published in the last 24 hours, where
  • The messages are tagged with PermIds (1, 2, 3, 4)
  • And the messages contain topic codes (x, y or z)  

To impelement the use case we are going to explore RDP news filters.  RDP news filtering is the primary focus of this use case, while specific filter example is chosen to illustrate it.

Approach

We are going to use RDP API ( HTTP REST) calls directly, as opposite to using Refinitiv Data Library Endpoint representation or Eikon Data Library Python get_news method.

  • Setup for RDP API Access
  • Obtain a Valid RDP Authentication Token
  • Request News Headlines - Define a Function
  • Request Headlines per Filter Requirements (by PermIds and Topics, for the last 24 hrs

Request News Headlines - Define a Function

    	
            

def getNewsHeadlines(query, numLines, return_as_text=False):

    news_category_URL = "/data/news"

    headlines_endpoint_URL = "/headlines?"

 

    REQUEST_URL = base_URL + news_category_URL + RDP_version + headlines_endpoint_URL+ query # +"&limit="+ str(numLines);

    

    accessToken = getToken();

    print("Requesting: ",REQUEST_URL)

    

    acceptValue = "*/*"

    dResp = requests.get(REQUEST_URL, headers = {"Authorization": "Bearer " + accessToken, "Accept": acceptValue});

    

    if dResp.status_code != 200:

        print("Unable to get data. Code %s, Message: %s" % (dResp.status_code, dResp.text));

        if dResp.status_code != 401:   # error other then token expired

            return("Error "+str(dResp.status_code)) 

        accessToken = getToken();     # token refresh on token expired

        dResp = requests.get(REQUEST_URL, headers = {"Authorization": "Bearer " + accessToken, "Accept": acceptValue});

        if dResp.status_code == 200:

            print("Resource access successful")

    else:

        print("Resource access successful")

    if return_as_text:

        return dResp.text

    jResp = json.loads(dResp.text);

    return jResp

next, we are going to use the defined function to

Request Headlines per Filter Requirements (by PermIds and Topics, for the last 24 hrs)

Some of the aspects to note:

  • Max headlines request size at this time is defined at 100 headlines, that is also what is returned by default
  • If the expected required result set of headlines is larger then 100, the result is paginated and the received portions of the result are assemebled into a single result dataframe
  • Each headline contains a storyID which allows to retrieve the corresponding story
    	
            

N = 100

query = 'query=(4295905573, 5030853586) and (searchIn:HeadlineOnly) and (last 24 hours)';

#query = 'query=(4295905573, 5030853586) and (Topic:E:F) and (searchIn:HeadlineOnly) and (last 24 hours)';

#query = 'query=(4295905573, 5030853586) and (Topic:SIG OR Topic:WEA) and (searchIn:FullStory) and (last 24 hours)';

#query = 'query=(4295905573, 5030853586) and (Topic:E:F) and (searchIn:HeadlineOnly) and (last 24 hours)';

jRespHeadlines = getNewsHeadlines(query, N)

 

dataAll = []

dfAll = pd.DataFrame()

iterations_count = 0;

while True:

    cursor = ""

    dfH = pd.json_normalize(jRespHeadlines,record_path =['data'])

    if not dfH.empty:

        dataAll.append(dfH)

        if 'meta' in jRespHeadlines:

            if 'next' in jRespHeadlines['meta']:

                # print('Meta.next=', jRespHeadlines['meta']['next'])

                cursor = urllib.parse.quote(jRespHeadlines['meta']['next'])

                print('Iteration= ',iterations_count,'Cursor= ', cursor)   

                query = 'cursor='+cursor;

    else:

        print('No results that qualify filter requirements')

    if not cursor:

        break

    iterations_count += 1

    jRespHeadlines = getNewsHeadlines(query, N)

 

if dataAll:    

    dfAll = pd.concat(dataAll,ignore_index=True)

    print('Displaying full result:')

    display(dfAll)

we may wish to mark the iterations, for a large result set, this allows us to knowhow we progress:

Once the headlines result is received in total, and there is no more results to be received, that qualify our filter requirement, we assess the received result in dataframe:

the columns of the result that are  of most interest to us are likley newsItem.contentMeta.subject and storyID.  As newsItem.contentMeta.subject is what the story is about should allow us to further zero in on the news stories that we require and storyID is used to request the stories that we select:

next we can just simply request a story or stories per storyID

Request a Story - Define a Function  

    	
            

def getStory(storyId, jsonOrHtml):

    news_category_URL = "/data/news"

    story_endpoint_URL = "/stories"

 

    REQUEST_URL = base_URL + news_category_URL + RDP_version + story_endpoint_URL+ "/" + storyId 

 

    accessToken = getToken();

    print("Requesting: ",REQUEST_URL)

    

    acceptValue = "application/json"

    if jsonOrHtml != True:

        acceptValue = "text/html"

    dResp = requests.get(REQUEST_URL, headers = {"Authorization": "Bearer " + accessToken, "Accept": acceptValue});

    if dResp.status_code != 200:

        print("Unable to get data. Code %s, Message: %s" % (dResp.status_code, dResp.text));

        if dResp.status_code != 401:   # error other then token expired

            return("") 

        accessToken = getToken();     # token refresh on token expired

    else:

        print("Resource access successful")

        return dResp.text

we can display the story in Json format:

for instance, we can see that the story we have selected, is not part of Top News ( for more on Top News and Top News Images see References).  Or display the selected story of interest to us in HTML:

Request Story for Display

    	
            

txt = getStory(dfAll.iloc[[0]]['storyId'].item(), False)

from IPython.core.display import display, HTML

display(HTML(txt))

or apply any required formatting on display.  To gain more insight into stories and associated rich metadata that is made available via RDP News service, we may wish to use tools such as Json Viewer, see References for more information.

We conclude our present brief discussion of RDP News filtering middle office compliance use case, to discuss more, you can always find us on Refinitiv Developers Forums

References

Article code on GitHUb

RDP News Metadata with JSON Viewer

How to retrieve RDP Top News using RDP Library for .NET

RDP News Images