Article

A Developers Guide to Sanctions Data

May 16,2022

Author:

Developer Community Team

Financial Institutions (FIs) are now in the spotlight as more sanctions have been imposed due to recent global events. For a higher level discussion of sanctions please see this Refinitive Perspectives article. The aim of this developer focused article is to show how you can access our sanctions offering programatically or via SFTP and monitor news around this using various APIs.

Broadly speaking there are two types of sanction:

Specially Designated Nationals (SDN) sanctions freeze any business or trading activity associated with an individual or company.

Sectoral Sanctions Identifications (SSI) sanctions restrict transactions in securities created by specific businesses, groups or individuals.

A quick summary of FI resposibilities:

Compliance can involve identifying and responding to top-line sanctions, sanctioned financial instruments related to these entities, those issued by their subsidiaries and those subject to majority ownership by sanctioned individuals in so-calleed sectoral sanctions.

The sanctions landscape remains dynamic and the latest rounds of both US & EU sanctions against Russia and China have added further complexity for compliance teams. A broad range of financial instruments - including equities, debt securities, options, futures, forwards, swaps, indices, ETFs and funds - is impacted.

Failure to identify such financial instruments and screen for related risk can result in both reputational damage and often heafty fines - but the process of identifying sactions-related financial instruments is frequently complex and time consuming.

Different Services Offered by Refinitiv

World Check Risk Intelligence Database - 100% Global coverage of 300+ sanctions lists over SFTP

Financial Instrument Risk Intelligence (FIRI) dataset - accurate, comprehensive sanctions content for both sanctioned entities (both explicit and implicit sanctions are included) and instruments, including entities that are directly listed by OFAC and the EU; subsidiaries of these entities; and lists of instruments issued by these entities after the application of any executive orders (EOs) - delivered by World Check over SFTP (csv,xml - zipped or unzipped)

DataScope Select uses WorldCheck to provide an instrument focused service with sanctions coverage of more than 250 lists via a REST API, as well as SFTP.

Plus:

Sanctions related News - available via many routes - we show an example using our Eikon Data API.

This article will show how you can daily sanctions files using our DataScope Select REST API and will also demonstrate how you can keep upto date on relevant Sanctions News using our Eikon and Refinitiv Data Platform News APIs. Finally we show how to access the FIRI dataset over SFTP via World Check. We note that there are differences in coverage between the different services if you have a granular query that isn't covered here, please contact your account team or click Contact Us. It is worth checking with us to see what is in-scope and not - especially as there are potentially large fines for breaches.

Firstly, lets look at some sanctions news topic codes you can subscribe to - to keep you abreast of changes in this dynamic environment programatically to feed any alerts mechanism you already have or wish to build. This is relevant for Eikon/Workspace or RDP users.

    	
            import refinitiv.dataplatform.eikon as ek
import pandas as pd
import matplotlib as plt
ek.set_app_key('YOUR APP KEY HERE')
import datetime
from datetime import datetime
import dateutil.relativedelta
now = datetime.now()
maxenddate = now - dateutil.relativedelta.relativedelta(days=30) #months,days
print(now, maxenddate)

2022-03-10 17:49:21.281355 2022-02-08 17:49:21.281355

Get Sanctions News Using a Topic Code and Eikon Data API get_news_headlines() API Call

This API call takes a flexible Query which can be built visually in the NEWS monitor app and if you double click on the query it forms it into query text which can be pasted into the API call below. Here we use the trade embaro topic code which covers sanctions and also add a regional topic (in this case Central and Eastern Europe) - but you could also leave that out to get all or add another Geographical Topic code you are interested in. More info on topic codes can be found using the TOPICS app.

Obviously some instuments and news queries are going to have more coverage than others so it really depends on the topics you are interested in. Our get_news_headlines() API call has a 100 headline count maximum - so we will need to iterate to build our dataframe to the desired coverage:

    	
            newsdf = pd.DataFrame()
startdf=now
while startdf >= maxenddate:
    try:
        df1 = ek.get_news_headlines(query ='Topic:TRDEMB AND Topic:CEEU', date_to = startdf, count=100)
        startdf = df1['versionCreated'].min().replace(second=0,microsecond=0,tzinfo=None).strftime('%Y/%m/%d %H:%M')
        startdf = datetime.strptime(startdf,'%Y/%m/%d %H:%M')
        if len(df1):
            newsdf = pd.concat([newsdf, df1], axis=0)
        else:
            newsdf = df1
    except Exception:
        break
 
newsdf.info()

<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 4300 entries, 2022-03-10 17:47:16.057000 to 2022-02-08 15:45:54
Data columns (total 4 columns):
#      Column                     Non-Null Count      Dtype
0      versionCreated      4300     non-null      datetime64[ns, UTC]
1       text                          4300     non-null      string
2      storyI                 4300     non-null      string
3      sourceCode            4300     non-null      string
dtypes: datetime64[ns, UTC](1), string(3)
memory usage: 168.0 KB

We can now simply aggregate the frame by date and use the count function to provide a news frequency count. We can clearly see the spike in related news frequency. Monitoring this type of thing (though it can be any topic code or query) can provide a valuable early warning system for a number of events. I suppose a number of these type of feature could be engineered relatively easily for programmatic consumption.

    	
            newsdf['versionCreated'].groupby(newsdf.index.date).count().plot()

For example we can see the impact of this type of financial sanctions on the financial heavy Cypriot Stock Index.

Retreiving Sanctions Files via DataScope Select REST API

Refinitiv’s DataScope Select (DSS) offers a targeted instrument sanctions service to enable smooth and effective sanctions-related trade compliance. Our pre-trade compliance solution delivers current lists of sanctioned entities and their subsidiaries (50 percent ownership or more), including affiliates, but importantly, also lists the individual securities issued by those entities.

This targeted and specific information helps professionals to make informed decisions based on up-to-date, trusted data. To ensure that information remains relevant and accurate, we closely monitor all EU, OFAC and UK published data, and we also regularly expand our service to include new areas and jurisdictions. Our detailed reports include identifiers and reference data for all equities and fixed income securities issued by the sanctioned entities, and detail both the effective and (where applicable) end dates of sanctions. Additionally, our Chinese report covers derivatives, warrants, indices and ETFs, with the file updated on a daily basis.

Jurisdictions covered include:

Iran and North Korea – sanctions imposed by the U.S. and the EU.

Russia – sanctions imposed by the U.S. and the EU.

Russia – additional sanctions imposed by the UK.

Venezuela – sanctions imposed by U.S.

China – sanctions imposed by U.S.

So lets jump straight in - there are 5 permissionable packages available over DSS:

The codes or PackageIDs for these are RUSS, RVEN, REIK, REBR, RCHN. We will download one of these reports - each contains a number of files. For example the RUSS package contains 3 files, one containing EU-Russian santions, one containing UK-Russian sanctions and one containing US-Russian sanctions. Each of these spreadsheets has 6 tabs - Sanctioned Parents, Sanctioned Subsidiaries, Sanctioned Fixed Income Instruments, Sanctioned Equity Instruments, Sanctioned Funds and Sanctioned Affiliates.

So lets authenticate, request a token and download the RUSS package. This abriged code is taken from a fuller sample available here.

Imports

    	
            import requests
import json
import shutil
import time
import urllib3
import gzip
import copy
import asyncio
 
reqStart = "https://selectapi.datascope"  #endpoint URL start
filePath = ".\\downloads\\"  #Location to save downloaded files
myUsername = "Your Username"
myPassword = "Your Password"

Request Authentication Token

    	
            def requestToken(dssUsername, dssRassword) :
    requestUrl = reqStart + ".refinitiv.com/RestApi/v1/Authentication/RequestToken"
 
    requestHeaders={
        "Prefer":"respond-async",
        "Content-Type":"application/json"
        }
 
    requestBody={
        "Credentials": {
        "Username": myUsername,
        "Password": myPassword
    }
    }
 
    r1 = requests.post(requestUrl, json=requestBody,headers=requestHeaders)
 
    if r1.status_code == 200 :
        jsonResponse = json.loads(r1.text.encode('ascii', 'ignore'))
        token = jsonResponse["value"]
    
    else:
        print ('Replace myUserName and myPassword with valid credentials, then repeat the request')
        token = 'None'
    
    return token
 
tokenValid = requestToken(myUsername, myPassword)
print("Authentication token (valid 24 hours): "+tokenValid)

Authentication token (valid 24 hours): _3o0zCYfkl5ZcGn2-Q8VUBP6iaD6fgyW3FjtMzvM4IYQwax5BUfCCaQa1Uv5hIG2LFQ7tzRRv7jo5gmNjF_8iFO82eRowi7iLKNwj_XveFg1TG38uR1ck7ToAnyHoWUrBqGR3ErLwV15-5KAIpjAORLLSDEZrPBVstSW4m6mL8ViQi48rddSErl-tAlWOTEORy8BoATKp4kCImUOI3YrabXQvcxSEpqoo5mO9ysDcwq8N7acILU8T97vS3T_SakyQbjg1qnG13_g2uOsQ4kAPRp-QJlOtS8LCM14tynDYmmI

Request packages

We need to know the UserPackageId for the sanctions pack we are interested in - we can find these programatically through the more involved code sample listed above - but I call them out here for the sake of brevity:

RUSS: "UserPackageId": "0x06559b6f5b6b2f96"

RVEN: "UserPackageId": "0x0701289068a165e6"

REIK: "UserPackageId": "0x07281b976916ae91"

REBR: "UserPackageId": "0x06b4df635d80ca1f"

RCHN: "UserPackageId": "0x076543e2e35728fa"

    	
            packageId = "0x06559b6f5b6b2f96" #RUSS package
 
def requestPackagesDeliveries():
    requestUrl = reqStart + '.refinitiv.com/RestApi/v1/StandardExtractions/UserPackageDeliveryGetUserPackageDeliveriesByPackageId(PackageId=\'' + packageId+'\')'
 
    requestHeaders={
        "Prefer":"respond-async",
        "Content-Type":"application/json",
        "Authorization": "token " + tokenValid
    }
 
    response = requests.get(requestUrl,headers=requestHeaders)
 
    if response.status_code != 200:
        if response.status_code == 401:   # error when token expired
                accessToken = getToken();     # token refresh on token expired
                headers['Authorization'] = "Bearer " + accessToken
                response = requests.request("GET", FILESET_ENDPOINT, data=payload, headers=headers, params=querystring)
    
    if response.status_code == 200:
        jsonFullResp = json.loads(response.text)        
        return jsonFullResp; 
    else:
        return '';
    
resp = requestPackagesDeliveries()
print(json.dumps(resp, indent=2));
 
# form the response, we select 1st packageDeliveryId '0'
packageDeliveryId = resp['value'][0]['PackageDeliveryId']
fileName = resp['value'][0]['Name']
print('Selected packageDeliveryId= ', packageDeliveryId, 'fileName= ',fileName)

{
"@odata.context": "https://selectapi.datascope.refinitiv.com/RestApi/v1/$metadata#UserPackageDeliveries",
"value": [
{
"PackageDeliveryId": "0x07ef871496cdad34",
"UserPackageId": "0x06559b6f5b6b2f96",
"SubscriptionId": "0x06516d2d08ab3036",
"Name": "UK_Russian_Ukrainian_Sanctions_OCP_20220314.xlsx",
"CreateDateTime": "2022-03-14T07:33:27.380Z",
"ReleaseDateTime": "2022-03-14T12:00:00.000Z",
"FileSizeBytes": 585122,
"Frequency": "Weekly",
"ContentMd5": ""
},
...
]
}
Selected packageDeliveryId= 0x07ef871496cdad34 fileName= UK_Russian_Ukrainian_Sanctions_OCP_20220314.xlsx

Retrieve Sanction File and Save- Define

    	
            def retrieveFile(token, packageDeliveryId, fileName):
    requestUrl = reqStart + '.refinitiv.com/RestApi/v1/StandardExtractions/UserPackageDeliveries(\''+packageDeliveryId+'\')/$value'
 
    requestHeaders={
            "Prefer":"respond-async",
            "Content-Type":"application/json",
            "Accept-Encoding":"gzip",
            "Authorization": "token " + token
    }
 
    r = requests.get(requestUrl,headers=requestHeaders,stream=True)    
    r.raw.decode_content = False
    
    fileName = filePath + fileName
    print ('Saving data to file:' + fileName )
   
    chunk_size = 1024
    rr = r.raw
    with open(fileName, 'wb') as fd:
        shutil.copyfileobj(rr, fd, chunk_size)
    fd.close
 
    print ('Finished saving data to file: ' + fileName + '\n')
    
retrieveFile(tokenValid, packageDeliveryId, fileName)

Saving data to file:.\downloads\UK_Russian_Ukrainian_Sanctions_OCP_20220314.xlsx
Finished saving data to file: .\downloads\UK_Russian_Ukrainian_Sanctions_OCP_20220314.xlsx

WorldCheck One FIRI Dataset Download

The Financial Instruments Risk Intelligence (FIRI) dataset has the most complete instrument list coverage of all our services. This dataset is available for download on a daily basis via the World-Check Online SFTP site. Full files and daily delta files are published at 5 PM UTC every day. To access the files, connect to the World-Check SFTP server (sftp.world-check.com) through command-line/PowerShell or through a 3rd party application, such as FileZilla or WINSCP. For example:

The files will appear in the firi directory. The files are available in CSV & XML format and follow the same format and layout as the World-Check Advanced Data Files. For example: