Requesting ESG Bulk PIT Content Set- Python

December 09, 2021

Zoya Farberov

Developer Advocate

Introduction

ESG stands for Environmental, Social and (Corporate) Governance data.

Refinitiv Data Platform (RDP) provides simple web based API access to a broad range of content, including ESG content and ESG content in bulk.

PIT content is newly made available on RDP, and this article will review the recommended approach to requesting complete ESG Bulk PIT content set and keeping it up-to-date.

For information on how to get started working with RDP, including the generation of CLIENT_ID, which is the same as AppKey, please refer to Quickstart guide for Refinitiv Data Platform included in References

Programmatic Approach

Every Monday 9am UTC, both full and delta ESG Bulk PIT filesets will be published.

There are currently 100 constituent initialization or full files, and as we add years, these will increase in 5’s.

At present, there are 5 delta or change files.

There are two ways to stay up-to date:

Every week, we initialize the local ESG Bulk PIT store or database by requesting the full set of initialization ESG Bulk PIT files (.F)
First week, we Initialize the local store by requesting the full set of initialization ESG Bulk PIT files (.F). Consequently, we request the delta ESG Bulk PIT files (.I) that will contain all the changes that were applied to the database since the last week's ESG Bulk PIT .F. files

Therefore, at any beginning of the week, depending on the approach that was selected, we need to request either full set of initialization files, or all the delta/change files, and we never require to request both types of files.

Requirements and Setup to Access RDP ESG Bulk

To interact with RDP platform we require valid RDP credentials and setup:

Import required libraries and define RDP endpoint paths and constants.
Load valid RDP credentials that are permissioned for RDP ESG Bulk access.
Authenticate with RDP using credentials to obtain a valid token

These steps are included in the companion code examples hosted on GitHub (see References section) and are described in detail in many RDP articles, for example https://developers.refinitiv.com/en/article-catalog/article/exploring-news-metadata-refinitiv-data-platform-and-python, so we omit what would be a redundant detailed discussion of these steps here as well, and focus solely on requesting RDP ESG Bulk PIT.

Request Available ESG Bulk PIT File Sets per Package ID

We are going to define bucket to use as a variable, at this time the bucket in use is

RDP_ESG_PIT_BUCKET = 'bulk-esg'

PackageID assigned to PIT content set should be known prior, and at this time it's '4173-aec7-8a0b0ac9-96f9-48e83ddbd2ad'

We are going to define a helper function accepting packageID as parameter, that populates this infromation into attributes of the request:

jsonFullResp = requestFileSets(accessToken, False, '','&packageId='+packageIdPIT);

The helper function will work with a packageID, valid at the time of the request and communicated prior to it.

let us observe that more then one Fileset ID is available

Select ESG Fileset ID

We are going to select FilesetID of a Fileset that is most recent - with maximum "created" timestamp

and next we are ready to

Request File IDs per selected Fileset ID

We are going to define a helper function:

We will call the helper function with pageSize of 100 (max) to obtain the first 100 available results as well as skipToken or nextLink, to obtain the remaining available results.