Author:
Overview
In this article, we explore the building blocks of futures RICs, including how the RIC changes after it expires. Then we build a Python object which helps to reconstruct and return futures RICs along with several metadata. The challenge we are trying to address here is that one cannot directly access expired futures through a single API call. To get historical data on futures, one should either use futures continuation RICs or will need to reconstruct RICs following the logic of Refinitiv RIC construction rules.
One reason a market participant may want to reconstruct the RIC instead of relying on prices from the continuation RIC, is the assumptions on when rolls take place. Refinitiv rolls on the last trading day, however many market participants may decide to roll with a different assumption in mind depending on the use case. For example, if a market is illiquid at the time of roll, exchange position caps may apply for large positions. In this case a market participant may want to spread the roll over several days or simply at a specific time of a day. So, if market participants can get individual futures, they can determine their own roll logic. Ultimately, prices from individual futures are much closer to reality than simply using a generically rolled series.
In the scope of this article, we show how to reconstruct futures RICs and provide an object class with functions to do that. This solution would be most useful to reconstruct expired futures for the purpose of getting historical prices which will be further used in multiple financial use cases, such as asset valuation and strategy back testing.
The image above provides example for bond, index, and commodity futures. The good news is, as we can see, all the futures, no matter the asset class, keep the same structure. If you have ever investigated options RIC reconstruction in one of your use cases, you would come across this challenge, when option RICs depending on the asset class, exchange they are traded etc., although share the same logic, have differing structures and components. If you are still having a challenge for reconstructing expired (and not only) option RICs, functions introduced one of my articles might be useful.
Moving back to the futures RIC components we see that the RICs consist of three main building blocks:
- RIC root code - which is the RIC code for the underlying asset, such as US for US T Bond, FFI for the FTSE100 and C for the corn,
- Delivery month Code - as the name indicates, this a letter demonstrating the month code where the futures will be delivered. The comprehensive list of the codes can be found in next image of this article,
- Expiry year code - this is the last digit of the year of the futures expiration.
Below, we also present the Delivery month and Quarter delivery code lists for futures contracts.
Below, we present also the Delivery month and Quarter delivery code lists for futures contracts.
Once a future has expired, focus moves to the next contract. If using a rolling contract RIC such as LCOc1, the contract will roll automatically. To go back and look at historical data one must use a specific RIC structure. The changes to the RIC structure after the futures contract expire are quite straightforward, particularly after the existing RIC we add:
- "^" carat symbol indicating that the futures are expired which is followed by:
- Decade of the expiry of the contract, e.g., 0 for 2000, 1 for 2015, 2 for 2022 etc.
The image below represents the RIC structure for Soybeans futures contract that expired in September 201
!pip install refinitiv-data
import refinitiv.data as rd
from refinitiv.data.content import search
from datetime import datetime
import calendar
import pandas as pd
To use the API, we first need to open either desktop or a platform session. In the scope of this article, we will be connected through a desktop session. More about the session types and how to connect the API can be found here.
rd.open_session()
<refinitiv.data.session.Definition object at 0x15da8bb50 {name='workspace'}>
Build Futurs RIC constructor object
To reconstruct futures RIC, we have built a Python class with a set of functions allowing us to construct the components of future RICs, form a search query and request data using Refinitiv Search API.
The class is using a constant named MONTH_CODES which lists month codes and month numbers as key/value pairs of a dictionary. Apart from this, the FuturesRICs class comprises of the following functions:
- get_futures - this is the main function of the object which accepts underlying (str), month (either month_code(str) or month number (int)) and year (int) as an input and returns a dataframe containing futures RIC. The process involves getting the numerical and code representation of the month, forming the search query, defining the search filtering criteria and making a request using Refinitiv Search API. This function is calling the following functions to achieve that:
- _get_month_number_and_code - this function enables users to provide either numerical or code input for the month argument when calling get_futures function from the FuturesRICs object. This function takes month (str, int) as an input and returns numerical(str) and code(str) representation of it by calling different logic depending on the type of user input for month parameter of get_futures function.
- _search_futures - this function implements the actual API call by accepting the search query (str) and filter (str)criteria as input. It returns a dataframe with future ric (df) and other related metadata.
- _build_filter_criteria - this function builds the criteria string to be used in the search
It should also be noted that to account for the RIC structure change related to the year code introduced in 2024 for set of RICs, we define two types of year_codes and check RIC for both. RIC structure data notification related to change can be found here.
class FuturesRICs:
MONTH_CODES = {'F': '01', 'G': '02', 'H': '03', 'J': '04', 'K': '05', 'M': '06',
'N': '07', 'Q': '08', 'U': '09', 'V': '10', 'X': '11', 'Z': '12'}
ASSET_CATEGORY = "Future"
DATE_FORMAT = "%Y-%m-%d"
def get_futures(self, underlying, month, year):
month_num, month_code = self._get_month_number_and_code(month)
month_last_day = calendar.monthrange(year, int(month_num))[1]-1
# we define two possible year_codes to account for RIC structure change
year_codes = [str(year)[-1], str(year)[-2:]]
for year_code in year_codes:
query = f'{underlying}{month_code}{year_code}*'
filter_criteria = self._build_filter_criteria(year, month_num, month_last_day, query)
response = self._search_futures(query, filter_criteria)
if response is not None and not response.empty:
return response
print(f'No futures contract for {underlying} expiring on {month_num} month of {year}')
return None
def _search_futures(self, query, filter_criteria):
response = search.Definition(
view=search.Views.SEARCH_ALL,
query=query,
select="DocumentTitle, RIC, ExchangeCode, ExpiryDate, " +
"UnderlyingQuoteRIC, RCSAssetCategoryLeaf, RetireDate",
filter=filter_criteria
).get_data()
return response.data.df if response else None
def _get_month_number_and_code(self, month):
if isinstance(month, int) or month.isnumeric():
month_num = f'{int(month):02}'
month_code = next((code for code, num in self.MONTH_CODES.items() if num == month_num), None)
if month_code is None:
raise ValueError(f"Invalid numeric month: {month}")
else:
month_code = month.upper()
month_num = self.MONTH_CODES.get(month_code)
if month_num is None:
raise ValueError(f"Invalid month code: {month_code}")
return month_num, month_code
def _build_filter_criteria(self, year, month_num, month_last_day, query):
return (
f"RCSAssetCategoryLeaf eq '{self.ASSET_CATEGORY}' and "
f"ExpiryDate ge {year-1}-{month_num}-{month_last_day} and "
f"ExpiryDate le {year+1}-{month_num}-{month_last_day} and "
f"(RIC xeq '{query[:-1]}' or RIC xeq '{query[:-1]}^{str(year)[-2]}')"
)
As one can rightfully notice after observing the object above, the key for the reconstruction process lies within the query and filtering criteria of the Search function. So, it worth explaining how these are constructed:
query - query variable includes the components of the futures RIC, such as the underlying, month code and the last digit of the year of future expiration as described in the first image of this article. This expression is then followed by "*" symbol which asks the API to return all inputs starting with the expression. This ensures we get the expired futures as well they have the same structure followed by the "^" symbol and decade of the expiration year.
filtering_criteria - As the query is quite generic and there can be many matches, we need to filter down the outputs:
- we ask for items which have Future under RCSAssetCategoryLeaf property. This will leave only the future contracts
- although we have only futures with the matched expression, the query output will include futures contracts from multiple expiration periods. Thus, we need to filter out the ones expiring +-1 year of the expiration period. The reason we ask for futures expiring +- the required year is to cover edge cases, such as:
- -1-year - to cover January contracts expired in December previous year,
- +1-year December contract expiring the January or even March (e.g., SON3) of the following year
We use the month from the function input and last day -1 (to accommodate for the leap year) of that month to form the final date ranges for ExpiryDate property . 3. as we have used date range for expiry date, it left out the outputs which ate out of the range, however it may still include a couple of outputs belonging the range for some underlings. The final criteria which will help us to ensure we return a single and correct futures contract RIC we ask for the output having RIC exactly equal to the expected (constructed according to the Refinitiv logic) RIC (active or matured).
As it comes to the actual outputs of the object, it returns the fields provided under select of the Search query. In our case, it includes Name of the futures (DocumentTitle), RIC, Exchange Code, Expiry and Retire Dates, UnderlyingQuoteRIC and asset type (RCSAssetCategoryLeaf). You can ask for more fields just adding a select property from available metadate. More on how to check the available metadata, or how to use search are presented in this article.
fr = FuturesRICs()
Below, let's request furures contract for Soybean expired on March 2021.
fr.get_futures('S', 3, 2021)
The result from the function above is a dataframe which contains the fields we have specified in the search select, including Futures contract name, RIC, Exchange Code and Expiration, Retirement dates.
It should be noted that different futures contracts have differing contract specifications, including the contract month. So that specifications should be considered when requesting an option contract RICs. To review futures contract specifications for an underlying we can search the continuation RIC in Workspace and click on the Contract Specification Tab as presented in the image below.
As we can see from the image, contract months for Soybean futures are Jan, Mar, May, Aug, Sep, and Nov, meaning if we request a soybean contract let us say for February 2021, a valid RIC will not exist. In that case, our object will return a response "No futures contract for S expiring on 02 month of 2021".
fr.get_futures('S', 2, 2021)
No futures contract for S expiring on 02 month of 2021
Now let us test our object for different underlings and with valid expiration periods. In order not to print the whole dataframe output we derive the RIC from the output and print it.
print(fr.get_futures('AD', 3, 2000)['RIC'][0])
print(fr.get_futures('ES', 3, 2002)['RIC'][0])
print(fr.get_futures('FLG', 9, 2004)['RIC'][0])
print(fr.get_futures('JY', 9, 2006)['RIC'][0])
print(fr.get_futures('NG', 5, 2008)['RIC'][0])
print(fr.get_futures('S', 1, 2010)['RIC'][0])
print(fr.get_futures('W', 7, 2012)['RIC'][0])
print(fr.get_futures('VX', 8, 2014)['RIC'][0])
print(fr.get_futures('SB', 10, 2016)['RIC'][0])
print(fr.get_futures('JGB', 3, 2018)['RIC'][0])
print(fr.get_futures('SON3', 6, 2020)['RIC'][0])
print(fr.get_futures('FFI', 9, 2022)['RIC'][0])
print(fr.get_futures('FDX', 12, 2023)['RIC'][0])
print(fr.get_futures('ES', 3, 2023)['RIC'][0])
print(fr.get_futures('LCO', 2, 2026)['RIC'][0])
ADH0^0
ESH2^0
FLGU4^0
JYU6^0
NGK8^0
SF0^1
WN2^1
VXQ4^1
SBV6^1
JGBH8^1
SON3M0^2
FFIU2^2
FDXZ3
ESH3
LCOG6
Finally, let's take one of the RICs above, e.g the one on S&P 500 (ES) expiring on March 2002, and request prices information using RD Libraries.
rd.get_history(
universe=['ESH2^0'],
fields = ['TRDPRC_1', 'OPEN_PRC', 'HIGH_1', 'LOW_1'],
interval="1D",
start="2001-10-01",
end="2002-03-30",
)
Conclusion
In this article, we presented the components of RIC futures contracts, including the ones expired and introduced a Python object to reconstruct futures RICs on different assets no matter the expiration fact. The reconstructed RICs then can be used to derive historical prices. Although, market participants could request pricing information using continuation RICs, they may not always agree on the futures rolling assumptions made by Refinitiv. To be able to utilize custom roll assumptions and get more realistic historical prices they need to get the individual contracts. As historical contracts are not directly accessible via a single API call, RICs for these contracts need to be reconstructed following the logic provided by Refinitiv. For this purpose, we have built a Python object consisting of several functions which simply accept the underlying, expiration month and year as an input and return a dataframe with contract RIC and several other metadata, including contract name, exchange, and expiration details.
- Register or Log in to applaud this article
- Let the author know how much this article helped you
