Datastream API (DSWS) Median Index Data

Author

    Jonathan Legrand
    Developer Advocate

In this article, we will create a Python function that will take the median measure of all (non 'NaN') values of a specific field for any index (or list of indices) of choice using Refinitiv's DataStream Web Services (DSWS).

Contents:

Get Coding 

Import Libraries 

For full replication, note that the version of libraries used

This code is running on Python version 3.7.7

We need to gather our data. Since Refinitiv's DataStream Web Services (DSWS) allows for access to ESG data covering nearly 70% of global market cap and over 400 metrics, naturally it is more than appropriate. We can access DSWS via the Python library "DatastreamDSWS" that can be installed simply by using pip install.

The libraries in the cell below are  native to Python.

The libraries in the cell below are not native to Python - id est (i.e.): they are not installed upon Python's installation. This means that they have their own version independently of Python's version used.

The dateutil library imported in this code is version: 2.8.1
The np library imported in this code is version: 1.18.5
The pd library imported in this code is version: 1.1.4

Collecting Data: A Simple Example 

1st: We have to specify our list of fields. In this example, we have:

  • EVT1FD12: Enterprise Value's ForwarD contract with 12 month maturity (12FRW).
  • EBT1FD12: Earnings Before Taxes' 12FRW.
  • EBD1FD12: Earnings Before Depreciation and amortisation's 12FRW.
  • X(CPS1FD12)*X(IBNOSH): product of the Cashflow Per Share amount's 12FRW and the Institutional Brokers Estimate Service( i.e.: I/B/E/S pulled from datastream's estimate data)'s Number Of Shares.
  • X(MV)~IBCUR: Market Value IBES currency.
  • PEFD12: market share Price to Equity ratio's 12FRW.
  • SAL1FD12: SALes' 12FRW.
  • X(BPS1FD12)*X(IBNOSH): product of the Book value Per Share's 12FRW and the Institutional Brokers Estimate Service's Number Of Shares.

The '  ~USD ' is there to revert the Currency to United States Dollars.

N.B.: X is the variable, so X(...)*X(...) means that we multiply them

Now that we know the fields that we want to capture, we will ask for DSWS to collect such data for all constituents of the MSCI World Index. We need to do a few changed to the mnemonics/tickers used in the DSWS function ' ds.get_data ':

  • We need to add an 'L' at the start to indicate that we are requesting for the list of the constituents of the index.
  • We need to remove '$' as it is not recognised by the backend. We need to remove all currency symbols for this to work.
  • We need to add the month and year (in mmyy format) at the end to specify the point in time that we are trying to replicate. This means that we are capturing data for the list of constituents of that index on that month.
  • We need to add '|L' at the end to indicate to the DSWS service that we are requesting data for a list, not a single object.

N.B.: start = '2013-08-31' collects data from August 2013. If someone was to use ds.get_data("LMSWRLD0313|L", start = '2014-08-31'), they'd be collecting data starting in August 2014 for the constituents listed under the MSWRLD index back in March 2013.

Now that we have stored the data we are looking for in the Python object 'df', we can sort through it:

  Instrument Datatype Value Dates
1608 MS:65955 X(EVT1FD12)~USD NA 2013-08-31
1609 MS:1016 X(EVT1FD12)~USD 49373.9 2013-08-31
1610 MS:388 X(EVT1FD12)~USD 52651.6 2013-08-31
1611 MS:10 X(EVT1FD12)~USD 54133.7 2013-08-31
1612 MS:96942 X(EVT1FD12)~USD 77434.2 2013-08-31
... ... ... ... ...
3211 MS:65552 X(EVT1FD12)~USD 11510.2 2013-08-31
3212 MS:3695 X(EVT1FD12)~USD 8305.29 2013-08-31
3213 MS:3698 X(EVT1FD12)~USD NA 2013-08-31
3214 MS:4502 X(EVT1FD12)~USD NA 2013-08-31
3215 MS:2138 X(EVT1FD12)~USD 82122.3 2013-08-31

1608 rows × 4 columns

 

There are a lot of repeated values, let's use ' .unique() ' to only retrieve unique values within the 'Instrument' column:

  Instrument Datatype Value Dates
0 MS:65955 NAME A P MOLLER MAERSK A 2013-08-31
1608 MS:65955 X(EVT1FD12)~USD NA 2013-08-31
  Instrument Datatype Value Dates
1608 MS:65955 X(EVT1FD12)~USD NA 2013-08-31
1609 MS:1016 X(EVT1FD12)~USD 49373.9 2013-08-31
1610 MS:388 X(EVT1FD12)~USD 52651.6 2013-08-31
1611 MS:10 X(EVT1FD12)~USD 54133.7 2013-08-31
1612 MS:96942 X(EVT1FD12)~USD 77434.2 2013-08-31
... ... ... ... ...
3211 MS:65552 X(EVT1FD12)~USD 11510.2 2013-08-31
3212 MS:3695 X(EVT1FD12)~USD 8305.29 2013-08-31
3213 MS:3698 X(EVT1FD12)~USD NA 2013-08-31
3214 MS:4502 X(EVT1FD12)~USD NA 2013-08-31
3215 MS:2138 X(EVT1FD12)~USD 82122.3 2013-08-31

1608 rows × 4 columns

X(EVT1FD12)~USD

10479.682

Creating Python Function to programmatically collect data

Example setting 'track_progress' and 'add_to_global_DF' to True

Note that - in this example - we made sure to normalize all currency-denominated values to the same currency (USD), otherwise taking the median value of different currencies wouldn't make much sense.

LMSWRLD0113|L
LMSWRLD0213|L
LMSWRLD0313|L

LS&PCOMP0113|L
LS&PCOMP0213|L
LS&PCOMP0313|L

  Instrument Datatype Value Dates
0 MS:65955 NAME A P MOLLER MAERSK A 2013-01-31
1 MS:1016 NAME A P MOLLER MAERSK B 2013-01-31
2 MS:388 NAME ABB LTD N 2013-01-31
3 MS:10 NAME ABBOTT LABORATORIES 2013-01-31
4 MS:96942 NAME ABBVIE 2013-01-31
... ... ... ... ...
43447 MS:65552 X(BPS1FD12)~USD*X(IBNOSH) 6266.1 2013-03-31
43448 MS:3695 X(BPS1FD12)~USD*X(IBNOSH) 3074.03 2013-03-31
43449 MS:3698 X(BPS1FD12)~USD*X(IBNOSH) 34543.6 2013-03-31
43450 MS:4502 X(BPS1FD12)~USD*X(IBNOSH) 4837.31 2013-03-31
43451 MS:2138 X(BPS1FD12)~USD*X(IBNOSH) 20133.9 2013-03-31

43452 rows × 4 columns

Index MSWRLD
Metrics X(EVT1FD12)~USD X(EBT1FD12)~USD X(EBD1FD12)~USD X(CPS1FD12)~USD*X(IBNOSH) X(MV)~USD X(PEFD12) X(SAL1FD12)~USD X(BPS1FD12)~USD*X(IBNOSH)
31/01/2013 9768.0295 1008.731 1159.4745 1033.763 9083.045 13.934 5136.3315 5321.7885
28/02/2013 9748.715 993.057 1166.615 1045.071 9107.255 14.3305 5113.3065 5254.477
31/03/2013 9876.253 1000.378 1151.172 1037.7785 9345.155 14.8205 5117.3 5243.3455
Index S&PCOMP
Metrics X(EVT1FD12)~USD X(EBT1FD12)~USD X(EBD1FD12)~USD X(CPS1FD12)~USD*X(IBNOSH) X(MV)~USD X(PEFD12) X(SAL1FD12)~USD X(BPS1FD12)~USD*X(IBNOSH)
2013-01-31 14746.49 1403.990 1956.5220 1353.3355 13275.135 14.0380 9011.652 5988.997
2013-02-28 15450.20 1404.525 1910.5655 1371.0870 13112.900 14.4570 9073.121 5870.180
2013-03-31 15799.75 1411.040 1921.3090 1368.2955 13821.085 14.8685 9089.930 5936.011
  Instrument Datatype Value Dates
0 MS:65955 NAME A P MOLLER MAERSK A 2013-01-31
1 MS:1016 NAME A P MOLLER MAERSK B 2013-01-31
2 MS:388 NAME ABB LTD N 2013-01-31
3 MS:10 NAME ABBOTT LABORATORIES 2013-01-31
4 MS:96942 NAME ABBVIE 2013-01-31
... ... ... ... ...
43447 MS:65552 X(BPS1FD12)~USD*X(IBNOSH) 6266.1 2013-03-31
43448 MS:3695 X(BPS1FD12)~USD*X(IBNOSH) 3074.03 2013-03-31
43449 MS:3698 X(BPS1FD12)~USD*X(IBNOSH) 34543.6 2013-03-31
43450 MS:4502 X(BPS1FD12)~USD*X(IBNOSH) 4837.31 2013-03-31
43451 MS:2138 X(BPS1FD12)~USD*X(IBNOSH) 20133.9 2013-03-31
  Instrument Datatype Value Dates
0 MS:65955 NAME A P MOLLER MAERSK A 2013-01-31
1 MS:1016 NAME A P MOLLER MAERSK B 2013-01-31
2 MS:388 NAME ABB LTD N 2013-01-31
3 MS:10 NAME ABBOTT LABORATORIES 2013-01-31
4 MS:96942 NAME ABBVIE 2013-01-31
... ... ... ... ...
43447 MS:65552 X(BPS1FD12)~USD*X(IBNOSH) 6266.1 2013-03-31
43448 MS:3695 X(BPS1FD12)~USD*X(IBNOSH) 3074.03 2013-03-31
43449 MS:3698 X(BPS1FD12)~USD*X(IBNOSH) 34543.6 2013-03-31
43450 MS:4502 X(BPS1FD12)~USD*X(IBNOSH) 4837.31 2013-03-31
43451 MS:2138 X(BPS1FD12)~USD*X(IBNOSH) 20133.9 2013-03-31
         
Index MSWRLD
Metrics X(EVT1FD12)~USD X(EBT1FD12)~USD X(EBD1FD12)~USD X(CPS1FD12)~USD*X(IBNOSH) X(MV)~USD X(PEFD12) X(SAL1FD12)~USD X(BPS1FD12)~USD*X(IBNOSH)
31/01/2013 9768.0295 1008.731 1159.4745 1033.763 9083.045 13.934 5136.3315 5321.7885
28/02/2013 9748.715 993.057 1166.615 1045.071 9107.255 14.3305 5113.3065 5254.477
31/03/2013 9876.253 1000.378 1151.172 1037.7785 9345.155 14.8205 5117.3 5243.3455
  Instrument Datatype Value Dates
0 MS:65955 NAME A P MOLLER MAERSK A 31/01/2013
1 MS:1016 NAME A P MOLLER MAERSK B 31/01/2013
2 MS:388 NAME ABB LTD N 31/01/2013
3 MS:10 NAME ABBOTT LABORATORIES 31/01/2013
4 MS:96942 NAME ABBVIE 31/01/2013
... ... ... ... ...
43447 MS:65552 X(BPS1FD12)~USD*X(IBNOSH) 6266.1 31/03/2013
43448 MS:3695 X(BPS1FD12)~USD*X(IBNOSH) 3074.03 31/03/2013
43449 MS:3698 X(BPS1FD12)~USD*X(IBNOSH) 34543.6 31/03/2013
43450 MS:4502 X(BPS1FD12)~USD*X(IBNOSH) 4837.31 31/03/2013
43451 MS:2138 X(BPS1FD12)~USD*X(IBNOSH) 20133.9 31/03/2013

You can find information inscribed inbetween the triple double quotes (*i.e.*:""" """) above with the Python functino ' help() ':

Help on function DSWS_Median_Index_Data in module __main__:

DSWS_Median_Index_Data(indices=['MSWRLD'], list_of_fields=['X(EVT1FD12)~USD', 'X(EBT1FD12)~USD', 'X(EBD1FD12)~USD', 'X(CPS1FD12)~USD*X(IBNOSH)', 'X(MV)~USD', 'PEFD12', 'X(SAL1FD12)~USD', 'X(BPS1FD12)~USD*X(IBNOSH)'], start='2013-01-01', end='2021-01-31', track_progress=False, add_to_global_DF=False)
' DSWS_Median_Index_Data ' requests data defined in ' list_of_fields ' from Datastream's API - DataStream Web
Services (DSWS) - for a series of indices of choice - defined in 'indices' - and returns three lists full of Pandas data-frames:
1st - the raw output from the requests (with columns 'Instrument', 'Datatype', 'Value' and 'Dates');
2nd - the monthly median from the raw output from the requests (with columns of the requested metrics and an index of dates);
3rd - the raw data pulled from the DSWS request (and where issues are most likely to lie).

One could re-write this basing themselves off datetime objects, it would be equivalent; its code may look neater too.

indices (list): List of strings of the DSWS mnemonic for the index for which data is requested.
Default: indices = ["MSWRLD"]

list_of_fields (list): List of strings of the DSWS fields we are after. This has to be comprised of 49 elements or less.
Don't include non numerical fields like 'GDSCN'. That will result in a printed error
Default: list_of_fields = ['EVT1FD12', 'EBT1FD12', 'EBD1FD12', 'X(CPS1FD12)*X(IBNOSH)', 'X(MV)~IBCUR', 'PEFD12', 'SAL1FD12', 'X(BPS1FD12)*X(IBNOSH)', '(X(SAL1FD12)*X(IBUNIT))', '354E(X)', 'X(INC1FD12)*X(IBUNIT)', 'X(FCF1FD12)*X(IBNOSH)', 'X(FCF1FD12)*X(IBNOSH)*X(IBUNIT)', 'GDSCN']

start (str): String of the start date from which we are asking for data in 'yyy-mm-dd' format.
Default: start = '2013-01-01'

end (str): String of the end date until which we are asking for data in 'yyy-mm-dd' format.
Default: end = '2021-01-31'

track_progress (Boolean): If set to True, the ' DSWS_Median_Index_Data ' function will print out each mnemonic requested of DSWS
Default: track_progress = False


track_progress (Boolean): If set to True, each string of the transformed index will be printed off - remember that there is one per month.
Default: track_progress = False

add_to_global_DF (Boolean): If set to True, on top of returning the 3 lists of Pandas data-frames described above, this function will append lists named DF, _DF, and debug_df.
This does mean that you need to include a line such as 'DF, _DF, debug_df = [], [], []' beforehand as a result.
Default: add_to_global_DF = False

 

References

You can find more detail regarding the DSWS API and related technologies for this article from the following resources:

For any question related to this example or Eikon Data API, please use the Developers Community Q&A Forum.

  • Login
  • Please Login
Contact Us MyRefinitiv