| Last update | Nov 2023 |
| Environment | Windows |
| Language | C# |
| Compilers | Microsoft Visual Studio 2019 |
| Prerequisites | DSS login, internet access, having done the previous tutorials |
| Source code | Download .Net SDK Tutorials Code |
This is the ninth tutorial in a series of .Net SDK tutorials. It is assumed that the reader has acquired the knowledge delivered in the previous tutorials before following this one.
This tutorial is similar to Tutorial 8, but replaces the T&C (Terms and Conditions) extraction with a Composite extraction, which combines T&C and Pricing data.
In this example we retrieve a few composite fields related to annualized dividends and the balance sheet.
Again, we also cross reference instrument codes, i.e. retrieve several instrument codes (RIC; Cusip, ISIN, Sedol, etc.), for a list of instruments (with different instrument codes). For each input instrument code, only 1 RIC is returned (the primary RIC).
Like tutorials 6 and 7, this sample also contains 2 extraction requests, one with the ExtractWithNotes endpoint, the other with the ExtractRaw endpoint, to illustrate the differences between the two.
The code installation was done in Tutorial 1.
Opening the solution is similar to what was done in the previous tutorials:
Before anything else, you must reference the DSS REST API .Net SDK in the Microsoft Visual Studio project.
Important: this must be done for every single tutorial, for both the learning and refactored versions.
This was explained in the tutorial 2; please refer to it for instructions.
In Microsoft Visual Studio, in the Solution Explorer, double click on Program.cs and on DssClient.cs to display both file contents. Each file will be displayed in a separate tab.
Before running the code, you must replace YourUserId with your DSS user name, and YourPassword with your DSS password, in these 2 lines of Program.cs:
private static string dssUserName = "YourUserId";
private static string dssUserPassword = "YourPassword";
Important reminder: this must be done for every single tutorial, for both the learning and refactored versions.
Failure to do so will result in an error at run time (see Tutorial 1 for more details).
We shall only describe what is new versus the previous tutorials.
This is the same as the version of Tutorial 8, except for the leading comment, and 2 added methods, to create and run a Composite extraction, one for JSON formatted data, the other for compressed CSV data.
These are both very similar to the other methods to create and run On Demand extractions, except that this time we declare a new CompositeExtractionRequest. Again, we use as parameters the instrument identifiers and requested field names arrays defined in Program.cs.
Here is the first method, for JSON data:
public ExtractionResult CreateAndRunCompositeExtraction(
InstrumentIdentifier[] instrumentIdentifiers, string[] contentFieldNames)
{
CompositeExtractionRequest extractionComposite = new CompositeExtractionRequest
{
IdentifierList = InstrumentIdentifierList.Create(instrumentIdentifiers),
ContentFieldNames = contentFieldNames
};
//Run the extraction.
//This call is blocking, it returns when the extraction is completed:
return extractionsContext.ExtractWithNotes(extractionComposite);
}
Here is the second one, for compressed CSV data:
public RawExtractionResult CreateAndRunCompositeRawExtraction(
InstrumentIdentifier[] instrumentIdentifiers, string[] contentFieldNames)
{
CompositeExtractionRequest extractionComposite = new CompositeExtractionRequest
{
IdentifierList = InstrumentIdentifierList.Create(instrumentIdentifiers),
ContentFieldNames = contentFieldNames
};
//Run the extraction.
//This call is blocking, it returns when the extraction is completed:
return extractionsContext.ExtractRaw(extractionComposite);
}
The Composite extraction request is just one of many possibilities. The Example Application explained in the Quick Start illustrates many more On Demand extractions.
We create an array of 4 instrument identifiers and populate it manually, using the same helper method as in Tutorial 8:
InstrumentIdentifier[] instrumentIdentifiers = PopulateInstrumentIdentifiers();
The method is very simple, as the purpose of this Tutorial is to concentrate on the extraction itself. For a more productized example with details on handling large instrument lists read from file, see tutorial 7.
For the demo we use 4 instruments, each one defined using a different identifier type: 1 Cusip, 1 Ric, 1 Isin and 1 Sedol.
We will use this array when we define the extraction.
To create the field name array we proceed just like in the previous tutorials, calling the helper method we created in Tutorial 2:
string[] requestedFieldNames = CreateRequestedFieldNames();
For this demo the helper method defines a list of fields including 5 instrument codes (the use case is cross referencing), and the currency code, and 4 pricing fields (these are not available in a T&C extraction):
static string[] CreateRequestedFieldNames()
{
string[] requestedFieldNames = { "RIC", "CUSIP", "ISIN", "SEDOL", "Issuer OrgID",
"Currency Code",
"Annualized Dividend Period Start Date",
"Annualized Dividend Adjusted Gross Amount",
"Balance Sheet - Enterprise Value",
"Balance Sheet - Market Value" };
return requestedFieldNames;
}
As a reminder, an explanation on how to choose field names is available in Tutorial 2, at the end of the section on report templates, under the heading: How to choose a report template format, and find out what field names are available ?
We will use this array when we define the extraction.
On Demand extractions do not require these, as explained in Tutorial 5.
This is what we did in the previous tutorials. The extraction uses the ExtractWithNotes endpoint, which delivers the data in JSON format. We call a new DSS client helper method (the only difference with the methods used in preceding tutorials is the type of data we request):
ExtractionResult extractionResult =
dssClient.CreateAndRunCompositeExtraction(instrumentIdentifiers, requestedFieldNames);
DssCollection<ExtractionRow> extractionDataRows = extractionResult.Contents;
It is a blocking API call which only returns when the extraction is complete, so there is no need for code to check for extraction completion.
The On Demand extraction returns extraction data rows. We process them using the helper methods from tutorial 6:
DisplayAndLogExtractedDataFieldNames(dataOutputFile, extractionDataRows);
DisplayAndLogExtractedDataFieldValues(dataOutputFile, extractionDataRows);
We also process the extraction notes:
DisplayAndLogAndAnalyzeExtractionNotes(notesOutputFile, errorOutputFile, extractionResult);
This is a different possibility we illustrate in this tutorial, like we also did in tutorials 6 and 7. Instead of retrieving JSON formatted data we want compressed CSV data, which is practical for storage, and optimises extractions of large data sets.
Instead of the ExtractWithNotes endpoint (which delivers the data in JSON format), the extraction uses the ExtractRaw endpoint, which delivers compressed CSV formatted data.
Depending on your use case, you can choose either ExtractWithNotes or ExtractRaw.
We call a new helper method, that is very similar to the one we used above:
RawExtractionResult rawExtractionResult =
dssClient.CreateAndRunCompositeRawExtraction(instrumentIdentifiers, requestedFieldNames);
The difference between this helper method and the previous one is the endpoint, and the fact that it returns a raw extraction result. This also implies that we do not extract data rows from the extraction result contents.
We process them using the helper methods from tutorial 6:
dssClient.SaveCompressedData(rawExtractionResult, gzipDataOutputFile);
DisplayAndLogAndAnalyzeRawExtractionNotes(gzipNotesOutputFile, errorOutputFile, rawExtractionResult);
As stated previously, cleanup on the DSS server is not required, as there is nothing to delete in this case.
The full code can be displayed by opening the appropriate solution file in Microsoft Visual Studio.
List of the main steps in the code:
We do not create a report template or schedule, and there is no need to cleanup.
There is none for this tutorial.
Don’t forget to reference the DSS SDK, and to set your user account in Program.cs !
After running the program, and pressing the Enter key when prompted, the final result should look like this:
Returned session token: <token>
Press Enter to continue
Cleared data output file C:\DSS REST API\DSS_API_tutorial_9l_output.csv
Cleared notes output file C:\DSS REST API\DSS_API_tutorial_9l_notes.txt
Cleared error output file C:\DSS REST API\DSS_API_tutorial_9l_errors.txt
Cleared compressed data output file C:\DSS REST API\DSS_API_tutorial_9l_gzipOutput.csv.gz
Cleared compressed data notes output file C:\DSS REST API\DSS_API_tutorial_9l_gzipNotes.txt
Press Enter to continue
We created an array with 5 instrument identifiers, and an array of field names, outside of DSS.
Next we will launch a direct on demand composite (T&C and pricing) extraction,
using these 2 arrays.
Press Enter to continue
Please be patient and wait for the extraction to complete ...
Returned list of field names:
RIC,CUSIP,ISIN,SEDOL,Issuer OrgID,Currency Code,Annualized Dividend Period Start Date,Annualized Dividend Adjusted Gro
ss Amount,Balance Sheet - Enterprise Value,Balance Sheet - Market Value,
Press Enter to continue
Returned field values:
======================
00209TAB1=RRPS, 00209TAB1, US00209TAB17, B7SDDX3, 115149232, USD, , , ,
IBM.N, 459200101, US4592001014, 2005973, 18228, USD, 6/10/2022 12:00:00 AM, 6.6, 160471580826, 126667500000
IBM.N, 459200101, US4592001014, 2005973, 18228, USD, 6/10/2022 12:00:00 AM, 6.6, 160471580826, 126667500000
LP71000002, , GB00B1YW4409, B1YW440, 100691344, GBp, , , ,
, 438516AC0, US438516AC05, , 18181, USD, , , ,
Extraction completed.
Number of data rows: 5
Number of valid (non empty) data rows: 5
Output was also written to file.
Press Enter to continue
Extraction Notes:
=================
Extraction Services Version 16.0.43633 (806c08a4ae8f), Built May 9 2022 17:21:13
Holiday Rollover of Universal Close Price waived.
Processing started at 06/09/2022 10:51:02.
User ID: 9008895
Extraction ID: 595094113
Correlation ID: CiD/9008895/rAyz2w.080b6da10c2df66a/RA/EXT.595094113
Schedule: _OnD_0x080b6da10c3df66a (ID = 0x080b6da10dfdf66a)
Input List (5 items): _OnD_0x080b6da10c3df66a (ID = 080b6da10c9df66a) Created: 06/09/2022 10:51:00 Last Modified: 06/0
9/2022 10:51:01
Schedule Time: 06/09/2022 10:51:01
Report Template (16 fields): _OnD_0x080b6da10c3df66a (ID = 0x080b6da10c4df66a) Created: 06/09/2022 10:51:00 Last Modif
ied: 06/09/2022 10:51:00
(CSP,438516AC0,EJV) is inactive.
Processing completed successfully at 06/09/2022 10:51:03, taking 1.22 Secs.
Extraction finished at 06/09/2022 09:51:03 UTC, with servers: x01a03, DSWS (0.5 secs), QSDHA1 (0.0 secs), QSHC20 (0.2
secs)
Usage Summary for User 9008895, Client 65508, Template Type Composite
Base Usage
Instrument Instrument Terms Price
Count Type Subtype Source Source
------- ----------------------------------- ---------------------------- -------------- ------------------------------
----------
1 Corporate Investment Grade N/A N/A
1 Corporate N/A N/A
2 Equities N/A N/A
1 Mutual Funds Lipper N/A
-------
5 Total instruments charged.
0 Instruments with no reported data.
=======
5 Instruments in the input list.
No Evaluated Pricing Service complex usage to report -- 5 Instruments in the input list had no reported data.
Writing RIC maintenance report.
Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType
===============================================================================
SUCCESS: processing completed successfully.
Inactive instruments messages:
(CSP,438516AC0,EJV) is inactive.
===============================================================================
Press Enter to continue
We now run the same extraction, to retrieve Gzipped CSV data.
Please be patient and wait for the extraction to complete ...
Saved the compressed data file to disk:
C:\DSS REST API\DSS_API_tutorial_9l_gzipOutput.csv.gz
Press Enter to continue
Extraction Notes:
=================
Extraction Services Version 16.0.43633 (806c08a4ae8f), Built May 9 2022 17:21:13
Holiday Rollover of Universal Close Price waived.
Processing started at 06/09/2022 10:51:37.
User ID: 9008895
Extraction ID: 595094223
Correlation ID: CiD/9008895/rAyz2w.080b6da34c4df66a/RA/EXT.595094223
Schedule: _OnD_0x080b6da34c5df66a (ID = 0x080b6da34ffdf66a)
Input List (5 items): _OnD_0x080b6da34c5df66a (ID = 080b6da34f2df66a) Created: 06/09/2022 10:51:34 Last Modified: 06/0
9/2022 10:51:34
Schedule Time: 06/09/2022 10:51:34
Report Template (10 fields): _OnD_0x080b6da34c5df66a (ID = 0x080b6da34c6df66a) Created: 06/09/2022 10:51:34 Last Modif
ied: 06/09/2022 10:51:34
(CSP,438516AC0,EJV) is inactive.
Processing completed successfully at 06/09/2022 10:51:37, taking 1.07 Secs.
Extraction finished at 06/09/2022 09:51:37 UTC, with servers: x01a03, DSWS (0.5 secs), QSDHA1 (0.0 secs), QSHC11 (0.2
secs)
Usage Summary for User 9008895, Client 65508, Template Type Composite
Base Usage
Instrument Instrument Terms Price
Count Type Subtype Source Source
------- ----------------------------------- ---------------------------- -------------- ------------------------------
----------
1 Corporate Investment Grade N/A N/A
1 Corporate N/A N/A
2 Equities N/A N/A
1 Mutual Funds Lipper N/A
-------
5 Total instruments charged.
0 Instruments with no reported data.
=======
5 Instruments in the input list.
No Evaluated Pricing Service complex usage to report -- 5 Instruments in the input list had no reported data.
Writing RIC maintenance report.
Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType
===============================================================================
SUCCESS: processing completed successfully.
Inactive instruments messages:
(CSP,438516AC0,EJV) is inactive.
===============================================================================
Press Enter to continue
Press any key to continue . . .
If the user name and password were not set properly, an error will be returned. See Tutorial 1 for details.
This tutorial continued with the simplified high level calls for On Demand extraction requests, with a simple use case for instrument cross referencing, and simultaneous price retrieval.
Like tutorials 6 and 7, it also compared the use of two endpoints; one that delivers JSON formatted data, for easy direct treatment, the other that delivers the data as a compressed CSV for effective download and storage of large data sets.
Now move on to the next tutorial, which covers a different topic.