Last update | Dec 2023 |
Environment | Windows |
Language | C# |
Compilers | Microsoft Visual Studio |
Prerequisites | DSS login, internet access, having done the previous tutorial |
Source code | Download .Net SDK Tutorials Code |
This is the fourth tutorial in a series of .Net SDK tutorials. It is assumed that the reader has acquired the knowledge delivered in the previous tutorials before following this one.
This tutorial looks at more programmatic GUI control. Building on the previous tutorial, it covers the following new topics:
The code installation was done in Tutorial 1.
Opening the solution is similar to what was done in Tutorial 1:
Important: this must be done for every single tutorial, for both the learning and refactored versions.
This was explained in the tutorial 2; please refer to it for instructions.
For this and the following tutorial an additional library must be referenced: SharpZipLib from ICSharpCode. The DLL is delivered with the code samples, but if you prefer you can also use NuGet.
In Microsoft Visual Studio, in the Solution Explorer, double click on Program.cs and on DssClient.cs to display both file contents. Each file will be displayed in a separate tab.
Before running the code, you must replace YourUserId with your DSS user name, and YourPassword with your DSS password, in these 2 lines of Program.cs::
private static string dssUserName = "YourUserId";
private static string dssUserPassword = "YourPassword";
Important reminder: this will have to be done again for every single tutorial, for both the learning and refactored versions.
Failure to do so will result in an error at run time (see Tutorial 1 for more details).
Once a schedule has run, and the requested data was extracted from the DSS server database, several resulting extraction files are generated. They are:
The names of the files (generated by a single extraction) share a common root, with differing extensions added. The root file name can be specified when creating a schedule; as this is optional, if none was specified the DSS server will automatically generate a (very complex) root file name.
Example file names, for a specified root file name myExtraction:
The extraction files are stored on the DSS server. They can be retrieved using various mechanisms:
This tutorial and the following demonstrate the 2rd method, using the API. Manual retrieval is out of the scope of these tutorials; for these, more information is available in the help pages of the web GUI.
When using the API:
Read on to understand how this is done.
We shall only describe what is new versus the previous tutorial.
This is nearly the same as the refactored version of Tutorial 3, except for the leading comment, and 2 small changes:
We expose our extractions context to our main program:
public ExtractionsContext extractionsContext;
We add Monday to our recurring schedule creation method (to avoid the demo failing on a Monday).
No additional explanations are required as the rest of the code was described in the previous tutorials.
At the top of the code we see the using directives. As we are using a whole set of API calls, several using directives are required to give us access to the types of the DSS API namespace, so that we do not have to qualify the full namespace each time we use them. As in the previous tutorials, these are followed by a using directive referring to the namespace of our code. In addition, we have a directive for the SharpZipLib library:
using ICSharpCode.SharpZipLib.GZip;
using DataScope.Select.Api.Extractions;
using DataScope.Select.Api.Content;
using DataScope.Select.Api.Extractions.SubjectLists;
using DataScope.Select.Api.Extractions.ReportTemplates;
using DataScope.Select.Api.Extractions.Schedules;
using DataScope.Select.Api;
using DataScope.Select.Api.Extractions.ReportExtractions;
using DataScope.Select.Api.Core;
using DssRestfulApiTutorials;
In the main code block, after connecting to the server, we expose the extractions context from DssClient (to avoid having to use the dssClient. prefix before each use of extractionsContext):
ExtractionsContext extractionsContext = dssClient.extractionsContext;
In the previous tutorial we created an instrument list, but did not worry about validating the instrument identifiers; we just assumed they were successfully appended to the list. In this tutorial we want to check if any issues occurred.
To illustrate this, this time we intentionally include 2 invalid instrument identifiers (in an array of 7):
new InstrumentIdentifier
{ Identifier = "INVALID", IdentifierType = IdentifierType.Cusip,
UserDefinedIdentifier = "INVALID INSTRUMENT_1" },
new InstrumentIdentifier
{ Identifier = "JUNK.JUNK", IdentifierType = IdentifierType.Ric,
UserDefinedIdentifier = "INVALID INSTRUMENT_2" }
To validate the instrument identifiers, we capture the result returned by the API call that appends instrument identifiers to an instrument list:
InstrumentsAppendIdentifiersResult appendResult =
extractionsContext.InstrumentListOperations.AppendIdentifiers(
instrumentList, instrumentIdentifiers, false);
This result contains useful instrument identifier validation information. For each invalid instrument identifier a message and severity are generated. We can retrieve and display them:
foreach (InstrumentValidationMessage message in appendResult.ValidationResult.Messages)
Console.WriteLine("- " + message.Message, message.Severity);
We can also get the valid instrument count:
DebugPrintAndWaitForEnter("Valid instrument count: " +
appendResult.ValidationResult.ValidInstrumentCount + "\nCheck The DSS GUI...");
The output therefore displays 1 line for each invalid instrument, and the valid instrument count:
Important note: inactive and historical instruments might be rejected, like a matured bond or an instrument that changed name. This tutorial includes a matured bond (Cusip 438516AC0). You can decide if historical and inactive instruments are rejected by setting user preferences in the web GUI:
This API feedback data can be used to log errors when managing instrument lists using the REST API.
In the web GUI the instrument list contains only the validated instrument identifiers, those not found were automatically rejected:
The results also contain some high level information on the segmentation by asset class of the instrument identifiers. The following code displays this information:
Console.WriteLine("Asset classes count: " + appendResult.ValidationResult.StandardSegments.Count);
Console.WriteLine("Asset classes codes and descriptions: ");
for (int i = 0; i < appendResult.ValidationResult.StandardSegments.Count; i++)
{
Console.WriteLine("- Code: " + appendResult.ValidationResult.StandardSegments[i].Code +
" - Description: " + appendResult.ValidationResult.StandardSegments[i].Description);
}
The valid instruments in this tutorial are all equity stocks. The segmentation results will be:
This is exactly the same as what we did in the previous tutorial. In the next step we use the returned report template ID.
We create a daily recurring schedule. This is a variation of the second schedule we created in the preceding tutorial, the difference being in the definition of the trigger. The preceding one was set to run at 2 am. This one is also for a set time, but for testing purposes it is set to run 2 minutes in the future, that way we quickly get results every time we run the tutorial:
int waitMinutes = 2;
DateTimeOffset dateTimeNow = DateTimeOffset.Now;
DateTimeOffset dateTimeSchedule = dateTimeNow.AddMinutes(waitMinutes);
int scheduleHour = dateTimeSchedule.Hour;
int scheduleMinute = dateTimeSchedule.Minute;
Schedule recurringSchedule = new Schedule
{
Name = "myRecurringSchedule",
TimeZone = TimeZone.CurrentTimeZone.StandardName,
Recurrence = ScheduleRecurrence.CreateWeeklyRecurrence(
new[] { DayOfWeek.Monday, DayOfWeek.Tuesday, DayOfWeek.Wednesday,
DayOfWeek.Thursday, DayOfWeek.Friday }),
// This trigger is on a set time:
Trigger = ScheduleTrigger. CreateTimeTrigger(
false, //Limit to today's data: no
new[] { new HourMinute { Hour = scheduleHour, Minute = scheduleMinute } }),
ListId = instrumentList.ListId,
ReportTemplateId = reportTemplateId
};
Note: you can also specify a time zone by name:
TimeZone = "Eastern Standard Time",
As in the previous tutorial, we create the extraction schedule, using the extraction schedule definition, ad retrieve the returned schedule ID:
extractionsContext.ScheduleOperations.Create(recurringSchedule);
string recurringScheduleId = recurringSchedule.ScheduleId;
DebugPrintAndWaitForEnter("Returned recurring schedule ID: " +
recurringScheduleId + "\nCheck the DSS GUI...");
In the previous tutorial we defined the output file name for the schedule. As we did not do it here, a default name will be generated by the DSS server. This is not important, because we do not intend to retrieve the file manually, we want to get the data programmatically, via the API, as described next.
The extraction will be launched by the schedule.
The next step is to wait for the extraction to complete. This part of the code is inside a try / catch block, to trap exceptions.
We use WaitForNextExtraction, a blocking API call which only returns when the extraction is complete (or times out), so there is no need for code to check for extraction completion.
To achieve this, the API call automatically polls the server at regular intervals, to check if the extraction has completed. The poll period is configurable; it should not be too frequent so as not to unnecessarily burden the network and server. The timeout period is also configurable. In the tutorial code we poll once a minute, and set the timeout to 5 minutes:
TimeSpan timeOut = TimeSpan.FromMinutes(5);
int pollPeriod = 60;
ReportExtraction extraction =
extractionsContext.ScheduleOperations.WaitForNextExtraction(recurringSchedule, pollPeriod, timeOut);
This is one way to do it, but it is also possible to program this differently, using other API calls to check the status of an extraction by Id, or to retrieve completed extractions for a date range, etc. These calls are documented in the REST API Reference tree, and in the Scheduled Extractions examples in the C# example application.
Once the extraction has completed, we can display its status and detailed status:
Console.WriteLine("\nStatus: {0} - Detailed status: {1}",
extraction.Status, extraction.DetailedStatus);
Here is the result:
We can then retrieve the list of files generated by the extraction. Be aware that the toolkit only populates this list in the extractions context when you explicitly call LoadProperty(), so we must do that systematically before checking the file count value which tells us how many extraction files were generated:
extractionsContext.LoadProperty(extraction, "Files");
Console.WriteLine("File count: " + extraction.Files.Count + "\n");
Here is the result:
We see that 2 files were generated. As explained above in the section on Understanding extraction files, they are:
If we wanted to retrieve only the data file, and ignore the other file, we could search for the first file where the file type is “Full” (only the data file will be of this type):
ExtractedFile extractedDataFile =
extraction.Files.FirstOrDefault(f => f.FileType == ExtractedFileType.Full);
But in our sample code we decided to retrieve all files instead:
IEnumerable<ExtractedFile> extractedFiles = extraction.Files;
Then we treat each file, using a helper method:
int fileNumber = 0;
foreach (ExtractedFile extractedFile in extractedFiles)
{
fileNumber++;
DisplayFileDetailsAndContents(extractionsContext, extractedFile, fileNumber);
}
if (fileNumber == 0) { DebugPrintAndWaitForEnter("ERROR: extracted data file was null"); }
The helper method is defined further in the code, it displays the file name, type, receive time stamp (in UTC) and its size. Furthermore, if the file size is greater than zero, it reads the file contents, and then displays them. Finally, if it is a notes file it checks for successful processing.
:Note that we disable automatic decompression. This is because we want to download the compressed data, and, if required, decompress it using a dedicated package. We used SharpZipLib, other libraries to consider include zlib and DotNetLib. The result is more reliable and robust code that can handle large data sets.
static void DisplayFileDetailsAndContents(
ExtractionsContext extractionsContext, ExtractedFile file, int fileNumber)
{
//Do not automatically decompress data file content:
extractionsContext.Options.AutomaticDecompression = false;
Console.WriteLine("================================ FILE DETAILS =================================" +
"\nFile number: " + fileNumber +
"\nFile name: " + file.ExtractedFileName +
"\nFile type: " + file.FileType +
"\nReceived date utc:" + file.ReceivedDateUtc +
"\nFile size:" + file.Size);
if (file.FileType.ToString() == "Note")
{
Console.WriteLine("=================================== COMMENT ===================================");
Console.WriteLine("File type is \"Note\": it contains the extraction notes.");
}
if (file.FileType.ToString() == "Full")
{
Console.WriteLine("=================================== COMMENT ===================================");
Console.WriteLine("File type is \"Full\": it contains the data.");
}
DebugPrintAndWaitForEnter("===============================================================================");
//Don't fetch the content if the file is empty:
if (file.Size == 0) return;
Console.WriteLine("================================ FILE CONTENTS ================================");
//Direct download from AWS ?
if (awsDownload && file.FileType.ToString() == "Full")
{
extractionsContext.DefaultRequestHeaders.Add("x-direct-download", "true");
};
DssStreamResponse streamResponse =
extractionsContext.ExtractedFileOperations.GetReadStream(file);
if (file.FileType.ToString() == "Full")
//The data file is compressed. Here we decompress it in stream.
//For robust decompression we use SharpZipLib, a 3rd party package.
{
int count = 0;
string line = "";
using (GZipInputStream gzip = new GZipInputStream(streamResponse.Stream))
{
using (StreamReader reader = new StreamReader(gzip, Encoding.UTF8))
{
line = reader.ReadLine();
if (string.IsNullOrEmpty(line))
{
Console.WriteLine("WARNING: no data returned. Check your request dates.");
}
else
{
//The first line is the list of field names:
Console.WriteLine(line);
count++;
//The remaining lines are the data:
while ((line = reader.ReadLine()) != null)
{
Console.WriteLine(line);
count++;
}
}
}
}
Console.WriteLine("Line count: " + count);
}
if (awsDownload && file.FileType.ToString() == "Full")
{
extractionsContext.DefaultRequestHeaders.Remove("x-direct-download");
};
if (file.FileType.ToString() == "Note")
// The notes file is small and uncompressed.
{
string streamContents;
using (StreamReader reader = new StreamReader(streamResponse.Stream))
Console.WriteLine(streamContents = reader.ReadToEnd());
Console.WriteLine("=================================== COMMENT ===================================");
if (streamContents.IndexOf("Processing completed successfully") != -1)
{
Console.WriteLine("SUCCESS: Processing completed successfully.");
}
else
{
Console.WriteLine("ERROR: Processing did not complete successfully !");
}
}
DebugPrintAndWaitForEnter("===============================================================================");
}
Note that if we use AWS downloads, we set the additional request header for the data file only, because that is the only one that can be delivered by AWS (AWS will not deliver the Note file).
The order of the returned files might vary.
Here is the result for the note file:
================================ FILE DETAILS =================================
File number: 1
File name: 9008895.myRecurringSchedule.20210421.155700.2000000250529492.x03q13.csv.gz.notes.txt
File type: Note
Received date utc:4/21/2021 8:59:47 AM +00:00
File size:2977
=================================== COMMENT ===================================
File type is "Note": it contains the extraction notes.
===============================================================================
Press Enter to continue
================================ FILE CONTENTS ================================
Extraction Services Version 14.5.42294 (737b0965c07f), Built Apr 8 2021 13:43:46
User ID: 9008895
Extraction ID: 2000000250529492
Schedule: myRecurringSchedule (ID = 0x078676d176cc78ab)
Input List (5 items): myInstrumentListName (ID = 0x078677cfb49c78b1) Created: 04/21/2021 08:49:46 Last Modified: 04/21
/2021 08:49:49
Report Template (5 fields): myBarTemplateName (ID = 0x07867814be6c78b3) Created: 04/21/2021 08:54:46 Last Modified: 04
/21/2021 08:54:46
Schedule dispatched via message queue (0x078676d176ec78ab), Data source identifier (C794A3903B7045308775EB8F66E64B13)
Schedule Time: 04/21/2021 08:57:00
Processing started at 04/21/2021 08:57:02
Processing completed successfully at 04/21/2021 08:59:46
Extraction finished at 04/21/2021 08:59:46 UTC, with servers: tm01n01
Historical Instrument <CSP,438516AC0> expanded to 2 RIC instances: TESTFINN=TEST to US438516AC05=JPNY.
Instrument <RIC,IBM.N> expanded to 1 RIC: IBM.N.
Instrument <RIC,6502.T> expanded to 1 RIC: 6502.T.
Instrument <RIC,UBSG.S> expanded to 1 RIC: UBSG.S.
Instrument <RIC,ALVG.DE> expanded to 1 RIC: ALVG.DE.
Total instruments after instrument expansion = 6
Quota Message: INFO: Tick History Cash Quota Count Before Extraction: 3190; Instruments Approved for Extraction: 1; Ti
ck History Cash Quota Count After Extraction: 3190, 638% of Limit; Tick History Cash Quota Limit: 500
Manifest: #RIC,Domain,Start,End,Status,Count
Manifest: 6502.T,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: ALVG.DE,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: IBM.N,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: TESTFINN=TEST,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: UBSG.S,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: US438516AC05=JPNY,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
=================================== COMMENT ===================================
SUCCESS: Processing completed successfully.
===============================================================================
Here is the beginning of the result for the data file. Note that it is a compressed file, which was uncompressed on the fly by our code, to display on screen:
================================ FILE DETAILS =================================
File number: 2
File name: 9008895.myRecurringSchedule.20210421.155700.2000000250529492.x03q13.csv.gz
File type: Full
Received date utc:
File size:1265
=================================== COMMENT ===================================
File type is "Full": it contains the data.
===============================================================================
Press Enter to continue
================================ FILE CONTENTS ================================
#RIC,Alias Underlying RIC,Domain,Date-Time,GMT Offset,Type,Open,High,Low,Last,Volume
6502.T,,Market Price,2016-09-29T00:00:00.000000000Z,+9,Intraday 1Hour,350,353.5,337.1,340.5,28304000
6502.T,,Market Price,2016-09-29T01:00:00.000000000Z,+9,Intraday 1Hour,340.4,340.8,338,338.4,7124000
6502.T,,Market Price,2016-09-29T02:00:00.000000000Z,+9,Intraday 1Hour,338.4,338.7,336.3,337.6,3079000
6502.T,,Market Price,2016-09-29T03:00:00.000000000Z,+9,Intraday 1Hour,337.9,339.1,335.1,339.1,6742000
6502.T,,Market Price,2016-09-29T04:00:00.000000000Z,+9,Intraday 1Hour,339.1,344.8,338.5,344.1,10470000
6502.T,,Market Price,2016-09-29T05:00:00.000000000Z,+9,Intraday 1Hour,344.3,347.4,343.7,345.8,10454000
6502.T,,Market Price,2016-09-29T06:00:00.000000000Z,+9,Intraday 1Hour,345.5,345.5,345.5,345.5,3184000
6502.T,,Market Price,2016-09-29T07:00:00.000000000Z,+9,Intraday 1Hour,,,,,
The resulting data might not contain all instruments, even though they were not rejected when we added them to the instrument list. This simply depends on the availability of data for the requested period.
In our specific case here there is no returned data for the bond, because it matured on the 6th January 2016, and we requested data for the 29th September 2016.
Like in the previous tutorial, the instrument list, report template and schedule are deleted.
The full code can be displayed by opening the appropriate solution file in Microsoft Visual Studio.
List of the main steps in the code:
Don’t forget to reference the DSS SDK and the SharpZipLib, and to set your user account in Program.cs !
After running the program, and pressing the Enter key when prompted, the final result should look like this:
Returned session token: _6IKjf4JnQP0XJfLQFBdsxP_4eKjpxwa7vnJRM0VyFwNQjjSDBJ90t-QwL3T2MQATr3-sLa3mK7iPiZmlXl9jFhlHYKMsEuHw60EvxTrsxi
pO5yKMFJrlZHHLlzOWBarXExXe0PB3CexZOpGgI3ZwkiGuNuaDqWIWAoAMTH1YGRb1IlD8K1lC1PgsLRbDf_VtNQf7QI40v32z5p9jGkCz9tpKC5HFU36waN-LWQDvX5gPn
IiEQdAmZSuRl9RUVKnm-Eg5a3kDGx-D-PhCtJHQXy6IAc
Press Enter to continue
Returned instrument list ID: 0x078677cfb49c78b1
Check The DSS GUI...
Press Enter to continue
Attempted to add 7 instruments to the list.
Instrument identifiers validation results:
Validation messages:
- CSP, INVALID (not found)
- RIC, JUNK.JUNK (not found)
Valid instrument count: 5
Check The DSS GUI...
Press Enter to continue
Asset classes count: 1
Asset classes codes and descriptions:
- Code: E - Description: Equity
Press Enter to continue
Returned bar report template ID: 0x07867814be6c78b3
Check The DSS GUI...
Press Enter to continue
Returned recurring schedule ID: 0x078676d176cc78ab
Check the DSS GUI...
Press Enter to continue
3:55:25 PM We set a schedule for 3:57 PM ...
We now wait for the next extraction, please be patient ...
4:00:31 PM Extraction done.
Status: Completed - Detailed status: Done
File count: 2
================================ FILE DETAILS =================================
File number: 1
File name: 9008895.myRecurringSchedule.20210421.155700.2000000250529492.x03q13.csv.gz.notes.txt
File type: Note
Received date utc:4/21/2021 8:59:47 AM +00:00
File size:2977
=================================== COMMENT ===================================
File type is "Note": it contains the extraction notes.
===============================================================================
Press Enter to continue
================================ FILE CONTENTS ================================
Extraction Services Version 14.5.42294 (737b0965c07f), Built Apr 8 2021 13:43:46
User ID: 9008895
Extraction ID: 2000000250529492
Schedule: myRecurringSchedule (ID = 0x078676d176cc78ab)
Input List (5 items): myInstrumentListName (ID = 0x078677cfb49c78b1) Created: 04/21/2021 08:49:46 Last Modified: 04/21/2021 08:49:4
9
Report Template (5 fields): myBarTemplateName (ID = 0x07867814be6c78b3) Created: 04/21/2021 08:54:46 Last Modified: 04/21/2021 08:5
4:46
Schedule dispatched via message queue (0x078676d176ec78ab), Data source identifier (C794A3903B7045308775EB8F66E64B13)
Schedule Time: 04/21/2021 08:57:00
Processing started at 04/21/2021 08:57:02
Processing completed successfully at 04/21/2021 08:59:46
Extraction finished at 04/21/2021 08:59:46 UTC, with servers: tm01n01
Historical Instrument <CSP,438516AC0> expanded to 2 RIC instances: TESTFINN=TEST to US438516AC05=JPNY.
Instrument <RIC,IBM.N> expanded to 1 RIC: IBM.N.
Instrument <RIC,6502.T> expanded to 1 RIC: 6502.T.
Instrument <RIC,UBSG.S> expanded to 1 RIC: UBSG.S.
Instrument <RIC,ALVG.DE> expanded to 1 RIC: ALVG.DE.
Total instruments after instrument expansion = 6
Quota Message: INFO: Tick History Cash Quota Count Before Extraction: 3190; Instruments Approved for Extraction: 1; Tick History Ca
sh Quota Count After Extraction: 3190, 638% of Limit; Tick History Cash Quota Limit: 500
Manifest: #RIC,Domain,Start,End,Status,Count
Manifest: 6502.T,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: ALVG.DE,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: IBM.N,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: TESTFINN=TEST,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: UBSG.S,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
Manifest: US438516AC05=JPNY,Market Price,2016-09-29T00:00:00.000000000Z,2016-09-29T23:00:00.000000000Z,Active,24
=================================== COMMENT ===================================
SUCCESS: Processing completed successfully.
===============================================================================
Press Enter to continue
================================ FILE DETAILS =================================
File number: 2
File name: 9008895.myRecurringSchedule.20210421.155700.2000000250529492.x03q13.csv.gz
File type: Full
Received date utc:
File size:1265
=================================== COMMENT ===================================
File type is "Full": it contains the data.
===============================================================================
Press Enter to continue
================================ FILE CONTENTS ================================
#RIC,Alias Underlying RIC,Domain,Date-Time,GMT Offset,Type,Open,High,Low,Last,Volume
6502.T,,Market Price,2016-09-29T00:00:00.000000000Z,+9,Intraday 1Hour,350,353.5,337.1,340.5,28304000
6502.T,,Market Price,2016-09-29T01:00:00.000000000Z,+9,Intraday 1Hour,340.4,340.8,338,338.4,7124000
6502.T,,Market Price,2016-09-29T02:00:00.000000000Z,+9,Intraday 1Hour,338.4,338.7,336.3,337.6,3079000
6502.T,,Market Price,2016-09-29T03:00:00.000000000Z,+9,Intraday 1Hour,337.9,339.1,335.1,339.1,6742000
6502.T,,Market Price,2016-09-29T04:00:00.000000000Z,+9,Intraday 1Hour,339.1,344.8,338.5,344.1,10470000
6502.T,,Market Price,2016-09-29T05:00:00.000000000Z,+9,Intraday 1Hour,344.3,347.4,343.7,345.8,10454000
6502.T,,Market Price,2016-09-29T06:00:00.000000000Z,+9,Intraday 1Hour,345.5,345.5,345.5,345.5,3184000
6502.T,,Market Price,2016-09-29T07:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T08:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T09:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T10:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T11:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T12:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T13:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T14:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T15:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T16:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T17:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T18:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T19:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T20:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T21:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T22:00:00.000000000Z,+9,Intraday 1Hour,,,,,
6502.T,,Market Price,2016-09-29T23:00:00.000000000Z,+9,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T00:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T01:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T02:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T03:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T04:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T05:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T06:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T07:00:00.000000000Z,+2,Intraday 1Hour,132.9,133.5,132.8,133.25,186918
ALVG.DE,,Market Price,2016-09-29T08:00:00.000000000Z,+2,Intraday 1Hour,133.3,133.7,132.7,132.85,104396
ALVG.DE,,Market Price,2016-09-29T09:00:00.000000000Z,+2,Intraday 1Hour,132.85,133.15,132.7,132.85,123768
ALVG.DE,,Market Price,2016-09-29T10:00:00.000000000Z,+2,Intraday 1Hour,132.85,132.9,132.7,132.75,76545
ALVG.DE,,Market Price,2016-09-29T11:00:00.000000000Z,+2,Intraday 1Hour,132.8,133.3,132.75,133.15,49522
ALVG.DE,,Market Price,2016-09-29T12:00:00.000000000Z,+2,Intraday 1Hour,133.15,133.2,132.5,132.7,116571
ALVG.DE,,Market Price,2016-09-29T13:00:00.000000000Z,+2,Intraday 1Hour,132.75,132.9,132.45,132.55,75991
ALVG.DE,,Market Price,2016-09-29T14:00:00.000000000Z,+2,Intraday 1Hour,132.55,132.85,131.35,131.45,156691
ALVG.DE,,Market Price,2016-09-29T15:00:00.000000000Z,+2,Intraday 1Hour,131.4,131.45,130.6,131.3,421795
ALVG.DE,,Market Price,2016-09-29T16:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T17:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T18:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T19:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T20:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T21:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T22:00:00.000000000Z,+2,Intraday 1Hour,,,,,
ALVG.DE,,Market Price,2016-09-29T23:00:00.000000000Z,+2,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T00:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T01:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T02:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T03:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T04:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T05:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T06:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T07:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T08:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T09:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T10:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T11:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T12:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T13:00:00.000000000Z,-4,Intraday 1Hour,158.55,160.52,158.55,159.53,194717
IBM.N,,Market Price,2016-09-29T14:00:00.000000000Z,-4,Intraday 1Hour,159.57,159.91,159.03,159.14,93953
IBM.N,,Market Price,2016-09-29T15:00:00.000000000Z,-4,Intraday 1Hour,159.14,159.8,158.6,159.28,107389
IBM.N,,Market Price,2016-09-29T16:00:00.000000000Z,-4,Intraday 1Hour,159.23,159.67,157.93,158.16,75752
IBM.N,,Market Price,2016-09-29T17:00:00.000000000Z,-4,Intraday 1Hour,158.24,158.37,157.49,157.68,70565
IBM.N,,Market Price,2016-09-29T18:00:00.000000000Z,-4,Intraday 1Hour,157.71,158.72,157.7,158.36,44732
IBM.N,,Market Price,2016-09-29T19:00:00.000000000Z,-4,Intraday 1Hour,158.33,158.61,158.04,158.14,114186
IBM.N,,Market Price,2016-09-29T20:00:00.000000000Z,-4,Intraday 1Hour,158.11,158.11,158.11,158.11,370188
IBM.N,,Market Price,2016-09-29T21:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T22:00:00.000000000Z,-4,Intraday 1Hour,,,,,
IBM.N,,Market Price,2016-09-29T23:00:00.000000000Z,-4,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T00:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T01:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T02:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T03:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T04:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T05:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T06:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T07:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T08:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T09:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T10:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T11:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T12:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T13:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T14:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T15:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T16:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T17:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T18:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T19:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T20:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T21:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T22:00:00.000000000Z,+0,Intraday 1Hour,,,,,
TESTFINN=TEST,,Market Price,2016-09-29T23:00:00.000000000Z,+0,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T00:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T01:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T02:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T03:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T04:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T05:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T06:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T07:00:00.000000000Z,+2,Intraday 1Hour,13.19,13.23,13.07,13.13,2115970
UBSG.S,,Market Price,2016-09-29T08:00:00.000000000Z,+2,Intraday 1Hour,13.13,13.15,13.02,13.04,1178221
UBSG.S,,Market Price,2016-09-29T09:00:00.000000000Z,+2,Intraday 1Hour,13.05,13.06,13.01,13.02,765354
UBSG.S,,Market Price,2016-09-29T10:00:00.000000000Z,+2,Intraday 1Hour,13.02,13.02,12.96,12.99,707369
UBSG.S,,Market Price,2016-09-29T11:00:00.000000000Z,+2,Intraday 1Hour,12.99,13.07,12.98,13.03,810917
UBSG.S,,Market Price,2016-09-29T12:00:00.000000000Z,+2,Intraday 1Hour,13.04,13.05,12.98,12.99,1022395
UBSG.S,,Market Price,2016-09-29T13:00:00.000000000Z,+2,Intraday 1Hour,13,13.03,12.95,13.03,1648776
UBSG.S,,Market Price,2016-09-29T14:00:00.000000000Z,+2,Intraday 1Hour,13.04,13.08,12.97,13.06,1348367
UBSG.S,,Market Price,2016-09-29T15:00:00.000000000Z,+2,Intraday 1Hour,13.05,13.05,13,13,1560142
UBSG.S,,Market Price,2016-09-29T16:00:00.000000000Z,+2,Intraday 1Hour,13.01,13.03,13.01,13.03,261378
UBSG.S,,Market Price,2016-09-29T17:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T18:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T19:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T20:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T21:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T22:00:00.000000000Z,+2,Intraday 1Hour,,,,,
UBSG.S,,Market Price,2016-09-29T23:00:00.000000000Z,+2,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T00:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T01:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T02:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T03:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T04:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T05:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T06:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T07:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T08:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T09:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T10:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T11:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T12:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T13:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T14:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T15:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T16:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T17:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T18:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T19:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T20:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T21:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T22:00:00.000000000Z,+0,Intraday 1Hour,,,,,
US438516AC05=JPNY,,Market Price,2016-09-29T23:00:00.000000000Z,+0,Intraday 1Hour,,,,,
Line count: 145
===============================================================================
Press Enter to continue
Clean up is done, will now exit ...
Press Enter to continue
Intermediary results are discussed at length in the code explanations in the previous section of this tutorial.
Press Enter one final time to close the pop-up and end the program.
If the user name and password were not set properly, an error will be returned. See Tutorial 1 for details.
If the program attempts to create an instrument list that has the same name as that of an instrument list that is already stored on the DSS server, an error will be generated. Similar errors will arise for report template or extraction schedule that has the same name as an existing one. See Tutorial 3 for details.
Obvious comments were removed. We modified the method that appends identifiers, to return validation results:
public InstrumentsAppendIdentifiersResult AppendIdentifiersToInstrumentList(
IEnumerable<InstrumentIdentifier> instrumentIdentifiers, InstrumentList instrumentList)
{
InstrumentsAppendIdentifiersResult appendResult =
extractionsContext.InstrumentListOperations.AppendIdentifiers(
instrumentList, instrumentIdentifiers, false);
return appendResult;
}
The method to create a recurring schedule was modified to take the schedule hour and minute as parameters:
public string CreateNextBusinessDayRecurringSchedule(
string scheduleName, string instrumentListId, string reportTemplateId,
int scheduleHour, int scheduleMinute, string outputFileName)
{
Schedule schedule = new Schedule
{
Name = scheduleName,
TimeZone = TimeZone.CurrentTimeZone.StandardName,
Recurrence = ScheduleRecurrence.CreateWeeklyRecurrence(
new[] { DayOfWeek.Monday, DayOfWeek.Tuesday, DayOfWeek.Wednesday,
DayOfWeek.Thursday, DayOfWeek.Friday, DayOfWeek.Saturday }),
//This trigger is on a set time (regardless of data availability):
Trigger = ScheduleTrigger.CreateTimeTrigger(
false, //Limit to today's data: no
new[] { new HourMinute { Hour = scheduleHour, Minute = scheduleMinute } }),
ListId = instrumentListId,
ReportTemplateId = reportTemplateId,
OutputFileName = outputFileName
};
extractionsContext.ScheduleOperations.Create(schedule);
return schedule.ScheduleId;
}
A new cleanup method was added, to delete the instrument list, report template and schedule, with minimal error trapping:
public void Cleanup(
DssClient dssClient, string instrumentListId, string reportTemplateId, string scheduleId)
{
try { dssClient.DeleteInstrumentList(instrumentListId); } catch { };
try { dssClient.DeleteReportTemplate(reportTemplateId); } catch { };
try { dssClient.DeleteSchedule(scheduleId); } catch { };
}
We removed the on screen comments telling us to check the DSS GUI.
To populate the instrument list and retrieve validation results, we call our new DSS client helper method:
InstrumentsAppendIdentifiersResult appendResult =
dssClient.AppendIdentifiersToInstrumentList(
instrumentIdentifiers, instrumentList);
To display the validation results, we call a helper method:
DisplayAppendResults(appendResult);
As this helper method is specific to our workflow, we declare it in Program.cs instead of DssClient.cs, after the main code:
static void DisplayAppendResults(InstrumentsAppendIdentifiersResult appendResult)
{
Console.WriteLine("Instrument identifiers validation results:");
Console.WriteLine("Validation messages:");
foreach (InstrumentValidationMessage message in appendResult.ValidationResult.Messages)
Console.WriteLine("- " + message.Message, message.Severity);
DebugPrintAndWaitForEnter("Valid instrument count: " +
appendResult.ValidationResult.ValidInstrumentCount);
//The standard segments are separations of data by asset class.
//Display them:
Console.WriteLine("Asset classes count: " + appendResult.ValidationResult.StandardSegments.Count);
Console.WriteLine("Asset classes codes and descriptions: ");
for (int i = 0; i < appendResult.ValidationResult.StandardSegments.Count; i++)
{
Console.WriteLine("- Code: " + appendResult.ValidationResult.StandardSegments[i].Code +
" - Description: " + appendResult.ValidationResult.StandardSegments[i].Description);
}
DebugPrintAndWaitForEnter("");
}
To create the schedule, we call our new DSS client helper method:
string recurringScheduleId = dssClient.CreateNextBusinessDayRecurringSchedule(
"myRecurringSchedule", instrumentListId, reportTemplateId,
scheduleHour, scheduleMinute, "myTimedRecurringExtractionOutput.csv");
As we need the schedule to launch the extraction, we retrieve it by name:
Schedule recurringSchedule =
extractionsContext.ScheduleOperations.GetByName("myRecurringSchedule");
To clean up, we call our new DSS client helper method:
dssClient.Cleanup(dssClient, instrumentListId, reportTemplateId, recurringScheduleId);
The full code can be displayed by opening the appropriate solution file in Microsoft Visual Studio.
Don’t forget to reference the DSS SDK, and to set your user account in Program.cs !
This tutorial shows how to do a scheduled data extraction, like we could do in the web GUI.
Now move on to the next tutorial, which shows how to do an On Demand data extraction.