Last update | Dec 2023 |
Environment | Windows |
Language | C# |
Compilers | Microsoft Visual Studio |
Prerequisites | DSS login, internet access, having done the previous tutorial |
Source code | Download .Net SDK Tutorials Code |
This is the third tutorial in a series of .Net SDK tutorials. It is assumed that the reader has acquired the knowledge delivered in the previous tutorials before following this one.
This tutorial is an introduction to the 3 basic core DSS operations. We shall take programmatic control of the DSS web GUI by creating:
After that, to cleanup, we delete these 3 items.
An installed copy of Microsoft Visual Studio is required to experiment with the code.
A DSS user account (user name and password) is required, because the API uses it to authenticate to the server. These username and password will be provided to LSEG Tick History customers and are valid in the API and the web GUI.
An internet connection is required to download the software, and run it, as it connects to the DSS server.
If you have not yet gone through Tutorial 1, which includes the installation of the SDK and the code of all tutorials, do it now before proceeding further.
The code installation was done in Tutorial 1.
Opening the solution is similar to what was done in Tutorial 1:
Important: this must be done for every single tutorial, for both the learning and refactored versions.
This was explained in the tutorial 2; please refer to it for instructions.
In Microsoft Visual Studio, in the Solution Explorer, double click on Program.cs and on DssClient.cs to display both file contents. Each file will be displayed in a separate tab.
Before running the code, you must replace YourUserId with your DSS user name, and YourPassword with your DSS password, in these 2 lines of Program.cs::
private static string dssUserName = "YourUserId";
private static string dssUserPassword = "YourPassword";
Important reminder: this will have to be done again for every single tutorial, for both the learning and refactored versions.
Failure to do so will result in an error at run time (see Tutorial 1 for more details).
Before explaining the code, let us briefly explain what we are trying to achieve.
To retrieve LSEG Tick History data from DSS, 3 basic elements are required:
This is the list of financial instrument identifiers for which data will be requested.
Using the API, an instrument list can be created, and instruments added to it.
It defines the format of the requested data, and the list of fields.
Depending on the requested data (pricing data, reference data, news, estimates, corporate actions, etc.), different report options and data fields are available.
Using the API, a report template can be created, and its data fields selected.
It defines when the data will be retrieved, and triggers the actual data extraction.
Its frequency could be once-off or recurring.
Its timing could be immediate, at a fixed time, or triggered when data is available.
They are all stored on the DSS server, and you can have several of each.
They can be created, displayed, modified and deleted manually, using the web GUI, and that might be the best solution if they are invariable in time and only need to be created once. But regular changes to some or all of these 3 elements might be required, and the purpose of the API is to give us the tools to automate that. The most frequent use case is the need to change the instrument list, which could be on a daily basis.
The purpose of this tutorial is to demonstrate how to do these things programmatically, using the API.
If your use case only requires regular changes to an instrument list, you could decide to manually create an empty instrument list, a report template and an extraction schedule, and then create a program that automatically changes the contents of the instrument list. You could also do it all programmatically, which is easier.
There are several ways you can proceed. In a nutshell, there are 2 approaches:
The choice is yours, and will depend on your use case.
We shall only describe what is new versus the previous tutorial.
This is nearly the same as the refactored version of Tutorials 1 or 2, except for the leading comment, and the fact that we have replaced the standard extractions context with an extractions context (which required adapting the using directives):
//extractionsContext temporarily set to public, to expose it to our learning program.
public ExtractionsContext extractionsContext;
private Uri dssUri = new Uri("https://selectapi.datascope.refinitiv.com/RestApi/v1/");
public void ConnectToServer(string dssUserName, string dssUserPassword)
{
extractionsContext = new ExtractionsContext(dssUri, dssUserName, dssUserPassword);
}
public string SessionToken
{
//The session token is only generated if the server connection is successful.
get { return extractionsContext.SessionToken; }
}
The reason for this is that VBD retrieval (the topic of Tutorial 2) requires a standard extractions context (VBD extractions are part of the standard solutions) whereas the calls we are covering now require an extractions context.
No additional explanations are required as the rest of the code was described in the previous tutorials.
At the top of the code we see the using directives. As we are using a whole set of API calls, several using directives are required to give us access to the types of the DSS API namespace, so that we do not have to qualify the full namespace each time we use them. As in Tutorial 1, these are followed by a using directive referring to the namespace of our code:
using System;
using System.Collections.Generic;
using DataScope.Select.Api.Extractions;
using DataScope.Select.Api.Extractions.SubjectLists;
using DataScope.Select.Api.Content;
using DataScope.Select.Api.Extractions.ReportTemplates;
using DataScope.Select.Api.Extractions.Schedules;
using DataScope.Select.Api;
using DssRestfulApiTutorials;
In the main code block, after connecting to the server, we expose the extractions context from DssClient (to avoid having to use the dssClient. prefix before each use of extractionsContext):
ExtractionsContext extractionsContext = dssClient.extractionsContext;
After those preliminary steps, we proceed to implement the three basic operations.
This is the first of our 3 basic operations. Programmatically, this is done in 3 steps:
Step 1: create an array of instrument identifiers, in memory:
IEnumerable<InstrumentIdentifier> instrumentIdentifiers = new[]
{
new InstrumentIdentifier
{
Identifier = "IBM.N",
IdentifierType = IdentifierType.Ric,
UserDefinedIdentifier = "EQUITYTEST"
},
new InstrumentIdentifier
{
Identifier = "438516AC0",
IdentifierType = IdentifierType.Cusip,
UserDefinedIdentifier = "BONDTEST"
}
};
A financial instrument identifier is created using at least 2 parameters (the third one here is optional):
Note:
Step 2: create an (empty) instrument list, on the DSS server; a list ID will be returned:
var instrumentList = new InstrumentList { Name = "myInstrumentListName" };
extractionsContext.InstrumentListOperations.Create(instrumentList);
DebugPrintAndWaitForEnter("Returned instrument list ID: " +
instrumentList.ListId + "\nCheck The DSS GUI...");
Here is the returned instrument list ID:
Let us display the result of this operation in the web GUI: click on DATASCOPE SELECT top left, then on Instrument Lists:
At this stage, the instrument list is created, but when we click on it we see it is empty:
If there are many and you can’t find it, sort the entries by clicking on the Creation Date column header.
Step 3: append the instrument identifiers to our instrument list, on the DSS server:
extractionsContext.InstrumentListOperations.AppendIdentifiers(
instrumentList, instrumentIdentifiers, false);
The third parameter defines if duplicate instrument identifiers should be kept or not.
The result of this operation can be seen in the web GUI. Now we find our instruments in the list:
Note: a validation process is automatically applied to all instruments, invalid ones are filtered out.
Important note: inactive and historical instruments might be rejected, like a matured bond or an instrument that changed name. This tutorial includes a matured bond (Cusip 438516AC0). You can decide if historical and inactive instruments are rejected by setting user preferences in the web GUI:
This is the second of our 3 basic operations. Programmatically, this is done in 3 steps:
Step 1: define a report template and set some of its properties, like its name and output format, in memory:
TickHistoryIntradaySummariesReportTemplate barReportTemplate =
new TickHistoryIntradaySummariesReportTemplate();
barReportTemplate.Name = "myBarTemplateName";
barReportTemplate.OutputFormat = ReportOutputFormat.CommaSeparatedValues;
Step 2: add the field list to the report template definition, in memory:
barReportTemplate.ContentFields.AddRange("Open", "High", "Low", "Last", "Volume");
Step 3: add the conditions:
barReportTemplate.Condition = new TickHistoryIntradaySummariesCondition
{
SummaryInterval = TickHistorySummaryInterval.OneHour,
DaysAgo = null,
MessageTimeStampIn = TickHistoryTimeOptions.GmtUtc,
QueryStartDate = new DateTimeOffset(2016, 09, 29, 0, 0, 0, TimeSpan.FromHours(0)),
QueryEndDate = new DateTimeOffset(2016, 09, 30, 0, 0, 0, TimeSpan.FromHours(0)),
ReportDateRangeType = ReportDateRangeType.Range,
ExtractBy = TickHistoryExtractByMode.Ric,
SortBy = TickHistorySort.SingleByRic,
Preview = PreviewMode.None,
DisplaySourceRIC = true,
TimebarPersistence = true
};
Step 4: create the report template, using the report template definition, on the DSS server; a report template ID will be returned:
extractionsContext.ReportTemplateOperations.Create(barReportTemplate);
DebugPrintAndWaitForEnter("Returned bar report template ID: " +
barReportTemplate.ReportTemplateId + "\nCheck The DSS GUI...");
Here is the returned report template ID:
To display the results in the web GUI, click on DATASCOPE SELECT top left, then on Report Templates:
We find our report template with 5 fields:
If there are many and you can’t find it, sort the entries by clicking on the Creation Date column header.
There are several report template formats, which depend on the data we want to retrieve, with different associated field lists. The easiest way to get familiar with these, and find the field names, is to look inside the web GUI and manually start to create a report template for the desired data, by clicking on DATASCOPE SELECT top left, then on Create just beneath Report Templates:
Even though we are starting to manually create a report template, we will not finish this creation. The idea here is not to do the work manually instead of with the API, but just to get familiar with what report template formats are available, and determine what data field names they contain, so that we can use these in our program.
Here we select a Tick History Intraday Summaries report:
After clicking on the Create button we see the list of available fields for that type of report. Depending on the type of report, many can be available ! Our sample will use, among others, the Last field:
Scroll through the list to see what is available, and decide what you need. Once you have your list of fields, you can quit this creation process and get back to programming.
This is the third of our 3 basic operations.
An extraction schedule defines when the data will be retrieved, and triggers the actual extraction. It must refer to an instrument list (for which the extraction will be done), and a report template (defining what data needs to be extracted).
An extraction schedule can run once or recurring. Its timing can be immediate, at a fixed time, or triggered when data is available.
It is possible to define several schedules.
In this tutorial we define two extraction schedules for our previously created instrument list and report template:
Programmatically, this is done in 2 steps:
As a first example, let us first define a schedule for an immediate extraction. Recurrence is set to run only once and now, the trigger is immediate, we refer to our previously defined instrument list ID and report template ID, and we define the output file name. This will be created on the DSS server; we will look at it further in this tutorial, in the section called Extracted data.
Step 1: define an extraction schedule and set some of its properties, in memory:
Schedule immediateSchedule = new Schedule
{
Name = "myImmediateSchedule",
TimeZone = TimeZone.CurrentTimeZone.StandardName,
Recurrence = ScheduleRecurrence.CreateSingleRecurrence(DateTimeOffset.UtcNow, true),
Trigger = ScheduleTrigger.CreateImmediateTrigger(true),
ListId = instrumentList.ListId,
ReportTemplateId = eodReportTemplate.ReportTemplateId,
OutputFileName = "myImmediateExtractionOutput.out",
};
Step 2: create the extraction schedule, using the extraction schedule definition, on the DSS server; a schedule ID will be returned:
extractionsContext.ScheduleOperations.Create(immediateSchedule);
DebugPrintAndWaitForEnter("Returned immediate extraction schedule ID: " +
immediateSchedule.ScheduleId + "\nCheck the DSS GUI...");
Here is the returned schedule ID:
Let us display the result of this operation in the web GUI: click on DATASCOPE SELECT top left, then on Schedules:
We find our schedule, as defined:
If there are many and you can’t find it, sort the entries by clicking on the Creation Date column header.
As a second example, let us define a recurring schedule for a triggered extraction. Recurrence is set to weekly, Tuesday to Saturday.
The extraction is triggered at a specific time.
Again, we refer to our instrument list and report IDs, and we define the output file name. A schedule ID is returned:
Step 1: define an extraction schedule and set some of its properties, in memory:
Schedule recurringSchedule = new Schedule
{
Name = "myRecurringSchedule",
TimeZone = TimeZone.CurrentTimeZone.StandardName,
Recurrence = ScheduleRecurrence.CreateWeeklyRecurrence(
new[] { DayOfWeek.Tuesday, DayOfWeek.Wednesday, DayOfWeek.Thursday,
DayOfWeek.Friday, DayOfWeek.Saturday }),
//Run the extraction at 2 am:
Trigger = ScheduleTrigger. CreateTimeTrigger(
false, //Limit to today's data: no
new[] { new HourMinute { Hour = 2, Minute = 0 } }),
ListId = instrumentList.ListId,
ReportTemplateId = barReportTemplate.ReportTemplateId,
OutputFileName = "myBarDataRecurringExtractionOutput.out"
};
Note: you can also specify a time zone by name:
TimeZone = "Eastern Standard Time",
Step 2: create the extraction schedule, using the extraction schedule definition, on the DSS server; a schedule ID will be returned:
extractionsContext.ScheduleOperations.Create(recurringSchedule);
DebugPrintAndWaitForEnter("Returned recurring extraction schedule ID: " +
recurringSchedule.ScheduleId + "\nCheck the DSS GUI...");
The result of this operation can again be seen in the web GUI, where we find our second schedule, as defined. We see the template, list, recurrence and trigger:
The extracted data is stored in files, on the DSS server. The web GUI allows us to manually display, preview and download these files: click on DATASCOPE SELECT top left, then on Extracted Files:
Here we find the resulting files, generated by the scheduled extractions. They might not all be here, one must wait for the schedules to run:
If there are many and you can’t find it, sort the entries by clicking on the Creation Date column header.
Using the action buttons on the right, the data contents can be previewed or downloaded. There is also a notes file that contains some information on the extraction.
The data can also be retrieved via the API. We do not do that in this tutorial, but we will in the next ones:
Both approaches are valid, and delivered as examples of what can be done.
Our last operation in this tutorial will be to cleanup.
The purpose of this tutorial is to demonstrate manipulating DSS just like through the DSS GUI, so we will stop here and cleanup what we created. We therefore delete our instrument list, report template and schedules:
extractionsContext.InstrumentListOperations.Delete(instrumentList.ListId);
extractionsContext.ReportTemplateOperations.Delete(barReportTemplate.ReportTemplateId);
extractionsContext.ScheduleOperations.Delete(immediateSchedule.ScheduleId);
extractionsContext.ScheduleOperations.Delete(recurringSchedule.ScheduleId);
In the context of these tutorials deleting these items is important, because in this example code their names are hard coded. Running the code a second time would fail if cleanup was not done, because one cannot create 2 items with the same name.
Also note that the second schedule never ran; it was just created to demonstrate different schedule types.
In real life you will probably want to re-use these items instead of deleting them. Deleting is easily done manually using the DSS web GUI, but if you wanted to do it programmatically at a later stage, you would have to store the required Ids.
As mentioned above in Basic operations: explanations - Important note on DSS programming approaches, it is also possible to use higher level API calls and dispense with creating instrument list, report template and schedules stored on the DSS server.
The full code can be displayed by opening the appropriate solution file in Microsoft Visual Studio.
List of the main steps in the code:
This is the same as what was done in Tutorial 1.
Open the top menu BUILD, and click on Rebuild Solution. This will check your code for errors, and prepare it to run:
In the lower part of the screen you should get a success message:
If there are errors, they will be displayed and highlighted in the code. Errors must be corrected before the code can run.
Click on
to run the code.
After running the program, and pressing the Enter key when prompted, the final result should look like this:
Intermediary results are discussed at length in the code explanations in the previous section of this tutorial.
Press Enter one final time to close the pop-up and end the program.
If the user name and password were not set properly, an error will be returned. See Tutorial 1 for details.
If the program attempts to create an instrument list that has the same name as that of an instrument list that is already stored on the DSS server, an error will be generated.
Scrolling down in the error popup displays the nature of the error:
Similar errors will arise for report template or extraction schedule that has the same name as an existing one.
Possible causes:
The most probable cause is that you ran the tutorial, and interrupted it before the end. In such a case the final cleanup is not done, the created instrument list, report template and schedules are not deleted. As stated before in the code explanations, deleting these items is important, because in this example code their names are hard coded, so running the code a second time will fail if cleanup was not done, as one cannot create 2 items with the same name.
It could also be possible that, by pure chance, you have an instrument list, report template and / or schedule name that has exactly the same name as one of those in this tutorial.
To solve this:
If the tutorial run was interrupted before the end:
If you have an instrument list, report template and / or schedule name that has exactly the same name as one of those in this tutorial:
The refactored version is located in directory \API TH REST\Tutorial 3\Refactored
To view the entire code, navigate to that folder and double click on restful_api_core_gui_basics.sln to open the solution in Microsoft Visual Studio.
We enhanced the DSS client helper class, with several methods, all related to the DSS API. Several of the tutorials use these, and you could also re-use them in your own projects.
At the top of the code we find the required using directives for the DSS REST API:
using DataScope.Select.Api.Extractions;
using DataScope.Select.Api.Extractions.SubjectLists;
using DataScope.Select.Api.Content;
using DataScope.Select.Api.Extractions.ReportTemplates;
using DataScope.Select.Api.Extractions.Schedules;
using DataScope.Select.Api;
The extraction context is set to private, as the main program does not access it:
private ExtractionsContext extractionsContext;
The first new method creates a new (empty) instrument list, taking as input parameter the instrument list name, and returning the instrument list:
public InstrumentList CreateInstrumentList(string instrumentListName)
{
InstrumentList instrumentList = new InstrumentList { Name = instrumentListName };
extractionsContext.InstrumentListOperations.Create(instrumentList);
return instrumentList;
}
The second one populates an instrument list by appending an array of instrument identifiers, taking as input parameters the array of instrument identifiers and the instrument list name. It does not return anything. This method can be called several times if required, to populate an instrument list in several increments. Duplicate instruments are discarded:
public void AppendIdentifiersToInstrumentList(
IEnumerable<InstrumentIdentifier> instrumentIdentifiers, InstrumentList instrumentList)
{
extractionsContext.InstrumentListOperations.AppendIdentifiers(
instrumentList, instrumentIdentifiers, false);
}
The third method combines the two preceding ones:
public string CreateAndPopulateInstrumentList(
string instrumentListName, IEnumerable<InstrumentIdentifier> instrumentIdentifiers)
{
InstrumentList instrumentList = CreateInstrumentList(instrumentListName);
AppendIdentifiersToInstrumentList(instrumentIdentifiers, instrumentList);
return instrumentList.ListId;
}
Note that instead of the instrument list, it returns the instrument list ID. If required you could retrieve the instrument list from its ID using the following API call:
instrumentList = extractionsContext.InstrumentListOperations.Get(instrumentListId);
The fourth method creates a report template for intraday summaries (bar) data, taking as input parameters the report template name, the output format, an array of field names and the conditions. It returns the report template ID:
public string CreateBarReportTemplate(
string reportTemplateName, ReportOutputFormat reportOutputFormat,
string[] contentFieldNames, TickHistoryIntradaySummariesCondition Condition)
{
TickHistoryIntradaySummariesReportTemplate reportTemplate =
new TickHistoryIntradaySummariesReportTemplate
{
Name = reportTemplateName,
OutputFormat = reportOutputFormat
};
reportTemplate.ContentFields.AddRange(contentFieldNames);
reportTemplate.Condition = Condition;
extractionsContext.ReportTemplateOperations.Create(reportTemplate);
return reportTemplate.ReportTemplateId;
}
The fifth method creates a schedule for an immediate extraction, taking as input parameters the schedule name, the instrument list ID, the report template ID and the output file name. It returns the schedule ID:
public string CreateImmediateSchedule(
string scheduleName, string instrumentListId, string reportTemplateId, string outputFileName)
{
Schedule schedule = new Schedule
{
Name = scheduleName,
TimeZone = TimeZone.CurrentTimeZone.StandardName,
Recurrence = ScheduleRecurrence.CreateSingleRecurrence(DateTimeOffset.UtcNow, true),
Trigger = ScheduleTrigger.CreateImmediateTrigger(true),
ListId = instrumentListId,
ReportTemplateId = reportTemplateId,
OutputFileName = outputFileName
};
extractionsContext.ScheduleOperations.Create(schedule);
return schedule.ScheduleId;
}
The sixth method also creates a schedule, but this one is a recurring schedule, which executes after every business day at 2 am. The inputs and output are the same as for the previous schedule creation method:
public string CreateNextBusinessDayRecurringSchedule (
string scheduleName, string instrumentListId, string reportTemplateId, string outputFileName)
{
Schedule schedule = new Schedule
{
Name = scheduleName,
TimeZone = TimeZone.CurrentTimeZone.StandardName,
Recurrence = ScheduleRecurrence.CreateWeeklyRecurrence(
new[] { DayOfWeek.Tuesday, DayOfWeek.Wednesday, DayOfWeek.Thursday,
DayOfWeek.Friday , DayOfWeek.Saturday }),
//Run the extraction at 2 am:
Trigger = ScheduleTrigger.CreateTimeTrigger(
false, //Limit to today's data: no
new[] { new HourMinute { Hour = 2, Minute = 0 } }),
ListId = instrumentListId,
ReportTemplateId = reportTemplateId,
OutputFileName = outputFileName
};
extractionsContext.ScheduleOperations.Create(schedule);
return schedule.ScheduleId;
}
The last three methods delete an instrument list, report template or schedule, taking their ID as input. They do not return anything:
public void DeleteInstrumentList(string instrumentListId)
{
extractionsContext.InstrumentListOperations.Delete(instrumentListId);
}
public void DeleteReportTemplate(string reportTemplateId)
{
extractionsContext.ReportTemplateOperations.Delete(reportTemplateId);
}
public void DeleteSchedule(string immediateScheduleId)
{
extractionsContext.ScheduleOperations.Delete(immediateScheduleId);
}
At the top of the code we list of using directives has been reduced, because we use less API calls:
using DataScope.Select.Api.Content;
using DataScope.Select.Api.Extractions.ReportTemplates;
The extraction context is not exposed, as the main code does not access it.
To create the array of instrument identifiers, we call a helper method:
IEnumerable<InstrumentIdentifier> instrumentIdentifiers = CreateInstrumentIdentifiers();
This helper method is not specific to DSS, so we declare it in Program.cs, after the main code:
static IEnumerable<InstrumentIdentifier> CreateInstrumentIdentifiers()
{
IEnumerable<InstrumentIdentifier> instrumentIdentifiers = new[]
{
new InstrumentIdentifier
{
Identifier = "IBM.N",
IdentifierType = IdentifierType.Ric,
UserDefinedIdentifier = "EQUITYTEST"
},
new InstrumentIdentifier
{
Identifier = "438516AC0",
IdentifierType = IdentifierType.Cusip,
UserDefinedIdentifier = "BONDTEST"
}
};
return instrumentIdentifiers;
}
To create and populate the instrument list, we call our new DSS client helper method:
string instrumentListId = dssClient.CreateAndPopulateInstrumentList(
"myInstrumentListName",
instrumentIdentifiers);
To create the array of field names, we call a helper method:
string[] requestedFieldNames = CreateRequestedFieldNames();
Again, as this helper method is not specific to DSS, we declare it in Program.cs, after the main code:
static string[] CreateRequestedFieldNames()
{
string[]requestedFieldNames = { "Open", "High", "Low", "Last", "Volume" };
return requestedFieldNames;
}
To create the two schedules, we call our two new DSS client helper methods:
string immediateScheduleId = dssClient.CreateImmediateSchedule(
"myImmediateSchedule",
instrumentListId,
reportTemplateId,
"myImmediateExtractionOutput.out");
string recurringScheduleId = dssClient.CreateNextBusinessDayRecurringSchedule(
" myRecurringSchedule",
instrumentListId,
reportTemplateId,
"myBarDataRecurringExtractionOutput.out");
To clean up, we call our new DSS client helper methods:
dssClient.DeleteInstrumentList(instrumentListId);
dssClient.DeleteReportTemplate(reportTemplateId);
dssClient.DeleteSchedule(immediateScheduleId);
dssClient.DeleteSchedule(recurringScheduleId);
The full code can be displayed by opening the appropriate solution file in Microsoft Visual Studio.
The refactored version is located in directory \API TH REST\Tutorial 3\Refactored
The actions described above for the learning version are applied similarly to the refactored version. Proceed to:
We have seen that using the API, we can create an instrument list, a report template, and schedules, like we could do in the web GUI. In other words, the API allows us to do programmatically what can be done manually in the web GUI.
We have also seen that the web GUI is very useful to help us check our code.
Now move on to the next tutorial, which: