Article

How to Retrieve Early Partial Delivery of Embargoed Reports via DataScope Select Web GUI

Author:

Jirapongse Phuriphanvichai
Developer Advocate Developer Advocate

LSEG DataScope Select is a hosted data extraction platform that provides access to global pricing, validated terms and conditions, historical data content, corporate actions, cross-reference data, and legal entity data. Supported instruments include equities, funds, derivatives, money/foreign exchange, warrants, fixed-income securities, commodities, and sell-side estimates, as well as user-defined instruments (Credit Default Swaps, Interest Rate Swaps, and Over-The-Counter Equity Options). Data is categorized into report templates, such as EOD Pricing, Intraday Pricing, Premium EOD Pricing, Terms and Conditions, and Composite report templates. 

This article focuses on the Early Partial Delivery of Embargoed Reports preference. You can enable this option to receive an early partial report with non-embargoed data before the complete report with embargoed intraday is available.

Embargoed Extractions

DataScope Select is governed by exchange rules that require delays in the release of Intraday Pricing and Premium End of Day Data Pricing extractions. Embargo delays apply only to users who don't have permission to receive real-time intraday data during the exchange’s operating hours. No embargo is applied if you have permission for real-time intraday data or if you request data outside of the exchange’s operating hours. Please contact your local account manager or sales specialist for information about real-time intraday permissions.

To retrieve embargo information for the instruments in your input list, you should run an Intraday Pricing or Premium EOD Pricing extraction using the following fields:

  • Current Embargo Delay: The remaining amount of time (in minutes) until an exchange embargo is lifted. 
  • Embargo Times: Operating hours of the exchange during which the full embargo will be applied, presented in the schedule-level preferred time zone.
  • Embargo Window: The time period during which you are not permitted to receive data from the exchange for the indicated instrument; Data requested during this time period will be delayed until the start of the next window.
  • Exchange Requiring Embargo: Exchange requiring the embargo delay for the indicated instrument.
  • Instrument Snap Time: The time at which the data was snapshot for the indicated instrument, presented in the time zone selected in your user preferences.
  • Last Update Time: The time at which the indicated instrument last received an update (tick) in DSS from the content feed for this extraction, presented in the time zone selected in your user preferences.
  • Maximum Embargo Delay: Maximum number of minutes required by the exchange that the instrument must be embargoed for, based on the specific users’ exchange permissions.
  • Real Time Permitted: Y/N flag indicating whether you have permission to retrieve real-time intraday data for the indicated instrument.

The extraction report looks like this:

RIC Current Embargo Delay Embargo Times Embargo Window Exchange Required Embargo Instrument Snap Time Last Update Time Maximum Embargo Delay Real Time Permitted
0001.HK 15 Monday-Friday 08:30:00-11:00:00 and Monday-Friday 12:00:00-15:00:00   HS1 (HKEX-HONG KONG STOCK EXCHANGE LEVEL 1), HKG (HKEX-HONG KONG STOCK EXCHANGE LEVEL 1 LEVEL 2), HSF (HKEX-HONG KONG STOCK EXCHANGE FULL TICK DATA) 23.06.2023 10:39:06 23.06.2023 10:38:53 15 N
IBM.N 0 Monday-Friday 20:30:00-03:02:00   NYS (NYSE LEVEL 1) 23.06.2023 10:39:06 23.06.2023 10:17:51 15 N
JPY= 0       23.06.2023 10:39:06 23.06.2023 10:39:06 0 Y
15793T 20 Monday-Friday 07:00:00-09:31:00 and Monday-Friday 10:30:00-13:05:00   TYO (TOKYO STOCK EXCHANGE), TYM (TOKYO SE FLEX FULL OPEN MKT MODEL) 23.06.2023 10:39:06 23.06.2023 10:39:06 20 N
PTT.BK 15 Monday-Friday 09:55:00-12:30:00 and Monday-Friday 14:25:00-16:40:00   SET (STOCK EXCHANGE OF THAILAND), ST2 (STOCK EXCHANGE OF THAILAND L1 L2) 23.06.2023 10:39:06 23.06.2023 10:39:03 15 N

You can also view embargo details for a selected instrument via the Embargo tab via the Details screen for equity instruments and futures & options. 

DataScope Select uses the last intraday update time plus the maximum embargo delay to calculate extraction delivery times.

Early Partial Delivery of Embargoed Reports

Typically, if users extract intraday data from both non-embargoed instruments and embargoed instruments, users will get a report after all instruments are extracted. For example, if an instrument list contains an embargoed instrument that has 20 minutes embargo delay, users will get a completed report twenty minutes later. 

This Early Partial Delivery of Embargoed Reports option lets users receive early, non-embargoed intraday data prior to receiving the complete extraction following all required exchange delays. When this option is selected, the following options are enabled:

  • Enable Intermediate Embargoed Reports

Select this option to receive intermediate embargoed extractions as soon as the data is available from the exchange. For extractions with multiple embargoes, a partial extraction will be delivered each time an embargo expires. In this instance, the complete file will be delivered repeatedly until all the embargoes have expired, each time adding the data for the instruments that have been released from the embargo.

When cleared, the non-embargoed extraction is delivered first, followed by the completed extraction with all embargoed data. No intermediate embargoed extractions will be delivered and all new extractions going forward will no longer include Xmins.

  • Enable Delta Capabilities for Embargoed Reports

Select this option to receive delta reports. If Enable Intermediate Embargoed Reports is also selected, only data that has changed since the previous intermediate embargoed report is output. If it is cleared, all embargoed data is output in a single report. 

Note that this report will be released once the exchange with the longest embargo has expired. Non-embargoed data released in the initial report will be excluded. Clear this preference to retrieve a cumulative list of data each time an embargo expires. 

You can enable these options by editing the User Preferences.

Click on the Edit Preferences button to edit the Preferences.

Select those options and then click on the Save button to save the settings.

Now, you have enabled the Early Partial Delivery of Embargoed Reports options with the Intermediate Embargoed Reports and Delta Capabilities for Embargoed Reports features. 

Embargoed Reports

After enabling these options, if you extract intraday embargoed data, you will get partial and intermediated embargoed reports according to the settings. For example, when performing an intraday pricing extraction that contains embargoed data with the above settings, you will get the first report file (<output file>.0min.csv)that contains non-embargoed data and the status of the extraction will be Embargoed

After that, you will receive immediate reports when each embargo expires. Finally, you will get all files and the status will be changed to Completed.

The notes contain the following information.

    	
            

Extraction Services Version 17.2.1.45837 (3cafbfe0269d), Built Jun  5 2023 17:25:14

User has overridden estimates broker entitlements.

Processing started at 23.06.2023 12:27:04.

User ID: 9008895

Extraction ID: 2000000573996616

Correlation ID: CiD/9007633/_vxmdA.0881a1dfa1d8d8ba/EQM/ED.0x088572dafb38e06d.0

Schedule: 23.06.2023 12.26.49 (ID = 0x088572dafb28e06d)

Input List (5 items): Embargo (ID = 0x08832f57fe98dc3f) Created: 16.06.2023 13:02:19 Last Modified: 22.06.2023 12:54:37

Report Template (8 fields): InstradayEmbargoPricesFields (ID = 0x088585afce78e0c0) Created: 23.06.2023 10:54:32 Last Modified: 23.06.2023 10:54:32

Schedule dispatched via message queue (0x088572dafb38e06d)

Schedule Time: 23.06.2023 12:27:01

Temporary Integration Test Checkpoint 22

Successful operation - data received from RDP

Real-time data was snapped at the following times:

   23.06.2023 12:27:01

   23.06.2023 12:27:04 for data scheduled to snap at 23.06.2023 12:27:01.

Processing completed successfully at 23.06.2023 12:27:04, taking 0.501 Secs.

Extraction finished at 23.06.2023 05:27:04 UTC, with servers: xc08bdm2Q18, QSDHA1 (0.0 secs), QSHC11 (0.1 secs)

Embargo delay of 15 minutes required by [ HS1 (HKEX-HONG KONG STOCK EXCHANGE LEVEL 1), HKG (HKEX-HONG KONG STOCK EXCHANGE LEVEL 1 LEVEL 2), HSF (HKEX-HONG KONG STOCK EXCHANGE FULL TICK DATA) ] for quotes from HKG

Embargo delay of 20 minutes required by [ TYO (TOKYO STOCK EXCHANGE), TYM (TOKYO SE FLEX FULL OPEN MKT MODEL) ] for quotes from TYO

Embargo delay of 15 minutes required by [ SET (STOCK EXCHANGE OF THAILAND), ST2 (STOCK EXCHANGE OF THAILAND L1 L2) ] for quotes from SET

The last report will be embargoed until 23.06.2023 12:47:04 (20 minutes) due to quote: RIC,1579.T,TYO - Last Update Time: 23.06.2023 12:27:04.

Usage Summary for User 9008895, Client 65507, Template Type Intraday Pricing

Base Usage

        Instrument                          Instrument                   Terms          Price

  Count Type                                Subtype                      Source         Source

------- ----------------------------------- ---------------------------- -------------- ----------------------------------------

      4 Equities                                                         N/A            N/A

      1 Money Market                                                     N/A            N/A

-------

      5 Total instruments charged.

      0 Instruments with no reported data.

=======

      5 Instruments in the input list.

No Evaluated Pricing Service complex usage to report -- 5 Instruments in the input list had no reported data.

The file testoutput.0min.csv will be available immediately.

The file testoutput.15min.csv will be embargoed until 23.06.2023 12:42:03.

The file testoutput.20min.csv will be embargoed until 23.06.2023 12:47:04.

Writing RIC maintenance report.

Outputs With Different Settings

This section shows the behaviors of the embargoed extractions when using the following scenarios.

1. Disable Early Partial Delivery of Embargoed Report

2. Enable Early Partial Delivery of Embargoed Reports

3. Enable Early Partial Delivery of Embargoed Reports with Enable Intermediate Embargoed Reports

4. Enable Early Partial Delivery of Embargoed Reports with Enable Delta Capabilities for Embargoed Reports

5. Enable Early Partial Delivery of Embargoed Reports with Enable Intermediate Embargoed Reports and Enable Delta Capabilities for Embargoed Reports

The instrument list contains the following instruments. 

The Intraday Price report contains the following fields. 

The current embargo delay values of those instruments are:

The behaviors and created files are shown in the following table. 

Scenarios Early Partial Delivery Intermediate Delta Capabilities 0 Minute 15 Minutes 20 Minutes Notes
1       NA NA Report.csv Notes
2 x     Report.partial.csv(*) NA Report.csv Notes
3 x x   Report.0min.csv Report.15min.csv Report.20min.csv Notes
4 x   x Report.partial.csv NA Report.csv Notes
5 x x x Report.0min.csv Report.15.min.csv Report.20min.csv Notes

The list of extracted files is:

In the first scenario (Disable Early Partial Delivery of Embargoed Report), the extraction report will be generated when the longest embargoed delay (20 minutes) expires. The extraction report will contain data for all requested instruments.

In the second scenario (Enable Early Partial Delivery of Embargoed Reports), the partial file that contains non-embargoed data will be available immediately. Embargoed data will be blank. The last extraction report that includes embargoed data will be created when the longest embargoed delay (20 minutes) expires. Therefore, the last extraction report will contain data for all requested instruments. (*)The partial file will be deleted after the last extraction report is created.

In the third scenario (Enable Early Partial Delivery of Embargoed Reports with Enable Intermediate Embargoed Reports), the partial file with the 0min suffix that contains non-embargoed data will be available immediately. Embargoed data will be blank. Then, when each embargoed delay expires, the new partial file with the xxmin suffix will be created and the embargoed data will be included in the new file. Finally, the last extraction report with the xxmin suffix will contain data for all requested instruments.

In the fourth scenario (Enable Early Partial Delivery of Embargoed Reports with Enable Delta Capabilities for Embargoed Reports), the partial file that contains non-embargoed data will be available immediately. The file only contains non-embargoed instruments. The last extraction report that contains embargoed data will be created when the longest embargoed delay (20 minutes) expires. The last extraction report will contain data for all embargoed instruments.

In the last scenario (Enable Early Partial Delivery of Embargoed Reports with Enable Intermediate Embargoed Reports and Enable Delta Capabilities for Embargoed Reports), the partial file with the 0min suffix that contains non-embargoed data will be available immediately. The file only contains non-embargoed instruments. Then, when each embargoed delay expires, the new partial file with the xxmin suffix will be created and the new file contain only the instruments that the embargoed delay has expired. Finally, the last extraction report with the xxmin suffix will contain data on the instruments that have the longest embargoed delay.

Summary

If users don’t have permission for real-time intraday data during the exchange’s operating hours, the Intraday Pricing and Premium End of Day Data Pricing extractions can be delayed according to the exchanges’ embargo rules. When the data is embargoed, the final extraction report will be impeded until all embargoed delays expire. DataScope Select provides the settings for users to receive an early partial report with non-embargoed data before the complete report with embargoed intraday is available.

The Early Partial Delivery of Embargoed Reports option lets users receive early, non-embargoed intraday data prior to receiving the complete extraction following all required exchange delays.

The Enable Intermediate Embargoed Reports option allows users to receive intermediate embargoed extractions as soon as the data is available from the exchange. For extractions with multiple embargoes, a partial extraction will be delivered each time an embargo expires. The extracted files will be suffixed with Xmins.

The Enable Delta Capabilities for Embargoed Reports option produces the report files that contain only data that has changed since the previous partial or intermediate report was created.

These settings can be set in the Preferences via the DataScope Select Web GUI.

References

1.     LSEG DataScope Select GUI User Guide (no date) LSEG Developers Portal. Available at: https://developers.lseg.com/en/api-catalog/datascope-select/datascope-select-rest-api/documentation#data-scope-select-gui-user-guide (Accessed: 03 July 2023).