LSEG Data Library for Python

Please click on the following link to access the document: LSEG Data Library for Python - Reference Guide

LSEG's Data Library for Python can access LSEG databases is two diferent ways, depending on how one decides to authenticate oneself to the LSEG Services, either using the 'Desktop Session' or the 'Platform Session'.  To use the 'Desktop Session', one will need the Eikon Desktop Application or the Workspace Desktop Application running on the same machine where one runs his/her code. No such Application needs to run when using the 'Platform Session', however a License allowing for 'Platform Session' use will be necessary. Note: If you wish to resell or redistribute any applications which you develop using the LSEG Data Platform, or the LSEG Data Library for Python to third parties, you must enter into a  developer's license agreement with LSEG.

Limits at a Glance

What are the detailed limits?

Call-based Limits

• Requests per second - The number of requests sent to LSEG's Data Platform via the LSEG Data Library (LDL) cannot exceed this limit. This number of requests is counted across all client applications connected to the same Workspace instance. For example if 4 applications connected to the same Workspace send 1 request per second, the throttling mechanism will consider an aggregated rate of 4 requests per seconds.The current limit value (10-Oct-2019) is 5 requests per second.
• Response volume per minute - The volume (size) of data retrieved from the platform via the LDL cannot exceed this limit. This volume of data is counted across all client applications connected to the same Workspace instance. The current limit value (10-Oct-2019) is 50 MB per minute.
• Datapoints returned per request - A datapoint is a 'cell', or a unique field value for a unique instrument on a unique time stamp. Datapoint limits vary by the content set being retrieved (for example, timeseries limits are different from news headline limits), but all are throttled on a per request basis and are not aggregated across all applications. Here are Datapoint limit examples per LDL function type 
  • get_data: The limit value is around 10,000 data points for version 1.0.2 and below. No enforced limit for version 1.1.0 and above. However, it still has a server timeout around 300 seconds.
  • get_history: The current limit value (10-Oct-2019) is 3,000 data points (rows) for interday intervals and 50,000 data points for intraday intervals. This limit applies to the whole request, whatever the number of requested instrument.
  • news.get_headlines: There is no current limit value other than the aggregate data limit (see the daily limits detailed below) (14-04-2024). The depth of history is 15 month.
  • news.get_story: The current limit value (10-Oct-2019) is 1 story per request.
  • content.symbol_conversion: No enforced limit (10-Oct-2019).

Daily Limits

• Requests per day - The number of requests sent to the platform per day via the Workspace cannot exceed this limit. This number of requests is counted across all client applications connected to the same Workspace instance.The current limit values (10-Oct-2019) are 10,000 requests per day and 5 requests per second.
• Response volume per day - The volume (size) of data retrieved from the platform per day cannot exceed this limit. This volume of data is counted across all client applications connected to the same Workspace instance.The current limit values (10-Oct-2019) are 5 GB per day and 50 MB per minute.

Content/Entitlement Limits

• Workspace Variant - LDL (LSEG's Data Library) content coverage follows the Workspace Variant model and any add-ons that have been added to that Variant. For example, access to specific content sets available in the Workspace Premium Variant may not be available in Workspace for Research and Advisory, and therefore the same limitations would be enforced on the LDL.
• Depth and Breadth of Content - Any content depth (like years of historical data) and content breadth (like the number of news sources or exchanges) available via Workspace and through Workspace apps like Workspace Excel, is for the most part also available in the LDL. There may be additional LDL limitations imposed by the providers of the underlying content.

Platform/Infrastructure Limits

• Overall concurrent request load - When the platform is overwhelmed by the number of requests it receives, it may take more time than usual to respond. When that happens, a timeout expires and the platform responds to the request with an error. It is important to note that the overloading may be due to requests sent by users other than the one who receives the error.

What happens when a limit is reached?

For Call-based and Daily limits, when the limit is reached, the LSEG platform returns an HTTP response with status code 429 and the message "Too many requests, please try again later.". Then, the Python library raises an Error exception with the following message:

When the limits for datapoints per request are reached, responses are simply truncated and only the available cells/headlines/results are returned - even though your request would have yielded a bigger response. For example, a news headline query may have yielded more headlines than those returned on the response, or a timeseries call could return a fewer number of years than what was requested.

If your Workspace Variant or additional entitlements do not include access to the LSEG Data API, your applicaiton may receive an HTTP response with status code 403 and the message “Your Workspace account is not allowed to run Workspace API Proxy.”. Then, the Python library raises an Error exception with the following message. It is recommended to contact your account manager if you get this error:

When a request fails because the platform is overwhelmed, an HTTP response with status code 400 and the message "Backend error. 400 Bad Request" is returned. Then the Python library raises an Error exception with the following message:.

What can be done when a limit is reached?

There is not a common behavior that every application should necessarily implement to deal with limits. What an application must do when a limit is hit is dependent on the goal of the application. Each application is unique in this domain. However, the following guidelines may help developers to define the right behavior for their applications.

Do not silently discard Exceptions

Each time a limit is reached, the function call that hits the limit (get_data, get_history…) throws an exception that conveys important additional information to your application. If you just discard the exception the user of your application will not know that the request failed and that the data may be incomplete. The exception and the information it conveys helps your application to inform the user and to behave appropriately, for example by re-requesting the data.

Tell your user a limit has been reached

It is important to give your users an appropriate and accurate feedback about what happens in your application. When it comes to throttling limits, the feedback your application should give really depends on the limit the application hit. For example, if your application reached a Daily Limit, you could display a message in a dialog box that explains that a daily limit has been reached and that the application will stop receiving data until tomorrow. On the other hand, if an instant Call-based limit is reached, you may not want to display a message box each time it happens. Sending feedback via a transient and non-blocking visual way is probably more appropriate. For example, an instant limit reached could be represented by a different color that fades away progressively or a progress bar that indicates the time to wait. In the case of instant throttling limits, doing nothing could even be an option if your application is able to resend throttled requests automatically.

Consider leveraging the information sent along with the exception

The information that comes with the exception provides useful indications that help your application to behave appropriately. This information can be used by your application to decide on the most appropriate way to send feedback to the user (Dialog box, transient message, color or progress bar), but it can also be very useful to improve the user experience. For example, thanks to the “time to wait” information, the application can automatically resend requests that fail at the appropriate time.

Try to detect and address datapoint limits

It is not easy to detect when a datapoint limit is reached because in such situations Workspace does not send errors but just truncates the data it returns. This can be spotted visually by the user but also programmatically. For example, your code should check that the returned response covers the time span or any other dimension specified in the request (for example, if the request asked for 5000 instruments, but only 3000 were returned). When a datapoint limit is reached, the application can try splitting the initial request into smaller requests and resending them. Smaller requests can be built by reducing the number of instruments or by requesting shorter periods of time for time series. If your application is not able to resend smaller requests it should tell the user the data is not complete either by displaying a message, showing a visual indication, or by any other mean that is appropriate for your application.