Author:
RIC is a convention for naming financial instruments, such that data related to these instruments can be easily accessed from any LSEG databases. The composition of each code depends on the type of instrument. The code is structured with a number of elements, and their complete combination makes up a unique RIC for each instrument.
RICs are typically used in LSEG financial products. Especially, RICs are uses in real-time products (Real-Time Optimized, Real-Time Distribution System, and Real-Time SDKs) for item subscriptions. There are several tools that can help developers search for RICs.
- RIC Search: This is a web UI tool available on the LSEG Developer Community
- Data Platform Search API: This is a REST API available on the Data Platform
These tools can be used to search RICs by any criteria such as asset class, country, or exchange. However, each tool has its own limitations. For example, users can’t download or export the search results from the RIC Search web UI. The Data Platform Search API is more flexible and powerful than the RIC Search, but it limits the returned results to 10,000 entries.
This article introduces the bulk search API for real-time users to search for the whole instrument universe from the specified query criteria. The credential used in this article is the Version 2 Authentication for Real-Time Optimized. The access token retrieved from the authentication service can be used in the Bulk Search API. For more information, please refer to the Real-Time WebSocket API: Refinitiv Real-Time Optimized Version 2 Authentication Migration Guide article.
Bulk Search APIs
Bulk Search APIs are available of the Data Platform. It is created to help improve Real-Time RICs searching/querying capabilities, by allowing Real-Time clients to search for the whole instrument universe. The key capabilities include:
- Free text search for instrument name
- The ability to combine/link multiple different criteria specified by different parameters, such as permission entity, domain, record type, and exchange name
- Search for RIC name and domain (base query) with specified query criteria
- Search for RIC name that changed (delta query) with the same query criteria
There are three endpoints in the Bulk Search APIs.
1. /realtime/bulksearch/v1/base-queries
This endpoint supports the HTTP POST request. It is used to search for RICs based on criteria specified in request body. The request body contains the query criteria in JSON format.
The query response contains results inline. In case the result is too large to fit as inline result, the query response contains an Amazon Web Service S3 URL that lets users retrieve the result from the S3 URL. The response also contains the “queryID” property which can be used in the delta query.
2. /realtime/bulksearch/v1/delta-queries
This endpoint supports the HTTP POST request. It is used to create a delta query response that contains RICs that were previously not in the base query (add) and RICs that are no longer in the base query (remove). The request body contains the “queryID” of the base query and the query criteria in JSON format.
The query response contains results inline. In case the result is too large to fit as inline result, the query response contains an Amazon Web Service S3 URL that lets users retrieve the result from the S3 URL. The response also contains the “queryID” of the base query.
3. /realtime/bulksearch/v1/health
This endpoint supports the HTTP GET request. It is used to check the health status of the Bulk Search API.
Query Request Message
Request body is in JSON format and is almost the same for base query and delta query, except that an extra property "queryID" is required by the delta query. The available properties are:
| Name | Type | Description |
|---|---|---|
| queryID | String | The queryID of the base query. It is only required by the delta query |
| queries | Object | Search criteria |
| permissionedResults | Boolean | Should the results be filtered to only include RICs permissioned for the user |
Query Properties
The following properties can be used in the queries object.
| Name | Type | Description | Example |
|---|---|---|---|
| ric | Array of strings | An array of RICs. The OR operator is used for items in the array. |
["IBM.N","LSEG.L"] |
| domain | Array of integers | An array of domain types. The value of domain types are:
|
[6] [7,8,9] |
| pe | Array of integers | An array of permission entities PE is in the PROD_PERM (1) field which provides product permissions information. For example:
The OR operator is used for items in the array. |
[62] [249,4741] |
| pc | Array of integers | An array of permission code IDs The OR operator is used for items in the array. |
[32] [32, 4723] |
| pcname | Array of strings | An array of pemission code names The OR operator is used for items in the array. |
["EXNNOSL"] ["EXNNOSL", "EXNNVIE"] |
| recordType | Array of integers | An array of record types. recordType is in the RECORDTYPE (259) field which indicates the type of record. For example:
For more information, please refer to the Using FID 259 to identify your market data records article. The OR operator is used for items in the array. |
[33] [50,113] |
| exchangeEnum | Array of integers | An array of exchange enumeration values. exchangeEnum is an enumeration value of the RDN_EXCHD2(1709) enumeration field which identifies the source from where the data originates. The possible values of the RDN_EXCHD2 enumeration field are defined in the enumtype.def file. For example: ! The OR operator is used for items in the array. |
[1] [2,3,4] |
| exchangeCode | Array of strings | An array of exchange enumeration codes. exchangeCode is an enumeration display value of the RDN_EXCHD2(1709) enumeration field which identifies the source from where the data originates. The possible values of the RDN_EXCHD2 enumeration field are defined in the enumtype.def file. For example: ! The OR operator is used for items in the array. |
["ASE"] ["NYS","BOS","BIN"] |
| exchangeName | Array of strings | An array of exchange names. exchangeName is an enumeration meaning value of the RDN_EXCHD2(1709) enumeration field which identifies the source from where the data originates. The possible values of the RDN_EXCHD2 enumeration field are defined in the enumtype.def file. For example: ! The OR operator is used for items in the array. |
["NYSE American"] ["New York Stock Exchange"," Boston Stock Exchange"] |
The string properties (ric, exchangeCode, pcName, and exchangeName) also support the startswith, endswith, and contains functions. For example:
"ric": ["startswith('IBM')", "endswith('BK')"]
"exchangeCode": ["startswith('N')"]
"exchangeName": ["startswith('NYSE')"]
The queries property supports only one query property in the request. To use many query properties, the "and" and "or" logic operators can be used. The query also supports the "not" operator. For example:
{
"queries": {
"and": {
"domain": [6],
"ric": [
"startswith('JPY=')",
"startswith('THB=')"
]
}
}
}
This query searches for RICs that start with "JPY=" or "THB=" and the domain type is Market Price (6).
Request and Response Examples
1. Use a base query to search for IBM.N and LSEG.L
| Endpoint | POST: https://api.refinitiv.com/realtime/bulksearch/v1/base-queries |
| Request | { "queries": { "ric": [ "IBM.N", "LSEG.L" ] } } |
| Response | { "queryID": "692b211e5671dca732a64b6e947c41e71dc41902c7488b67c632a21c0d92ae30", "results": [ "IBM.N,6", "LSEG.L,6", "LSEG.L,7", "LSEG.L,8" ] } |
The response contains the Marke Price (IBM.N, LSEG.L), Market By Price (LSEG.L), and Market By Order (LSEG.L) RICs.
2. Use a base query to search for Market Price RICs of IBM.N and LSEG.L
The query uses the "and" operator with the ric, and domain query preperties.
| Endpoint | POST: https://api.refinitiv.com/realtime/bulksearch/v1/base-queries |
| Request | { "queries": "and": { "ric": [ "IBM.N", "LSEG.L" ], "domain": [ 6 ] } } } |
| Response | { "queryID": "bf3b84826bcd532ff1f781444004cf1baa1ea4d1501fd28f9fe7e5cfc75a7310", "results": [ "IBM.N,6", "LSEG.L,6" ] } |
The response contains the Marke Price RICs (IBM.N, LSEG.L)
3. Use a base query to search for Market Price Equity Cash Instrument RICs that start with "IBM." and "LSEG."
The query uses the "and" operator with the ric, domain, and recordType query preperties.
| Endpoint | POST: https://api.refinitiv.com/realtime/bulksearch/v1/base-queries |
| Request | { "queries": { "and": { "ric": [ "startswith('IBM.')", "startswith('LSEG.')" ], "domain": [ 6 ], "recordType": [ 113 ] } } } |
| Response | { "queryID": "dbc19d762da281098ac56515eca728758da9d6944f38cbdc86fbfb77409c8630", "results": [ "IBM.C,6", "IBM.N,6", "IBM.TH,6", "IBM.NEO,6", … "IBM.DY,6", "LSEG.LO,6", "IBM.GTX,6", "IBM.LTS,6", "IBM.F,6", "IBM.H,6", … "IBM.SBX,6", "LSEG.L,6", "IBM.LNXd,6", "IBM.EI,6", … ] } |
4. Use a base query to search for Market By Order Equity Cash Instrument RICs in the London Stock Exchange
The query uses the "and" operator with the exchangeCode, domain, and recordType query preperties.
| Endpoint | POST: https://api.refinitiv.com/realtime/bulksearch/v1/base-queries |
| Request | { "queries": { "and": { "exchangeCode": [ "LSE" ], "domain": [ 7 ], "recordType": [ 113 ] } } } |
| Response | { "queryID": "cac7fe37f44254458b0c2b56486ee4dc628c5c9abc55e7572074d2bbc7fc98e1", "results": [ "SVME.L,7", "METP.L,7", "3JD.L,7", "EMLI.L,7", "AIRAq.L,7", "MIVO.L,7", "PPH.L,7", "SAGS.L,7", "GBB01QM88=L,7", … ] } |
5. Use a base query to search for permissioned Market Price Equity Cash Instrument RICs in the London Stock Exchange
The query sets the permissionedResults to true and uses the "and" operator with the exchangeCode, domain, and recordType query preperties.
| Endpoint | POST: https://api.refinitiv.com/realtime/bulksearch/v1/base-queries |
| Request | { "queries": { "and": { "exchangeCode": [ "LSE" ], "domain": [ 6 ], "recordType": [ 113 ] } }, "permissionedResults":true } |
| Response | { "queryID": "c08bb52f3c2c0bd43a8179ab5d8241d1e6811c52437da4e8018de4de6a3518d1", "results": [ "/FMVde.TRE,6", "/0QPFs.TRE,6", "/THEONas.TRE,6", "/WT0Ytg.TRE,6", "/JE13de.TRE,6", "/ICAG.L,6", "/TIJf.TRE,6", "/0LSM.L,6", "/0QW1s.TRE,6", "/EQ6f.TRE,6", "/0FTW.L,6", "/STHEl.TRE,6", "/COPL.L,6", … ] } |
The response contains the delayed RICs in the London Stock Exchange which indicates that I only have permission to access delayed RICs in the London Stock Exchange.
6. Use a base query to search all RICs in the London Stock Exchange (Bulk Response)
The query uses the "and" operator with the exchangeCode query preperty.
| Endpoint | POST: https://api.refinitiv.com/realtime/bulksearch/v1/base-queries |
| Request | { "queries": { "and": { "exchangeCode": [ "LSE" ] } } } |
| Response | { "queryID": "57ef522588e98ed928a9cc8adb43d49a7deeeb6f9a1282deb3658131da52db31", "resultsURL": "https://a250649-use1-prod-bulk-search-query.s3.amazonaws.com/query/result/57ef522588e98ed928a9cc8adb43d49a7deeeb6f9a1282deb3658131da52db31.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAXLY6VSE7JPFTWE3P%2F20240507%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20240507T051340Z&X-Amz-Expires=21600&X-Amz-SignedHeaders=host&X-Amz-Security-Token=..." } |
In this example, the result is too large to fit as inline result so the query response contains an Amazon Web Service S3 URL in the resultsURL property that lets users download the result CSV file from the S3 URL.
ric,domain
/US89352HAZ29=TRE,6
USU9226VAF41=TRE,6
US29278GAK40=TRE,6
/XS1791704932=TRE,6
UBUQbe.TRE,6
ALINSpa.TRE,6
0RH9st.TRE,6
/0FTW.L,6
DE000MHB38J4=TRE,6
/XS2778205281=TRE,6
/EZ6NKT9BCQG1=TRE,6
/STHEl.TRE,6
1#0A0C.l,6
/XS2778181839=TRE,6
1#0HL8.l,6
7. Use a delta query to create a delta response against the query from the first example
The base query in the first example searches for IBM.N and LSEG.L and the queryID is “692b211e5671dca732a64b6e947c41e71dc41902c7488b67c632a21c0d92ae30”. The new delta query will search for IBM.N, LSEG.L, and GOOG.O on the Market Price domain.
| Endpoint | POST: https://api.refinitiv.com/realtime/bulksearch/v1/delta-queries |
| Request | { "queryID": "692b211e5671dca732a64b6e947c41e71dc41902c7488b67c632a21c0d92ae30", "queries": { "and": { "ric": [ "IBM.N", "LSEG.L", "GOOG.O" ], "domain": [ 6 ] } } } |
| Response | { "queryID": "692b211e5671dca732a64b6e947c41e71dc41902c7488b67c632a21c0d92ae30", "results": { "add": [ "GOOG.O,6" ], "remove": [ "LSEG.L,7", "LSEG.L,8" ] } } |
The response contains the add and remove properties. These indicate that the Market Price RIC of GOOG.O was added to the base query (the first example) and the Market By Price and Market By Order RICs of LSEG.L were removed from the base query (the first example).
8. Check the health status of the Bulk Search API
| Endpoint | GET: https://api.refinitiv.com/realtime/bulksearch/v1/health |
| Response | { "message": "OK", "details": { "region": "us-east-1", "environment": "prod", "db_conn": "ready", "data_update": "ready", "odps": "ready" } } |
These examples are also available in the public Postman workspace.
Python WebSocket Example
Bulk Search API can be used with Real-Time WebSocket applications that consume real-time data from Refinitiv Real-Time Optimized. The Python example is on GitHub. This example uses the Bulk Search API to search RICs that are in specific exchanges and record types. The example accepts the following parameters:
--clientid <ClientID> --clientsecret <ClientSecret> --exchangecode <Exchange Codes> --recordtype <Record Types>
For example:
python market_price_rdpgw_client_buk_search.py --clientid <ClientID> --clientsecret <ClientSecret> --exchangecode SET,LSE --recordtype 113
With the above parameters, the example will search for permissioned Market Price Equity Cash Instrument RICs in the London Stock Exchange and Stock Exchange of Thailand. The request message is:
{
"queries": {
"and": {
"exchangeCode": [
"SET",
"LSE"
],
"domain": [
6
],
"recordType": [
113
]
}
},
"permissionedResults": true
}
Then, it uses batch requests to retrieve snapshot data of those RICs.
{
"ID": "2",
"Streaming": false,
"Key": {
"Name": [
"/PTT.BK",
"/LSEG.L",
…
],
"Service": "ELEKTRON_DD"
},
"View": [
"DSPLY_NAME",
"BID",
"ASK",
"TRDPRC_1",
"TIMACT",
"HST_CLOSE",
"HSTCLSDATE"
]
}
The output looks like this:
Summary
Bulk Search API is the REST API available on the Data Platform. It can be used to search RICs for the whole instrument universe by using instrument names, domain types, record types, permission entities, and exchange information. It supports the startswith, endswith, and contains functions for the string properties. It also supports the "and", "or", and "not" login operators to create complex queries. The search request can be in the response. If the result is too large to fit in the response, the response will contain an Amazon Web Service S3 URL that lets users download the result CSV file from this S3 URL.
Bulk Search API can be used in any real-time applications that connect Refinitiv Real-Time Optimized to subscribe real-time data. For example, a real-time WebSocket application can use Bulk Search API to get all equity market price RICs in London Stock Exchange and then uses batch requests in the WebSocket API to get real-time data of those RICs.
