Getting Start Unit Test with Pytest for an HTTP REST Python Application

You can find more detail about the unit test concept from the following resources:

• Python Guide: Testing Your Code article.
• How and when to use Unit Testing properly post.

Introduction to Pytest framework

The pytest (sometimes called py.test) is one of the most popular all-purpose Python testing frameworks. This open-source framework lets developers/QAs write small, readable, and scalable test cases that are suitable for both simple function testing and complex applications. Compared to the bulky class-based unit test framework like Python's built-in unittest, the pytest framework has an easier learning curve with more flexibility.

Pytest Key Features:

• Use the Python standard assert statement for verifying expectations and values in Python tests, no more self.assertXXX methods like the unittest
• Auto-discovery of test modules and functions
• Modular fixtures for managing small or parametrized long-lived test resources
• You can run unittest cases with pytest too!
• Provide a lot of official and community plugins for extending the framework capability and integrations.

The pytest framework has the following requirements:

• The test cases file name must follow the form test_.py* or *_test.py in the current directory and its subdirectories
• The test cases can be test prefixed test functions or methods outside of class
• The test cases can be test prefixed test functions or methods inside Test prefixed test classes (without an init method)

Example from pytest official page:

To run the test, just run the following command

Result:

Please find more detail about the pytest framework from the following resources:

• pytest official page
• pytest getting started page
• Using pytest - Real Python

Let’s leave the pytest framework overview there.

What is Refinitiv Data Platform (RDP) APIs?

Now let me turn to our example HTTP API for this project. The Refinitiv Data Platform (RDP) APIs provide various Refinitiv data and content for developers via easy-to-use Web-based API.

RDP APIs give developers seamless and holistic access to all of the Refinitiv content such as Environmental Social and Governance (ESG), News, Research, etc, and commingled with their content, enriching, integrating, and distributing the data through a single interface, delivered wherever they need it. The RDP APIs delivery mechanisms are the following:

• Request - Response: RESTful web service (HTTP GET, POST, PUT or DELETE)
• Alert: delivery is a mechanism to receive asynchronous updates (alerts) to a subscription.
• Bulks: deliver substantial payloads, like the end-of-day pricing data for the whole venue.
• Streaming: deliver real-time delivery of messages.

This example project is focusing on the Request-Response: RESTful web service delivery method only.

For more detail regarding the Refinitiv Data Platform, please see the following APIs resources:

• Quick Start page.
• Tutorials page.

RDP APIs Application Workflow

Refinitiv Data Platform entitlement check is based on OAuth 2.0 specification. The first step of an application workflow is to get a token from RDP Auth Service, which will allow access to the protected resource, i.e. data REST API.

The API requires the following access credential information:

• Username: The username.
• Password: Password associated with the username.
• Client ID: This is also known as AppKey, and it is generated using an App key Generator. This unique identifier is defined for the user or application and is deemed confidential (not shared between users). The client_id parameter can be passed in the request body or as an “Authorization” request header that is encoded as base64.

Once the authentication success, the function gets the RDP Auth service response message and keeps the following RDP token information in the variables.

• access_token: The token used to invoke REST data API calls as described above. The application must keep this credential for further RDP APIs requests.
• refresh_token: Refresh token to be used for obtaining an updated access token before expiration. The application must keep this credential for access token renewal.
• expires_in: Access token validity time in seconds.

Next, after the application received the Access Token (and authorization token) from RDP Auth Service, all subsequent REST API calls will use this token to get the data. Please find more detail regarding RDP APIs workflow in the following resources:

• RDP APIs: Introduction to the Request-Response API page.
• RDP APIs: Authorization - All about tokens page.

Project Structure

That brings us to the project structure. This example project is a Python console application that login to the RDP platform, then requests the company's Environmental Social and Governance (ESG) data and meta information from the RDP ESG and Search Explore services respectively. The project structure is as follows:

Note: The README, Docker, and git-related files are not shown in the project structure above.

• app.py: The main console application.
• pytest.ini: The pytest settings configuration file.
• rdp_controller/rdp_http_controller.py: The main HTTP operations class. This is our focus class for unit testing.
• tests/conftest.py: The root file that provides fixtures for all test cases in the tests folder.
• tests/test_rdp_http_controller.py: The main test cases file that tests all rdp_http_controller.py class's methods. This is our focus test suite in this project.
• tests/test_app.py: The test suite class that tests some app.py methods.
• tests/data: The test suite resource files.

Setting Unit Test Environment

Let’s start with the class that operates HTTP request-response messages with the RDP services.

It loads the test configurations such as the RDP APIs URLs from a .env.test environment variables file.

Let's start with setting up the test configurations, resources, and environment variables. This project keeps the test configurations such as mock RDP credentials, and API URLs in a ${project root}/.env.test environment variables file. And then uses the pytest-dotenv plugin to the os.environ variable without hardcoded path reference to the file.

To load a custom environment variables into pytest, we create a pytest configuration file named pytest.ini at the root of project folder to specify where the env_var is and overriding any variables already defined in the process' environment.

Note:

• This plugin uses the python-dotenv under the hood, so the python-dotenv dependency will be installed too.
• If you have a .env file in the same folder, somehow the plugin always loads .env file content instead even if you did not specify it. I am workaround this issue by using a .env.development file name for running the application.

This os.environ environment variables and the RDPHTTPController class will be used in almost every test case, so we set them as a fixtures. The fixture can be are data, class, preconditions states, context, or resources needed to run a test. Unlike the unittest framework,the pytest fixture is in a functional form that can be used in a modular manner.

The pytest fixtures are defined using the @pytest.fixture decorator. It can be defined in a test case (supports only in that test case) or in a conftest.py file (for sharing fixtures to all test cases in the same directory). This project uses the latter case, so the content of a conftest.py file is as follows:

The supply_test_config fixture function is for the os.environ environment variable dictionary and the supply_test_class fixture function is for sharing the RDPHTTPController class among test cases. The test cases can use those fixture functions separately based on each test requirement.

You may be noticed the `scope='class' in the @pytest.fixture(scope='class') decorator, which means the fixture will be destroyed during teardown of the last test in the class.

That is all for the fixture preparation.

Unit Testing RDP APIs Authentication with Pytest

Moving on to the next topic, the class that operates HTTP request-response messages with the RDP services. The rdp_controller/rdp_http_controller.py class uses the Requests library to send and receive data with the RDP HTTP REST APIs. The code for the RDP authentication is shown below.

The rdp_authentication() method above just create the request message payload, and send it to the RDP Auth service as an HTTP Post request. The return values can be as follows

• If the authentication success, returns the access_token, refresh_token, and expires_in information to the caller.
• If the URL or credentials parameters are empty or none, raise the TypeError exception to the caller.
• If the authentication fails, raise the Requests' HTTPError exception to the caller with HTTP status response information.

Let’s leave the rdp_authentication() method there and continue with the test case. The basic test case scenario is to check if the rdp_authentication() method can handle a valid RDP login and empty parameters scenarios.

The test class is tests\test_rdp_http_controller.py file (please noticed a tests prefixed). The basic test case code for the rdp_authentication() method is as follows:

The test_login_rdp_success() test case requests the supply_test_config and supply_test_class fixtures as function parameters. We get the RDPHTTPController object (as app function variable) and the RDP Auth URL endpoint string (as auth_endpoint function variable) from the supply_test_config and supply_test_class fixtures to use as the test case's resources.

The test_login_rdp_success() is a test case for the successful RDP Authentication login scenario. It just sends the RDP Auth Service URL and RDP credentials to the RDPHTTPController.rdp_authentication() method and checks the response token information. Please noticed that a unit test just focuses on if the rdp_authentication() returns no empty/zero token information only. The token content validation would be in a system test (or later) phase.

Please see more detail about the assertions in the test on the pytest framework page.

What is about checking how the rpd_authentication() handles empty or none parameters? We create a new test_login_rdp_none_empty_params() test case to check if the method throws the TypeError exception and does not return token information to a caller as expected. The code is shown below.

The test_login_rdp_none_empty_params() test case uses pytest.raises() as a context manager to check if the called method throws the expected exception type. If the code block does not raise the expected exception (TypeError in the test above), or no exception at all, the check will fail instead.

Please note that the pytest.raises() method returns the ExceptionInfo object which can be used to inspect the details of the captured exception via .type, .value, and .traceback attributes. This behavior is a bit different from unittest assertRaises() which returns the context manager that stores the caught exception as is.

The example of a test runner result is shown below.

However, the test suite above makes HTTP requests to RDP APIs in every run. It is not a good idea to flood requests to external services every time developers run a test suite when they have updated the code or configurations.

Unit test cases should be able to run independently without relying on external services or APIs. The external dependencies add uncontrolled factors (such as network connection, data reliability, etc) to unit test cases. Those components-to-components testing should be done in an integration testing phase.

So, how can we unit test HTTP request method calls without sending any HTTP request messages to an actual server? Fortunately, developers can simulate the HTTP request and response messages with a mock object.

Mocking Python HTTP API call with Responses

A mock is a fake object that is constructed to look and act like real data within a testing environment. We can simulate the various scenario of the real data with a mock object, then use a mock library to trick the system into thinking that that mock is the real one.

The purpose of mocking is to isolate and focus on the code being tested and not on the behavior or state of external dependencies. By mocking out external dependencies, developers can run tests as often without being affected by any unexpected changes or irregularities of those dependencies. Mocking also helps developers save time and computing resources if they have to test HTTP requests that fetch a lot of data.

I have demonstrated how to use the Responses mocking library in the previous unittest framework project. I am using other popular Requests mocking library which is requests-mock with this pytest example.

Adding a mock Object to the test case

So, I will start with a mock object for testing a successful RDP login case. Firstly, create a supply_test_mock_json fixture method with a dummy content of the RDP authentication success response message in a contfest.py file.

And then we can modify test_login_rdp_success test case and other test cases to request this supply_test_mock_json fixture as a function argument.

The requests-mock library provides an external fixture registered with pytest for developers. Developers can simply just specifying requests-mock as a function parameter without the need to import requests-mock library in the test files (you still need to install the library). Once the requests-mock is loaded to a test case, developers can specify the endpoint URL, HTTP method, status response, response message, etc of that request via a requests_mock.post() and requests_mock.get() methods.

Example:

The code above set a Responses mock object with the https://api.refinitiv.com/auth/oauth2/v1/token URL and HTTP POST method. The requests-mock library then returns a valid_auth_json JSON message with HTTP status 200 and Content-Type application/json from the supply_test_mock_json fixture to the application for all HTTP POST request messages to https://api.refinitiv.com/auth/oauth2/v1/token URL without any network operations between the machine and the actual RDP endpoint.

Testing Invalid RDP Client ID Authentication Request-Response

This mock object is also useful for testing false cases such as invalid login too. The test_login_rdp_invalid() method is a test case for the RDP Authentication login failure scenario. I am starting by adding a new invalid_clientid_auth_json dummy content of the RDP invalid ClientID log response to a supply_test_mock_json fixture method.

And then we set a Responses mock object for the https://api.refinitiv.com/auth/oauth2/v1/token URL and HTTP POST method with the expected error response message and status (401 - Unauthorized) that is loaded from this supply_test_mock_json fixture.

Once the rdp_authentication() method is called, the test case verifies if the method raises the requests.exceptions.HTTPError exception with the expected error message and status. The test case also makes assertions to check if the method does not return token information to a caller.

With mocking, a test case never needs to send actual request messages to the RDP APIs, so we can test more scenarios for other RDP services too.

That covers some of the basic RDP authentication methods testing.

Unit Testing for RDP APIs Data Request

That brings us to requesting the RDP APIs data. All subsequent REST API calls use the Access Token via the Authorization HTTP request message header as shown below to get the data.

• Header:
  • Authorization = Bearer <RDP Access Token>

Please notice the space between the Bearer and RDP Access Token values.

The application then creates a request message in a JSON message format or URL query parameter based on the interested service and sends it as an HTTP request message to the Service Endpoint. Developers can get RDP APIs the Service Endpoint, HTTP operations, and parameters from Refinitiv Data Platform's API Playground page - which is an interactive documentation site developers can access once they have a valid Refinitiv Data Platform account.

The example console application consumes content from the following RDP Services:

• ESG Service /data/environmental-social-governance//views/scores-full endpoint that provides full coverage of Refinitiv's proprietary ESG Scores with full history for consumers.
• Discovery Search Explore Service /discover/search//explore endpoint that explore Refinitiv data based on searching options.

However, this development article covers the Search Explore Service test cases only. The ESG Service's test cases have the same test logic as the Discovery Search Explore's test cases.

Unit Testing HTTP Request Source Code for The RDP Discovery Search Explore Service

Now let me turn to test the Discovery Search Explore service endpoint. The code in the rdp_controller/rdp_http_controller.py class for requesting the search result data is shown below.

The rdp_request_search_explore() method above just create the request message payload, and send it to the RDP Search Explore service as an HTTP POST request with the Requests requests.post() method. The return values can be as follows

• If the request success (HTTP status is 200), returns the response data in JSON message format.
• If the URL or access token, or universe values are empty or none, raise the TypeError exception to the caller.
• If the request fails, raise the Requests' HTTPError exception to the caller with HTTP status response information.

Testing a valid RDP Search Explore Request-Response

The first test case is the request data success scenario. I will begin by creating a test-content file with a valid Search Explore response message. A file is rdp_test_esg_fixture.json in a tests/data folder.

We are not going to load this data into the conftest.py fixture because only one test case uses this dummy data. We are using it as a test data resource with the pytest-datadir library. The pytest-datadir will copy the original file to a temporary folder, so changing the file contents won't change the original data file and the test case can use it in the test assertions.

However, the RDP Search request message will be reused by multiple test cases as a based line for each test's request message payload, so we add it as a new search_explore_payload content to a supply_test_mock_json fixture method as follows.

Next, create the test_request_search_explore() method in the test_rdp_http_controller.py file to test the easiest test case, the successful ESG data request-response scenario.

The code above does the following tasks:

• Injects the test data location via a shared_datadir function parameter to the test case
• Read the test_search_fixture.json test data content from a shared_datadir folder to a valid_response variable
• Create a request message based on the search_explore_payload message for the supply_test_mock_json fixture
• Mock a request to the RDP https://api.refinitiv.com/discovery/search/v1/explore endpoint URL and valid_response variable.

When the Requests library receives the HTTP POST request for that URL, it returns a mock Search result JSON object with HTTP Status 200 and Content-Type application/json to the application. The test_request_search_explore() method then verifies if the response data is in JSON/Dictionaries type, and checks if the message contains the basic fields.

Testing Requesting RDP Search service with an expired token

That brings us to one of the most common RDP APIs failure scenarios, applications request data from RDP with an expired access token. The test_request_search_explore_token_expire() method is the one for testing this case with the RDP ESG Service.

The first step is to add a mock token expired message token_expire_json to a supply_test_mock_json fixture. We add this to a fixture because we can reuse this token expired message in all RDP data services test cases as well.

The token expires error message is sent from the RDP services to applications with HTTP error status (401 - As of Feb 2023), the test_request_search_explore_token_expire() method simulates the token expire error message and HTTP 401 status with a mock object as follows.

Next, this test case verifies if the rdp_request_search_explore() method raises the requests.exceptions.HTTPError exception with the expected token expired error message and status.

The other common RDP APIs failure scenario is the application sends the request message to RDP without the access token in the HTTP request header. However, the access token is one of the rdp_request_search_explore() method required parameters. If the access token is not presented (None or Empty), the method raises the TypeError exception and does not send an HTTP request message to the RDP. The test_request_search_explore_none_empty() method is the one that covers this test case. There is also a test_request_search_explore_invalid_json() test case method in the file that demonstrates how to test an invalid JSON request message scenario.

That covers the Discovery Search Explore service test cases. If you are interested in RDP ESG service test cases, please check the test_request_esg_xxx methods which have the same testing and mocking logic as all previous cases that I have mentioned above.

Bonus: Pytest Markers

Now we come to one of the most unique features of pytest, the markers. The pytest framework supports test case metadata settings known as markers (pytest.mark). The markers are used by plugins and are commonly used to run or skip tests with specific markers. Here are some of the builtin markers:

• skip: always skip a test function
• skipif - skip a test function if a certain condition is met
• xfail - produce an “expected failure” outcome if a certain condition is met
• parametrize - perform multiple calls to the same test function.

The framework also supports custom markers to match the project's requirements too. Once the custom markers have been set, developers can run pytest -m <mark> to run test-specific tests.

To create custom markers, the first thing you need to do is define your markers for grouping test cases in the pytest.ini configuration file as follows:

Then, add the marker to test methods in test_rdp_http_controller.py file with @pytest.mark.<custom mark> decorator. Please see the example modified code below.

Please be noticed that you can set multiple markers for each test method.

To run specific markers, you can run pytest with -m <mark> command as follows:

Example: Run only success scenarios test cases:

You can select multiple markers with and, or, not operators to combine multiple markers. The following example shows how to specify only RDP ESG service unsuccess test cases.

Please see more detail about pytest markers and custom markers from the following resources:

• pytest document: How to mark test functions with attributes
• pytest document: Working with custom markers
• pytest document: Raising errors on unknown marks

That’s all I have to say about unit testing the Python HTTP code with Requests and Responses libraries.

Prerequisite

This demo project requires the following dependencies.

1. RDP Access credentials.
2. Python Anaconda or MiniConda distribution/package manager.
3. Internet connection.

Please contact your Refinitiv representative to help you to access the RDP account and services. You can find more detail regarding the RDP access credentials set up from the lease see the Getting Started for User ID section of the Getting Start with Refinitiv Data Platform article.

How to run the example test suite

The first step is to unzip or download the example project folder into a directory of your choice, then set up Python or Docker environments based on your preference.

Run example test suite with Conda

Firstly, open Anaconda Prompt and go to the project's folder.

Then run the following command in the Anaconda Prompt application to create a Conda environment named rdp_pytest for the project.

Once the environment is created, activate a Conda rdp_pytest environment with this command in Anaconda Prompt.

Run the following command to the test dependencies (requirements_test.txt) in the rdp_pytest environment

Alternatively, you can use VS Code Python testing tool to run test cases too. The .vscode project setting is included with the project.

That covers how to run an example test suite.

Conclusion and Next Steps

That brings me to the end of this pytest project example. Unit testing is now the mandatory process of a software development lifecycle for both modern and legacy applications. It helps to expose unintentional behaviors of a tiny part of the application quicker than trying to find bugs in a big complex phase. It speeds up the overall feedback loop and improves trust among the project team. Unit testing also helps improves application source code quality, developers have more confidence in refactoring the source code for better performance and cleaner code. As the author of this article, I also have learned a lot from this project. There are a lot of error handlers or code logic that I never think of until I started to write unit test cases.

This example project demonstrates the manual unit testing method. However, developers should run unit test cases automatically every time they made changes to the code (or configurations). The most practical technique is running automated unit tests as part of the developers' Continuous Integration/Continuous Delivery (CI/CD) pipeline. Developers can apply the TDD (Test-driven development) approach that writing and correcting the failed tests before writing new code with their project too.

The pytest test framework and requests-mock mocking library are other good choices for learning a unit testing with Python and HTTP REST applications. They are easy to learn, provide powerful but simple-to-use features for developers, and support many plugins and Python development frameworks.

At the same time, the Refinitiv Data Platform (RDP) APIs provide various Refinitiv data and content for developers via an easy-to-use Web-based API. The APIs are easy to integrate into any application and platform that supports the HTTP protocol and JSON message format.

That’s all I have to say about unit testing with pytest.

References

That brings me to the end of my unit test example project. For further details, please check out the following resources:

• Refinitiv Data Platform APIs page on the Refinitiv Developer Community website.
• Refinitiv Data Platform APIs Playground page.
• Refinitiv Data Platform APIs: Introduction to the Request-Response API.
• Refinitiv Data Platform APIs: Authorization - All about tokens.
• Limitations and Guidelines for the RDP Authentication Service article.
• Getting Started with Refinitiv Data Platform article.
• Python pytest framework official page.
• Python requests-mock library page.
• pytest-datadir library page.
• Python Guide: Testing Your Code article.
• Using pytest - Real Python website.
• How and when to use Unit Testing properly post.
• 13 Tips for Writing Useful Unit Tests blog post.
• pytest document: How to mark test functions with attributes
• pytest document: Working with custom markers

For any questions related to Refinitiv Data Platform APIs, please use the RDP APIs Forum on the Developers Community Q&A page.