Technologies | February 15, 2024

PyTest + AWS Lambda function. Introduction to Python Unit Testing

In today’s fast-developing cloud environment, developers need tools for building scalable and flexible serverless applications. However, to maintain the high quality of code and application functionality, the effective unit testing of serverless functions is crucial. If you are working in an AWS environment and wondering how to effectively perform unit testing for AWS Lambda with the PyTest framework, you should find some tips in my article. Let’s take a look at some practical examples of testing AWS Lambda code.

pytest aws lambda

AWS Lambda at a glance

Let’s start with the fact that serverless services are currently some of the most advanced features of cloud computing. Today, every cloud provider has function-as-a-service solutions in their offer. In the project I am involved in, we use AWS Lambda (read more about its capabilities in this article: AWS Lambda Functions). 

In a nutshell, AWS Lambda is a serverless service provided by Amazon Web Services that allows developers to run code in the cloud in response to various events. The serverless model means that infrastructure management is on the cloud provider’s side. This makes it just perfect for developing applications requiring rapid scaling and efficient resource management. 

Also read: 

The testing challenge. Why do you need to test serverless applications?

What do standard applications and serverless applications have in common? Both should pass unit tests. Testing serverless Lambda function code is quite a challenge. Using my project as an example, I would like to present the challenges we face as a team in the context of the unit testing of our application. 
 
Firstly, AWS Lambda functions are often tightly integrated with other cloud resources, such as databases or message queues. This can cause problems if developers want to isolate unit tests, as we need to create tests independent of other components. In any scenario, we need to ensure that our unit tests do not affect the state of other services (such as changing records in a database). 

Another challenge is handling asynchronous communication. In many scenarios, our application responds to asynchronous events such as processing an incoming message from a queue. Testing such cases requires appropriate strategies to keep control of the data flow and ensure that the application behaves as we expect.  

Why is test coverage within a Lambda function so important?

Before getting into the practical aspects of unit testing, let’s see why it is so important, especially in the context of Lambda functions. 

Keeping functions working correctly  

Lambda functions are often small pieces of code invoked by specific events. Unit tests help ensure that these functions behave as expected in different scenarios.  

Quick error detection  

AWS Lambda can respond to various events. If your unit tests are well-written, it can quickly detect errors in the code while the product is still being developed. Unit testing is crucial, as it allows you to avoid introducing errors into the production environment and thus ensures that the application is reliable at this stage. 

Shortening the development cycle  

Unit testing enables developers to iterate quickly when making changes to code. By shortening the development cycle, the application is developed faster and the value is delivered quicker.  
 
Now that we know why unit testing is important, let’s consider how to do it effectively with the PyTest framework. 

What exactly is PyTest?

PyTest is a tool for testing code in Python which allows you to write both simple and complex tests. Among the available tools, Selenium, Cypress, Playright, or Robot Framework are popular (we have already discussed them for you in other articles and on webinars). It’s time to take a closer look at PyTest, which is particularly useful in an AWS Lambda function environment, as it allows local testing before deployment. 

What is more, thanks to its integration with mocking libraries such as Moto, you can simulate (“mock”) AWS services directly in your local environment. This will allow you to test your code thoroughly before deploying to the AWS cloud. 

Practical applications of unit testing

1. Handling expected errors 

Let’s assume we are developing a Lambda function to process online orders. The unit tests may include error-handling scenarios, such as no connection to the database or a problem with a third-party payment provider. By ensuring that the function handles these situations correctly, we can increase the system’s resilience to failures. 

# An example of code for error-handling 

 

def test_error_handling():  

 

 # We simulate a database communication error  

 

 event = {"order_id": "123"}  

 

 result = process_order(event, None)  

 

  

 

 assert result == {"statusCode": 500, "message": "Internal server error"}  

2. Validation of input data  

Lambda functions are often called by events, and the input data is crucial for the function to work correctly. Unit tests can cover different instances of input data. You gain confidence that the function validates them correctly and responds as expected. 

 #Example of a unit test for validating input data
def test_input_validation():  

 

 event = {"customer_id": "123", "order_total": 50.0}  

 

 result = process_order(event, None)  

 

 assert result == {"statusCode": 200, "message": "Order processed successfully"}  

# We test invalid input data
 
 event_invalid = {"customer_id": "abc", "order_total": "invalid"}  

 

 result_invalid = process_order(event_invalid, None)  

 

 assert result_invalid == {"statusCode": 400, "message": "Invalid input data"}  

3. Integrations with AWS services  

Lambda functions often use other AWS services, such as the storage service S3 or the NoSQL database service DynamoDB. We can use mocking techniques to avoid actual calls to these services while testing. PyTest offers the pytest-mock module, which makes it easy to create mocks for selected services. 

import pytest  

 

from my_module import process_s3_event  

# An example of a unit test using pytest-mock to mock an AWS service 

def test_s3_upload(mocker):  
 # We create a mock of the S3 client 

 

 s3_client_mock = mocker.Mock()  

 

 # We update the (“patch”) "boto3.client" function in such a way that it returns our mock  

 

 mocker.patch("boto3.client", return_value=s3_client_mock)  

 

  

 

 # We create an example of an event  

 

 event = {"data": "example"}  

 

 # We call the function that processes the event  

 

 result = process_s3_event(event, None)  

 

  

 

 # We check if the function has returned the expected result  

 

 assert result == {"statusCode": 200, "message": "S3 event processed successfully"}  

 

 # We check that the "upload_file" method on the S3 client's mockup was called exactly once 

 

 s3_client_mock.upload_file.assert_called_once_with("source_file", "bucket", "destination_key")  

4. Multi-layer applications  

In a microservice architecture, where Lambda functions act as microservices, unit testing becomes even more complex. A unit testing framework like PyTest allows you to create layered tests that check integrations between various microservices. 

import pytest  

 

from my_module import process_microservices  

 

  

 

# An example of a multi-layer unit test 

 

def test_microservices_integration(mocker):  

 

 # We create a mock for Lambda A function  

 

 lambda_a_mock = mocker.Mock()  

 

 # We create a mock for Lambda B function  

 

 lambda_b_mock = mocker.Mock()  

 

  

 

 # We define what data will be passed between microservices 

 

 event_a_to_b = {"data": "example_data_from_A_to_B"}  

 

 event_b_to_a = {"data": "example_data_from_B_to_A"}  

 

  

 

 # We determine what should be returned by mocks of the Lambda function  

 

 lambda_a_mock.return_value = {"statusCode": 200, "message": "Response from Lambda A"}  

 

 lambda_b_mock.return_value = {"statusCode": 200, "message": "Response from Lambda B"}  

 

  

 

 # We patch the imports in the process_microservices function to use our mocks 

 

 mocker.patch("your_module.lambda_a_function", lambda_a_mock)  

 

 mocker.patch("your_module.lambda_b_function", lambda_b_mock)  

 

  

 

 # We call the process_microservices function with sample data 

 

 result = process_microservices(event_a_to_b, None)  

 

  

 

 # We check if the function has returned the expected result after the integration of microservices 

 

 assert result == {"statusCode": 200, "message": "Microservices processed successfully"}  

 

  

 

 # We check that Lambda functions A and B were called with the right data 

 

 lambda_a_mock.assert_called_once_with(event_a_to_b, None)  

 

 lambda_b_mock.assert_called_once_with(event_b_to_a, None)  
 

Techniques of testing AWS Lambda functions

1. Fault tolerance testing  

This is a key component of AWS Lambda function testing. Fault tolerance testing covers scenarios in which one of the AWS services working with the function is unavailable or returns errors. An example of the test may look like this: 

import pytest  

 

from unittest.mock import patch, Mock  

 

  

 

  

 

# Example of a fault-tolerance test  

 

def test_error_resilience(mocker):  

 

 # We mock a service that returns an error  

 

 external_service_mock = mocker.Mock()  

 

 external_service_mock.side_effect = Exception("Service unavailable")  

 

  

 

 mocker.patch("boto3.client", return_value=external_service_mock)  

 

  

 

 event = {"data": "example"}  

 

 result = process_event(event, None)  

 

 assert result == {"statusCode": 500, "message": "Internal server error"}  

 

  

 

  

 

2. Test-Driven Development (TDD) tests 

The practice of TDD means writing unit tests before implementing functions. This allows for a more informed code design and ensures that tests cover every line of code. 

Below is an example of TDD for a Lambda function: 

import pytest  

 

  

 

# Example of unit testing with TDD  

 

def test_tdd():  

 

 event = {"data": "example"}  

 

 result = process_event(event, None)  

 

 assert result == {"statusCode": 200, "message": "Event processed successfully"}  

 

  

 

 # We add a new functionality and write a test for it first 

 

 new_feature_result = new_feature(event)  

 

 assert new_feature_result == {  

 

 "statusCode": 200,  

 

 "message": "New feature processed successfully",  

 

 }  
  

 

3. Using Fixture in PyTest  

PyTest Fixtures are functions that run before tests (and optionally after them) that can prepare the necessary resources. For example, a fixture can create a fake input or configure a mock of any AWS resource before running tests. This way, each test runs in an isolated and controlled environment.

import boto3  

 

import pytest  

 

from moto import mock_s3  

 

  

 

@pytest.fixture()  

 

def mock_s3_resource():  

 

 with mock_s3():  

 

 # We use Moto library to simulate the S3 service  

 

 import boto3  

 

  

 

 s3 = boto3.resource("s3", region_name="us-east-1")  

 

 # Crearing mock – AWS S3 resource 

 

 s3.create_bucket(Bucket="mocked-bucket")  

 

 yield s3  

 

 # Teardown (resource cleansing) is not required, Moto performs automatic cleansing  

 

  

 

# We use a decorator to turn on S3 simulation for the test  

 

@mock_s3  

 

def test_s3_operations(mock_s3_resource):  

 

 s3 = mock_s3_resource  

 

  

 

 # Perform requested operations on S3  

 

 s3.Object("mocked-bucket", "test.txt").put(Body="Hello, World!")  

 

  

 

 # Check if the performed operations were successful  

 

 obj = s3.Object("mocked-bucket", "test.txt").get()  

 

 assert obj["Body"].read().decode("utf-8") == "Hello, World!"  

4. Parameterization of tests 

Parameterization allows you to run the same test with different input data sets. Using the @pytest.mark.parametrize decorator, you can specify different data sets for a single test function. This way, you can easily verify the performance of Lambda functions in different scenarios without writing multiple similar tests. 

@pytest.mark.parametrize(  

 

 "input, expected",  

 

 [  

 

 ({"data": "example1"}, {"statusCode": 200, "message": "Success"}),  

 

 ({"data": "example2"}, {"statusCode": 400, "message": "Error"}),  

 

 ],  

 

)  

 

def test_lambda_function(input, expected):  

 

 result = lambda_handler(input, None)  

 

 assert result == expected  

How to start unit testing with the PyTest framework

PyTest installation 

  1. Make sure you have Python installed on your system.  
  2. Open a terminal or command line.  
  3. To install PyTest, type the following command: 
pip install pytest  

3. If you plan to use libraries like Moto for more advanced testing, of course, you will need to install them as well – for example: 

pip install moto  

Test preparation  

  1. Prepare your unit tests in Python (.py) files that contain test functions. Your tests should be organized following the PyTest convention, in which test function names start with “test_”.  

Running tests  

  1. In the terminal, navigate to the directory where your test files are located, or, if they are in the same directory as the terminal, you can skip this step. 
  1. To execute all defined tests from a given file, use the following command, specifying the name of the test file: 
pytest file_name.py  

For example, if your tests are in the my_tests.py file, the command will look like this:  

pytest my_tests.py  

If you want to test all files located in the current directory, use the command: 

pytest .  

Test results analysis 

  1. Once the tests are executed, PyTest will provide detailed information about the test run and its results. You will see which tests succeeded, which failed, and why. 

Practical application of more advanced unit testing and mocking of AWS services for Lambda 

Now that you know what the PyTest framework is, what unit tests are for, and how they can be used with various techniques, it’s time to move on to a more advanced, practical application of unit testing and AWS service mocking for Lambda functions. When developing serverless event-driven applications using AWS Lambda, the best practice is to validate individual components and services.  

Now let’s get into an example of a function for handling notifications in the cloud. Imagine that we have an application that needs to quickly notify users about various events – from a meeting reminder to a sale notification.  

That’s where AWS SNS, a service that allows you to send notifications in the form of messages to a wide range of subscribers, comes in. We can think of it as a cloud-based instant messaging system. Here, the AWS Lambda function acts as an intermediary – it accepts specific requests and forwards them to SNS, which then handles the distribution of messages. 

How does it work? The process is simple:  

  1. Message preparation: The Lambda function takes two key elements – the identifier of the SNS subject (to whom we want to send the notification) and the message content.  
  2. Sending the message: The function communicates with the SNS using a special API to deliver the message to the specified subject. 

You may be wondering: how can we trust that this function works properly? The answer lies in unit testing itself. We run test scenarios to make sure our code does exactly what it’s supposed to do in different situations. We simulate (“mock”) the operation of the SNS service to see how the function behaves when everything goes successfully, and how it handles any errors. 

Before writing the first unit test, we will look at an example of the AWS Lambda function we want to test: 

main.py:  

 

import boto3  

 

import json  

 

  

 

def send_notification(sns_topic_arn, message):  

 

 if not message:  

 

 raise ValueError("Message cannot be empty")  

 

 try:  

 

 sns_client = boto3.client("sns")  

 

 response = sns_client.publish(TopicArn=sns_topic_arn, Message=message)  

 

 return response  

 

 except boto3.exceptions.Boto3Error as e:  

 

 print(f"Error sending notification: {e}")  

 

 raise  

 

  

 

def lambda_handler(event, context):  

 

 # Parsowanie zdarzenia wejściowego  

 

 sns_topic_arn = event.get("sns_topic_arn")  

 

 message = event.get("message")  

 

  

 

 # Validation of input data  

 

 if not sns_topic_arn or not message:  

 

 return {"statusCode": 400, "body": json.dumps("Missing sns_topic_arn or message")}  

 

  

 

  try:  

 

 # Calling function sending notification  

 

 response = send_notification(sns_topic_arn, message)  

 

 return {"statusCode": 200, "body": json.dumps("Notification sent successfully")}  

 

 except Exception as e:  

 

 return {"statusCode": 500, "body": json.dumps(str(e))}  

 

  

  

 

import pytest  

 

import boto3  

 

from moto import mock_sns  

 

from unittest.mock import patch  

 

from main import lambda_handler # Importing from a handler  

 

  

 

# Preparing test environment  

 

@mock_sns  

 

class TestLambdaFunction:  

 

 # Test to see if we correctly handle the situation when input is missing 

 

 def test_missing_data(self):  

 

 response = lambda_handler({"sns_topic_arn": "", "message": ""}, None)  

 

 assert response["statusCode"] == 400  

 

 assert "Missing sns_topic_arn or message" in response["body"]  

 

  

 

 # Test to see if the function sends the notification correctly 

 

 @patch("main.boto3.client")  

 

 def test_send_notification_success(self, mock_boto3_client):  

 

 mock_sns_client = boto3.client("sns")  

 

 mock_boto3_client.return_value = mock_sns_client  

 

 mock_sns_client.publish.return_value = {"MessageId": "12345"}  

 

  

 

 response = lambda_handler(  

 

 {  

 

 "sns_topic_arn": "arn:aws:sns:region:123456789012:testTopic",  

 

 "message": "Test message",  

 

 },  

 

 None,  

 

 )  

 

 assert response["statusCode"] == 200  

 

 assert "Notification sent successfully" in response["body"]  

 

  

 

 # Test to check how the functions cope with SNS errors  

 

 @patch("main.boto3.client")  

 

 def test_sns_error(self, mock_boto3_client):  

 

 mock_sns_client = boto3.client("sns")  

 

 mock_boto3_client.return_value = mock_sns_client  

 

 mock_sns_client.publish.side_effect = boto3.exceptions.Boto3Error  

 

  

 

  response = lambda_handler(  

 

 {  

 

 "sns_topic_arn": "arn:aws:sns:region:123456789012:testTopic",  

 

 "message": "Test message",  

 

 },  

 

 None,  

 

 )  

 

 

 assert response["statusCode"] == 500  

How to interpret tests

  • test_missing_data: We check if the function responds correctly when we do not provide the required data. We expect a response with error code 400, meaning the query was invalid.  
  • test_send_notification_success: Here we simulate a situation in which everything goes successfully (the so-called “happy path”). We use mock_boto3_client to simulate the interaction with SNS, and then check if the function returns a success code of 200 and the corresponding message.  
  • test_sns_error: We test how the function will behave when the SNS reports an error. Again, we simulate interactions with the SNS and check if the function handles the error correctly, returning a status code of 500. 

Remember, these tests are just examples. You may want to test more scenarios to make sure a function works as it should in all possible situations. Unit tests can greatly improve the quality and reliability of your code, so it’s worth paying proper attention to them! 

FAQ – frequently asked questions

What and how should we test in AWS Lambda functions?  

Testing AWS Lambda functions should cover all key aspects, such as business logic, error handling, interactions with AWS services, and performance. It’s also a good idea to test functions with different inputs to make sure they behave correctly in various scenarios.  

Why should you test your AWS Lambda functions?  

Testing AWS Lambda functions is key to maintaining high-quality code and application functionality. Through unit testing, developers can quickly identify and eliminate errors, resulting in more reliable and efficient applications. 

AWS cloud function testing in Python – summary  

Unit testing of AWS Lambda functions with PyTest is more than just a code validation process! It’s also an opportunity to develop more stable, reliable, and efficient applications. Practical usages of unit testing include error handling, input validation, integration with AWS services, performance optimization, and fault tolerance testing. Advanced techniques, such as using PyTest fixtures and test parameterization, allow you to keep even tighter control of code quality. Unit testing is not only a pre-implementation step, but also a key part of continuous application improvement in the dynamic world of the cloud.  

I hope this article has helped you understand how we can use unit testing and AWS service mocking to ensure the reliability and correctness of our AWS Lambda functions.

"Passionate about cloud technologies, especially AWS, and a Python programming enthusiast. His experience and passion for creating scalable and innovative cloud solutions mean that he constantly desires to take on new challenges. Outside of work, he is interested in sports, personal development, and is also a car enthusiast."

Exclusive Content Awaits!

Dive deep into our special resources and insights. Subscribe to our newsletter now and stay ahead of the curve.

Information on the processing of personal data

Exclusive Content Awaits!

Dive deep into our special resources and insights. Subscribe to our newsletter now and stay ahead of the curve.

Information on the processing of personal data

Subscribe to our newsletter to unlock this file

Dive deep into our special resources and insights. Subscribe now and stay ahead of the curve – Exclusive Content Awaits

Information on the processing of personal data

Almost There!

We’ve sent a verification email to your address. Please click on the confirmation link inside to enjoy our latest updates.

If there is no message in your inbox within 5 minutes then also check your *spam* folder.

Already Part of the Crew!

Looks like you’re already subscribed to our newsletter. Stay tuned for the latest updates!

Oops, Something Went Wrong!

We encountered an unexpected error while processing your request. Please try again later or contact our support team for assistance.

    Get notified about new articles

    Be a part of something more than just newsletter

    I hereby agree that Inetum Polska Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as: my full name, e-mail address, telephone number and Skype ID/name for commercial purposes.

    I hereby agree that Inetum Polska Sp. z o.o. shall process my personal data (hereinafter ‘personal data’), such as: my full name, e-mail address and telephone number for marketing purposes.

    Read more

    Just one click away!

    We've sent you an email containing a confirmation link. Please open your inbox and finalize your subscription there to receive your e-book copy.

    Note: If you don't see that email in your inbox shortly, check your spam folder.