Testing AWS Lambda Function in Python

In this Blog Article, I show some techniques to test python-based AWS Lambda Functions.

Python is a perfect match for the AWS ecosystem. The reasons are boto3 and moto - there is nothing like this in other languages.

The project is a small poetry project, which gets deployed by AWS CDK, but these are irrelevant details for this post. The code creates signed cookies to request resources from a protected CloudFront Distribution.

More details on the implementation can be found in the articles:

• Access Cloudfront with signed cookies by arrrrrmin
• Setting up S3 & CloudFront to Deliver Static Assets Across the Web by Suraj Patil

Tools to test the code are:

• pytest - the testing framework
• moto - to mock aws-services
• python-lambda-local - to run the lambda locally and perform an exploratory test against a real endpoint

Topics included:

• test and fixture setup with conftest.py
• how to create fake_time in python
• how to mock aws-services in general with the secretsmanager as an example
• how to expect an raised error

#1 test and fixture setup with conftest.py

The file tests/conftest.py should contain all fixtures which are used by the tests.

To mock AWS services, it is required to set some ENV-Vars:

@pytest.fixture(scope="function")
def aws_credentials():
    os.environ["AWS_ACCESS_KEY_ID"] = "testing"
    os.environ["AWS_SECRET_ACCESS_KEY"] = "testing"
    os.environ["AWS_SECURITY_TOKEN"] = "testing"
    os.environ["AWS_SESSION_TOKEN"] = "testing"
    os.environ["AWS_REGION"] = "eu-central-1"

This fixture can be included in your test later like this:

def test_aws_service(aws_credentials):
    # ... test code ..

But, you can not only include fixtures in your test - fixtures can also be included in other fixtures, too:

@pytest.fixture(scope="function")
def mock_aws_service(aws_credentials):
    # ... setup aws service mock ..

#2 Create a fake_time in python

There are many different approaches to implementing this, according to this StackOverflow answer.

The technique explained by Shinji Matsumoto in the answer worked best for me.

@pytest.fixture()
def fake_time(monkeypatch):
    datetime_mock = MagicMock(wraps=datetime.datetime)
    datetime_mock.now.return_value = datetime.datetime(2015, 9, 21)
    monkeypatch.setattr(datetime, "datetime", datetime_mock)

    yield datetime_mock.now()

Two things happening here:

1. Wrap datetime.datetime in MagickMock

   By using MagickMock, the return_value of this function can be overwritten.

2. Monkeypatch datetime module

   Monkeypatch is a pytest fixture to modify values and it can be used to replace the datetime.datetime implementation by datetime_mock.

At the end, the timestamp of the mocked datetime is yielded so the test can use the timestamp as input for the system under test.

#3 Mock AWS Services, with SecretsManager as an example

To mock AWS Services, we will use the excellent moto lib.

In this Example I will mock the secretsmanager create_secret method, so the system under test can call get_secret_value to get the response from the mocked service.

There are two convenient ways to mock the service with moto:

• using the mock_* function as decorator
• using the mock_* function as context manager

I will use the mock_* function as context manager:

@pytest.fixture(scope="function")
def mock_cdn_secret(aws_credentials, env_cdn_key):
    with moto.mock_secretsmanager():
        secretsmanager = boto3.client("secretsmanager", region_name="eu-central-1")
        secretsmanager.create_secret(
            Name="any-secret",
            SecretString=json.dumps(
                {
                    "private-key": """
-----BEGIN RSA PRIVATE KEY-----
✂snipsnip✂
-----END RSA PRIVATE KEY-----
                """.strip(),
                    "public-key": """
-----BEGIN PUBLIC KEY-----
✂snipsnip✂
-----END PUBLIC KEY-----
                """.strip(),
                }
            ),
        )

        yield

As you see, instead of mocking the response with a fixture, we mock the AWS service simply by using it.

One important detail: The yield on the last line is important, don’t overlook it. If you do, the mock stops before the test starts.

#4 Expect an raised Error

This is a small one. It is important to test for expected Errors.

Following test expects that a PermissionError is raised:

def test_permission_check():
    with pytest.raises(PermissionError):
        handler(
            {
                "ownerId": "d005b2de-d70c-478a-8cc2-287fcb6d3b09",
                "source": {
                    "ownerId": "a43d3b16-e77f-4014-8d47-eb3d51004c84", 
                    "state": "DOC_STATE_READY"
                },
            },
            None,
        )

Pretty straight forward!

#5 Exploratory testing by running the lambda locally

To run lambdas locally, use the tool python-lambda-local.

Have the event and optionally the env-vars in a json-file, then run the lambda like:

python-lambda-local -e tests/fixture_env.json index.py tests/fixture_query.json

If needed add your AWS_PROFILE so the lambda can access AWS resources or start local-stack for full local testing.

---

---

---