Testing Django on steroids with Pytest

I love Django and I love to create web projects that don’t need high performance. It’s a very convenient way to create an admin interface for your data or to create a REST API. I’d been working with Django until 2015 and testing was the most hated part of it. Several months ago I started to use Django again and I took some time for research and I found pytest + tox runners. Pytest framework makes it easy to write functional tests and tox is a tool for testing with the aim to automate and standardize testing in Python.

Why is writing automated tests for your application so important?

Copy link

1. You do not waste your time to run manual tests;
2. Design of your code becomes better;
3. You prove with a working test that your code works as expected;
4. You can make changes in your code without worrying if something is broken or you can break it;
5. It’s easy to refactor your code with tests;

Django’s official documentation has a lot of information about testing.

Following the documentation, you can find an example for unittest library, like this one:

I’m not a big fan of this way to write tests with many lines of code. My teammates copy and paste tests with small modifications of them. Plus the standard test runner is too slow and sometimes 1500 tests can take up to 20 minutes to run.

Pytest, on the other hand, has this example on the main page of the documentation:

def inc(x):
    return x + 1
def test_answer():
    assert inc(3) == 5

It looks amazing at first glance. Let’s try to use it in our project. Let’s make a simple project:

1. An API endpoint for returning list of users
2. An API endpoint for weather forecast

Run this to bootstrap the project:

$ pipenv install Django requests django-environ
$ pipenv shell
$ django-admin startproject example
$ django-admin startapp api

The default Django user model will be used for users and openweathermap.org for a weather forecast. Check this guide on how to get the API key for openweathermap.org. Let’s design our API methods

1. GET /api/users will return the list of users. For example, it doesn’t need any query parameters. The name for the endpoint will be fetch-users
2. GET /api/forecast will return weather forecast proxying it from openweathermap.org. city parameter required to get a forecast for a city. The name for endpoint will be fetch-forecast

The fetch-forecastendpoint needs a Django form to work with the query string. The form is simple:

The view is even simpler:

How to write tests with Pytest

Copy link

For the Django, the framework Pytest has pytest-django or pytest-djangoapp with convenient fixtures. To install it run this command:

pipenv install -d pytest pytest-django tox==3.7.0

We installed the 3.7.0 version of Tox because after 3.8 there is an error with virtualenv. Tox will be used as tests runner because:

1. It creates a virtualenv to run tests for each of the environments;
2. It’s convenient to use in CI, because of Tox automation.

Tests Structure

Copy link

Here is the folder structure in the project for testing:

|-- tests
|   `-- api # Tests for specific application goes here
|       `-- test_views.py
`-- tox.ini

Tests folder is for all our tests created in the root folder of the project. tox.ini file stands for tox test runner configuration.

Okay, fetch-users view interacts with the database, thus we need to mark the test with @pytest.mark.django_db decorator to allow the test to work with the database.

Let’s run it with Pytest:

$ pytest tests
====================== test session starts =========================
platform linux -- Python 3.8.0, pytest-5.3.1, py-1.8.0, 
pluggy-0.13.1
Django settings: example.settings (from ini file)
rootdir: 
/home/gen1us2k/work/src/github.com/gen1us2k/django_pytest_example, 
inifile: tox.ini
plugins: django-3.7.0
collected 1 item
tests/api/test_views.py .                                                                                                                                                      [100%]
====================== 1 passed in 0.28s ==========================

Okay. Let’s configure Tox and run it:

$ cat tox.ini
[tox]
envlist = tests
skipsdist = True

[testenv]
# install pytest in the virtualenv where commands will be executed
whitelist_externals = pytest
commands = pytest tests

[pytest]
DJANGO_SETTINGS_MODULE=example.settings

Tox output:

$ tox
tests run-test-pre: PYTHONHASHSEED='4095331360'
tests runtests: commands[0] | pytest tests
====================== test session starts =========================
platform linux -- Python 3.8.0, pytest-5.3.1, py-1.8.0, pluggy-0.13.1
cachedir: .tox/tests/.pytest_cache
Django settings: example.settings (from ini file)
rootdir: 
/home/gen1us2k/work/src/github.com/gen1us2k/django_pytest_example, 
inifile: tox.ini
plugins: django-3.7.0
collected 1 item
tests/api/test_views.py .                                                                                                                                                      [100%]

======================== 1 passed in 0.23s =========================
___________________________ summary ________________________________
  tests: commands succeeded
  congratulations :)

So. Pretty easy, huh? Pytest has adjango_user_modelfixture that provides access to the User model. Yes, for custom user model too. The test with the created user will look like:

As you can see, tests are written with pytest look way shorter than standard Django’s unittests. It’s amazing, isn’t it? We don’t need to copy and paste test cases from app to app, and we can focus on writing the logic of tests, instead of spending time on setup/teardown and creating an environment for tests.

Mocking Data with Pytest

Copy link

Mocking of external API is an emulation. Without it, you’ll depend on network and API availability. Yet, tests without mocking of data run slower and you’ll get feedback slower too. Quick feedback matters, because you can find errors and bugs in your code faster and fix it.

As far as you know we have fetch-forecast API endpoint which uses external API. Here’s a simple test without mocking external API.

Pytest has great integration with VCR which allows you automatically mock any HTTP API. It can be an external resource, ElasticSearch API, API of your microservice. Let’s install it.

pipenv install pytest-vcr

All you need is to add @pytest.mark.vcr() to the test, you want to mock data for. On the next test run, it will create a cassette with required data stored in tests/api/cassettes/test_fetch_forecast.yaml

The code of the test with VCR

Conclusion

Copy link

Writing functional tests with Pytest can be a great time-saver for you. The full code of the example is hosted on Github. Happy testing!

Read about Multiple Django Media Storage or my first open source experience.