Python Unit Testing Using Options with pytest

In a real situation, we will have numerous test files and each file will have various tests. Tests will cover different modules and functionalities. Assume, we need to run just a particular set of tests; how would we go about it?

Let’s create a sample.py file which is having the following code:

def add(a, b=5):
    return a + b


def mul(a, b=5):
    return a * b

Now, let’s create the file which will perform testing with the name of test_sample.py with the following code:

import sample

def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) != 525


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result

Pytest provides two different ways to run the subset of the test suite.

Select tests to run dependent on substring matching of test names.
Select tests groups to run based on the markers applied.

Let’s try to execute test_add() function:

pytest test_sample.py::test_add -v

Suppose you want to execute the tests which are having the add function, the type the following command on cmd:

pytest -v -k "add"

We can see that 2 passed and 2 deselected. The passed tests are those which had the add function and deselected the remaining functions.
Suppose you want to execute tests which will have add or string function, then do the following:

pytest -v -k "add or string"

We can see it tested all functions which had add or string function. Hence it passed 3 tests and deselected the remaining.
Similarly, we can use the and function which will test function having add and string both:

pytest -v -k "add and string"

Pytest marking

We can use markers to organize tests into units.

So, to check how markers work make the following changes in your test_sample.py file:

import sample
import pytest

@pytest.mark.numb
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15

@pytest.mark.numb
def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) != 525

@pytest.mark.str
def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result

@pytest.mark.str
def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result

Suppose we want to execute “numb” marker, so we need to type the following:

Suppose you want to check failure assertation(make sure you make changes in your assert so that test cases fail), run the following command:

pytest -v -x

-x signifies stops after 1 failure

Suppose you want to test for maximum 2 failed test cases, the code is as follows:

pytest -v -x --maxfail=2

We can see all the test cases are performed since, we had only one fail test test cases, if there would have have been more than 2 failed test cases it would have stopped then and there.

Pytest skip

With the skip decorator, we can skip the specified tests. There are multiple reasons for skipping test; for instance, a database/online service is not available at the moment or we skip Linux specific tests on Windows.
Make the following changes in your test_sample.py file:

import sample
import pytest
import sys

@pytest.mark.skip(reason="do not run number addd test")
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) == 25


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result

Run using the following command:

pytest -v

We can see that the add function is skipped and the remaining functions are tested.
If you want to show the reason for skips in verbose mode on the terminal, you can pass -rsx to report skipped tests. For example:

pytest -v -rsx

Similarly you can skip a test using a if condition, make the following changes:

import sample
import pytest
import sys

@pytest.mark.skipif(sys.version_info < (3,3), reason="do not run number add test")
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) == 25


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result

The add test will get skipped if the Python version is less than 3.3

Now if you want to to print a certain statement make the following changes in your test_sample.py file:

import sample
import pytest
import sys

@pytest.mark.skipif(sys.version_info < (3,3), reason="do not run number add test")
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15
    print(sample.add(10, 10)), '------------------------'


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) == 25


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result

Execute using the following command:

pytest -v -s