Python Unit Testing Using Options with pytest

In a real situation, we will have numerous test files and each file will have various tests. Tests will cover different modules and functionalities. Assume, we need to run just a particular set of tests; how would we go about it?

Let’s create a sample.py file which is having the following code:

def add(a, b=5):
    return a + b


def mul(a, b=5):
    return a * b

Now, let’s create the file which will perform testing with the name of test_sample.py with the following code:

import sample

def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) != 525


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result

Pytest provides two different ways to run the subset of the test suite.

  • Select tests to run dependent on substring matching of test names.
  • Select tests groups to run based on the markers applied.

Let’s try to execute test_add() function:

pytest test_sample.py::test_add -v

  • ¬†Suppose you want to execute the tests which are having the add function, the type the following command on cmd:
pytest -v -k "add"

  • We can see that 2 passed and 2 deselected. The passed tests are those which had the add function and deselected the remaining functions.
  • Suppose you want to execute tests which will have add or string function, then do the following:
pytest -v -k "add or string"

  • We can see it tested all functions which had add or string function. Hence it passed 3 tests and deselected the remaining.
  • Similarly, we can use the and function which will test function having add and string both:
pytest -v -k "add and string"

Pytest marking

  • We can use markers to organize tests into units.

So, to check how markers work make the following changes in your test_sample.py file:

import sample
import pytest

@pytest.mark.numb
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15

@pytest.mark.numb
def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) != 525

@pytest.mark.str
def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result

@pytest.mark.str
def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result
  • Suppose we want to execute “numb” marker, so we need to type the following:

  • Suppose you want to check failure assertation(make sure you make changes in your assert so that test cases fail), run the following command:
pytest -v -x
  • -x signifies stops after 1 failure

  • Suppose you want to test for maximum 2 failed test cases, the code is as follows:
pytest -v -x --maxfail=2

  • We can see all the test cases are performed since, we had only one fail test test cases, if there would have have been more than 2 failed test cases it would have stopped then and there.

Pytest skip

  • With the skip decorator, we can skip the specified tests. There are multiple reasons for skipping test; for instance, a database/online service is not available at the moment or we skip Linux specific tests on Windows.
  • Make the following changes in your test_sample.py file:
import sample
import pytest
import sys

@pytest.mark.skip(reason="do not run number addd test")
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) == 25


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result
  • Run using the following command:
pytest -v

  • We can see that the add function is skipped and the remaining functions are tested.
  • If you want to show the reason for skips in verbose mode on the terminal, you can pass -rsx to report skipped tests. For example:
pytest -v -rsx

  • Similarly you can skip a test using a if condition, make the following changes:
import sample
import pytest
import sys

@pytest.mark.skipif(sys.version_info < (3,3), reason="do not run number add test")
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) == 25


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result
  • The add test will get skipped if the Python version is less than 3.3

  • Now if you want to to print a certain statement make the following changes in your test_sample.py file:
import sample
import pytest
import sys

@pytest.mark.skipif(sys.version_info < (3,3), reason="do not run number add test")
def test_add():
    assert sample.add(7, 3) == 10
    assert sample.add(10) == 15
    print(sample.add(10, 10)), '------------------------'


def test_mul():
    assert sample.mul(5, 5) == 25
    assert sample.mul(5) == 25


def test_add_strings():
    result = sample.add('Hello', ' World')
    assert result == 'Hello World'
    assert type(result) is str
    assert 'Welcome' not in result


def test_mul_strings():
    assert sample.mul('Hello ', 3) == 'Hello Hello Hello '
    result = sample.mul('Hello ')
    assert result == 'Hello Hello Hello Hello Hello '
    assert type(result) is str
    assert 'Hello' in result
  • Execute using the following command:
pytest -v -s