In a real situation, we will have numerous test files and each file will have various tests. Tests will cover different modules and functionalities. Assume, we need to run just a particular set of tests; how would we go about it?
Let’s create a sample.py file which is having the following code:
def add(a, b=5): return a + b def mul(a, b=5): return a * b
Now, let’s create the file which will perform testing with the name of test_sample.py with the following code:
import sample def test_add(): assert sample.add(7, 3) == 10 assert sample.add(10) == 15 def test_mul(): assert sample.mul(5, 5) == 25 assert sample.mul(5) != 525 def test_add_strings(): result = sample.add('Hello', ' World') assert result == 'Hello World' assert type(result) is str assert 'Welcome' not in result def test_mul_strings(): assert sample.mul('Hello ', 3) == 'Hello Hello Hello ' result = sample.mul('Hello ') assert result == 'Hello Hello Hello Hello Hello ' assert type(result) is str assert 'Hello' in result
Pytest provides two different ways to run the subset of the test suite.
- Select tests to run dependent on substring matching of test names.
- Select tests groups to run based on the markers applied.
Let’s try to execute test_add() function:
pytest test_sample.py::test_add -v
- Suppose you want to execute the tests which are having the add function, the type the following command on cmd:
pytest -v -k "add"
- We can see that 2 passed and 2 deselected. The passed tests are those which had the add function and deselected the remaining functions.
- Suppose you want to execute tests which will have add or string function, then do the following:
pytest -v -k "add or string"
- We can see it tested all functions which had add or string function. Hence it passed 3 tests and deselected the remaining.
- Similarly, we can use the and function which will test function having add and string both:
pytest -v -k "add and string"
Pytest marking
- We can use markers to organize tests into units.
So, to check how markers work make the following changes in your test_sample.py file:
import sample import pytest @pytest.mark.numb def test_add(): assert sample.add(7, 3) == 10 assert sample.add(10) == 15 @pytest.mark.numb def test_mul(): assert sample.mul(5, 5) == 25 assert sample.mul(5) != 525 @pytest.mark.str def test_add_strings(): result = sample.add('Hello', ' World') assert result == 'Hello World' assert type(result) is str assert 'Welcome' not in result @pytest.mark.str def test_mul_strings(): assert sample.mul('Hello ', 3) == 'Hello Hello Hello ' result = sample.mul('Hello ') assert result == 'Hello Hello Hello Hello Hello ' assert type(result) is str assert 'Hello' in result
- Suppose we want to execute “numb” marker, so we need to type the following:
- Suppose you want to check failure assertation(make sure you make changes in your assert so that test cases fail), run the following command:
pytest -v -x
- -x signifies stops after 1 failure
- Suppose you want to test for maximum 2 failed test cases, the code is as follows:
pytest -v -x --maxfail=2
- We can see all the test cases are performed since, we had only one fail test test cases, if there would have have been more than 2 failed test cases it would have stopped then and there.
Pytest skip
- With the skip decorator, we can skip the specified tests. There are multiple reasons for skipping test; for instance, a database/online service is not available at the moment or we skip Linux specific tests on Windows.
- Make the following changes in your test_sample.py file:
import sample import pytest import sys @pytest.mark.skip(reason="do not run number addd test") def test_add(): assert sample.add(7, 3) == 10 assert sample.add(10) == 15 def test_mul(): assert sample.mul(5, 5) == 25 assert sample.mul(5) == 25 def test_add_strings(): result = sample.add('Hello', ' World') assert result == 'Hello World' assert type(result) is str assert 'Welcome' not in result def test_mul_strings(): assert sample.mul('Hello ', 3) == 'Hello Hello Hello ' result = sample.mul('Hello ') assert result == 'Hello Hello Hello Hello Hello ' assert type(result) is str assert 'Hello' in result
- Run using the following command:
pytest -v
- We can see that the add function is skipped and the remaining functions are tested.
- If you want to show the reason for skips in verbose mode on the terminal, you can pass -rsx to report skipped tests. For example:
pytest -v -rsx
- Similarly you can skip a test using a if condition, make the following changes:
import sample import pytest import sys @pytest.mark.skipif(sys.version_info < (3,3), reason="do not run number add test") def test_add(): assert sample.add(7, 3) == 10 assert sample.add(10) == 15 def test_mul(): assert sample.mul(5, 5) == 25 assert sample.mul(5) == 25 def test_add_strings(): result = sample.add('Hello', ' World') assert result == 'Hello World' assert type(result) is str assert 'Welcome' not in result def test_mul_strings(): assert sample.mul('Hello ', 3) == 'Hello Hello Hello ' result = sample.mul('Hello ') assert result == 'Hello Hello Hello Hello Hello ' assert type(result) is str assert 'Hello' in result
- The add test will get skipped if the Python version is less than 3.3
- Now if you want to to print a certain statement make the following changes in your test_sample.py file:
import sample import pytest import sys @pytest.mark.skipif(sys.version_info < (3,3), reason="do not run number add test") def test_add(): assert sample.add(7, 3) == 10 assert sample.add(10) == 15 print(sample.add(10, 10)), '------------------------' def test_mul(): assert sample.mul(5, 5) == 25 assert sample.mul(5) == 25 def test_add_strings(): result = sample.add('Hello', ' World') assert result == 'Hello World' assert type(result) is str assert 'Welcome' not in result def test_mul_strings(): assert sample.mul('Hello ', 3) == 'Hello Hello Hello ' result = sample.mul('Hello ') assert result == 'Hello Hello Hello Hello Hello ' assert type(result) is str assert 'Hello' in result
- Execute using the following command:
pytest -v -s