Pytest-BDD-NG Tutorial #2

In the first installment of this tutorial series we learned how Pytest-BDD-NG improved on Behave and Pytest-BDD, and we began to write and automate requirements for searching a library catalog. In this installment we will add the supporting classes and methods to fully illustrate catalog searches. We will also spend considerable time explaining how to properly pass state between steps. Proper management of state is critical for reliable tests.

Step 1 – Create a Python Virtual Environment

The first thing to do is to create a virtual environment; doing this will keep Pytest-BDD-NG and all of its dependencies separate from all of your other Python projects. I like the Real Python tutorial explaining how to do this.

Step 2 – Install Pytest-BDD-NG

Installation is easy; activate your new virtual environment and then run:

pip install pytest-bdd-ng

Step 3 – Properly Pass State

In the first installment of this tutorial we created a context object like this:

@fixture
def context():
    """
    Create a placeholder object to use in place of Cucumber's context object.
    The context object allows us to pass state between steps.
    CAUTION: DO NOT USE THIS FIXTURE unless you also write the code to
    Destroy and recreate it for every scenario. Failure to do this may 
    result in scenarios that depend on order of execution; this is a Bad 
    Thing.
    """
    class dummy():
        pass

    return dummy()

This is a really simple way to pass state from Given to When to Then, but we can’t use it in the real world. This context object never gets deleted and recreated, so it could pass state, not just from one step to the next, but from one Scenario to the next. This is a Bad Thing; it would mean that the outcome of a scenario could change depending on which scenarios were run previously. The outcome could also change depending on whether the previous scenarios passed or failed. Flickering tests cause everyone to lose confidence in our testing, so we can’t use this context object.

Pytest-BDD-NG has a solution for this; we can create fixtures for the variables that we need to pass between steps, and inject those fixtures into the step definition. Fixtures that are injected into a step definition exist for the life of the scenario, so every following step has access to them. However, all of these fixtures are destroyed at the end of the scenario, so they can’t accidentally pass state from one Scenario to the next.

What state do we need to pass in our library requirement? This is the only requirement we have written:

Feature: Library book searches and book delivery

Scenario: The catalog can be searched by author name.
    Given these books in the catalog
    | Author          | Title                       |
    | Stephen King    | The Shining                 |
    | James Baldwin   | If Beale Street Could Talk  |
    When a name search is performed for Stephen
    Then only these books will be returned
    | Author          | Title                       |
    | Stephen King    | The Shining                 |

The Given step will have to pass the catalog into the When step so we can perform the search. The When step will have to pass the search results to the Then step so we can compare the search results to the expected results. We can pass these two pieces of state information if we create these two fixtures:

@fixture
def catalog():
    """
    Create an empty catalog class instance.
    """
    return Catalog()


@fixture
def search_results():
    """
    Create an empty list of search results.
    """
    return []

Now we must inject the catalog fixture into the Given step definition:

@given("these books in the catalog")
def these_books_in_the_catalog(step: PickleStep, request: FixtureRequest):
    catalog = request.getfixturevalue('catalog')
    catalog.add_books_to_catalog(step.data_table)

Adding the ‘request’ fixture to the parameters for our Given step definition allowed us to inject the ‘catalog’ fixture. In the When step we will need the catalog fixture and also the search_results fixture, in order to save the results and pass them to the Then step:

@when(parsers.re("a (?P<search_type>name|title) search is performed for " +
                 "(?P<search_term>.+)"))
def a_SEARCH_TYPE_is_performed_for_SEARCH_TERM(search_type: str, search_term: str, request):
    if search_type == "title":
        raise NotImplementedError("Title searches are not yet implemented.")
    catalog = request.getfixturevalue('catalog')
    search_results = request.getfixturevalue('search_results')
    search_results.append(catalog.search_by_author(search_term))

These fixtures will be destroyed at the end of the scenario, so there is no risk of passing state to the next scenario. Now our complete step definition file looks like this:

from pytest import fixture, FixtureRequest
from pytest_bdd import given, parsers, then, when
from pytest_bdd.model import PickleStep

from helper_methods.library_catalog import Catalog
from helper_methods.verification_helper_methods import verify_returned_books


@fixture
def catalog():
    """
    Create an empty catalog fixture.
    """
    return Catalog()


@fixture
def search_results():
    """
    Create an empty list of search results.
    """
    return []


@given("these books in the catalog")
def these_books_in_the_catalog(step: PickleStep, request: FixtureRequest):
    catalog = request.getfixturevalue('catalog')
    catalog.add_books_to_catalog(step.data_table)

@when(parsers.re("a (?P<search_type>name|title) search is performed for " +
                 "(?P<search_term>.+)"))
def a_SEARCH_TYPE_is_performed_for_SEARCH_TERM(search_type: str, 
						 search_term: str, 
    						 request: FixtureRequest):
    if search_type == "title":
        raise NotImplementedError("Title searches are not yet implemented.")
    catalog = request.getfixturevalue('catalog')
    search_results = request.getfixturevalue('search_results')
    search_results.append(catalog.search_by_author(search_term))


@then("only these books will be returned")
def only_these_books_will_be_returned(step: PickleStep,
					request: FixtureRequest):
    catalog = request.getfixturevalue('catalog')
    search_results = request.getfixturevalue('search_results')
    expected_books = []
    expected_books.append(catalog.read_books_from_table(step.data_table))
    verify_returned_books(search_results, expected_books)

I put the step definitions in tests/steps/library_steps.py.

Step 4 – Write the ‘Catalog’ Class

 The Catalog class will need a way to populate the catalog from the ‘Given’ data table, and a way to search the catalog by author’s name. We need to write these methods:

  • add_books_to_catalog
  • search_by_author

The add_books_to_catalog method must first read the books from the data table – but that method will also be needed in the ‘Then’ statement, so we should make it a separate method. That makes add_books_to_catalog pretty simple:

def add_books_to_catalog(self, table_of_books):
    """
    Read the author names and titles from the data table_of_books,
    and add them to the catalog
    """
    books = self.read_books_from_table(table_of_books)
    self._catalog = self._catalog + books

def read_books_from_table(table_of_books):
    """
    Read all of the authors and titles from a data table. The data table has
    this format:
    | Author                | Title         |
    | first author name     | first title   |
    | second author name    | second title  |
    .
    .
    .

    """
    books = []
    FIRST_ROW = 0
    FIRST_COLUMN = 0
    if table_of_books.rows[FIRST_ROW].cells[FIRST_COLUMN].value == "Author":
        # If the first column is 'Author', assume that the second column is
        # 'Title'. Of course this is poor programming, but it is adequate for
        # this simple example.
        author_column = 0
        title_column = 1
    else:
        # If the first column is NOT 'Author', assume that it is'Title' and
        # the second column is 'Author'.
        author_column = 1
        title_column = 0
    # Now read each row, starting with row 1 (the first row containing an
    # author’s name and book title).
    last_row = len(table_of_books.rows) - 1
    this_row = 1
    while this_row <= last_row:
        author = table_of_books.rows[this_row].cells[author_column].value
        title = table_of_books.rows[this_row].cells[title_column].value
        books.append((author, title))
        this_row = this_row + 1
    return books

I put this code in tests/Features/helper_methods /library_catalog.py. The code above also needs these few lines to define the class:

class Catalog(object):
    """
    Define a class that represents a library catalog, with book titles,
    authors, and searches by title or author.
    """
    
    def __init__(self):
        # Store the catalog as a list of tuples.
        self._catalog = []

Step 5 – Write a Verification Function

We have written step definitions for all three lines of Gherkin in our requirement, and we have automated the Given and When steps. The only thing remaining is to automate the Then step – verifying the results returned from the search. That is only slightly tricky – we must verify two things:

  1. All of the expected books were returned
  2. No unexpected books were returned

We could write this as a method of the Catalog class, but it doesn’t seem to be an inherent part of a catalog search system. Because of that I have written it in a separate file. Here is our verification code:

def verify_returned_books(actual_books, expected_books):
    """
    Verify that the set of actual books matches the expected books.
    """
    # If no books were expected and no books were returned, then verification
    # passes.
    if len(actual_books) == 0 and len(expected_books) == 0:
        return
    for author_and_title in actual_books:
        assert (author_and_title in expected_books), \
                f"The books returned don't match what was expected." \
                f"\nThis book: '{author_and_title}' was not expected, but "\
                f"was returned."
    # All of the books returned by the search were expected, but we don't yet
    # know that all of the expected books were returned. If actual_books has
    # the same number of entries as expected_books, then all of the expected
    # books were returned.
    if not len(actual_books) == len(expected_books):
        # There is at least one expected book that wasn't returned; find the
        # first one.
        for author_and_title in expected_books:
            assert (author_and_title in actual_books), \
                    f"The books returned don't match what was expected." \
                    f"\nThis book: '{author_and_title}' was expected, but “\
                    f”wasn't returned."

It is important to note that, although this entire tutorial is about Pytest-BDD-NG, we run it simply by typing ‘pytest’ – not ‘pytest-bdd-ng’:

% pytest Features/library_catalog_searches.feature 
=========================== test session starts ===========================
platform darwin -- Python 3.11.5, pytest-7.4.4, pluggy-1.3.0
rootdir: /Users/leslie/Development/pytest-bdd-ng/tests
plugins: bdd-ng-2.1.3
collected 1 item                                                                                                                                                              

Features/library_catalog_searches.feature .                                                                                                                             [100%]

=========================== 1 passed in 0.01s ===========================

This doesn’t look like the standard output from Cucumber or Behave; if you prefer the more verbose results you can add ‘–gherkin-terminal-reporter -vv’:

 

Step 7 – Test Your Test Code!

When the requirement passes, that doesn’t mean that we are done. How do we know that our code doesn’t always declare that the scenario passed? Declaring a Pass when the library search actually failed is a Really Bad Thing – so we should test that. In this case we should test two different ways:

  1. Change the When statement in the scenario so that nothing in the catalog matches the search. For example, change the When to search for an author’s name of ‘Steve’. Then the search will return nothing, but the Then statement will expect ‘Stephen King’ to be returned. Our code should report that an expected book wasn’t returned.
  2. Change the When statement back to ‘Stephen’, and add a second book to the data table in the Then statement. Our code should report that an expected book wasn’t returned.

When these tests both pass (i.e., fail as we expect them to) then we are done!

In the next tutorial in this series we will show how to use the Cucumber Expressions parser, and how to convert the parameters in the Gherkin into Enums. Using Enums makes our code cleaner and easier to modify and extend.

Leave a Comment