[Python] Unexpected PyTest test failure with mock side-effects

I'm trying to generate come C code based on other C code that I'm parsing with libclang.

The code under test (TypeWriter methods write and _write_struct) is supposed to take a structure node from the abstract syntax tree (AST) and print a C structure definition to the a comment (if one was detected and the structure element. These appear as children in the AST. I'm using the pytest test function test_write_struct to create a mock AST node and feed it to the type writer which should then write it to the file.

The code looks something like:

import logging

from clang import cindex

import pytest
from unittest import mock


class TypeWriter:
def __init__(self, types_file):
self._file = types_file
self._space = self._file.TAB

self._log = logging.getLogger(self.__class__.__name__)

self._writers = {cindex.CursorKind.STRUCT_DECL: self._write_struct}

def write(self, node: cindex.Cursor):
if node.brief_comment:
self._file.writeln(f'// {node.brief_comment}')
try:
self._writers[node.kind](node)
except KeyError as exc:
self._log.error(f'Unknown datatype {exc.args[0]}')

def _write_struct(self, node: cindex.Cursor):
self._log.info(f'Handling Struct node {node.displayname}')
lines = [f'struct {node.displayname} {{']
for elem in node.get_children():
if elem.brief_comment:
lines.append(f'{self._space * 2}// {elem.brief_comment}')
lines.append(f'{self._space * 2}{elem.type.spelling} {elem.displayname};')
lines.append(f'')
self._file.write_lines(lines)


@pytest.fixture
def mock_header(mocker):
#return mocker.Mock(spec_set=SourceFile)
return mocker.Mock()


@pytest.fixture
def mock_node(mocker):
ret = mocker.Mock(get_children=mocker.Mock())
ret.brief_comment = None
return ret


@pytest.fixture
def mock_struct_node(mock_node):
mock_node.kind = cindex.CursorKind.STRUCT_DECL
return mock_node


def test_write_struct(mock_struct_node, mock_header):
writer = TypeWriter(mock_header)

mock_struct_node.get_children.side_effect = [
mock.Mock(type=mock.PropertyMock(spelling=f'type_1'), displayname=f'property_1'),
mock.Mock(type=mock.PropertyMock(spelling=f'type_2'), displayname=f'property_2', brief_comment='hello world'),
mock.Mock(type=mock.PropertyMock(spelling=f'type_3'), displayname=f'property_3')
]
writer.write(mock_struct_node)

mock_header.write_lines.assert_called_once()
assert 1 == len(mock_header.write_lines.call_args)

lines = mock_header.write_lines.call_args.args[0]
curr_line = 0
if getattr(mock_struct_node, 'brief_comment') is not None:
assert '// {mock_struct_node.brief_comment}' in lines[curr_line]
curr_line += 1

assert f'struct {mock_struct_node.displayname} {{' in lines[curr_line]
curr_line += 1

for element in elements:
if element.brief_comment is not None:
assert '// {element.brief_comment}' in lines[curr_line]
curr_line += 1
assert f'{element.type.spelling} {element.displayname};' in lines[curr_line]
curr_line += 1


if __name__ == '__main__':
pytest.main()


The test is failing in TypeWriter._write_struct with an error:

self._log.info(f'Handling Struct node {node.displayname}')
lines = [f'struct {node.displayname} {{']
> for elem in node.get_children():
E TypeError: 'Mock' object is not iterable


I have been trying to work this out in the debugger. When I break on the line, I can see that node.get_children.side_effect is a list iterator:

>>> node.get_children.side_effect
<list_iterator object at 0x...>


I can enumerate this and it appears valid.

Stepping into this line sends me into unittest.Mock:

• __call__
  • _mock_call
  • _execute_mock_call
    • effect is my list iterator
    • next(effect) returns a mock object which is returned and passed back up the stack

When I get back to the line for elem in node.get_children(): in the debugger, I expect elem to be set to my mock node when I step again, and for the test to continue. Unfortunately, when I step into (or over), I move to _pytest.stash.Stash.get and the python stack has unwound to:

• 0: <lambda>, runner.py:242
• 1: from_call, runner.py:341
• 2: call_and_report, runner.py:241
• ...

I can't see what's causing this test to crash and would really appreciate ideas on what's going wrong. It looks like an exception was thrown in the test but I can't see where it came from in the python code.

I'm running Python 3.12.3 on Ubuntu 24.04.1 LTS (noble) x86_64 and have tried running this through PyCharm and from the bash prompt with python -m pytest and pytest with the same results. I'm using the following packages:

• libclang: 18.1.1
• pytest: 8.3.4

Continue reading...