DocTest unittest discovery adapter

[villanyibalint]

I have looked at previous issues and feature requests but have found no mention of this problem. Please let me know if I missed something or have made a mistake below.

Behaviour

When using both unittest tests and doctests in a project the official python docs recommend adapting the doctests into the unittest discovery process, as you can see here:

import unittest
import doctest
import my_module_with_doctests

def load_tests(loader, tests, ignore):
    tests.addTests(doctest.DocTestSuite(my_module_with_doctests))
    return tests

This allows us to run doctests systematically along with regular unit tests.

Under the hood this adapter creates doctest.DocTestCase instances, a subclass of unittest.TestCase. This is not a publicly documented class, but exploring the source we can see that it overrides the id() method to point to the underlying doctest.DocTest instance's name attribute, which in turn is the qualified name of the object that contains the doctest (could be a module, a class or a function).

In vscode-python's unittest adapter script unittest.TestCase.id() is used to build the test tree. This means that doctests discovered in the way described above are displayed "incorrectly" in the test explorer tab:

• Since the script expects a pattern of *folders, filename, class_name, function_name in the id, doctest trees are only correct if the doctest is on a member function of a class. Module and class level doctests have a tree view where a random component of their module name is labeled .py. For example a doctest on a class with a qualified name of some.path.to.module.SomeClass would have a tree of

  some
  └── path
      └── to.py
          └── module
              └── SomeClass
  

• You (usually) cannot open the original source file by double clicking the test, as:
  • This could only work if the doctest is on a member function of a class (as mentioned above)
  • Doesn't work if the module name of the doctest source object doesn't conform to its location in the repository. This can happen if you use a src layout.

To be clear all other functionality of these tests work, they can be run from the test explorer and they report their results correctly. Only the tree and Go to test don't work.

Solution

I think a minor change of creating a special case for doctest.DocTestCase instances in build_test_tree could solve all of these issues. As mentioned doctest.DocTestCase instances have a _dt_test attribute which holds a reference to the underlying doctest.DocTest. This has both a filename and lineno attribute and all necessary information is present to build a correct tree.

I have not researched how the gutter icon is displayed in the editor, but could such a change also automatically fix those?

I'd be happy to take a crack at a PR if possible.

[eleanorjboyd]

Hi! Thank you for your careful and thorough investigation! Your suggested solution is right on! To give you a bit more context on how things are displayed in the editor we use the vscode testing api which handles all displaying in the correct places. This means that the filename and lineno created in build_test_tree is returned to the resultResolver.ts file which then build the test items here. These are vscode interfaces that are then used in vscode-core to render correctly.

Long story short- yes if you fix the filename and lineno there it will resolve the issue! A PR would be great since I can't promise when I could get around to this.

Thank you and I look forward to working with you on this PR! (just FYI I will be out 12/18 - 1/6 so apologies as I wont respond till the new year)