doctest —- 测试交互性的Python示例

源代码 Lib/doctest.py

doctest 模块寻找像Python交互式代码的文本，然后执行这些代码来确保它们的确就像展示的那样正确运行，有许多方法来使用doctest：

  • 通过验证所有交互式示例仍然按照记录的方式工作，以此来检查模块的文档字符串是否是最新的。

  • To perform regression testing by verifying that interactive examples from atest file or a test object work as expected.

  • To write tutorial documentation for a package, liberally illustrated withinput-output examples. Depending on whether the examples or the expository textare emphasized, this has the flavor of "literate testing" or "executabledocumentation".

下面是一个小却完整的示例模块:

  1. """
  2. This is the "example" module.
  3.  
  4. The example module supplies one function, factorial().  For example,
  5.  
  6. >>> factorial(5)
  7. 120
  8. """
  9.  
  10. def factorial(n):
  11.     """Return the factorial of n, an exact integer >= 0.
  12.  
  13.     >>> [factorial(n) for n in range(6)]
  14.     [1, 1, 2, 6, 24, 120]
  15.     >>> factorial(30)
  16.     265252859812191058636308480000000
  17.     >>> factorial(-1)
  18.     Traceback (most recent call last):
  19.         ...
  20.     ValueError: n must be >= 0
  21.  
  22.     Factorials of floats are OK, but the float must be an exact integer:
  23.     >>> factorial(30.1)
  24.     Traceback (most recent call last):
  25.         ...
  26.     ValueError: n must be exact integer
  27.     >>> factorial(30.0)
  28.     265252859812191058636308480000000
  29.  
  30.     It must also not be ridiculously large:
  31.     >>> factorial(1e100)
  32.     Traceback (most recent call last):
  33.         ...
  34.     OverflowError: n too large
  35.     """
  36.  
  37.     import math
  38.     if not n >= 0:
  39.         raise ValueError("n must be >= 0")
  40.     if math.floor(n) != n:
  41.         raise ValueError("n must be exact integer")
  42.     if n+1 == n:  # catch a value like 1e300
  43.         raise OverflowError("n too large")
  44.     result = 1
  45.     factor = 2
  46.     while factor <= n:
  47.         result *= factor
  48.         factor += 1
  49.     return result
  50.  
  51.  
  52. if __name__ == "__main__":
  53.     import doctest
  54.     doctest.testmod()

如果你直接在命令行里运行 example.py ， doctest 将发挥他的作用。

  1. $ python example.py
  2. $

There's no output! That's normal, and it means all the examples worked. Pass-v to the script, and doctest prints a detailed log of whatit's trying, and prints a summary at the end:

  1. $ python example.py -v
  2. Trying:
  3.     factorial(5)
  4. Expecting:
  5.     120
  6. ok
  7. Trying:
  8.     [factorial(n) for n in range(6)]
  9. Expecting:
  10.     [1, 1, 2, 6, 24, 120]
  11. ok

And so on, eventually ending with:

  1. Trying:
  2.     factorial(1e100)
  3. Expecting:
  4.     Traceback (most recent call last):
  5.         ...
  6.     OverflowError: n too large
  7. ok
  8. 2 items passed all tests:
  9.    1 tests in __main__
  10.    8 tests in __main__.factorial
  11. 9 tests in 2 items.
  12. 9 passed and 0 failed.
  13. Test passed.
  14. $

That's all you need to know to start making productive use of doctest!Jump in. The following sections provide full details. Note that there are manyexamples of doctests in the standard Python test suite and libraries.Especially useful examples can be found in the standard test fileLib/test/test_doctest.py.

简单用法：检查Docstrings中的示例

开始使用doctest的最简单方法（但不一定是你将继续这样做的方式）是结束每个模块 M 使用:

  1. if __name__ == "__main__":
  2.     import doctest
  3.     doctest.testmod()

doctest then examines docstrings in module M.

Running the module as a script causes the examples in the docstrings to getexecuted and verified:

  1. python M.py

This won't display anything unless an example fails, in which case the failingexample(s) and the cause(s) of the failure(s) are printed to stdout, and thefinal line of output is Test Failed N failures., where N is thenumber of examples that failed.

Run it with the -v switch instead:

  1. python M.py -v

and a detailed report of all examples tried is printed to standard output, alongwith assorted summaries at the end.

You can force verbose mode by passing verbose=True to testmod(), orprohibit it by passing verbose=False. In either of those cases,sys.argv is not examined by testmod() (so passing -v or nothas no effect).

There is also a command line shortcut for running testmod(). You caninstruct the Python interpreter to run the doctest module directly from thestandard library and pass the module name(s) on the command line:

  1. python -m doctest -v example.py

This will import example.py as a standalone module and runtestmod() on it. Note that this may not work correctly if the file ispart of a package and imports other submodules from that package.

For more information on testmod(), see section Basic API.

Simple Usage: Checking Examples in a Text File

Another simple application of doctest is testing interactive examples in a textfile. This can be done with the testfile() function:

  1. import doctest
  2. doctest.testfile("example.txt")

That short script executes and verifies any interactive Python examplescontained in the file example.txt. The file content is treated as if itwere a single giant docstring; the file doesn't need to contain a Pythonprogram! For example, perhaps example.txt contains this:

  1. The example module

Using factorial

This is an example text file in reStructuredText format. First importfactorial from the example module:

  1. &gt;&gt;&gt; from example import factorial

Now use it:

  1. &gt;&gt;&gt; factorial(6)
  2. 120

Running doctest.testfile("example.txt") then finds the error in thisdocumentation:

  1. File "./example.txt", line 14, in example.txt
  2. Failed example:
  3.     factorial(6)
  4. Expected:
  5.     120
  6. Got:
  7.     720

As with testmod(), testfile() won't display anything unless anexample fails. If an example does fail, then the failing example(s) and thecause(s) of the failure(s) are printed to stdout, using the same format astestmod().

By default, testfile() looks for files in the calling module's directory.See section Basic API for a description of the optional argumentsthat can be used to tell it to look for files in other locations.

Like testmod(), testfile()'s verbosity can be set with the-v command-line switch or with the optional keyword argumentverbose.

There is also a command line shortcut for running testfile(). You caninstruct the Python interpreter to run the doctest module directly from thestandard library and pass the file name(s) on the command line:

  1. python -m doctest -v example.txt

Because the file name does not end with .py, doctest infers thatit must be run with testfile(), not testmod().

For more information on testfile(), see section Basic API.

How It Works

This section examines in detail how doctest works: which docstrings it looks at,how it finds interactive examples, what execution context it uses, how ithandles exceptions, and how option flags can be used to control its behavior.This is the information that you need to know to write doctest examples; forinformation about actually running doctest on these examples, see the followingsections.

Which Docstrings Are Examined?

The module docstring, and all function, class and method docstrings aresearched. Objects imported into the module are not searched.

In addition, if M.test exists and "is true", it must be a dict, and eachentry maps a (string) name to a function object, class object, or string.Function and class object docstrings found from M.test are searched, andstrings are treated as if they were docstrings. In output, a key K inM.test appears with name

  1. <name of M>.__test__.K

Any classes found are recursively searched similarly, to test docstrings intheir contained methods and nested classes.

CPython implementation detail: Prior to version 3.4, extension modules written in C were not fullysearched by doctest.

How are Docstring Examples Recognized?

In most cases a copy-and-paste of an interactive console session works fine,but doctest isn't trying to do an exact emulation of any specific Python shell.

  1. >>> # comments are ignored
  2. >>> x = 12
  3. >>> x
  4. 12
  5. >>> if x == 13:
  6. ...     print("yes")
  7. ... else:
  8. ...     print("no")
  9. ...     print("NO")
  10. ...     print("NO!!!")
  11. ...
  12. no
  13. NO
  14. NO!!!
  15. >>>

Any expected output must immediately follow the final '>>> ' or '… 'line containing the code, and the expected output (if any) extends to the next'>>> ' or all-whitespace line.

The fine print:

  • Expected output cannot contain an all-whitespace line, since such a line istaken to signal the end of expected output. If expected output does contain ablank line, put <BLANKLINE> in your doctest example each place a blank lineis expected.

  • All hard tab characters are expanded to spaces, using 8-column tab stops.Tabs in output generated by the tested code are not modified. Because anyhard tabs in the sample output are expanded, this means that if the codeoutput includes hard tabs, the only way the doctest can pass is if theNORMALIZE_WHITESPACE option or directiveis in effect.Alternatively, the test can be rewritten to capture the output and compare itto an expected value as part of the test. This handling of tabs in thesource was arrived at through trial and error, and has proven to be the leasterror prone way of handling them. It is possible to use a differentalgorithm for handling tabs by writing a custom DocTestParser class.

  • Output to stdout is captured, but not output to stderr (exception tracebacksare captured via a different means).

  • If you continue a line via backslashing in an interactive session, or for anyother reason use a backslash, you should use a raw docstring, which willpreserve your backslashes exactly as you type them:

  1. >>> def f(x):
  2. ...     r'''Backslashes in a raw docstring: m\n'''
  3. >>> print(f.__doc__)
  4. Backslashes in a raw docstring: m\n

Otherwise, the backslash will be interpreted as part of the string. For example,the \n above would be interpreted as a newline character. Alternatively, youcan double each backslash in the doctest version (and not use a raw string):

  1. >>> def f(x):
  2. ...     '''Backslashes in a raw docstring: m\\n'''
  3. >>> print(f.__doc__)
  4. Backslashes in a raw docstring: m\n
  • The starting column doesn't matter:
  1. >>> assert "Easy!"
  2.       >>> import math
  3.           >>> math.floor(1.9)
  4.           1

and as many leading whitespace characters are stripped from the expected outputas appeared in the initial '>>> ' line that started the example.

What's the Execution Context?

By default, each time doctest finds a docstring to test, it uses ashallow copy of M's globals, so that running tests doesn't change themodule's real globals, and so that one test in M can't leave behindcrumbs that accidentally allow another test to work. This means examples canfreely use any names defined at top-level in M, and names defined earlierin the docstring being run. Examples cannot see names defined in otherdocstrings.

You can force use of your own dict as the execution context by passingglobs=your_dict to testmod() or testfile() instead.

What About Exceptions?

No problem, provided that the traceback is the only output produced by theexample: just paste in the traceback. 1 Since tracebacks contain detailsthat are likely to change rapidly (for example, exact file paths and linenumbers), this is one case where doctest works hard to be flexible in what itaccepts.

Simple example:

  1. >>> [1, 2, 3].remove(42)
  2. Traceback (most recent call last):
  3.   File "<stdin>", line 1, in <module>
  4. ValueError: list.remove(x): x not in list

That doctest succeeds if ValueError is raised, with the list.remove(x):x not in list detail as shown.

The expected output for an exception must start with a traceback header, whichmay be either of the following two lines, indented the same as the first line ofthe example:

  1. Traceback (most recent call last):
  2. Traceback (innermost last):

The traceback header is followed by an optional traceback stack, whose contentsare ignored by doctest. The traceback stack is typically omitted, or copiedverbatim from an interactive session.

The traceback stack is followed by the most interesting part: the line(s)containing the exception type and detail. This is usually the last line of atraceback, but can extend across multiple lines if the exception has amulti-line detail:

  1. >>> raise ValueError('multi\n    line\ndetail')
  2. Traceback (most recent call last):
  3.   File "<stdin>", line 1, in <module>
  4. ValueError: multi
  5.     line
  6. detail

The last three lines (starting with ValueError) are compared against theexception's type and detail, and the rest are ignored.

Best practice is to omit the traceback stack, unless it adds significantdocumentation value to the example. So the last example is probably better as:

  1. >>> raise ValueError('multi\n    line\ndetail')
  2. Traceback (most recent call last):
  3.     ...
  4. ValueError: multi
  5.     line
  6. detail

Note that tracebacks are treated very specially. In particular, in therewritten example, the use of … is independent of doctest'sELLIPSIS option. The ellipsis in that example could be left out, orcould just as well be three (or three hundred) commas or digits, or an indentedtranscript of a Monty Python skit.

Some details you should read once, but won't need to remember:

  • Doctest can't guess whether your expected output came from an exceptiontraceback or from ordinary printing. So, e.g., an example that expectsValueError: 42 is prime will pass whether ValueError is actuallyraised or if the example merely prints that traceback text. In practice,ordinary output rarely begins with a traceback header line, so this doesn'tcreate real problems.

  • Each line of the traceback stack (if present) must be indented further thanthe first line of the example, or start with a non-alphanumeric character.The first line following the traceback header indented the same and startingwith an alphanumeric is taken to be the start of the exception detail. Ofcourse this does the right thing for genuine tracebacks.

  • When the IGNORE_EXCEPTION_DETAIL doctest option is specified,everything following the leftmost colon and any module information in theexception name is ignored.

  • The interactive shell omits the traceback header line for someSyntaxErrors. But doctest uses the traceback header line todistinguish exceptions from non-exceptions. So in the rare case where you needto test a SyntaxError that omits the traceback header, you will need tomanually add the traceback header line to your test example.

  • For some SyntaxErrors, Python displays the character position of thesyntax error, using a ^ marker:

  1. >>> 1 1
  2.   File "<stdin>", line 1
  3.     1 1
  4.       ^
  5. SyntaxError: invalid syntax

Since the lines showing the position of the error come before the exception typeand detail, they are not checked by doctest. For example, the following testwould pass, even though it puts the ^ marker in the wrong location:

  1. >>> 1 1
  2.   File "<stdin>", line 1
  3.     1 1
  4.     ^
  5. SyntaxError: invalid syntax

Option Flags

A number of option flags control various aspects of doctest's behavior.Symbolic names for the flags are supplied as module constants, which can bebitwise ORed together and passed to various functions.The names can also be used in doctest directives,and may be passed to the doctest command line interface via the -o option.

3.4 新版功能: The -o command line option.

The first group of options define test semantics, controlling aspects of howdoctest decides whether actual output matches an example's expected output:

  • doctest.DONT_ACCEPT_TRUE_FOR_1

  • By default, if an expected output block contains just 1, an actual outputblock containing just 1 or just True is considered to be a match, andsimilarly for 0 versus False. When DONT_ACCEPT_TRUE_FOR_1 isspecified, neither substitution is allowed. The default behavior caters to thatPython changed the return type of many functions from integer to boolean;doctests expecting "little integer" output still work in these cases. Thisoption will probably go away, but not for several years.

  • doctest.DONT_ACCEPT_BLANKLINE

  • By default, if an expected output block contains a line containing only thestring <BLANKLINE>, then that line will match a blank line in the actualoutput. Because a genuinely blank line delimits the expected output, this isthe only way to communicate that a blank line is expected. WhenDONT_ACCEPT_BLANKLINE is specified, this substitution is not allowed.

  • doctest.NORMALIZE_WHITESPACE

  • When specified, all sequences of whitespace (blanks and newlines) are treated asequal. Any sequence of whitespace within the expected output will match anysequence of whitespace within the actual output. By default, whitespace mustmatch exactly. NORMALIZE_WHITESPACE is especially useful when a line ofexpected output is very long, and you want to wrap it across multiple lines inyour source.

  • doctest.ELLIPSIS

  • When specified, an ellipsis marker (…) in the expected output can matchany substring in the actual output. This includes substrings that span lineboundaries, and empty substrings, so it's best to keep usage of this simple.Complicated uses can lead to the same kinds of "oops, it matched too much!"surprises that .* is prone to in regular expressions.

  • doctest.IGNORE_EXCEPTION_DETAIL

  • When specified, an example that expects an exception passes if an exception ofthe expected type is raised, even if the exception detail does not match. Forexample, an example expecting ValueError: 42 will pass if the actualexception raised is ValueError: 3*14, but will fail, e.g., ifTypeError is raised.

It will also ignore the module name used in Python 3 doctest reports. Henceboth of these variations will work with the flag specified, regardless ofwhether the test is run under Python 2.7 or Python 3.2 (or later versions):

  1. >>> raise CustomError('message')
  2. Traceback (most recent call last):
  3. CustomError: message
  4.  
  5. >>> raise CustomError('message')
  6. Traceback (most recent call last):
  7. my_module.CustomError: message

Note that ELLIPSIS can also be used to ignore thedetails of the exception message, but such a test may still fail basedon whether or not the module details are printed as part of theexception name. Using IGNORE_EXCEPTION_DETAIL and the detailsfrom Python 2.3 is also the only clear way to write a doctest that doesn'tcare about the exception detail yet continues to pass under Python 2.3 orearlier (those releases do not support doctest directives and ignore them as irrelevant comments). For example:

  1. >>> (1, 2)[3] = 'moo'
  2. Traceback (most recent call last):
  3.   File "<stdin>", line 1, in <module>
  4. TypeError: object doesn't support item assignment

passes under Python 2.3 and later Python versions with the flag specified,even though the detailchanged in Python 2.4 to say "does not" instead of "doesn't".

在 3.2 版更改: IGNORE_EXCEPTION_DETAIL now also ignores any information relatingto the module containing the exception under test.

  • doctest.SKIP
  • When specified, do not run the example at all. This can be useful in contextswhere doctest examples serve as both documentation and test cases, and anexample should be included for documentation purposes, but should not bechecked. E.g., the example's output might be random; or the example mightdepend on resources which would be unavailable to the test driver.

The SKIP flag can also be used for temporarily "commenting out" examples.

  • doctest.COMPARISON_FLAGS
  • A bitmask or'ing together all the comparison flags above.

The second group of options controls how test failures are reported:

  • doctest.REPORT_UDIFF

  • When specified, failures that involve multi-line expected and actual outputs aredisplayed using a unified diff.

  • doctest.REPORT_CDIFF

  • When specified, failures that involve multi-line expected and actual outputswill be displayed using a context diff.

  • doctest.REPORT_NDIFF

  • When specified, differences are computed by difflib.Differ, using the samealgorithm as the popular ndiff.py utility. This is the only method thatmarks differences within lines as well as across lines. For example, if a lineof expected output contains digit 1 where actual output contains letterl, a line is inserted with a caret marking the mismatching column positions.

  • doctest.REPORT_ONLY_FIRST_FAILURE

  • When specified, display the first failing example in each doctest, but suppressoutput for all remaining examples. This will prevent doctest from reportingcorrect examples that break because of earlier failures; but it might also hideincorrect examples that fail independently of the first failure. WhenREPORT_ONLY_FIRST_FAILURE is specified, the remaining examples arestill run, and still count towards the total number of failures reported; onlythe output is suppressed.

  • doctest.FAIL_FAST

  • When specified, exit after the first failing example and don't attempt to runthe remaining examples. Thus, the number of failures reported will be at most1. This flag may be useful during debugging, since examples after the firstfailure won't even produce debugging output.

The doctest command line accepts the option -f as a shorthand for -oFAIL_FAST.

3.4 新版功能.

  • doctest.REPORTING_FLAGS
  • A bitmask or'ing together all the reporting flags above.

There is also a way to register new option flag names, though this isn'tuseful unless you intend to extend doctest internals via subclassing:

  1. MY_FLAG = register_optionflag('MY_FLAG')

Directives

Doctest directives may be used to modify the option flags for an individual example. Doctest directives arespecial Python comments following an example's source code:

  1. directive             ::=  "#" "doctest:" directive_options
  2. directive_options     ::=  directive_option ("," directive_option)\*
  3. directive_option      ::=  on_or_off directive_option_name
  4. on_or_off             ::=  "+" \| "-"
  5. directive_option_name ::=  "DONT_ACCEPT_BLANKLINE" \| "NORMALIZE_WHITESPACE" \| ...

Whitespace is not allowed between the + or - and the directive optionname. The directive option name can be any of the option flag names explainedabove.

An example's doctest directives modify doctest's behavior for that singleexample. Use + to enable the named behavior, or - to disable it.

For example, this test passes:

  1. >>> print(list(range(20))) 
  2. [0,   1,  2,  3,  4,  5,  6,  7,  8,  9,
  3. 10,  11, 12, 13, 14, 15, 16, 17, 18, 19]

Without the directive it would fail, both because the actual output doesn't havetwo blanks before the single-digit list elements, and because the actual outputis on a single line. This test also passes, and also requires a directive to doso:

  1. >>> print(list(range(20))) 
  2. [0, 1, ..., 18, 19]

Multiple directives can be used on a single physical line, separated bycommas:

  1. >>> print(list(range(20))) 
  2. [0,    1, ...,   18,    19]

If multiple directive comments are used for a single example, then they arecombined:

  1. >>> print(list(range(20))) 
  2. ...                        
  3. [0,    1, ...,   18,    19]

As the previous example shows, you can add … lines to your examplecontaining only directives. This can be useful when an example is too long fora directive to comfortably fit on the same line:

  1. >>> print(list(range(5)) + list(range(10, 20)) + list(range(30, 40)))
  2. ... 
  3. [0, ..., 4, 10, ..., 19, 30, ..., 39]

Note that since all options are disabled by default, and directives apply onlyto the example they appear in, enabling options (via + in a directive) isusually the only meaningful choice. However, option flags can also be passed tofunctions that run doctests, establishing different defaults. In such cases,disabling an option via - in a directive can be useful.

警告

doctest is serious about requiring exact matches in expected output. Ifeven a single character doesn't match, the test fails. This will probablysurprise you a few times, as you learn exactly what Python does and doesn'tguarantee about output. For example, when printing a set, Python doesn'tguarantee that the element is printed in any particular order, so a test like

  1. >>> foo()
  2. {"Hermione", "Harry"}

is vulnerable! One workaround is to do

  1. >>> foo() == {"Hermione", "Harry"}
  2. True

instead. Another is to do

  1. >>> d = sorted(foo())
  2. >>> d
  3. ['Harry', 'Hermione']

注解

Before Python 3.6, when printing a dict, Python did not guarantee thatthe key-value pairs was printed in any particular order.

There are others, but you get the idea.

Another bad idea is to print things that embed an object address, like

  1. >>> id(1.0) # certain to fail some of the time
  2. 7948648
  3. >>> class C: pass
  4. >>> C()   # the default repr() for instances embeds an address
  5. <__main__.C instance at 0x00AC18F0>

The ELLIPSIS directive gives a nice approach for the last example:

  1. >>> C() 
  2. <__main__.C instance at 0x...>

Floating-point numbers are also subject to small output variations acrossplatforms, because Python defers to the platform C library for float formatting,and C libraries vary widely in quality here.

  1. >>> 1./7  # risky
  2. 0.14285714285714285
  3. >>> print(1./7) # safer
  4. 0.142857142857
  5. >>> print(round(1./7, 6)) # much safer
  6. 0.142857

Numbers of the form I/2.**J are safe across all platforms, and I oftencontrive doctest examples to produce numbers of that form:

  1. >>> 3./4  # utterly safe
  2. 0.75

Simple fractions are also easier for people to understand, and that makes forbetter documentation.

Basic API

The functions testmod() and testfile() provide a simple interface todoctest that should be sufficient for most basic uses. For a less formalintroduction to these two functions, see sections 简单用法：检查Docstrings中的示例and Simple Usage: Checking Examples in a Text File.

  • doctest.testfile(filename, module_relative=True, name=None, package=None, globs=None, verbose=None, report=True, optionflags=0, extraglobs=None, raise_on_error=False, parser=DocTestParser(), encoding=None)
  • All arguments except filename are optional, and should be specified in keywordform.

Test examples in the file named filename. Return (failure_count,test_count).

Optional argument module_relative specifies how the filename should beinterpreted:

  • If module_relative is True (the default), then filename specifies anOS-independent module-relative path. By default, this path is relative to thecalling module's directory; but if the package argument is specified, then itis relative to that package. To ensure OS-independence, filename should use/ characters to separate path segments, and may not be an absolute path(i.e., it may not begin with /).

  • If module_relative is False, then filename specifies an OS-specificpath. The path may be absolute or relative; relative paths are resolved withrespect to the current working directory.

Optional argument name gives the name of the test; by default, or if None,os.path.basename(filename) is used.

Optional argument package is a Python package or the name of a Python packagewhose directory should be used as the base directory for a module-relativefilename. If no package is specified, then the calling module's directory isused as the base directory for module-relative filenames. It is an error tospecify package if module_relative is False.

Optional argument globs gives a dict to be used as the globals when executingexamples. A new shallow copy of this dict is created for the doctest, so itsexamples start with a clean slate. By default, or if None, a new empty dictis used.

Optional argument extraglobs gives a dict merged into the globals used toexecute examples. This works like dict.update(): if globs andextraglobs have a common key, the associated value in extraglobs appears inthe combined dict. By default, or if None, no extra globals are used. Thisis an advanced feature that allows parameterization of doctests. For example, adoctest can be written for a base class, using a generic name for the class,then reused to test any number of subclasses by passing an extraglobs dictmapping the generic name to the subclass to be tested.

Optional argument verbose prints lots of stuff if true, and prints onlyfailures if false; by default, or if None, it's true if and only if '-v'is in sys.argv.

Optional argument report prints a summary at the end when true, else printsnothing at the end. In verbose mode, the summary is detailed, else the summaryis very brief (in fact, empty if all tests passed).

Optional argument optionflags (default value 0) takes thebitwise OR of option flags.See section Option Flags.

Optional argument raise_on_error defaults to false. If true, an exception israised upon the first failure or unexpected exception in an example. Thisallows failures to be post-mortem debugged. Default behavior is to continuerunning examples.

Optional argument parser specifies a DocTestParser (or subclass) thatshould be used to extract tests from the files. It defaults to a normal parser(i.e., DocTestParser()).

Optional argument encoding specifies an encoding that should be used toconvert the file to unicode.

  • doctest.testmod(m=None, name=None, globs=None, verbose=None, report=True, optionflags=0, extraglobs=None, raise_on_error=False, exclude_empty=False)
  • All arguments are optional, and all except for m should be specified inkeyword form.

Test examples in docstrings in functions and classes reachable from module m(or module main if m is not supplied or is None), starting withm.doc.

Also test examples reachable from dict m.test, if it exists and is notNone. m.test maps names (strings) to functions, classes andstrings; function and class docstrings are searched for examples; strings aresearched directly, as if they were docstrings.

Only docstrings attached to objects belonging to module m are searched.

Return (failure_count, test_count).

Optional argument name gives the name of the module; by default, or ifNone, m.name is used.

Optional argument exclude_empty defaults to false. If true, objects for whichno doctests are found are excluded from consideration. The default is a backwardcompatibility hack, so that code still using doctest.master.summarize() inconjunction with testmod() continues to get output for objects with notests. The exclude_empty argument to the newer DocTestFinderconstructor defaults to true.

Optional arguments extraglobs, verbose, report, optionflags,raise_on_error, and globs are the same as for function testfile()above, except that globs defaults to m.dict.

  • doctest.rundocstring_examples(_f, globs, verbose=False, name="NoName", compileflags=None, optionflags=0)
  • Test examples associated with object f; for example, f may be a string,a module, a function, or a class object.

A shallow copy of dictionary argument globs is used for the execution context.

Optional argument name is used in failure messages, and defaults to"NoName".

If optional argument verbose is true, output is generated even if there are nofailures. By default, output is generated only in case of an example failure.

Optional argument compileflags gives the set of flags that should be used bythe Python compiler when running the examples. By default, or if None,flags are deduced corresponding to the set of future features found in globs.

Optional argument optionflags works as for function testfile() above.

Unittest API

As your collection of doctest'ed modules grows, you'll want a way to run alltheir doctests systematically. doctest provides two functions that canbe used to create unittest test suites from modules and text filescontaining doctests. To integrate with unittest test discovery, includea load_tests() function in your test module:

  1. import unittest
  2. import doctest
  3. import my_module_with_doctests
  4.  
  5. def load_tests(loader, tests, ignore):
  6.     tests.addTests(doctest.DocTestSuite(my_module_with_doctests))
  7.     return tests

There are two main functions for creating unittest.TestSuite instancesfrom text files and modules with doctests:

  • doctest.DocFileSuite(*paths, module_relative=True, package=None, setUp=None, tearDown=None, globs=None, optionflags=0, parser=DocTestParser(), encoding=None)
  • Convert doctest tests from one or more text files to aunittest.TestSuite.

The returned unittest.TestSuite is to be run by the unittest frameworkand runs the interactive examples in each file. If an example in any filefails, then the synthesized unit test fails, and a failureExceptionexception is raised showing the name of the file containing the test and a(sometimes approximate) line number.

Pass one or more paths (as strings) to text files to be examined.

Options may be provided as keyword arguments:

Optional argument module_relative specifies how the filenames in _paths_should be interpreted:

  • If module_relative is True (the default), then each filename inpaths specifies an OS-independent module-relative path. By default, thispath is relative to the calling module's directory; but if the _package_argument is specified, then it is relative to that package. To ensureOS-independence, each filename should use / characters to separate pathsegments, and may not be an absolute path (i.e., it may not begin with/).

  • If module_relative is False, then each filename in paths specifiesan OS-specific path. The path may be absolute or relative; relative pathsare resolved with respect to the current working directory.

Optional argument package is a Python package or the name of a Pythonpackage whose directory should be used as the base directory formodule-relative filenames in paths. If no package is specified, then thecalling module's directory is used as the base directory for module-relativefilenames. It is an error to specify package if module_relative isFalse.

Optional argument setUp specifies a set-up function for the test suite.This is called before running the tests in each file. The setUp functionwill be passed a DocTest object. The setUp function can access thetest globals as the globs attribute of the test passed.

Optional argument tearDown specifies a tear-down function for the testsuite. This is called after running the tests in each file. The tearDown_function will be passed a DocTest object. The setUp function canaccess the test globals as the _globs attribute of the test passed.

Optional argument globs is a dictionary containing the initial globalvariables for the tests. A new copy of this dictionary is created for eachtest. By default, globs is a new empty dictionary.

Optional argument optionflags specifies the default doctest options for thetests, created by or-ing together individual option flags. See sectionOption Flags. See function set_unittest_reportflags() belowfor a better way to set reporting options.

Optional argument parser specifies a DocTestParser (or subclass)that should be used to extract tests from the files. It defaults to a normalparser (i.e., DocTestParser()).

Optional argument encoding specifies an encoding that should be used toconvert the file to unicode.

The global file is added to the globals provided to doctests loadedfrom a text file using DocFileSuite().

  • doctest.DocTestSuite(module=None, globs=None, extraglobs=None, test_finder=None, setUp=None, tearDown=None, checker=None)
  • Convert doctest tests for a module to a unittest.TestSuite.

The returned unittest.TestSuite is to be run by the unittest frameworkand runs each doctest in the module. If any of the doctests fail, then thesynthesized unit test fails, and a failureException exception is raisedshowing the name of the file containing the test and a (sometimes approximate)line number.

Optional argument module provides the module to be tested. It can be a moduleobject or a (possibly dotted) module name. If not specified, the module callingthis function is used.

Optional argument globs is a dictionary containing the initial globalvariables for the tests. A new copy of this dictionary is created for eachtest. By default, globs is a new empty dictionary.

Optional argument extraglobs specifies an extra set of global variables, whichis merged into globs. By default, no extra globals are used.

Optional argument test_finder is the DocTestFinder object (or adrop-in replacement) that is used to extract doctests from the module.

Optional arguments setUp, tearDown, and optionflags are the same as forfunction DocFileSuite() above.

This function uses the same search technique as testmod().

在 3.5 版更改: DocTestSuite() returns an empty unittest.TestSuite if _module_contains no docstrings instead of raising ValueError.

Under the covers, DocTestSuite() creates a unittest.TestSuite outof doctest.DocTestCase instances, and DocTestCase is asubclass of unittest.TestCase. DocTestCase isn't documentedhere (it's an internal detail), but studying its code can answer questions aboutthe exact details of unittest integration.

Similarly, DocFileSuite() creates a unittest.TestSuite out ofdoctest.DocFileCase instances, and DocFileCase is a subclassof DocTestCase.

So both ways of creating a unittest.TestSuite run instances ofDocTestCase. This is important for a subtle reason: when you rundoctest functions yourself, you can control the doctest options inuse directly, by passing option flags to doctest functions. However, ifyou're writing a unittest framework, unittest ultimately controlswhen and how tests get run. The framework author typically wants to controldoctest reporting options (perhaps, e.g., specified by command lineoptions), but there's no way to pass options through unittest todoctest test runners.

For this reason, doctest also supports a notion of doctestreporting flags specific to unittest support, via this function:

  • doctest.setunittest_reportflags(_flags)
  • Set the doctest reporting flags to use.

Argument flags takes the bitwise OR of option flags. Seesection Option Flags. Only "reporting flags" can be used.

This is a module-global setting, and affects all future doctests run by moduleunittest: the runTest() method of DocTestCase looks atthe option flags specified for the test case when the DocTestCaseinstance was constructed. If no reporting flags were specified (which is thetypical and expected case), doctest's unittest reporting flags arebitwise ORed into the option flags, and the option flagsso augmented are passed to the DocTestRunner instance created torun the doctest. If any reporting flags were specified when theDocTestCase instance was constructed, doctest'sunittest reporting flags are ignored.

The value of the unittest reporting flags in effect before the functionwas called is returned by the function.

Advanced API

The basic API is a simple wrapper that's intended to make doctest easy to use.It is fairly flexible, and should meet most users' needs; however, if yourequire more fine-grained control over testing, or wish to extend doctest'scapabilities, then you should use the advanced API.

The advanced API revolves around two container classes, which are used to storethe interactive examples extracted from doctest cases:

  • Example: A single Python statement, paired with its expectedoutput.

  • DocTest: A collection of Examples, typically extractedfrom a single docstring or text file.

Additional processing classes are defined to find, parse, and run, and checkdoctest examples:

The relationships among these processing classes are summarized in the followingdiagram:

  1.                             list of:
  2. +------+                   +---------+
  3. |module| --DocTestFinder-> | DocTest | --DocTestRunner-> results
  4. +------+    |        ^     +---------+     |       ^    (printed)
  5.             |        |     | Example |     |       |
  6.             v        |     |   ...   |     v       |
  7.            DocTestParser   | Example |   OutputChecker
  8.                            +---------+

DocTest 对象

  • class doctest.DocTest(examples, globs, name, filename, lineno, docstring)
  • A collection of doctest examples that should be run in a single namespace. Theconstructor arguments are used to initialize the attributes of the same names.

DocTest defines the following attributes. They are initialized bythe constructor, and should not be modified directly.

  • examples

  • A list of Example objects encoding the individual interactive Pythonexamples that should be run by this test.

  • globs

  • The namespace (aka globals) that the examples should be run in. This is adictionary mapping names to values. Any changes to the namespace made by theexamples (such as binding new variables) will be reflected in globsafter the test is run.

  • name

  • A string name identifying the DocTest. Typically, this is the nameof the object or file that the test was extracted from.

  • filename

  • The name of the file that this DocTest was extracted from; orNone if the filename is unknown, or if the DocTest was notextracted from a file.

  • lineno

  • The line number within filename where this DocTest begins, orNone if the line number is unavailable. This line number is zero-basedwith respect to the beginning of the file.

  • docstring

  • The string that the test was extracted from, or None if the string isunavailable, or if the test was not extracted from a string.

Example Objects

  • class doctest.Example(source, want, exc_msg=None, lineno=0, indent=0, options=None)
  • A single interactive example, consisting of a Python statement and its expectedoutput. The constructor arguments are used to initialize the attributes ofthe same names.

Example defines the following attributes. They are initialized bythe constructor, and should not be modified directly.

  • source

  • A string containing the example's source code. This source code consists of asingle Python statement, and always ends with a newline; the constructor addsa newline when necessary.

  • want

  • The expected output from running the example's source code (either fromstdout, or a traceback in case of exception). want ends with anewline unless no output is expected, in which case it's an empty string. Theconstructor adds a newline when necessary.

  • exc_msg

  • The exception message generated by the example, if the example is expected togenerate an exception; or None if it is not expected to generate anexception. This exception message is compared against the return value oftraceback.format_exception_only(). exc_msg ends with a newlineunless it's None. The constructor adds a newline if needed.

  • lineno

  • The line number within the string containing this example where the examplebegins. This line number is zero-based with respect to the beginning of thecontaining string.

  • indent

  • The example's indentation in the containing string, i.e., the number of spacecharacters that precede the example's first prompt.

  • options

  • A dictionary mapping from option flags to True or False, which is usedto override default options for this example. Any option flags not containedin this dictionary are left at their default value (as specified by theDocTestRunner's optionflags). By default, no options are set.

DocTestFinder 对象

  • class doctest.DocTestFinder(verbose=False, parser=DocTestParser(), recurse=True, exclude_empty=True)
  • A processing class used to extract the DocTests that are relevant toa given object, from its docstring and the docstrings of its contained objects.DocTests can be extracted from modules, classes, functions,methods, staticmethods, classmethods, and properties.

The optional argument verbose can be used to display the objects searched bythe finder. It defaults to False (no output).

The optional argument parser specifies the DocTestParser object (or adrop-in replacement) that is used to extract doctests from docstrings.

If the optional argument recurse is false, then DocTestFinder.find()will only examine the given object, and not any contained objects.

If the optional argument exclude_empty is false, thenDocTestFinder.find() will include tests for objects with empty docstrings.

DocTestFinder defines the following method:

  • find(obj[, name][, module][, globs][, extraglobs])
  • Return a list of the DocTests that are defined by obj'sdocstring, or by any of its contained objects' docstrings.

The optional argument name specifies the object's name; this name will beused to construct names for the returned DocTests. If name isnot specified, then obj.name is used.

The optional parameter module is the module that contains the given object.If the module is not specified or is None, then the test finder will attemptto automatically determine the correct module. The object's module is used:

  1. -

As a default namespace, if globs is not specified.

  1. -

To prevent the DocTestFinder from extracting DocTests from objects that areimported from other modules. (Contained objects with modules other thanmodule are ignored.)

  1. -

To find the name of the file containing the object.

  1. -

To help find the line number of the object within its file.

If module is False, no attempt to find the module will be made. This isobscure, of use mostly in testing doctest itself: if module is False, oris None but cannot be found automatically, then all objects are consideredto belong to the (non-existent) module, so all contained objects will(recursively) be searched for doctests.

The globals for each DocTest is formed by combining globs andextraglobs (bindings in extraglobs override bindings in globs). A newshallow copy of the globals dictionary is created for each DocTest.If globs is not specified, then it defaults to the module's dict, ifspecified, or {} otherwise. If extraglobs is not specified, then itdefaults to {}.

DocTestParser 对象

  • class doctest.DocTestParser
  • A processing class used to extract interactive examples from a string, and usethem to create a DocTest object.

DocTestParser defines the following methods:

  • getdoctest(_string, globs, name, filename, lineno)
  • Extract all doctest examples from the given string, and collect them into aDocTest object.

globs, name, filename, and lineno are attributes for the newDocTest object. See the documentation for DocTest for moreinformation.

  • getexamples(_string, name='')

  • Extract all doctest examples from the given string, and return them as a listof Example objects. Line numbers are 0-based. The optional argumentname is a name identifying this string, and is only used for error messages.

  • parse(string, name='')

  • Divide the given string into examples and intervening text, and return them asa list of alternating Examples and strings. Line numbers for theExamples are 0-based. The optional argument name is a nameidentifying this string, and is only used for error messages.

DocTestRunner 对象

  • class doctest.DocTestRunner(checker=None, verbose=None, optionflags=0)
  • A processing class used to execute and verify the interactive examples in aDocTest.

The comparison between expected outputs and actual outputs is done by anOutputChecker. This comparison may be customized with a number ofoption flags; see section Option Flags for more information. If theoption flags are insufficient, then the comparison may also be customized bypassing a subclass of OutputChecker to the constructor.

The test runner's display output can be controlled in two ways. First, an outputfunction can be passed to TestRunner.run(); this function will be calledwith strings that should be displayed. It defaults to sys.stdout.write. Ifcapturing the output is not sufficient, then the display output can be alsocustomized by subclassing DocTestRunner, and overriding the methodsreport_start(), report_success(),report_unexpected_exception(), and report_failure().

The optional keyword argument checker specifies the OutputCheckerobject (or drop-in replacement) that should be used to compare the expectedoutputs to the actual outputs of doctest examples.

The optional keyword argument verbose controls the DocTestRunner'sverbosity. If verbose is True, then information is printed about eachexample, as it is run. If verbose is False, then only failures areprinted. If verbose is unspecified, or None, then verbose output is usediff the command-line switch -v is used.

The optional keyword argument optionflags can be used to control how the testrunner compares expected output to actual output, and how it displays failures.For more information, see section Option Flags.

DocTestParser defines the following methods:

  • reportstart(_out, test, example)
  • Report that the test runner is about to process the given example. This methodis provided to allow subclasses of DocTestRunner to customize theiroutput; it should not be called directly.

example is the example about to be processed. test is the testcontaining example. out is the output function that was passed toDocTestRunner.run().

  • reportsuccess(_out, test, example, got)
  • Report that the given example ran successfully. This method is provided toallow subclasses of DocTestRunner to customize their output; itshould not be called directly.

example is the example about to be processed. got is the actual outputfrom the example. test is the test containing example. out is theoutput function that was passed to DocTestRunner.run().

  • reportfailure(_out, test, example, got)
  • Report that the given example failed. This method is provided to allowsubclasses of DocTestRunner to customize their output; it should notbe called directly.

example is the example about to be processed. got is the actual outputfrom the example. test is the test containing example. out is theoutput function that was passed to DocTestRunner.run().

  • reportunexpected_exception(_out, test, example, exc_info)
  • Report that the given example raised an unexpected exception. This method isprovided to allow subclasses of DocTestRunner to customize theiroutput; it should not be called directly.

example is the example about to be processed. exc_info is a tuplecontaining information about the unexpected exception (as returned bysys.exc_info()). test is the test containing example. out is theoutput function that was passed to DocTestRunner.run().

  • run(test, compileflags=None, out=None, clear_globs=True)
  • Run the examples in test (a DocTest object), and display theresults using the writer function out.

The examples are run in the namespace test.globs. If clear_globs istrue (the default), then this namespace will be cleared after the test runs,to help with garbage collection. If you would like to examine the namespaceafter the test completes, then use clear_globs=False.

compileflags gives the set of flags that should be used by the Pythoncompiler when running the examples. If not specified, then it will default tothe set of future-import flags that apply to globs.

The output of each example is checked using the DocTestRunner'soutput checker, and the results are formatted by theDocTestRunner.report_*() methods.

  • summarize(verbose=None)
  • Print a summary of all the test cases that have been run by this DocTestRunner,and return a named tuple TestResults(failed, attempted).

The optional verbose argument controls how detailed the summary is. If theverbosity is not specified, then the DocTestRunner's verbosity isused.

OutputChecker 对象

  • class doctest.OutputChecker
  • A class used to check the whether the actual output from a doctest examplematches the expected output. OutputChecker defines two methods:check_output(), which compares a given pair of outputs, and returns Trueif they match; and output_difference(), which returns a string describingthe differences between two outputs.

OutputChecker defines the following methods:

  • checkoutput(_want, got, optionflags)

  • Return True iff the actual output from an example (got) matches theexpected output (want). These strings are always considered to match ifthey are identical; but depending on what option flags the test runner isusing, several non-exact match types are also possible. See sectionOption Flags for more information about option flags.

  • outputdifference(_example, got, optionflags)

  • Return a string describing the differences between the expected output for agiven example (example) and the actual output (got). optionflags is theset of option flags used to compare want and got.

调试

Doctest provides several mechanisms for debugging doctest examples:

  • Several functions convert doctests to executable Python programs, which can berun under the Python debugger, pdb.

  • The DebugRunner class is a subclass of DocTestRunner thatraises an exception for the first failing example, containing information aboutthat example. This information can be used to perform post-mortem debugging onthe example.

  • The unittest cases generated by DocTestSuite() support thedebug() method defined by unittest.TestCase.

  • You can add a call to pdb.set_trace() in a doctest example, and you'lldrop into the Python debugger when that line is executed. Then you can inspectcurrent values of variables, and so on. For example, suppose a.pycontains just this module docstring:

  1. """
  2. >>> def f(x):
  3. ...     g(x*2)
  4. >>> def g(x):
  5. ...     print(x+3)
  6. ...     import pdb; pdb.set_trace()
  7. >>> f(3)
  8. 9
  9. """

Then an interactive Python session may look like this:

  1. >>> import a, doctest
  2. >>> doctest.testmod(a)
  3. --Return--
  4. > <doctest a[1]>(3)g()->None
  5. -> import pdb; pdb.set_trace()
  6. (Pdb) list
  7.   1     def g(x):
  8.   2         print(x+3)
  9.   3  ->     import pdb; pdb.set_trace()
  10. [EOF]
  11. (Pdb) p x
  12. 6
  13. (Pdb) step
  14. --Return--
  15. > <doctest a[0]>(2)f()->None
  16. -> g(x*2)
  17. (Pdb) list
  18.   1     def f(x):
  19.   2  ->     g(x*2)
  20. [EOF]
  21. (Pdb) p x
  22. 3
  23. (Pdb) step
  24. --Return--
  25. > <doctest a[2]>(1)?()->None
  26. -> f(3)
  27. (Pdb) cont
  28. (0, 3)
  29. >>>

Functions that convert doctests to Python code, and possibly run the synthesizedcode under the debugger:

  • doctest.scriptfrom_examples(_s)
  • Convert text with examples to a script.

Argument s is a string containing doctest examples. The string is convertedto a Python script, where doctest examples in s are converted to regular code,and everything else is converted to Python comments. The generated script isreturned as a string. For example,

  1. import doctest
  2. print(doctest.script_from_examples(r"""
  3.     Set x and y to 1 and 2.
  4.     >>> x, y = 1, 2
  5.  
  6.     Print their sum:
  7.     >>> print(x+y)
  8.     3
  9. """))

displays:

  1. # Set x and y to 1 and 2.
  2. x, y = 1, 2
  3. #
  4. # Print their sum:
  5. print(x+y)
  6. # Expected:
  7. ## 3

This function is used internally by other functions (see below), but can also beuseful when you want to transform an interactive Python session into a Pythonscript.

  • doctest.testsource(module, name)
  • Convert the doctest for an object to a script.

Argument module is a module object, or dotted name of a module, containing theobject whose doctests are of interest. Argument name is the name (within themodule) of the object with the doctests of interest. The result is a string,containing the object's docstring converted to a Python script, as described forscript_from_examples() above. For example, if module a.pycontains a top-level function f(), then

  1. import a, doctest
  2. print(doctest.testsource(a, "a.f"))

prints a script version of function f()'s docstring, with doctestsconverted to code, and the rest placed in comments.

  • doctest.debug(module, name, pm=False)
  • Debug the doctests for an object.

The module and name arguments are the same as for functiontestsource() above. The synthesized Python script for the named object'sdocstring is written to a temporary file, and then that file is run under thecontrol of the Python debugger, pdb.

A shallow copy of module.dict is used for both local and globalexecution context.

Optional argument pm controls whether post-mortem debugging is used. If pm_has a true value, the script file is run directly, and the debugger getsinvolved only if the script terminates via raising an unhandled exception. Ifit does, then post-mortem debugging is invoked, via pdb.post_mortem(),passing the traceback object from the unhandled exception. If _pm is notspecified, or is false, the script is run under the debugger from the start, viapassing an appropriate exec() call to pdb.run().

  • doctest.debugsrc(_src, pm=False, globs=None)
  • Debug the doctests in a string.

This is like function debug() above, except that a string containingdoctest examples is specified directly, via the src argument.

Optional argument pm has the same meaning as in function debug() above.

Optional argument globs gives a dictionary to use as both local and globalexecution context. If not specified, or None, an empty dictionary is used.If specified, a shallow copy of the dictionary is used.

The DebugRunner class, and the special exceptions it may raise, are ofmost interest to testing framework authors, and will only be sketched here. Seethe source code, and especially DebugRunner's docstring (which is adoctest!) for more details:

  • class doctest.DebugRunner(checker=None, verbose=None, optionflags=0)
  • A subclass of DocTestRunner that raises an exception as soon as afailure is encountered. If an unexpected exception occurs, anUnexpectedException exception is raised, containing the test, theexample, and the original exception. If the output doesn't match, then aDocTestFailure exception is raised, containing the test, the example, andthe actual output.

For information about the constructor parameters and methods, see thedocumentation for DocTestRunner in section Advanced API.

There are two exceptions that may be raised by DebugRunner instances:

  • exception doctest.DocTestFailure(test, example, got)
  • An exception raised by DocTestRunner to signal that a doctest example'sactual output did not match its expected output. The constructor arguments areused to initialize the attributes of the same names.

DocTestFailure defines the following attributes:

  • DocTestFailure.test

  • The DocTest object that was being run when the example failed.

  • DocTestFailure.example

  • The Example that failed.

  • DocTestFailure.got

  • The example's actual output.

  • exception doctest.UnexpectedException(test, example, exc_info)

  • An exception raised by DocTestRunner to signal that a doctestexample raised an unexpected exception. The constructor arguments are usedto initialize the attributes of the same names.

UnexpectedException defines the following attributes:

  • UnexpectedException.test

  • The DocTest object that was being run when the example failed.

  • UnexpectedException.example

  • The Example that failed.

  • UnexpectedException.exc_info

  • A tuple containing information about the unexpected exception, as returned bysys.exc_info().

Soapbox

As mentioned in the introduction, doctest has grown to have three primaryuses:

  • Checking examples in docstrings.

  • Regression testing.

  • Executable documentation / literate testing.

These uses have different requirements, and it is important to distinguish them.In particular, filling your docstrings with obscure test cases makes for baddocumentation.

When writing a docstring, choose docstring examples with care. There's an art tothis that needs to be learned—-it may not be natural at first. Examples shouldadd genuine value to the documentation. A good example can often be worth manywords. If done with care, the examples will be invaluable for your users, andwill pay back the time it takes to collect them many times over as the years goby and things change. I'm still amazed at how often one of my doctestexamples stops working after a "harmless" change.

Doctest also makes an excellent tool for regression testing, especially if youdon't skimp on explanatory text. By interleaving prose and examples, it becomesmuch easier to keep track of what's actually being tested, and why. When a testfails, good prose can make it much easier to figure out what the problem is, andhow it should be fixed. It's true that you could write extensive comments incode-based testing, but few programmers do. Many have found that using doctestapproaches instead leads to much clearer tests. Perhaps this is simply becausedoctest makes writing prose a little easier than writing code, while writingcomments in code is a little harder. I think it goes deeper than just that:the natural attitude when writing a doctest-based test is that you want toexplain the fine points of your software, and illustrate them with examples.This in turn naturally leads to test files that start with the simplestfeatures, and logically progress to complications and edge cases. A coherentnarrative is the result, instead of a collection of isolated functions that testisolated bits of functionality seemingly at random. It's a different attitude,and produces different results, blurring the distinction between testing andexplaining.

Regression testing is best confined to dedicated objects or files. There areseveral options for organizing tests:

  • Write text files containing test cases as interactive examples, and test thefiles using testfile() or DocFileSuite(). This is recommended,although is easiest to do for new projects, designed from the start to usedoctest.

  • Define functions named _regrtest_topic that consist of single docstrings,containing test cases for the named topics. These functions can be included inthe same file as the module, or separated out into a separate test file.

  • Define a test dictionary mapping from regression test topics todocstrings containing test cases.

When you have placed your tests in a module, the module can itself be the testrunner. When a test fails, you can arrange for your test runner to re-run onlythe failing doctest while you debug the problem. Here is a minimal example ofsuch a test runner:

  1. if __name__ == '__main__':
  2.     import doctest
  3.     flags = doctest.REPORT_NDIFF|doctest.FAIL_FAST
  4.     if len(sys.argv) > 1:
  5.         name = sys.argv[1]
  6.         if name in globals():
  7.             obj = globals()[name]
  8.         else:
  9.             obj = __test__[name]
  10.         doctest.run_docstring_examples(obj, globals(), name=name,
  11.                                        optionflags=flags)
  12.     else:
  13.         fail, total = doctest.testmod(optionflags=flags)
  14.         print("{} failures out of {} tests".format(fail, total))

脚注

  • 1
  • Examples containing both expected output and an exception are not supported.Trying to guess where one ends and the other begins is too error-prone, and thatalso makes for a confusing test.