test

Run unit tests on notebooks in parallel

test_nb


def test_nb(
    fn, # file name of notebook to test
    skip_flags:NoneType=None, # list of flags marking cells to skip
    force_flags:NoneType=None, # list of flags marking cells to always run
    do_print:bool=False, # print completion?
    showerr:bool=True, # print errors to stderr?
    basepath:NoneType=None, # path to add to sys.path
):

Execute tests in notebook in fn except those with skip_flags

test_nb can test a notebook, and skip over certain flags:

_nb = Path('../../tests/directives.ipynb')
success,duration = test_nb(_nb, skip_flags=['notest'])
assert success

In that notebook the cell flagged notest raises an exception, which will be returned as a bool:

_nb = Path('../../tests/directives.ipynb')
success,duration = test_nb(_nb, showerr=False)
assert not success

Sometimes you may wish to override one or more of the skip_flags, in which case you can use the argument force_flags which will remove the appropriate tag(s) from skip_flags. This is useful because skip_flags are meant to be set in the tst_flags field of settings.ini, whereas force_flags are usually passed in by the user.

source

nbdev_test


def nbdev_test(
    path:str=None, # A notebook name or glob to test
    flags:str='', # Space separated list of test flags to run that are normally ignored
    n_workers:int=None, # Number of workers
    timing:bool=False, # Time each notebook to see which are slow
    do_print:bool=False, # Print start and end of each notebook
    pause:float=0.01, # Pause time (in seconds) between notebooks to avoid race conditions
    ignore_fname:str='.notest', # Filename that will result in siblings being ignored
    symlinks:bool=False, # Follow symlinks?
    file_glob:str='*.ipynb', # Only include files matching glob
    file_re:str=None, # Only include files matching regex
    folder_re:str=None, # Only enter folders matching regex
    skip_file_glob:str=None, # Skip files matching glob
    skip_file_re:str='^[_.]', # Skip files matching regex
    skip_folder_re:str='^[_.]', # Skip folders matching regex
):

Test in parallel notebooks matching path, passing along flags

nbdev_test(n_workers=0)

Success.

You can even run nbdev_test in non nbdev projects, for example, you can test an individual notebook like so:

nbdev_test --path ../../tests/minimal.ipynb --do_print

Or you can test an entire directory of notebooks filtered for only those that match a regular expression:

nbdev_test --path ../../tests --file_re '.*test.ipynb' --do_print