When a test run hangs, ctrl-c should say what each notebook was executing. Each worker installs a SIGINT handler that prints the in-flight notebook’s stack — the sync frames plus every pending asyncio task, since a hang inside an awaited coroutine lives on the task, not the sync stack — as a single block (one os.write, so parallel workers’ dumps don’t interleave) and then exits immediately, rather than letting the interrupt surface inside the cell where the shell would catch it and carry on. An idle worker exits silently.
def test_nb( fn, # file name of notebook to test skip_flags:NoneType=None, # list of flags marking cells to skip force_flags:NoneType=None, # list of flags marking cells to always run do_print:bool=False, # print completion? showerr:bool=True, # print errors to stderr? basepath:NoneType=None, # path to add to sys.path verbose:bool=False, # stream stdout/stderr from cells to console? save:bool=False, # write outputs back to notebook on success? profile:bool=None, # load the IPython profile, as `ipykernel` does? (default: `exec_profile` config key) cell_timeout:int=600, # seconds before each cell times out (None: no limit) cell_timing_min:float=None, # print cells slower than this many seconds (None: no timing output)):
Execute tests in notebook in fn except those with skip_flags
test_nb is sync: every cell runs on the CaptureShell’s own loop thread, and the caller simply blocks on each result. parallel workers call it directly, and because the worker’s main thread stays free while cells run, the SIGINT handler above can dump the shell thread’s stack and its pending tasks in the middle of a hang. cell_timeout bounds each cell even when it blocks the loop in sync code – the timed-out notebook fails with a TimeoutError naming the cell and the stuck tasks, instead of hanging the run.
test_nb can test a notebook, and skip over certain flags. A notebook whose frontmatter sets skip_exec: true (e.g. as a - skip_exec: true list item in its title cell) is skipped entirely and reported as passing; use it for notebooks that can’t run under test at all, such as those needing credentials or live services:
Pass cell_timing_min to identify slow cells without flooding ordinary test output. Each cell over the threshold prints its notebook name, stable cell id, and elapsed seconds; --cell-timing-min=0.1 is a useful first pass.
test_nb loads the IPython profile by default, like ipykernel does, so notebooks are tested the way their author’s kernel ran them (startup files, extensions, shell config). Set exec_profile = false under [tool.nbdev], or pass profile=False, to run without it:
Sometimes you may wish to override one or more of the skip_flags, in which case you can use the argument force_flags which will remove the appropriate tag(s) from skip_flags. This is useful because skip_flags are meant to be set in the tst_flags field of [tool.nbdev] in pyproject.toml, whereas force_flags are usually passed in by the user.
def nbdev_test( path:str=None, # A notebook name or glob to test flags:str='', # Space separated list of test flags to run that are normally ignored n_workers:int=None, # Number of workers timing:bool=False, # Time each notebook to see which are slow do_print:bool=False, # Print start and end of each notebook pause:float=0.01, # Pause time (in seconds) between notebooks to avoid race conditions ignore_fname:str='.notest', # Filename that will result in siblings being ignored verbose:bool=False, # Print stdout/stderr from notebook cells? save:bool=False, # Write outputs back to notebooks on success? cell_timeout:int=600, # Seconds before each cell times out (0: no limit) cell_timing_min:float=None, # Print cells slower than this many seconds (None: no timing output) symlinks:bool=False, # Follow symlinks? file_glob:str='*.ipynb', # Only include files matching glob file_re:str=None, # Only include files matching regex folder_re:str=None, # Only enter folders matching regex skip_file_glob:str=None, # Skip files matching glob skip_file_re:str='^[_.]', # Skip files matching regex skip_folder_re:str='^[_.]', # Skip folders matching regex):
Test in parallel notebooks matching path, passing along flags
nbdev_test(n_workers=0)
Success.
You can even run nbdev-test in non nbdev projects, for example, you can test an individual notebook like so:
test_nb decides which cells run through the eval cascade (fastcore.nbio.does_cell_eval): a cell’s own #| eval: directive wins; otherwise the notebook-level eval directive — in frontmatter, or the notebook’s metadata.nbdev mapping — sets the default; with neither, cells run. #| eval: false therefore skips one cell, as it always has, while a notebook-level eval: false flips the whole notebook to opt-in: only cells marked #| eval: true run, which suits a slow or service-dependent notebook where just a few cells are worth testing. Unlike skip_exec: true, which skips a notebook unconditionally, marked cells still run — here the unmarked cell would raise if executed, so the passing test is the proof it was skipped:
with tempfile.TemporaryDirectory() as td: cells = [mk_cell('---\neval: false\n---', 'raw'), mk_cell('raise Exception("unmarked: must not run")'), mk_cell('#| eval: true\nx = 1')] fn = Path(td)/'optin.ipynb' write_nb(new_nb(cells), fn) success,_ = test_nb(fn)assert success