processors

Some processors for NBProcessor

Helpers

On this page we’ll be using this private helper to process a notebook and return the results, to simplify testing:

def _run_procs(procs=None, preprocs=None, postprocs=None, return_nb=False, path=_test_file):
    nbp = NBProcessor(path, procs, preprocs=preprocs, postprocs=postprocs)
    nbp.process()
    if return_nb: return nbp.nb
    return '\n'.join([str(cell) for cell in nbp.nb.cells])

Notebook preprocessors

---

yml2dict

 yml2dict (s:str, rm_fence=True)

convert a string that is in a yaml format to a dict

---

is_frontmatter

 is_frontmatter (nb)

List of raw cells in nb that contain frontmatter

---

populate_language

 populate_language (nb)

Insert cell language indicator based on notebook metadata. You should to use this before lang_identify

---

insert_warning

 insert_warning (nb)

Insert Autogenerated Warning Into Notebook after the first cell.

This preprocessor inserts a warning in the markdown destination that the file is autogenerated. This warning is inserted in the second cell so we do not interfere with front matter.

res = _run_procs(preprocs=[insert_warning])
assert "<!-- WARNING: THIS FILE WAS AUTOGENERATED!" in res

L('foo', None, 'a').filter(lambda x:x == 1)
_tstre = re.compile('a')

---

add_show_docs

 add_show_docs (nb)

Add show_doc cells after exported cells, unless they are already documented

---

cell_lang

 cell_lang (cell)

res = _run_procs(preprocs=[populate_language, add_show_docs])
assert "show_doc(some_func)'" in res
assert "show_doc(and_another)'" in res
assert "show_doc(another_func)'" not in res

---

yaml_str

 yaml_str (s:str)

Create a valid YAML string from s

---

nb_fmdict

 nb_fmdict (nb, remove=True)

Infer the front matter from a notebook’s markdown formatting

_testnb = read_nb('../tests/docs_test.ipynb')
_res = nb_fmdict(_testnb)
test_eq(_res, dict(key1='value1', key2='value2', categories=['c1', 'c2'], title='a title', description='A description'))

---

filter_fm

 filter_fm (fmdict:dict)

Filter front matter

---

construct_fm

 construct_fm (fmdict:dict)

Construct front matter from a dictionary

_testdict = nb_fmdict(read_nb('../tests/docs_test.ipynb'))
_res = construct_fm(filter_fm(_testdict))
test_eq(len(_res.splitlines()), 8)
print(_res)

---
categories:
- c1
- c2
description: A description
title: a title

---

---

insert_frontmatter

 insert_frontmatter (nb, fm_dict:dict)

Add frontmatter into notebook based on filter_keys that exist in fmdict.

---

infer_frontmatter

 infer_frontmatter (nb)

Insert front matter if it doesn’t exist automatically from nbdev styled markdown.

_raw_res = _run_procs()
_res = _run_procs(preprocs=infer_frontmatter)
assert '# a title' in _raw_res and '# a title' not in _res
assert r'description: A description\n' in _res
assert r'categories:\n- c1\n- c2\n' in _res
assert r'output-file: foobar.html\n' in _res

If you already have front matter in a raw cell that will take precedence over markdown style frontmatter. For example, this notebook has the the following front matter defined in a raw cell:

---
title: fastcore my favorite python library
---

…as well as a Markdown front matter shortcuts like this:

# "fastcore: An Underrated Python Library"

> A unique python library that extends the python programming language and provides utilities that enhance productivity.
- author: "<a href='https://twitter.com/HamelHusain'>Hamel Husain</a>"
- toc: false
- image: images/copied_from_nb/fastcore_imgs/td.png
- comments: true
- search_exclude: true
- hide: true
- categories: [fastcore, fastai]
- permalink: /fastcore/
- badges: true

When this notebook is processed, the front matter looks like this, as you can see the title from the raw front matter takes precedence:

_res = _run_procs(preprocs=infer_frontmatter, path='../tests/2020-09-01-fastcore.ipynb', return_nb=True)
print(_res['cells'][0].source)

---
author: <a href='https://twitter.com/HamelHusain'>Hamel Husain</a>
categories:
- fastcore
- fastai
comments: true
description: A unique python library that extends the python programming language
  and provides utilities that enhance productivity.
image: fastcore_imgs/td.png
title: fastcore my favorite python library

---

Additionally, as shown above, front matter from fastpages is automatically aliased to conform to Quarto including modifying the image path, which minimizes burden on folks migrating from fastpages.

Cell processors

---

nbflags_

 nbflags_ (nbp, cell, *args)

Store flags marked with nbflags

nbp = NBProcessor('../tests/01_everything.ipynb', nbflags_)
nbp.process()
test_eq(nbp.nb._nbflags, ('foobar',))

---

add_links

 add_links (cell)

Add links to markdown cells

res = _run_procs(add_links)
assert "[`numpy.array`](https://numpy.org/doc/stable/reference/generated/numpy.array.html#numpy.array)" in res
assert "[`ModuleMaker`](https://nbdev.fast.ai/maker.html#modulemaker) but not a link to `foobar`." in res
assert "A link in a docstring: [`ModuleMaker`](https://nbdev.fast.ai/maker.html#modulemaker)." in res
assert "And not a link to <code>dict2nb</code>." in res

Gets rid of colors that are streamed from standard out, which can interfere with static site generators:

---

strip_ansi

 strip_ansi (cell)

Strip Ansi Characters.

res = _run_procs(strip_ansi)
assert not _re_ansi_escape.findall(res)

---

strip_hidden_metadata

 strip_hidden_metadata (cell)

Strips “hidden” metadata property from code cells so it doesn’t interfere with docs rendering

---

hide_

 hide_ (nbp, cell)

Hide cell from output

res = _run_procs(hide_)
assert 'you will not be able to see this cell at all either' not in res

---

hide_line

 hide_line (cell)

Hide lines of code in code cells with the directive hide_line at the end of a line of code

res = _run_procs(hide_line)
assert r"def show():\n    a = 2\n    b = 3" not in res
assert r"def show():\n    a = 2"                in res

---

filter_stream_

 filter_stream_ (nbp, cell, *words)

Remove output lines containing any of words in cell stream output

res = _run_procs(filter_stream_)
exp=r"'A line\n', 'Another line.\n'"
assert exp in res

---

clean_magics

 clean_magics (cell)

A preprocessor to remove cell magic commands

res = _run_procs(clean_magics)
assert "%%" not in res

---

lang_identify

 lang_identify (cell)

A preprocessor to identify bash/js/etc cells and mark them appropriately

When we issue a shell command in a notebook with !, we need to change the code-fence from python to bash and remove the !:

res = _run_procs(lang_identify)
assert "'language': 'bash'" in res

---

rm_header_dash

 rm_header_dash (cell)

Remove headings that end with a dash -

res = _run_procs(rm_header_dash)
assert 'some words' in res
assert 'A heading to Hide' not in res
assert 'Yet another heading to hide' not in res

---

rm_export

 rm_export (cell)

Remove cells that are exported or hidden

res = _run_procs(rm_export)
assert 'dontshow' not in res

---

clean_show_doc

 clean_show_doc (cell)

Remove ShowDoc input cells

---

exec_show_docs

 exec_show_docs (nb)

Execute cells needed for show_docs output, including exported cells and imports

res = _run_procs([exec_show_docs])
assert res

Notebook postprocessors

There are no notebook postprocessors yet.