nbdev1 Migration

How to change your nbdev1 repo to work with nbdev2

nbdev v2 is a new from-scratch rewrite of nbdev that’s not backwards compatible. This page describes the changes you need to make to upgrade your nbdev v1 repo to work with the new version. The steps shown here should work on macOS or Linux (including Windows WSL)

The biggest change is that nbdev2 uses Quarto to generate your website, whereas nbdev1 used nbconvert and jekyll. You can use all of Quarto’s features directly in nbdev, so checkout the Quarto website to see all the amazing functionality it supports.

Initial setup

If you’ve pinned nbdev in requirements.txt or settings.ini (e.g nbdev<2) remove the version pin. (If you don’t know what this means, then you don’t have it, so you can ignore this step).

Install the latest version of nbdev by typing:

pip install -U nbdev

or:

conda install -c fastai nbdev

You may need to restart your terminal for the new commands to be visible to your shell.

Upgrade directives

nbdev has slightly changed how “directive comments” like export and default_exp work, in order to align with how Quarto does things. Now, instead of just adding a # to the start to indicate a directive (e.g #export), you now need to use #| (e.g #|export). You can also optionally add a space (e.g #| export).

To automatically upgrade your directives to the new format, run in the root of your repo:

nbdev_migrate

You should now test that you can export your module by running:

nbdev_export

Note that nbdev_export replaces nbdev_build_lib. Run nbdev_export -h to see the options you can pass to it (normally you won’t need to pass any). To see a list of all the commands available in nbdev2, run nbdev_help.

Add and remove files

First set a variable with the name of your library, by running the following (replacing “yourlib” with the name of your library’s subdirectory)

export LIBNAME=yourlib

Now run the following:

git rm Makefile
git add $LIBNAME/_modidx.py
rm -rf docs
rm -f .gitconfig 
rm -f .git/hooks/post-merge

rm -f setup.py
curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/styles.css
curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/setup.py

cat >>.gitignore <<EOF
_docs/
_proc/
EOF

As you see above, we’ve remove the Makefile – that’s because all the things done by make before are now handled by nbdev commands directly.

Note

All documentation related files should be included in your nbs_path, and all paths should be relative to it. If you have set the nbs_path in your settings.ini file, then copy your styles.css file inside of your nbs_path folder.

If you use GitHub Actions for continuous integration (CI) you can update this to use nbdev too as follows:

rm -f .github/workflows/main.yml
curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/.github/workflows/test.yaml
curl -O https://raw.githubusercontent.com/fastai/nbdev-template/master/.github/workflows/deploy.yaml
mv deploy.yaml test.yaml .github/workflows/

Update directive names

A number of directives have changed names. We’ll use perl to fix them. Run these lines in the root of your repo:

find . -name '*.ipynb' -exec perl -pi -e 's/#\|\s*hide_input/#| echo: false/' {} +
find . -name '*.ipynb' -exec perl -pi -e 's/#\|\s*hide_output/#| output: false/' {} +
find . -name '*.ipynb' -exec perl -pi -e 's/#\|\s*skip/#| eval: false/' {} +
find . -name '*.ipynb' -exec perl -pi -e 's/from nbdev.export import notebook2script/from nbdev import nbdev_export/' {} +
find . -name '*.ipynb' -exec perl -pi -e 's/notebook2script/nbdev_export/' {} +

These change the following directives to use functionality built into Quarto:

  • hide_input –> echo: false
  • hide_output –> output: false
  • skip –> eval: false

They also update the new location and name of the nbdev_export python function.

If you have any notebooks that you’ve asked nbdev1 to skip (using all_slow), you’ll need to add a raw cell to the top of your notebook containing YAML frontmatter. The frontmatter needs to include skip_showdoc: true to avoid running cells when rendering docs, and skip_exec: true to skip this notebook when running tests. E.g to do both, you would add a raw cell (or update your existing frontmatter raw cell) to contain:

---
skip_showdoc: true
skip_exec: true
---

Or you can also add these flags in a markdown cell,

# title
> description

- skip_showdoc: true
- skip_exec: true

Edit Workflow Permissions

Make sure your workflow permissions are set to “Read and write permissions”, which you can find in Settings → Actions → General → Workflow permissions:

GitHub Pages settings
Important

Failure to set the correct permissions may result in an error message like this:

 fatal: unable to access 'https://github.com/user/repo.git/': The requested URL returned error: 403
  Error: Action failed with "The process '/usr/bin/git' failed with exit code 128"

Edit GitHub Pages Permissions

At this point you will want to commit the files with the changes you made to GitHub. Wait for GitHub Actions to run and pass. A new branch in your repo will automatically be created called gh-pages. You want to enable GitHub Pages to work off this branch by configuring your Pages to settings to look like this:

Access this screen under Settings → Pages

  • Select “Deploy from a branch” in the drop down list for Source.
  • Specify gh-pages as the branch
  • Specify the /root as the location
  • Click save

Actions workflow permissions

Final steps

You should now edit settings.ini, and change doc_path from docs to _docs, since that’s where nbdev2 will build your website.

If you use a custom domain for your website, you should move your CNAME file into the directory containing your notebooks.

Before pushing to GitHub, check that your website looks OK locally by running:

nbdev_preview

Now prepare to commit to GitHub:

nbdev_prepare

You can now commit to GitHub as usual. Finally, update Github Pages by clicking on the Settings tab in your repo, then click Pages on the left side bar. Set “Source” to gh-pages branch and the /root folder.