nbdev v2 evaluation: Git-friendly Jupyter Notebooks | Tech Zen

There are various methods to program. Some of the productive paradigms is interactive: you employ a REPL (read-evaluate-print loop) to put in writing and take a look at your code as you code, after which copy the examined code to a file.

The REPL methodology, which originated in LISP growth environments, is effectively suited to Python programming, since Python has at all times had good interactive growth instruments. The draw back to this type of programming is that after you’ve got written the code, it’s important to extract the assessments and write the documentation individually, maintain all of that in a repository, package deal and publish your package deal and documentation.

Donald Knuth’s literate programming paradigm prescribes writing documentation and code in the identical doc, with documentation meant for people interspersed with code meant for the pc. Literate programming has been extensively used for scientific programming and information science, typically utilizing moveable environments similar to Jupyter Notebooks, Jupyter Lab, Visible Studio Code, and PyCharm. One downside with notebooks is that they often do not work effectively with repositories as a result of they maintain an excessive amount of data, together with metadata that no one cares about. That creates an issue when there are merge conflicts, since notebooks are cell-oriented and supply code repositories like Git are line-oriented.

Jeremy Howard and Hamel Husain of quick.ai, together with some two dozen minor collaborators, created a set of command line utilities that not solely permit Jupyter Notebooks to work effectively with Git, but additionally allow extremely interactive literate programming. productive. type. Along with producing appropriate Python code shortly, you’ll be able to produce documentation and assessments on the identical time, save the whole lot to Git with out concern of corruption resulting from merge conflicts, and publish to PyPI and Conda with only a few instructions. Whereas there’s a studying curve to those utilities, that funding pays dividends in you could get it performed together with your growth venture in concerning the time it could usually take to only write the code.

As you’ll be able to see within the following diagram, nbdev works with Jupyter Notebooks, GitHub, Quarto, Anaconda, and PyPI. To summarize what each bit of this method does:

• You may generate documentation utilizing Quarto and host it on GitHub Pages. Paperwork are LaTeX appropriate, searchable and mechanically linked.
• You may publish packages to PyPI and Conda, in addition to instruments to simplify package deal releases. Python finest practices are mechanically adopted, for instance, solely exported objects are included in __all__.
• There may be two-way synchronization between the notebooks and the plain textual content supply code, permitting you to make use of your IDE to navigate the code or make fast edits.
• Exams written as peculiar pocket book cells are executed in parallel with a single command.
• There may be seamless integration with GitHub Actions that runs your assessments and rebuilds your docs.
• Git-compliant notebooks with Jupyter/Git hooks that clear up undesirable metadata and generate merge conflicts in a human-readable format.

IDG

nbdev set up

nbdev works on macOS, Linux, and most Unix-style working techniques. Requires a latest model of Python 3; I used Python 3.9.6 on macOS Ventura, working it on a MacBook Professional M1. nbdev works on Home windows beneath WSL (Home windows Subsystem for Linux), however not beneath cmd or PowerShell. You may set up nbdev with pip or conda. I used pip:

pip set up nbdev

That put in 29 command line utilities, which you’ll be able to listing utilizing nbdev_help:

% nbdev_help
nbdev_bump_version              Increment model in settings.ini by one
nbdev_changelog                 Create a CHANGELOG.md file from closed and labeled GitHub points
nbdev_clean                     Clear all notebooks in `fname` to keep away from merge conflicts
nbdev_conda                     Create a `meta.yaml` file able to be constructed right into a package deal, and optionally construct and add it
nbdev_create_config             Create a config file.
nbdev_docs                      Create Quarto docs and README.md
nbdev_export                    Export notebooks in `path` to Python modules
nbdev_filter                    A pocket book filter for Quarto
nbdev_fix                       Create working pocket book from conflicted pocket book `nbname`
nbdev_help                      Present assist for all console scripts
nbdev_install                   Set up Quarto and the present library
nbdev_install_hooks             Set up Jupyter and git hooks to mechanically clear, belief, and repair merge conflicts in notebooks
nbdev_install_quarto            Set up newest Quarto on macOS or Linux, prints directions for Home windows
nbdev_merge                     Git merge driver for notebooks
nbdev_migrate                   Convert all markdown and pocket book information in `path` from v1 to v2
nbdev_new                       Create an nbdev venture.
nbdev_prepare                   Export, take a look at, and clear notebooks, and render README if wanted
nbdev_preview                   Preview docs regionally
nbdev_proc_nbs                  Course of notebooks in `path` for docs rendering
nbdev_pypi                      Create and add Python package deal to PyPI
nbdev_readme                    None
nbdev_release_both              Launch each conda and PyPI packages
nbdev_release_gh                Calls `nbdev_changelog`, enables you to edit the end result, then pushes to git and calls `nbdev_release_git`
nbdev_release_git               Tag and create a launch in GitHub for the present model
nbdev_sidebar                   Create sidebar.yml
nbdev_test                      Check in parallel notebooks matching `path`, passing alongside `flags`
nbdev_trust                     Belief notebooks matching `fname`
nbdev_update                    Propagate change in modules matching `fname` to notebooks that created them

The builders at nbdev recommend watching this 90 minute video or following this roughly one hour written tutorial. I did each and likewise learn extra documentation and a number of the supply code. I discovered totally different materials from every one, so I recommend watching the video first, then doing the tutorial. For me, the video gave me a transparent sufficient thought of ​​the usefulness of the package deal to encourage me to observe the tutorial.

Begin the nbdev tutorial

The tutorial begins with putting in Jupyter Pocket book:

pip set up pocket book

After which launching Jupyter:

jupyter pocket book

The set up continues on the laptop computer, first creating a brand new terminal after which utilizing the terminal to put in nbdev. You may skip that set up if you happen to already did it in a shell, like I did.

You may then use nbdev to put in Quarto:

nbdev_install_quarto

That requires root entry, so you may have to enter your password. You may learn Quarto’s supply code or paperwork to confirm that it’s protected.

At this level, you might want to navigate to GitHub and create an empty repository (repo). I adopted the tutorial and referred to as mine nbdev_hello_world, and added a reasonably generic description. Create the repository. See the directions if you happen to want them. Then clone the repository to your native machine. The directions recommend utilizing the Git command line in your machine, however I like to make use of GitHub Desktop, which additionally labored nice.

In any case, cd in your repository in your terminal. It does not matter if you happen to use a terminal in your desktop or in your laptop computer. now run nbdev_new, which is able to create a bunch of information in your repository. Then commit and push your additions to GitHub:

git add .
git commit -m'Preliminary commit'
git push

Return to your repository on GitHub and open the Actions tab. You will note one thing like this:

IDG

Now allow GitHub Pages, following the non-obligatory directions. It ought to appear like this:

IDG

Open the Actions tab once more and you will note a 3rd workflow:

IDG

Now open your generated web site, at https://person.github.io/repo. Mine is at https://meheller.github.io/nbdev-hello-world/. You may copy that and alter meheller to your individual GitHub identifier and see one thing just like the next:

IDG

Proceed the nbdev tutorial

Now we’re lastly attending to the good things. You’ll set up internet hyperlinks to mechanically clear the notebooks if you register them,

nbdev_install_hooks

export your library,

nbdev_export

set up your package deal,

nbdev_export

preview your paperwork,

nbdev_preview

(and click on the hyperlink) and eventually begin modifying your Python pocket book:

jupyter pocket book

(and click on on nbs, and click on on 00_core.ipynb).

Edit the pocket book as described, then put together your adjustments:

nbdev_prepare

Edit index.ipynb as described, then push your adjustments to GitHub:

git add .
git commit -m'Add `say_hello`; replace index'
git push

If you’d like, you’ll be able to go forward and add superior options.

IDG

As you’ve got seen, particularly if you happen to’ve labored via the tutorial your self, nbdev can allow a extremely productive Python growth workflow in notebooks, working seamlessly with a GitHub repository and the Quarto documentation proven on the pages of GitHub. If you have not labored via the tutorial but, what are you ready for?

—

Contact: quick.ai, https://nbdev.quick.ai/

Price: Free open supply beneath Apache License 2.0.

Platforms: macOS, Linux, and most Unix-style working techniques. It really works on Home windows beneath WSL, however not beneath cmd or PowerShell.