Notebook Driven Development
Learn to create reproducible research with data science notebooks, hvPlot visualizations, and automated multi-notebook pipelines using PyDoIt and Papermill.
Authors
This course offers researchers and graduate students a practical, hands-on introduction to working with data science notebooks for scientific analysis and reproducible research. Data science notebooks provide an integrated environment where code, figures, and narrative explanations come together to support exploration, documentation, and communication of results.
Participants will learn to use tools such as hvPlot for creating informative visualizations, and explore techniques for generating presentation-ready outputs directly from their analyses. The course also introduces advanced methods using PyDoIt and Papermill to build modular, multi-notebook pipelines that can run analyses across varying parameters and datasets. By the end of the workshop, participants will be equipped to create reusable, well-documented code libraries and automate batch processing workflows. No prior programming experience is required—this course is open to researchers from all disciplines.
Credits
Installation
To run the course materials on your own machine, it is recommended that you:
- Install VSCode as your editor
- Install pixi or alternatively conda to create virtual Python environments (see the lessons on environment and package management)
- Create a dedicated folder for this course and install the virtual environment:
Download the pixi.toml file and install the environment:
pixi install --manifest-path pixi.toml
pixi shellDownload the environment.yml file and install the environment:
conda env create -f environment.yml
conda activate nddCourse Contents
Exploratory data analysis with Pandas and hvPlot on Jupyter Lab
Jupyter Notebooks in Depth
Understanding jupyter notebook structure and features
Jupyter Notebooks for Dashboards
How to build dashboard using Jupyter notebooks
Building Modular and Automated Analysis Workflows
Developing Reusable Code in Jupyter Notebooks
Modularizing notebooks
Automation in and of Jupyter Notebooks
How to automate jupyter notebooks