Skip to content

Getting started#

There are a few different ways to install and use Bookshelf. We provide these by use case below. As a short summary, if you:

By use case#

For data consumers#

bookshelf can be installed via pip:

pip install bookshelf

Fetching and using Books requires very little setup in order to start playing with data.

>> import bookshelf
>> shelf = bookshelf.BookShelf()
# Load the latest version of the MAGICC specific rcmip emissions
>> book = shelf.load("rcmip-emissions")
INFO:/home/user/.cache/bookshelf/v0.1.0/rcmip-emissions/volume.json downloaded from https://cr-prod-datasets-bookshelf.s3.us-west-2.amazonaws.com/v0.1.0/rcmip-emissions/volume.json
# On the first call this will fetch the data from the server and cache locally
>> book.timeseries("magicc")
INFO:/home/user/.cache/bookshelf/v0.1.0/rcmip-emissions/v0.0.2/magicc.csv downloaded from https://cr-prod-datasets-bookshelf.s3.us-west-2.amazonaws.com/v0.1.0/rcmip-emissions/v0.0.2/magicc.csv
<ScmRun (timeseries: 1683, timepoints: 751)>
Time:
        Start: 1750-01-01T00:00:00
        End: 2500-01-01T00:00:00
Meta:
                 activity_id mip_era        model region          scenario       unit                    variable
        0     not_applicable   CMIP5          AIM  World             rcp60   Mt BC/yr                Emissions|BC
        1     not_applicable   CMIP5          AIM  World             rcp60  Mt CH4/yr               Emissions|CH4
        2     not_applicable   CMIP5          AIM  World             rcp60   Mt CO/yr                Emissions|CO
        3     not_applicable   CMIP5          AIM  World             rcp60  Mt CO2/yr               Emissions|CO2
        4     not_applicable   CMIP5          AIM  World             rcp60  Mt CO2/yr  Emissions|CO2|MAGICC AFOLU
        ...              ...     ...          ...    ...               ...        ...                         ...
        1678  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt NH3/yr               Emissions|NH3
        1679  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt NOx/yr               Emissions|NOx
        1680  not_applicable   CMIP5  unspecified  World  historical-cmip5   Mt OC/yr                Emissions|OC
        1681  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt SO2/yr            Emissions|Sulfur
        1682  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt VOC/yr               Emissions|VOC

        [1683 rows x 7 columns]

# Subsequent calls use the result from the cache
>> book.timeseries("magicc")

For data curators#

If you wish to build/modify Books some additional dependencies are required. These can be installed using:

pip install bookshelf-producer

Building and deploying datasets is managed via Jupyter notebooks and a small yaml file that contains metadata about the dataset. These notebooks are stored as plain text Python files using the jupytext plugin for Jupyter. See notebooks/example.py for an example dataset.

Once the dataset has been developed, it can be deployed to the remote BookShelf so that other users can consume it.

The dataset can deployed using the publish CLI as shown below:

bookshelf publish my-dataset

Note

Publishing to the remote bookshelf requires valid credentials. Creating or obtaining these credentials is not covered in this documentation.

For developers#

For development, we rely on uv for all our dependency management. To get started, you will need to make sure that uv is installed (instructions here).

This project is a uv workspace, which means that it contains more than one Python package. uv commands will by default target the root bookshelf package, but if you wish to target another package you can use the --package flag.

For all of work, we use our Makefile. You can read the instructions out and run the commands by hand if you wish, but we generally discourage this because it can be error prone. In order to create your environment, run make virtual-environment.

If there are any issues, the messages from the Makefile should guide you through. If not, please raise an issue in the issue tracker.