Converting and plotting¶
Reading a published book covered addressing a book and pulling pandas out of it. This guide covers the rest of the converter family, the local content cache, and getting a chart on screen.
from bookshelf import Bookshelf
bs = Bookshelf()
entry = bs.book("rcmip-emissions", "v5.1.0")["magicc"]
The converter family¶
Every converter reads the whole resource and takes the same local year window and filters. They differ only in what they hand back.
as_df()returns wide indexed pandas.as_long_df()returns tidy pandas.as_polars()returns a Polars DataFrame.as_arrow()returns a PyArrow Table.as_scmrun()returns anscmdata.ScmRun.
as_polars() needs Polars installed. as_scmrun() needs the scmrun extra.
uv add polars "bookshelf[scmrun]"
selection = dict(region="World", variable="Emissions|CO2", year_min=2000, year_max=2100)
entry.as_polars(**selection).shape
(22, 108)
entry.as_arrow(**selection).schema.names[:8]
['2000', '2001', '2002', '2003', '2004', '2005', '2006', '2007']
The optional imports are resolved before any request is made, so a missing Polars fails immediately rather than after downloading data.
Working in scmdata¶
as_scmrun() is the route into the wider Climate Resource tooling. ScmRun requires region, unit, variable, model and scenario to be present, so those index dimensions have to be intact. The converters filter locally and never drop a dimension, so that holds.
run = entry.as_scmrun(year_min=1900, year_max=2100)
run
/home/runner/work/bookshelf/bookshelf/.venv/lib/python3.12/site-packages/scmdata/database/_database.py:9: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html import tqdm.autonotebook as tqdman
<ScmRun (timeseries: 1683, timepoints: 201)> Time: Start: 1900-01-01T00:00:00 End: 2100-01-01T00:00:00 Meta: activity_id mip_era model region scenario \ 0 ZECMIP CMIP6 idealised World esm-bell-1000PgC 1 ZECMIP CMIP6 idealised World esm-bell-2000PgC 2 ZECMIP CMIP6 idealised World esm-bell-750PgC 3 not_applicable CMIP5 AIM World rcp60 4 not_applicable CMIP5 IMAGE World rcp26 ... ... ... ... ... ... 1678 not_applicable CMIP6 REMIND-MAGPIE World|R5.2REF ssp534-over 1679 not_applicable CMIP6 REMIND-MAGPIE World|R5.2REF ssp585 1680 not_applicable CMIP6 idealised World esm-pi-CO2pulse 1681 not_applicable CMIP6 idealised World esm-pi-cdr-pulse 1682 not_applicable CMIP6 idealised World esm-piControl unit variable 0 Mt BC/yr Emissions|BC 1 Mt BC/yr Emissions|BC 2 Mt BC/yr Emissions|BC 3 Mt BC/yr Emissions|BC 4 Mt BC/yr Emissions|BC ... ... ... 1678 Mt VOC/yr Emissions|VOC 1679 Mt VOC/yr Emissions|VOC 1680 Mt VOC/yr Emissions|VOC 1681 Mt VOC/yr Emissions|VOC 1682 Mt VOC/yr Emissions|VOC [1683 rows x 7 columns]
From here the usual scmdata vocabulary applies.
co2 = run.filter(variable="Emissions|CO2", region="World")
sorted(co2.get_unique_meta("scenario"))[:8]
['esm-bell-1000PgC', 'esm-bell-2000PgC', 'esm-bell-750PgC', 'esm-pi-CO2pulse', 'esm-pi-cdr-pulse', 'esm-piControl', 'historical', 'historical-cmip5']
Plotting¶
ScmRun carries its own plotting helpers.
from matplotlib import pyplot as plt
fig, ax = plt.subplots(figsize=(10, 5), dpi=120)
co2.filter(scenario=["ssp119", "ssp245", "ssp585"], year=range(1990, 2101)).lineplot(hue="scenario", ax=ax)
ax.set_title("RCMIP CO2 emissions by scenario")
plt.tight_layout()
Pandas works just as well when scmdata is not wanted.
wide = entry.query(
region="World",
variable="Emissions|CO2",
year_min=1990,
year_max=2100,
drop_constant=True,
)
fig, ax = plt.subplots(figsize=(10, 5), dpi=120)
wide.T.plot(ax=ax, legend=False)
ax.set_title("The same data straight from pandas")
plt.tight_layout()
Files and the content cache¶
The converters hand back a frame. To get the stored file itself, use fetch() for bytes or as_path() for a local path.
Both verify the declared SHA256 before handing anything back. A mismatch raises HashMismatchError rather than returning suspect data. The verified bytes land in a local content cache, so a second call for the same resource does no network work. The cache also remembers each resource's hash and every pinned edition you resolve, so bs.book(volume, version, edition=n)[name].as_path() makes no request at all once warm. A remembered edition is checked with one request after a day, in case it was retracted. Tune that with Bookshelf(book_ttl=seconds) or BOOKSHELF_CACHE_BOOK_TTL. Only the latest edition lookup, bs.book(volume, version), always asks the platform.
path = entry.as_path()
path.stat().st_size
1552066
The cache is content addressed and shared across every book that points at the same bytes. Manage it from the command line.
bookshelf cache path
bookshelf cache clear