Converting and plotting¶
Reading a published book covered addressing a book and pulling pandas out of it. This guide covers the rest of the converter family, the local content cache, and getting a chart on screen.
import os
os.environ.setdefault("BOOKSHELF_URL", "https://bookshelf-staging.ovh.climateresource.com.au")
from bookshelf import Bookshelf
bs = Bookshelf()
entry = bs.book("rcmip-emissions", "v5.1.0")["magicc"]
The converter family¶
Every converter takes the same trimming and filter arguments. They differ only in what they hand back.
as_df()returns wide indexed pandas.as_long_df()returns tidy pandas.as_polars()returns a Polars DataFrame.as_arrow()returns a PyArrow Table.as_scmrun()returns anscmdata.ScmRun.
Polars and PyArrow need the dataframes extra. as_scmrun() needs the scmrun extra.
uv add "bookshelf[dataframes,scmrun]"
selection = dict(region="World", variable="Emissions|CO2", year_min=2000, year_max=2100)
entry.as_polars(**selection).shape
(22, 105)
entry.as_arrow(**selection).schema.names[:8]
['2000', '2001', '2002', '2003', '2004', '2005', '2006', '2007']
The optional imports are resolved before any request is made, so a missing extra fails immediately rather than after downloading data.
Working in scmdata¶
as_scmrun() is the route into the wider Climate Resource tooling. ScmRun requires region, unit, variable, model and scenario to be present, so the query has to leave those index dimensions intact.
A year window and row filters are safe. top_n and limit are not, because the server drops index columns that carry a single value across the trimmed result.
run = entry.as_scmrun(year_min=1900, year_max=2100)
run
/home/runner/work/bookshelf/bookshelf/.venv/lib/python3.12/site-packages/scmdata/database/_database.py:9: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html import tqdm.autonotebook as tqdman
<ScmRun (timeseries: 1683, timepoints: 201)> Time: Start: 1900-01-01T00:00:00 End: 2100-01-01T00:00:00 Meta: activity_id mip_era model region scenario unit \ 0 not_applicable CMIP5 AIM World rcp60 Mt BC/yr 1 not_applicable CMIP5 AIM World rcp60 Mt CH4/yr 2 not_applicable CMIP5 AIM World rcp60 Mt CO/yr 3 not_applicable CMIP5 AIM World rcp60 Mt CO2/yr 4 not_applicable CMIP5 AIM World rcp60 Mt CO2/yr ... ... ... ... ... ... ... 1678 not_applicable CMIP5 unspecified World historical-cmip5 Mt NH3/yr 1679 not_applicable CMIP5 unspecified World historical-cmip5 Mt NOx/yr 1680 not_applicable CMIP5 unspecified World historical-cmip5 Mt OC/yr 1681 not_applicable CMIP5 unspecified World historical-cmip5 Mt SO2/yr 1682 not_applicable CMIP5 unspecified World historical-cmip5 Mt VOC/yr variable 0 Emissions|BC 1 Emissions|CH4 2 Emissions|CO 3 Emissions|CO2 4 Emissions|CO2|MAGICC AFOLU ... ... 1678 Emissions|NH3 1679 Emissions|NOx 1680 Emissions|OC 1681 Emissions|Sulfur 1682 Emissions|VOC [1683 rows x 7 columns]
From here the usual scmdata vocabulary applies.
co2 = run.filter(variable="Emissions|CO2", region="World")
sorted(co2.get_unique_meta("scenario"))[:8]
['esm-bell-1000PgC', 'esm-bell-2000PgC', 'esm-bell-750PgC', 'esm-pi-CO2pulse', 'esm-pi-cdr-pulse', 'esm-piControl', 'historical', 'historical-cmip5']
Plotting¶
ScmRun carries its own plotting helpers.
from matplotlib import pyplot as plt
fig, ax = plt.subplots(figsize=(10, 5), dpi=120)
co2.filter(scenario=["ssp119", "ssp245", "ssp585"], year=range(1990, 2101)).lineplot(hue="scenario", ax=ax)
ax.set_title("RCMIP CO2 emissions by scenario")
plt.tight_layout()
Pandas works just as well when scmdata is not wanted.
wide = entry.as_df(
region="World",
variable="Emissions|CO2",
year_min=1990,
year_max=2100,
drop_constant=True,
)
fig, ax = plt.subplots(figsize=(10, 5), dpi=120)
wide.T.plot(ax=ax, legend=False)
ax.set_title("The same data straight from pandas")
plt.tight_layout()
Files and the content cache¶
The converters go through the query API, which trims and filters on the server. To get the stored file itself, use fetch() for bytes or as_path() for a local path.
Both verify the declared SHA256 before handing anything back. A mismatch raises HashMismatchError rather than returning suspect data. The verified bytes land in a local content cache, so a second call for the same resource does no network work.
path = entry.as_path()
path.stat().st_size
3489639
The cache is content addressed and shared across every book that points at the same bytes. Manage it from the command line.
bookshelf cache path
bookshelf cache clear
Warning: Prefer the book entry for timeseries
entry.as_resource()drops the book context and returns the lean resource handle. That handle accepts the richercol.opfilter grammar, butas_df()on a lean timeseries resource does not currently reassemble the year columns correctly. Read timeseries through the book entry, as this guide does.