Converting and plotting¶

Reading a published book covered addressing a book and pulling pandas out of it. This guide covers the rest of the converter family, the local content cache, and getting a chart on screen.

In [1]:

  Copied!     
 
from bookshelf import Bookshelf

bs = Bookshelf()
entry = bs.book("rcmip-emissions", "v5.1.0")["magicc"]
from bookshelf import Bookshelf bs = Bookshelf() entry = bs.book("rcmip-emissions", "v5.1.0")["magicc"]

The converter family¶

Every converter returns every selected row and takes the same year window and filters. They differ only in what they hand back.

as_df() returns wide indexed pandas.
as_long_df() returns tidy pandas.
as_polars() returns a Polars DataFrame.
as_arrow() returns a PyArrow Table.
as_scmrun() returns an scmdata.ScmRun.

as_polars() needs Polars installed. as_scmrun() needs the scmrun extra.

uv add polars "bookshelf[scmrun]"

In [2]:

  Copied!     
 
selection = dict(filters={"region": "World", "variable": "Emissions|CO2"}, year_min=2000, year_max=2100)

entry.as_polars(**selection).shape
selection = dict(filters={"region": "World", "variable": "Emissions|CO2"}, year_min=2000, year_max=2100) entry.as_polars(**selection).shape

Out[2]:

(22, 108)

In [3]:

  Copied!     
 
entry.as_arrow(**selection).schema.names[:8]
entry.as_arrow(**selection).schema.names[:8]

Out[3]:

['2000', '2001', '2002', '2003', '2004', '2005', '2006', '2007']

The optional imports are resolved before any request is made, so a missing Polars fails immediately rather than after downloading data.

Working in scmdata¶

as_scmrun() is the route into the wider Climate Resource tooling. ScmRun requires region, unit, variable, model and scenario to be present, so those index dimensions have to be intact. The converters never drop a dimension, so that holds.

In [4]:

  Copied!     
 
run = entry.as_scmrun(year_min=1900, year_max=2100)
run
run = entry.as_scmrun(year_min=1900, year_max=2100) run

/home/runner/work/bookshelf/bookshelf/.venv/lib/python3.12/site-packages/scmdata/database/_database.py:9: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  import tqdm.autonotebook as tqdman

Out[4]:

<ScmRun (timeseries: 1683, timepoints: 201)>
Time:
	Start: 1900-01-01T00:00:00
	End: 2100-01-01T00:00:00
Meta:
	         activity_id mip_era        model region          scenario       unit  \
	0     not_applicable   CMIP5          AIM  World             rcp60   Mt BC/yr   
	1     not_applicable   CMIP5          AIM  World             rcp60  Mt CH4/yr   
	2     not_applicable   CMIP5          AIM  World             rcp60   Mt CO/yr   
	3     not_applicable   CMIP5          AIM  World             rcp60  Mt CO2/yr   
	4     not_applicable   CMIP5          AIM  World             rcp60  Mt CO2/yr   
	...              ...     ...          ...    ...               ...        ...   
	1678  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt NH3/yr   
	1679  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt NOx/yr   
	1680  not_applicable   CMIP5  unspecified  World  historical-cmip5   Mt OC/yr   
	1681  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt SO2/yr   
	1682  not_applicable   CMIP5  unspecified  World  historical-cmip5  Mt VOC/yr   
	
	                        variable  
	0                   Emissions|BC  
	1                  Emissions|CH4  
	2                   Emissions|CO  
	3                  Emissions|CO2  
	4     Emissions|CO2|MAGICC AFOLU  
	...                          ...  
	1678               Emissions|NH3  
	1679               Emissions|NOx  
	1680                Emissions|OC  
	1681            Emissions|Sulfur  
	1682               Emissions|VOC  
	
	[1683 rows x 7 columns]

From here the usual scmdata vocabulary applies.

In [5]:

  Copied!     
 
co2 = run.filter(variable="Emissions|CO2", region="World")
sorted(co2.get_unique_meta("scenario"))[:8]
co2 = run.filter(variable="Emissions|CO2", region="World") sorted(co2.get_unique_meta("scenario"))[:8]

Out[5]:

['esm-bell-1000PgC',
 'esm-bell-2000PgC',
 'esm-bell-750PgC',
 'esm-pi-CO2pulse',
 'esm-pi-cdr-pulse',
 'esm-piControl',
 'historical',
 'historical-cmip5']

Plotting¶

ScmRun carries its own plotting helpers. lineplot() draws with seaborn and matplotlib, which the figures extra installs.

uv add "bookshelf[scmrun,figures]"

In [6]:

  Copied!     
 
from matplotlib import pyplot as plt

fig, ax = plt.subplots(figsize=(10, 5), dpi=120)
co2.filter(scenario=["ssp119", "ssp245", "ssp585"], year=range(1990, 2101)).lineplot(hue="scenario", ax=ax)
ax.set_title("RCMIP CO2 emissions by scenario")
plt.tight_layout()
from matplotlib import pyplot as plt fig, ax = plt.subplots(figsize=(10, 5), dpi=120) co2.filter(scenario=["ssp119", "ssp245", "ssp585"], year=range(1990, 2101)).lineplot(hue="scenario", ax=ax) ax.set_title("RCMIP CO2 emissions by scenario") plt.tight_layout()

No description has been provided for this image

Pandas works just as well when scmdata is not wanted.

In [7]:

  Copied!     
 
wide = entry.as_df(
    filters={"region": "World", "variable": "Emissions|CO2"},
    year_min=1990,
    year_max=2100,
)
fig, ax = plt.subplots(figsize=(10, 5), dpi=120)
wide.T.plot(ax=ax, legend=False)
ax.set_title("The same data straight from pandas")
plt.tight_layout()
wide = entry.as_df( filters={"region": "World", "variable": "Emissions|CO2"}, year_min=1990, year_max=2100, ) fig, ax = plt.subplots(figsize=(10, 5), dpi=120) wide.T.plot(ax=ax, legend=False) ax.set_title("The same data straight from pandas") plt.tight_layout()

Files and the content cache¶

The converters hand back a frame. To get the stored file itself, use fetch() for bytes or download(destination) for a copy on disk. as_path() returns the cached file itself, which the cache may evict later.

Both verify the declared SHA256 before handing anything back. A mismatch raises HashMismatchError rather than returning suspect data. The verified bytes land in a local content cache, so a second call for the same resource does no network work. The converters read through the same cache, so as_df() after as_path() downloads nothing. The cache also remembers each resource's hash and every pinned edition you resolve, so bs.book(volume, version, edition=n)[name].as_path() makes no request at all once warm. A remembered edition is checked with one request after a day, in case it was retracted. Tune that with Bookshelf(book_ttl=seconds) or BOOKSHELF_CACHE_BOOK_TTL. Only the latest edition lookup, bs.book(volume, version), always asks the platform.

In [8]:

  Copied!     
 
path = entry.as_path()
path.stat().st_size
path = entry.as_path() path.stat().st_size

Out[8]:

The cache is content addressed and shared across every book that points at the same bytes. Manage it from the command line.

bookshelf cache path
bookshelf cache clear --yes