Finding volumes and books¶

Reading a published book assumed you already knew the coordinates. This guide covers the step before that: working out what is on the shelf at all.

Discovery needs no credentials, though an authenticated caller sees more, because private volumes belonging to your organisation join the results.

In [1]:

  Copied!     
 
import os

os.environ.setdefault("BOOKSHELF_URL", "https://bookshelf-staging.ovh.climateresource.com.au")

from bookshelf import Bookshelf

bs = Bookshelf()
import os os.environ.setdefault("BOOKSHELF_URL", "https://bookshelf-staging.ovh.climateresource.com.au") from bookshelf import Bookshelf bs = Bookshelf()

Listing the catalogue¶

search_volumes() with no arguments returns everything you can see.

In [2]:

  Copied!     
 
catalogue = bs.search_volumes()
len(catalogue.items), catalogue.has_more
catalogue = bs.search_volumes() len(catalogue.items), catalogue.has_more

Out[2]:

(16, False)

Each result carries enough to decide whether it is worth opening, including the latest version and edition, so a summary needs no further requests.

In [3]:

  Copied!     
 
for volume in sorted(catalogue.items, key=lambda item: item.name)[:8]:
    licence = volume.discovery.license.root if volume.discovery.license else "-"
    print(f"{volume.name:32} {volume.latest_version or '-':16} {licence}")
for volume in sorted(catalogue.items, key=lambda item: item.name)[:8]: licence = volume.discovery.license.root if volume.discovery.license else "-" print(f"{volume.name:32} {volume.latest_version or '-':16} {licence}")

ceds                             v2021_04_21      CC-BY
demo-lineage                     v1.0.0           CC-BY-4.0
demo-revisions                   v1.0.0           CC-BY-4.0
demo-types                       v1.0.0           CC-BY-4.0
demo-versions                    v2.0.0           CC-BY-4.0
demo-visibility                  v1.0.0-public    CC-BY-4.0
gdp-ndc-tool                     2023-10          Proprietary
imf-weo                          v202603          IMF-Data

Searching¶

The first argument is free text over the name, title and summary.

In [4]:

  Copied!     
 
found = bs.search_volumes("emissions")
[volume.name for volume in found.items]
found = bs.search_volumes("emissions") [volume.name for volume in found.items]

Out[4]:

['ceds',
 'primap-hist',
 'primap-hist-energy-downscaled',
 'rcmip-emissions',
 'un-br-ctf']

Filters narrow it further, and they combine with AND rather than OR. The vocabulary matches the discovery profile a volume is published with: topic, keyword, region, publisher, license, coverage_year, resource_type and deprecated.

In [5]:

  Copied!     
 
[volume.name for volume in bs.search_volumes(license="CC-BY-4.0").items]
[volume.name for volume in bs.search_volumes(license="CC-BY-4.0").items]

Out[5]:

['demo-lineage',
 'demo-revisions',
 'demo-types',
 'demo-versions',
 'demo-visibility',
 'primap-hist',
 'wb-population']

deprecated=False is worth knowing about. A superseded volume stays readable so old analyses keep working, so excluding it is how you avoid building something new on a dataset that has moved on.

In [6]:

  Copied!     
 
active = bs.search_volumes("emissions", deprecated=False)
[volume.name for volume in active.items]
active = bs.search_volumes("emissions", deprecated=False) [volume.name for volume in active.items]

Out[6]:

['ceds',
 'primap-hist',
 'primap-hist-energy-downscaled',
 'rcmip-emissions',
 'un-br-ctf']

Paging¶

The response carries total, limit, offset and has_more. Read has_more rather than comparing counts, and page with offset.

In [7]:

  Copied!     
 
page = bs.search_volumes(limit=5)
print(f"showing {len(page.items)} of {page.total}, more to come: {page.has_more}")

if page.has_more:
    following = bs.search_volumes(limit=5, offset=5)
    print(f"next page starts at {following.items[0].name}")
page = bs.search_volumes(limit=5) print(f"showing {len(page.items)} of {page.total}, more to come: {page.has_more}") if page.has_more: following = bs.search_volumes(limit=5, offset=5) print(f"next page starts at {following.items[0].name}")

showing 5 of 16, more to come: True
next page starts at demo-visibility

Every book in a volume¶

A volume holds many books, one per version and edition. list_books() returns all of them, oldest first, walking the pages itself so you do not have to.

In [8]:

  Copied!     
 
books = bs.list_books("primap-hist")
[f"{book.version}_e{book.edition:03}" for book in books]
books = bs.list_books("primap-hist") [f"{book.version}_e{book.edition:03}" for book in books]

Out[8]:

['v2.3_e005',
 'v2.3.1_e005',
 'v2.4_e005',
 'v2.4.1_e005',
 'v2.4.2_e005',
 'v2.5_e005',
 'v2.5.1_e005',
 'v2.6_e005',
 'v2.7_e005']

Versions sort component by component, comparing numeric runs as numbers, so v2.10 lands after v2.9 instead of before it. The last entry is therefore the newest book, which is what bs.book(volume, version) resolves when you leave edition= off.

In [9]:

  Copied!     
 
newest = books[-1]
newest.version, newest.edition, newest.status
newest = books[-1] newest.version, newest.edition, newest.status

Out[9]:

('v2.7', 5, <BookStatus.published: 'published'>)

Pass status= to look at something other than published books. Drafts are only visible to the organisation that owns them.

From discovery to data¶

Discovery hands back coordinates, and reading a book takes it from there.

In [10]:

  Copied!     
 
entry = bs.book("primap-hist", newest.version)["by_region"]
entry.as_df(year_min=2018, year_max=2020, top_n=5).iloc[:5, :3]
entry = bs.book("primap-hist", newest.version)["by_region"] entry.as_df(year_min=2018, year_max=2020, top_n=5).iloc[:5, :3]

Out[10]:

		2018	2019	2020
category	gwp_context
0	AR6GWP100	51119499.0	51983668.0	49955598.0
	AR5GWP100	50883661.0	51739829.0	49701639.0
	AR4GWP100	50351551.0	51196267.0	49186145.0
M.0.EL	AR6GWP100	49587740.0	50170250.0	48553159.0
M.0.EL	AR5GWP100	49353822.0	49928687.0	48301725.0

From the command line¶

The same catalogue is searchable without writing any Python, which is often faster when you are just orienting yourself.

bookshelf search emissions
bookshelf search --licence CC-BY-4.0 --no-deprecated
bookshelf search --facets            # the valid values for each filter
bookshelf show primap-hist           # one volume, its versions and editions
bookshelf show primap-hist@v2.6/by_region

Every command takes --json for a machine readable form.