Resources#
bookshelf.Resource #
Bases: Describable
Lean immutable resource handle for machine and provenance reads.
tracking_id instance-attribute #
The platform's id for this resource.
resource_type #
Return the canonical resource type, from memory or disk before the platform.
describe #
Return what the platform records about the resource, fetching it at most once.
content_hash #
Return the declared sha256: digest, from memory or disk before the platform.
as_df #
as_df(*, filters: Filters | None = None, year_min: int | None = None, year_max: int | None = None, server_side: bool = False, order: Order | None = None, int_years: bool = False) -> pd.DataFrame
Return every selected row as pandas, using wide indexed form for timeseries.
Values listed for one filter are alternatives, and separate filters must all hold. The year window is inclusive. order names columns to sort by, each prefixed with - to sort descending, with missing values first either way and no promised order among ties. By default the selection applies to the verified cached file, and server_side has the platform select instead, which transfers only the selected rows. Both pick the same rows, but without order the platform returns them in its own order. int_years labels a timeseries' year columns as integers rather than strings.
as_long_df #
as_long_df(*, filters: Filters | None = None, year_min: int | None = None, year_max: int | None = None, server_side: bool = False, legacy_columns: bool = False, dropna: bool = False) -> pd.DataFrame
Return tidy pandas timeseries data, with integer year and a value column.
Every series has a row for every year, missing values included. dropna leaves out the rows with no value, without ever building them, which keeps a sparse resource far smaller in memory. legacy_columns reproduces the 0.4 long format instead: a values column, a year column of YYYY-01-01 00:00:00 strings, and rows sorted by the dimensions and then the year. It is deprecated and will be removed in bookshelf 2.0.
as_polars #
as_polars(*, filters: Filters | None = None, year_min: int | None = None, year_max: int | None = None, server_side: bool = False, order: Order | None = None) -> pl.DataFrame
Return the selection as a Polars DataFrame, with the dimensions as columns.
as_arrow #
as_arrow(*, filters: Filters | None = None, year_min: int | None = None, year_max: int | None = None, server_side: bool = False, order: Order | None = None) -> pa.Table
Return the selection as a PyArrow table, with the dimensions as columns.
as_scmrun #
as_scmrun(*, filters: Filters | None = None, year_min: int | None = None, year_max: int | None = None, server_side: bool = False) -> ScmRun
Return timeseries data as an scmdata ScmRun.
scmdata rejects rows with duplicate metadata.
preview #
preview(*, limit: int = 100, filters: Filters | None = None, year_min: int | None = None, year_max: int | None = None, order: Order | None = None, top_n: int | None = None, drop_constant: bool = False) -> DataPreview
Return up to limit selected rows, selected on the platform, and whether that is all.
order names columns to sort by first, each prefixed with - to sort descending. top_n keeps the timeseries with the largest latest values, and cannot take an order. drop_constant drops the dimensions that hold one value across the returned series.
fetch #
Return verified bytes, using memory proportional to the resource size.
Use download() to write large resources to disk without loading them into memory.
download #
Stream and verify the resource, then copy it to destination and return that path.
as_path #
Stream and verify the resource, then return its path in the cache.
The cache may evict the file later, so use download() for a copy that stays.
bookshelf.DataPreview dataclass #
A bounded look at a selection, saying whether it holds every selected row.
A wide timeseries row is one series, so limit counts series rather than observations.
completeness instance-attribute #
Whether data holds every row the request matches, after any top_n.
bookshelf.ResourceInfo dataclass #
ResourceInfo(tracking_id: UUID, resource_type: ResourceType, content_hash: str, visibility: Visibility, readers: tuple[str, ...], metadata: Mapping[str, Any], discovery: Mapping[str, Any], created_at: datetime, updated_at: datetime)
bookshelf.ResourceType #
Bases: _OpenStrEnum
The kind of data a resource holds, which decides the reads it answers.
A type a newer platform adds arrives as a member of its own, so the resource stays readable as bytes rather than failing to load.
bookshelf.Visibility #
Bases: _OpenStrEnum
Who can read a book or resource.
A tier a newer platform adds arrives as a member of its own rather than an error.
bookshelf.Facets dataclass #
Facets(facets: tuple[Facet, ...], categorical_columns: tuple[str, ...], numeric_columns: tuple[str, ...], total_rows: int)
The index columns of an entry and the distinct values each one holds.
__getitem__ #
Look up one column's facet, raising KeyError for a column the entry does not index.
bookshelf.Facet dataclass #
One index column and its distinct values.
total_unique instance-attribute #
How many distinct values the whole entry holds, which values may truncate.
bookshelf.FacetValue dataclass #
One distinct value of a column, and how many series use it.
bookshelf.SeriesMetadata dataclass #
SeriesMetadata(index: tuple[Mapping[str, Any], ...], columns: tuple[str, ...], total_rows: int, limit: int, offset: int, has_more: bool)
One page of an entry's series, each described by its index values alone.