Command line#

bookshelf is the command line interface installed with the package. It is a machine interface first, so scripts, CI jobs and agents can drive it:

Payload goes to stdout and diagnostics go to stderr.
Nothing changes when a terminal is attached, so piped output matches what you see.
The exit code says what went wrong, as listed under exit codes.

Addressing covers how a command names a volume, a book or an entry. Configuration lists the environment variables it reads.

Stability#

The CLI is outside the semantic versioning promise that covers the Python API. Commands, flags, --json keys and exit codes may change in a minor release. Every such change is listed in the changelog. Pin the bookshelf version a script depends on, and read the changelog before raising it.

Output#

Every command that reports something takes --json. Without it the same fields print as aligned rows, under labels derived from the JSON keys.

A command that reports one thing prints one JSON document.
A listing (search, auth list) prints one JSON object per line, and nothing when it is empty.
search --json ends with one {"page": {...}} line carrying offset, limit, returned, total and next_offset, which is null on the last page.
Errors are plain text on stderr, with or without --json.

The --json keys follow a few rules:

A byte count ends in _bytes: size_bytes for one file, total_size_bytes, max_size_bytes and freed_size_bytes for the cache. The flag that sets one is spelt the same way, such as cache prune --max-size-bytes.
A hash is content_hash, written sha256:<hex>.
A count ends in _count, such as resource_count and entry_count.
A named file in a book is an entry, so a book lists its entries.
A moment is UTC ISO 8601 to the second, with a Z suffix.

bookshelf auth token and bookshelf cache path print a bare value by default, so they drop straight into shell interpolation.

Exit codes#

Code	Meaning	Typical errors
0	success
1	any other failure, including a bug worth reporting	`BookshelfError` with no more specific code
2	usage: bad arguments, a malformed address, or unusable local setup	`ConfigurationError`, `AuthConfigurationError`, `RequestValidationError`, `SelectionError`
3	no accepted credential: log in, or refresh the one in play	`AuthenticationError`, `AuthenticationRequiredError`
4	the credential lacks a permission	`ForbiddenError`
5	the volume, book, entry or resource does not exist	`NotFoundError` and its subclasses
6	network, gateway or server failure, worth retrying	`TransportError`, `ServerError`, `RateLimitError`, `GatewayError`
7	the bundle is malformed or refused	`InvalidBundleError`
8	the request conflicts with what the platform holds	`ConflictError`
9	the server answered outside the API contract: upgrade bookshelf	`UnexpectedResponseError`, `ContractError`

Some cases worth knowing:

A bundle directory that does not exist is a usage error (2), where one that exists but is malformed is 7.
A build that cannot read its code ref from git, for example outside a repository, is a usage error (2).
Incomplete credential settings are a usage error (2). For example BOOKSHELF_CLIENT_ID and BOOKSHELF_CLIENT_SECRET without BOOKSHELF_TOKEN_URL, or auth login against a deployment with no WorkOS client ID and no BOOKSHELF_WORKOS_CLIENT_ID. BOOKSHELF_CLIENT_ID without its secret is ignored rather than refused.
A credentials file written by a newer bookshelf is read but never written, so a login or logout against it exits 2 before anything is sent.
An error status the API contract does not declare for the operation, such as a proxy's JSON 403, exits by its status: 400 and 422 exit 2, 401 exits 3, 403 exits 4, 404 exits 5 and 409 exits 8.
A rate limit the retries could not wait out is a network failure (6).

bookshelf --help prints the same table.

Credentials file#

bookshelf auth login stores credentials in a file under your user configuration directory. Its format is private and may change in any release, so do not read or write it directly. Use bookshelf auth list --json to see what is stored, and bookshelf auth token to get a usable token.

A file written by a newer bookshelf is read for whatever this version understands, and never overwritten, so going back to an older version cannot log the newer one out. Logging in or out with the older version fails with exit code 2 until you upgrade. The older version also never refreshes a token from that file, because refreshing spends a secret the newer one needs. So auth list marks an expired token from that file as needs_login.

Command reference#

Hidden commands, such as preview which the feedstock CI workflow runs, are left out.

`bookshelf`#

Bookshelf data platform CLI.

$ bookshelf [OPTIONS] COMMAND [ARGS]...

Options:

--api-url <str>: Deployment to act against. Defaults to $BOOKSHELF_URL.
--version: Print the installed version and exit.

`bookshelf search`#

Search volumes with free text and filters, which combine with AND.

$ bookshelf search [OPTIONS] [query]

Arguments:

query: Free text over name, title and summary. Optional.

Options:

--topic <str>: Topic the volume must carry.
--keyword <str>: Keyword the volume must carry.
--region <str>: Region the volume must cover.
--publisher <str>: Publisher organisation.
--licence, --license <str>: SPDX licence identifier.
--coverage-year <int>: Year the volume's data must cover.
--type <timeseries|geospatial|tabular|document|binary|figure>: Resource type the volume contains.
--deprecated / --no-deprecated: Restrict to deprecated or to active volumes. Omitted means both.
--limit <int range>: Maximum results. [default: 20; 1<=x<=1000]
--offset <int range>: Results to skip. [default: 0; x>=0]
--facets: List every valid filter value instead of searching. Takes no query or filters.
--json: One JSON object per result, then one 'page' object with the totals and next offset.

`bookshelf show`#

Resolve one address and describe what is there, at whatever depth it is given.

$ bookshelf show [OPTIONS] {ADDRESS}

Arguments:

ADDRESS: volume[@version[_eNNN]][/entry], or the bookshelf:// reference naming the same thing. [required]

Options:

--json: Emit the description as JSON.

`bookshelf record`#

Execute a build file and record it into a reviewable bundle.

$ bookshelf record [OPTIONS] [build]

Arguments:

build: Standalone Jupytext build file. Defaults to the recipe's notebook.

Options:

--recipe <path>: Sectioned Bookshelf recipe. [default: bookshelf.yaml]
--bundle <path>: Bundle directory to write. [default: bundle]
--book VERSION: Required. Version to build, naming a book under 'books:' in the recipe.
-p, --parameter KEY=VALUE: Value for a top-level assignment in the build file, read as YAML. Repeatable.
--force: Replace an existing bundle directory.
--json: Emit the summary as JSON.

`bookshelf validate`#

Assert a recorded bundle is a replayable published book.

$ bookshelf validate [OPTIONS] [bundle]

Arguments:

bundle: Bundle directory to validate. [default: bundle]

Options:

--json: Emit the summary as JSON.

`bookshelf publish`#

Replay a recorded bundle to publish it, converging on one edition.

$ bookshelf publish [OPTIONS] [bundle]

Arguments:

bundle: Bundle directory to replay. [default: bundle]

Options:

--dry-run: Report what would be sent without sending it. Resolves no edition.
--json: Emit the summary as JSON.

`bookshelf discard`#

Delete a draft edition, so a publish that failed validation leaves no debris.

Only a draft can be discarded. A published book is protected by the API, and the CLI refuses one before it asks.

$ bookshelf discard [OPTIONS] {address}

Arguments:

address: Draft edition to discard, as volume@version_eNNN. [required]

Options:

--json: Emit the outcome as JSON.

`bookshelf upload`#

Upload a file as a standalone input and print the bookshelf URI that names it.

The URI is bookshelf://sha256/<hex>, which a recipe declares under resources: as its uri. The file is readable by your organisation alone.

$ bookshelf upload [OPTIONS] {file}

Arguments:

file: File to upload. [required]

Options:

--type <timeseries|geospatial|tabular|document|binary|figure>: Resource type the file registers under. Never inferred from the name. [required]
--name <str>: Resource name. Defaults to the file name, flattened.
--description <str>: What the file is.
--tag <str>: Catalogue tag. Repeatable.
--json: Emit the outcome as JSON.

`bookshelf auth`#

Manage authentication for the Bookshelf API.

$ bookshelf auth [OPTIONS] COMMAND [ARGS]...

`bookshelf auth login`#

Log in through WorkOS.

$ bookshelf auth login [OPTIONS]

Options:

--no-browser: For a box that cannot open a browser.
--json: Emit the credential summary as JSON.

`bookshelf auth token`#

Print the current access token to stdout and nothing else.

$ bookshelf auth token [OPTIONS]

Options:

--json: Emit the token and its deployment as JSON.

`bookshelf auth whoami`#

Report the identity in play and which resolution step supplied it.

$ bookshelf auth whoami [OPTIONS]

Options:

--offline: Report the stored credential without calling the API.
--json: Emit the report as JSON.

`bookshelf auth logout`#

Clear stored credentials.

$ bookshelf auth logout [OPTIONS]

Options:

--all: Clear every stored login for every deployment.
--json: Emit the outcome as JSON.

`bookshelf auth list`#

List every stored login.

$ bookshelf auth list [OPTIONS]

Options:

--json: Emit one JSON object per login.

`bookshelf cache`#

Manage the local content cache.

$ bookshelf cache [OPTIONS] COMMAND [ARGS]...

`bookshelf cache info`#

Show cache size, entry count, age range and the configured cap.

$ bookshelf cache info [OPTIONS]

Options:

--json: Emit the summary as JSON.

`bookshelf cache prune`#

Evict oldest entries until the cache fits the cap.

$ bookshelf cache prune [OPTIONS]

Options:

--max-size-bytes <int range>: Cap to prune the cache down to. [default: 5368709120; x>=0]
--json: Emit the result as JSON.

`bookshelf cache clear`#

Remove everything. Requires --yes, so a cache is never wiped by accident.

$ bookshelf cache clear [OPTIONS]

Options:

--yes: Confirm removal of every cached entry.
--json: Emit the result as JSON.

`bookshelf cache path`#

Print the cache directory as a bare string, for shell interpolation.

$ bookshelf cache path [OPTIONS]

Options:

--json: Emit the path as JSON.

`bookshelf volume`#

Create, update and delete volumes.

$ bookshelf volume [OPTIONS] COMMAND [ARGS]...

`bookshelf volume create`#

Create a volume, which a first publish into a new collection needs.

Creating needs WRITE and deleting needs ADMIN, so you may not be able to delete what you create here.

$ bookshelf volume create [OPTIONS] {name}

Arguments:

name: Volume name, in alphanumerics, hyphens and underscores. [required]

Options:

--licence, --license <str>: SPDX licence identifier. [required]
--description <str>: Long-form description.
--author <str>: Name of somebody who made the data. Repeatable.
--maintainer <str>: Name of somebody who maintains the feedstock. Repeatable.
--metadata <path>: JSON file holding arbitrary volume metadata.
--json: Emit the volume as JSON.

`bookshelf volume update`#

Update a volume's metadata. Each field given replaces what is there, and the licence is fixed.

$ bookshelf volume update [OPTIONS] {name}

Arguments:

name: Volume to update. [required]

Options:

--description <str>: Long-form description.
--author <str>: Name of somebody who made the data. Repeatable.
--maintainer <str>: Name of somebody who maintains the feedstock. Repeatable.
--metadata <path>: JSON file holding arbitrary volume metadata.
--json: Emit the volume as JSON.

`bookshelf volume delete`#

Delete a volume and every book in it. This needs ADMIN, where creation needs WRITE.

$ bookshelf volume delete [OPTIONS] {name}

Arguments:

name: Volume to delete, with every book in it. [required]

Options:

--yes: Confirm the deletion, which is not reversible.
--json: Emit the outcome as JSON.

Command line#

Stability#

Output#

Exit codes#

Credentials file#

Command reference#

bookshelf#

bookshelf search#

bookshelf show#

bookshelf record#

bookshelf validate#

bookshelf publish#

bookshelf discard#

bookshelf upload#

bookshelf auth#

bookshelf auth login#

bookshelf auth token#

bookshelf auth whoami#

bookshelf auth logout#

bookshelf auth list#

bookshelf cache#

bookshelf cache info#

bookshelf cache prune#

bookshelf cache clear#

bookshelf cache path#

bookshelf volume#

bookshelf volume create#

bookshelf volume update#

bookshelf volume delete#

`bookshelf`#

`bookshelf search`#

`bookshelf show`#

`bookshelf record`#

`bookshelf validate`#

`bookshelf publish`#

`bookshelf discard`#

`bookshelf upload`#

`bookshelf auth`#

`bookshelf auth login`#

`bookshelf auth token`#

`bookshelf auth whoami`#

`bookshelf auth logout`#

`bookshelf auth list`#

`bookshelf cache`#

`bookshelf cache info`#

`bookshelf cache prune`#

`bookshelf cache clear`#

`bookshelf cache path`#

`bookshelf volume`#

`bookshelf volume create`#

`bookshelf volume update`#

`bookshelf volume delete`#