wags_tails.base_source#

Define core data source classes.

All source classes should inherit - directly or indirectly - from DataSource. Each class defined here is an abstract base class and cannot be instantiated directly.

exception wags_tails.base_source.RemoteDataError[source]#

Raise when unable to parse, navigate, or extract information from a remote resource, like a data API

class wags_tails.base_source.DataSource(data_dir=None, silent=True)[source]#

Abstract base class for a data source.

_src_name: str[source]#
_filetype: str[source]#
_versioned: bool = True[source]#
__init__(data_dir=None, silent=True)[source]#

Set common class parameters.

Parameters:

  • data_dir (Optional[Path]) – direct location to store data files in, if specified. See get_data_dir() in the storage_utils module for further configuration details.

  • silent (bool) – if True, don’t print any info/updates to console

abstract _get_latest_version()[source]#

Acquire value of latest data version.

Return type:

str

Returns:

latest version value

abstract _download_data(version, outfile)[source]#

Download data file to specified location.

Parameters:

  • version (str) – version to acquire

  • outfile (Path) – location and filename for final data file

Return type:

None

get_latest(from_local=False, force_refresh=False)[source]#

Get path to latest version of data.

Provides logic for both versioned and unversioned data here, rather than in the UnversionedDataSource child class, to support CustomData instances regardless of whether they’re versioned.

Parameters:

  • from_local (bool) – if True, use latest available local file

  • force_refresh (bool) – if True, fetch and return data from remote regardless of whether a local copy is present

Return type:

Tuple[Path, str]

Returns:

Path to location of data, and version value of it

Raises:

ValueError – if both force_refresh and from_local are True

class wags_tails.base_source.UnversionedDataSource(data_dir=None, silent=True)[source]#

Data acess tool for unversioned data. Provides some additional defaults.

_versioned: bool = False[source]#
_get_latest_version()[source]#

Return blank version. Unversioned data sources shouldn’t need to implement anything further.

Return type:

str

Returns:

empty string

class wags_tails.base_source.GitHubDataSource(data_dir=None, silent=True)[source]#

Class for data sources provided via GitHub releases, where versioning is defined by release tag names.

_repo: str[source]#
iterate_versions()[source]#

Lazily get versions (i.e. not the files themselves, just their version strings), starting with the most recent value and moving backwards.

Return type:

Generator

Returns:

Generator yielding version strings

_get_latest_version()[source]#

Acquire value of latest data version.

Return type:

str

Returns:

latest version value