wags_tails.utils.downloads#

Provide helper functions for downloading data.

wags_tails.utils.downloads.handle_zip(dl_path, outfile_path)[source]#

Extract the largest file within a given zipfile and save it within the appropriate data directory. Can be passed as a callback to a downloader method.

Parameters:

  • dl_path (Path) – path to temp data file

  • outfile_path (Path) – path to save file within

Return type:

None

wags_tails.utils.downloads.handle_gzip(dl_path, outfile_path)[source]#

Extract file from gzip. Can be passed as a callback to a downloader method.

Parameters:

  • dl_path (Path) – path to temp data file

  • outfile_path (Path) – path to save file within

Return type:

None

wags_tails.utils.downloads.download_ftp(host, host_directory_path, host_filename, outfile_path, handler=None, tqdm_params=None)[source]#

Perform FTP download of remote data file.

Parameters:

  • host (str) – FTP hostname

  • host_directory_path (str) – path to desired file on host

  • host_filename (str) – name of desired file on host

  • outfile_path (Path) – path to where file should be saved. Must be an actual Path instance rather than merely a pathlike string.

  • handler (Optional[Callable[[Path, Path], None]]) – provide if downloaded file requires additional action, e.g. it’s a zip file

  • tqdm_params (Optional[Dict]) – Optional TQDM configuration.

Return type:

None

wags_tails.utils.downloads.download_http(url, outfile_path, headers=None, handler=None, tqdm_params=None)[source]#

Perform HTTP download of remote data file.

Parameters:

  • url (str) – URL to retrieve file from

  • outfile_path (Path) – path to where file should be saved. Must be an actual Path instance rather than merely a pathlike string.

  • headers (Optional[Dict]) – Any needed HTTP headers to include in request

  • handler (Optional[Callable[[Path, Path], None]]) – provide if downloaded file requires additional action, e.g. it’s a zip file.

  • tqdm_params (Optional[Dict]) – Optional TQDM configuration.

Return type:

None