utilities

Download and management utilities for syncing time and auxiliary files

Can list a directory on a ftp host

Can list the available static gravity field models from the ICGEM

Can download a file from a ftp or http host

Checks MD5 or sha1 hashes between local and remote files

Source code

General Methods

geoid_toolkit.utilities.get_data_path(relpath: list | str | Path)[source]

Get the absolute path within a package from a relative path

Parameters:

relpath: list, str or pathlib.Path: relative path

geoid_toolkit.utilities.import_dependency(name: str, extra: str = '', raise_exception: bool = False)[source]

Import an optional dependency

Adapted from pandas.compat._optional::import_optional_dependency

Parameters:

name: str: Module name
extra: str, default “”: Additional text to include in the ImportError message
raise_exception: bool, default False: Raise an ImportError if the module is not found

Returns:

module: obj: Imported module

geoid_toolkit.utilities.get_hash(local: str | IOBase | Path, algorithm: str = 'md5')[source]

Get the hash value from a local file or BytesIO object

Parameters:

local: obj, str or pathlib.Path: BytesIO object or path to file
algorithm: str, default ‘md5’: hashing algorithm for checksum validation

geoid_toolkit.utilities.get_git_revision_hash(refname: str = 'HEAD', short: bool = False)[source]

Get the git hash value for a particular reference

Parameters:

refname: str, default HEAD: Symbolic reference name
short: bool, default False: Return the shorted hash value

geoid_toolkit.utilities.get_git_status()[source]: Get the status of a git repository as a boolean value

geoid_toolkit.utilities.url_split(s: str)[source]

Recursively split a url path into a list

Parameters:

s: str: url string

geoid_toolkit.utilities.convert_arg_line_to_args(arg_line)[source]

Convert file lines to arguments

Parameters:

arg_line: str: line string containing a single argument and/or comments

geoid_toolkit.utilities.get_unix_time(time_string: str, format: str = '%Y-%m-%d %H:%M:%S')[source]

Get the Unix timestamp value for a formatted date string

Parameters:

time_string: str: formatted time string to parse
format: str, default ‘%Y-%m-%d %H:%M:%S’: format for input time string

geoid_toolkit.utilities.even(value: float)[source]

Rounds a number to an even number less than or equal to original

Parameters:

value: float: number to be rounded

geoid_toolkit.utilities.ceil(value: float)[source]

Rounds a number upward to its nearest integer

Parameters:

value: float: number to be rounded upward

geoid_toolkit.utilities.copy(source: str | Path, destination: str | Path, move: bool = False, **kwargs)[source]

Copy or move a file with all system information

Parameters:

source: str or pathlib.Path: source file
destination: str or pathlib.Path: copied destination file
move: bool, default False: remove the source file

geoid_toolkit.utilities.check_ftp_connection(HOST: str, username: str | None = None, password: str | None = None)[source]

Check internet connection with ftp host

Parameters:

HOST: str: remote ftp host
username: str or NoneType: ftp username
password: str or NoneType: ftp password

List a directory on a ftp host

Parameters:

HOST: str or list: remote ftp host path split as list
username: str or NoneType: ftp username
password: str or NoneType: ftp password
timeout: int or NoneType, default None: timeout in seconds for blocking operations
basename: bool, default False: return the file or directory basename instead of the full path
pattern: str or NoneType, default None: regular expression pattern for reducing list
sort: bool, default False: sort output list

Returns:

output: list: items in a directory
mtimes: list: last modification times for items in the directory

geoid_toolkit.utilities.from_ftp(HOST: str | list, username: str | None = None, password: str | None = None, timeout: int | None = None, local: str | ~pathlib.Path | None = None, hash: str = '', chunk: int = 8192, verbose: bool = False, fid=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, mode: oct = 509)[source]

Download a file from a ftp host

Parameters:

HOST: str or list: remote ftp host path
username: str or NoneType: ftp username
password: str or NoneType: ftp password
timeout: int or NoneType, default None: timeout in seconds for blocking operations
local: str, pathlib.Path or NoneType, default None: path to local file
hash: str, default ‘’: MD5 hash of local file
chunk: int, default 8192: chunk size for transfer encoding
verbose: bool, default False: print file transfer information
fid: obj, default sys.stdout: open file object to print if verbose
mode: oct, default 0o775: permissions mode of output local file

Returns:

remote_buffer: obj: BytesIO representation of file

geoid_toolkit.utilities._create_default_ssl_context() → SSLContext[source]: Creates the default SSL context

geoid_toolkit.utilities._create_ssl_context_no_verify() → SSLContext[source]: Creates an SSL context for unverified connections

geoid_toolkit.utilities._set_ssl_context_options(context: SSLContext) → None[source]: Sets the default options for the SSL context

geoid_toolkit.utilities.check_connection(HOST: str, context: ~ssl.SSLContext = <ssl.SSLContext object>)[source]

Check internet connection with http host

Parameters:

HOST: str: remote http host
context: obj, default geoid_toolkit.utilities._default_ssl_context: SSL context for urllib opener object

geoid_toolkit.utilities.from_http(HOST: str | list, timeout: int | None = None, context: ~ssl.SSLContext = <ssl.SSLContext object>, local: str | ~pathlib.Path | None = None, hash: str = '', chunk: int = 16384, verbose: bool = False, fid=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>, mode: oct = 509)[source]

Download a file from a http host

Parameters:

HOST: str or list: remote http host path split as list
timeout: int or NoneType, default None: timeout in seconds for blocking operations
context: obj, default geoid_toolkit.utilities._default_ssl_context: SSL context for urllib opener object
local: str, pathlib.Path or NoneType, default None: path to local file
hash: str, default ‘’: MD5 hash of local file
chunk: int, default 16384: chunk size for transfer encoding
verbose: bool, default False: print file transfer information
fid: obj, default sys.stdout: open file object to print if verbose
mode: oct, default 0o775: permissions mode of output local file

Returns:

remote_buffer: obj: BytesIO representation of file

geoid_toolkit.utilities.icgem_list(host: str = 'http://icgem.gfz-potsdam.de/tom_longtime', timeout: int | None = None, context: ~ssl.SSLContext = <ssl.SSLContext object>, parser=<lxml.etree.HTMLParser object>)[source]

Parse the table of static gravity field models on the GFZ International Centre for Global Earth Models (ICGEM) server

Parameters:

host: str: url for the GFZ ICGEM gravity field table
timeout: int or NoneType, default None: timeout in seconds for blocking operations
context: obj, default geoid_toolkit.utilities._default_ssl_context: SSL context for urllib opener object
parser: obj, default lxml.etree.HTMLParser(): HTML parser for lxml

Returns:

colfiles: dict: Static gravity field file urls mapped by field name