scmdata.filters

Helpers for filtering data in scmdata.run.ScmRun.

Based upon pyam.utils.

is_in

is_in(vals, items)

Find elements of vals which are in items.

Parameters:

• vals (typing.Iterable[typing.Any]) – The list of values to check

• items (typing.Iterable[typing.Any]) – The options used to determine whether each element of vals is in the desired subset or not

Returns:

numpy.ndarray of bool – Array of the same length as vals where the element is True if the corresponding element of vals is in items and False otherwise

find_depth

find_depth(meta_col, s, level, separator='|')

Find all values which match given depth from a filter keyword.

Parameters:

• meta_col (pandas.core.series.Series) – Column in which to find values which match the given depth

• s (str) – Filter keyword, from which level should be applied

• level (typing.Union[int, str]) – Depth of value to match as defined by the number of separator in the value name. If an int, the depth is matched exactly. If a str, then the depth can be matched as either “X-”, for all levels up to level “X”, or “X+”, for all levels above level “X”.

• separator (str) – The string used to separate levels in s. Defaults to a pipe (“|”).

Returns:

numpy.ndarray of bool – Array where True indicates a match

Raises:

ValueError – If level cannot be understood

pattern_match

pattern_match(meta_col, values, level=None, regexp=False, separator='|')

Filter data by matching metadata columns to given patterns.

Parameters:

• meta_col (pandas.core.indexes.base.Index) – Column to perform filtering on

• values (typing.Union[typing.Iterable[typing.Union[str, int, float]], str, int, float]) – Values to match

• level (typing.Union[str, int, float, None]) – Passed to find_depth(). For usage, see docstring of find_depth().

• regexp (bool) – If True, match using regexp rather than our pseudo regexp syntax.

• has_nan – If True, convert all nan values in meta_col to empty string before applying filters. This means that “” and “*” will match rows with numpy.nan. If False, the conversion is not applied and so a search in a string column which contains numpy.nan will result in a TypeError.

• separator (str) – String used to separate the hierarchy levels in values. Defaults to ‘|’

Returns:

numpy.ndarray of bool – Array where True indicates a match

Raises:

TypeError – Filtering is performed on a string metadata column which contains numpy.nan and has_nan is False

years_match

years_match(data, years)

Match years in time columns for data filtering.

Parameters:

• data (typing.Iterable[typing.Any]) – Input data to perform filtering on

• years (typing.Union[typing.Iterable[int], numpy.ndarray, int]) – Years to match

Returns:

numpy.ndarray of bool – Array where True indicates a match

Raises:

TypeError – If years is not int or list of int

month_match

month_match(data, months)

Match months in time columns for data filtering.

Parameters:

• data (typing.Iterable[typing.Any]) – Input data to perform filtering on

• months (typing.Union[typing.Iterable[typing.Union[str, int]], int, str]) – Months to match

Returns:

numpy.ndarray of bool – Array where True indicates a match

day_match

day_match(data, days)

Match days in time columns for data filtering.

Parameters:

• data (typing.Iterable[typing.Any]) – Input data to perform filtering on

• days (typing.Union[typing.List[str], typing.List[int], int, str]) – Days to match

Returns:

numpy.ndarray of bool – Array where True indicates a match

hour_match

hour_match(data, hours)

Match hours in time columns for data filtering.

Parameters:

• data (typing.Iterable[typing.Any]) – Input data to perform filtering on

• hours (typing.Union[typing.Iterable[int], int]) – Hours to match

Returns:

numpy.ndarray of bool – Array where True indicates a match

time_match

time_match(data, times, conv_codes, strptime_attr, name)

Match times by applying conversion codes to filtering list.

Parameters:

• data (typing.Iterable[typing.Any]) – Input data to perform filtering on

• times (typing.Union[typing.Iterable[typing.Union[str, int]], int, str]) – Times to match

• conv_codes (typing.List[str]) – If times contains strings, conversion codes to try passing to time.strptime() to convert times to datetime.datetime

• strptime_attr (str) – If times contains strings, the datetime.datetime attribute to finalize the conversion of strings to integers

• name (str) – Name of the part of a datetime to extract, used to produce useful error messages.

Returns:

numpy.ndarray of bool – Array where True indicates a match

Raises:

ValueError – If input times cannot be converted understood or if input strings do not lead to increasing integers (i.e. “Nov-Feb” will not work, one must use [“Nov-Dec”, “Jan-Feb”] instead)

datetime_match

datetime_match(data, dts)

Match datetimes in time columns for data filtering.

Parameters:

• data (typing.Iterable[typing.Any]) – Input data to perform filtering on

• dts (typing.Union[typing.Iterable[datetime.datetime], datetime.datetime, str]) – Datetimes to use for filtering

Returns:

numpy.ndarray of bool – Array where True indicates a match

Raises:

TypeError – dts contains int