scmdata.filters
Helpers for filtering data in scmdata.run.ScmRun
.
Based upon pyam.utils
.
is_in
- is_in(vals, items)[source]
Find elements of vals which are in items.
- Parameters:
vals (
typing.Iterable
[typing.Any
]) – The list of values to checkitems (
typing.Iterable
[typing.Any
]) – The options used to determine whether each element ofvals
is in the desired subset or not
- Returns:
numpy.ndarray
ofbool
– Array of the same length asvals
where the element isTrue
if the corresponding element ofvals
is initems
and False otherwise
find_depth
- find_depth(meta_col, s, level, separator='|')[source]
Find all values which match given depth from a filter keyword.
- Parameters:
meta_col (
pandas.core.series.Series
) – Column in which to find values which match the given depths (
str
) – Filter keyword, from which level should be appliedlevel (
typing.Union
[int
,str
]) – Depth of value to match as defined by the number of separator in the value name. If an int, the depth is matched exactly. If a str, then the depth can be matched as either “X-”, for all levels up to level “X”, or “X+”, for all levels above level “X”.separator (
str
) – The string used to separate levels in s. Defaults to a pipe (“|”).
- Returns:
numpy.ndarray
ofbool
– Array whereTrue
indicates a match- Raises:
ValueError – If
level
cannot be understood
pattern_match
- pattern_match(meta_col, values, level=None, regexp=False, separator='|')[source]
Filter data by matching metadata columns to given patterns.
- Parameters:
meta_col (
pandas.core.indexes.base.Index
) – Column to perform filtering onvalues (
typing.Union
[typing.Iterable
[typing.Union
[str
,int
,float
]],str
,int
,float
]) – Values to matchlevel (
typing.Union
[str
,int
,float
,None
]) – Passed tofind_depth()
. For usage, see docstring offind_depth()
.regexp (
bool
) – IfTrue
, match using regexp rather than our pseudo regexp syntax.has_nan – If
True
, convert all nan values inmeta_col
to empty string before applying filters. This means that “” and “*” will match rows withnumpy.nan
. IfFalse
, the conversion is not applied and so a search in a string column which containsnumpy.nan
will result in aTypeError
.separator (
str
) – String used to separate the hierarchy levels in values. Defaults to ‘|’
- Returns:
numpy.ndarray
ofbool
– Array whereTrue
indicates a match- Raises:
TypeError – Filtering is performed on a string metadata column which contains
numpy.nan
andhas_nan
isFalse
years_match
- years_match(data, years)[source]
Match years in time columns for data filtering.
- Parameters:
data (
typing.Iterable
[typing.Any
]) – Input data to perform filtering onyears (
typing.Union
[typing.Iterable
[int
],numpy.ndarray
,int
]) – Years to match
- Returns:
numpy.ndarray
ofbool
– Array where True indicates a match- Raises:
month_match
- month_match(data, months)[source]
Match months in time columns for data filtering.
- Parameters:
data (
typing.Iterable
[typing.Any
]) – Input data to perform filtering onmonths (
typing.Union
[typing.Iterable
[typing.Union
[str
,int
]],int
,str
]) – Months to match
- Returns:
numpy.ndarray
ofbool
– Array whereTrue
indicates a match
day_match
- day_match(data, days)[source]
Match days in time columns for data filtering.
- Parameters:
data (
typing.Iterable
[typing.Any
]) – Input data to perform filtering ondays (
typing.Union
[typing.List
[str
],typing.List
[int
],int
,str
]) – Days to match
- Returns:
numpy.ndarray
ofbool
– Array whereTrue
indicates a match
hour_match
- hour_match(data, hours)[source]
Match hours in time columns for data filtering.
- Parameters:
data (
typing.Iterable
[typing.Any
]) – Input data to perform filtering onhours (
typing.Union
[typing.Iterable
[int
],int
]) – Hours to match
- Returns:
numpy.ndarray
ofbool
– Array whereTrue
indicates a match
time_match
- time_match(data, times, conv_codes, strptime_attr, name)[source]
Match times by applying conversion codes to filtering list.
- Parameters:
data (
typing.Iterable
[typing.Any
]) – Input data to perform filtering ontimes (
typing.Union
[typing.Iterable
[typing.Union
[str
,int
]],int
,str
]) – Times to matchconv_codes (
typing.List
[str
]) – Iftimes
contains strings, conversion codes to try passing totime.strptime()
to converttimes
todatetime.datetime
strptime_attr (
str
) – Iftimes
contains strings, thedatetime.datetime
attribute to finalize the conversion of strings to integersname (
str
) – Name of the part of a datetime to extract, used to produce useful error messages.
- Returns:
numpy.ndarray
ofbool
– Array whereTrue
indicates a match- Raises:
ValueError – If input times cannot be converted understood or if input strings do not lead to increasing integers (i.e. “Nov-Feb” will not work, one must use [“Nov-Dec”, “Jan-Feb”] instead)
datetime_match
- datetime_match(data, dts)[source]
Match datetimes in time columns for data filtering.
- Parameters:
data (
typing.Iterable
[typing.Any
]) – Input data to perform filtering ondts (
typing.Union
[typing.Iterable
[datetime.datetime
],datetime.datetime
,str
]) – Datetimes to use for filtering
- Returns:
numpy.ndarray
ofbool
– Array whereTrue
indicates a match- Raises: