Metrics¶
A Metric
is part of a Benchmark
and scores how similar two sets of data are.
Typically these two sets are model and primate measurements, but metrics are agnostic of the data source
and can also be used to compare two primate measurements (e.g. for ceiling estimates).
- class brainscore_core.metrics.Metric[source]¶
Metric interface. A metric compares two sets of data and outputs a score of how well they match (1 = identical, 0 = no match).
- __call__(assembly1: DataAssembly, assembly2: DataAssembly) Score [source]¶
Compare two assemblies on their similarity. These assemblies are typically neural or behavioral measurements, e.g. model and primate recordings.
- Parameters:
assembly1 – the first assembly to compare against the second
assembly2 – the second assembly to compare against the first
- Returns:
a
Score
denoting the match between the two assemblies (1 = identical, 0 = no match).
- class brainscore_core.metrics.Score(*args, **kwargs)[source]¶
Scores are used as the outputs of metrics, benchmarks, and ceilings. They indicate similarity or goodness-of-fit of sets of data. The high-level score is typically an aggregate of many smaller scores, e.g. the median of neuroid correlations. To keep records of these smaller scores, a score can store “raw” scores in its attributes (score.attrs[‘raw’]).
- RAW_VALUES_KEY = 'raw'¶
- all(dim=None, axis=None, **kwargs)¶
Reduce this Score’s data by applying all along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply all.
- axisint or sequence of int, optional
Axis(es) over which to apply all. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then all is calculated over axes.
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating all on this object’s data.
Returns¶
- reducedScore
New Score object with all applied to its data and the indicated dimension(s) removed.
- any(dim=None, axis=None, **kwargs)¶
Reduce this Score’s data by applying any along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply any.
- axisint or sequence of int, optional
Axis(es) over which to apply any. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then any is calculated over axes.
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating any on this object’s data.
Returns¶
- reducedScore
New Score object with any applied to its data and the indicated dimension(s) removed.
- count(dim=None, axis=None, **kwargs)¶
Reduce this Score’s data by applying count along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply count.
- axisint or sequence of int, optional
Axis(es) over which to apply count. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then count is calculated over axes.
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating count on this object’s data.
Returns¶
- reducedScore
New Score object with count applied to its data and the indicated dimension(s) removed.
- cumprod(dim=None, axis=None, skipna=None, **kwargs)¶
Apply cumprod along some dimension of Score.
Parameters¶
- dimstr or sequence of str, optional
Dimension over which to apply cumprod.
- axisint or sequence of int, optional
Axis over which to apply cumprod. Only one of the ‘dim’ and ‘axis’ arguments can be supplied.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to cumprod.
Returns¶
- cumvalueScore
New Score object with cumprod applied to its data along the indicated dimension.
- cumsum(dim=None, axis=None, skipna=None, **kwargs)¶
Apply cumsum along some dimension of Score.
Parameters¶
- dimstr or sequence of str, optional
Dimension over which to apply cumsum.
- axisint or sequence of int, optional
Axis over which to apply cumsum. Only one of the ‘dim’ and ‘axis’ arguments can be supplied.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to cumsum.
Returns¶
- cumvalueScore
New Score object with cumsum applied to its data along the indicated dimension.
- expand_dims(*args, _apply_raw=True, **kwargs)[source]¶
Return a new object with an additional axis (or axes) inserted at the corresponding position in the array shape. The new object is a view into the underlying array, not a copy.
If dim is already a scalar coordinate, it will be promoted to a 1D coordinate consisting of a single value.
Parameters¶
- dimhashable, sequence of hashable, dict, or None, optional
Dimensions to include on the new variable. If provided as str or sequence of str, then dimensions are inserted with length 1. If provided as a dict, then the keys are the new dimensions and the values are either integers (giving the length of the new dimensions) or sequence/ndarray (giving the coordinates of the new dimensions).
- axisint, list of int or tuple of int, or None, default: None
Axis position(s) where new axis is to be inserted (position(s) on the result array). If a list (or tuple) of integers is passed, multiple axes are inserted. In this case, dim arguments should be same length list. If axis=None is passed, all the axes will be inserted to the start of the result array.
- **dim_kwargsint or sequence or ndarray
The keywords are arbitrary dimensions being inserted and the values are either the lengths of the new dims (if int is given), or their coordinates. Note, this is an alternative to passing a dict to the dim kwarg and will only be used if dim is None.
Returns¶
- expandedsame type as caller
This object, but with an additional dimension(s).
- isel(*args, _apply_raw=True, **kwargs)[source]¶
Return a new DataArray whose data is given by integer indexing along the specified dimension(s).
Parameters¶
- indexersdict, optional
A dict with keys matching dimensions and values given by integers, slice objects or arrays. indexer can be a integer, slice, array-like or DataArray. If DataArrays are passed as indexers, xarray-style indexing will be carried out. See indexing for the details. One of indexers or indexers_kwargs must be provided.
- dropbool, optional
If
drop=True
, drop coordinates variables indexed by integers instead of making them scalar.- missing_dims{“raise”, “warn”, “ignore”}, default: “raise”
What to do if dimensions that should be selected from are not present in the DataArray: - “raise”: raise an exception - “warn”: raise a warning, and ignore the missing dimensions - “ignore”: ignore the missing dimensions
- **indexers_kwargs{dim: indexer, …}, optional
The keyword arguments form of
indexers
.
See Also¶
Dataset.isel DataArray.sel
Examples¶
>>> da = xr.DataArray(np.arange(25).reshape(5, 5), dims=("x", "y")) >>> da <xarray.DataArray (x: 5, y: 5)> array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24]]) Dimensions without coordinates: x, y
>>> tgt_x = xr.DataArray(np.arange(0, 5), dims="points") >>> tgt_y = xr.DataArray(np.arange(0, 5), dims="points") >>> da = da.isel(x=tgt_x, y=tgt_y) >>> da <xarray.DataArray (points: 5)> array([ 0, 6, 12, 18, 24]) Dimensions without coordinates: points
- item(*args)¶
Copy an element of an array to a standard Python scalar and return it.
Parameters¶
*args : Arguments (variable number and type)
none: in this case, the method only works for arrays with one element (a.size == 1), which element is copied into a standard Python scalar object and returned.
int_type: this argument is interpreted as a flat index into the array, specifying which element to copy and return.
tuple of int_types: functions as does a single int_type argument, except that the argument is interpreted as an nd-index into the array.
Returns¶
- zStandard Python scalar object
A copy of the specified element of the array as a suitable Python scalar
Notes¶
When the data type of a is longdouble or clongdouble, item() returns a scalar array object because there is no available Python scalar that would not lose information. Void arrays return a buffer object for item(), unless fields are defined, in which case a tuple is returned.
item is very similar to a[args], except, instead of an array scalar, a standard Python scalar is returned. This can be useful for speeding up access to elements of the array and doing arithmetic on elements of the array using Python’s optimized math.
Examples¶
>>> np.random.seed(123) >>> x = np.random.randint(9, size=(3, 3)) >>> x array([[2, 2, 6], [1, 3, 6], [1, 0, 1]]) >>> x.item(3) 1 >>> x.item(7) 0 >>> x.item((0, 1)) 2 >>> x.item((2, 2)) 1
- max(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying max along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply max.
- axisint or sequence of int, optional
Axis(es) over which to apply max. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then max is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating max on this object’s data.
Returns¶
- reducedScore
New Score object with max applied to its data and the indicated dimension(s) removed.
- mean(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying mean along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply mean.
- axisint or sequence of int, optional
Axis(es) over which to apply mean. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then mean is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating mean on this object’s data.
Returns¶
- reducedScore
New Score object with mean applied to its data and the indicated dimension(s) removed.
- median(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying median along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply median.
- axisint or sequence of int, optional
Axis(es) over which to apply median. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then median is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating median on this object’s data.
Returns¶
- reducedScore
New Score object with median applied to its data and the indicated dimension(s) removed.
- classmethod merge(*scores, exception_handling: str = 'ignore')[source]¶
Merges the raw values in addition to the score assemblies. Raw values are indexed on the first score.
- Parameters:
exception_handling – how to deal with raised exceptions, one of ‘ignore’ (do not raise or log), ‘warn’ (log but do not raise), ‘raise’ (raise exception).
- min(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying min along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply min.
- axisint or sequence of int, optional
Axis(es) over which to apply min. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then min is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating min on this object’s data.
Returns¶
- reducedScore
New Score object with min applied to its data and the indicated dimension(s) removed.
- prod(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying prod along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply prod.
- axisint or sequence of int, optional
Axis(es) over which to apply prod. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then prod is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- min_countint, default: None
The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA. Only used if skipna is set to True or defaults to True for the array’s dtype. New in version 0.10.8: Added with the default being None. Changed in version 0.17.0: if specified on an integer array and skipna=True, the result will be a float array.
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating prod on this object’s data.
Returns¶
- reducedScore
New Score object with prod applied to its data and the indicated dimension(s) removed.
- reduce(*args, _apply_raw=False, **kwargs)[source]¶
Reduce this array by applying func along some dimension(s).
Parameters¶
- funccallable
Function which can be called in the form f(x, axis=axis, **kwargs) to return the result of reducing an np.ndarray over an integer valued axis.
- dimhashable or sequence of hashable, optional
Dimension(s) over which to apply func.
- axisint or sequence of int, optional
Axis(es) over which to repeatedly apply func. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then the reduction is calculated over the flattened array (by calling f(x) without an axis argument).
- keep_attrsbool, optional
If True, the variable’s attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- keepdimsbool, default: False
If True, the dimensions which are reduced are left in the result as dimensions of size one. Coordinates that use these dimensions are removed.
- **kwargsdict
Additional keyword arguments passed on to func.
Returns¶
- reducedDataArray
DataArray with this object’s array replaced with an array with summarized data and the indicated dimension(s) removed.
- searchsorted(v, side='left', sorter=None)¶
Find indices where elements of v should be inserted in a to maintain order.
For full documentation, see numpy.searchsorted
See Also¶
numpy.searchsorted : equivalent function
- sel(*args, _apply_raw=True, **kwargs)[source]¶
Return a new DataArray whose data is given by selecting index labels along the specified dimension(s).
In contrast to DataArray.isel, indexers for this method should use labels instead of integers.
Under the hood, this method is powered by using pandas’s powerful Index objects. This makes label based indexing essentially just as fast as using integer indexing.
It also means this method uses pandas’s (well documented) logic for indexing. This means you can use string shortcuts for datetime indexes (e.g., ‘2000-01’ to select all values in January 2000). It also means that slices are treated as inclusive of both the start and stop values, unlike normal Python indexing.
Warning
Do not try to assign values when using any of the indexing methods
isel
orsel
:da = xr.DataArray([0, 1, 2, 3], dims=['x']) # DO NOT do this da.isel(x=[0, 1, 2])[1] = -1
Assigning values with the chained indexing using
.sel
or.isel
fails silently.Parameters¶
- indexersdict, optional
A dict with keys matching dimensions and values given by scalars, slices or arrays of tick labels. For dimensions with multi-index, the indexer may also be a dict-like object with keys matching index level names. If DataArrays are passed as indexers, xarray-style indexing will be carried out. See indexing for the details. One of indexers or indexers_kwargs must be provided.
- method{None, “nearest”, “pad”, “ffill”, “backfill”, “bfill”}, optional
Method to use for inexact matches:
None (default): only exact matches
pad / ffill: propagate last valid index value forward
backfill / bfill: propagate next valid index value backward
nearest: use nearest valid index value
- toleranceoptional
Maximum distance between original and new labels for inexact matches. The values of the index at the matching locations must satisfy the equation
abs(index[indexer] - target) <= tolerance
.- dropbool, optional
If
drop=True
, drop coordinates variables in indexers instead of making them scalar.- **indexers_kwargs{dim: indexer, …}, optional
The keyword arguments form of
indexers
. One of indexers or indexers_kwargs must be provided.
Returns¶
- objDataArray
A new DataArray with the same contents as this DataArray, except the data and each dimension is indexed by the appropriate indexers. If indexer DataArrays have coordinates that do not conflict with this object, then these coordinates will be attached. In general, each array’s data will be a view of the array’s data in this DataArray, unless vectorized indexing was triggered by using an array indexer, in which case the data will be a copy.
See Also¶
Dataset.sel DataArray.isel
Examples¶
>>> da = xr.DataArray( ... np.arange(25).reshape(5, 5), ... coords={"x": np.arange(5), "y": np.arange(5)}, ... dims=("x", "y"), ... ) >>> da <xarray.DataArray (x: 5, y: 5)> array([[ 0, 1, 2, 3, 4], [ 5, 6, 7, 8, 9], [10, 11, 12, 13, 14], [15, 16, 17, 18, 19], [20, 21, 22, 23, 24]]) Coordinates: * x (x) int64 0 1 2 3 4 * y (y) int64 0 1 2 3 4
>>> tgt_x = xr.DataArray(np.linspace(0, 4, num=5), dims="points") >>> tgt_y = xr.DataArray(np.linspace(0, 4, num=5), dims="points") >>> da = da.sel(x=tgt_x, y=tgt_y, method="nearest") >>> da <xarray.DataArray (points: 5)> array([ 0, 6, 12, 18, 24]) Coordinates: x (points) int64 0 1 2 3 4 y (points) int64 0 1 2 3 4 Dimensions without coordinates: points
- squeeze(*args, _apply_raw=True, **kwargs)[source]¶
Return a new object with squeezed data.
Parameters¶
- dimNone or Hashable or iterable of Hashable, optional
Selects a subset of the length one dimensions. If a dimension is selected with length greater than one, an error is raised. If None, all length one dimensions are squeezed.
- dropbool, optional
If
drop=True
, drop squeezed coordinates instead of making them scalar.- axisNone or int or iterable of int, optional
Like dim, but positional.
Returns¶
- squeezedsame type as caller
This object, but with with all or a subset of the dimensions of length 1 removed.
See Also¶
numpy.squeeze
- std(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying std along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply std.
- axisint or sequence of int, optional
Axis(es) over which to apply std. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then std is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating std on this object’s data.
Returns¶
- reducedScore
New Score object with std applied to its data and the indicated dimension(s) removed.
- sum(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying sum along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply sum.
- axisint or sequence of int, optional
Axis(es) over which to apply sum. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then sum is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- min_countint, default: None
The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA. Only used if skipna is set to True or defaults to True for the array’s dtype. New in version 0.10.8: Added with the default being None. Changed in version 0.17.0: if specified on an integer array and skipna=True, the result will be a float array.
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating sum on this object’s data.
Returns¶
- reducedScore
New Score object with sum applied to its data and the indicated dimension(s) removed.
- var(dim=None, axis=None, skipna=None, **kwargs)¶
Reduce this Score’s data by applying var along some dimension(s).
Parameters¶
- dimstr or sequence of str, optional
Dimension(s) over which to apply var.
- axisint or sequence of int, optional
Axis(es) over which to apply var. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then var is calculated over axes.
- skipnabool, optional
If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
- keep_attrsbool, optional
If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
- **kwargsdict
Additional keyword arguments passed on to the appropriate array function for calculating var on this object’s data.
Returns¶
- reducedScore
New Score object with var applied to its data and the indicated dimension(s) removed.