Metrics

A Metric is part of a Benchmark and scores how similar two sets of data are. Typically these two sets are model and primate measurements, but metrics are agnostic of the data source and can also be used to compare two primate measurements (e.g. for ceiling estimates).

class brainscore_core.metrics.Metric[source]

Metric interface. A metric compares two sets of data and outputs a score of how well they match (1 = identical, 0 = no match).

__call__(assembly1: DataAssembly, assembly2: DataAssembly) → Score[source]

Compare two assemblies on their similarity. These assemblies are typically neural or behavioral measurements, e.g. model and primate recordings.

Parameters:

assembly1 – the first assembly to compare against the second
assembly2 – the second assembly to compare against the first

Returns:

a Score denoting the match between the two assemblies (1 = identical, 0 = no match).

class brainscore_core.metrics.Score(*args, **kwargs)[source]

Scores are used as the outputs of metrics, benchmarks, and ceilings. They indicate similarity or goodness-of-fit of sets of data. The high-level score is typically an aggregate of many smaller scores, e.g. the median of neuroid correlations. To keep records of these smaller scores, a score can store “raw” scores in its attributes (score.attrs[‘raw’]).

RAW_VALUES_KEY = 'raw'

all(dim=None, axis=None, **kwargs)

Reduce this Score’s data by applying all along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply all.
axisint or sequence of int, optional: Axis(es) over which to apply all. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then all is calculated over axes.
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating all on this object’s data.

Returns

reducedScore: New Score object with all applied to its data and the indicated dimension(s) removed.

any(dim=None, axis=None, **kwargs)

Reduce this Score’s data by applying any along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply any.
axisint or sequence of int, optional: Axis(es) over which to apply any. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then any is calculated over axes.
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating any on this object’s data.

Returns

reducedScore: New Score object with any applied to its data and the indicated dimension(s) removed.

count(dim=None, axis=None, **kwargs)

Reduce this Score’s data by applying count along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply count.
axisint or sequence of int, optional: Axis(es) over which to apply count. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then count is calculated over axes.
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating count on this object’s data.

Returns

reducedScore: New Score object with count applied to its data and the indicated dimension(s) removed.

cumprod(dim=None, axis=None, skipna=None, **kwargs)

Apply cumprod along some dimension of Score.

Parameters

dimstr or sequence of str, optional: Dimension over which to apply cumprod.
axisint or sequence of int, optional: Axis over which to apply cumprod. Only one of the ‘dim’ and ‘axis’ arguments can be supplied.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to cumprod.

Returns

cumvalueScore: New Score object with cumprod applied to its data along the indicated dimension.

cumsum(dim=None, axis=None, skipna=None, **kwargs)

Apply cumsum along some dimension of Score.

Parameters

dimstr or sequence of str, optional: Dimension over which to apply cumsum.
axisint or sequence of int, optional: Axis over which to apply cumsum. Only one of the ‘dim’ and ‘axis’ arguments can be supplied.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to cumsum.

Returns

cumvalueScore: New Score object with cumsum applied to its data along the indicated dimension.

expand_dims(*args, _apply_raw=True, **kwargs)[source]

Return a new object with an additional axis (or axes) inserted at the corresponding position in the array shape. The new object is a view into the underlying array, not a copy.

If dim is already a scalar coordinate, it will be promoted to a 1D coordinate consisting of a single value.

Parameters

dimhashable, sequence of hashable, dict, or None, optional: Dimensions to include on the new variable. If provided as str or sequence of str, then dimensions are inserted with length 1. If provided as a dict, then the keys are the new dimensions and the values are either integers (giving the length of the new dimensions) or sequence/ndarray (giving the coordinates of the new dimensions).
axisint, list of int or tuple of int, or None, default: None: Axis position(s) where new axis is to be inserted (position(s) on the result array). If a list (or tuple) of integers is passed, multiple axes are inserted. In this case, dim arguments should be same length list. If axis=None is passed, all the axes will be inserted to the start of the result array.
**dim_kwargsint or sequence or ndarray: The keywords are arbitrary dimensions being inserted and the values are either the lengths of the new dims (if int is given), or their coordinates. Note, this is an alternative to passing a dict to the dim kwarg and will only be used if dim is None.

Returns

expandedsame type as caller: This object, but with an additional dimension(s).

isel(*args, _apply_raw=True, **kwargs)[source]

Return a new DataArray whose data is given by integer indexing along the specified dimension(s).

Parameters

indexersdict, optional: A dict with keys matching dimensions and values given by integers, slice objects or arrays. indexer can be a integer, slice, array-like or DataArray. If DataArrays are passed as indexers, xarray-style indexing will be carried out. See indexing for the details. One of indexers or indexers_kwargs must be provided.
dropbool, optional: If drop=True, drop coordinates variables indexed by integers instead of making them scalar.
missing_dims{“raise”, “warn”, “ignore”}, default: “raise”: What to do if dimensions that should be selected from are not present in the DataArray: - “raise”: raise an exception - “warn”: raise a warning, and ignore the missing dimensions - “ignore”: ignore the missing dimensions
**indexers_kwargs{dim: indexer, …}, optional: The keyword arguments form of indexers.

Examples

>>> da = xr.DataArray(np.arange(25).reshape(5, 5), dims=("x", "y"))
>>> da
<xarray.DataArray (x: 5, y: 5)>
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24]])
Dimensions without coordinates: x, y

>>> tgt_x = xr.DataArray(np.arange(0, 5), dims="points")
>>> tgt_y = xr.DataArray(np.arange(0, 5), dims="points")
>>> da = da.isel(x=tgt_x, y=tgt_y)
>>> da
<xarray.DataArray (points: 5)>
array([ 0,  6, 12, 18, 24])
Dimensions without coordinates: points

item(*args)

Copy an element of an array to a standard Python scalar and return it.

Parameters

*args : Arguments (variable number and type)

none: in this case, the method only works for arrays with one element (a.size == 1), which element is copied into a standard Python scalar object and returned.

int_type: this argument is interpreted as a flat index into the array, specifying which element to copy and return.

tuple of int_types: functions as does a single int_type argument, except that the argument is interpreted as an nd-index into the array.

Returns

zStandard Python scalar object: A copy of the specified element of the array as a suitable Python scalar

Notes

When the data type of a is longdouble or clongdouble, item() returns a scalar array object because there is no available Python scalar that would not lose information. Void arrays return a buffer object for item(), unless fields are defined, in which case a tuple is returned.

item is very similar to a[args], except, instead of an array scalar, a standard Python scalar is returned. This can be useful for speeding up access to elements of the array and doing arithmetic on elements of the array using Python’s optimized math.

Examples

>>> np.random.seed(123)
>>> x = np.random.randint(9, size=(3, 3))
>>> x
array([[2, 2, 6],
       [1, 3, 6],
       [1, 0, 1]])
>>> x.item(3)
1
>>> x.item(7)
0
>>> x.item((0, 1))
2
>>> x.item((2, 2))
1

max(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying max along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply max.
axisint or sequence of int, optional: Axis(es) over which to apply max. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then max is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating max on this object’s data.

Returns

reducedScore: New Score object with max applied to its data and the indicated dimension(s) removed.

mean(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying mean along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply mean.
axisint or sequence of int, optional: Axis(es) over which to apply mean. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then mean is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating mean on this object’s data.

Returns

reducedScore: New Score object with mean applied to its data and the indicated dimension(s) removed.

median(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying median along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply median.
axisint or sequence of int, optional: Axis(es) over which to apply median. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then median is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating median on this object’s data.

Returns

reducedScore: New Score object with median applied to its data and the indicated dimension(s) removed.

classmethod merge(*scores, exception_handling: str = 'ignore')[source]

Merges the raw values in addition to the score assemblies. Raw values are indexed on the first score.

Parameters:: exception_handling – how to deal with raised exceptions, one of ‘ignore’ (do not raise or log), ‘warn’ (log but do not raise), ‘raise’ (raise exception).

min(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying min along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply min.
axisint or sequence of int, optional: Axis(es) over which to apply min. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then min is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating min on this object’s data.

Returns

reducedScore: New Score object with min applied to its data and the indicated dimension(s) removed.

prod(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying prod along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply prod.
axisint or sequence of int, optional: Axis(es) over which to apply prod. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then prod is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
min_countint, default: None: The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA. Only used if skipna is set to True or defaults to True for the array’s dtype. New in version 0.10.8: Added with the default being None. Changed in version 0.17.0: if specified on an integer array and skipna=True, the result will be a float array.
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating prod on this object’s data.

Returns

reducedScore: New Score object with prod applied to its data and the indicated dimension(s) removed.

reduce(*args, _apply_raw=False, **kwargs)[source]

Reduce this array by applying func along some dimension(s).

Parameters

funccallable: Function which can be called in the form f(x, axis=axis, **kwargs) to return the result of reducing an np.ndarray over an integer valued axis.
dimhashable or sequence of hashable, optional: Dimension(s) over which to apply func.
axisint or sequence of int, optional: Axis(es) over which to repeatedly apply func. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then the reduction is calculated over the flattened array (by calling f(x) without an axis argument).
keep_attrsbool, optional: If True, the variable’s attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
keepdimsbool, default: False: If True, the dimensions which are reduced are left in the result as dimensions of size one. Coordinates that use these dimensions are removed.
**kwargsdict: Additional keyword arguments passed on to func.

Returns

reducedDataArray: DataArray with this object’s array replaced with an array with summarized data and the indicated dimension(s) removed.

searchsorted(v, side='left', sorter=None)

Find indices where elements of v should be inserted in a to maintain order.

For full documentation, see numpy.searchsorted

Parameters

indexersdict, optional

A dict with keys matching dimensions and values given by scalars, slices or arrays of tick labels. For dimensions with multi-index, the indexer may also be a dict-like object with keys matching index level names. If DataArrays are passed as indexers, xarray-style indexing will be carried out. See indexing for the details. One of indexers or indexers_kwargs must be provided.

method{None, “nearest”, “pad”, “ffill”, “backfill”, “bfill”}, optional

Method to use for inexact matches:

None (default): only exact matches
pad / ffill: propagate last valid index value forward
backfill / bfill: propagate next valid index value backward
nearest: use nearest valid index value

toleranceoptional

Maximum distance between original and new labels for inexact matches. The values of the index at the matching locations must satisfy the equation abs(index[indexer] - target) <= tolerance.

dropbool, optional

If drop=True, drop coordinates variables in indexers instead of making them scalar.

**indexers_kwargs{dim: indexer, …}, optional

The keyword arguments form of indexers. One of indexers or indexers_kwargs must be provided.

Returns

objDataArray: A new DataArray with the same contents as this DataArray, except the data and each dimension is indexed by the appropriate indexers. If indexer DataArrays have coordinates that do not conflict with this object, then these coordinates will be attached. In general, each array’s data will be a view of the array’s data in this DataArray, unless vectorized indexing was triggered by using an array indexer, in which case the data will be a copy.

Examples

>>> da = xr.DataArray(
...     np.arange(25).reshape(5, 5),
...     coords={"x": np.arange(5), "y": np.arange(5)},
...     dims=("x", "y"),
... )
>>> da
<xarray.DataArray (x: 5, y: 5)>
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24]])
Coordinates:
  * x        (x) int64 0 1 2 3 4
  * y        (y) int64 0 1 2 3 4

>>> tgt_x = xr.DataArray(np.linspace(0, 4, num=5), dims="points")
>>> tgt_y = xr.DataArray(np.linspace(0, 4, num=5), dims="points")
>>> da = da.sel(x=tgt_x, y=tgt_y, method="nearest")
>>> da
<xarray.DataArray (points: 5)>
array([ 0,  6, 12, 18, 24])
Coordinates:
    x        (points) int64 0 1 2 3 4
    y        (points) int64 0 1 2 3 4
Dimensions without coordinates: points

squeeze(*args, _apply_raw=True, **kwargs)[source]

Return a new object with squeezed data.

Parameters

dimNone or Hashable or iterable of Hashable, optional: Selects a subset of the length one dimensions. If a dimension is selected with length greater than one, an error is raised. If None, all length one dimensions are squeezed.
dropbool, optional: If drop=True, drop squeezed coordinates instead of making them scalar.
axisNone or int or iterable of int, optional: Like dim, but positional.

Returns

squeezedsame type as caller: This object, but with with all or a subset of the dimensions of length 1 removed.

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply std.
axisint or sequence of int, optional: Axis(es) over which to apply std. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then std is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating std on this object’s data.

Returns

reducedScore: New Score object with std applied to its data and the indicated dimension(s) removed.

sum(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying sum along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply sum.
axisint or sequence of int, optional: Axis(es) over which to apply sum. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then sum is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
min_countint, default: None: The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA. Only used if skipna is set to True or defaults to True for the array’s dtype. New in version 0.10.8: Added with the default being None. Changed in version 0.17.0: if specified on an integer array and skipna=True, the result will be a float array.
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating sum on this object’s data.

Returns

reducedScore: New Score object with sum applied to its data and the indicated dimension(s) removed.

var(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying var along some dimension(s).

Parameters

dimstr or sequence of str, optional: Dimension(s) over which to apply var.
axisint or sequence of int, optional: Axis(es) over which to apply var. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then var is calculated over axes.
skipnabool, optional: If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).
keep_attrsbool, optional: If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.
**kwargsdict: Additional keyword arguments passed on to the appropriate array function for calculating var on this object’s data.

Returns

reducedScore: New Score object with var applied to its data and the indicated dimension(s) removed.

Metrics

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

See Also

Examples

Parameters

Returns

Notes

Examples

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

Returns

Parameters

Returns

See Also

Parameters

Returns

See Also

Examples

Parameters

Returns

See Also

Parameters

Returns

Parameters

Returns

Parameters

Returns