Metrics

A Metric is part of a Benchmark and scores how similar two sets of data are. Typically these two sets are model and primate measurements, but metrics are agnostic of the data source and can also be used to compare two primate measurements (e.g. for ceiling estimates).

class brainscore_core.metrics.Metric[source]

Metric interface. A metric compares two sets of data and outputs a score of how well they match (1 = identical, 0 = no match).

__call__(assembly1: DataAssembly, assembly2: DataAssembly) → Score[source]

Compare two assemblies on their similarity. These assemblies are typically neural or behavioral measurements, e.g. model and primate recordings.

Parameters:

  • assembly1 – the first assembly to compare against the second

  • assembly2 – the second assembly to compare against the first

Returns:

a Score denoting the match between the two assemblies (1 = identical, 0 = no match).

class brainscore_core.metrics.Score(*args, **kwargs)[source]

Scores are used as the outputs of metrics, benchmarks, and ceilings. They indicate similarity or goodness-of-fit of sets of data. The high-level score is typically an aggregate of many smaller scores, e.g. the median of neuroid correlations. To keep records of these smaller scores, a score can store “raw” scores in its attributes (score.attrs[‘raw’]).

RAW_VALUES_KEY = 'raw'
all(dim=None, axis=None, **kwargs)

Reduce this Score’s data by applying all along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply all.

axisint or sequence of int, optional

Axis(es) over which to apply all. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then all is calculated over axes.

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating all on this object’s data.

Returns

reducedScore

New Score object with all applied to its data and the indicated dimension(s) removed.

any(dim=None, axis=None, **kwargs)

Reduce this Score’s data by applying any along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply any.

axisint or sequence of int, optional

Axis(es) over which to apply any. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then any is calculated over axes.

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating any on this object’s data.

Returns

reducedScore

New Score object with any applied to its data and the indicated dimension(s) removed.

count(dim=None, axis=None, **kwargs)

Reduce this Score’s data by applying count along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply count.

axisint or sequence of int, optional

Axis(es) over which to apply count. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then count is calculated over axes.

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating count on this object’s data.

Returns

reducedScore

New Score object with count applied to its data and the indicated dimension(s) removed.

cumprod(dim=None, axis=None, skipna=None, **kwargs)

Apply cumprod along some dimension of Score.

Parameters

dimstr or sequence of str, optional

Dimension over which to apply cumprod.

axisint or sequence of int, optional

Axis over which to apply cumprod. Only one of the ‘dim’ and ‘axis’ arguments can be supplied.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to cumprod.

Returns

cumvalueScore

New Score object with cumprod applied to its data along the indicated dimension.

cumsum(dim=None, axis=None, skipna=None, **kwargs)

Apply cumsum along some dimension of Score.

Parameters

dimstr or sequence of str, optional

Dimension over which to apply cumsum.

axisint or sequence of int, optional

Axis over which to apply cumsum. Only one of the ‘dim’ and ‘axis’ arguments can be supplied.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to cumsum.

Returns

cumvalueScore

New Score object with cumsum applied to its data along the indicated dimension.

expand_dims(*args, _apply_raw=True, **kwargs)[source]

Return a new object with an additional axis (or axes) inserted at the corresponding position in the array shape. The new object is a view into the underlying array, not a copy.

If dim is already a scalar coordinate, it will be promoted to a 1D coordinate consisting of a single value.

Parameters

dimhashable, sequence of hashable, dict, or None, optional

Dimensions to include on the new variable. If provided as str or sequence of str, then dimensions are inserted with length 1. If provided as a dict, then the keys are the new dimensions and the values are either integers (giving the length of the new dimensions) or sequence/ndarray (giving the coordinates of the new dimensions).

axisint, list of int or tuple of int, or None, default: None

Axis position(s) where new axis is to be inserted (position(s) on the result array). If a list (or tuple) of integers is passed, multiple axes are inserted. In this case, dim arguments should be same length list. If axis=None is passed, all the axes will be inserted to the start of the result array.

**dim_kwargsint or sequence or ndarray

The keywords are arbitrary dimensions being inserted and the values are either the lengths of the new dims (if int is given), or their coordinates. Note, this is an alternative to passing a dict to the dim kwarg and will only be used if dim is None.

Returns

expandedsame type as caller

This object, but with an additional dimension(s).

isel(*args, _apply_raw=True, **kwargs)[source]

Return a new DataArray whose data is given by integer indexing along the specified dimension(s).

Parameters

indexersdict, optional

A dict with keys matching dimensions and values given by integers, slice objects or arrays. indexer can be a integer, slice, array-like or DataArray. If DataArrays are passed as indexers, xarray-style indexing will be carried out. See indexing for the details. One of indexers or indexers_kwargs must be provided.

dropbool, optional

If drop=True, drop coordinates variables indexed by integers instead of making them scalar.

missing_dims{“raise”, “warn”, “ignore”}, default: “raise”

What to do if dimensions that should be selected from are not present in the DataArray: - “raise”: raise an exception - “warn”: raise a warning, and ignore the missing dimensions - “ignore”: ignore the missing dimensions

**indexers_kwargs{dim: indexer, …}, optional

The keyword arguments form of indexers.

See Also

Dataset.isel DataArray.sel

Examples

>>> da = xr.DataArray(np.arange(25).reshape(5, 5), dims=("x", "y"))
>>> da
<xarray.DataArray (x: 5, y: 5)>
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24]])
Dimensions without coordinates: x, y

>>> tgt_x = xr.DataArray(np.arange(0, 5), dims="points")
>>> tgt_y = xr.DataArray(np.arange(0, 5), dims="points")
>>> da = da.isel(x=tgt_x, y=tgt_y)
>>> da
<xarray.DataArray (points: 5)>
array([ 0,  6, 12, 18, 24])
Dimensions without coordinates: points
item(*args)

Copy an element of an array to a standard Python scalar and return it.

Parameters

*args : Arguments (variable number and type)

  • none: in this case, the method only works for arrays with one element (a.size == 1), which element is copied into a standard Python scalar object and returned.

  • int_type: this argument is interpreted as a flat index into the array, specifying which element to copy and return.

  • tuple of int_types: functions as does a single int_type argument, except that the argument is interpreted as an nd-index into the array.

Returns

zStandard Python scalar object

A copy of the specified element of the array as a suitable Python scalar

Notes

When the data type of a is longdouble or clongdouble, item() returns a scalar array object because there is no available Python scalar that would not lose information. Void arrays return a buffer object for item(), unless fields are defined, in which case a tuple is returned.

item is very similar to a[args], except, instead of an array scalar, a standard Python scalar is returned. This can be useful for speeding up access to elements of the array and doing arithmetic on elements of the array using Python’s optimized math.

Examples

>>> np.random.seed(123)
>>> x = np.random.randint(9, size=(3, 3))
>>> x
array([[2, 2, 6],
       [1, 3, 6],
       [1, 0, 1]])
>>> x.item(3)
1
>>> x.item(7)
0
>>> x.item((0, 1))
2
>>> x.item((2, 2))
1
max(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying max along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply max.

axisint or sequence of int, optional

Axis(es) over which to apply max. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then max is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating max on this object’s data.

Returns

reducedScore

New Score object with max applied to its data and the indicated dimension(s) removed.

mean(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying mean along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply mean.

axisint or sequence of int, optional

Axis(es) over which to apply mean. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then mean is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating mean on this object’s data.

Returns

reducedScore

New Score object with mean applied to its data and the indicated dimension(s) removed.

median(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying median along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply median.

axisint or sequence of int, optional

Axis(es) over which to apply median. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then median is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating median on this object’s data.

Returns

reducedScore

New Score object with median applied to its data and the indicated dimension(s) removed.

classmethod merge(*scores, exception_handling: str = 'ignore')[source]

Merges the raw values in addition to the score assemblies. Raw values are indexed on the first score.

Parameters:

exception_handling – how to deal with raised exceptions, one of ‘ignore’ (do not raise or log), ‘warn’ (log but do not raise), ‘raise’ (raise exception).

min(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying min along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply min.

axisint or sequence of int, optional

Axis(es) over which to apply min. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then min is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating min on this object’s data.

Returns

reducedScore

New Score object with min applied to its data and the indicated dimension(s) removed.

prod(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying prod along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply prod.

axisint or sequence of int, optional

Axis(es) over which to apply prod. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then prod is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

min_countint, default: None

The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA. Only used if skipna is set to True or defaults to True for the array’s dtype. New in version 0.10.8: Added with the default being None. Changed in version 0.17.0: if specified on an integer array and skipna=True, the result will be a float array.

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating prod on this object’s data.

Returns

reducedScore

New Score object with prod applied to its data and the indicated dimension(s) removed.

reduce(*args, _apply_raw=False, **kwargs)[source]

Reduce this array by applying func along some dimension(s).

Parameters

funccallable

Function which can be called in the form f(x, axis=axis, **kwargs) to return the result of reducing an np.ndarray over an integer valued axis.

dimhashable or sequence of hashable, optional

Dimension(s) over which to apply func.

axisint or sequence of int, optional

Axis(es) over which to repeatedly apply func. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then the reduction is calculated over the flattened array (by calling f(x) without an axis argument).

keep_attrsbool, optional

If True, the variable’s attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

keepdimsbool, default: False

If True, the dimensions which are reduced are left in the result as dimensions of size one. Coordinates that use these dimensions are removed.

**kwargsdict

Additional keyword arguments passed on to func.

Returns

reducedDataArray

DataArray with this object’s array replaced with an array with summarized data and the indicated dimension(s) removed.

searchsorted(v, side='left', sorter=None)

Find indices where elements of v should be inserted in a to maintain order.

For full documentation, see numpy.searchsorted

See Also

numpy.searchsorted : equivalent function

sel(*args, _apply_raw=True, **kwargs)[source]

Return a new DataArray whose data is given by selecting index labels along the specified dimension(s).

In contrast to DataArray.isel, indexers for this method should use labels instead of integers.

Under the hood, this method is powered by using pandas’s powerful Index objects. This makes label based indexing essentially just as fast as using integer indexing.

It also means this method uses pandas’s (well documented) logic for indexing. This means you can use string shortcuts for datetime indexes (e.g., ‘2000-01’ to select all values in January 2000). It also means that slices are treated as inclusive of both the start and stop values, unlike normal Python indexing.

Warning

Do not try to assign values when using any of the indexing methods isel or sel:

da = xr.DataArray([0, 1, 2, 3], dims=['x'])
# DO NOT do this
da.isel(x=[0, 1, 2])[1] = -1

Assigning values with the chained indexing using .sel or .isel fails silently.

Parameters

indexersdict, optional

A dict with keys matching dimensions and values given by scalars, slices or arrays of tick labels. For dimensions with multi-index, the indexer may also be a dict-like object with keys matching index level names. If DataArrays are passed as indexers, xarray-style indexing will be carried out. See indexing for the details. One of indexers or indexers_kwargs must be provided.

method{None, “nearest”, “pad”, “ffill”, “backfill”, “bfill”}, optional

Method to use for inexact matches:

  • None (default): only exact matches

  • pad / ffill: propagate last valid index value forward

  • backfill / bfill: propagate next valid index value backward

  • nearest: use nearest valid index value

toleranceoptional

Maximum distance between original and new labels for inexact matches. The values of the index at the matching locations must satisfy the equation abs(index[indexer] - target) <= tolerance.

dropbool, optional

If drop=True, drop coordinates variables in indexers instead of making them scalar.

**indexers_kwargs{dim: indexer, …}, optional

The keyword arguments form of indexers. One of indexers or indexers_kwargs must be provided.

Returns

objDataArray

A new DataArray with the same contents as this DataArray, except the data and each dimension is indexed by the appropriate indexers. If indexer DataArrays have coordinates that do not conflict with this object, then these coordinates will be attached. In general, each array’s data will be a view of the array’s data in this DataArray, unless vectorized indexing was triggered by using an array indexer, in which case the data will be a copy.

See Also

Dataset.sel DataArray.isel

Examples

>>> da = xr.DataArray(
...     np.arange(25).reshape(5, 5),
...     coords={"x": np.arange(5), "y": np.arange(5)},
...     dims=("x", "y"),
... )
>>> da
<xarray.DataArray (x: 5, y: 5)>
array([[ 0,  1,  2,  3,  4],
       [ 5,  6,  7,  8,  9],
       [10, 11, 12, 13, 14],
       [15, 16, 17, 18, 19],
       [20, 21, 22, 23, 24]])
Coordinates:
  * x        (x) int64 0 1 2 3 4
  * y        (y) int64 0 1 2 3 4

>>> tgt_x = xr.DataArray(np.linspace(0, 4, num=5), dims="points")
>>> tgt_y = xr.DataArray(np.linspace(0, 4, num=5), dims="points")
>>> da = da.sel(x=tgt_x, y=tgt_y, method="nearest")
>>> da
<xarray.DataArray (points: 5)>
array([ 0,  6, 12, 18, 24])
Coordinates:
    x        (points) int64 0 1 2 3 4
    y        (points) int64 0 1 2 3 4
Dimensions without coordinates: points
squeeze(*args, _apply_raw=True, **kwargs)[source]

Return a new object with squeezed data.

Parameters

dimNone or Hashable or iterable of Hashable, optional

Selects a subset of the length one dimensions. If a dimension is selected with length greater than one, an error is raised. If None, all length one dimensions are squeezed.

dropbool, optional

If drop=True, drop squeezed coordinates instead of making them scalar.

axisNone or int or iterable of int, optional

Like dim, but positional.

Returns

squeezedsame type as caller

This object, but with with all or a subset of the dimensions of length 1 removed.

See Also

numpy.squeeze

std(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying std along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply std.

axisint or sequence of int, optional

Axis(es) over which to apply std. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then std is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating std on this object’s data.

Returns

reducedScore

New Score object with std applied to its data and the indicated dimension(s) removed.

sum(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying sum along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply sum.

axisint or sequence of int, optional

Axis(es) over which to apply sum. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then sum is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

min_countint, default: None

The required number of valid values to perform the operation. If fewer than min_count non-NA values are present the result will be NA. Only used if skipna is set to True or defaults to True for the array’s dtype. New in version 0.10.8: Added with the default being None. Changed in version 0.17.0: if specified on an integer array and skipna=True, the result will be a float array.

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating sum on this object’s data.

Returns

reducedScore

New Score object with sum applied to its data and the indicated dimension(s) removed.

var(dim=None, axis=None, skipna=None, **kwargs)

Reduce this Score’s data by applying var along some dimension(s).

Parameters

dimstr or sequence of str, optional

Dimension(s) over which to apply var.

axisint or sequence of int, optional

Axis(es) over which to apply var. Only one of the ‘dim’ and ‘axis’ arguments can be supplied. If neither are supplied, then var is calculated over axes.

skipnabool, optional

If True, skip missing values (as marked by NaN). By default, only skips missing values for float dtypes; other dtypes either do not have a sentinel missing value (int) or skipna=True has not been implemented (object, datetime64 or timedelta64).

keep_attrsbool, optional

If True, the attributes (attrs) will be copied from the original object to the new one. If False (default), the new object will be returned without attributes.

**kwargsdict

Additional keyword arguments passed on to the appropriate array function for calculating var on this object’s data.

Returns

reducedScore

New Score object with var applied to its data and the indicated dimension(s) removed.