scikit_posthocs.posthoc_demsar

scikit_posthocs.posthoc_demsar(a: list | ndarray | DataFrame, y_col: str | None = None, group_col: str | None = None, block_col: str | None = None, block_id_col: str | None = None, control: str | None = None, alternative: Literal['two-sided', 'less', 'greater'] = 'two-sided', p_adjust: str | None = None, melted: bool = False, sort: bool = False, to_matrix: bool = True) → Series | DataFrame

Demsar’s many-to-one test for unreplicated blocked data [1], comparing several treatments against one control (e.g. a baseline algorithm) across a set of blocks (e.g. datasets), based on Friedman-type ranks.

Parameters:

  • a (array_like or pandas DataFrame object) –

    An array, any object exposing the array interface or a pandas DataFrame.

    If melted is set to False (default), a is a typical matrix of block design, i.e. rows are blocks, and columns are groups. In this case you do not need to specify col arguments.

    If a is an array and melted is set to True, y_col, block_col and group_col must specify the indices of columns containing elements of correspondary type.

    If a is a Pandas DataFrame and melted is set to True, y_col, block_col and group_col must specify columns names (strings).

  • y_col (str or int) – Must be specified if a is a pandas DataFrame object. Name of the column that contains y data.

  • group_col (str or int) – Must be specified if a is a pandas DataFrame object. Name of the column that contains treatment (group) factor values.

  • block_col (str or int) – Must be specified if a is a pandas DataFrame object. Name of the column that contains blocking factor values.

  • block_id_col (str or int) – Must be specified if a is a pandas DataFrame object. Name of the column that contains identifiers of blocking factor values.

  • control (str, optional) – Name of the control group within the group_col column. Must be specified if a is a pandas DataFrame.

  • alternative (['two-sided', 'less', or 'greater'], optional) – Whether to get the p-value for the one-sided hypothesis (‘less’ or ‘greater’) or for the two-sided hypothesis (‘two-sided’). Defaults to ‘two-sided’.

  • p_adjust (str, optional) – Method for adjusting p values across the treatment-vs-control comparisons. See statsmodels.sandbox.stats.multicomp for details.

  • melted (bool, optional) – Specifies if data are given as melted columns “y”, “blocks”, and “groups”.

  • sort (bool, optional) – If True, sort data by block and group columns.

  • to_matrix (bool, optional) – Specifies whether to return a DataFrame or a Series. If True, a DataFrame is returned with some NaN values since it’s not a pairwise comparison. Default is True.

Returns:

result – P values.

Return type:

pandas.Series or pandas.DataFrame

Notes

P values are computed from the standard normal distribution applied to the difference in mean Friedman-type ranks between each treatment and the control. This is the standard procedure for comparing multiple classifiers/algorithms against a baseline across multiple datasets [1].

References

Examples

>>> import scikit_posthocs as sp
>>> import numpy as np
>>> x = np.array([[31,27,24],[31,28,31],[45,29,46],[21,18,48],[42,36,46],[32,17,40]])
>>> sp.posthoc_demsar(x, control=0)