Comments (4)
Hey! I think this may be due to insufficient statistical power of Dunn's test. You can try Conover's test instead. I will look for the relevant research on this. And yes, Dunn's and Conover's tests require p values correction.
from scikit-posthocs.
@maximtrp Thanks for your reply! I just tried Conover's test and it is still the same. In fact, the corrected pairwise p-values are actually higher with Conover's test.
I followed this with Bonferroni-Holm correction (p_adjust='holm'
)
from scikit-posthocs.
Hi,
I met the same situation recently. Below is the example from [1] regarding four algorithms over 14 datasets.
# %%
import numpy as np
data = [
[0.763, 0.768, 0.771, 0.798],
[0.599, 0.591, 0.590, 0.569],
[0.954, 0.971, 0.968, 0.967],
[0.628, 0.661, 0.654, 0.657],
[0.882, 0.888, 0.886, 0.898],
[0.936, 0.931, 0.916, 0.931],
[0.661, 0.668, 0.609, 0.685],
[0.583, 0.583, 0.563, 0.625],
[0.775, 0.838, 0.866, 0.875],
[1.000, 1.000, 1.000, 1.000],
[0.940, 0.962, 0.965, 0.962],
[0.619, 0.666, 0.614, 0.669],
[0.972, 0.981, 0.975, 0.975],
[0.957, 0.978, 0.946, 0.970]
]
data = np.array(data)
# %%
import scikit_posthocs as sp
sp.posthoc_dunn(data.T, p_adjust='bonferroni')
But it returns meaningless results:
1 | 2 | 3 | 4 | |
---|---|---|---|---|
1.0 | 1.0 | 1.0 | 1.0 | |
1.0 | 1.0 | 1.0 | 1.0 | |
1.0 | 1.0 | 1.0 | 1.0 | |
1.0 | 1.0 | 1.0 | 1.0 |
I observed that the implementation of posthoc_dunn ranks the entire data matrix, while [1] is row-wise. Did this make any difference ?
Thanks a lot !
[1] Dem\check{s}ar, J. Statistical comparisons of classifiers over multiple data sets. JMLR, 2006.
from scikit-posthocs.
I have checked the algorithm and found no errors. Dunn suggests ranking all data in the original paper.
from scikit-posthocs.
Related Issues (20)
- Is it possible to also return the test statistic instead of only p-values? HOT 1
- ValueError: Format specifier missing precision HOT 1
- Add hypo argument for outliers_gesd HOT 1
- Can't Install HOT 1
- Post-hocs test for dataframes with different group / block / y column names break HOT 4
- Rotation of the vertical ticks in the significance matrix. HOT 1
- v0.6.8 in __init__.py HOT 1
- Cannot import under python 2.7 by using Spyder HOT 1
- Significance Plots Colour Bar Breaks Using Matplotlib >= 3.5.0 HOT 1
- Solving ValueError; 'All numbers are identical in mannwhitneyu' HOT 3
- Use posthoc tests to plot critical difference diagram HOT 1
- [Feature request] outlier_gesd to return data even when report=True HOT 1
- Wilcoxon paired test is not really paired HOT 1
- posthoc_tukey yields p-values of 0.900 and 0.100 which are also different to scipy.stats.tukey_hsd HOT 1
- custom comparisons on scikit_posthocs.posthoc_tukey
- posthoc_conover_friedman calculates incorrect p-values HOT 9
- Dunn test incorrect repeating p values after adjustment HOT 1
- Alpha cannot be specified for critical difference diagram HOT 1
- add method argument in posthoc_wilcoxon
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scikit-posthocs.