Rather than using false_positive_percentage to set your stringency, we use a new column that I created that is called false_positive_rate. about sccomp HOT 18 CLOSED

This is the schematics I showed before

from sccomp.

Hi Stefano,

I am still confused about how the one-step discovery rate is calculated. Is there a paradigm or algorithm for it?

from sccomp.

Hi Stefano,

I am still confused about how the one-step discovery rate is calculated. Is there a paradigm or algorithm for it?

Can you show me the results from an execution with the dev branch?

from sccomp.

from sccomp.

By one execution, do you refer to the sccomp_glm function?

Yes, can you paste the result table here?

from sccomp.

from sccomp.

The first excel is the data for regression, and the second one is table of result.

I don't see EXCELs here in github comments. You can simply copy and paste here the result table. I want to see what columns there are in your result table.

from sccomp.

cell_group .lower_(Intercept) .lower_type1 .median_(Interce~ .median_type1 .upper_(Interc~ .upper_type1 significant

1 1 0.0514 0.181 0.430 0.761 0.841 1.36 TRUE
2 2 -0.481 -0.823 -0.0625 -0.264 0.423 0.327 FALSE
3 3 -0.479 -0.566 -0.0739 0.00293 0.396 0.619 FALSE
4 4 -0.588 -1.01 -0.156 -0.391 0.281 0.169 FALSE
5 5 -0.384 -0.813 0.00390 -0.139 0.452 0.418 FALSE
6 6 -0.452 -0.0177 -0.0679 0.577 0.393 1.15 FALSE
7 7 -0.340 -0.211 0.0538 0.372 0.488 0.968 FALSE
8 8 -0.688 -0.596 -0.271 0.000966 0.208 0.553 FALSE
9 9 -0.903 -0.744 -0.469 -0.173 0.00544 0.378 FALSE
10 10 -0.523 -1.14 -0.0574 -0.559 0.394 0.0581 FALSE
11 11 -0.00232 -0.595 0.398 0.00719 0.797 0.636 FALSE
12 12 -0.0513 -0.398 0.351 0.161 0.806 0.718 FALSE
13 13 -0.277 -0.0959 0.126 0.509 0.594 1.12 FALSE
14 14 -0.319 -0.812 0.114 -0.253 0.645 0.321 FALSE
15 15 -0.854 -0.862 -0.375 -0.290 0.155 0.270 FALSE
16 16 -0.437 -0.772 -0.0335 -0.182 0.410 0.396 FALSE
17 17 -0.378 -1.21 0.0478 -0.616 0.560 -0.0261 TRUE
18 18 -0.524 -0.495 -0.139 0.120 0.321 0.748 FALSE
19 19 0.100 -0.385 0.472 0.188 0.872 0.729 FALSE
20 20 -0.925 -0.402 -0.452 0.169 0.0590 0.754 FALSE

from sccomp.

If you install the dev version you should have false_positive_rate

from sccomp.

prob_non_zero false_discovery_rate concentration

1 0.0275 0.0275 <tibble [1 x 3]>
2 0.466 0.215 <tibble [1 x 3]>
3 0.990 0.470 <tibble [1 x 3]>
4 0.286 0.154 <tibble [1 x 3]>
5 0.702 0.388 <tibble [1 x 3]>
6 0.109 0.0732 <tibble [1 x 3]>
7 0.274 0.132 <tibble [1 x 3]>
8 0.999 0.496 <tibble [1 x 3]>
9 0.642 0.367 <tibble [1 x 3]>
10 0.142 0.0905 <tibble [1 x 3]>
11 0.987 0.441 <tibble [1 x 3]>
12 0.627 0.326 <tibble [1 x 3]>
13 0.158 0.104 <tibble [1 x 3]>
14 0.488 0.243 <tibble [1 x 3]>
15 0.392 0.184 <tibble [1 x 3]>
16 0.620 0.300 <tibble [1 x 3]>
17 0.083 0.0552 <tibble [1 x 3]>
18 0.74 0.409 <tibble [1 x 3]>
19 0.560 0.271 <tibble [1 x 3]>
20 0.632 0.347 <tibble [1 x 3]>

from sccomp.

Ok, did you learn everything about false discovery rate?

Do you know now how to test for how many false positives are in the first N positions of your false discovery rate rank?

from sccomp.

If you studied false discovery rate you will see that if you order the column according to that, the first N estimates, should have on average false_discovery_rate probability of being false. Then you can test whether this is the case from N that goes from 1 to ...

from sccomp.

In this case, false discovery rate should be a non-decreasing function on (number of) category N ? N*false_discovery_rate == false_positive_rate?

from sccomp.

N*false_discovery_rate == average number of false discoveries.

For example

1. if you have 20 categories with a false discovery rate < 0.5, on average you will have 10 categories that will be errors/false positives
2. if you have 30 categories with a false discovery rate < 0.5, on average you will have 15 categories that will be errors/false positives
3. if you have 30 categories with a false discovery rate < 0.1, on average you will have 3 categories that will be errors/false positives

from sccomp.

Then, what do I need to calculate or analysize?

from sccomp.

Same calibration you have done already.

Estimated false positives vs real false positive, for false_discovery_rate of 0.01, 0.02, 0.03, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5 ...

from sccomp.

the Estimated false positives should be N*false discovery rate, right?

from sccomp.

fp_aimed fp_estimated
0.00 0.000
0.01 0.000
0.02 0.015
0.03 0.040
0.05 0.070
0.10 0.165
0.20 0.370
0.30 0.570
0.40 0.785
0.50 0.940
0.60 1.000
0.70 1.000
0.80 1.000
0.90 1.000
1.00 1.000

from sccomp.