mlm pval_adj: Adjusted or not?

[zanebeckwith]

Report

Thanks for the great project!

I'm confused about the reported p-values from the mlm module.

The name of the value padj_mlm indicates that it's been FDR adjusted. But, the implementation looks like it does no such correction.

And, I saw this question in the R version of decoupleR, saying that mlm doesn't perform p-value adjustment. (And, I also see this unit test in this repo which appears to be explicitly applying the BH correction to the padj_mlm)

I'm mostly just confused by the name, and by the fact that the MethodMeta created for the mlm function hardcodes adj=True

Just want to make sure I'm interpreting that value correctly!

Versions

[PauBadiaM]

Hi @zanebeckwith,

I see where the confusion is coming from. The adj=True you noticed in the metadata is unrelated to p-value adjustment, it indicates that the method takes the network as an adjacency matrix rather than as sets of genes.

What actually controls FDR correction is the test=True flag in the metadata, which mlm has, so padj_mlm is indeed FDR-adjusted (see here and here).

The unit test you linked is just checking the FDR utility in isolation: it passes in a set of p-values and verifies they get corrected properly, regardless of whether those inputs happened to already be adjusted.

Regarding the R version, there p-values are not corrected. As stated in the repository documentation, the package is deprecated and hasn't been updated in a few years.

Hope this is helpful!

[zanebeckwith]

Ah, yes, this is very helpful, thank you for clarifying this for me!

OK, that makes much more sense!

And, just to make sure, the ulm method also does FDR correction, right? Based on the test=True in the MethodMeta created in the _ulm.py module

(The context for this is the fact that you also told me, in the scverse discourse, that ulm is preferred over mlm for my used case. Sorry to open two semi-parallel threads! But, thank you for such great responses!)

[PauBadiaM]

It's fine 😉 Yes, in the python version of decoupler, if a method returns a p-value it is FDR corrected, so ulm p-values are corrected indeed.

[zanebeckwith]

@PauBadiaM one question: I just noticed that, in the _run method you linked to, the mlm model is explicitly kept from having the FDR correction applied, isn't it? (I.e. the check if name != "mlm": on line 116)

If we use ulm that won't matter, but I just wanted to check my understanding

[PauBadiaM]

Aha! You're absolutely right, I actually forgot about this, my bad! Yes, actually we don't apply FDR correction for mlm because it's a single multivariate test per observation that produces scores for all TFs/pathways at once, while ulm runs one test per TF per observation, so the correction is needed there.