Scaling predictor variables both in training and test datasets about bigstatsr HOT 5 OPEN

• big_spLinReg() may perform some scaling internally, but it will always return effect sizes corresponding to the input data (i.e. before any internal scaling).
• I guess the way you compute r2 is wrong since it should be invariant of any scale of either pred and true; just use cor()^2 or pcor()^2 (possibly also keeping the sign).

from bigstatsr.

Thank you for the very fast response!

Do you mean that if I want my predictors to be on the same scale in test data than in training data, I should scale them externally? How to do that then in case of genotype data?

Thanks for pointing out the mistake in R2 calculation. However, I'm a little confused because I have been taught that when calculating R2, MSE should be used to ensure that predictions are on the correct scale of the outcome variable, and that using cor()^2 can be misleadingly optimistic. Can you explain why I should ignore the linear scale difference between the prediction and the outcome?

from bigstatsr.

I am not saying that you should not care about the prediction not being calibrated; I'm just saying that R2 does not measure calibration.

If you use the training data as is, which is then scale internally in big_spLinReg() but unscale to report the effects, then you should not have to scale the test data.

from bigstatsr.