Can I read Cryptic Multiple Hypotheses Testing in Linear Models: Overestimated Effect Sizes and the Winner's Curse on EtoBox?

Cryptic Multiple Hypotheses Testing in Linear Models: Overestimated Effect Sizes and the Winner's Curse by Wolfgang Forstmeier; Holger Schielzeth is a Agricultural and Biological Sciences article available to read on EtoBox.

What is Cryptic Multiple Hypotheses Testing in Linear Models: Overestimated Effect Sizes and the Winner's Curse about?

Fitting generalised linear models (GLMs) with more than one predictor has become the standard method of analysis in evolutionary and behavioural research. Often, GLMs are used for exploratory data analysis, where one starts with a complex full model including interaction terms and then simplifies by removing non-significant terms. While this approach can be useful, it is problematic if significant effects are interpreted as if they arose from a single a priori hypothesis test. This is because model selection involves cryptic multiple hypothesis testing, a fact that has only rarely been acknowledged or quantified. We show that the probability of finding at least one ‘significant’ effect is high, even if all null hypotheses are true (e.g. 40% when starting with four predictors and their two-way interactions). This probability is close to theoretical expectations when the sample size (__N__) is large relative to the number of predictors including interactions (__k__). In contrast, type I error rates strongly exceed even those expectations when model simplification is applied to models that are over-fitted before simplification (low __N__/__k__ ratio). The increase in false-positive re

Who reads Cryptic Multiple Hypotheses Testing in Linear Models: Overestimated Effect Sizes and the Winner's Curse?

Author

Wolfgang Forstmeier; Holger Schielzeth

Publisher

Springer; Springer-Verlag; Springer Verlag; Springer Science and Business Media LLC; Society for Mining, Metallurgy and Exploration Inc. (ISSN 0340-5443)

Published

2010

Language

EN

Field

Agricultural and Biological Sciences (Life Sciences)