How Strong is the Confirmation of a Hypothesis by Significant Data?

Publication Year:
Usage 460
Downloads 460
Social Media 1
Tweets 1
Repository URL:
Bartelborth, Thomas
preprint description
The aim of the article is to determine how much a hypothesis H is actually confirmed if it has successfully passed a classical significance test. Bayesians have already raised many serious objections against significance testing, but in doing so they have always had to rely on epistemic probabilities and a further Bayesian analysis, which are rejected by classical statisticians. Therefore, I will suggest a purely frequentist evaluation procedure for significance tests, that should also be accepted by a classical statistician. This procedure likewise indicates some additional problems of significance tests. Such tests generally offer only incremental support of a hypothesis, although an absolute confirmation is necessary, and they overestimate positive results for small effects, since the confirmation of H in these cases is often rather marginal. This phenomenon leads in specific cases, for example, in cases of ESP-hypotheses, such as precognition, too easily to a significant confirmation. I will propose a method of how to evaluate and supplement significance tests so that we can avoid their epistemic deficits.