HomePage | RecentChanges | Preferences

Statistical Criticism

HELP! We really need this kind of criticism

From pb's comments on Kuro5hin (http://www.kuro5hin.org/story/2003/7/29/17659/1721) :

I'd like to mention that just doing 10 runs isn't really going to generate reliable data. From a little ad-hoc experimentation, I can tell you that doing 100 runs could get you to about 2% of doing 1000 runs, which should be a lot closer to the actual values. However, only doing 10 runs could give you margins of error of 40% or so, easily.

You could account for this by noting how much the data varies between runs, and doing more runs until the average stabilizes. Or you could do the math, and try to compute a confidence interval of some sort.

Accusations of Bias Damaging the Project

Other Methodological Problems

Back to WeNeedYourCriticism

HomePage | RecentChanges | Preferences
This page is read-only | View other revisions
Last edited July 29, 2003 11:45 pm by PhilJones (diff)