A topic of great interest and controversy in the speech industry is competitive benchmarking, studies designed to determine which of two or more recognizers is most accurate. Unfortunately, it is easy to produce skewed results, either by mistake or deliberately, so that the result are misleading. We will discuss common errors in experimental design or analysis and ways to ensure that tests accurately correspond user experience. "