“Recently, A/B testing has come under (unjust) criticism from different
circles on the Internet. Even though this criticism contains some
relevant points, the basic argument against A/B testing is flawed. It
seems to confuse the A/B testing methodology with a specific
implementation of it (e.g. testing red vs. green buttons and other
trivial tests). Let’s look at different criticisms that have surfaced on
the Web recently and see why they are unfounded.”
http://www.smashingmagazine.com/2010/08/26/in-defense-of-a-b-testing/