Reminds me of that classic scam of sending out sports (or anything else) predictions to a huge mailing list, and basically AB testing in such a way that after a little time there are at least x% who think your success rate is incredible.
Yeah. Done historically by stockbrokers making cold calls. Every week, "Now, don't put your money with me just yet. Here's what I'm putting my clients into this week. Just watch Stock XYZ this week to see what happens." If you pick three highly volatile stocks every week, make a lot of calls, and drop the prospects to whom you have given "bad" advice, you'll end-up with a decent sized set of potential clients who think you are a god.
In a scam, you don't have any way of making any future correct predictions--it's just random. This analyst has a reasoned system that can be tested against past players and against future prospects. Sure, there's going to be someone out there that picks Jeremy Lin. But the trick is, can you correctly identify players who are already known to be good based on their college stats, make the right call on Jeremy Lin, and go forward and continue to make good calls on draft prospects?
I'm not saying this sabermetrician generally makes good calls; I haven't looked. But dismissing his call as the result of a series of Bernoulli trials is completely missing the point.
His point is the number of people making predictions is high enough that it raises the bar for people to stand out from the pack. Basically run an ANOVA test on 10 groups vs 10,000 groups and it takes a larger divination to stand out from the pack.
PS: It does not matter if there is a system behind their predictions, what matters is how well their predictions hold up over time. Otherwise your just: http://xkcd.com/904/
In the scam one person is making loads of predictions knowing that some will pay off. In this story, lots of people are making predictions and again, the chances are some of them will be right. Read the comment I replied to, his point was exactly the same as the scam's logic, which is basically that if enough monkeys spend enough time with typewriters... etc.
There's a very important difference that you're missing. In the scam you're applying you're random machine over all possible results. In the sabermetrician example, the random machine is being applied across methodologies of picking prospects, ie bunch of different "moneyball" hobbyists trying out different stats and weights to rank players. The end result is the possibility of the discovery of a very good methodology that has found stats that traditionally are not important.
But it takes more than one result to tell whether the methodology is good or if it was just lucky - otherwise it's just a guy tossing a coin once and saying "see, I predicted heads!"
