> ... It's called beat the cheating detector. If you have access to the detector, it becomes trivial even: no matter how advanced the detector, using a chess engine to generate a set of moves and picking the best ones which go undetected works.
Except that you have to beat all future detectors as well, which is a lot harder to get access to. E.g. this is how athletes sometimes get caught out doping; they use methods which are undetectable today, but fail in a year or two.
I don't really want to press this comparison too far though, not only are the prevalent levels completely different, but doping is also a lot less clear cut than cheating in chess / counter strike. It's pretty obvious when you're cheating in the one, whereas you could argue that using as-yet not banned substances is fair game. Mind you, we have had players type in recoil-resetting scripts at competitions, and argue that it's just a configuration, so perhaps it's not as clear cut as I make it sound.
As long as the detectors don't directly detect cheating, but instead improvement at a rate that is too high (like in the article), then if you limit your rate of improvement to a rate and distribution similar to human improvement, it is unlikely that any future program will be able to detect this.
Detecting performance enhancement solely by looking at performance is a losing battle for the detecting side.
Sure, but forcing cheaters to rate-limit their improvements to within reasonable limits would already be a huge win for the detecting side, so I'm not sure I'd call it a losing battle. It'd severely limit the utility of cheating, while at the same time forcing a long history of slow improvement which can be scrutinised.
Taken by itself, I'd tend to agree with you. In practice though, I think this can easily swing the balance in favour of the detecting side. Another angle to analyse chess games would be to see the degree to which a player's moves correlate with known chess engines. Over a long term (say, a few dozen tournaments worth of games), this type of evidence would be particularly damning; the types of chess engines which play relatively human-like, yet still are top-tier simply don't exist. Given the difficulty in creating them, and the relative rewards, I wonder if it will ever be practical to cheat.
Except that you have to beat all future detectors as well, which is a lot harder to get access to. E.g. this is how athletes sometimes get caught out doping; they use methods which are undetectable today, but fail in a year or two.
I don't really want to press this comparison too far though, not only are the prevalent levels completely different, but doping is also a lot less clear cut than cheating in chess / counter strike. It's pretty obvious when you're cheating in the one, whereas you could argue that using as-yet not banned substances is fair game. Mind you, we have had players type in recoil-resetting scripts at competitions, and argue that it's just a configuration, so perhaps it's not as clear cut as I make it sound.