Separating Skill from Luck in Investment Performance
As previously written by Covenant Capital Management
“So, how did you guys do last month?”
It seems like the most natural and logical question to ask an investment manager. But is it the right question to ask? I suspect that most managers either cringe or rejoice internally upon hearing this question depending on how they actually performed the previous month. Almost implied in the question is the notion that the manager’s recent performance is some sort of indicator of his ‘skill’ at investing. But was it skill or was it luck (good or bad) that constituted that month’s performance? And how can one possibly separate the two?
A Laboratory for Luck and Skill
Luckily (or should we say skillfully), there is a scientific way to approach the problem. We can consider a game of chance with some known degree of ‘inherent skill’. For instance imagine a trader who has a 50% chance of winning each trade he takes. Suppose the trader makes twice as much on his winning trades than he loses on his losing trades. This is equivalent to being paid 2 to 1 on a fair coin flip. The long run expected outcome of each trade is $0.501. This $0.50 expectancy is the ‘inherent skill’ of the trader. But an outside observer does not have this knowledge. The outside observer only has the trader’s track record and must use it to somehow decipher the trader’s abilities. So how is this best accomplished?
Another way to pose the question above is to say “How should one look at a track record in a way that will minimize the effects of randomness?” In the scenario above, randomness is manifested in whether or not a series of actual trades turns out to be winners or losers. Even though the chance of winning each trade is 50%, this does not mean that in 10 trades exactly 5 will be winners. There is some possibility that the trader will only win 3 of the next 10 trades – it is the exact same possibility that 10 coin flips will yield only 3 heads. If the trader wins 3 (or fewer) trades over the course of 10, his actual performance per trade is negative (-$0.10 per trade for 3 wins) as opposed to his positive $0.50 expectancy. Thus his actual performance over these 10 trades is vastly different from his long term expectancy.
The mathematical tool for determining the likelihood of observing 3 or fewer wins in 10 trades if the underlying chance of winning is 50% is the binomial distribution. It turns out that the chance of winning 3 or fewer trades in this scenario is 17.2%. Put another way, even though the trader has a skill which implies he should expect to make $0.50 per trade, there is roughly a 1 in 5 chance that over the course of 10 trades he will lose money – simply due to randomness. Similarly, there is some chance that the trader will win 6 or more trades out of 10, in which case his realized performance is well above his $0.50 expectancy. According to the binomial distribution, the chance of the trader winning 6 or more out of 10 trades is 37.7%. That is there is a 37.7% chance that the trader’s performance over 10 trades will be better than his long term expectancy.
Time Will Tell
The key to separating ‘skill’ from ‘luck’ lies in the fact that the odds in the scenarios given above actually change depending on the number of (independent) observations3. For instance while the chance of winning 3 or fewer trades out of 10 is 17.2%, the chance of winning 6 or fewer trades out of 20 is only 5.8%. Figure 1 illustrates how the odds of observing a winning percentage of less than or equal to 30% for a situation with a true winning percentage of 50% diminish as the number of observations increases.
From the exercise above it should be clear that if a trader truly has skill, it will be more obvious over the long run as opposed to the short run. Put another way, the short term good and bad luck offset over the long run and all that remains is the manager’s skill or ‘market edge’.
About the author: Scot Billington co-founded Covenant Capital Management, a boutique CTA that has been managing client assets for more than 15 years. He co-developed the CCM trading models and manages the CCM office in Chicago. Founded in 1999, Covenant Capital Management, LLC (CCM) is an alternative asset manager. CCM is registered with the Commodity Futures Trading Commission as a Commodity Trading Advisor (CTA), and manages client assets using global futures markets as an investment vehicle. CCM currently trades in over 40 different markets including equity indexes, fixed income, foreign exchange, agricultural, metals, soft commodities, and energy. CCM offers three primary programs, the Original Program, the Aggressive Program and the Optimal Program. The Aggressive Program is a more leveraged version of the Original Program.
1 The expectancy or long run average is $0.50 since 50% of the trades result in a $2.00 gain and 50% of the trades result in a $1.00 loss. Expected Outcome = $0.50 = $2*(0.50) – $1*(0.50).