This post was first published in April 2010 on the FiduciaryX website. I am posting it again today on the new Bdellium blog just to have a complete record of my (few) previous posts in one location.
In my blog post Lessons From The Copula Function I raised the concern that some investment fiduciaries might be following the same path to self destruction as the credit derivatives industry, paved by the indiscriminate use of decision-making methods that are at best ineffective and at worst actually dangerous.
Ask The Right Questions
If you want to avoid the worst dangers you need to be able to confidently answer “yes” to each of the following three questions in respect of any method you use or that is used by your advisors:
1. Do I understand the method?
2. Can I verify the method?
3. Can I use the method to achieve better results?
Superficially these questions seem simple and you might be tempted either to dismiss them or quickly check the “Yes” box and move on. However, as I have learned from direct experience over almost 30 years, the more diligently you apply these standards the more effective you will become at achieving your goals and avoiding disaster.
Simple, Transparent, Practical
The corollary to these questions is that software vendors, rating agencies, advisors and other service providers must ensure that the methods they use are simple so that they can be understood, transparent so that they can be verified and practical so that they can be put to use. One of the ways I judge prospective service providers and partners is by how willing and able they are to help me fully answer these three questions.
Let’s consider each question in turn, starting with understanding the method. The critical lesson here is that there is a world of difference between being familiar with a general methodology and understanding the concrete steps used in a specific implementation of that methodology.
For example, Morningstar is very good at publishing its major methodologies but how many investors or fiduciaries actually read and understand them. If you use any tools from Morningstar try this simple test – do you know how the rescaling step in the Morningstar Style Box Methodology can lead to misinterpretation of their popular graphs? I will provide the answer in a future post.
Secret Sauces are For Chefs
Some service providers will claim that they cannot provide a detailed explanation of their methods because that would require them to reveal their “secret sauce”. Instead they present marketing material that is designed to impress but not to inform. I can understand why traders closely guard their methods but I am very skeptical of companies who sell software, publish ratings or provide advice based on methods that they are not willing to fully disclose.
I once contacted a company that was receiving huge publicity for a new service they were providing. I wanted to understand their methods and I was encouraged by their PR which heralded their commitment to transparency. In fairness to them they were willing to answer most of my questions but they told me that they would not be publishing their methods because – you guessed it – they had to guard their secret sauce.
As a result of our discussions I was able to identify a number of serious deficiencies in their methods which they acknowledged at the time. They assured me however that that they would be making improvements as they learned from experience. Interestingly, the media and some industry pundits continue to hail this company as a white knight of transparency without seeming to recognize the irony that the company continues to keep their own methods secret. I also see no evidence that they have fixed the acknowledged flaws in their calculations while they increasingly claim to represent “best practices” in their chosen field.
Call me cynical but I suspect that in too many cases the “secret sauce” excuse is more an attempt to avoid third-party scrutiny that might reveal flaws in a service provider’s method than it is a valid commercial necessity. Humans inevitably make mistakes, particularly if they are trying to expand the boundaries of achievement. Therefore a service provider who acknowledges, corrects and learns from an occasional error should not be automatically disqualified. However if somebody deliberately tries to hide possible errors for their commercial gain fiduciaries need to be unforgiving or face legal challenges when problems inevitably emerge.
Trust The Person, Verify The Method
Even if the service provider is trustworthy that is no guarantee that their methods are always sound. Ironically, because it is human nature not to want to challenge a good person too strongly, honest, well-meaning people can sometimes inflict more damage through unintentional mistakes than more questionable operators against whom we are normally on our guard. By all means trust the person but always verify the calculations.
One of the most unfortunate side effects of ERISA is that the potential for personal liability can make plan sponsors afraid to exercise independent judgment. Common sense is not enough, it must be “expert” common sense and there is now a host of “experts” ready to tell plan sponsors what to do. It is not unusual for service providers to advertise the academic qualifications and experience of their executives but sometimes there is a subtle undertone that investors or fiduciaries should defer to these experts in areas where they do not have similar formal qualifications. My advice is never to be afraid to challenge an expert opinion if it conflicts with common sense.
The true measure of an expert is not the complexity of the methods they use but their ability to deliver valuable, actionable insights using methods that are intelligible to normal business professionals.
Effective Fiduciaries Value Function Over Fashion
In general, provided a fiduciary makes an honest effort to understand, I think that the main burden for explaining and proving the accuracy of any method falls on the service provider who uses it. However, when it comes to answering our third question – whether the method is useful to achieving better results, the pendulum swings back to the fiduciary. In my opinion, the plan sponsor or institutional investor retain primary responsibility for clearly defining their goals and determining which methods and tools will usefully help achieve them. Effective fiduciaries are willing to reject fashionable methods if they don’t work and make the effort to understand and validate better alternatives.