To view this article in digital magazine format, click here.
“Suppose you go into 32 bars. In 16 of them, you say the Celtics are going to win tonight, and in 16 you say the Celtics are going to lose. Return to the 16 you were correct at, and repeat the process for the next five games. Then, go into the bar where you’ve been right six times, and say, ‘How much are you going to pay me to tell you who’s going to win tonight?’”
Kenneth French—the Dartmouth professor who constitutes one half of the famed duo behind the Fama-French three-factor model of stock returns, which questioned the underpinnings of the Capital Asset Pricing Model—offers his students that hypothetical as Skill v. Luck 101. And he doesn’t stop there.
“For the average dollar, I found moving from active to passive management would increase returns by 67 basis points per year,” French says, speaking of a landmark 2008 study he conducted on US stock market data from 1980 to 2006. For individuals, that figure would be somewhat higher, whereas the lower investment fees offered to institutions would reduce it slightly. “This is all about cost. If I choose to hold a passive market portfolio, my return is the return on the market minus my costs, and everybody else’s is theirs. So if my costs are lower, I know I’m going to win. And not just over the long term: I’m going to win in literally every instance.”
It’s possible some managers do earn their fees through skill, French says. But guaranteed there are superstar managers—and CIOs—who owe their performance entirely to luck. “If I’m that manager, that track record is all I know: I’ve got this alpha of 6% per year. I’m walking around with my chest all puffed out, hitting on women in bars.”
So how can asset owners pick out the true Celtics savant from the lucky guesser—one whose next game prediction is just a well-paid shot in the dark? French doesn’t know. “Either before or after the fact, it’s an extremely difficult statistical problem. To think that one could learn anything meaningful by looking at three years of securities returns… Well, they’d have to be very peculiar, Madoff-style returns. In reality, the signal-to-noise ratio is about zero.”
Even if skillful managers do exist, French argues that without the tools to identify them, there is no apparent justification to pay for anything but passive in public markets. “If you insist on hiring an active manager,” he says, “at least get someone cheap.”
Skill Strategies on Trial
“Do you swear to tell the truth, the whole truth, and nothing but the truth, so help you God?”
“Yes, sir.”
Thus began the testimony of South Carolina Treasurer Curtis Loftis at the Senate Finance Committee Special Subcommittee hearing on January 16, 2014.
This is not a court case—officially. Such hearings are not typically held under oath. However, complying with Loftis’ special request, four senators and the treasurer vowed to tell the truth, transforming the Gressette Building in downtown Columbia, South Carolina, into a courtroom. If the treasurer was lead prosecutor, the state pension commission represented the defense as its aggressively skilled-oriented investment strategy went on trial.
“This is serious business,” Loftis said in his opening statement. “The numbers are large; there are an incredible number of people involved; and we want to know everything.”
The senate finance committee has met numerous times since September of 2011, eventually forming a special unit to review the investments of South Carolina’s Retirement System Investment Commission (RSIC). It has called five hearings so far, comprising various testimonies from Loftis, Inspector General Patrick Maley, RSIC members, and consulting firm Hewitt EnnisKnupp (HEK). And it’s still going.
South Carolina’s public pension came under fire most recently from the media-friendly treasurer who has launched an initiative to address the pension’s “fees issues.” His statement is simple: “We pay too much, and we earn too little.” The $27 billion fund paid out $418.2 million in fees in the fiscal year 2013, according to HEK’s report.
“The fee issue is a manifestation of errors in execution,” Loftis said. “From 2006 to the present, we have been in a rush to diversify our portfolio, pushing away many prudent investing standards we should hold dear: comprehensive due diligence, reliable reporting, governance policies on everything from conflicts of interests, third-party marketing, cash management, investment sourcing, and internal auditing. We’ve delayed and discarded them.”
RSIC’s race towards the endowment model—one heavy on alternatives—has been swift. The fund started 2006 with a 60/40 portfolio. Now, it boasts a 43% allocation to alternatives and has more money invested with hedge funds than directly in equity markets. Investment expenses have ballooned along with the alternatives bucket, and that’s no surprise to architects of the strategy. “Our fees are consistent with and appropriate for our exposures to alternatives,” RSIC’s Chairman Reynolds Williams said in an interview. “We work hard to negotiate all fees in the manager selecting process, but fees are fees. It’s going to be expensive.”
The forces of luck exerted themselves in the turn to this skill-based strategy, and not to the fund’s advantage. The portfolio’s final tilt towards expensive, private-market investments coincided neatly with some of the best years in decades for passive equity investors. South Carolina’s pension underperformed US public funds as a whole by 250 basis points in the 2013 fiscal year, according to Wilshire Associates data, returning 9.9% after fees. The differential is reversed over five years, with RSIC gaining 196 basis points above the sector’s median. For the last decade, however, the fund fell short with 5.29% returns to its peers’ 6.92%. A passive 60/40 would have outperformed both—but the alpha-oriented universe of large endowments and foundations topped all three.
To anyone who claims manager skill has little impact on performance—like Kenneth French—the default counterexample is David Swensen’s Yale endowment fund. According to the Ivy League university’s investment office, the fund had $20.8 billion in assets as of June 30, 2013, and a market-leading annual return of 13.5% produced $18.4 billion in added relative value over the last 20 years.
And yet, the question remains: Was Yale’s success the result of Swensen’s immaculate selection of managers, the right exposure to risk, or a lucky combination of both? Swensen asserts in his book Unconventional Success: A Fundamental Approach to Personal Investment that alternatives are not for everyone. He, the father of the endowment model, avoids top-performing strategies and the so-called “top” managers. Ordinary Joes should stick to index funds if they’re unable to sift through the jungle of managers and negotiate on fees, access, and liquidity, he says.
Research better supports the existence of attainable, skill-based alpha in alternative asset classes—or at least doesn’t disprove it as conclusively as with equities. According to French, the jury is still out among academics. “We don’t have the data to really dig in and see if the world works here the way it seems to work in public spaces,” he says. “There is also the argument that private markets are not as efficient, which could make skill a larger factor.” For all asset classes, consultants, researchers, and asset owners alike consider market efficiency a leading factor in the value proposition for active management. The “barbell” distribution of assets into ultra-cheap passive vehicles and alpha-oriented private funds—but skipping traditional active equity and fixed income—has become increasingly popular.
Consultant Joe Nankof, a partner at Rocaton Investment Advisors, sees this concern about paying alpha fees for beta performance at work with hedge funds as well. “Since the financial crisis, clients are more wary of beta-driven strategies,” he says. “For example, hedge fund options selling strategies that earn a fairly consistent 10% or 12% per year—which evaporates in market downturns—don’t have the appeal they once did.” Asset owners may be learning from past follies, but there is little evidence to suggest that selection processes for alternatives managers are broadly better at identifying skill than those with traditional assets.
Of course, many asset owners have their own special technique. Larry Schloss, former CIO of New York City’s retirement systems, had dinner one-on-one with every potential manager before writing a check (or not, as the case may be). Josh Kaplan, head of hedge funds for the $26 billion health care investor Ascension Investment Management, follows his intuition. “You shouldn’t be allocating your firm’s money with the minimal effort of checking the boxes,” Kaplan explains. “It’s my responsibility and duty to go above and beyond. And experience is crucial as a lot of it is instinctual. It comes from the gut.”
For the $12 billion it has signed over to alternatives managers, the South Carolina retirement system has followed a traditional path. “Our manager selection process is pretty standard,” says Chairman Williams. “We first notify our consultant on what kinds of managers we’re looking for in a particular asset class and receive a list of six to a dozen potential names. A team of three staffers and one commissioner is assigned to vet the list and narrow it down to those we’d like to invite down to Columbia.” This is where the first level of due diligence is conducted, he adds. “After the first interview, we further narrow down to a few managers we see at their offices. Sometimes we may be able to squeeze in three a day. That would be an insanely busy day, but we do it on occasion.”
Williams asserts that “no stone is left unturned” in the due diligence process. It’s important to not only look at a manager’s historical returns, but also how they performed relative to market environments, the key personnel in the organization, the quality of their reporting, and even the stability of their compensation, he says. “The most important thing to consider in the due diligence process is our assessment of managers’ wisdom in their strategies.”
The phrase “due diligence” says a lot about the standard public fund allocation process. The tendency to rely on consultants’ recommendations, past performance, established industry players, and, at one point, funds-of-hedge-funds, suggests that the primary aim of these institutions is weeding out the Madoffs, not identifying the skillful. To this end, public funds have largely been successful. But if the unstated goal is defensible beta with minimized headline risk, Vanguard may be as robust a choice as Brevan Howard.
Research suggests that any institution—be it a corporate pension, university endowment, or public retirement system—working from a consultant’s shortlist has already cut their chances of selecting an alpha-generator. A 2013 study of active equity managers found that consultant recommendations correlated with substantially higher fund inflows but slightly worse performance. The authors, all researchers out of Oxford University’s Saïd Business School, make a familiar qualification: “Our analysis focuses on one asset class, US active equity, which may be more efficient than other asset classes, and it is possible that elsewhere the recommendations of investment consultants are more prescient.”
Moneyball for Hedge Funds
Stan Altshuller intends to be the Billy Beane of hedge funds.
As general manager of the impoverished Oakland A’s baseball team in the late 1990s, Beane and his quant discovered major inefficiencies in the player market using a statistical analysis system called sabermetrics—and exploited them. He built a record-breaking team out of underpriced skill, the outperformers that standard statistical analysis missed. Altshuller, co-founder of New York-based analytics firm Novus, believes the same can be done for manager selection. He also believes that French, and others, might actually be wrong on a few points.
One: It’s too early to tell if the hedge fund industry can deliver persistent, skill-based alpha. “It’s fewer than a quarter of managers,” Altshuller argues, “but the numbers show that some are winning the same way again and again. And the opposite is true, too: Trading data shows how much alpha is left on the table by managers who might, for example, tend to sell their winners too early or trade too much.”
Two: There is no reliable way to identify which managers will outperform before they’ve already done it for years. “If you think you need a six-year track record to indicate a manager is skilled rather than lucky, you are looking at the wrong data. With daily trading data, you can start to make a statistically sound conclusion in two years.”
Three: Investment consultants may be more prescient in recommending managers for asset classes that are less efficient than US equities. “I haven’t seen a single consulting firm that recommends managers based on anything resembling robust data. They think that having years of experience means they can sit down across the table from a hedge fund manager and tell if he’s a moneymaker just by looking him in the eyes. I don’t think their gut feeling is any better than the next guy’s.”
Altshuller presents every inch the Silicon Valley success story: a data wizard in jeans and hiking boots, and a job title of “chief research officer” with an actual team to be chief of. Typical of his generation, he is also impatient with the industry status quo.
One pet project epitomizes this ethos. Altshuller is developing a tool that overlays the portfolios of Tiger Management-affiliated hedge funds with one another, based on public data about holdings and performance. It spits out statistics on security, popularity, uniqueness, and liquidity—among myriad other measures—for every possible fund grouping and pairing. Eventually, Altshuller says, the tool will encompass all 3,000 hedge funds (and counting) in the Novus system. The leap from fund prospectus data—isolated, static, packaged by the manager—to the data offered by a system like Altshuller’s could be greater than the gap between traditional baseball metrics and Billy Beane’s sabermetrics. Of course, all of those bells and whistles may not help investors to parse skill from luck, but statistics are one of the few things that ever have.
Playing hedge fund moneyball can suit institutions with tight finances. Good data is cheaper than bad management, after all. But for funds with scarce human resources, gathering nuanced information won’t do much good if there’s no one to look at it.
Opting out of the skill chase in favor of simplicity has proven successful for some of the rare asset owners who try it. Oklahoma’s $8.3 billion public pension system has done just that, with 10-year returns that put it in the 27th percentile of public plans. The plan spent $8.6 million on the management of its strictly stocks and bonds portfolio in the 2013 fiscal year, and earned a net 12%. South Carolina, in contrast, rang up nearly 50 times the investment expenses ($418 million) for a portfolio just over triple the size, with 9.9% returns.
“We try to control the things that we can control,” says CIO Brad Tillberg, and that means fees. Tillberg is well aware that the 69% passive and indexed fund makes it “an exception” among its institutional peers. The plan has been roughly 60/40 and largely passive since that was the standard institutional portfolio. The industry has evolved away, now favoring actively managed public assets and a side of alternatives, but Oklahoma’s board and CIO made the conscious decision not to follow suit.
“It’s a function of the organization’s investment philosophy,” Tillberg says. Still, he acknowledges, “there is a competitive pressure to be innovative in every aspect of asset management. But I think it’s important to maintain skepticism. I’m perfectly comfortable not being held out as a key innovator.”
Oklahoma’s retirement system once ventured into real estate, but has not gone back. “I don’t think the experience in real estate was very good,” Tillberg says. “It was perhaps not as well timed as it could have been.” There are no immediate plans to try out any more private, skill-focused investments, according to the CIO. “We get everything we need from the asset classes that we have. It’s not a priority.” Within the portfolio’s two buckets, Tillberg allocates active management carefully, focusing it on markets he sees as less efficient, such as fixed income and international small-cap equities.
But why go active at all? By all accounts, the Oklahoma retirement system has a sophisticated and engaged board, one bold enough to stay the 60/40 course as its peers play with hedge funds and risk parity. Tillberg knows what the research says—he’s a technical guy—and yet really does believe he can be the exception and spot skillful managers. “I would justify our active allocation with our manager selection,” he says, noting that over 10 years the fund has added value over its policy benchmark, net of fees. “It’s an attempt to garner alpha.”
So even Tillberg, who presides over one of the most index-heavy large portfolios around, isn’t immune to the pull of alpha. But with fees of 11.8 basis points last year, Oklahoma’s plan is at least following Kenneth French’s advice: “If you insist on hiring an active manager, at least get someone cheap.”
The Wrong Question?
Oklahoma’s mistake, according to Rick Di Mascio, CEO of UK data firm Inalytics, could be holding its managers too long. Most, Tillberg says, have been with the fund for years and years. For managers and CIOs, Di Mascio says, knowing when to sell is a rare—and crucial—marker of skill.
And Di Mascio believes skill exists. Based on his firm’s analysis of 800 portfolios around the world, he says it is possible for asset owners to identify talent. The debate, Di Mascio says, should move to “why this question of skill versus luck even exists in the investment industry. It comes down to the fact that the current process of focusing on people, philosophy, and process is fine because you need to know this stuff,” he says. “But the question is, ‘How do these translate into performance? What’s the link?’”
Empirically, the query has been answered in the simplest manner: “Skillful fund managers have the ability to find winners and know when to sell.” However, the answer that seems so self-evident has boggled many talented minds in the industry—and many CIOs and investment teams are still unable to spot the “skillful” among a jungle of managers. Yet asset owners also cannot be faulted entirely for this inability to discern ability from luck, according to Di Mascio.
“The industry camouflages the obvious,” he says. “We’re often swamped by Greek letters, information coefficients, Sharpe ratios, and complicated equations, which make the job of identifying skillful managers more difficult than it needs to be.” Di Mascio suggests looking at a manager’s buying decisions to pinpoint talent. Buying choices show a manager’s ability to discover winners, representing his or her “best ideas in a moment in time.” Studying a manager’s holdings can only add to the noise, as it takes into account decisions that may have been made in the distant past.
The second criteria of a talented manager may be more difficult to understand, Di Mascio adds. “The industry is actually pretty good at finding winners,” he says. ”The problem is knowing when to sell. Virtually everyone sells their winners way too early and hangs onto their losers too long.” Knowing when to sell is an area that generally needs subtlety, expertise, and wisdom.
“Due diligence is more than giving the asset owners the answers about managers—good or bad, to hire or to fire, skillful or just lucky,” Di Mascio says. “It’s more about listening to the managers’ answers to really understand how they work, what makes them tick, and how their processes actually function in reality.”
Evidence is all that matters, says the data firm CEO. If a manager’s history reveals he or she consistently sells winners, an asset owner should ask why. It could be that the manager believed selling would fit the price target or had a tracking error constraint—neither of which are wrong answers, Di Mascio says.
Another crucial mistake common in manager selection is following the herd and defaulting to top-quartile managers, says Peter Corippo, senior strategist at Russell Investments. “Looking back in time is not a real reliable way to trying to identify who is going to be the best manager going forward.” Top-quartile managers are selected based on their past performance, and while it may represent some level of skill and success, he says it could misguide CIOs to select those who may underperform in the short term. It’s important for a fiduciary to know what to expect in advance.
The Solution?
According to Martijn Cremers, a finance professor at the University of Notre Dame, a measurement of active share—the portion of a portfolio that’s unique from the benchmark—can help further differentiate competence from chance. “Active share ignores returns completely,” Cremers says. “It only looks at holdings and can serve as a complementary measure to tracking error volatility.” The concept derives from the basic premise that a fund can only outperform a benchmark if it’s different, and quantifies the proportion of the portfolio skill that is actually applied and contributes to said outperformance.
Historically, high active share managers have outperformed benchmarks more consistently than low active share managers, Cremers says. With guts, strong conviction, and opportunities to translate these positions into performance, high active share managers can garner a strong track record. And a manager who is able to overcome the risk of short-term underperformance that accompanies high active share counts as truly talented, he says. “A manager who runs a high active share fund but is unskilled cannot survive the industry for very long,” he says. “The competition of today’s investment market will naturally weed out the lucky in the group.”
But mathematical equations that calculate the efficacy of active managers may be a futile effort without a robust governance and allocation system. What good is manager skill when the execution process simply isn’t there?
Despite South Carolina’s efforts to construct a high-power alpha-generating portfolio, the fund suffers from serious governance problems. Internal disputes among the six commission members have diseased important investment decisions, casting the fund’s questionable choices as a mere catfight. Southern gentility is quickly thrown out the door as hostile words jet from one fiduciary to another. Whispers of corruption permeate discussions of the Palmetto State’s allocation process.
These distractions only dig RSIC’s fiduciaries into a deeper hole. Hearings and infighting make sufficient due diligence and proper searches for manager skill fall behind the highly evolved, but risk-taking, portfolio. RSIC’s official investment policy underscores the ordinary and vague manager selection process: “The commission will only invest in alternative assets when there is sufficient transparency and policy compliance reporting. Accordingly, the commission expects that extensive due diligence will be performed in evaluating and fully understanding all aspects of an alternative investment opportunity.”
This nebulous statement and the way it has been implemented at RSIC are enough to make Treasurer Loftis nervous. “The staff should not make any recommendations to the commission without an explanation of why they believe a manager can beat passive results, and the commission shouldn’t approve any investments unless it believes the manager can surpass passive performance,” Loftis says.
And yet South Carolina is just one small state, in one large country, in a world of institutions trying to be skillful at what most experts—like Kenneth French—think is a lucky man’s game.
In Columbia, South Carolina, however, few at the retirement system seem to be losing much sleep. “We don’t really consider how lucky a manager is,” the fund’s chairman says. “If a lucky manager is consistently outperforming benchmarks, wouldn’t that luck transform to skill at a certain point?”