Opinion: Consumer’s got it wrong ... again!

• Tweet

Could you use Consumers' review of managed funds to make an investment decison? Michael Chamberlain says No. He reckons the review is based on incomplete analysis of investment options and that it reaches flawed conclusions.

Monday, July 16th 2001, 5:55PM

The July 2001 issue of the Consumers' Institute magazine recommended where investors should invest their money. "Where" in this context meant which product (i.e. managed fund), and not which investment or necessarily what strategy, and "recommended" meant what "was best" and not what "will be best". "Best" also related more to past performance. Consumer's recommendations were based on an analysis of 448 funds, of which only 48 met its criteria as being potentially suitable or "good". Of these 48, 16 were identified as "meeting the special needs of consumers looking for long-term growth". Details of the final 48 funds, and the highlighted 16 funds, were included in Consumer's article.

In my view, Consumer's conclusions are flawed - they're based on an incomplete analysis of the investment options available. They incorporated "analysis" that was little more than unsubstantiated opinion and that does not pass any form of close scrutiny. This lack of thorough analysis and inconsistent conclusions, highlights the problems the financial services industry has in ensuring that unbiased quality information is available to the investing public. With such mis-information, from what is considered a reputable organisation, Consumer has undone the good it does elsewhere. Perhaps it should go back to dropping things on washing machines until they break.

What you need to be aware of in respect of the consumer analysis is that:

• Flawed processes lead to flawed conclusions.
• Volatility is volatility - unless the comparison is on a "like with like basis", comparative volatility analysis is a waste of time.
• Past performance is a poor indicator of future performance - in fact it is close to useless.
• Managed funds are not less risky - managed funds are managed funds.
• High fees and good products are negatively correlated. Most managed funds can't provide good returns because of high fees.
• Property is not "low risk".
• International shares do not a "medium risk portfolio" make.
• Property is not "low risk".
• If the best products are not considered, they won't make the recommended list.

And what Consumer doesn't tell you is that:

• NZ based, active overseas share funds don't produce the goods.
• The best products have low fees, are transparent and are tax efficient.

In fact all Consumer showed is that, if you select your data, pick your period and then misinterpret the results, you can claim absolutely anything.

Consumer's universe

Consumer claimed, "we've analysed the structure and performance of all managed funds operating in New Zealand". However, the funds Consumer considered were restricted to the 445 funds within FundSource's (formerly IPAC) database and 3 funds recommended by brokers, ABN Amro Craigs. FundSource was also contracted to undertake the basic analysis.

While a full list of the 445 funds in FundSource's database was not published, it is clear, from the information provided, that Consumer did not evaluate all available options. In fact, the analysis appears to be limited to those funds promoted by financial planners. One could be excused from concluding that the article set out to recommend only funds promoted or supported by financial planners, and not funds that might be best suited to the needs of the investing public. This view is further highlighted by emotional claims and throw away comments throughout the article like "managed funds [are] less risky". If the best products are not considered, they won't make the recommended list, or as one All Black turned rugby commentator once said, "If you don't have the ball, you can't score a try".

The 1999 vintage

This is the second time that Consumer has undertaken this type of analysis. The first time was in 1999. To give Consumer credit, while not reproducing the 1999 results, it did comment on its previous success. It's worth repeating its conclusions. "We have undertaken this analysis once before, in 1999 (Consumer 385, 386). Then 32 funds met the criteria. Of these, only eight have passed again (see table). Two were super schemes; the rest were unit trusts."

So, based on the criteria that Consumer believes separates the good from the bad (see box 1), 32 funds were good two years ago but only 8 of them are now. At this rate of success, only 2 of the 1999 recommendations will still make it in a further 2 years. I wonder what it feels like to have bought one of the 24, or even if you are still holding one of the lucky 8, which may soon be only 2! I also wonder how many of the current 48 identified (or 16 highlighted) funds will still be "good" in one year's time, let alone two years.

Consumer noted that the main reason for previously recommended funds failing this time, was poor investment performance. Excuse me, "poor investment performance"! Surely any criteria that leads to funds being recommended in 1999, but where 75% immediately (in the following two years) didn't meet the performance criteria must question the whole performance analysis undertaken, and the wider criteria.

What Consumer did not tell you is that if you randomly select 32 funds from the total funds available, the laws of probability say that one quarter of the 32 (i.e. 8) will have upper quartile (i.e. top 25%) performance. This is unsurprisingly the same result as Consumer found. Is Consumer's process no better than a random process?

40 (of the 48) funds that passed this time, failed last time. Was it because they were not considered? Was it because they had poor performance? Whatever the reason, it brings to question the type of analysis consumer has undertaken.

Finally, pity the investor who didn't chase that last bit of return and adopted a conservative approach in 1999. No "lower" risk fund recommended in 1999 made the final list in 2001!

Process

To find out why Consumer got it so wrong, you need look no further than the process followed and its criteria. Consumer said it applied rigidly a set of nine criteria (see box 1, back page) against the funds considered.

Consumer applied six initial criteria (access, longevity, size and entry, ongoing and exit fees) to all 448 funds. This reduced the 448 funds to 163. Consumer then applied a further 3 criteria which reduced the 163 to 48. Consumer finally highlighted 16 funds as being capable of meeting the special needs of consumers looking for long-term growth.

The funds highlighted, along with all 48 funds meeting the criteria, were also broken down into 3 categories of "like" funds; "higher risk", "medium risk" and "lower risk". "Like" in this context means having similar historical (last 3 years) volatility but not necessarily similar investment strategy or similar expected future returns.

Criteria

I question the total reliance on the longevity, size and access criteria. I don't think these should automatically eliminate candidates. In respect of size, a fund obviously needs to have critical mass. However, in many cases one fund invests in a larger fund and, though the first fund might have less than $10m, it could be profitable and well diversed from day one, because it invests in other funds. Applying a rigid screen of a minimum of $10m will therefore exclude a number of good funds, and will not exclude a number of poor funds. If the diversification level and the fund profitability are the issues, these should be the criteria.

Access and longevity criteria will also have the same affect i.e. eliminating "good" funds and not eliminating "bad" funds. If a longevity criterion is strictly applied, new good funds will initially be excluded for at least 3 years. This is why passive funds took so long to get on recommended lists even though they should have been there from the outset.

High fees and good products are negatively correlated

My main concerns with the initial six criteria relates to the three fee-based criteria. It is very difficult to see how a product that has an entry fee of 6%, or even using Consumer's words "aim for 3% or 4% at most", can be a good product. For this to be so, the others would have to be really bad. How much more than 6% can you charge?

There are two types of "entry" fees:

• brokerage - when an investor puts money into a fund, the manager has to buy assets and usually has to pay brokerage to do that. The reverse applies when the investor pulls money out. Provided the brokerage charged is the actual brokerage incurred this is a cost of buying assets, and not a fee for the manager. In fact brokerage has to be paid and will either be explicitly disclosed (best practice) or hidden but still paid.
• fees - on top of brokerage, the saver pays fees to the manager.

Any manager's entry fee, is a cost and is not supportable in a "good" product. Typically, entry fees of this nature cover commission for the salesman. These have nothing to do with the product, and by definition, are a negative feature. Such commissions can, in some cases, be justified, but this is up to the investor to determine, and honest disclosure is required. In my view, any product that cannot be bought by the general public without manager's entry fees must by definition be a poor product relative to one that can be. A key criterion, therefore should be that the product, if available to the general public, is readily available at a zero entry fee.

At the same time, I can't see any justification for a manager's exit fee. Genuine costs of brokerage or transaction costs are reasonable, as these are incurred when assets are sold. However, anything over and above this, is unsupportable and by definition must make the product less competitive and therefore a poorer product.

Consumer highlighted that it was important that a product had a high level of flexibility. One of the important features of flexible products is not only being able to access your investments, or change contributions or change your investment strategy, but also to withdraw funds without penalties or fees. That's one reason why managers' entry fees and exit fees must be a feature of a poor product.

Of similar concern is the criterion that ongoing fees must be a percentage of assets and therefore deducted as a percentage of the return - that is not "fair enough". The best products, by definition, must be those with the lower fees that also meet the normal quality criteria associated with sound investment management. If a particular product chooses to charge a low administration or dollar fee and a low management fee (which is lower in total for an investor is less than a typical straight percentage of assets), that product should be commended and recommended. At the end of the day, the best products must be able to produce the best returns, and "returns" in this context must be net of tax and net of all fees no matter how they are calculated.

The best products must have transparent fees. Many of the products Consumer recommends, and within the 448 funds considered, have quite high fees and in many cases do not fully disclose their fees and costs. Many deduct in-fund costs (such as audit, legal, printing etc.) that are not quantified and can often see a further 1% deducted from the return.

I am not sure why Consumer has adopted this approach (excluding funds that charge dollars, not percentages) and can only assume that it was someone's laziness. The required level of analysis would otherwise have been more complex and more technical. Also, such funds are not, as a rule, promoted by financial planners. Accordingly this information is often not captured by FundSource that made the recommendations to Consumer.

Flawed process leads to flawed conclusions

Only two index funds made it to the final 48 so Consumer concluded, "it seems that actively managed funds still produce the goods". In reality only two made the final 48 as not all funds were considered and the criteria was designed to exclude them (that's why none made it in 1999).

There is no doubt that actively managed, New Zealand-based global share funds might occasionally outperform passive global share funds but, over the long-term, there is now compelling evidence that passive funds will outperform actively managed funds in the New Zealand environment. This stems from lower fees and lower tax.

If Consumer understood its own analysis, it should have reached the same conclusion. Of the 10 funds in the final 48 "growth" funds, all New Zealand-based, active funds under-performed the index funds because such funds don't "produce the goods". In fact the only general global funds that outperformed the index funds were overseas investment trusts that typically enjoy the same tax treatment to a passive or individual investor. It is also clear, and is shown by Consumer, that selecting one or two active funds that might in a particular period outperform passive management will not over any reasonable period, in fact it is nearly impossible.

Also you have to ask why other index funds, for example the BNZ International Index fund, which did not make the final 48, yet are effectively identical to the products that did make it. Likewise, other index funds that have significantly lower fees than both Tower's and AMP's, did not make the recommended list and some were not even considered - perhaps they don't pay high enough commissions to financial planners.

In reality only two passive funds made the final 48 as the others were excluded because they have not existed for 3 years, or charged low dollar fees and not high percentages of asset fees. The fact that 2 did make the list, given the top 25% performance criteria, highlights that most active funds must have under-performed the index funds.

Tax reserving

Consumer doesn't analyse the internal tax position of each of the products and should have. Some products do not fully reserve for tax in the sense that if they were wound up today, their current tax reserve would not be sufficient to meet the tax liability. These products have therefore overstated their historical performance and present higher risks to investors than is the case with other products. Does Consumer think that investors would not want to know this?

Understanding the tax position is important but difficult. Again, Consumer has shied away from the complicated stuff. Many products reserve for tax at a 33% tax rate but only provide for tax at a lower rate. They assume that the 33% is not payable for some time and so they need to deduct only 30% or 15% today. A lower deduction today means a higher after-tax return today so boosting its investment performance today, but ultimately the chickens will come home to roost.

Consumer did not look at the adequacy of a product's tax reserve as one of its criteria. I think it should have.

Volatility is volatility

Throughout the article, Consumer interprets "risk" as volatility (the rate at which a product's returns go up and down). This is a poor definition of risk, as it is something most investors do not understand, focus on, or should focus on. It also leads to inconsistent conclusions. Unfortunately financial planners and research houses like it because they can kick it and measure it and it sounds good.

If volatility is to be used, it should be a relative measure in the sense that an international share fund's performance should have its returns analysed on a return and volatility basis relative to an international share index or other international share funds. If this had been done, Consumer's recommendations would have been different.

Instead, Consumer used volatility to allocate funds into three categories; higher risk, lower risk and medium risk. This was just really dumb! Consumer really doesn't understand what it was doing here. It is very difficult to see how products that are international share or share-only funds, or bond-only funds, can fall into the category of being "medium risk". The fact that they might have had moderate volatility over the last three years is purely a coincidence. This highlights the flaw in Consumer's emphasis on historical performance or volatility in this type of exercise, particularly over such a short period.

Property is not low risk

To further highlight the futility of Consumer's analysis on volatility, you need go no further than its classification of property funds. It is hard to see why anyone could classify a property fund as "low risk" though Consumer noted that normally such funds have higher risk. Consumer's analysis put them in the lower risk category because such funds, in recent times, have performed poorly (i.e. had low returns). And, because they have consistently had low returns (from one year to the next), they have had low volatility. Consistently poor performance means "poor product", it does not mean "lower risk". The fact that the three property funds that made it to the final 48 were compared with, and all returned less than, cash over the measured period, highlights the stupidity of this as a risk measure. Again, Consumer really doesn't understand what it's doing here.

 

Two swallows don't make a summer

Consumer also looked at a couple of common investment "concepts". It concluded that high returns mean higher risk, and that managed funds are less risky. While I agree that investments, such as shares, are likely to perform better over the long run than investments in fixed interest and are more "risky", Consumer's analysis in this instance was superficial and very much time-dependent. It ignored fees and shows that there can be quite long periods of time, even excluding times such as the sharemarket crash, when bonds outperform shares. Also, why did Consumer do its analaysis using NZ Shares when everywhere else it recommended overseas shares? Consumer's graph is misleading and its conclusions flawed. What its graph also showed was:

• between 1991 and 1998, shares and bonds performed about the same in total but had different year-by-year results.
• since 1994 bonds have out-performed shares.

On Consumer's basis of analysis it could have concluded high returns mean higher risk, which is not as good as high returns and lower risk. It chose not to.

However, of greater concern is the superficial nature of the analysis that shows that managed funds are less risky. Consumer did this by taking the average of all managed funds invested in the New Zealand share market and comparing it to Telecom's share price since November 1994. If you take a long enough period, it is highly unlikely that a single share will always outperform the wider sharemarket or a larger pool. This is not to say a few won't. However, you just need to wait for the required period as sooner or later, other shares will catch up or the single share will go down.

However, Consumer made its comparison based on the average of all managed funds in the New Zealand market place, and thinks this is relevant. By definition this must be a lower risk portfolio because of the higher level of diversification. Not only are there diversification by shares, but also diversification by managers. Interestingly, had Consumer undertaken the same analysis at the time they previously did an "in depth" analysis (i.e. 1999), it would have reached the opposite conclusion. All the Consumer analysis shows is that the average of everything is average and a single security will at times be above average and, at other times, below average.

What is really important

When it comes to selecting an investment product, here's what is important:

 

Strategy

The investment strategy and objectives of the product/investment must be consistent with the investment needs and objectives of the investor. Any analysis must therefore compare like with like and summarise investments by category i.e. overseas share products with other overseas share products and not overseas share products with overseas bond products. This will let investors identify the better funds of the type that the investors think are appropriate for their needs.

While not a requirement, the best products will let investors choose or create their own strategy, and change strategies with no or low costs. Alternatively, they will be products that can be conveniently combined with other products to produce the required overall strategy.

Returns

If Consumer really wants to do its job properly, it needs to analyse products on their ability to deliver after-tax, after-fee performance that is competitive in the future and not in the past. A good product will have the capacity to produce good returns relative to the investor's objectives. "Returns" in this context are net of tax and net of fees. Did Consumer measure "after all fees' " returns? Any analysis must therefore focus on identifying which products are invested efficiently, are tax effective, do not have fees that act as a drain on performance, and are products which deliver what they promise. Selecting a fund with a high gross performance is of no value if most of that return is eaten up in tax and fees. Likewise full provision for tax must be made and can't be assumed.

Other important criteria relate to security, convenience, communication, and flexibility. If a product does not also meet these criteria, it will not be a robust solution. However the main criteria have to be:

• the ability of the product to produce future performance consistent with the stated strategy so that there are no surprises, and
• the product's ability to produce a performance that is consistently high relative to other comparable funds on an after-tax and after-fees basis.

Consumer got it just plain wrong.

The last word

New Zealand is crying out for high quality, impartial information on investment products. This is Consumer's second attempt. Its 2001 exam mark, based on its 1999 attempt was only 25%, a failure by School C standards. If Consumer doesn't learn from its 2001 experience it's possible that it won't be even that high if the exercise is repeated again in two years. If Consumer wants to provide a quality service and to become the independent commentator on investment issues, it has to change what it does and how it doesn't.

Regardless of what I might think of Consumer's technical weaknesses, the proof is in the pudding. With 8 of 32 funds recommended in 1999 surviving in 2001, it gives me no confidence that any of the 48 funds (that met the nine criteria) will perform the best in the immediate future, never mind in the long-term which is really my concern. Consumer (or its adviser) needs to go back to school. It has produced a second dose of simplistic, flawed analysis that won't help its readers.

Box 1:

Selection criteria. The first six criteria applied by Consumer to the 448 considered funds were: Access. The fund had to be available to the general public. Longevity. The fund had to have been in business for at least 3 years. The sole purpose of this was to let Consumer judge its investment performance. However, there is now sufficient academic evidence to suggest that no conclusions about future performance can be reached based on 3 years of past performance. This is a serious deficiency. Size. The fund had to have more then $10m assets. Entry fees. The funds had to have a maximum entry fee of 6% with the intention that the aim should be 3% or 4% at most. Ongoing fees. Management fees had to be expressed as a percentage of assets as apparently "that [is] fair enough" and were not allowed to be dollar based. Products with low dollar based fees were excluded for no logical reason. Exit fees. Exit fees had to be below 2%. Having passed the above six criteria, (which eliminated 285 of the 448 funds), the remaining 163 funds were subjected to a further three criteria. Continuity of management. Where funds were actively managed, the investment management team had to have been stable for at least 2 years. Investment performance. The investment performance of the fund had to place it in the top 25%, presumably of similar funds, over the last 3 years. Consumer noted "past performance is not necessarily an indicator of future performance, but it is a useful guide". Subsequent analysis in the article actually demonstrated that good past performance is no guide at all to good future performance but more on this later. Initial investments. Funds had to allow a minimum investment of no more than $2,000.

Michael Chamberlain is an investment adviser and a principal of consulting actuary firm MCA NZ Limited

« Tax review puts funds on equal footings

King builds an empire »

Commenting is closed

 

Printable version

Email to a friend