I think he also has a point when he says “It’s not just enough to process data and come up with a rating”. Can a multi-dimensional space of data really be mapped onto a single dimension (a rating) without losing a lot of information and insight?

To address a few of your concerns.

Other Plans: We already have the capability and encourage plan sponsors and advisors to build peer groups taking into account whether the other sponsors in their peer group offer a DB or other benefit plan.

Data Timeliness: Right now most of our data is accurate as of 12/31/2007. Most of our clients agree that in the absence of anything else, it is still valuable having 2 year old data until such time as fresher data becomes available. The alternative is no benchmarking at all, and that is not very palatable to most. We are in the process of updating to 2008 over the next month. Due to EFAST2 we will be able to add 2009 data before the end of the year. I think the issue of data timeliness will disappear literally overnight.

Again, I appreciate you taking the time to share your concerns. If at some point you want to come get a demo I would be happy to accomodate you.

Certainly, companies have done benchmarking, however, most benchmarking is focused on plan design:
(1) First, whether you are talking about the Hewitt Benefit Index or other consulting firm efforts, the actuaries are always careful to focus on the value of the design – not the cost,
(2) Second, to get at the value of the design, they use a uniform demographic group and run the formula/features through calculations – which may or may not produce a reasonable view of where you stand – depending on whether your firm’s demographics are typical,
(3) Third, they tend to only look at new hires because of all of the grandfathering from the massive level of changes over the past two decades (for example, one of my retirement plans has two totally different values for two halves of the population due to grandfathers), and
(4) Fourth, they also remember to aggregate analysis into “events” – a retirement event, a medical event, etc. – because they are well aware that the designs focus on all plans as a package, and, specifically, all retirement plans (income, medical, etc.) as a separate package/event.

A caveat about benchmarking is that whatever BrightScope thinks is “best practices” as demonstrated by what other firms do would not necessarily be “best practices” for your firm; even if it were attainable.

How the plan defines covered compensation, accumulated service for vesting, age of the participants, whether the plan uses automatic features, etc. – and how those features interact with the specific demographics of the population – all tend to have a significant impact. The same design can produce different results depending on those and other factors. The folks at Hewitt and elsewhere know this and expend more effort on confirming designs – versus focusing their attention on 5500 data.

Finally, the plan must also be considered within the context of the total rewards strategy, affordability, etc.

No need to be immediately reactive or defensive. Our “critics” are always our best source of new ideas and improvements. Let’s remember:
- The DC system is still very new and evolving – the very first generation to retire using it is only now starting to happen
- No one expected it to be so popular and such a massive phenomena – so we re always playing “catch-up.”
- It has become critical to the global financial system and the health of US society

Any new effort will need evolve and data collection and analysis is always a moving target. This all seems like a healthy and needed debate. We encourage it to continue.