logo

The Perils Of Data-Fitting
By: Jeffrey A. Miller   Tuesday, September 01, 2009 10:46 AM

Vote for next session
The next market session will close:

We are delighted to see the Felix Salmon piece (highlighted by Abnormal Returns) reviewing the supposed relationship between Japan in the 90's and the current US market.  Check out the article to see the "great fit."

Here at "A Dash" we have frequently exposed such  bogus relationships, even when the loud voices took the other side.  There is a simple process for generating a misleading relationship:

  1. Examine a bunch of charts.  Look for variables that are all correlated to the economy in some way.
  2. Find a couple of prominent features to the charts.  It is especially good if one of them (the leader) shows a move not yet reflected in the other.
  3. Adjust the scales to make the prominent features line up.  Ignore any other activity.
  4. Disseminate the predicted next move as widely as possible.

We exposed this process in the "Batman" chart allegedly showing a relationship between bank lending criteria and personal consumption expenditures.  This widely publicized chart was never defended by any of the original big-time proponents.  The collapse in PCE did not occur in the predicted time frame, as we showed here.  No one was keeping score.

It was an obvious case of data-fitting, although our analysis did not get the buzz that we see for Felix Salmon's piece.  Perhaps it will do better in the book version.

What You Do Not Understand

Let us assume that everyone now sees the peril of twisting old chart data to fit a pattern.  This is a lesson that can be applied to something you do not already know.

When someone reaches into history for an analogy -- -it is the same thing.  It is back-fitting of data, using the human mind, the most powerful computer.

When someone presents a trading system -- based upon many runs of old data, it is back-fitting.  The spectacular returns are not real.

Our Take

It is a shame that people are drawn to bad charts, bad analogies, and bad systems.

Too bad for them.  For the rest of us, it is a time of opportunity.


(0)
No Comments
Post Comment
Name:  
Alert for new comments:
Your email:
Your Website:
Title:
Comments:
   
 
 
 
 
   
 

The above story is the opinion of the author only and it does not reflect iStockAnalyst opinion. Further, the author is not personally advising you regarding the suitability of the story for your investment needs. In no event iStockAnalyst will be liable for any loss or damage including without limitation, indirect or consequential loss or damage, or any loss or damage whatsoever arising from or arising out of, or in connection with the use of this information. Please consult your investment advisor before making any investment decision.
  
Advertisement
Popular Articles
Related Press Releases
Advertisement
Partner Center
Recent Articles by Jeffrey A. Miller



Subscribe to Email Alerts rss feed or RSS feeds rss feed for articles from more than 500 contributors, press releases, SEC filings and full text news from more than four thousand sources.
Fundamental data is provided by Zacks Investment Research, market data is provided by AlphaTrade. , and Commentary and Press Releases provided by Quotemedia