So I started looking at the Shiller file, and I saw stuff in there like this:
=B21*7/12+B33*5/12
What the... ??
Turns out, Shiller is taking seven twelfths of a previous actual number and adding it to five twelfths of a subsequent actual number, to calculate a monthly value. It is all like that, with a number for January of each year and calculated trend-values for the eleven intervening months, from 1871 all the way up to 1953. After that there are numbers for every month, rather than calculations.
Shiller is figuring points on straight lines that run from January to January. That's why the EconomPic graph appears relatively smooth (up to 1953), then jiggy:
 |
| Graph #1: 110 Years of Interest Rates (EconomPic) |
In the previous post I wrote
Gotta go with annual, because otherwise I'm just making things up.
Apparently, sometimes it's okay to just make things up.
Okay. But instead of taking averages of Shiller's calculated values, I'm just gonna go with the actual values that are given for January of each year. That's definitely my plan for the numbers up to 1953. After that I could take averages of yearly values. But it would be easier just to go with the January numbers all the way through. We'll see.
Hi Art, you wrote:
ReplyDeleteApparently, sometimes it's okay to just make things up.
___________________________
Shiller is not making things up. He is just using a 12 month moving average filter as a resampling filter. This is an excepted method for expnding yearly data into monthly data.
His method is equivalent to first starting with the assumption that every month of the year all have a single value and then applying a 12 month moving average to the data. That means each month is some linear combination of 2 years values and you get a formula for each month of the year that looks like:
month_t = [A*(12-t)/12 + B*t/12]
Where A is the current years value and B is the following years value.
That is equivalent to populating a monthly data base with 12 month blocks of data and then smoothing that data with a 12 month moving average filter.
Hope that made sense.
Also, I should point out that your data is around 1300 months long but your graph is only 800 pixels wide (for the large version) and 400 pixels wide (for the small version). That means the software you are using for image processing is also applying some kind of similar resampling filter to the data to get the images as we see them.
-jim
Nice explanation, Jim.
ReplyDeleteThanx!
JzB
I agree: Great explanation. The word that came to mind for me is "interpolating" but that's a schoolword from a long time ago. Definitely interesting, fitting 1300 months of numbers into 400 pixels.
ReplyDeleteAnd I should have pointed out that Shiller's file includes several categories of data. I don't know what purpose he had in mind. But I can imagine that he needed monthly interest rate numbers to go with his other numbers.
Still, his method does explain the not-jiggy and jiggy on the interest rate graph.
Hi Art,
ReplyDeleteThe term "interpolating" is correct. But that would be a generic description. This not the only way to interpolate.
If you took the post-WW2 data and just took, let's say, the July value for each year and then applied the same method that would get rid of much of the wiggles in that part of the data.
But it also looks to me that the interest rates became more volatile after about the mid 60's.
Art
ReplyDeleteThis is a discussion you need to get into
http://worthwhile.typepad.com/worthwhile_canadian_initi/2011/11/why-has-private-debt-increased.html
Sorry for the relatively off topic comment
Thanks Greg. The Worthless Canadian site rejected my comments...
ReplyDeleteJim, Shiller's numbers look much less volatile if you look only at the Januaries.