How to Change the Past

I've fiddled with Hodrick-Prescott trend lines a bit, and one thing I noticed is that they have a sorry kind of foresight. They can predict the future -- but only in hindsight.

I did a post a while ago -- a post named Hindsight, actually -- in which I showed this FRED graph:

Graph #1: Natural Log of the Inflation-Adjusted Change in "Total Credit Market Debt Owed"

Then I showed the same graph in Excel, with a Hodrick Prescott trendline added in:

Graph #2: Same as Graph #1, with Hodrick-Prescott Trend thru 4th Quarter 2007

I stopped the trend line at the high point of the blue line, and wrote:

I stopped the HP calc at that point so that the later decline did not drag down the earlier years' numbers.

So that the later decline did not drag down the earlier years' numbers.

The HP calc is entered in Excel as an "array formula". It takes as input a whole series of FRED data. For Graph #2, I fed it data from 4th Quarter 1952 to 4th Quarter 2007.

For Graph #3, I fed it data from 4th Quarter 1952 to 4th Quarter 2012. That's the only difference.

Graph #3: The Blue Line is the Same as on Graph #2, but the Red Line Continues to 4th Quarter 2012

Notice that the red line no longer reaches up to the high blue peak. The red line bends down early now, around 2004, and begins the plunge while the blue line is still trending up to its high peak. The trend line anticipates the future.

Adding data for the years after 2007 to the Hodrick-Prescott calculation changed the results we got for a few years before 2007. If that's not a sorry kind of foresight, I don't know what is!


Another example, this from A second look at the heteconomist post:

Graph #4: Percent Change in the Non-Federal Portion of TCMDO (blue) and the H-P trend (red)

The red line on Graph #4 is a Hodrick-Prescott trend of the debt data through 2007. Graph #5 shows the same debt data, and the H-P trend again, but this time the trend is based on all the data through 2012:

Graph #5: Same as Graph #4, except the H-P Trend is figured through 2012

On Graph #4 the red line trends upward for the last few years before the 2007 end. On Graph #5 the red line runs flat until about 2004, when it starts to fall.

The decline since 2004 on this graph is the result of figuring in data for 2008-2012. Look at it from the start of 2008, and it appears that the future changes the past.


The Excel files, on Google Drive:

NonFederal Debt Growth.xls for the "Change in Non-Federal Debt" graph.
dTCMDO rel GDPDEF.xls for the "Log of Change in TCMDO" graph.

Both files contain Kurt Annen's VBA code for the Hodrick Prescott calculation.

Creating malaise

Graph #1: Real GDP Growth Rate, and Trend Lines

Look at the range of dates often called the Great Inflation -- from the mid-1960s to the early 1980s. The red trend-line is low then, a flatland between 5% hills and 4% hills.

Growth in those years is often said to have been disappointing:

  • Chris Kissell:

In the 1970s, the American economy faced the dual problems of high unemployment and high inflation. The origins of this problem began in the late 1960s, and the economy was adversely impacted through the early 1980s. Overall, the American economy experienced malaise for almost 15 years.

  • U.S. History.org:

Nothing fuels a strong case of malaise like a sputtering economy.

  • Wikipedia:

The 1970s were perhaps the worst decade of most industrialized countries' economic performance since the Great Depression. Although there was no severe economic depression as witnessed in the 1930s, economic growth rates were considerably lower than previous decades.

"Economic growth rates were considerably lower" in the 1970s. You can see it on the graph, between the mid-1960s peak and the mid-1980s peak. But what else do you see?

I see the Federal Reserve fighting inflation by slowing growth, creating troughs in 1967 and 1970 and 1975 and 1980 and 1982. Those troughs did a remarkable job of creating "malaise" and making growth look bad.

Seeing for myself

I looked before at the Econbrowser guest post The Myth of 'Jobless Recoveries' by Laurence Ball (Johns Hopkins University), Daniel Leigh (IMF) and Prakash Loungani (IMF).

Since then I found Kurt Annen's Visual BASIC code for calculating Hodrick Prescott values. So now I can try again to duplicate the graphs from the Econbrowser post, and maybe learn more about them.

From that post:

Using annual U.S. data from 1948 to 2011, we find that the Okun’s Law has a coefficient of –0.4 or –0.5, with an R² in the neighborhood of 0.8. Chart 1 illustrates the fit of the estimated Okun’s Law by plotting the unemployment gap (the gap between unemployment and the natural rate) against the output gap (output relative to potential). The relationship is very tight. No year is a major outlier in the graphs.

Chart 1. United States: Okun’s Law, 1948-2011 (Annual data) (Natural Rates Based on Hodrick-Prescott (HP) Filter with λ = 100)

NOTE: In the Econbrowser article the text reads "...a coefficient of –0.4 or –0.5, with an in the neighborhood of 0.8." Something is obviously missing between the words "an" and "in". I tracked down Laurence Ball at IDEAS, clicked the IMF link to paper, opened the PDF Okun's Law: Fit at 50?, and searched it for the phrase "coefficient of –0.4". Yup, what's missing is "R-squared", given in the PDF as capital R with an overbar, followed by a superscript 2. So I inserted that into the Econbrowser excerpt.


I went to FRED for "annual" data on "unemployment" and "output". Retrieved the seasonally adjusted Civilian Unemployment Rate UNRATE and seasonally adjusted Real Gross Domestic Product GDPC1. For both I selected "annual" data and the default aggregation method, "average". Used Excel to calculate the Hodrick Prescott values and figure the gaps -- the differences between the FRED values and the HP values. Created a scatterplot showing gap versus gap:

Graph #2

Well I'll be darned. The chart's a good match. I got my axes the wrong way around from Chart 1, but the data points are definitely grouped along the trend line, and I even got an R² comparable to the 0.8 reported in the Econbrowser post. And no major outliers.

Huh.

Okay. What I'm looking at is the difference of output values from their trend, versus the difference of unemployment values from their trend. I have to think about it a while.


Two points.

1. On my graph, the unemployment gap values range roughly from 2 to -2, or a difference of four percentage points. And the output gap values range roughly from 4 to -4, a difference of 8 percentage points. Roughly, the change in output is twice the change in unemployment. This agrees with Okun's law.

2. The output gap for this exercise is assumed to be the difference between inflation-adjusted output, and the trend of that output. In other applications, the output gap is taken to be the difference between inflation-adjusted output and potential output, which is not the same as the trend of realized output.

According to CBO's A Summary of Alternative Methods for Estimating Potential GDP (PDF),

Statistical filters (such as centered moving averages, bandpass filters, the Hodrick-Prescott filter, and the Kalman filter) are often used to extract the trend from GDP directly. These methods do not generally use Okun’s law...

Having actually used the HP filter now two or three times, having seen it for myself, I understand this better and I see it applies to the work of Laurence Ball et al. Ball's work is *not* circular because he uses an alternative to CBO's calculation, one that does not use Okun's law.

And yet, as the CBO paper states:

There are many ways to estimate the trend in GDP (and other economic data) as well as to compute the economy’s productive potential. Some methods rely on purely statistical techniques. Others, such as CBO’s method, rely on models guided by economic theory. Many methods used to compute potential output do not benchmark their trends to inflation or any independent measure of capacity and therefore cannot be interpreted as estimating the level of maximum sustainable output. That is, they provide a measure of trend output but not potential output.

If we, Laurence Ball and I, are not measuring the output gap as the difference from potential output, then we are likely understating the size of the gap.

If there is a long-term trend of decline in realized GDP, as I believe, then there may be a long-term increase in the size of the output gap that we are understating.


Thought about it. Given the actual unemployment we've had over the years, it is reasonable to draw a "trend line" that minimizes the actual variations, and helps us picture the general trend. Likewise output.

And then is is interesting to compare the differences-from-trend of unemployment against the differences-from-trend of output. This is what we see in the above graphs. There is nothing circular in the arithmetic, that I can see. And the correlation appears to be strong.


Regarding the second graph in the Econbrowser post, we read:

Our finding of a stable Okun’s Law is robust to various methods of measuring short-run movements in output and unemployment. We try alternatives to the Hodrick Prescott (HP) filter. We also estimate the relationship in “changes”, that is between the change in the unemployment rate and the change in (log) output, which does not require using the HP or any other filter. The relationship holds for quarterly as well as annual data. Chart 2 shows the tight fit between actual unemployment and the estimate based on Okun’s Law. Some residuals are evident during the early years of the Great Recession, for which Ferrara and Mignon provide some conjectures.

Chart 2. United States: Actual and Fitted Unemployment Rate, 1948Q2-2011Q4. Notes: Figure reports fitted unemployment rate from Okun specification estimated on quarterly data in levels with two lags and natural rates based on Hodrick-Prescott filter with λ = 100.

Trying to get clear on what they're saying. This part is easy: "Chart 2 shows the tight fit between actual unemployment and the estimate based on Okun’s Law." In this case, they definitely *are* using Okun's law to determine "fitted" unemployment.

The note below the graph refers to "levels with two lags" which must be some specific detail of the calculation, but the meaning escapes me.

The note below also refers to "natural rates based on Hodrick-Prescott". Rates, plural, so I think they figured HP trends for both output and unemployment, and used these somehow in their calculation. Then Okun's law comes into play. Working it backwards, they use the output gap to estimate unemployment. They call this estimate "fitted" unemployment, and they compare it to "actual" unemployment in their graph.


Okay. I think I figured out a way to estimate unemployment numbers using Okun's law and the HP trends. In my post (linked at the top) I wrote:

To figure potential output, the Congressional Budget Office uses the unemployment gap. They take that gap and stretch it to fit over actual output. That gives them the output gap. The output gap looks like the unemployment gap by design

I'll do the same thing now, but reverse the direction. First I'll figure the HP trends for unemployment and GDP. Then I'll take the difference of GDP and its trend, shrink it by half -- that's the Okun's law part -- and add it to the unemployment trend, then use the result for my estimated unemployment numbers. Now it's simple.

Graph #4

Holy crap! It's a really good match.

Okay, again: Take the discrepancy between RGDP and its HP trend, shrink it by half per Okun's law, and add it to the unemployment HP trend. That's the red line. The blue line is FRED's UNRATE.

What I've learned:
1. The HP trend calc is a really good trend calc.
2. Laurence Ball's "output gap" is the difference between RGDP and its trend, without regard for inflation or the Phillips curve which play a role in the CBO calc. Ball's output gap is based on the trend of realized RGDP, not on Potential Output. This is a significant difference, but I now think not relevant to Ball's Econbrowser post.
3. Laurence Ball's calc is *not* circular, and it *is* interesting.


On Graph #4, the blue line is above the red line when actual unemployment is higher than the estimate. The blue line is lower when actual unemployment is lower than the estimate.

We like low unemployment. We like the blue line low.

Looking at Graph #4, I see two periods where the blue line is clearly below the red line for more than a very brief period. Those periods are 1960-1964 approximately, and 1994-2000 approximately. Those were both periods of exceptionally good economic growth, so we should expect the unemployment to be relatively low in those times. We should expect the blue line to be relatively low in those times.

I decided to look at the discrepancy between the two lines. I subtracted the blue line from the red. This gives me the "error" of the unemployment estimate, relative to realized unemployment.

Graph #5

For the most part, the error is within plus or minus half a percentage point. Not bad.

//

The Excel file containing my graphs and calcs is available for download from Google Drive.

Note that the file contains Kurt Annen's Hodrick-Prescott filter in Visual Basic code, and also some routines I use for formatting my graphs. If your security settings are reasonable, when you open the file Excel will warn you of a potential problem because the file contains VB code.

Excel doesn't know there's a problem. It only knows there could be a problem.

Excel lets you disable the code and open the file. But with the code disabled, the H-P filter calcs won't work and the whole thing will probably be garbage. So maybe you don't care to mess with it. But anyway, the file is available if you want.

Hindsight

Change in TCMDO, corrected for inflation. Log of that, so that a constant rate of growth looks like a straight line.

Graph #1: Natural Log of the Inflation-Adjusted Change in "Total Credit Market Debt Owed"

Same graph in Excel, with a Hodrick Prescott trendline:

Graph #2: Ditto, plus Trend

Straight lines indeed. The red line is flat, briefly, before 1958... Straight line uphill, 1958-1986... Straight line downhill, 1986-1992... And straight line uphill, 1992 to the crisis. I stopped the HP calc at that point so that the later decline did not drag down the earlier years' numbers.

The media would tell you debt is a problem over there on the right, where the blue line is broken and there is no red line.

Most people would tell you debt was a problem in the 2000s, where the red line goes above the 3.5 level, or maybe where it goes above 3.

Lots of people would tell you debt went up during the Reagan years. For some reason, most people don't seem to know about the great slowdown of debt growth, 1986-1992. Too many media stories, maybe.

Apart from that downtrend, I would say the only time the graph does not show a problem is in the early years, the early 1950s. That's the only time the trend does not show increase. Looking at this graph, you could have known by 1961 that debt was going to be a problem.

A second look at the heteconomist post

The other day I looked briefly at PeterC's Why Neoliberals Pretend Private Debt Doesn't Matter and Public "Debt" Does. Time for a second look.

Consider the opening sentence of the post. PeterC writes:

The neoliberal policy approach in the decades leading up to the crisis basically amounted to enticing or pushing people into increasing levels of private debt.

What is "neoliberal"? Wikipedia doesn't help. Bill Mitchell writes of "the early 1980s (as the neo-liberal onslaught began in earnest)."

The early 1980s, then. Since Reagan, basically. Same as my K-R Shift. And actually, in his post PeterC writes

...the neoliberal attack on workers' pay and conditions from the early 1980s onwards was highly orchestrated.

The early 1980s it is. So reconsider Peter's opening thought, revised:

The policy approach since the early 1980s and up to the crisis basically amounted to enticing or pushing people into increasing levels of private debt.

Is it true? It must be true, right? I mean, how else did we end up with all this debt.

Yeah, but take a look at the rate of debt growth since the 1950s:

Graph #2: Percent Change in the Non-Federal Portion of TCMDO (blue)
If I did it right, the red line is a Hodrick-Prescott trend of the debt data.

The first thing that happened "after the early 1980s" was a slowdown in debt growth! A slowdown beginning in the mid-1980s. Immediately after the early 1980s there was a major downtrend in the growth of debt.

But by the late 1990s, the growth rate of debt was back to normal. Back to 10% annual, give or take. Back where it was before the neoliberals took over. Back where it was in the Keynesian years.

What's different is that in the Keynesian years, excessive debt growth led to inflation. In the neoliberal years, excessive debt growth led to unemployment. That's the main difference.

The problem is the excessive debt growth: the excessive reliance on credit, the excessive cost of circulating money, excessive finance, excessive accumulated savings, excessive accumulated debt, all the same thing. It's a cost-push problem.

The liberal solution to the cost-push problem is inflation. The conservative solution is unemployment. The Arthurian solution is to reduce financial costs by reducing the size of finance, providing more money to balance against less debt, using policy to limit the amount of creditmoney that can be generated from a dollar of money.

The Arthurian solution is to reduce the factor cost of money.

De-Trending

It was easy. I couldn't even remember the name Hodrick, so I Googled prescott filter excel. Jackpot.

There is a how to use PDF, short, says you give it a range of cells and a number (a constant). The long version: Select a range of cells for your results, type =HP( , select the range of cells containing the data to be filtered, type in a comma, type in the constant, and close the parentheses. But don't hit ENTER.

Instead, hold down CTRL and SHIFT and then press ENTER. That's it. Then you can make a graph or whatever, from the results.

Holding down CTRL and SHIFT while you press ENTER is standard Excel stuff. They call it "array formulas" which sounds pretty complicated... but all you have to do is hold down CTRL and SHIFT while pressing ENTER. How complicated is that?

None of that works, though, if you don't have the Hodrick Prescott filter installed. No problem. Among the search results is a link to Kurt Annen's HP-Filter Excel Add-In at IDEAS. There are three separate files you can download:

1. an XLA file, an Excel add-in which adds the HP( ) function to the built-in Excel functions.
2. the Visual BASIC source code for the HP( ) function.
3. an example.

For most people the simple thing would be to install the add-in. (The PDF linked above tells how.) For me, the simple thing was to copy the Visual BASIC source code and paste it into a code module. That way I got to look the code over a bit. It looks like a very complicated (arithmetically) version of a "moving average" calculation, or something comparable. That's a crude description; I didn't work through all the arithmetic. But the result you get from the HP filter serves the same sort of purpose as the result you get from the moving average calc.

Anyway, after it's installed one way or the other, it's as easy to use as =SUM( ) or any other Excel function. I expect to explore it, and to use it.


What number do you use for the constant?... for the "Lambda" as they call it?

At the EViews User Forum, Trubador explains:

Rule of thumb is:
Lambda = 100*(number of periods in a year)^2

In this respect, for:
Annual data = 100*1^2 = 100
Quarterly data = 100*4^2 = 1,600
Monthly data = 100*12^2 = 14,400
Weekly data = 100*52^2 = 270,400

I think you have got the idea...

1600 for quarterly data. That was easy to remember. I didn't even have to look it up again, the first time I tried it.

Drewtedlock quotes Trubador

Rule of thumb is:
Lambda = 100*(number of periods in a year)^2

and responds:

There is additional research that suggests using a power of 4 instead of 2. See Ravn and Uhlig (2002). http://ideas.repec.org/a/tpr/restat/v84 ... 1-375.html

(I didn't explore his link.)

And a hint from Trubador: "I'd suggest you to seasonally adjust your series first."


When I have an hour or nine to spend on it, web:reg provides an interesting discussion.

// Update 30 March 2014: For a look at how changing the constant affects the result, see mine of 19 March 2014.

// Update 16 September 2014, Recommended Reading: There is an interesting analysis of the Hodrick-Prescott filter, how it works, and some problems with it, in Blogs review: HP Filters and business cycles at bruegel.org.