Utopia, you are standing in it!

Various bold claims have been made about the payoff from investing more in retrofitting insulation into housing. The government recently spent $600 million on such retrofitting of insulation.

https://twitter.com/PhilTwyford/status/728137160113557505

There is a private member’s bill before Parliament to introduce minimum standards for rental properties with regard to insulation and other matters. Little is by the Leader of the Opposition Andrew Little said for the consequences for rents of this additional expense to landlords.

Ian Harrison of Tail Risk Economics initially estimated that the $600 million invested in retrofitting of insulation will save barely half of that:

After correcting for this major error and taking a more realistic view of the benefit estimates in other studies, the net benefits of $630 million disappear.

The $600 million insulation investment will probably generate benefits of closer to $170 million, for an economic loss of $430 million.

After meeting with Ian, I read through the rather dull background documents behind a cost benefit analysis relied upon by the government to spend the $600 million dollars.

The most interesting part of the cost benefit analysis is most of the benefits come from fewer cardiovascular related hospitalisation of the elderly and not from respiratory diseases among children.

I found the error was far more fundamental than a incorrect transfer of a calculation between tables discussed in the first publication by Harrison. I had to read the background documents several times to understand what had been done wrong.

The cost benefit analysis for the Warm Up New Zealand Heat Smart Programme assumes that the number of elderly occupants of the newly insulated house increases by one each year and after 5 years, one of these dies but is replaced by a new elderly occupant.

We have modelled the probability of a vulnerable person avoiding mortality as a result of the intervention. The probability of this is (112.7/1000)*0.27= 0.03 (3%). We treat avoidance of mortality by treatment in each year as independent events.

The multi-year benefit calculated above would accrue based on the life years gained as a result of deaths avoided in year one.

However, we would expect these benefits to accrue in year two for different vulnerable individuals (aged 65 and over with a cardiovascular related hospitalisation in previous 18 months), and for different individuals again in every subsequent year that the treatment continues to have an effect, i.e. an on-going stream of benefits of $1,050.74 per year. This assumes a constant proportion of people aged 65+ who have recently been hospitalised with circulatory problems….( p.38).

In the first year of the new insulation, the first occupant benefits and the net present value is included in the benefit cost analysis calculation – the erroneous benefit cost analysis calculations which its authors still defend.

In the 2nd year, another elderly person moves into that same house and the same calculation is done for them. In the following year, yet another elderly person moves into the same house and the net present value calculation is repeated.

By the end of 5 years, there are 5 occupants in this house all benefiting from the same insulation investment. In the 6th year, the first elderly occupant dies to be replaced by a new elderly occupant who then gains from the insulation upgrade.

There was double counting of the number of people who benefited from the insulation as Iain Harrison explains

The analysis assumed that there was not one, but five occupants who had been hospitalised with a cardiovascular illness in the previous 18 months in each of the relevant insulated houses. There should have been only one such occupant.

The retrofitting of insulation was estimated to cost $600 million. Iain Harrison estimated the benefits to be $300 million, not $1.2 billion. That is a benefit cost ratio of 0.5.

Source: Iain Harrison, The mortality reduction benefits of insulation: the error identified.