How accurate is your house energy model compared to tools like Rem/RATE?

The energy model I use I built myself. There are lots of software packages out there but the reviews of them are terrible. They have a very poor track record of predicting the real world energy performance of any particular house. See this review I wrote a couple of years ago:

Zero-Energy Ready Home (ZERH) and Home Energy Rating System (HERS) 

The Department of Energy offers its Zero Energy Ready Home (ZERH) program but it is more aimed at certifying builders rather than buildings. Hence, just like the PassiveHaus and LEED programs it is focused on new construction, not how to go zero on your existing home. The ZERH program relies heavily on EnergyStar standards for appliances and windows and the HERS (Home Energy Rating System) for performance. HERS is focused on energy use relative to a benchmark house (i.e., how your home compares to a model house of the same floor area)  rather than minimizing energy or spending. A HERS rating is only available on new houses, not for existing ones. A review of the HERS rating system in Home Energy magazine found that,  in practice, “there was no clear relationship between the rating score of an individual home and actual energy cost.” Hmmm.

My model began with simple curiosity. I began with just correlating (drawing a line graph) the actual energy (heating fuel plus electricity) that was used every day in my home (I have two years worth of daily data) and the average outside temperature. The r-squared (statistical correlation) of these models (there were 5 of them – one built each time I added one of the fab four) was over 80%. This is a very high correlation for a model that left out known influencers of energy demand like solar heat gain and drafts. Nevertheless, despite these obvious weaknesses, the outside temperature was by far the biggest driver of energy use and hence energy bills. This was an “ahha” moment.
In a separate “ahha” moment I realized that the u-value for windows was not just an arbitrary scale (unlike say a HERS rating) but actually was the rate of energy flow across the window. Since I could approximate the R-values of all my walls, attic and basement, (and the u-value is 1/R value) I could build a mathematical model of how the energy flowed out of my house. Basic physics requires that over any period longer than a few hours, the energy flowing into a house must equal the energy flowing out. This allowed me to build a predictive model of how the energy flows into and out of a house. Since I know the energy flowing in (the combined energy in the electricity plus that burned as heating fuel) I could anchor the model to reality before we even got started. Hence, the inaccuracies in my model are going to be in allocating where the energy flows out (e.g., I might over overestimate the energy flowing out through the walls and underestimate the energy flowing out through the attic) but the overall amount of energy lost must be correct because energy cannot be created or destroyed, the energy lost by your house must be equal to the energy you put in. Put another way, if you cut off the electricity and turned off the heating, your house would eventually reach the outside temperature.

As far as I know, all the other software modeling packages out there start with modeling the thermal envelope of the house. They then model the heating inputs and then hope that they have got it right. One of the most respected models out there is Rem/RATE. I have repeatedly asked the owners of this software for data on how accurate it is in real-world situations. They have never answered my questions. The most that they were willing to say is that “it meets the standards”, but could not even tell me what the standards were. There is almost no published data on the performance of Rem/RATE software. The only data that I have been able to find is the following chart from 2009:

Although the average prediction of the model (which is not plotted on the graph, the straight line is what at 100% r-squared would look like – clearly the model is no where near 100% r-squared, and they never publish this most basic statistic) it is obvious that there is enormous error in the predictions for one house vs another. In some cases the error is equal to the mean value! We use models to predict the heating or cooling load for one individual house, not the average of thousands of homes. Hence, I really doubt the value of Rem/RATE for helping homeowners cut their bills and carbon footprints. The HERS rating systems (which is reviewed negatively in the Home Energy magazine article that I quote above) is built on the Rem/RATE software.  I, and Home Energy magazine, are not the only one with these concerns, see this quote below from the same article:

“Of course, modeling older homes and heating, water heating, lights, and appliance loads is a different matter, and the divergence between modeled and actual energy consumption may be quite different. According to Blasnik, “I know from experience that many energy modeling tools—REM included—often do a poor job of modeling heating loads in older, leaky, poorly-insulated homes.”

 And yet, cutting energy use on “older, leaky, poorly-insulated homes” is exactly the problem we need to overcome! 

So I built my own model, and it not only predicted my actual annual heating bills to within 10% of the actual bills but it has proven itself in practice with all of my consulting clients. It has enabled me to make predictions of the real-world impact on both the energy bills and the financial bills for actual changes to that home like adding insulation or adding triple-glazed windows. This is why I do not use any off-the-shelf energy modeling software. They simply have a poor track record of predicting real-world energy and financial performance on houses such as those most people live in. 

If you use one of these software packages please let me know how you get on. I welcome any feedback that I can use to improve the model.