The study released this week titled “Do Energy Efficiency Investments Deliver? Evidence from the Weatherization Assistance Program” by Meredith Fowlie, Michael Greenstone, and Catherine Wolfram on energy savings from residential energy programs is seriously flawed and does not present a balanced assessment of the benefits of investing in energy efficiency in either its final or working draft form. This study was supported by a grant from The Alfred P. Sloan Foundation and run through E2e. E2e is a joint initiative of the Energy Institute at the University of California at Berkeley’s Haas School of Business and the Center for Energy and Environmental Policy Research at the Massachusetts Institute of Technology.
Residential energy efficiency is responsible for 22% of the energy consumed in the United States, accounting for more than $230 billion annually in home energy bills. Many assessments have confirmed the energy savings potential that exists in residential homes. The focus of the recent study was the Weatherization Assistance Program, which has been responsible for weatherizing more than 7.4 million homes during the nearly forty year history of the program. Weatherization has helped ease the disproportionate burden of high energy costs that falls on low income Americans, especially senior citizens, those who are disabled, families with children, and low income households. The program also is responsible for creating new work pathways as well as contributing to community revitalization and reduced carbon emissions.
Other more broad-based and comprehensive evaluations of the weatherization program negate the findings in the recent study and prove what those who have received or provided weatherization services know: this program saves energy and reduces the burden of utility costs for the most vulnerable populations in our society. (Examples of evaluations include but are not limited to those linked here from ORNL, here from Berkeley Lab, here from Policy Matters Ohio, here from independent contractors, and here from Harvard)
Weatherization is the largest residential energy efficiency program in the country. Conclusions by studies, including but not limited to those referenced here, are important and relevant to shaping policy; therefore, it is absolutely imperative that such studies are held to rigorous standards and sample a sufficient number of homes to be conclusive, are peer-reviewed for accuracy and efficacy, draw sound and logical conclusions without relying on assumptions or unfair estimations, and hold up under scrutiny. The weatherization study conducted in Michigan by Fowlie, Greenstone, and Wolfram does not meet these standards. The sample included only two weatherization providers and as few as 349 homes were sampled for specific data segments. Additionally, though the abstract cites more than 30,000 households in the sample, the researchers were only able to match audit data for 1,638 homes — an insignificant and statistically irrelevant number compared to the more than 1 million homes weatherized during the American Recovery and Reinvestment Act period.
Additionally, the study draws illogical conclusions and makes unfair assumptions about the populations of weatherized customers. The researchers conducted what they refer to in the study as an “encouragement campaign,” in which potential customers are inundated with information to encourage voluntary participation in the program. However, what the researchers fail to articulate is that advertising does not drive people to the program. Instead, crisis largely drives participation. For example, the research states that ” given that households had detailed, specific information about the program, it seems reasonable to surmise that some combination of high perceived costs of applying for the program, low expectation of an application leading to weatherization, high unmeasured process costs, and low expected benefits of participating in the program are impediments to WAP participation.” This is a broad and unfounded assumption. Any first year weatherization staff person realizes that people do not apply for weatherization because they are convinced of the importance of energy efficiency measures for their home — they are driven to request services because they are in crisis. They cannot afford their utility bills. In Michigan, where the study was conducted, customers may reach out for help because their heating system has failed and the weather grows cold, they might not be able to afford to pay their energy bill, or are seeking help to remain safe and healthy in their homes.
People reach out for weatherization assistance due to practical or crisis concerns, not because lots of flashy materials told them their homes needed energy efficiency measures. Low-income weatherization customers live day-to-day and their main concern is surviving rather than thriving. Fortunately, weatherization provides for both. The researchers admit that “…the aggressive encouragement efforts were disappointing.” Indeed, they were, not because of any lacking on the part of the customers, but because the researchers themselves failed to understand the values of the population served, their circumstances, and their motivations.
We would be as remiss if we also failed to note that some homes have no heating or cooling system (example) or have a dysfunctional system prior to weatherization (example). Correction of this health and safety concern by installing a high efficient appliance will mean that utility bills and usage actually increase because the family has a working system post-weatherization, and had no system prior to the weatherization work. It is unclear whether the researchers considered this key factor when conducting the study. There are no references to elimination of these homes from the sample or correction for this variable. Failure to take into account this important factor would unfavorably skew the data conclusions drawn by the research.
Of critical note also is the fact that the research uncovered — but elected not to emphasize — a number of positive outcomes of weatherization, including the following:
- WAP energy efficiency investments reduce monthly energy consumption by 10-20 percent on average.
- Weatherized households have historically consumed significantly less natural gas than non-weatherized homes during both winter and summer months.
- WAP participation reduced energy consumption by roughly 8-10 percent in the relatively small sample of homes from two agencies in one state.
- The study estimated energy savings of approximately $155 per year for weatherized homes. This represents a savings for vulnerable participants who may then apply those savings to other household needs. Although the savings number is only an estimate and is based on assumptions that may or may not be valid, any savings is positive for the populations mainly serviced by weatherization (priority for service is given to the elderly, disabled, families with children in the home and those with high energy usage of burden).
- Average natural gas bills during winter months are $85.95 for non-weatherized households. For weatherized households, the average bill drops to $57.96.
These positive outcomes were mostly buried in an overall negative and unbalanced perspective provided by the researchers. Outputs by studies such as this receive attention from policymakers and should therefore adhere to rigorous standards of quantitative analysis and present facts in an unbiased light. It is clear that this report misses the mark. Policy debate regarding the costs of energy efficiency should be driven by the best available analyses, and the goal of research should be to narrow the range of uncertainty and produce better cost estimates useful in the ongoing discussion of energy efficiency policy.
The study focuses almost exclusively on the impact of weatherization from an energy-saving perspective. This is to be expected. Weatherization is, after all, an energy efficiency program. However, energy savings is not the only benefit conferred by weatherization. There are tremendous benefits to households that are not reflected in increased indoor air temperatures — to compare the benefits of weatherization exclusively using this one metric is akin to judging a painting based on one brush stroke. There are comfort, safety, and health benefits to participants; structures are improved; communities are enhanced as housing stock increase in value; the health and safety of the residents in the home is improved, and customers served often are able to live independently and remain in their homes thanks to the work of the program. These unmeasured benefits are not recognized by the research produced by Fowlie, Greenstone, and Wolfram, yet they are powerful testaments to the value of weatherization.
