One major challenge with improving energy assistance programs is with defining what a “good” program is, whether it’s for discount programs, low-income weatherization or arrearage management.
Some metrics are easy to calculate but don’t actually tell you anything about the quality of the program.
- We serve 5000 customers a year. Participation is the quintessential vanity metric. How much of these customers’ energy burden is actually reduced? Are those the right customers to be serving?
- 100% of participants are satisfied with our program. Who doesn’t like free, whether in the form of home upgrades or money? Is the high customer satisfaction being translated to an improved image of the utility more widely? Do customers even know the utility’s role in the program?
- Our annual energy assistance budget is $XX million. Bigger isn’t always better—are these funds being used cost-effectively? Could these funds be used in different ways for more persistent benefits?
Before developing metrics or KPIs, utilities should be crystal clear on the goals of their programs – steering clear from vague, unmeasurable benefits and vanity metrics.
The two main value propositions for energy assistance programs are improved on-time payment rates for low-income customers and enhanced customer satisfaction. These assume that the programs manage to improve energy affordability for program participants and that this impact is communicated to the utility’s customers.
Realizing the benefits of energy assistance programs starts with demonstrating a reduction of energy burden for high-burden customers.
Read on to learn how an energy burden lens on energy assistance programs can give you simple metrics that can be quantified, tracked and used to drive concrete improvements to programs. Utilities can leverage this energy burden framework to optimize their programs—whether delivered in-house or with third-parties, such as community action agencies—to make sure that utility funds are being used effectively.
Short on time? Download the full Comprehensive Guide to Quantifying Energy Equity for your reference.
Send download link to:
What Problem Are We Solving?
Our goal is to reduce energy assistance need. In other words, we don’t want high-burden, low-income customers to spend more than a certain percent of their incomes on energy (this threshold can be anywhere from 4% to 10%).
To get a clear picture of program performance, we need to calculate four values (see image below):
Energy assistance need is a single dollar value that can be calculated and tracked year-over-year, usually through low-income needs assessments. Some approaches to calculating this number are discussed in a later section.
In most areas, the total energy assistance funding that is available to customers is some fraction of the energy assistance need. But funding levels by themselves do not capture the success of a program. You could theoretically dump millions of dollars in a program and not affect energy assistance need by a single dollar, because the funds aren’t reducing energy burden for high-burden customers. This takes us to the concept of avoided burden.
Avoided burden is the actual dollar reduction in customer energy bills resulting from energy assistance programs. This can be lower than the total energy assistance funding due to overhead expenses or the installation of non-cost-effective conservation measures. This number is an output of program impact evaluations. Ideally, it would be calculated annually or every couple of years, but, unfortunately, many assistance programs rarely, if ever, get evaluated.
This is not the end of the story. Remember, as utilities, we’re trying to help payment-troubled, high-burden customers, not simply offer free cash to low-income customers. So the final value we need to calculate is the avoided need. This is the avoided burden specifically for high-burden customers and can easily be calculated from program participation data. It’s usually much smaller than avoided burden because most low-income programs do not target high-burden customers.
[Background Reading: The Six Key Energy Equity Terms You Need to Know]
Program Effectiveness Metrics
Using this energy burden framework, effective energy assistance programs achieve a high level of avoided need and demonstrate continuous progress by shrinking the gap between avoided need and total energy assistance need.
We can express this energy assistance to avoided need gap with three ratios. Each ratio represents a lever we can use to improve our energy assistance program effectiveness.
- Funding Ratio: the ratio between energy assistance funding and energy assistance need
- Operational Effectiveness: the ratio between avoided burden and energy assistance funding
- Targeting Effectiveness: the ratio between the avoided burden and avoided need
These three ratios multiplied by each other yield the Overall Program Effectiveness at reducing the energy burden of high-burden customers.
So, we have three levers to create a great energy assistance program: increase funding, improve efficiency of operations and effectively target high-burden customers.
A great program has enough funding, streamlined operations and is designed to target high-burden customers. It looks like this:
Most programs, however, have insufficient funding and aren’t particularly intentional about targeting or operational effectiveness. Three small ratios multiplied by each other result in a much smaller overall effectiveness.
How do we get to the gold standard of effective assistance programs?
To push that green circle up, you can influence the three levers: funding, operations and targeting.
Unfortunately, whenever a utility or program administrator considers doing more for low-income customers, the default reaction is to use the funding lever by pumping more money into program budgets. This overlooks the fact that the success metric is avoided need not just program funding.
If the programs are inefficient at delivery and targeting, when we rely on this option, we are hoping for a “trickle-down” effect. Funds are injected in the program budget. Some of it will be spent on program administration, operations and customers who don’t need the assistance. Only a portion of the additional funding will eventually make its way to the right customers. Most of these funds aren’t actually addressing energy assistance need.
One alternative to increasing program budgets alone is to leave program budgets unchanged and instead devote some of the funds to doing things smarter by optimizing operations or targeting.
Operational effectiveness encompasses things like program workflows, marketing, customer service, choice of incentive levels and measures, performance tracking and KPIs, among others. Program evaluations, when well-executed, can yield valuable insight and actionable recommendations for improving operational effectiveness or even guiding a full program (re)design.
Improving targeting effectiveness requires a comprehensive understanding of the demographic and geographic characteristics of high-burden customers to guide targeted marketing and outreach approaches. This can be accomplished through low income needs assessments. Program designs can also support the purpose of targeting by designing incentive or discount structures that are better aligned with energy burden. Integrated marketing that intentionally focuses on key customer segments is also vital to improving overall program targeting effectiveness.
How Do we Calculate Program Effectiveness Metrics?
Metric #1. Energy Burden
The calculation of energy burden requires data on the annual energy bills and gross income for a group of customers or an entire service territory. These can be obtained from census microdata (for example from the American Community Survey), from customer surveys administered by the utility or by using a combination of utility billing data and customer-level demographics (from customer data aggregators). In most cases, some level of modeling will be required to fill in data gaps, but the degree of modeling will vary based on the extent of available data. For example, census microdata covers less that 5% of customers in a service territory and requires extensive modeling, while utility data requires minimal modeling.Metric #2. Affordability Threshold
Once energy burden has been calculated, you need to determine a threshold value above which a customer would be considered to have “high energy burden” for your service territory. Sometimes, this value is set by regulators. A program/utility could set its own threshold—usually varying from 4-10%, with 6% being a very common threshold. Alternatively, utilities can deploy a well-designed survey that identifies this threshold for their service territory. This type of survey would tie the level of customer energy burden with their ability to afford basic necessities, their likelihood of being late on their bill payment and the practice of keeping homes at unhealthy temperatures to save on bills. The advantage of this approach is that it takes into account specific needs and perceptions of a utility’s customers, along with competing expenses for other essentials and the general standard of living in the area.Metric #3. Low-Income Threshold
The low income threshold is more of a program design question revolving around eligibility rules for programs than a metric for program effectiveness. However, it is useful to incorporate various low income thresholds when evaluating programs or performing needs assessments to understand the repercussions of this choice. Low income thresholds are typically set as a percent of the federal poverty limit or the area median income.Metric #4. Energy Assistance Need
The total energy assistance need in a service territory depends on several factors:- Household energy use and efficiency
- Household income levels and, by extension, unemployment rates
- Weather, especially the severity of winters in northern climates and summers in southern climates
- Approach #1. Econometric Modeling of Sampled Data
The first “econometric” method of estimating energy assistance need relies on sampled survey data along with extrapolation models that yield metrics across a county or service territory. One excellent example of this approach is the Low-Income Energy Affordability (LEAD) tool published by the Office of Energy Efficiency and Renewable Energy at the Department of Energy. (Note that the LEAD tool only provides estimated averages of energy burden, not the actual energy assistance need – some additional analysis would be required to arrive at the latter.)
Pros: This class of methods can be very useful for policy purposes, as it offers consistent calculations that can be applied across an entire state or even the whole country for comparative analysis.
Cons: However, these methods can suffer from drawbacks that limit their applicability in energy assistance programs, specifically:
- Timeline: Most of these approaches (including the LEAD tool) are based on 5-year American Community Survey microdata. So, the results are based on data that may be outdated and also too smoothed out to detect year-over-year changes in the future.
- Sampling accuracy: The data used in these methods is sampled from a small portion of the population (under 10%) and extrapolated across a service territory. When using the American Community Survey, the energy use data is self-reported and for a single month. The accuracy of extrapolating energy use from one month to a full year will depend on when the survey was answered and the level of seasonal variability for a service territory, calling into question the reliability of energy burden estimates.
- Granularity: Even if we were to overlook potential questions of timing or accuracy, these approaches do not tie data to utility customer accounts and often only go down to the census tract level. This means that results can be affected by “outlier” meters that do not represent most customer accounts (for example, vacation homes, garages, commercial uses, etc.). This also means that the results are too broad to use for specific program design and marketing strategies because the geographical units are too broad.
These drawbacks mean that the approaches can fall short of providing actionable data for driving program design and informing targeted outreach for specific utility programs. However, they could still be useful for comparative analysis in academic or high-level policy contexts.
A second, “data-science” approach to estimating energy assistance need relies on gathering as much real data as possible from the service territory, with minimal modeling to fill in data gaps. For example, the utility has energy use data for 100% of its customers. Income data can be purchased data from data aggregators or credit bureaus. Building level data can be obtained from county assessors. One example of this approach is delivered through our Equity Dashboard (see image below), which allows utility program managers to slice and dice their data and develop customized program delivery and marketing strategies for their service territory.
Pros: The advantage of working with customer or meter-level data is the ability to control the quality of data that goes into the energy assistance need estimates. For example, meters that are not tied to households can be identified and eliminated. Meters that show minimal energy consumption can be flagged as potentially unoccupied. If a utility wishes to monitor its energy equity progress, it can always use the most recent data available. Finally, performing the analysis at the household level means that insights can be extracted at various levels of granularity.
Cons: One drawback with this type of approach is the level of effort required to gather the disparate datasets and perform the analysis, but it is more than balanced by the level of insight that can be gained – the accuracy and granularity of the approach makes it appropriate to designing specific energy assistance programs.
A third approach combines elements of econometric modeling with data science. One such method leverages a modeling approach with a statistical procedure called “iterative proportional fitting” as the backbone, but uses real data wherever possible for calibration. For example, actual energy consumption data can be easily obtained from utilities and used in place of surveyed estimates and building characteristics (e.g. building type) can be obtained from county assessors. Demographic data like income, ethnicity and homeownership is harder to obtain and more sensitive. With this approach, you could rely on American Community Survey estimates of these attributes.
Pros: This approach would enhance the reliability and data relevance of energy assistance estimates while avoiding sensitive data. It also requires a lower level of effort than a pure data science approach, making it suitable for assisting policy makers or setting energy equity targets for utilities.
Cons: This approach would have low granularity, so it may or may not be useful for in-depth program design.
[Not interested in applying statistical methods yourself? Get in touch to figure out what it takes to do this for your service territory.]
Metric #5. Energy Assistance Funding
This is the total dollar amount of funding flowing through energy assistance programs, including discount, donation, arrearage management and weatherization programs. This is typically well-known to program administrators and can be retrieved from the program accounting systems. One minor tweak to program accounting practices is to attribute funding to specific customers, so that service gaps can be identified for various customer segments.
Metric #6. Avoided Burden
Avoided burden can be determined through program impact evaluations, which identify the actual bill reductions for program participants. Program evaluations rely on data collected from program tracking databases and accounting systems. For conservation programs, program impact is determined by performing an analysis of customer energy consumption prior to and after the installation of efficiency measures.
Metric #7. Avoided Need
Avoided need is calculated by identifying which program participants would qualify as “high energy burden” based on the affordability threshold. The total bill reductions actually experienced by this customer group is the avoided need. The data required for this calculation (income and energy use) is usually stored in program tracking databases as it is required for checking customer program eligibility.
Interested in learning more about quantifying low-income programs?
Send download link to:
How Do We Segment Customers When Quantifying Equity?
Most of the discussion so far has revolved around aggregate metrics across a service territory. The true value of understanding energy burden within this framework is when these same metrics can be studied for specific customer segments. This “slicing and dicing” is especially valuable for designing specific marketing and outreach strategies, as well as for tweaking program application workflows and incentive levels for maximum impact.
Some of these relevant segmentation dimensions are:
- Geographical Location: Where are the customers with high energy burden located? Where does the current energy assistance funding go?
- Income: Is high burden concentrated in customers with the lowest incomes? Or is it a function of high energy costs?
- Age: Do older customers on fixed incomes need the most assistance? How do we accommodate working-age families?
- Building Type and Homeownership: How does energy burden compare in single family and multifamily properties? Do renters shoulder a higher burden than homeowners, and do they have equal access to energy assistance programs?
- Race/Ethnicity/Language: Are there barriers in the existing programs that preclude certain demographics from learning about assistance programs or accessing assistance funds?
- Urban/Rural: For larger utilities, are the energy burden and program participation rates markedly different across rural and metropolitan areas?
Equity Indices for Customer Segments
The overall metrics discussed earlier should be supplemented with some key indices that are applicable to specific customer segments. These indices help quantify equity across customer segments and highlight segment gaps in program delivery.
- Burden Index: The ratio between a customer segment’s proportion of burdened households and their proportion of the total population. For example, if a certain customer segment comprises 10% of burdened households and is 5% of the population, then the burden index is 2. An index of less than 1 indicates an under-burdened segment, while greater than 1 indicates an over-burdened segment.
- Program Equity Index: The ratio between the percent of total energy assistance budget received by a given customer segment and their proportion of the total population. For example, if a certain customer segment receives 2.5% of total assistance funding and is 5% of the population, , then their equity index is 0.5. An index less than 1 indicates an underserved segment and greater than 1 indicates an overserved segment.
- Energy Cost Index: The ratio of the median annual energy bill for a given customer segment and the median annual energy bill for customers outside this segment. For example, if the median annual energy bill is $1500/year for a certain customer segment and $1000/year for everyone else, the energy cost index is 1.5. An index greater than 1 indicates higher than average energy use.
- Late Payment Index: The ratio of the late bill payment rate for a given customer segment and the late bill payment rate for customers outside this segment. For example, if the late bill payment rate is 10% for a certain customer segment and 5% for everyone else, then the late payment index is 2. An index greater than 1 indicates a customer segment with more frequent late bill payments than average.
Closing the Energy Assistance Gap
We’ve shared one framework for delivering more effective energy assistance programs. But as with most things in life, it’s all about execution.
[Read next: Smart Strategies for Optimizing Energy Efficiency Programs]
The easiest step that an energy assistance program administrator can take is to start laying the foundation for quantifying energy assistance programs. All of the data you’ll need exists in one form or another, and it’s usually just a matter of combining the data in a coherent manner. The metrics are also relatively easy to calculate and understand, and once they are placed in the context of a specific utility, it becomes easier to spot potential areas of improvement, underserved customer segments and funding needs.
Ready to make your programs more effective and equitable?
Stop relying on vanity metrics and start quantifying your actual program performance using this energy burden framework.
Learn more about our energy equity services or schedule a free consultation
Or interested in learning about strategies to improve your energy assistance programs?
Download your free copy of our Comprehensive Guide to Quantifying Energy Equity.
Send download link to: