Energy or water end use assessment projects typically focus on individual energy or water systems in a particular market sector or building type. Monitoring usually requires separate meters or data collection channels for each end use, and analysts must account for all factors that may affect energy or water use. Examples of this approach include detailed utility load research efforts, energy or water audits, evaluation of utility incentive programs, and end use calibration of computer simulations. Depending on the project objectives, the frequency of data collection may range from one-time measurements of full-load operation to continuous time-series measurements.

Specific technology assessment projects monitor field performance of particular equipment or technologies that affect building energy or water use, such as envelope retrofit measures, major end use system loads or savings from retrofits (e.g., lighting, plumbing fixtures), or retrofits to, or performance of, mechanical equipment.

The typical goal of retrofit performance monitoring projects is to estimate savings resulting from the retrofit despite potentially significant variation in indoor/outdoor conditions, building characteristics, and occupant behavior unrelated to the retrofit. The frequency and complexity of data collection depend on project objectives and site-specific conditions. Projects in this category assess variations in performance between different buildings or for the same building before and after the retrofit.

Field tests of end use equipment are often characterized by detailed monitoring of all critical performance parameters and operational modes. In evaluating equipment performance or improvements to energy/water efficiency, it is preferable to measure in situ performance. Although manufacturers and laboratory performance measurements can provide excellent data for sizing and selecting equipment, installed performance can differ significantly from that at design conditions. The project scope may include reliability, maintenance, design, energy or water efficiency, sizing, and environmental effects (Phelan et al. 1997a, 1997b).

Accountability is increasingly necessary in energy and water performance retrofits, whether they are performed as part of energy savings performance contracting (ESPC) or directly by the owner. In either case, savings measurement and verification (M&V) is an important part of the project. Because the actual energy or water savings cannot be measured directly, the appropriate role of energy or water monitoring methodology is to

Proper assessment of a retrofit involves comparing before and after energy or water use, and adjusting for all non-retrofit changes that affected that use. Weather and occupancy are examples of factors that often change. To assess the effectiveness of the retrofit alone, the influence of these other complicating factors must be removed as best possible. To do so, relationships must be found between and these factors and energy or water use. These relationships are usually determined through data analysis, not textbook equations. Because data analysis can be conducted in an infinite number of ways, there can be no absolute certainty about a given relationship. The need for accuracy must be carefully balanced with measurement and analysis costs, recognizing that absolute certainty is not achievable. Among the numerous sources of uncertainty are instrumentation or measurement error, normalization or model error, sampling error, and errors of assumption. Each source can be minimized (to varying degrees) by using more sophisticated measurement equipment, analysis methods, sample sizes, and assumptions. However, more certain savings determinations generally follow the law of diminishing returns, where further increases in certainty come at progressively greater expense. Total certainty is seldom achievable, and even less frequently cost-effective (ASHRAE Guideline 14).

Other resources are also available. One of the widest known is the International Performance Measurement and Verification Protocol (IPMVP 2014). The IPMVP is more general than ASHRAE Guideline 14-2014 but provides important background for understanding the larger context of M&V efforts.

Diagnostic projects measure physical and operating parameters that determine the energy or water use of buildings and systems. Usually, the project goal is to determine the cause of problems, model or improve energy or water performance of a building or system(s), or isolate effects of components. Diagnostic tests frequently involve one-time measurements or short-term monitoring. To give insight, the frequency of measurement must be several times faster than the rate of change of the effect being monitored. Some diagnostic tests require intermittent, ongoing data collection.

The most basic energy or water diagnostic for buildings is determining rate of use or demand for a specific period, ranging from a single point in time to a few weeks. The scope of measurement may include the whole building or only one component. The purpose can range from measurement system parameter estimation to verification of nameplate information. Daily or weekly profiles may also be of interest.

A large number of diagnostic procedures are used for energy and water measurements in residential (particularly single-family) buildings. Typical measurements for single-family residences include (1) flue gas and other analysis procedures to determine steady-state furnace combustion efficiency and the efficiency of other end uses, such as air conditioners, refrigerators, and water heaters; (2) fan pressurization tests to measure and locate building envelope air leakage (ASTM Standard E779) and tests to measure airtightness of air distribution systems (Modera 1989; Robison and Lambert 1989); (3) infrared thermography to locate thermal defects in the building envelope and other methods to determine overall building envelope parameters (Subbarao 1988); and (4) faucet and shower flow meter bags.

Energy and water systems in multifamily buildings can be much more complex than those in single-family homes, but the types of diagnostics are similar: combustion equipment diagnostics, air leakage measurements, flow meter bags, and infrared thermography to identify thermal defects or moisture problems (DeCicco et al. 1995). Some techniques are designed to determine the operating efficiency of steam and hot-water boilers, identify plumbing leaks, and measure air leakage between apartments.

Diagnostic techniques have also been created to measure the overall airtightness of office building envelopes and the thermal performance of walls (Armstrong et al. 2001; Persily et al. 1988; Sellers et al. 2004). Practicing engineers also use a host of monitoring techniques to aid in diagnostics and analysis of equipment energy performance. Portable data loggers are often used to collect time-synchronized distributed data, allowing multiple data sets (e.g., chiller performance and ambient conditions) to be collected and quickly analyzed. Similar short-term monitoring procedures are used to provide more detailed and complete commercial building system commissioning. Short-term, in situ tests have also been developed for pumps, fans, and chillers (Phelan et al. 1997a, 1997b).

Diagnostics are also well suited to support development and implementation of building energy or water management programs (see Chapter 37). Long-term diagnostic measurements support improvements (Liu et al. 1994). Diagnostic measurement projects can generally be designed using procedures adapted to specific project requirements (see the section on Steps for Project Design and Implementation).

Equipment for diagnostic measurement may be installed temporarily or permanently to aid energy and water management efforts. Designers should consider providing permanent or portable check metering of major electrical loads and plumbing fixtures in new building designs. Building automation systems also can be used to collect the data required for diagnostics. The same concept can be extended to fuel and thermal energy use.

Protocols for monitoring residential building retrofit performance can answer specific questions associated with actual measured performance. For example, Ternes (1986) developed a single-family retrofit monitoring protocol, a data specification guideline that identifies important parameters to be measured. Both one-time and time-sequential data parameters are covered, and parameters are defined carefully to ensure consistency and comparability between experiments. Discrepancies between predicted and actual performance, as measured by the energy or water bills, are common. This protocol improves on billing data methods in two ways: (1) internal temperature is monitored, which eliminates a major unknown variable in data interpretation; and (2) data are taken more frequently than monthly, which potentially shortens monitoring duration. Utility bill analysis generally requires a full season of pre- and post-retrofit data. The single-family retrofit protocol may require only a single season.

Ternes (1986) identified both a minimum set of data, which must be collected in all field studies that use the protocol, and optional extensions that can be used to study additional issues. See Table 3 for details. Szydlowski and Diamond (1989) developed a similar method for multifamily buildings.

The single-family retrofit monitoring protocol recommends a before/after experimental design, and the minimum data set allows performance to be measured on a normalized basis with weekly time-series data (some researchers recommend daily). The protocol also allows hourly recording intervals for time-integrated parameters, an extension of the basic data requirements in the minimum data set. The minimum data set may also be extended through optional data parameter sets for users seeking more information.

Data parameters in this protocol have been grouped into four data sets: basic parameters, occupant behavior, microclimate, and distribution system (Table 3). The minimum data set consists of a weekly option of the basic data parameter set. Time-sequential measurements are monitored continuously during the field study. These are all time-integrated parameters (i.e., appropriate average values of a parameter over the recording period, rather than instantaneous values).

This protocol also addresses instrumentation installation, accuracy, measurement frequency, and expected ranges for all time-sequential parameters (Table 4). The minimum data set (weekly option of the basic data) must always be collected. At the user’s discretion, hourly data may be collected, which allows two optional parameters to be monitored. Parameters from the optional data sets may be chosen, or other data not described in the protocol added, to arrive at the final data set.

This protocol standardizes experimental design and data collection specifications, enabling independent researchers to compare project results more readily. Moreover, including both minimum and optional data sets and two recording intervals accommodates projects of varying financial resources.

Several related guidelines have been created for evaluating retrofit savings (measurement and verification [M&V]). ASHRAE Guideline 14-2014 provides methods for effectively and reliably measuring the energy, water and demand savings due to building improvement projects. The guideline defines a minimum acceptable level of performance in measuring energy, water, and demand savings from conservation measures in residential, commercial, or industrial buildings. These measurements can serve as the basis for commercial transactions between energy services providers and customers who rely on measured energy or water savings as the basis for financing payments. Three approaches are discussed: whole building, retrofit isolation, and calibrated simulation. The guideline includes an extensive resource on physical measurement, uncertainty, and regression techniques. Example M&V plans are also provided.

The International Performance Measurement and Verification Protocol (IPMVP 2014) provides guidance to buyers, sellers, and financiers of energy projects on quantifying energy savings performance of energy retrofits. The Federal Energy Management Program has produced guidelines specific to federal projects, which include many procedures applicable to calculating retrofit savings in nonfederal buildings (FEMP 2015).

On a more detailed level, ASHRAE research project RP-827 resulted in separate guidelines for in situ testing of chillers, fans, and pumps to evaluate installed energy efficiency (Phelan et al. 1996, 1997a, 1997b). The guidelines specify the physical characteristics to be measured; number, range, and accuracy of data points required; methods of artificial loading; and calculation equations with a rigorous uncertainty analysis.

In addition to these specialized protocols for particular monitoring applications, a number of specific laboratory and field measurement standards exist, and many monitoring source books are in circulation.

Finally, MacDonald et al. (1989) developed a protocol for field monitoring studies of energy improvements (retrofits) for commercial buildings. Similar to the residential protocol, it addresses data requirements for monitoring studies. Commercial buildings are more complex, with a diverse array of potential efficiency improvements. Consequently, the approach to specifying measurement procedures, describing buildings, and determining the range of analysis must differ.

The strategy used for this protocol is to specify data requirements, analysis, performance data with optional extensions, and a building core data set that describes the field performance of efficiency improvements. This protocol requires a description of the approach used for analyzing building energy or water performance. The necessary performance data, including identification of a minimum data set, are outlined in Table 5.

New building construction offers the potential for monitoring building subsystem energy consumption at a reasonable cost. Information obtained by monitoring operating hours or direct energy consumption can benefit building owners and managers by

To provide data necessary to improve building systems operation, monitoring should be considered for boilers, chillers, cooling towers, heat pumps, air-handling unit fans, large fan-coil units, major exhaust fans, major pumps, comfort cooling compressors, lighting panels, electric heaters, receptacle panels, substations, motor control centers, major feeders, service water heaters, plumbing systems, process loads, and computer rooms.

Guidance on energy monitoring to determine energy savings for new construction design modifications is available in IPMVP (2014). Construction documents may include provisions for various meters to monitor equipment and system operation. Some equipment can be specified to have factory-installed hour meters that record actual operating hours of the equipment. Hour meters can also be easily field-installed on any electrical motor.

More sophisticated power-monitoring systems, with electrical switchgear, substations, switchboards, and motor control centers, can be specified. These systems can monitor energy demand, energy consumption, power factor, neutral current, etc., and can be linked to a computer. These same systems can be installed on circuits to existing or retrofit fans, chillers, lighting panels, etc. Some equipment commonly used for improving system efficiency, such as variable-frequency drives, can be provided with capability to monitor kilowatt output, kilowatt-hours consumed, and other variables.

Using direct digital control (DDC) or BAS for monitoring is particularly appropriate in new construction. These systems can monitor, calculate, and record system status, water use, energy use at the main meter or of particular end-use systems, demand, and hours of operation; as well as start and stop building systems, control lighting, and report alarms when systems do not operate within specified limits. Initial specification of the new control system should include specific requirements for sensors, calculations, and trend logging and reporting functions. Special issues related to sensors and monitoring approaches can be found in Piette et al. (2000).

Many common problems in monitoring projects can be avoided by effective and comprehensive planning.

Project Goals. Project goals and data requirements should be established before hardware is selected. Unfortunately, projects are often driven by hardware selection rather than by project objectives, either because monitoring hardware must be ordered several months before data collection begins or because project initiation procedures are lacking. As a result, the hardware may be inappropriate for the particular monitoring task, or critical data points may be overlooked.

Project Costs and Resources. After goal setting, the feasibility of the anticipated project should be reviewed in light of available resources. Projects to which significant resources can be devoted usually involve approaches different from those with more limited resources. This issue should be addressed early on and reviewed throughout the course of the project. Although it is difficult at an early stage to assess with certainty the cost of an anticipated project, rough estimates can be quite helpful.

Data Products. It is important to establish the type and format of the final results calculated from data before selecting data points. Failure to plan these data products may lead to failure to answer critical questions.

Data Management. Failure to anticipate the typically large amounts of data collected can lead to major difficulties. The computer and personnel resources needed to verify, retrieve, analyze, and archive data can be estimated based on experience with previous projects.

Data Quality Control. It is also important to validate the quality and plausibility of data before use. Failure to use some type of quality control often results in data errors and invalid results.

Commitment. Many projects require long-term commitment of personnel and resources. Project success depends on long-term, daily attention to detail and staff continuity.

Accuracy Requirements. The required accuracy of data products and accuracy of the final data and experimental design needed to meet these requirements should be determined early on. After the required accuracy is specified, the sample size (number of buildings, control buildings, or pieces of equipment) must be chosen, and the required measurement precision (including error propagation into final data products) must be determined. Because trade-offs must usually be made between cost and accuracy, this process is often iterative. It is further complicated by a large number of independent variables (e.g., occupants, operating modes) and the stochastic nature of many variables (e.g., weather).

Advice. Expert advice should be sought from others who have experience with the type of monitoring envisioned.

The following steps can facilitate smooth project implementation and data management:

For most projects, the collected data must be analyzed and reported. Because the objective of the project is to translate these data into information, and ultimately into knowledge and action, the importance of this step cannot be overemphasized. Clear, convenient, and informative formats should be devised in the planning stages and adhered to throughout the project.

Data analysis should be carefully defined before the project begins. Close attention must be paid to resource allocation to ensure that adequate resources are dedicated to verification, management, and analysis of data and to ongoing maintenance of monitoring equipment. As a quality control procedure and to make data analysis more manageable, these activities should be ongoing.

Start with a clear understanding of the decision or action that the project will inform. The goals and objectives statement determines the overall direction and scope of the data collection and analysis effort. The statement should also list questions to be answered by empirical data, noting the error or uncertainty associated with the desired result. Realistic assessment of error is needed because requiring too small an uncertainty leads to an overly complex and expensive project. It is important in monitoring projects that a data acquisition plan be developed and followed with a clear idea of the research questions to be answered.

Resource requirements for equipment, personnel, and other items must feed into budget estimates to determine expected funding needed for different project objectives. Scheduling requirements must also be considered, and project constraints defined and considered. Trade-offs on budget and objectives require that priorities be established.

Even if a project is not research-oriented, it is attempting to obtain information, which can be expressed in the form of questions. Research questions can have varying scopes and levels of detail, addressing entire systems or specific components. Examples of research questions include

Research questions vary widely in technical complexity, generally taking one of the following three forms:

The first form of question can sometimes be answered generically for a class of typical buildings without detailed monitoring and analysis, although planning and thorough analysis are still required. The second and third forms usually require detailed monitoring and analysis and, thus, detailed planning.

In general, more detailed and precise goal statements are better. They ensure that the project is constrained in scope and developed to meet specific accuracy and reliability requirements. Usually, projects attempt to answer more than one research question, and often consider both primary and secondary questions. All data collected should have a purpose of helping to answer a project question; the more specific the questions, the easier it becomes to identify required data.

Measured energy or water data will not be meaningful to people who were not involved in the project unless the characteristics of the building being monitored, and its use, have been documented. To meet this need, a data structure (e.g., a characteristic database) can be developed to describe the buildings.

Building characteristics can be collected at many levels of detail, depending on the type of monitoring project and the parameters that affect results. For projects that determine whole-building performance, it is important to provide at least enough detail to document the following:

The minimum level of detail is known as summary characteristics data. Simulation-level characteristics (detailed information collected for hourly simulation model input) may be desirable for some buildings. Regardless of the level of detail, the data should provide a context for analysts, who may not be familiar with the project, to understand the building and its energy use.

The objective of a monitoring project is not to merely produce data, but to answer a question. However, the data must be of high quality and must be presented to key decision makers and analysts in a convenient, informative format. The specific data products (scope, format, and content of data needed to meet project goals and objectives) must be identified and evaluated for feasibility and usefulness in answering project questions identified in Part One. Final data products must be clearly specified, together with the minimum acceptable data requirements for the project. It is important to clearly define an analysis path showing what will be calculated and what data are necessary to achieve desired results; assurance should be given that the resources needed to analyze the data, once collected, are available. Clear communication is critical to ensure that project requirements are satisfied and factors contributing to monitoring costs are understood.

Evaluation results can be presented in many forms, often as interim and final reports (possibly by heating and/or cooling season), technical notes, or technical papers. These documents must convey specific results of the field monitoring clearly and concisely. They should also contain estimates of the accuracy of the results.

The composition of data presentations and analysis summaries should be determined early to ensure that all critical parameters are identified (Hough et al. 1987). For instance, mock-ups of data tables, charts, and graphs can be used to identify requirements. Previously reported results can be used to provide examples of useful output. Data products should also be prioritized to accommodate possible cost trade-offs or revisions resulting from other steps in the process such as error analysis (see Part Seven).

Although requirements for the minimum acceptable data results can often be specified during planning, data analysis typically reveals further requirements. Thus, budget plans should include allowances and optional data product specifications to handle additional or unique project output requirements uncovered during data analysis.

Longer-term goals and future information needs should be anticipated and explained to project personnel. For example, a project may have short- and long-term data needs (e.g., demonstrating reductions in peak electrical demand versus demonstrating cost effectiveness or reliability to a target audience). Initial results on demand reduction may not be the ultimate goal, but rather a step toward later presentations on cost reductions achieved. Thus, it is prudent to consider long-term and potential future data needs so that additional supporting information, such as photographs or testimonials, may be identified and obtained.

A general monitoring design must be developed that defines three interacting factors: the number of buildings in the study, the monitoring approach (or experimental design), and the level of detail in the data being measured. A less detailed or precise approach can be considered if the number of buildings is increased, and vice versa. If the goal is related to a specific product, the monitoring design must isolate the effects of that product. Haberl et al. (1990) discuss monitoring designs. For example, for retrofit M&V, protocols have been written allowing a range of monitoring methods, from retrofit verification to retrofit isolation (ASHRAE Guideline 14-2014; FEMP 2015; IPMVP 2014). Some monitoring approaches are better suited than others to larger numbers of buildings.

Specifying the approach is particularly important because total building performance is a complex function of several variables, changes in which are difficult to monitor and to translate into performance. Unless care is taken with measurement organization and accuracy, uncertainties, errors (noise), and other variations (e.g., weather) can make it difficult to detect performance changes of less than 20% (Fracastoro and Lyberg 1983).

In some cases, judgment may be required in selecting the number of buildings involved in the project. If an owner seeks information about a particular building, the number of buildings in the experiment is fixed at one. However, for other monitoring applications, such as drawing conclusions about effects in a sample population of buildings, some selection is involved. Generally, error in the derived conclusions decreases as the square root of the number of buildings increases (Box 1978). A specific project may be directed at

For projects of the first type, accuracy requirements are usually resolved initially by determining expected variations of measured quantities (dependent variables) about their average values in response to expected variations of independent variables. For buildings, a typical concern is the response of heating and cooling loads to changes in temperature or other weather variables. The response of lighting energy use to daylighting is another example of the relationship between dependent and independent variables. Fluctuations in response are caused by (1) outside influences not quantified by measured energy use data and (2) limitations and uncertainties associated with measurement equipment and procedures. Thus, accuracy must often be determined using statistical methods to describe mean tendencies of dependent variables.

For projects of the second and third types, the increased number of buildings improves confidence in the mean tendencies of the dependent response(s) of interest. Larger sample sizes are also needed for experimental designs with control groups, which adjust for some outside influences. For more information, see Box (1978), Fracastoro and Lyberg (1983), and Hirst and Reed (1991). Projects can also use more complex, multilevel measurement and modeling approaches to handle an array of technologies or to improve confidence in results (Hughes and Shonder 1998).

Most monitoring procedures use one or more of the following general experimental approaches:

Before/After. Building, system, or component energy consumption is monitored before and after a new component or retrofit improvement is installed. Changes in factors not related to the retrofit, such as the weather and building operation, must be accounted for, often requiring a model-based analysis (Fels 1986; Hirst et al. 1983; Kissock et al. 1992; Robison and Lambert 1989; Sharp and MacDonald 1990). This experimental design is the primary concern of most current building energy monitoring documents (ASHRAE Guideline 14-2014; FEMP 2015; IPMVP 2014).

Test/Reference. The building energy consumption data of two “identical” buildings, one with the product or retrofit being investigated, are compared. Because buildings cannot be absolutely identical (e.g., different air leakage distributions, insulation effectiveness, temperature settings, and solar exposure), measurements should be taken before installation as well, to allow calibration. Once the product or retrofit is installed, any deviation from the calibration relationship can be attributed to the product or retrofit (Fracastoro and Lyberg 1983; Levins and Karnitz 1986).

On/Off. If the retrofit or product can be activated or deactivated at will, energy consumption can be measured in a number of repeated on/off cycles. On-period consumption is then compared to off-period consumption (Cohen et al. 1987; Woller 1989).

Simulated Occupancy. In some cases, the desire to reduce noise can lead the experimenter to postulate certain standard profiles for temperature set points, internal gains, moisture release, or window manipulation and to introduce this profile into the building by computer-controlled devices. The reference is often given by the test/reference design. In this case, both occupant and weather variations are held constant in the comparison (Levins and Karnitz 1986).

Nonexperimental Reference. A reference for assessing the performance of a building can be derived nonexperimentally using (1) a normalized, stratified performance database, such as energy use per unit area classified by building type (MacDonald and Wasserman 1989) or (2) a representative standard (peer) building, simulated by a calculated hourly or bin-method calibrated building energy performance model subject to the same weather, equipment type, and occupancy as the monitored building.

This design, also called calibrated simulation, is a secondary concern of current building energy monitoring documents (ASHRAE Guideline 14-2014; FEMP 2015; IPMVP 2014).

Engineering Field Test. When an experiment focuses on testing a particular piece of equipment, actual (in situ) performance in a building is often of interest. The building provides a realistic environment for testing the equipment for reliability, maintenance requirements, and comfort and noise levels, as well as energy or water usage. Because energy consumption of mechanical equipment is significantly affected by the system control strategy testing procedures should be designed to incorporate the control strategy of the equipment and its system (Phelan et al. 1997a, 1997b). This type of monitoring and testing can also be used to calibrate computer simulation models of as-built and as-operated buildings, which can then be used to evaluate whole-building energy consumption. The equipment may be extensively instrumented.

Some of the general advantages and disadvantages of these approaches are listed in Table 6 (Fracastoro and Lyberg 1983). Combining monitoring design choices can been successful (e.g., the before/after and test/reference approaches).

Questions to be considered in choosing a monitoring approach include the following:

Data are useless unless they are distilled into meaningful products that allow conclusions to be drawn. Too often, data are collected and never analyzed. This planning step focuses on specifying the minimum acceptable data analysis procedures and algorithms and detailing how collected data will be processed to produce desired data products. In this step, monitoring practitioners should

Determine proper National Institute of Standards and Technology (NIST)-traceable calibration standards for each sensor type to be used. For details, see the references in ASHRAE Guideline 14-2014 and IPMVP (2014) for specific types of sensors. However, it is often impractical to implement standards in the field. For example, maintaining the length of straight ductwork required for an airflow sensor is usually difficult, requiring compromise.

Algorithm inputs can be assumed values (e.g., energy value of a unit volume of natural gas), one-time measurements (e.g., leakage area of a house), or time-series measurements (e.g., fuel consumption and outdoor and indoor temperatures at the site). The algorithms may pertain to (1) utility level aggregates of buildings, (2) whole-building performance of particular buildings, or (3) performance of instrumented components.

Chapter 19 of the 2017 ASHRAE Handbook—Fundamentals contains a lengthy discussion on modeling procedures, and readers should consult this material for more information on modeling. In this chapter, the discussion is categorized differently, with a view toward procedures and issues related to field energy monitoring projects.

Table 7 provides a guide to selecting an analysis method. The error quotations are rough estimates for a single-building scenario.

Empirical Methods. Although analysis methods based on measured data are the simplest, they can have large uncertainty and may generate little or no information for small sample sizes. The simplest empirical methods are based on annual consumption values, tracking annual numbers and looking for degradation. Questions about building performance relative to other buildings are based on comparing certain performance indices between the building and an appropriate (peer) reference. The ENERGY STAR tools (www.energystar.gov) are probably the best-known performance comparison tools. The ASHRAE Building Energy Quotient (Building EQ) tool has also been developed to not just provide performance comparison (benchmarking), but also streamline the energy audit process to include improvement measures in the report generated from Building EQ.

For commercial buildings, the most common comparison index is the energy utilization index (EUI), which is annual consumption, either by fuel type or summed over all fuel types, divided by the gross floor area (see MacDonald and Wasserman [1989] for a discussion of indices). Comparison is often made only on the basis of general building type, which can ignore potentially large variations in how much floor area is heated or cooled, climate, number of workers in a building, number and type of computers in a building, and HVAC systems. Variations can be accommodated somewhat by stratifying the database from which the reference EUI is chosen. Computer simulations are often used to set reasonable comparison values.

The Commercial Buildings Energy Consumption Survey (CBECS) database (summarized in Chapter 37) has been used to develop ENERGY STAR and Building EQ energy use benchmarking methods for several building types in the United States. Many of the building tools use 2003 data and are updating to the 2012 CBECS survey data Building types covered as of June 2009 include

For some of these building types, some secondary spaces are allowed:

Initial work in this area covered only electricity use for office buildings (Sharp 1996), but the methods have been extended to cover all fuels for the listed building types and water. Results show that electricity use of office buildings is most significantly explained by the number of workers in the building, number of computers, whether the building is owner-occupied, and number of operating hours each week. Only a subset of these parameters might be used to determine a benchmark within a specific census division. ENERGY STAR documentation on current tools and the performance normalization factors for a range of building types can be found at www.energystar.gov/buildings.

Simple empirical methods applied to retrofit applications should include at least some periods of data on daily energy or water use and average daily temperature (recorded locally) to account for variations in occupancy and building schedules. Monthly EUI or billing data provide more information for empirical analysis and can be used for extended analysis of energy impacts of retrofit applications, for example, in conditional demand analysis (Hirst and Reed 1991). Monthly data can also be used to detect billing errors, improper equipment operation during unoccupied hours, and seasonal space condition problems (Haberl and Komor 1990a, 1990b). Daily data are often used in these analyses, and raw hourly total building consumption data, when available, provide more detailed information on occupied versus unoccupied performance. Hourly, daily, monthly, and annual EUI across buildings can be directly compared when reduced to average power per unit area (power density). To avoid false correlations, the method of analysis should have statistical significance that can be traced to realistic parameters (Haberl et al. 1996). ASHRAE (2012) gives best-practice protocols for measurements at different time periods.

Model-Based Methods. These techniques allow a wide range of additional data normalization to potentially improve the accuracy of comparisons and provide estimates of cause-and-effect relations. The analyst must carefully define the system and postulate a useful form of the governing energy balance/system performance equation or system of equations, as often embodied in hour-by-hour simulation programs. Explicit terms are retained for equipment or processes of particular interest. As part of the data analysis, whole-building data (driving forces and thermal or energy response) are used to determine the model’s significant parameters. The parameters themselves can provide insight, although parameter interpretation can be difficult, particularly with time-integrated billing data methods. The model can then be used for normalization as well as future diagnostic and control applications. Two general classes of models are used in analysis methods: time-integrated methods and dynamic techniques (Balcomb et al. 1993). Simulation modeling results used for design of new buildings may not cover all energy used in a building and thus could be difficult to compare with empirical data.

Time-Integrated Methods. Based on algebraic calculation of the building energy balance, time-integrated methods are often used before data comparison to correct annual consumption for variations in outdoor temperature, internal gains, and internal temperature (ASHRAE 2002; Fels 1986; Haberl and Claridge 1987). This type of calibrated model is essential for most retrofit applications.

Time-integrated methods can be used with whole-building energy consumption data (billing data) or with submetered end-use data. For example, standard time-integrated methods are often used to separately analyze end-use consumption data on heating, cooling (Ternes and Wilkes 1993), domestic water heating, and others for comparison and analysis. Time-integrated methods are generally reliable, as long as the following three conditions are accounted for:

For whole-building energy consumption data (billing data), reasonable results can be expected from heating analysis models when the building is dominantly responsive to indoor/outdoor temperature differences. Billing data analysis yields little of interest when internal gains are large compared to envelope loads, as in large commercial buildings and industrial applications. Daily, weekly, and monthly whole-building heating season consumption integration steps can been used (Claridge et al. 1991; Fels 1986; Sharp and MacDonald 1990; Ternes 1986). Cooling analysis results have been less reliable because cooling load is not strictly proportional to variable-base cooling degree-days (Fels 1986; Haberl et al. 1996; Kissock et al. 1992). Problems also arise when solar gains are dominant and vary by season.

Dynamic Techniques. Dynamic models, both macrodynamic (whole-building) or microdynamic (component-specific), offer great promise for reducing monitoring duration and increasing conclusion accuracy. Furthermore, individual effects from multiple measures and system interactions can be examined explicitly. Dynamic whole-building analysis is generally accompanied by detailed instrumentation of specific technologies.

Dynamic techniques create a dynamic physical model for the building, adjusting model parameters to fit experimental data (Duffy et al. 1988; Subbarao 1988). In residential applications, computer-controlled electric heaters can be used to maintain a steady interior temperature overnight, extracting from these data an experimental value for the building steady-state load coefficient. A cooldown period can also be used to extract information on internal building mass thermal storage. Daytime data can be used to renormalize the building response (computed from a microdynamic model) to solar radiation, which is particularly appropriate for buildings with glazing areas over 10% of the building floor area (Subbarao et al. 1986). Once the data with electric heaters have been taken, the building can be used as a dynamic calorimeter to assess the performance of auxiliary heating and cooling systems.

Similar techniques have been applied to commercial buildings (Burch et al. 1990; Norford et al. 1985). In these cases, delivered energy from the HVAC system must be monitored directly in lieu of using electric heaters. Because ventilation is a major variable in the building energy balance, outdoor airflow rate should also be monitored directly. Simultaneous heating and cooling (common in large buildings) requires a multizone treatment, which has not been adequately tested in any of the dynamic techniques.

Equipment-specific monitoring guidelines using dynamic modeling have been successfully tested in a variety of applications. For fans and pumps, relatively simple regression techniques from short-term monitoring provided accurate estimates of annual energy consumption when combined with an annual equipment load profile. For chillers, a thermodynamic model used with short-term monitoring captured the most important operating parameters for estimating annual chiller energy performance. In all cases, the key to accurate model results was capturing a wide enough range of the independent load variable in monitored data to reflect annual operating characteristics (Phelan et al. 1997a, 1997b).

Careful specification of field monitoring points is critical to identifying variables that need to be measured to produce required data. The analysis method determines the data to be measured. The simplest methods require no on-site instrumentation. As methods become more complex, data channels increase. For engineering field tests conducted with dynamic techniques, up to 100 data channels may be required.

Because metering projects are often conducted in buildings with changing conditions, special consideration must be given to identifying and monitoring significant changes in climate, systems, and operation during the monitoring period. Additional monitoring points may be required to measure variables that are assumed to be constant, insignificant, or related to other measured variables to draw sound conclusions from the measurements. Because the necessary data may be obtained in several ways, data analysts, equipment installers, and data acquisition system engineers should work together to develop tactics that best suit the project requirements. It is important to anticipate the need for supplemental measurements in response to project needs that may not become apparent until equipment is installed.

The cost of data collection is a nonlinear function of the number, accuracy, and duration of measurements considered while planning within budget constraints. Costs per data point typically decrease as the number of points increases, but increase with accuracy requirements. Duration of monitoring can have many different effects. If the extent of data applications is unknown, such as in research projects, the value of other concurrent measurements should be considered, because the incremental cost of additional analyses may be small.

For any project involving large amounts of data, data quality verification (see Part Eight) should be automated (Lopez and Haberl 1992). Although this may require adding monitoring points to facilitate energy balances or redundancy checks, the added costs are likely to be offset by savings in data verification.

If multiple sites are to be monitored, common protocols for selecting and describing all field monitoring points should be established so data can be more readily verified, normalized, compared, and averaged. Protocols, such as those recommended in ASHRAE (2010), also add consistency in selecting monitoring points. Pilot installations should be conducted to provide data for a test of the system and to ensure that the necessary data points have been properly specified and described.

Monitoring Equipment. General considerations in selecting monitoring equipment include

Using a building energy management system or direct digital control systems for data acquisition may decrease costs. This should be considered only when the sensors and their accuracy and limitations (scan rate, etc.) are thoroughly understood. When merging data from two data acquisition systems, problems may arise such as differing reliability and low data resolution (e.g., 1 kW resolution of a circuit that draws 10 kW fully loaded). These problems can often be avoided, however, by adding appropriate sensors and setting up custom logging or calculations with point, memory, and programming capacity.

Once the required field data monitoring points are specified, these requirements should be clearly communicated to all members of the project team to ensure that the actual monitoring points are accurately described. This can be accomplished by publishing handbooks for measurement plan development and equipment installation and by outlining procedures for diagnostic tests and technology assessments.

Because hardware needs vary considerably by project, specific selection guidelines are not provided here. However, general characteristics of data acquisition hardware components are shown in Table 8. Some typical concerns for selecting data acquisition hardware are outlined in Table 9. In general, data logger and instrumentation hardware should be standardized, with replacements available in the event of failure. Also consider redundant measurements for critical data components that are likely to fail, such as modems, flowmeters, shunt resistors on current transformers, and devices with moving parts (O’Neal et al. 1993). Some measurements that are more difficult to obtain accurately because of instrumentation limitations are summarized in Table 10.

Safety must be considered in equipment selection and installation. Installation teams of two or more individuals reduce risks. When contemplating thermal metering, the presence of asbestos insulation on water piping should be determined. Properly licensed trades personnel, such as an electrician or welder, should be a fundamental part of any team installing electrical monitoring equipment.

To prevent inadvertent tampering, occupants and maintenance personnel should be carefully briefed on what is being done and the purpose of sensors and equipment. Data loggers should have a dedicated (non-occupant-switchable), hard-wired power supply to prevent accidental power loss.

Sensors. Sensors should be selected to obtain each measurement on the field data list. Next, conversion and proportion constants should be specified for each sensor type, and the accuracy, resolution, and repeatability of each sensor should be noted. Sensors should be calibrated before they are installed, preferably with a NIST-traceable calibration procedure. They should be checked periodically for drift, recalibrated, and then post-calibrated at the conclusion of the experiment (Haberl et al. 1996). Instrument calibration is particularly important for flow and power measurement.

Particular attention must be paid to sensor location. For example, if the method requires an average indoor temperature, examine the potential for internal temperature variation; data from several temperature sensors must often be averaged. Alternatively, temperature sensors adjacent to HVAC thermostats detect the temperature to which the HVAC equipment reacts. For temperature or pitot tube flow measurements the location must be sufficiently downstream of louvers, filters, and pipe turns to obtain accurate results.

Scanning and Recording Intervals. Measurement frequency and data storage can affect the accuracy of results. Scanning differs from storage in that data channels may be read (scanned) many times per second, for example, whereas average data may be recorded and stored every 15 min. Most data loggers maintain temporary storage registers, accumulating an integrated average of channel readings from each scan. The average is then recorded at the specified interval.

After the channel list is compiled and sensor accuracy requirements established, scan rates should be assigned. Some sensors, such as indoor and outdoor temperature sensors, may require low scan rates (once every 5 min). Others, such as total electric sensors, may contain high-frequency transients that require rapid sampling (many times per second). The scan rate must be fast enough to ensure that all significant effects are monitored.

The maximum sampling rate is usually programmed into the logger, and averages are stored at a specified time step (e.g., hourly). Some loggers can scan different channels at different rates. The logger’s interrupt capability can also be used for rapid, infrequent transients. Interrupt channels signal the data logger to start monitoring an event only once it begins. In some cases, online computation of derived quantities must be considered. For example, if heating or cooling flow in an air duct is required, it can be computed from a differential temperature measurement multiplied by an air mass flow rate determined from a one-time measurement. However, it should be computed and totaled only when the fan is operating.

Data collected by monitoring equipment are usually used for the following purposes:

In all three cases, the value of the measurements is dramatically increased if the associated uncertainty can be quantified.

Basic Concepts. Uncertainty can be better understood in terms of confidence limits, which define the range of values that can be expected to include the true value with a stated probability (ASHRAE Guideline 2). Thus, a statement that the 95% confidence limits are 5.1 to 8.2 implies that the true value is between 5.1 and 8.2 in 19 out of 20 observations or, more loosely, that we are 95% confident that the true value lies between 5.1 and 8.2. For a given set of n observations with normal (Gaussian) error distribution, the total variance about the mean predicted value X′ provides a direct indication of the confidence limits. Thus the “true” mean value X′ of the random variable is bounded as follows:

(1)

where

The terms accuracy and precision are often used to distinguish between bias errors and random errors. A set of measurements with small bias errors is said to have high accuracy, and a set of measurements with small random errors is said to have high precision. In repeated measurements of a given sample by the same technique (single-sample data), each measurement has the same bias. Bias errors include those that are (1) known and can be removed by calibration, (2) negligible and are ignored, and (3) estimated and are included in the uncertainty analysis. It is usually difficult to estimate bias limits, and this effect is often overlooked. However, a proper error analysis should include bias error, which is usually written as a plus-minus error. ASME Standard PTC 19.1 has a more complete discussion.

Because bias errors b_m and random errors ε_m are usually uncorrelated, measurement variance σ² can be expressed as(b_m, ε_m) = σ²(b_m) + σ²(ε_m)

(2)

For further information on uncertainty, see Chapter 37 of the 2017 ASHRAE Handbook─Fundamentals. For more information on uncertainty calculation methods related specifically to building energy savings monitoring, see ASHRAE Guideline 14-2014, Annex B.

Primary Measurement Uncertainty. Sensor and measuring equipment manufacturers usually specify measurement variances; frequent recalibration minimizes bias errors. As indicated by Equation (1), increasing the number of measurements n reduces the uncertainty bounds.

Uncertainty in Derived Quantities. Once a specific algorithm or equation for obtaining final data from physical measurements has been established, standard techniques can be used to incorporate primary measurement uncertainties into the final data product. For random errors, the Kline and McClintock (1953) error propagation method, based on a first-order Taylor series expansion, is widely used to determine measurement uncertainties in derived variables in single-sample experiments. Bias errors are difficult to account for; the usual practice is to remove them through calibration and exclude them from the uncertainty analysis.

Uncertainty in Statistical Regression Models. Statistical regression models developed from measured data are usually used for predictive purposes. Measurement errors are much smaller than model errors, which arise because the regression model is imperfect; that is, it is unable to explain the entire variation in the regression variable (Box 1978). Measurement error is inherently included in the identified model, so total prediction variance is simply given by the model prediction uncertainty.

Determining prediction errors from regression models is subject to different types of problems. The sources of error can be classified into three categories (Reddy et al. 1998):

Misspecification and extrapolation errors are likely to introduce bias and random error. If ordinary least-squares regression is used for parameter estimation, and if the model is subsequently used for prediction, model prediction will be purely random. Thus, models identified from short data sets and used to predict seasonal or annual energy use are affected by misspecification and extrapolation errors.

The least-squares method of calculating linear regression coefficients cannot produce unbiased estimators of slope and intercept if there are errors associated with measuring the predictor variable. The uncertainty analysis methodology developed for in situ equipment testing uses standard linear regression practices to find the functional relationship and then estimates the increased uncertainty in the regression prediction because of random and bias errors in both variable measurements (Phelan et al. 1996).

Experimental Design. Errors can also be estimated based on historical experience (e.g., using results from previous similar projects). Alternatively, a pilot study can obtain an estimate of potential errors in a proposed analysis. Some estimate of potential error must be available to determine whether project goals and objectives are reasonable.

Estimating data uncertainty is one part of the iterative procedure associated with proper experimental design. If the final data product uncertainty determined using the given evaluation procedure is unacceptable, uncertainty can be reduced in one or more of the following ways:

On the other hand, if simulations or estimates indicate that the expected bias in the final data product is unacceptable, the bias may be reduced by one or more of the following steps:

Accuracy Versus Cost. The need for accuracy must be carefully balanced against measurement and analysis costs. Accuracy loss can stem from instrumentation or measurement error, normalization or model error, sampling or statistical error, and errors in assumptions. Each of these sources can be controlled to varying degrees. However, in general, more accurate methods follow the law of diminishing returns, in which further reductions in error come at progressively greater expense.

Because of this trade-off, the optimal measurement solution is usually found by an iterative approach, where incremental improvements in accuracy are assessed relative to the increase in measurement cost. Such optimization requires that a value be placed on increasing levels of accuracy. One method of evaluating the uncertainty of a proposed method is to calculate results using the highest and lowest values in the confidence interval. The difference between these values can be translated into a monetary amount that is at risk. The question that must be answered is whether further measurement investment is warranted to reduce this risk.

Establishing and using data quality assurance (QA) procedures can be very important to the success of a field monitoring project. The amount and importance of the data to be collected help determine the extent and formality of QA procedures. For most projects, the entire data path, from sensor installation to procedures that generate results for the final report, should be considered for verification tests. In addition, the data flow path should be checked routinely for failure of sensors or test equipment, as well as unexpected or unauthorized modifications to equipment.

QA often requires complex data handling. Building energy monitoring projects collect data from sensors and manipulate those data into results. Data handling in a project with only a few sensors and required readings can consist of a relatively simple data flow on paper. Computers, which are generally used in one or more stages of the process, require a different level of process documentation because much of what occurs has no direct paper trail.

Computers facilitate collection of large data sets and increase project complexity. To maximize automation, computers require development of specific software. Often, separate computers are involved in each step, so passing information from one computer to another must be automated in large projects. To move data as smoothly as possible, an automated data pipeline should be developed; this minimizes the delay from data collection to results production and maximizes the cost-effectiveness of the entire project.

Because collected data are valuable, data back-up procedures must also be part of QA. At a minimum, the basic data, either raw or first-level processed, should be stored in at least one, and preferably two, different back-up locations, apart from the main data storage location, to allow data recovery in case of hardware failure, fire, vandalism, etc. Back-ups should occur at regular intervals, probably not less than weekly for larger projects.

Frequent data acquisition (preferably automated), with a quality control review of summarized and plotted data, is essential to ensure that reliable data are collected. Verification procedures should be performed at frequent intervals (daily or weekly), depending on the importance of missing data. This minimizes data loss due to equipment failure and/or changes at a building site. It also allows processed information to be applied quickly.

The following QA actions should take place:

Three aspects of a monitoring project that require QA are shown in Table 11: hardware, engineering data, and characteristics data. Three QA reviews are necessary for each aspect: (1) initial QA confirms that the project starts correctly; (2) ongoing QA confirms that information collected by the project continues to satisfy quality requirements; and (3) periodic QA involves additional checks, established at the beginning of the project, to ensure acceptable performance quality.

Information about data quality and the QA process should be readily available to data users. Otherwise, significant analytical resources may be expended to determine data quality, or the analyses may never be performed because of uncertainties.

This step specifies the formats in which data are supplied to the end user or other data analysts. Both raw and processed (adjusted for missing data or anomalous readings) data formats should be specified. In addition, if supplemental analyses are planned, the medium and format to be used (type of storage, possibly magnetic tape type, spreadsheet, character encoding standard) should be specified. These requirements can be determined by analyzing the software data format specifications. Common formats for raw data are comma- and blank-delimited American Standard Code for Information Exchange (ASCII), which do not require data conversion.

Data documentation is essential for all monitoring projects, especially when several organizations are involved. Data usability is improved by specifying and adhering to data recording and exchange formats. Most data transfer problems are related to inadequate documentation. Other problems include hardware or software incompatibility, errors in electronic storage media, errors or inconsistencies in the data, and transmittal of the wrong data set. The following precautions can prevent some of these problems: