Learning from failure: understanding the anticipated–achieved building energy performance gap

Over the past 20 years a number of studies have identified and provided explanations for a significant ‘performance gap' between designed and actual energy performance of buildings. The anticipated and achieved energy performance of an advanced, innovative building that aspired to net-positive energy performance is studied: the Centre for Interactive Research on Sustainability (CIRS) building at the University of British Columbia in Vancouver, Canada. Selected performance ‘failures’ that became evident during operation of CIRS are studied for how they were discovered and the efforts required for their resolution: the energy systems and associated controls and monitoring. The key findings show the barriers were neither economic nor technical. Instead, the primary impediments were institutional regimes – arising from the ways that various life-cycle stages were specified, contracted and implemented. The key issues emphasize the importance of having meaningful and effective building energy monitoring capabilities, an understanding of energy system boundaries in design and analysis, crossing the gaps between different stages of a building life cycle, and feedback processes throughout design and operation. The disclosure of ‘failure’ and lessons learned is a valuable contribution to subsequent advancement for the building stakeholders and the wider professional and research communities.


Introduction
Although almost every building can be considered a discrete undertaking -shaped by different owner/ user goals and requirements, budget, experience of the design team, physical context, and by norms and conventions that are themselves culturally bound -performance goals and approaches to design can be, and typically are, more generally prescribed. During periods where technologies of building construction are considered to be mature, and design criteria have been evaluated and specified over several decades, one can reasonably expect a relatively close match between the specification of design and performance criteria by building owners and operators, their subsequent interpretation and implementation by the design and construction team and, hopefully, the performance of the eventual building.
The overlaying of new, more demanding sustainability considerations on the building design and construction process brings the inevitable difficulties of reassessing priorities, acquiring new skills, and developing and integrating new information into an existing project delivery process. New, more demanding performance-based criteria also carry with them the uncertainty of their acceptance or successful outcome (Butera, 2013). Such uncertainty is particularly the case when performance criteria are measured through predictive modelling techniques, with limited feedback of verified performance. It is perhaps not surprising, therefore, that there has been increasing concern and controversy surrounding the actual performance of 'green' or 'sustainable' buildings. Indeed, a number of studies over the past 20 years have identified and provided explanations for a significant 'performance gap' between designed and actual energy performance of buildings (Arup, 2013;Bordass, Cohen, & Field, 2004;Butera, 2013;Gann, Salter, & Whyte, 2003;Newsham, Mancini, & Birt, 2009;Norford, Socolow, Hsieh, & Spadaro, 1994;NYC, 2012;Schofield, 2013;Stafford, Bell, & Gorse, 2012;Turner & Frankel, 2008). Such performance gaps are of considerable interest for a number of reasons including: building owners and tenants potentially seeking redress for buildings that are costing more to operate than anticipated; regulatory requirements to reduce carbon emissions or meet netzero energy performance targets and the credibility and market appeal of high-performance buildings.
The building industry is a fragmented sector -it involves a large number of actors. The diversity of backgrounds, knowledge, position, objectives and interests means that those involved often have differing perspectives on the goals that need to be achieved and how to achieve them (Van Bueren, 2009). More effective communication between key stakeholders associated with the design, construction and operation of buildings is considered key to achieving successful performance and establishing approaches to do so has been a continual area of study (Butera, 2013;Fedoruk, 2013;Newsham et al., 2009;. Dammann and Elle (2006), for example, investigated if, and to what extent, consensus on environmental indicators could form a 'common language for green building'. Their work identified two of the key areas of conflict in preferred indicators and tools to be 'simplicity versus complexity' and 'checklist indicators versus life cycle assessments'. The authors conclude that closure of the debate 'on the basis of an all-actors consensus' is unlikely in the near future (Dammann & Elle, 2006). In order to resolve some of this fragmentation of the building sector, case study evidence suggests there is a need to reframe the way building projects are conceptualized and implemented (Kaatz, Root, & Bowen, 2005;Loftness et al., 2009). This will require fundamental changes to the processes and communication channels of a project, moving from a linear project conceptualization to one that looks at the whole building life cycle. Beyond technical considerations or simply injecting new information, a rethink is required of how buildings are planned, designed, constructed, commissioned and operated in order to close the performance gap. Bordass et al. (2004) reference the multi-stakeholder nature of the performance gap and suggest that to achieve genuine step-change improvements, procuring clients, design and building teams, users and managers will all need to engage much more closely with achieved performance.
(p. 1) Moreover, 'credibility gaps' between performance expectations of energy performance and actual fuel consumption outcomes, they suggest, arise not so much because predictive techniques are 'wrong', but because the assumptions often used are not well enough informed by what really happens in practice, because few people who design buildings go on to monitor their performance.
(p. 1) The value of monitoring raised by Bordass et al. (2004) is a critical aspect of the work presented in this paper. With respect to these 'credibility gaps', this paper uses a high-profile case study to elucidate some of the opportunities to close the gap between predicted and actual performance. It also examines the cultural and institutional barriers that that can significantly limit closing the performance gap.

Moving beyond green
Building environmental assessment systems such as the US Green Building Council's Leadership in Energy and Environmental Design (LEED) have provided a common framework for green design and a common language, and set of performance requirements and metrics that are increasingly utilized by key stakeholders. For example, a study of 446 post-secondary buildings in the US that targeted LEED certification showed a steady increase in points achievement over time, allowing the study authors to conclude that indeed a market transformation and understanding of how to achieve the specified checklists of green building practices had emerged (Chance, 2012). Today, many North American architectural practices have accumulated experience in green design and, indeed, are consistently producing buildings achieving LEED 'Platinum' -the highest ranking possible under the LEED green building certification process. This maturing of green building practice has meant that leadingedge 'green' practitioners and building owners who have operated at this level are increasingly seeking to push further than the performance aspirations embedded in current assessment methods (Cole, 2012). Indeed, increasingly reference is made to aspirations that call for performance of 'Beyond LEED' and 'net positive' design. As these demanding performance expectations become reality, this will further challenge current design skills and operational capabilities, based as they are on a previous era of established design norms and their specification, interpretation and implementation (Mang & Reed, 2012;McDonough & Braungart, 2002;Moore, Gelfand, & Whitsett, 2015;Reed, 2007). Bunn (2008), in characterizing the innovative Council House 2 (CH2) building in Melbourne, Victoria, Australia, raises the issue that 'some of building innovation is running ahead of institutional norms and would need to change the expectation of the occupants' and that while the CH2 building enjoys public renown, it is 'overcomplicated and finicky, and very demanding of highly skilled premises management ' (p. 42). In instances where innovation outpaces the knowledge base of industry, setting objective benchmarks and seeking feedback mechanisms such as metering and monitoring protocols becomes particularly important. Innovation in both engineering and management needs to occur across the entire building life cycle in order to advance sustainable building design and construction (Chen, Clements-Croome, Hong, Li, & Xu, 2006).
This paper is concerned with the anticipated and achieved energy performance of the Centre for Interactive Research on Sustainability (CIRS) building at the University of British Columbia (UBC), in Vancouver, Canada. It examines some of the performance 'failures' that became evident after the completion and a subsequent period of operation of CIRS, how these failures were discovered and the efforts required to resolve them. Although performance problems occurred across a number of resource flows -energy, carbon, water and waste-water treatment -the paper focuses on examining the energy systems and associated controls and monitoring. The key findings emphasize the importance of having meaningful and effective building energy monitoring capabilities, an understanding of energy system boundaries in design and analysis, crossing the gaps between different stages of a building life cycle, and feedback processes throughout design and operation.

CIRS building
CIRS is a 5675 m 2 building on the UBC campus that was designed to operate at the frontier of sustainable performance in both environmental and human terms, and to serve as a living laboratory of, and research test-bed for, net-positive performance, and sustainable urban development practices over its lifetime (Robinson, Cole, Kingstone, & Cayuela, 2013).
In the context of design for this building, 'net-positive energy' was defined in the following way: adding the CIRS building to the UBC campus should reduce the amount of energy used and carbon emitted at the campus scale. The CIRS building, which was occupied in September 2011, was also designed to sequester more carbon in its structure than required to construct the building, and reduce the campus demand for potable water. 1 This is intended to be supportive of UBC's long-term goal of showcasing the Vancouver campus as 'the world's first net positive energy and water campus' (UBC, 2009).
The CIRS building was intended to show how the use of aspirational design strategies could result in net benefits in human and environmental terms both within and beyond the building boundary, considering the building not as a stand-alone, isolated project, but as an integral part of the campus-wide system and infrastructure. CIRS is part of a campus-level planning approach that is attempting to find design strategies for sustainable energy, water, waste and food systems through practical implementation and learning. 2 The broad range of performance aspirations and goals for the CIRS building have been described elsewhere (Robinson et al., 2013) and only those related to its energy and control systems are presented here. 3

CIRS energy systems
The design of the CIRS building incorporated a physical connection for waste-heat exchange with the adjacent Earth and Ocean Sciences (EOS) laboratory building. This connection to EOS became one of the main points of physical systems integration of the project as well as a symbol for campus-scale integrations in a broader sense. Such inter-building energy exchange as a design basis is not typical of building projects, which tend only to accommodate design aspects within the boundary of the building. It brings an element of multi-energy system optimization to building design, which can cause substantial integration issues (Fabrizio, Corrado, & Filippi, 2010). High-level design goals, which provided some of the key guiding principles of the project relating to energy performance of CIRS, included: . minimizing the energy needs for building operation . matching the quality of energy with its use . installation of controls and monitoring systems to minimize total building energy consumption, 4 thereby ensuring the building works by itself and that it responds actively and autonomously to environmental stimuli . enabling a net effect of reducing campus energy use through renewable technologies and wasteheat harvesting 5 The main energy source for the CIRS building is electricity. Electrical systems in the building include the photovoltaic cells embedded in shading devices and skylight panels, heating ventilation and the limited cooling equipment, electric lighting, associated metering and controls, and laboratory equipment and computer systems included in building plug-loads. Metering of the electrical system is accomplished with a utility-grade electricity meter owned by the utility provider, BC Hydro, as well as by the more than 30 electrical sub-meters installed in the building monitored by the building management system (BMS). These individual electrical sub-meters monitor the electrical use of each electrical panel as well as the electrical power at each transformer. Electrical panels were meant to be fully segregated by end use, such as lighting or heating, ventilation and cooling equipment, so that measuring the connected loads at the panel would accurately measure those end uses.
The design and performance of the building envelope enabled the space heating systems in CIRS to be low temperature (based on convective heat perimeter hydronic radiators and radiant floors) which is compatible with the low-grade heat sources available to the building including waste heat from the nearby EOS laboratory building, local heat recovery from CIRS ventilation exhaust, and an on-site geo-exchange system. Solar hot collectors for domestic water pre-heating and a backup electric hot-water boiler are also part of the CIRS thermal system. All thermal energy sources and sinks require electrical energy to run various associated pumps and fans that transport the heat, as well as to operate the heat pumps that upgrade the harvested waste-heat energy so that it can be used in the building.

Methods
This research was conducted by performing an analysis of both the design process and the operational performance of the CIRS building at UBC. The system design process was examined through analysis of major design and contract documentation for the building, including minutes from design meetings such as facilitated design charettes. These meeting minutes were coded based on whom was speaking, what type of information was being shared, whether that information brought about a new path forward for the design or was a source of conflict, and what type of information followed information that brought about conflict.
The system operation was examined through on-site physical assessment of building systems, through analysis of data recorded by the CIRS BMS, and through an online survey of inhabitants and operators experience with and perception of the building systems. This paper explores the energy performance of the CIRS building at approximately one year post-occupancy (April 2012-March 2013). Through the process of evaluating energy performance from the BMS system, issues were uncovered within monthly data sets that resulted in a physical inspection of the rooftop integration between CIRS and EOS. This examination was conducted in February 2013 by one of the design team leaders, in accompaniment with the building operations leader and the lead author of this paper. The CIRS BMS system logs data from the building's over 3000 monitoring points. In order to evaluate the performance of the building systems relative to their design intent, the BMS database was queried through a Structured Query Language (SQL) interface for data on electrical and heat systems at hourly and daily resolutions. Specifically, energy meters meant to measure the thermal or electrical transfer of energy were queried as to their current reading, as well as sub-points of temperature and flow rate so that comparisons could be made. Any data deemed to be erroneous, such as those data points that read zero or negative numbers or were read as an error, were removed and replaced with interpolated data. Also those data points that produced values outside of reasonable system functioning, such as the building using more water in one hour than could flow through its pipes, was replaced with interpolated data from the nearest clean data sets. A time period of data was observed such that it was possible to obtain a data set where there was an overall energy use difference of less than 2% between the utility meter readings and the calculated data set for the overall energy use of the building. For some sub-systems metering installation and commissioning issues were such that a highquality data set could not be gathered, and these systems have been left out of the quantitative analysis. Twelve months of detailed hourly data from these electrical and thermal systems and subcomponents were used to produce the results found in this paper.
In addition to the use of BMS data to evaluate energy and controls at CIRS, an online survey was conducted with building inhabitants and operators during the summer of 2013. This survey gauged satisfaction and understanding of building systems and controls on a six-point scale, and invited longer narrative answers to explain responses. Forty-five respondents took the inhabitant survey, a statistical analysis of which is provided in the Appendix (which can be found in the Supplemental data online). The BMS data-collection and survey method enabled the detailed study of a highperformance building during the early stages of postconstruction operation, using both qualitative data such as long-form discursive answers to the survey and experience with the system through the data collection process and quantitative data such as data readings, number of data points reading correctly and survey results to inform conclusions. It is, however, limited by its specific focus on one case study and relatively short time period of analysis as compared with the full design cycle and future operational lifetime of the building. Without a data set related to a larger number of buildings, conclusions from the case study are potentially indicative of other building case studies but not conclusive.

Anticipated and actual energy performance
Anticipated performance During design, it was estimated that CIRS would use 585 MWh of electricity yearly, be able to harvest 306 MWh of waste heat from the EOS fume hood exhaust manifold for its own use, and be able to Learning from failure transfer an additional 600 MWh of waste heat from the fume hood exhaust into the EOS air-handling roof-top units to preheat air before it is supplied into the building. This would therefore reduce the EOS building heating demand by 600 MWh of steam, which, in turn, was estimated to correspond to a reduction of 860 MWh of natural gas use at the UBC steam plant (UBC has a district energy system with natural gas-fired boilers that produce medium-pressure steam to heat buildings on campus). This understanding was presented by the design team to the owners and operational team at UBC during the design process and became the basis of understanding by the university. These goals and energy targets were shared with an audience that extended beyond those involved in initial design charettes through presentations to the university and community. This was not a typical design process for UBC, as neither the collaborative process with university oversight nor the ambitious design goals were standard. Local greenhouse gas emission factors, as used by the university for carbon reporting purposes, for natural gas and electricity at the time of calculating design assumptions were 0.19 and 0.025 tonnes CO 2 equivalent per MWh eq , respectively. 6 This would mean that adding CIRS to the UBC campus would increase campus electricity use by 585 MWh and reduce campus natural gas use by 860 MWh, making CIRS 'net positive' in energy use. 7 These estimates were not contractual. These figures were estimated from simulation done during the design process and shared with the university and design team. Based on a net-positive design aspiration, the connection at EOS is therefore an integral component of the system, allowing the design to meet its intent. The 'performance goal', as the university understood it, was to target a energy-use intensity (EUI) of 70 kWh/m 2 /year, not including the reduction of energy at the nearby building, as per the energy model documentation for LEED compliance. In addition to performance goals for energy flows, CIRS also had goals related to the operational monitoring and optimization of building systems. The overarching goals for the project were intended to be broad, and included the installation of controls and monitoring systems to minimize total building energy consumption, thereby ensuring the building works by itself and that it responds actively and autonomously to environmental stimuli. Figure 1 and Table 1 show the anticipated annual energy flows for CIRS, based on design modelling and estimated performance.

Actual performance
In spite of a series of measures put in place by the CIRS design and operation team, such as a measurement and verification (M&V) plan and reporting process; comprehensive performance tracking and reporting; and a continuous optimization process, the measured energy totals for the building were found to be incongruent with both the anticipated design and a more comprehensive analysis of the data. For example, energy totals were significantly less than the predicted values for heat exchange between the two buildings.  buildings. 9 A negligible amount of heat was returned to the EOS preheat connection, resulting in no reduction of natural gas used by EOS. Given that the design of CIRS relies heavily on the connection at EOS to achieve its energy goals, it is clear that operation of the building in 2012 -13 did not meet the design intent.
The implementation of the CIRS controls and monitoring systems was also not consistent with the design intent, with the system being ineffective for building troubleshooting and system optimization as deemed by the building operational team. Issues included lack of metering for key systems such as the solar collectors or plug-load energy use of CIRS and the connected air and heating systems or energy uses at EOS. This lack of metering at key points meant that the overall energy use of the entire system -inclusive of EOScould not be properly verified or optimized. In addition, cumbersome data access portals and representations of data sets that were difficult to interpret prevented the metering and monitoring system from achieving its design intent. Figure 2 and Table 2 show the energy flows for CIRS for the 2012 -13 period described above. Figure 2 shows a combination of actual values as discussed above, and extrapolated values based on the best data available from this period. 10 Some of the sub-metered data are shown in subsequent figures, including heat transfer and electricity use (Figure 3).
Interestingly, even the above calculated heat transferred from CIRS to EOS had to be further distilled, looking at the heat flows on the EOS side of the heat pump in order to understand when heat was truly being accepted by EOS. This heat acceptance turned out to only occur one month of the year-long period under investigation, in December 2012, when approximately 1 MWh of heating was accepted by the makeup air (MUA) unit of EOS. During other months of operation, it was found that heat was sent from CIRS to EOS but was not able to be used due to the issues discussed below. Overall electricity use can be seen on a monthly basis in Figure 4.

Discrepancies between anticipated and actual performance
A physical examination of a new MUA unit, rooftop air-handling units (AHUs) and the fume hood exhaust manifold on the EOS roof where collectively heat is exchanged between the two buildings revealed

Figure 2 Actual CIRS energy performance
Learning from failure that the system did not function as assumed during the design stage. In particular: . The EOS building was taking in much less outside air than had been assumed during design (with the majority of air in the building being recirculated).
. The EOS exhaust temperature was 168C instead of 208C assumed during design. This was due to the effect of a water-based fume hood exhaust manifold scrubber that lowers the temperature of the exhausted air, resulting in less heat availability.
. The older design of the intake EOS rooftop AHU was based on a cold-deck, hot-deck concept designed to dehumidify the air before it is mixed with recirculated building air and heated for distribution as conditioned air. The design of this unit was such that the cooling coil for the EOS building was located sequentially before the heating coil of the building, meaning that if the conditioned air heated using upgraded heat from CIRS was actually being accepted by the new MUA unit, it would have been being cooled before entering Notes: a Estimate based on a small data set (less than one year of properly metered data).
b Calculated (not a metered system) based on the assumed percentage of the total, same as theoretical. the EOS building. This has prevented the pump on the EOS side of the MUA unit from running and enabling heat transfer. EOS will likely be able to accept very little heat from CIRS until a retrofit of the intake system is undertaken.
As these physical realities prevent the building system from achieving its design intent, it is clear that neither of these facts about the operation of the existing EOS rooftop units was fully understood during the design of the energy system. With regard to outdoor air intake, since the EOS building is a laboratory, typically a building type requiring several air changes per hour, one would expect that the building would take in a large amount of outside air to avoid possible issues with indoor air quality. However, EOS operation inspections show that in actual operation a relatively small proportion of outside air is used. With regard to the intake of heated air from CIRS at EOS, since the cooling coil is located before the building's heating coil, there are very few hours of operation in a year when the outdoor air is cold enough that it could be mixed with the preheated air being delivered from the new MAU which has a heating coil supplied by CIRS and still be below the cooling coil temperature set point that would call for cooling. 11 The consequence of this is that CIRS is currently unable to send large quantities of useful heat to EOS; as mentioned above this occurred only during December 2012.

Understanding the discrepancies between measured and actual performance
In general, a key lesson from the examination of the heat transfer systems of CIRS was that it is necessary to consider all modes of operation and potential flows of heat transfer in order to design and implement adequate controls, measurement and operational strategies. Without this understanding, it is likely that monitoring strategies will not deliver accurate reports due to misrepresentation or insufficient information.
In the case of the monitoring systems for CIRS, significant faults were found such as meters that did not recognize flow directionality, were mislabelled, incorrectly installed and calibrated, left out of the energy understanding boundary, or which used differing conventions of directionality for various system components. It is not enough simply to have access to high quantities of data. Without a proper understanding of how to interpret data sets and their relative importance, more data may simply result in more confusion and operational problems, as was evidenced through this case study's performance data and survey data, which showed a low level of building system understanding of inhabitants ( Figure 5), despite this being a high-level project goal. Further survey data can be found in the Appendix in the Supplemental data online.
In addition to doing additional technical analysis, examining the data from the CIRS BMS revealed that in many cases the measurements being reported by the BMS were not accurately representing the performance of the building systems. This means that not only was there a discrepancy between predicted and measured performance, but also there was a discrepancy between measured and actual performance. In order to obtain a more accurate picture of the energy flows, it is necessary to look at the energy rate as well as the supply and return temperatures at each heat pump. 12 An example of this can be seen in Figure 6, which shows the difference between the metered heat transfer from EOS to CIRS and the values calculated based on sub-metering points of temperature and flow. This metering error was due to the meter not recognizing flow directionality. It is also an operational issue in that the operational sequence did not recognize when CIRS was venting heat to this connection, and therefore did not check this against whether there was a call for heating, or whether heating was in fact available to be captured from the EOS connection.
As systems become more complex, the needs of various parties that will interface with that information becomes increasingly important and, as such, it is necessary to assess how to make information accessible, interpretable and actionable. With the increased use of terms such as 'smart' and 'intelligent' to describe infrastructure, it is important to remember that the true 'intelligence' of a facility comes from its designed ability to serve specific functions in a useful and convenient way (Chen, 2010).

Implications
There were likely many occasions when the issues identified in the 'Anticipated and actual energy performance' section above might have been discovered and addressed. For example: . During pre-design and the design phase, if there had been better communication or more thorough site visits to EOS, then the systems compatibility issue could have been discovered and the design modified accordingly.
. System integration could have been further questioned during construction and commissioning, with a better understanding of system boundaries contributing to better design implementation and operational performance.
At any point during life cycle of the project, had the energy flows been disaggregated, understood and articulated in detail, these issues could have been addressed prior to post-occupancy performance evaluation. System integration and sequences of operations could then have been evaluated during construction, commissioning and operation. However, it is worth noting that such detailed analysis is not typical, even of newer buildings that undertake some degree of energy modelling as part of their design. As a result, such issues are either overlooked by the various parties associated with the project, not considered within their realm of responsibility or control, or their recognition considered politically problematic if raised with institutional management. An analysis of energy flows by each party responsible for overseeing integration would have included designers during design, commissioning agents during commissioning, and an overlap of responsibility and sharing of understanding during other construction phases.
While the process of identifying, quantifying and characterizing CIRS energy performance problems described in this paper has resulted in valuable learning generally applicable to the campus and to building teams, this detailed analysis was undertaken fundamentally to correct the performance problems themselves. At the time of writing (2014), tangible solutions are being implemented by UBC to resolve the identified building systems issues. This iterative process of implementation, analysis and feedback has proven fundamental to the advancement of sustainable building practices at UBC and is anticipated to provide significant future performance improvements. It has resulted in a continual improvement of the systems in CIRS, and an understanding of the processes required to improve new and existing buildings on the university campus, feeding forward to the operations and design teams across UBC. The pervasiveness of some of these performance and process issues may require a fundamental re-evaluating of the processes used to achieve high-performing buildings.
Aside from the need to examine detailed energy flows and operational sequences, there are several key general lessons that can be drawn from this brief review, including the need: to understand system boundaries better; to ensure contractual arrangements with consultants cover the full range of work required to achieve the design intent (in this case, a two-building analysis); to look at performance metrics that do not misleadingly aggregate important building information; to focus on stakeholder communication; and to incorporate intentional feedback into all stages of the project life cycle.

System boundaries
While there have been energy performance issues in CIRS, the energy exchange system with EOS still represents a particularly interesting design solution in that it takes into account the local context of the building and puts to the test the concept of system optimization at a level beyond a single building. This allows the building to be part of a multi-building resourcesharing network or building integrated distributed energy system. It extends the boundary of the energy systems beyond the building footprint and supportive of UBC's priority of considering individual buildings as part of campus or multi-building scale system much like a community-energy utility. With this intended network operation, it is imperative to draw the boundary of analysis at a larger scale in order to optimize design. As such, all analysis and testing throughout the whole project cycle and especially during design and commissioning should have included both EOS and CIRS as a two-building system. Prior to any intervention at EOS, a full assessment of the building systems and performance should have been undertaken so as to provide the design and operations team of CIRS with accurate information. In addition, metering should have been installed on the EOS side in order to represent accurately the impact CIRS has on EOS steam use as well as to understand how issues such as air intake and outlet air temperatures would affect the two-building heat exchange system. No such analysis, testing or optimization was undertaken, or meter or monitoring system installed at EOS so as to provide accurate information on the two-building system.
A key lesson from this investigation comes from the new understanding of the functioning of the two building energy transfer system, which has not performed according to the design intent. The monitored performance and operation has shown that it is necessary to understand not just one operational sequence but all the possible operating sequences and cascades of energy flows in networked energy systems. If a move towards fully integrated energy systems is to be realized on the campus, then rigorous articulation and conceptualization of system and component affects should be undertaken throughout the project's life cycle. Both the design teams and operational teams should undertake this conceptualization. This would assist in enabling intended system functioning and overall efficiency gains afforded by integrated systems. Stated more generally, it appears that the design process for CIRS did not adequately take into account the fact that this was a two-building system involving multiple institutional entities for operation, which required significant assessment of the characteristics and performance of the pre-existing building. Buildings and their system components should be thought of in the context of system integration at a larger scale than only the building site.
When project scope or boundaries change such that design or operational teams may begin to consider new components that are in fact integral to the operation and function of the system as a whole, it is necessary to understand the ways in which components could affect each other and to incorporate this information into design documents, e.g. in specifications and predictive modelling techniques. This understanding of building and system physics is particularly important for the designers as well as those commissioning and connecting the system components. As energy exchanges between adjacent buildings is atypical, the process of understanding and questioning system boundaries should be prioritized. This may require redefining 'ownership' over project responsibilities and prioritizing integration processes (Sorrell, 2003).
When the system operational boundary includes multiple buildings, the integrated design clearly needs to be extended to the operation of the respective buildings, and of all institutional parties involved in implementation and operation. Re-scoping of integrated design could lead to better quality assurance processes (Sorrell, 2003). Such re-scoping can be understood in terms of who are engaged throughout the design and construction of buildings, the work presented here is equally concerned with what is included in the process. The primary emphasis of current building energy performance relates to that of individual buildings and this is the scale at which building codes focus and which energy services are metered, and where design responsibilities reside. However, viewing buildings as part of a larger energy system means that the boundaries of analysis are expanded and, in doing so, new metrics and methods will be required to evaluate and ensure success.
Performance metrics: energy-use intensity (EUI) CIRS' operational EUI proved to be an inadequate indicator of building performance and functioning.
As is the case with many aggregate and average measures used to indicate performance of complex systems, this metric fails to show the manifold interactions of sub-system performance and provide understanding of how to ascertain energy savings and optimizations. Survey data showed that both operators and inhabitants found monitoring systems to be difficult to understand and interpret in general, with inhabitants rating monitoring systems poorly and operators commenting on the query process being cumbersome and time consuming. An operator specifically stated: 'Extracting the data using a query through Excel is cumbersome, time consuming and the options for the sample frequency is limited'; and 'numerous sensors (space temperature & duct air flow) have never worked correctly'. As discovered by other researchers, energy savings cannot be realized through metering and advanced systems if the human element is not carefully considered (Carrie Armel, Gupta, Shrimali, & Albert, 2012). Utilization of BMS to increase building operating performance has been found elsewhere to be deficient due to perceived complexity, lack of good user interfaces, and misperceptions from both users and designers (Lowry, 2002).

Feedback
The issues with the performance of CIRS and in particular the systems' connections to EOS were discovered during the operation and performance evaluation of the building, which in this case arose late in the life cycle of the project. This underscores the importance of having performance feedback for a building's operation as compared with its design. Continual reference back to the overall project goals was extremely valuable during the design process, as was shown by coding of design charettes meeting minutes whereby conflicts seemed to be overcome through linking back to overarching goals, and would clearly have been useful to close the loop between building operations and design. Systems such as the 'Soft Landings' framework developed in the UK may assist in closing the performance gap and speak to the need to improve communication, feedback and 'soft' handover of building projects (Clark, 2012). Since in many cases the implementation of new 'smart' technologies is outpacing the understanding of the subsequent complexities that are introduced (Blumsack & Fernandez, 2012), it is important to take time to produce lessons learned that industry can access and to feed these lessons forward to future projects.
The operational issues discovered during performance evaluation of the CIRS building highlight lessons related to implementing controls and monitoring systems for advanced sustainable buildings. The design of feedback systems needs to be carefully considered for such buildings, keeping in mind the target audience who will be interacting with the system and data, the frequency with which that interaction will occur, and the way that whole-system and subsystem information is incorporated in an understandable way that can result in ongoing performance improvements. For control systems to be optimal troubleshooting aids for operators, this research illustrates that data should be easily accessible, interpretable in terms of how system operation is affecting overall performance and aligned with overall design intent, and actionable in that it provides information that can support changes in building performance. Other research has come to similar conclusions on controls design, and has found that controls shortcomings are a widespread buildings issue (Bordass, Cohen, Standeven, & Leaman, 2001). In general, this and other research shows that the feedback and feed-forward from buildings in operation is a valuable resource for the design community and for building owners and operators (Bordass, Leaman, & Ruyssevelt, 2001). Moreover, the work highlights the importance of (1) establishing a strong culture of learning and (2) targeting the overall performance that is needed for capitalizing on opportunities for improvement. Table 3 shows some of the key lessons learned from this analysis of the performance of the CIRS energy and control systems.

Conclusions
This paper has presented an analysis of energy performance 'gap' in the CIRS. While the study relates IDP principles need to be extended beyond the design stage to include the entire project life cycle IDP should also be extended to the appropriate system boundary, which, in the case of CIRS, is inclusive of the entire multi-building system System boundaries Systems boundaries and network design implications need to be considered from the outset, inclusive of all stakeholder responsibilities for the system Limitations of simpli¢ed models and tools need to be fully understood at the design stage Individual optimization of small components will not translate into signi¢cant savings if the underlying operation of systems is incorrect

Feedback
Feedback is needed at all stages and particularly from commissioning back to design and from operation back to design Monitoring and assessment is a key mechanism to ensure that systems are performing as intended and this capacity should be considered and accommodated during design

Institutional capacity
Without institutional support, capacity and opportunities to implement solutions or communicate issues, projects will not perform as intended

Performance
Large commissioning and operational opportunities for energy e⁄ciency exist, even in 'green' or 'sustainable' buildings Learning Culture of learning and embracing failures is necessary in order to achieve overall building performance success to a single building during its first year of operation, the monitoring capabilities built into CIRS, despite the problems noted above, have permitted a greater level of scrutiny and explanation of the performance 'gap' not often possible in other buildings. Clearly even in buildings that aspire to exemplary performance, if the basic functioning of building systems is not correct, fine-tuning individual component efficiencies and other details within the design will not result in desired performance outcomes. Again somewhat obviously, the work reinforces the need for team members to understand better the basic flows and thermodynamics of all component building systems prior to detailed component design.
It is often assumed that the main barriers to achieving low energy consumption levels in new buildings have to do with the extra costs associated with such goals or the technical difficulties of achieving them. It is, however, apparent from the review of the energy and controls systems in CIRS presented in this paper, that the main barriers, in practice, to achieving its very ambitious design goals with regard to energy performance (summarized in Table 4) were not economic or technical. In the case of CIRS, the primary impediments were institutional, in the sense that they arose from the way the various stages of the building life cycle were specified, contracted and implemented. The problems had much more to do with a lack of useful information, interpreting of information, communications, feedback and interaction than with the expense and technical difficulty involved in implementing the CIRS design goals. To the extent that this lesson is generalizable, this would suggest the desirability of focusing attention and effort on these institutional issues, particularly for buildings attempting to reach aggressive energy or other sustainability performance goals. A re-scoping of the integrated design process for these buildings may help to ensure optimal outcomes and a better process of feedback and feedforward. Table 4 summarizes key barriers to performance of highly sustainable buildings based on the lessons learned from CIRS.
A key issue emerging from this paper relates to the process of learning from failure, which in turn is closely connected to the way actual building energy performance information is made publically available. Successes are clearly more comfortable for building owners and design teams to broadcast than perceived failures and indeed offer considerable promotional value. But the disclosure of 'failures', when explained and accompanied by the lessons drawn from them, are perhaps much more valuable in making subsequent performance advances. As public institutions with a mandate for research and publication, universities are well positioned to publish performance data on their buildings, and to derive lessons from ongoing experience of, and research about, the technical and behavioural dimensions of performance failures. The extent of the performance gap in sustainable buildings, and the concomitant need for it to be addressed in a timely fashion, suggest that universities may have an important role in becoming transparent living laboratories of sustainable building performance, and thereby contributing to narrowing the performance gap. UBC has taken the position that the university also has a responsibility to act as a test-bed for innovative sustainable practices by viewing the 'Campus as a Living Lab'. This has not only created the necessary receptivity to, and rethinking of, the notion of 'failure', but also permitted both academics and operational personnel to engage in and learn from such 'experiments'.

Questions for research and for practice
The lessons emerging from the energy analysis of the CIRS raise a number of key questions related to building design practice and research particularly given that performance requirements of new buildings are expected to ramp up significantly within the next decade, including: . What institutional barriers exist, and what are the current disciplinary and professional boundaries of design and client teams, that may hinder their achievement of high-performing buildings and what is necessary to overcome them?
. What present practices of building design currently hinder interdisciplinary interaction and shared responsibility for overall system integration?
. Are new design support tools such as building information modelling (BIM) software reducing the building energy performance gap?
. What stages of a building's life cycle offer the greatest opportunities to close the gap between predicted and actual performance?
. Does the present form of implementation of integrated design and operation encourage overcoming institutional barriers and creating optimal processes?
. What new skills and competences are required by design team members, inhabitants and operators -individually and collectively -to engage in building design that involve resource sharing?
. How can communication and feedback be more effectively encouraged, not just among the design team but over the life cycle of buildings, design, construction, commissioning and operations?
. What performance metrics more effectively characterize performance improvements in more complex systems design?

Disclosure statement
No potential conflict of interest was reported by the authors.

Supplemental data
Supplemental data for this article can be accessed at 10.1080/ 09613218.2015.1036227.