Intent and outcomes from the Retrofit for the Future programme: key lessons

The Retrofit for the Future programme, sponsored by UK government's Technology Strategy Board (TSB) from 2009 to 2013, demonstrated innovative approaches to deep retrofitting of social housing, using a whole-house approach for achieving an 80% CO2 reduction target. The intent and outcomes of this programme (in which all authors participated) are critically examined through a cross-project meta-study of the primary data, substantiated by insights from secondary sources. Given that only three (out of 45) projects met the expected CO2 target in reality, despite generous funding and professional expertise, it suggests that decarbonizing existing housing will not be particularly easy. Important lessons are found in this initiative's formulation, target setting, monitoring and evaluation procedures, and feedback mechanisms. These lessons can inform the formulation, delivery and effectiveness of future national energy retrofit programmes. Furthermore, to support the ‘scaling up’ of effective retrofit programmes and reduce the gap between intent and outcome, it is recommended that attention be moved from what level of CO2 reductions are to be achieved to how (delivery models) these radical reductions can be achieved and by whom (supply chain). Such alternative delivery models to the ‘whole house’ approach include retrofit over time, city-scale retrofit and community-based energy retrofits.


Introduction
This paper uses a cross-project meta-study approach to examine critically the intent and outcomes of the UK government's Technology Strategy Board's (TSBnow Innovate UK) Retrofit for the Future (RfF) programme. It outlines what programme-level lessons in terms of set-up, delivery and evaluation could inform policy-making and help produce more successful energy retrofit initiatives in countries faced with the challenge of large-scale improvement of their building stock. The paper also debates whether a 'whole house' low carbon retrofitting approach on a house-by-house basis is an appropriate means to meet radical CO 2 targets, by undertaking a comparative cross-project analysis of primary data (covering the technical performance of retrofits) to reveal the gap between intent and outcome. The remit of the paper addresses a number of key questions raised by the special issue, which include: . Are there instances where intended outcomes have been followed up and assessed in light of actual experience?
. What lessons could inform better policy-making and help produce more successful initiatives?
programme, the techniques used for monitoring and evaluation (M&E), and the data collected. The third section presents the meta-study by conducting a cross-project analysis of the primary data on energy consumption, air-tightness levels, ventilation systems and environmental conditions. The following section then extracts learnings from the RfF programme in terms of programme formulation, delivery and evaluation; while the final section uses specific learnings to construct wider lessons and recommendations for future initiatives and policy-making on energy retrofits.

Energy retro¢ts and performance gap
Along with the European Union, the UK is required to cut greenhouse gas emissions (CO 2 emissions are often used as proxy) by 80% over 1990 levels by 2050 (Department of Energy and Climate Change (DECC), 2012a). To ensure incremental progress is made en route to meeting this goal, five-year carbon budgets have been established in the Climate Change Act, to which the UK is legally committed (CCC, 2014). According to the 2011 UK Carbon Plan (HM Government, 2011), by 2050 all buildings will need to have an emissions footprint close to zero, and it is widely understood that the goal will not be achieved without significant energy retrofit of existing houses (Jones, Lannon, & Patterson, 2013;Li et al., 2015). This will unquestionably require a paradigm shift from existing approaches that tend to support the most cost-effective measures in worst-performing dwellings, as evidenced by some of the recommendations for meeting the next carbon budget, which include continual insulating of lofts and cavity walls and establishing of minimum energy performance standards for the private-rented sector (CCC, 2014). Even the Golden Rule, 1 which is a key element of the UK's Green Deal programme (DECC, 2012a), 2 limits the size of the loan (and, therefore, the extent of retrofit) to what can be repaid through energy savings (so as to protect the homeowner from financial burden). As such this rule has been observed to be a restriction on the level of obtainable CO 2 savings from wholehouse retrofitting (Dowson, Poole, Harrison, & Susman, 2012). Instead the approach of deep and whole-house energy retrofitting promotes the interaction of multiple measures to be considered (e.g. fabric, ventilation, heating, lighting and microgeneration) at the earliest stages, thereby improving the expected energy performance of the building from all end uses of energy, by an average factor of four or within a range of between 65% and 95% compared with pre-retrofit levels (National Energy Foundation, 2014). From the German experience, it is realized that since economic benefits are never as great as predicted, deep retrofits should be encouraged through policy, using alternative motivators other than economic savings (Galvin, 2014a).
The success of any carbon-reduction policy including an energy retrofit programme depends on achieving predicted energy reductions in practice (Summerfield, Oreszczyn, Pathan, & Hong, 2009). For this reason, recent policy initiatives such as the UK government's Electricity Demand Reduction Pilot 3 and the Energy Savings Opportunity Scheme (ESOS) 4 emphasize verification of energy savings in reality. However, an increasing body of research shows that large differences can occur between the predicted and measured energy performance of retrofits (Chiu, Lowe, Raslan, Altamirano-Medina, & Wingfield, 2014;Galvin, 2014b;Gupta & Gregg, 2012;TSB, 2013;Tweed, 2013). This energy performance gap can occur at any stage of retrofit delivery as follows: . During the design and specification stage, there can be a lack of understanding regarding the impact of early design decisions on energy performance, and lack of communication of design intent through all work stages. Also domestic energy modelling software, such as the Standard Assessment Procedure (SAP) in the UK, is reliant on the expertise of the user, quality of data input and appropriateness of the model to the particular context (Zero Carbon Hub, 2014). Energy models for retrofits must also consider the occurrence of underheating in poorly insulated dwellings (Sunikka-Blank & Galvin, 2012) and higher temperatures in well-insulated dwellings (Kelly et al., 2013).
. During the construction stage, substitution of specified products with products of inferior performance due to supply chain issues and inadequate on-site understanding of the performance implication of different products also contribute to the performance gap. This is further compounded by poor workmanship and lack of quality assurance procedures on-site (Gupta, Gregg, & Cherian, 2013). In retrofit, during the design and construction stages, an insufficient understanding of existing conditions can result in a failure to integrate new measures and technology appropriately.
. Finally at handover, delivery from the retrofit team is usually piecemeal with no formal aftercare arrangements, which lead to unfamiliarity amongst residents to operate, control, and maintain new and unfamiliar technologies resulting in suboptimal use, settings or unexpected behaviour (Gupta & Kapsali, 2014;Summerfield et al., 2009). As a solution, Bartiaux, Gram-Hanssen, Fonseca, Ozolina, and Christensen (2014) found that for retrofit work in Denmark, Portugal, Latvia and Belgium, knowledge networks were beneficial in providing advice and help for homeowners, both before and during the retrofit work.
For reasons such as these, policy-makers recognize the need to allow an offset for the performance gap in planning, therefore the Green Deal programme and the Energy Company Obligation (ECO) 5 include a set of 'in use' factors (Jones, Lannon, & Patterson, 2013).
Recent research has also revealed a building fabric performance gap since the fabric is not as well understood and far less predictable than originally thought. Li et al. (2015) have shown that a large sample of solid wall U-values were found to be better (and covering a wide range) than the standard U-value used in energy estimation. This suggests that standard UK solid-wall U-values may be inappropriate for energy certification or for evaluating the investment economics of solidwall insulation. This research has exposed the diversity in physical building characteristics as a significant source for anomalies. Furthermore, as demonstrated by Elton and Turrent (2011), moisture ingress into the building fabric and the resultant structural damage can be the result of misunderstood material relationships and poor construction quality. Research needs to address the lack of understanding regarding moisture movement through structures (particularly traditional solid-wall structures) from inside and out, as the addition of insulation can potentially lead to rotting of embedded timbers. In recent years, English Heritage and the Sustainable Traditional Buildings Alliance have done much useful research in this area, which addresses modelling inaccuracies and complexities in the study of moisture management in buildings and fabric (English Heritage, 2014). It is clear that whatever the reason, differences between the measured and predicted performance of retrofits have the potential to undermine national CO 2 reduction targets. It is against this context that the RfF programme was developed and implemented.
Retro¢t for the Future (RfF) programme: intent The RfF programme was sponsored by the UK government's Technology Strategy Board (TSB -now Innovate UK) from 2009 to 2013 with £17 million of funding to test and demonstrate innovative approaches to deep retrofitting 6 of the UK's social housing stock, using a whole-house 7 approach for achieving an 80% CO 2 emission reduction target. Through a two-stage process, 194 projects were awarded funding of up to £20 000 to develop a strategy towards meetings these targets in Phase 1, and about 86 projects (covering 119 dwellings) across the UK were awarded up to £150 000 each to demonstrate the effectiveness of their strategy in real homes during Phase 2. In order to set a single target across the programme independent of location, building type and condition, an estimated average emissions figure for the UK housing stock was used. Rather than a percentage reduction for each individual house, an absolute target was set for CO 2 emissions/m 2 /year as a standard that would achieve an 80% reduction in CO 2 emissions against a 1990 baseline when averaged across the existing housing stock. This is because percentage reduction against individual dwelling baselines would have been relatively easy to achieve for a poorly insulated home but considerably more difficult for more recent, better insulated, construction. The targets included all areas of energy consumption, including that for appliances, and were based on emissions from a typical 80 m 2 semi-detached house. There were two CO 2 targets, both based on calculated emissions: . 17 kgCO 2 /m 2 /year calculated using Standard Assessment Procedure (SAP) 2005 with an extension worksheet to account for unregulated uses . 20 kgCO 2 /m 2 /year calculated using Passive House Planning Package (PHPP) The difference reflects the use of lower carbon intensities in SAP as compared with PHPP. There was also a primary energy target of 115 kWh/m 2 /year. To assess the post-retrofit CO 2 performance, one year's worth of electricity and heating fuel energy data were collected and normalized using floor area and common CO 2 metrics.
The RfF programme was designed to target the social housing sector (representing 20% of UK housing stock), which is relatively 'homogenous' with an established organizational structure of tenants and social landlords who were able to facilitate the delivery of retrofits and select suitable tenants. This does imply that the findings will not be completely representative of the private housing sector where householders have a stronger role in the decision-making process. However, the programme was a 'living lab' of many different experiments, and involved rigorous and systematic evaluation of each project, comprising shortterm physical tests of the building fabric; long-term physical monitoring of energy consumption and environmental conditions; standardized post-occupancy evaluation (POE) of occupant (primary resident) experiences; post-construction review (PCR) of construction quality and holistic review of projects (TSB, 2014). Most of the primary data collected from the evaluation have been made available in the public domain through an interactive data platform called the Energy Monitoring and Building Evaluation Database (EMBED). Table 1 summarizes the types of data collected to evaluate the programme outcomes, the respective M&E technique deployed, and the organization responsible.
Data generated by each method, apart from POE (work is ongoing to anonymize the POE data for Intent and outcomes from RfF analysis and release), are available through a range of primary and secondary sources, as listed in Table 2.
Overall a maximum of 86 dwellings had some level of data available that have been used for this meta-study. However, depending on data availability, different aspects of data analysis are conducted on subsets of the 86 dwellings. For example, annual CO 2 emissions data (pre-and post-retrofit) are available for 45 dwellings, while only 36 dwellings have indoor monitored temperature and relative humidity (RH) data. In some cases the sample size becomes quite small. For instance, there are only 23 dwellings where pre-and post-retrofit electricity and gas figures can be compared. Another limitation of the data set is that it represents only one retrofit programme located in the UK. Figure 1 identifies RfF project teams/data collated by EST through EMBED † Meters for gas, electricity, water and oil (where necessary) with remote data collection † Internal temperature and relative humidity in three locations^living room, hall and principal bedroom † External temperature and relative humidity † Indoor air quality: CO 2 (as a proxy for overall air quality)

Occupant experience with the installation process/ how the occupant(s) is interacting with the measures/occupant thermal and overall comfort
Post-occupancy evaluation (POE) (immediately after occupancy)

RfF project teams
Post-programme assessment led by EST Site visit and walk-through inspection to review the following: Post-construction review (PCR) ESTand industry experts † Inspect building fabric, installed technologies and measures † Check and record delivery to the planned retro¢ts and comment on the appropriateness/ e¡ectiveness of any changes † Comment on quality of works † Record electrical appliances installed † Identify all defects and serious faults at the property which have arisen from the retro¢t To achieve 80% (deep) carbon reductions through a 'whole house' retrofit approach, a combination of energy-saving measures and low/zero carbon technologies (LZTs) were adopted across the RfF projects such as a super air-tight fabric in combination with mechanical ventilation with heat recovery (MVHR) systems, solar photovoltaics (PV), solar hot water systems (SHW), biomass boilers and heat pumps. As shown in Figure 2, unsurprisingly fabric improvements were most popular wherein wall insulation was installed widely followed by roof/loft insulation. While external wall insulation (EWI) was the most popular wall insulation strategy for all types of dwellings built post-1900 including post-1950s dwellings; internal wall insulation (IWI) was more prevalent in pre-1900 dwellings (historic) due to planning requirements, historic preservation and/or aesthetic concerns. Although deployment of heat pumps was high, gasfired boilers remained the most common heating system installed across two-thirds of dwellings because of resident familiarity. Due to high air-tightness levels, MVHR systems were installed in threequarters of dwellings to provide background ventilation and good indoor air quality.

Assessing outcomes through meta-study
The meta-study was conducted as a comparative analysis of RfF projects using primary data sets listed in Table 2, which are linked through unique identification numbers for each dwelling (TSB assigned identification numbers are not revealed in this paper). The Appendix in the Supplemental data online lists the dwellings included in the study along with characteristics and retrofit features. The main focus of the analysis is to examine the achievement of RfF targets in practice in terms of overall energy and CO 2 reduction, and also by fuel type. Since the ambitious targets set by the programme required high levels of insulation and airtightness, the meta-study investigates the effectiveness of significant improvements to the building fabric, through a comparison of measured pre-and post-retrofit air-tightness levels, and assessment of post-retrofit air-tightness levels against post-retrofit gas usage.
Given the large uptake of MVHR systems (to provide reliable ventilation whilst minimizing heat loss) in the RfF sample, the appropriateness of these systems and their performance (air flow rates) are also assessed. Finally, indoor CO 2 levels, temperature and humidity as indicators of environmental conditions are also examined, albeit for a small sample of properties due to issues with data availability.
Of the 45 RfF dwellings with available data on postretrofit measured CO 2 emissions, it is found that only three dwellings met the absolute RfF target of 17 kgCO 2 /m 2 /year (and 20 kgCO 2 /m 2 /year for dwellings modelled using PHPP). About 34 out of the 45 dwellings had data available for pre-retrofit modelled CO 2 emissions (Figure 3), of which six dwellings achieved a reduction of less than 30%; five had a reduction of between 30% and 49%; 11 dwellings achieved a reduction of between 50% and 69%; while nine achieved a reduction of between 70% and 79.9%; and only three dwellings achieved a reduction of 80% or more as compared with pre-retrofit modelled CO 2 emissions. Given that most dwellings (20 out of 34) achieved CO 2 reductions from 50% to just above 70%, it raises questions about the level of CO 2 targets that should be set for future programmes so that they remain ambitious but achievable.
As was expected, achieving the absolute RfF target did not necessarily equate to an 80% reduction over pre-retrofit modelled CO 2 emissions (baseline). For example, in one retrofit (H28), although CO 2 emissions were reduced by merely 55% over pre-retrofit modelled emissions (since it was a modern terrace with very low pre-retrofit emissions), the project was still able to meet the RfF target. This reinforces the need to establish clear and consistent absolute carbon targets (along with percentage reductions) in energy retrofit programmes, so as to avoid poorly performing dwellings easily meeting ambitious percentage reductions, while dwellings with a low baseline (pre-retrofit) CO 2 emissions struggle to meet the same reductions. Figure 4 compares the calculated pre-retrofit CO 2 emissions (calculated using SAP/PHPP along with an SAP extension worksheet to calculate CO 2 emissions from appliances and lighting) and measured (actual) post-retrofit CO 2 emissions for 43 dwellings (for which data were available) by retrofit wall type and heating system. While all the dwellings that achieved the RfF target had installed EWI, dwellings with all kinds of heating systems (from gas boilers to heat pumps to Figure 3 CO 2 emissions Note: n represents the number of dwellings with post-retro¢t CO 2 emissions. Not all these dwellings have pre-retro¢t CO 2 emissions data biomass boilers) experienced significant reductions in CO 2 emissions. This indicates the influence of the most popular strategy adopted across the RfF projects, the 'fabric first' approach (minimizing heat loss by prioritizing insulation and air-tightness before generating more efficient heat) on reducing actual CO 2 emissions.
Out of the 45 dwellings, for a subset of 23 dwellings, preretrofit modelled and post-retrofit actual data for gas and electricity consumption were available, as shown in Figure 5. Although only one retrofitted dwelling experienced an increase in energy consumption and CO 2 emissions (indicated by an up arrow in Figure 5), about 15 dwellings with (gas) boiler improvements reduced mean measured annual gas consumption by 78%, though their mean annual electricity use increased by 44% suggesting possible addition of electricity-using equipment. Interestingly no association is found between the installation of an (always on) mechanical ventilation (MV) system and increase in electricity use.
In eight dwellings with heat pump installations there was an expected rise in annual electricity consumption due to fuel switching, which was equal to a mean increase of 178% in annual electricity use, although the proportion of electricity used for heating is not provided in the data. Although the heating fuel figures are not weather corrected, generally in the south-east of the UK (based on the Heathrow weather station), the period before retrofit was cooler than the period postretrofit (Environmental Change Institute, 2015); however, both periods were warmer than average.
As seen in Figure 6, measured annual electricity use across all 23 dwellings (irrespective of the heating Figure 4 Predicted pre-retro¢t and actual post-retro¢t CO 2 emissions by wall insulation type and heating source. n represents the number of dwellings with post-retro¢t CO 2 emissions and heating technology information Figure 5 Electricity and gas consumption with CO 2 emissions. The ¢rst bar represents modelled pre-retro¢t energy consumption, the second bar post-retro¢t measured energy consumption. Heating fuel ¢gures are not weather corrected system) is greater than forecasted, implying that the difference between predicted and actual electricity use (rather than gas) is a significant factor in the energy performance gap. To add to this, only about half of the dwellings with gas fired boilers reduced gas consumption.
Investigating post-retrofit electricity consumption further, Figure 7 shows the change in electricity consumption for dwellings that had solar PV installations and those without. Within this sample, dwellings that had heat pumps and PV installed are differentiated in the graph. It is expected that even with PV there would be a possible increase in electricity consumption for dwellings that installed heat pumps in place of gas boilers. However, of the 11 dwellings with only PV (no heat pumps), five reduced their electricity consumption.
To add to this, most of the six PV only dwellings that experienced a rise in electricity use increased it to a greater degree than those that had a reduction. Possible reasons for this could be a change in household compositions of these dwellings, underperformance of PV systems and/or increase in the use of electrical appliances.
To investigate the fabric performance of the RfF projects given the high levels of insulation that the programme necessitated, measured pre-and post-retrofit air-permeability rates (as an indicator of air-tightness level) for 86 dwellings are compared, as shown in Figure 8. Apart from eight dwellings, post-retrofit When measured (maximum, mean and minimum) air permeability is assessed against categories of built form, dwelling age or retrofit wall type, no obvious relationships are found. Interestingly, a weak correlation (r ¼ 0.3) emerges between post-retrofit measured air-permeability rate and post-retrofit gas consumption (normalized by area) for a limited sample of 15 gas-heated dwellings ( Figure 9). (It is important to remember that a proportion of gas consumption is also used for domestic hot water and cooking.) Within this sample, one dwelling (H14) with the lowest gas consumption had an air-permeability .12 m 3 /m 2 /h@50 Pa. This is likely due to changes in household occupancy and composition. Based on these findings, it is difficult objectively to infer the impact and effectiveness of building fabric improvements on energy consumption; however, it is obvious that in order to achieve the ambitious levels of improvements targeted by the RfF projects, all the elements of retrofit need to work sufficiently together.
It is recognized that high levels of air-tightness necessitate the use of MV systems for providing reliable ventilation whilst minimizing heat loss. Mostly MVHR and sometimes decentralized mechanical extract ventilation (d-MEV) systems were installed in 66 RfF dwellings. However, only 23 (out of the 66) dwellings achieved post-retrofit air permeability of 3 m 3 /m 2 / h@50 pa or less, the level at which MV is recommended by the Energy Saving Trust (EST) (2007). The presence of MV systems (mainly MVHR) in dwellings (n ¼ 43) with higher air permeability than recommended suggests that MVHR was a redundant technology with an additional upfront cost (the average cost of a domestic MVHR system was about £6117 in the programme; Sweett, 2014) that also added an ongoing parasitic electricity load due to its 'always on' status. This trend of having MVHR in dwellings with inappropriate levels of air tightness is happening in new-build social housing projects as well (Gupta et al., 2013). Testing air tightness midway (as undertaken by some RfF projects) during the retrofit works can help to identify and fix unforeseen areas of airflow effectively and relatively cheaply. When the performance of MVHR systems is further assessed using measured air flow data for 42 out of the 66 dwellings ( Figure 10), a vast discrepancy is found between the whole-house supply and extract rates. This indicates a system imbalance, possibly due to issues with the design, installation or commissioning of these systems. Many of the tested dwellings are over-ventilated, which in turn increases space heating demand (Lowe & Johnston, 1997) and also the amount of electricity consumed by the MVHR systems.
The effect of air permeability and ventilation systems on physical performance is assessed using measured internal environmental conditions within a selection of RfF dwellings. Figure 11 displays the minimum, mean and maximum indoor temperatures and RH conditions observed in 36 dwellings. Looking at temperature to begin with, it is apparent that the annual mean indoor temperature of 228C is high and towards the warmer end of the recommended comfort range of 20 -228C. Whilst, on the one hand, this illustrates the ability of well-insulated dwellings to provide whole-house comfort, such temperature levels have implications for heating energy consumption. Furthermore 21 out of 36 dwellings experience maximum temperatures above 268C Post-retro¢t gas consumption and air-tightness results and in some cases over 288C (in the living rooms), indicating very high demand temperatures. More detailed analysis of individual cases is necessary to establish the risk of overheating. On the other hand, seven out of 36 dwellings also experience minimum indoor temperatures ,168C and in some cases up to 148C, indicating design or control issues or unexpected patterns of use. The overall mean RH is 52% across the 36 dwellings, with all the dwellings experiencing mean RH levels within the expected comfort range of 40 -70%. A few dwellings have minimum RH levels ,30%, which is likely to be due to high internal temperatures.
In assessing the internal conditions, it is important to consider the indoor CO 2 levels which are a good indicator of ventilation rates and indoor air quality. Figure 12 displays the CO 2 concentrations in living rooms of 12 dwellings allowing for comparative analysis. There is a general acceptance that CO 2 levels above 1000 ppm are indicative of poor ventilation rates (Sharpe, Porteous, Foster, & Shearer, 2014). Interestingly, though representing a small sample, two out of the three dwellings with the lowest mean CO 2 concentrations (,1000 ppm) are naturally ventilated. On the other hand, eight out of 10 MVHR dwellings have experienced CO 2 concentrations above 1000 ppm, while four of these experience concentrations greater than 1500 ppm. Given the small sample, while it is not possible to associate poor indoor CO 2 levels with underperformance of MVHR systems, it reemphasizes the role of

Lessons from the RfF programme
Findings of the meta-study provide useful insights and learnings for improving the formulation, delivery and evaluation of future energy retrofit initiatives.

Programme formulation and target setting
The overall experience of the RfF programme shows how difficult it is to make deep cuts in energy consumption and CO 2 emissions from existing housing. One of the reasons for this is the dilemma that lies in the formulation of the programme itself, between the intent and approach. Although the programme was designed to deliver clear and ambitious CO 2 targets, it adopted an innovation-based approach, which led most projects to use effectively experimental (untested at that time) strategies to meet the goals. Whilst innovation should be encouraged, risk and uncertainty associated with innovation, and their impact on delivery of targets should also be considered during programme formulation. Furthermore innovation needs to sociotechnical, as research has shown that performance gap occurs where technological innovation takes place without social innovation (new ways of working with the user) (Gill, Tierney, Pegg, & Allan, 2010;Usable Building Trust & BSRIA, 2009). Experimentation and innovation, however, are not the chosen policy route in the recent carbon budget for achieving the prescribed emissions reductions (CCC, 2014). Whether future programmes adopt experimental or conventional measures or both, it is vital that the approach selected is aligned with the intent of the programme, and integrated and effective supply chains are established for delivery.
The 'whole house' approach adopted in the RfF programme was designed to address all end-uses of energy and associated CO 2 emissions as a whole, but in practice this approach rarely delivered radical energy reductions, despite generous funding available for the implementation and involvement of committed and expert professionals (architects, engineers, constructors and university researchers). This indicates the complexity associated with the design and delivery of whole-house energy retrofits. Also the 'whole house' approach was 'technologically focused' and driven by physical energy models, while the human dimension was often omitted and not addressed through behavioural interventions. Future programmes must adopt a sociotechnical approach taking into account both the building itself and the occupants within to address both physical and social factors that influence energy consumption, so that energy savings are realized and sustained over the long-term, as observed in some recent community-led energy retrofit programmes (Gupta, Barnfield, & Hipwood, 2014).
The RfF analysis showed that a majority of the dwellings achieved around 50% CO 2 reductions (50% of the RfF baseline is roughly equivalent to 43 kgCO 2 / m 2 /year), which coincidently is the percentage target set (50% CO 2 reduction by 2025) for the fourth carbon budget (CCC, 2014). However, in initiatives where capital cost is a concern and deep low carbon retrofitting is likely to be unattainable, ambitious CO 2 cuts will be even more difficult to achieve. Galvin (2014a) has questioned whether the existing domestic sector should be even burdened with the same CO 2 reduction expectations as other sectors, given the history of inconsistency, unexpected results and complexity around the issue. Nevertheless, if the buildings sector does not deliver ambitious energy savings, meaningful CO 2 reduction targets will become very difficult to achieve nationally, since over 80% of existing homes will still be standing in 2050 (CCC, 2014). This then raises an important question for future programmes, whether house-by-house deep low carbon retrofitting is a viable route to reducing CO 2 emissions by 80%, or rather more localized area-based approaches need to be investigated, so that economies of scale can be used to drive down both capital costs and CO 2 emissions. Furthermore, even with CO 2 reduction as the goal, the majority of RfF projects in the programme consumed more electricity than predicted, exposing the challenges associated with cutting electricity use. To improve the uptake and effectiveness of electricity-saving interventions in future initiatives, it is necessary to develop a deeper understanding of the variety of ways in which electricity is used in existing homes across different tenures and why people react to particular electricity-saving interventions in the ways that they do. The recent Household Electricity Survey commissioned by the government (Godoy-Shimizu, Palmer, & Terry, 2014) provides insights into patterns of electricity use of owner-occupied houses, but needs to be expanded to include social housing and electrically heated dwellings.
This leads to a discussion about whether CO 2 reduction should be considered as the main driver for retrofitting. Obviously for the UK government and the European Union reduction of CO 2 emissions has been the goal for deep retrofitting. But as Galvin (2014a) points out, the tunnel vision of achieving a specific emissions target has created policy measures that become barriers to homeowners potentially improving comfort on their own terms or reducing energy consumption incrementally in an affordable manner. Swan, Ruddock, and Smith (2013) interviewed social housing providers and found that fuel poverty and energy bills for residents were the top drivers for retrofitting. Chahal, Swan, and Brown (2012) interviewed social housing residents and found that energy costs and improved comfort were the leading drivers for interest in retrofitting. In both studies climate change as a driver received the least responses. As also shown by Tweed (2013) it was evident in some RfF projects (including the current authors' experience) that some residents prioritized comfort over greater energy or cost savings when afforded the ability to do so. Such findings imply that outcomes of energy retrofit programmes must be reframed to focus on outcomes wider than energy or CO 2 reduction, such as comfort, indoor air quality, health and elimination of fuel poverty.

Monitoring and evaluation (M&E), and feedback
For the first time, from the outset, the RfF programme prescribed clear M&E protocols for undertaking physical tests, co-incident energy and environmental monitoring over a long-term (12 -24 months) and social science surveys. Although physical tests such as air-permeability tests and thermal imaging surveys (for some projects) were conducted pre-and post-retrofits, the programme did not require the assessment of pre-retrofit measured energy consumption. Although the difficulty of obtaining consistent pre-retrofit energy data is acknowledged, such data could have helped in establishing baseline energy performance of the dwelling which could be used for validating energy models, providing a reference for measuring actual energy savings to verify the success of the retrofits and, most importantly, in revealing any unusual behavioural characteristics (such as excessive use of electric lighting or tumble dryer) that could be addressed through the retrofits. This is especially important for future energy programmes, given that collating reliable data about energy consumption is recognized to be a perennial problem internationally (Godoy-Shimizu, Palmer, & Terry, 2014).
Despite the effort put into M&E and the experimental nature of the RfF programme, the feedback mechanisms to share the learning from the M&E were fragmented. PCRs were undertaken by third-party experts with almost no feedback provided to the project teams apart from an overall report. However in seven projects, post-construction (hindsight) interviews regarding perception of occupants' experience, lessons learnt and viability of technical solutions were undertaken as part of a separate European Regional Development Fund (ERDF) sponsored project on Facilitation, Learning and Sharing (FLASH) and published to identify future opportunities (Lowe, Chiu, Raslan, & Altamirano, 2013), but no feedback on the dwelling performance was provided to the residents. Occupancy surveys (comprising 90-min interviews and walk-throughs) managed by EST served as a useful way to gather feedback from residents on their experiences with the retrofit process and technologies installed, but no feedback or advice was provided to the residents about managing and reducing their household energy consumption, or the project teams. In future programmes, retrofit project teams, housing associations and supply chains should be urged to integrate the lessons learnt from retrofit evaluation into their knowledgemanagement systems, so that mistakes revealed through the evaluation are not repeated elsewhere. This could be captured through a domestic variant of the Soft Landings approach (Usable Buildings Trust & BSRIA, 2009). 8 Given that M&E of retrofits is a form of action research (Oreszczyn & Lowe, 2010), it is vital that evaluation methods are integrated with occupant training, education and advice to avoid unintended consequences such as frequent window opening in dwellings with MVHR during winter for 'fresh air', which causes heat loss. In an ongoing project studying the impact of low carbon community groups, it has been found that person-to-person feedback through advice 'helped a lot' or was 'crucial' in informing residents on how to reduce energy consumption in their home . Another method of feedback is through social learning, which can involve local communities and networks to share occupant experiences of energy retrofits through home visits and open days for the wider community (Berry, Sharp, Hamilton, & Killip, 2014). This would also enable greater diffusion of energy retrofits amongst the wider population.

Conclusions
Reducing energy consumption associated with buildings has long been identified as a key, relatively easy and cost-effective strategy, and as a focus for policy that can deliver a major impact in the next 30 years (CCC, 2014). However, evidence from the meta-study reinforces the finding by Oreszczyn and Lowe (2010) that it will not be particularly easy (and cheap) to reduce energy use in existing housing. Given that whole-house retrofits, as a route to achieving radical cuts in CO 2 emissions, did not deliver all that was intended, it might be more appropriate then to shift the focus of attention from what level of CO 2 reductions are to be achieved, to how (delivery models, implementation challenges) these radical reductions can be achieved, and by whom (supply chain, capacity in the system), in order to support 'scaling up' of effective retrofit programmes. While 'deep renovations' have political traction through the European Union Energy Efficiency Directive (European Parliament & Council of the European Union, 2012), it is worth debating whether the 'whole house' approach is the desired route, given that the scope of works in a wholehouse retrofit is extremely challenging (high levels of insulation, MV, LZT etc.), involving a variety of suppliers and project management of multiple measures. Other alternative and complementary models for delivery of energy retrofits also need to be examined.
Given that physical changes to dwellings take place incrementally and usually privately (Fawcett, 2014), models such as 'retrofit over time' (planned or emergent) are gaining recognition, given the experiences of owner-occupied Superhomes that have achieved 60% CO 2 reductions or more (Fawcett & Killip, 2014). Whilst individual action at building level remains important, an overemphasis at this scale risks fragmentation and overreliance on individual building owners and tenants (Dixon & Eames, 2013). This is why city-level (urban-scale) retrofit has gained prominence, operating across the building, neighbourhood and city-regional scales, and involving and motivating a range of stakeholders such as developers, financers and policy-makers (Dixon & Eames, 2013). Even at a sub-city level, widespread upgrades to the UK housing stock can be achieved by involvement of local, community-based programmes that simultaneously address the social and technical issues of domestic energy consumption (DECC, 2014c;Karvonen, 2013). Customized solutions to groups of dwellings can be developed, which can also be replicated in neighbouring areas. As observed in DECC's Low Carbon Communities Challenge (DECC, 2012b) and Local Energy Assessment Fund (DECC, 2013), community retrofit programmes could be community led, local authority led or even a community/local authority partnership, but importantly they are able to engage with the owner-occupied (70% of stock) and private rented (10% of stock) sectors that are vital for scaling up. As a corollary, it also brings local support for retrofit delivery, given that local planning was found to be an obstacle for many RfF projects despite the programme being government funded (TSB, 2014).
Independent POE studies of 10 RfF projects by the Institute for Sustainability (IfS) (2012) and 18 RfF projects by the EST (TSB, 2013) revealed that the type of lead organization influenced the integration of the project team and communication, as well as occupant satisfaction. It was found that dwellings with higher occupant satisfaction and team integration were led by project architects as opposed to local authority or housing associations. The occupants of these dwellings were found to be either satisfied or very satisfied with the handover process and claimed to understand how the technologies were operated, thereby implying the significant role of supply chains in the effective delivery and aftercare of energy retrofits. Although it is suggested that community groups and local authorities lead retrofit programmes in an umbrella sense, the role of integrated supply chains in delivering effective retrofits has to be emphasized. However, given the disjointed character of the retrofit industry (Clarke, 2006), this is a significant challenge to address and an area for further research.
Given the scale and scope of deep retrofits, success of these programmes will be dependent on resident experience. Whether or not the residents are decanted from homes during the retrofit process can impact the quality of work. In one example reported (TSB, 2014), when minimization of disruption was prioritized as residents remained in the home, opportunities to improve airtightness were not taken fully as the extra works were considered too disruptive. It is preferred that for deep retrofits occupants should be decanted to achieve excellence in detailing the building envelope, specifically when internally wrapping the envelope and installing floor insulation (Baeli, 2013;TSB, 2013;Tweed, 2013). Further research is required to understand residents' responses to living amongst building works, and how their tolerance to disruption caused by retrofit works can be increased, as retrofit scales up and the private sector market grows (where moving people out temporarily will be more challenging). Active engagement of residents, right from the briefing and design stage, through construction, handover and aftercare, will undoubtedly play an important role in this.
The innovative and experimental nature of the RfF programme has warranted the high capital costs of £150 000 per RfF project (including design fees and monitoring costs), but it is obvious that these costs cannot be scaled up. However, as with any innovation, it is expected that there would be considerable refinements and improvements as subsequent projects are rolled out. What is important is that if predicted energy savings are not realized in reality, the costeffectiveness of any interventions could be jeopardized. Examining the relation between costs and CO 2 savings, Jones et al. (2013) show that significant improvements are achievable at low costs, but as CO 2 savings go above 40% the costs rise steeply, while major cost increases begin to set in beyond 60% reduction. As experience in Germany shows, sometimes a simple cheap solution that does not meet the most advanced standards can bring significant benefits and allowing these solutions could mobilize actions in low-to-medium-income households (Galvin 2014a). On the other hand, mandatory and highly restrictive standards for thermal retrofitting of existing homes have had a negative impact, slowing the rate and depth of retrofitting required for achieving national goals in Germany. Energy retrofit policy should take account of the experience of the RfF projects in setting retrofit targets for existing housing that are acceptable to householders and are technically and financially viable, whilst recognizing that costs will fall, construction standards will rise and technologies will be fine-tuned as a mass market develops.
Many of the wider lessons outlined in this section may seem obvious; however, these issues are often overlooked even in large policy-based dwelling energy retrofit interventions. No doubt the RfF programme succeeds in capturing learning from projects by providing a repository of empirical evidence (though with varying data quality across projects) on the actual performance of deep retrofits. Until this programme, such a catalogue of empirical data on dwelling retrofit performance has been largely absent from the debate. Given the large differences that are found to occur between predicted and measured energy performance of deep energy retrofits, it is vital that such differences are captured through procedures similar to RfF, but it is also essential that the learning from such programmes is considered in the planning of incremental five-year carbon budgets to bridge the gap between expectation (targets) and practice (delivery). Otherwise there is a risk that retrofits (deep or shallow) will save less energy than expected, and meaningful CO 2 reduction targets will become very difficult to achieve.
Usable Building Trust and BSRIA. (2009). The soft landings framework: For better briefing, handover and building performance in-use. Retrieved from https://www.bsria.co.uk/ services/design/soft-landings/free-guidance/ Zero Carbon Hub. (2014). Closing the gap between design and as-built performance: Evidence review report. Retrieved from www.zerocarbonhub.org/ Endnotes 1 The Golden Rule is the central mechanism for determining which measures can be financed by using the Green Deal. The rule: 'Estimated savings must be greater than or equal to repayments.' The caveat: 'Actual savings may be less than these repayments (if your energy use changes or if energy prices fall)' (DECC, 2012a).

2
The Green Deal is a UK government-initiated private investment in the carbon reduction of the existing building stock. Energy-efficiency improvements will be offered by the private sector to homeowners and businesses at little or no upfront cost with payment recouped through customers' energy bills (DECC, 2012a).

3
Electricity demand reduction (EDR) summarizes electricity savings as a result of the installation of more efficient electrical equipment. Savings are calculated by comparing energy use before and after a project, whilst making appropriate adjustments (DECC, 2014a). 4 ESOS is a mandatory energy assessment and energy saving identification scheme for large 'undertakings' in the UK. Large undertakings are defined by either a large number of employees or a large annual financial turnover and balance sheet (DECC, 2014b). 5 ECO operates alongside the Green Deal placing the obligation on larger energy suppliers to deliver energy-efficiency measures to domestic premises of vulnerable consumer groups and hardto-treat homes (Ofgem, 2014). 6 A deep retrofit implies that interventions introduced will have a strong impact on the energy use and CO 2 emissions of the existing building, typically aiming for a 80% reduction in line with the UK's Climate Change Act target figure. 7 A whole-house approach means considering a household's energy needs and CO 2 impacts as a whole, and establishing a comprehensive package of measures to reduce them. 8 The Soft Landings approach provides a five-stage alternative to the conventional brief, design, build and occupy system, which aims to close the performance gap (Usable Building Trust and BSRIA, 2009). Though designed for non-domestic use, the Soft Landings approach provides a methodical approach to improving the review and communication of intent, assessment of knowledge and understanding of the construction team, and appropriate handover and aftercare steps for the occupant, which could all be applicable to construction in the domestic sector.