Content Analysis of U.S. Local Adaptation Plans.xlsx
Purpose of the data:
This data represents the content and quality of 44 local climate change adaptation plans in the United States. Climate change already affects local communities across the country, and these impacts are projected to become more severe and intense in the future. In response, many local communities are investing in adaptation – actions to limit the negative consequences of climate change – and, in many cases, creating climate adaptation plans. These plans detail how climate change is projected to impact the community and what actions should be taken to prepare. Adaptation planning represents systematic attention to climate change and, as a result, is expected to help prepare communities and lower the cost of climate-related impacts. This data represents how adaptation plans align with agreed-upon criteria of plan quality and allows users to identify specific strengths and areas for improvement.
How this data was created:
This data was created using a content analysis based plan evaluation. First, we developed a set of criteria corresponding to the plan quality characteristics based on existing planning guidance documents. Second, we created a sample of 44 local adaptation plans in the U.S. Third, we systematically read plans to identify whether they fulfill the specified criteria. Assessing the presence/absence of criteria allows the conversion of text to a quantitative measurement of plan quality, which eases comparisons between plans and permits statistical analyses. Finally, we used a grounded theory approach to analyze the text originally coded as strategies to ensure an accurate representation of adaptation strategies proposed by local governments in the sample.
Researchers have used plan evaluation to analyze plans from multiple domains, including hazard mitigation, affordable housing, and sustainability planning. As plan quality evaluation has become more common, consensus has emerged on the core characteristics, or principles, of plan quality. Among these core characteristics are goals, fact base, policies, public participation in the planning process, and details concerning implementation and monitoring. These characteristics are applicable across planning domains and scales (e.g. municipal, county, state). When used effectively, plan quality evaluations can identify specific strengths and review the effectiveness of planning processes, providing the opportunity improve planning in the future. It is important to note that plan quality evaluation and plan quality guidance focus on plans themselves and not the outcomes plans produce.
Coding protocol
We developed a coding protocol to assess seven principles of plan quality: 1) goals, 2) fact base, 3) strategies, 4) public participation in plan creation, 5) inter-organizational coordination, 6) details regarding implementation and monitoring, and 7) how plans deal with uncertainty. To ensure that the protocol captured the most current theory on adaptation, metrics for each principle were based on an analysis of nine adaptation guidance documents published by international, federal, state, and non-governmental organizations. Despite the diversity in guidance documents considered, there was a high level of agreement on adaptation processes and factors that should be considered in climate adaptation planning. From this analysis, we extracted processes and considerations that are present across multiple adaptation guidance instruments and therefore could serve as evaluation metrics. In total, we developed a protocol with 124 metrics across the seven principles. A description of each metric and how it was applied in the coding process are provided in column B and C, respectively.
The metrics pre-tested on eight local adaptation plans from Europe and Australia. The pre-testing allowed us to train the three coders and refine the metrics and coding instructions so that they captured the concepts intended.
Sample Selection
We selected plans to include in our sample based on three criteria: (1) the central topic of the plan was adaptation, resilience, or preparedness; (2) the plan was written by or for a U.S. city or county government; and (3) the plan took a comprehensive approach to adaptation by focusing on more than just one or two topics (i.e., we exclude sector-based adaptation plans). These criteria excluded plans that integrate adaptation components but do not focus entirely on adaptation (e.g., climate action plans and sustainability plans that dedicate a chapter to adaptation), plans that are written by regional entities (e.g., the Southeast Florida Regional Climate Change Compact), and plans that are written without local government involvement (e.g., plans written by local environmental non-profits).
We evaluated all U.S. plans that we could find that met these criteria and were released between 2007, when Keene, NH published the first adaptation plan in the U.S., and 2014. We developed the sample based on a search of three adaptation clearinghouse websites: the Georgetown Climate Center, the Climate Adaptation Knowledge Exchange (CakeX), and the Center for Climate and Energy Solutions. In addition, we collected plans through three 100-page Google searches for the terms “local adaptation plan,” “local resilience plan,” and “local preparedness plan.”
Coding Procedures
Each plan was coded independently by two of the trained coders in line with recommendations from the communications literature on content analysis and recommendations from the plan evaluation literature on methodology. Before coding plans in the sample, we calculated inter-coder reliability to ensure that the coders fell within an appropriate range of inter-coder agreement (0.80 or greater).
Coders used the NVivo version 10 qualitative analysis software package to link coding items with the content of plans. After the coders completed a plan, we compared their quantitative data to identify disagreements on a metric-by-metric basis. All disagreements were discussed and reconciled by referring to the qualitative plan content, and the final, agreed-upon codes were integrated into a master dataset.
We calculated inter-coder reliability scores for each plan and code using two measures: percent agreement and Krippendorff’s alpha reported in column AY for each code and row 149 for each plan. To achieve equal weighting of the codes for each principle, we calculated index scores for each plan principle by summing the reconciled scores for the principle and dividing by the number of codes in that principle. We calculated total plan quality by averaging index scores for each plan principle. These results were presented in Woodruff and Stults (2016).
Strategies
Once all 44 plans had been analyzed, we extracted all of the text related to the type of adaptation strategy and re-analyzed it using a modified Grounded Theory Method. Each author independently reviewed and classified strategies, moving strategies to more appropriate types and creating new types of strategies, if necessary, to more accurately reflect what the local plans were proposing. To be consistent, we coded all strategies based on the action being proposed, not the intent of the action. This meant that if a strategy was a policy change that would incentivize more resilient building codes, we coded it as being a policy strategy and not a building code strategy. By doing this, we were able to code the strategies as presented by the plan authors, avoiding the need to interpret the plan authors’ intent. All differences were collaboratively reconciled by referring back to the adaptation plan and looking for similarities between the strategy in question and other strategies. In the end, we classified each adaptation activity as one of seventeen types. When appropriate, adaptation strategies were co-tagged as multiple types. For example, Baltimore MD’s strategy to “encourage the development of integrated flood protection systems that use structural (engineering) and non-structural (wetlands) measures” was double tagged as being both a physical infrastructure strategy and a green infrastructure strategy. Because coding of the strategies was based on a more qualitative and iterative process, no inter-coder agreement scores are reported for the strategy-type metrics.
Time and date of creation:
Plans were collected and analyzed in 2014 – 2015. No plans released after December 31, 2014 were included in our sample. Results of this work was published in Nature Climate Change in May 2016 and has lead to several subsequent articles.
For additional details and analysis of this data please see:
Woodruff, S. C., & Stults, M. (2016). Numerous strategies but limited implementation guidance in US local adaptation plans. Nature Climate Change. http://doi.org/10.1038/nclimate3012
