Evaluating Winter Orographic Cloud Seeding: Design of the Wyoming Weather Modiﬁcation Pilot Project (WWMPP)

An overview of the Wyoming Weather Modiﬁcation Pilot Project (WWMPP) is presented. This project, funded by the State of Wyoming, is designed to evaluate the effectiveness of cloud seeding with silver iodide in the Medicine Bow and Sierra Madre Ranges of south-central Wyoming. The statistical evaluation is based on a randomized crossover design for the two barriers. The description of the experimental design includes the rationalebehind thedesign choice, the criteria for case selection,facilities for operations and evaluation, and the statistical analysis approach. Initial estimates of the number of cases needed for statistical signiﬁcance used historical Snow Telemetry (SNOTEL) data (1987–2006), prior to the beginning of the randomized seeding ex- periment. Reﬁned estimates were calculated using high-resolution precipitation data collected during the initial seasons of the project (2007–10). Comparing the sample size estimates from these two data sources, the initial estimates are reduced to 236 (110) for detecting a 10% (15%) change. The sample size estimates are highly dependent onthe assumed effect of seeding, onthe correlations between thetwotargetbarriers andbetweenthe target and control sites, and on the variance of the response variable, namely precipitation. In addition to the statistical experiment, a wide range of physical studies and ancillary analyses are being planned and conducted.


Introduction
The concept of modifying supercooled clouds with artificial ice nuclei (IN) was discussed by Findeisen as early as 1938 as part of his seminal paper on ice formation in the presence of supercooled liquid water (Findeisen 1938). In the 1940s, with the discovery of artificial ice nucleation by both dry ice and silver iodide (AgI), the potential for modifying precipitation from supercooled clouds heightened the interest of many researchers (e.g., Schaefer 1946;Vonnegut 1947;Kraus and Squires 1947;Langmuir 1948;Coons et al. 1948;Bergeron 1949). Early on it was recognized that winter orographic clouds might be especially amenable to seeding because of the frequency and persistence of supercooled clouds. Ludlum (1955) was one of the first scientists to present a conceptual model of seeding mountain clouds to enhance snowfall.
Precipitation in winter orographic storms generally develops when ice crystals form on natural IN (typically dust particles) and grow through deposition, riming, and/or aggregation. In many storms, an inefficient precipitation process exists because of the lack of natural IN active at warmer temperatures. Measurements suggest that most layer clouds do not contain much ice until temperatures less than 2128C are reached (e.g., Geresdi et al. 2005). This is attributed to the fact that natural IN are not efficient until in-cloud temperatures fall below about 2128C (Hoose and M€ ohler 2012). Furthermore, the weak updrafts in these layer clouds and narrow cloud droplet distributions limit the impacts of any ice * The National Center for Atmospheric Research is sponsored by the National Science Foundation. multiplication processes (e.g., Hallett and Mossop 1974). As a result, many shallow clouds, especially winter orographic clouds, may be relatively devoid of ice crystals through large regions and thus have an inefficient precipitation process. The absence of ice crystals allows supercooled water to persist for long periods in winter orographic clouds instead of being depleted by vapor diffusion, riming, and their feedback on precipitation formation (increased riming and enhanced aggregation). This fact is well attested to by the measurement of sustained supercooled liquid water (SLW) in orographic clouds by aircraft and ground-based instruments such as radiometers (e.g., Rauber et al. 1986;Huggins 1995). In contrast to natural IN, artificial IN, such as AgI, can be highly efficient at nucleating ice crystals at temperatures as warm as 258C, providing the ability to create ice crystals in clouds warmer than 2128C by ''seeding'' them with an AgI aerosol (DeMott et al. 1995).
Since 1948, there have been numerous research programs showing that AgI seeding could produce additional precipitation in winter orographic clouds (see Huggins 2009 for a summary). Programs in the Rocky Mountains that may be most relevant to winter storm conditions in the mountains of Wyoming include the Climax experiments in the central Colorado mountains (Mielke et al. 1981;Grant 1986), the Colorado Orographic Seeding Experiment (COSE) in the northern Colorado mountains (Rauber and Grant 1986), and the Bridger Range Experiment in southwestern Montana (Super and Heimbach 1983). The Climax program included exploratory and confirmatory randomized seeding experiments, and used existing instruments and observations as covariates and for ancillary (ex post facto) studies. The COSE program employed special airborne and ground-based observations to elucidate the characteristics and evolution of SLW in orographic storms. It focused on how the distribution of SLW impacts precipitation development and the implications for cloud seeding. However, no randomized seeding was conducted. The Bridger Range Experiment deployed a network of snow gauges to show statistically that AgI seeding from a single generator likely enhanced precipitation through the use of a target/control evaluation of the randomized seeding experiment. This project also collected aircraft measurements that established the vertical extent and concentrations of IN in seeding plumes (Super 1974).
Other noteworthy studies include the Sierra Cooperative Pilot Project and the Snowy Mountain study in Australia. The Sierra Cooperative Pilot Project showed through detailed case studies with observations and modeling that dry ice and AgI seeding likely caused additional precipitation over the Sierra Nevada Range (Deshler et al. 1990;Reynolds 1988). This project, however, did not conduct a randomized experiment and statistical evaluation. The Snowy Mountain randomized cloudseeding program in Australia has provided recent evidence of an increase in precipitation because of AgI seeding of winter orographic clouds based on a 5-yr statistical program (Manton and Warren 2011). Only limited physical evidence was collected as part of this program. Few of these programs have combined both statistical and physical evidence of the quantitative impact of AgI seeding on wintertime orographic precipitation. However, these well-controlled field experiments suggest that orographic seeding has the potential to enhance precipitation under certain well-constrained conditions. Results from programs such as these, coupled with an extended drought in the western United States, led the State of Wyoming to fund a winter orographic precipitation enhancement study, called the Wyoming Weather Modification Pilot Project (WWMPP).

WWMPP
In response to requests from the Wyoming Association of Conservation Districts and other stakeholders, the Wyoming State Legislature through the Wyoming Water Development Commission (WWDC) funded a feasibility study in 2004 to examine the potential to increase winter orographic precipitation through cloud seeding (Weather Modification Inc. 2005). Two regions of the state, the Wind River Range in west-central Wyoming and the Medicine Bow-Sierra Madre Ranges of south-central Wyoming, were selected based on past studies by the University of Wyoming. These studies had documented the occurrence of SLW and hence the potential for seeding orographic clouds (e.g., Auer and Veal 1970;Dirks 1973;Politovich and Vali 1983). Furthermore, these areas are important to the generation of spring streamflow in the Green, Wind-Bighorn, and Platte River basins (States West Water Resources Corporation 2001;BRS Inc. 2003;TriHydro Corporation 2006).
Subsequently, in 2005, the WWDC approved and the state legislature funded a 5-yr program to test this potential. A key component of the program was the inclusion of a substantial evaluation effort independent of the cloud-seeding operations. A team of scientists and statisticians at the National Center for Atmospheric Research (NCAR) was selected for the evaluation component, while Weather Modification, Inc. (WMI), was chosen to conduct the cloud-seeding operations. The independence of these two aspects of the program allows the seeding effect to be evaluated by an entity without any vested interest in future operational cloud seeding. At the end of 2007, after initial measurements were collected FEBRUARY 2014 and permissions for facility sites were obtained, the Medicine Bow and Sierra Madre Ranges were established as targets for a randomized wintertime cloudseeding experiment. The final plan for the randomized seeding experiment (RSE), in the form of a crossover design using ground-based seeding generators over an annual seeding period from 15 November through 15 April, commenced during the 2008/09 season (NCAR 2008). Because of required data collection, design changes and refinements, and additional instrument deployments, the Wyoming legislature extended funding beyond the original 5-yr appropriation to obtain a sufficient number of experimental units to reach statistically significant conclusions for the WWMPP. The field program is now scheduled to conclude in the spring of 2014.
The American Meteorological Society (AMS) Statement on Planned Weather Modification Through Cloud Seeding (AMS 2011) emphasizes the need for a comprehensive approach to cloud-seeding experiments that includes both a statistical and physical evaluation. The physical effects of glaciogenic seeding of winter orographic clouds have been fairly well documented (Super and Heimbach 1983;Reynolds 1988;Super and Boe 1988;Deshler et al. 1990;Reinking and Martner 1995;Huggins 2009). This is understated in the AMS statement. More recent aircraft measurements collected during the initial years of the WWMPP (Geerts et al. 2010) further elucidate the physical effects of seeding. Numerical modeling capabilities of simulated seeding effects are progressing rapidly (e.g., Xue et al. 2013a,b) and will play a key role in understanding the results of the WWMPP. While a major part of the WWMPP is focused on a randomized statistical experiment, the project also encompasses efforts to measure and model the physical effects of seeding. The overall goal of the WWMPP is to provide the best advice possible to the WWDC regarding the potential for meaningful augmentation of snowpack through cloud seeding. As a first step, this paper provides an overview of the WWMPP, focusing on the design and implementation of the statistical evaluation of the RSE.

Seeding procedures and facilities
a. Seeding concept and the definition of a case The following chain of events is conceptualized for seeding winter orographic clouds containing SLW with ground-based generators: This concept is not new, having been outlined in the very early work mentioned previously and illustrated schematically in several studies (e.g., Grant 1986;Rauber and Grant 1986;Huggins 2009). The annotated photograph in Fig. 1 schematically highlights some of the seeding concepts important to the WWMPP. The WWMPP target ranges are inset as a coarse map in Fig. 1. The final conceptual stage of enhanced snowfall on the target area is into the page of Fig. 1 and hence not visible in the photograph (and not labeled).
Based on this seeding concept, seedable conditions require clouds with SLW at temperatures cold enough for the AgI nuclei to be effective IN and with winds that allow the AgI plume(s) to affect the clouds such that augmented precipitation will fall on the mountain in the target area. Since the elevation of the target areas is approximately 3000 m MSL and precipitating clouds would need to be at least that high, the reference level for seedable conditions was chosen to be 700 hPa (;3000 m). The ground-based generators were located based on the prevailing upslope winds such that wind directions between southwest (SW) and northwest (NW) (specifically from 2108 to 3158) will carry the AgI IN into clouds that might affect precipitation in the target areas. AgI IN are effective as warm as 258C but are much more effective at temperatures colder than about 288C. Therefore, the following criteria were established for seedable conditions in the WWMPP: SLW present, 700-hPa temperatures #288C, and 700-hPa winds from SW to NW.
The length of time seedable conditions can exist is quite variable, evident both in storm period and in precipitation intensity. Several studies have documented the typical variability and persistence of SLW (Heggli and Rauber 1988;Long et al. 1990;Long and Huggins 1992;Rauber and Grant 1987;Sassen et al. 1990). Limited observations in 2006 from a radiometer, snow gauges, and airborne instruments (NCAR 2008) showed SLW periods and precipitation periods of 4 h and more to be fairly common in the Medicine Bow Range. A 4-h seeding period is short enough to minimize variations in seeding conditions, within cases and across targets, yet long enough to ensure an opportunity for the clouds to respond to seeding and to measure the response. Therefore, for the RSE, a seeding period of 4 h was decided upon based on the past studies and the early observations collected in the WWMPP. Other randomized programs have used similar time periods (6 h) for their statistical analysis (Super and Heimbach 2009;Manton and Warren 2011).
Various studies and overviews have reported precipitation rate increases from seeding over a range of about 0.1-1.0 mm h 21 (Reynolds 1988;Super and Heimbach 1988;Super and Boe 1988;Super 1999). This range of values suggests that over a 4-h period, an increase in precipitation of 0.4-4.0 mm might be expected as a result of seeding. The resolution of the snow gauges (;0.1 mm) implies that seeded differences of this magnitude are measurable.
The transport and dispersion of AgI IN is an important consideration. This is being addressed in the WWMPP through ground-based IN observations, airborne IN observations, trace chemistry analysis of snow samples, and numerical modeling with seeding included (e.g., Geerts et al. 2010;Xue et al. 2013a,b). Of particular concern is experimental contamination-when seeding material may inadvertently affect precipitation in an area considered to be unseeded. To help guard against experimental contamination, a buffer period following seeding can be established to allow AgI IN to clear the target areas before allowing another seeding period to begin. Surface observations of AgI IN in the Medicine Bow target area, collected in 2007-08 with an acoustic IN counter (Langer 1973;Heimbach et al. 2008;Super et al. 2010), indicated that for some conditions, experimental contamination of the Medicine Bow target occurred up to 4 h after seeding in the Sierra Madre Range. Therefore, a 4-h buffer period was chosen for the RSE.
In a randomized seeding experiment involving two alternating barriers, such as in a crossover design, seedable conditions must exist over both barriers for the specified seeding period. Thus, for the WWMPP, when the criteria for seedable conditions were met over both the Medicine Bow Range and the Sierra Madre Range, a 4-h RSE case (the experimental unit) was declared and seeding began on one of the barriers, selected randomly (equal  chance for either barrier). The randomization procedure consisted of a predetermined list of seeded barriers selected randomly, but limited to no more than four seeding cases in a row on the same barrier. It was felt that a longer string of cases might result in a seasonal bias of seeding decisions for a particular barrier. The seeding decision for a declared case was accessible only to the WMI operational personnel, those responsible for starting and stopping the generators. This information has subsequently been carefully guarded so as not to influence the forecasters/case callers or the evaluation team, particularly those responsible for quality control of the precipitation data.

b. Procedure for case declaration
Project meteorologists (WMI personnel) monitor weather conditions over the target ranges constantly (24 h per day, 7 days per week) to be ready for any seeding events. General weather information was obtained primarily from Internet sources such as satellite imagery, synoptic surface reports, and visual observations of the ranges (during daylight hours). Expectations are honed by numerical model output, especially that from the Weather Research and Forecasting (WRF) model, which was run specifically for the WWMPP by NCAR. The WRF model is nested down to 2 km over most of Wyoming and is nudged with observations every 3 h using real-time fourdimensional data assimilation (RT-FDDA), producing a 24-h forecast every 3 h. This modeling system is described by Liu et al. (2008).
As low clouds develop over both ranges, microwave radiometers sited to view each mountain range detect liquid water (LW) along the radiometer sensing path/ beam. For the cold conditions in the RSE, LW is assumed to be SLW. These observations are reported in real time and available via a secure link through NCAR. When LW is detected by both radiometers, and satellite imagery and/or visual observations indicate that orographic cloud is fully covering both ranges, the SLW criterion is considered satisfied. The 700-hPa temperature and wind direction are initially obtained from prognostic model output, particularly the WRF RT-FDDA, which is also helpful in estimating the extent and duration of SLW over the barriers. A few examples of graphical output from the model used by the project meteorologists are depicted in Fig. 2. When these forecast criteria are satisfied, a radiosonde is released from Saratoga. If the temperature and wind direction are within the criteria established for the RSE (i.e., 700-hPa temperature #288C and wind direction 2108-3158), a case is declared. If not, declaration is delayed until conditions change and another sounding is released. This process is illustrated in the decision tree of Fig. 3.

c. Facilities used for the WWMPP
A site map of the RSE target barriers, the Medicine Bow and Sierra Madre Ranges in south-central Wyoming, and the facilities deployed for the WWMPP (sounding station, radiometers, ground-based generators, and precipitation gauges) and facilities generally available [Natural Resources Conservation Service Snow Telemetry (SNOTEL) sites] is shown in Fig. 4. The radiometers and sounding station supplement other observations and numerical models to declare cases. The generators disperse the AgI IN during seeding operations. The precipitation gauges are used for both seeding evaluation and for controls in the statistical analysis. The SNOTEL sites provide a precipitation climatology and contribute to quality control of the high-resolution WWMPP precipitation gauges.

1) SOUNDING SITE
The sounding site (magenta-colored circle in Fig. 4) is located midway between the Medicine Bow and Sierra Madre Ranges in the town of Saratoga. Soundings using Vaisala, Inc., GPS sondes are taken by WMI staff for each storm to provide information on the environmental conditions and to determine whether the storm qualifies for seeding based on temperature and wind criteria.

2) RADIOMETER SITES
The two radiometer sites (inverted purple triangles in Fig. 4) are upwind of each of the ranges. The microwave radiometers consist of a 2-channel Radiometrics Corporation WVR-1100 series, west of the Medicine Bow Range, and a 5-channel Radiometrics WVP-1500 series, west of the Sierra Madre Range. Both radiometers scan to low elevation angles to intercept clouds forming over the ranges. The radiometer data are transferred over Internet connections every 10 min. The primary variable of interest measured by the radiometers is liquid water path, which is calculated in real time and displayed on the project Internet page. Figure 5 shows photographs of an AgI seeding generator as well as a precipitation gauge site (discussed in the next section). Super and Heimbach (2005) have recently summarized studies on generator siting issues. Issues relevant to the current study are storm direction and precipitation trajectories, generator spacing, and optimizing the seeding solution for the temperature range of interest.

(i) Storm directions and precipitation trajectories
The distributions of storm directions and precipitation trajectories are needed to determine generator sites that will treat the majority of seedable precipitation events. Upper-air observations are sparse in southern Wyoming, which is about equidistant (230-250 km) from the Denver (DNR) and Riverton (RIW) sounding sites. When the 700-hPa winds from RIW were examined in the feasibility study (Weather Modification Inc. 2005), the wind direction distribution during 2003-04 precipitation events in the Sierra Madres was clearly peaked in the southwest-northwest quadrant. Given that sparse upper-air wind observations from one year may not be particularly representative of precipitation trajectories in the southern Wyoming mountains, output from WRF model simulations for about 200 precipitation events were investigated. Periods of snowfall were identified from SNOTEL data in the target area in each mountain range, which then defined the time and coordinates to initiate back-trajectory calculations using the WRF model output. Figure 6 shows the results of back-trajectory calculations for precipitation events in the Sierra Madre Range over the 2005/06 (blue) and 2006/07 (red) winters. The crosses represent the beginning points for trajectories with 30-min of air parcel travel before ending at the SNOTEL site in the target area (one trajectory per event/ storm). Observations and modeling studies (e.g., Prasad et al. 1989;Huggins 2007) indicate that a 30-min time period represents the minimum amount of time necessary for precipitation to form, and thus the trajectory starting points should be somewhat downwind or east of the generator locations, as observed for the majority of the trajectories in Fig. 6 (except for those storms with easterly winds). Similar results are shown for the trajectories that end at a SNOTEL site during precipitation events in the Medicine Bow Range (Fig. 7). The trajectory starting points should be close to the ground level if they are to represent plumes from ground-based seeding generators. The majority of the points are close to the ground: 75% are below 200 m AGL for the Sierra Madre events and 90% are below 100 m AGL for the Medicine Bow events. Overall, the modeling study showed primarily westerly trajectories in agreement with the RIW upperair data. The trajectories also showed that winds were sometimes too strong for effective targeting from the initial complement of AgI generators (deployed in late 2006), which were located close to the barriers at high elevations (blue sites in Figs. 6 and 7). These results motivated the addition of generators farther upstream in 2007 (yellow sites in Figs. 6 and 7).

(ii) Generator spacing and targeting
A number of studies (e.g., outlined in Super and Heimbach 2005) suggest that horizontal mixing results in a plume spread of 158-208 several kilometers downwind of a source, which translates to roughly a 5-10-km width at 15-30 km downwind. Estimating the coverage of the target areas by the AgI plume was further investigated using a transport and diffusion model called Second-Order Closure Integrated Puff (SCIPUFF; Sykes and Gabruk 1997) for a number of the WRF model case studies in 2006 and 2007. The results of these studies, though limited in number, confirmed the results discussed by Super and Heimbach and suggest 10 km as being optimal for the WWMPP generator spacing. The spacing of the generators in the Sierra Madres is close to this guideline, being #10 km apart for the generators 18-22 km upwind. However, as seen in Fig. 6, precipitation events often occurred with relatively strong winds and thus air parcel trajectory starting points west of the initial complement of generators. Additional generators were added 28-31 km upwind to better cover these strong-wind events. The Medicine Bow generators have somewhat larger spacing than in the Sierra Madres, as much as 15 km apart, for generators 17-23 km upwind. The strong-wind events seem to be more northwesterly in the Medicine Bow Range (Fig. 7), so two additional generators were added 29-30 km upwind as well as one to fill a gap to the southwest. This generator placement covers mean wind speeds up to about 18 m s 21 . The

288
generators are listed in Table 1, which shows that the generators lie within about 850 m (elevation) of the target sites. The coverage of wind direction and speed are comparable between the two ranges. The generators are remotely operated and monitored via satellite communications by WMI technical staff.

(iii) Seeding rate and effectiveness
The seeding rate of the ground-based generators was set at approximately 25 g h 21 , based on past experiments and laboratory work (e.g., Super 1999;DeMott 1997) and as commonly practiced and described in the American Society of Civil Engineers (ASCE) standards manual (ASCE 2004). This seeding rate also attempts to maximize the effectiveness of the seeding plume at relatively warm temperatures (from 268 to 288C).
The activation of AgI as an ice nucleus is strongly temperature dependent. This is the case for the solution being used by WMI in Wyoming (Table 2), which is very close to a formulation tested in the Colorado State University (CSU) Simulation Laboratory (DeMott 1997) and used by the North Dakota Atmospheric Resource Board (ARB) (D. Langerud 2005, personal communication). The CSU tests of the ARB seeding solution show that the number of ice crystals produced increases by almost two orders of magnitude as temperature decreases from about 268 to 288C (Fig. 8). Although the specific WMI solution has not been tested, the close similarity of the two formulations suggest that they likely exhibit similar performance. Both formulations produce nuclei that function via the condensation-freezing mechanism (DeMott 1995;Finnegan 1998).
Both AgI solutions have two important advantages over the rather pure AgI aerosol used during earlier projects: 1) they produce higher ice crystal concentrations FIG. 4. Site map of the WWMPP randomized experiment in the Medicine Bow and Sierra Madre Ranges in Wyoming. Topography is color coded (scale at the bottom) and facilities are denoted with color-coded symbols. These include the SNOTEL sites (in or near the ranges), the sounding site, the two radiometer sites, the groundbased seeding generators (see Table 1), and the high-resolution precipitation gauge sites (see Table 3). at the warm end of the effective temperature range, and 2) they produce much faster nucleation through condensation freezing by the hydrophilic AgI-NH 3 I complex than is the case with the slower contact-freezing nucleation from pure AgI (Finnegan 1998). It is important to initiate seeded crystals as soon as practical to maximize their growth times while they are transported over mountain barriers.

4) PRECIPITATION GAUGE SITES
To statistically evaluate the effects of seeding, precipitation gauge sites were located in each of the target areas along with gauges at control sites, which are always unseeded. While they are referred to as controls, precipitation from these sites should more properly be called covariates. While recognizing this inconsistency, we have kept the reference to controls to be consistent with other program documentation. In the Medicine Bow Range, the target site (black square labeled GL in Fig. 4) is located near the Brooklyn Lake SNOTEL, within a U.S. Forest Service experimental site called the Glacier Lakes Ecosystem Experiments Site (GLEES; Musselman 1994). In the Sierra Madre Range, the target area precipitation gauge site (black square labeled HY in Fig. 4) is near Highway 47 between the Old Battle and Webber Springs SNOTEL sites. Two control sites, one upwind and one crosswind, are also associated with each target range. These are denoted by black squares in Fig. 4, labeled for Barrett Ridge (BR) and Chimney Park (CP) in the Medicine Bow Range, and Sandstone (SS) and Elk River (ER) in the Sierra Madre Range. Table 3 lists the precipitation gauge site coordinates and elevations. The deployment of precipitation gauges at the HY site is shown in the photograph of Fig. 5. Data from the nearby SNOTEL sites will be used to help evaluate the quality of the project precipitation gauge data over a period of a day or longer.
All but one of the precipitation gauge sites include three high-resolution precipitation gauges with surface meteorological sensors (temperature, relative humidity, and wind). One site, ER, has only two precipitation gauges. In some seasons, the GL site, as the only field measurement location with access to commercial power, has had particle disdrometers, hotplate sensors, a microwave radiometer, a K-band radar, and snow-depth sensors deployed to support physical studies. Initially, two  Fig. 6, but for the Medicine Bow Range, where the back trajectories are initiated from the Brooklyn Lake SNOTEL, marked with a black star. 290 precipitation gauges were located at each site for redundancy and to minimize measurement variance, but the difficulty of measuring snowfall led to the addition of another gauge at all but the ER site. These difficulties included partial capping of gauges resulting from buildup of snow on the orifice or inside walls of the container, uncompensated temperature dependencies of the transducers, and electrical noise. Comparing data from three gauges makes it easier to identify an errant gauge with poor data. Three types of precipitation gauges have been used in the RSE, but all sites now have ETI Instrument Systems, Inc., NOAH II and GEONOR, Inc., T-200B gauges deployed.
Undercatchment of snow falling into the gauges because of winds, as identified by Rasmussen et al. (2012) at the exposed Marshall field site, is not as much of an issue for the well-protected forested sites in the WWMPP. Wind speeds at the gauges were seldom over 2 m s 21 . The precipitation data used for the statistical analysis will be rechecked for calibration changes and periodic field tests, corrected and/or filtered as per World Meteorological Organization standards, and bias corrected if consistent differences remain. The processed and bias-corrected gauge data will then be averaged for each site, resulting in one precipitation value to represent the measurements of all the gauges at each site.

Statistical evaluation a. Experimental design-Pooled versus crossover
Initially, it was assumed that each of the target areas (Sierra Madre and Medicine Bow Ranges) would be treated independently with regard to seeding decisions, and the results would be pooled for the statistical evaluation. However, a number of factors suggested that storm conditions and precipitation would be far from independent in the two ranges. The Medicine Bow and Sierra Madre Ranges are of similar size, located relatively close together, and have similar elevations (Fig. 4). Annual precipitation amounts are similar, and an analysis of daily SNOTEL data over 28 years showed a correlation of about 0.5 between the ranges for all storms. This correlation made pooling of the data more problematic since many cases in the two target ranges would not be independent. On the other hand, the correlation between the two areas provided the opportunity to use a more efficient crossover approach (Gabriel 1999).
The strength of a crossover design, which requires two target areas, is that it produces paired data and is thus more efficient than pooled data at decreasing sample size, just as a paired t test requires many fewer data than a t test applied to two independent samples. Gabriel (1999) showed for example that the variance of a ratio test statistic is reduced by a factor of 1 2 t for a crossover design versus a single target design, where t is the correlation between target areas. Moreover, the sample size needed for an assumed statistical significance is directly related to the variance. Since the correlation between the target areas in the two ranges is on the order of 0.5, or possibly higher given that experimental cases would be selected based on similar meteorological conditions, the number of cases needed to achieve statistical significance would be reduced by a factor of 2 or more in the crossover design relative to the single target design with pooling of two target areas.
The requirements for confidence in a statistical test of precipitation enhancement for the WWMPP were set at a one-tailed statistical significance level of 0.05, to control the possibility of false positive results, and a statistical power of 0.8, to control for false negative results. Given these values of significance and power, the arguments outlined above, the constraints of a limited number of seeding experiments per season, and a total project period limited to 5-6 yr, the decision was made to develop a randomized crossover design. The other approach, to use a single target design, could require 12 yr or more of data (Gabriel 1999), a daunting and unfundable task. The randomized crossover design has been used in numerous weather modification studies and has been documented extensively in the weather modification literature (Moran 1959;Mielke et al. 1981Mielke et al. , 1982Heimbach and Super 1996;Gabriel 1999Gabriel , 2002List et al. 1999;Super 1999). The details of this design are repeated here as they apply to the WWMPP.
b) Randomized crossover design applied to the WWMPP

1) DEFINITION OF THE RESPONSE VARIABLE
In a statistical experiment, the response variable is the variable being investigated for the effects of a ''treatment.'' The response variable for the Medicine Bow and Sierra Madre targets is the 4-h accumulation of liquidequivalent precipitation in each target area as measured by high-resolution precipitation gauges. Subsequently, this will be referred to as simply the precipitation. These measurements are used in the statistical tests that will assess the effectiveness of the seeding actions. The 4-h period of precipitation, during which the effects of seeding will be considered, starts 30 min after seeding begins and ends 30 min after seeding ceases. The 30-min delay is to allow time for the AgI to reach the cloud and eventually the target area, as discussed in section 3c. Also, 30 min is about the lower limit for seeding to impact precipitation development and fallout (e.g., Prasad et al. 1989;Huggins 2007).

STATISTICAL TEST
In general, a test statistic estimates whether there is a change in the response variable between treated and untreated cases. A more useful test statistic also provides an estimate of any change that may have occurred for the seeded cases compared to the unseeded cases. A useful and appropriate test statistic for the WWMPP design is the ratio test. There are various forms of the ratio test, which have been described in Gabriel (1999). The ratio test statistic used for the WWMPP is outlined below.
The ratio test statistic called the root regression ratio (RRR) will be used to compare precipitation during seeded and unseeded events (Gabriel 1999;List et al. 1999). To understand the RRR and its power, we begin with the simple situation of a single target with no controls. Then the estimate of the seeding effect is the single ratio (SR) of the mean precipitation measured during seeded and unseeded events: where u i is an indicator (or ''switch'') equal to 1 during seeded events and 0 during nonseeded events, and y i is the precipitation amount for event i. This equation is equivalent to forming the ratio of the average precipitation during seeded events at target Y to the average precipitation during unseeded events at target Y (y seeded /y unseeded ). A ratio greater than 1.0 would suggest seeding has a positive effect. For experiments where large differences are expected between treated and untreated populations, with small variances, the single ratio test might be appropriate; however, for cloud-seeding experiments, variances can be high, effects small, and cases limited. To account for and reduce some of the variability, control gauges can be used. Ideally, control gauge sites are located near the target gauge site to maximize their correlation but are unaffected by seeding actions. In theory, control sites can measure any number of variables related to conditions at the target, but in the RSE, as in most other seeding experiments, only precipitation  The use of a control gauge site provides a mechanism for reducing the natural variability by calculating a SR for conditions at the control gauge site(s) for the seeded versus unseeded time periods, in spite of the fact that the control gauge site is not affected by seeding. This allows the SR of the control gauge site to scale the SR at the target gauge site, reducing the effects of natural precipitation variability. Following Gabriel (1999), the resulting statistic is referred to as the double ratio (DR): The effect of the double ratio is to normalize any change in SR Y (target) with SR Y (control), which captures any natural differences that occurred by chance during the same two sets of cases, seeded and unseeded. Equation (2) can be generalized to include multiple control gauge sites.

(i) Crossover design without control gauge sites
For a crossover design with two targets (Y and X) and no control gauge sites, during each case, one target area is randomly selected for seeding. When the experimental results are evaluated, the SR is calculated for each target. These ratios are combined as shown in Eq. (3) to create a statistic called the root double ratio (RDR): The precipitation values at the two targets for case i are denoted by x i and y i . For this ratio test statistic, u i 5 0 when target X is seeded and u i 5 1 when target Y is seeded. With this notation, an increase in the SR because of seeding at one target would result in an increase of the reciprocal of the SR at the other target. Thus, the overall measure of the seeding effect for the crossover design is the geometric mean of SR Y and 1/SR X . Because the measures of interest are ratios rather than simple scalar values, it is more appropriate to use a geometric mean rather than an arithmetic mean. With these definitions, the RDR can be restated as the geometric mean of the product of ratios representing the precipitation on the seeded barrier compared to the precipitation on the unseeded barrier. For example, when X is seeded (e.g., u i 5 0) the only terms contributing to the RDR are the denominator of the first term in Eq. (3), y i (which is unseeded), and the numerator of the second term, x i (which is seeded). When Y is seeded (e.g., u i 5 1), the contributions are switched but the seeded precipitation remains in the numerator as before.
(ii) Crossover design with control gauge sites When controls are used in a crossover design, precipitation recorded at the control gauge sites is used to predict the naturally occurring differences in measured precipitation at the two targets. This approach compensates for the fact that during any case, regardless of the seeding decision, one target is likely to receive more snowfall than the other simply because of environmental factors and random variability.
When dealing with multiple gauge sites, the data are more easily combined if the values are first scaled or standardized so that measurements from all of the gauge sites have the same mean. This standardization is achieved by dividing the values measured at each site by the mean value for the site (i.e., mean precipitation for all the cases), irrespective of seeding, resulting in relative values. As a result of this operation, the mean of the relative values for each gauge site is 1. A tilde is used to denote relative values; for example,ỹ i is the relative value of a measurement at gauge site Y for case i.
Data collected at the control gauge sites are used to create a statistical model that predicts the difference in measured precipitation between the targets. Note that this model does not use information about seeding actions. Intuitively, if a difference in precipitation can be predicted from data measured at the control gauge sites, that difference should not be attributed to a seeding effect. Removing this predictable difference reduces some of the natural variability in the precipitation, which makes seeding differences easier to detect. This adjustment can be accomplished by estimating the coefficients, b 1 and b 2 , which minimize the value of the squared error: whereỹ i andx i denote relative precipitation values measured at the two target gauge sites (Y and X) andz 1i andz 2i denote relative precipitation values measured at the two control gauge sites (Z 1 and Z 2 ). This results in a multiple regression equation that relates the differences in relative precipitation at the two targets to the relative precipitation at the two controls (Gabriel 1999). If the precipitation values from the control gauge sites are not correlated with the precipitation values at the target gauge sites, the coefficients b 1 and b 2 , which represent weights of the control gauges, will nearly equal zero.
This information is used to scale the RDR to provide the RRR: The true but unknown population parameters b 1 and b 2 are estimated by b 1 and b 2 [Eq. (4)]. Essentially the square root term (the denominator) adjusts RDR (the estimated seeding effect) according to information from the controls, which is weighted by the strength of the relationship between the precipitation differences at the control and target gauge sites. While Eq. (5) assumes that there are two control gauge sites, the equation can be expanded to include more control gauge sites. This RRR test statistic is used to perform the null hypothesis testing described in the next section.

3) STATISTICAL HYPOTHESIS
A RRR value of 1.0 indicates that there is no difference between seeded and unseeded populations. Thus, it was decided in this study to evaluate the null hypothesis for the ratio test statistic H0: RRR 5 1.0. Since we are interested in whether seeding leads to an increase in precipitation, the alternative hypothesis is HA: RRR . 1.0.
The evaluation of the p value, the probability that the null hypothesis is correct, for the WWMPP will be done using a rerandomization procedure as recommended in Tukey et al. (1978). The rerandomization procedure (also known as a permutation test) is a nonparametric test, meaning that it does not make any distribution assumptions. In theory, the value of the test statistic is calculated under all possible permutations of the seeding decision-that is, every possible ordering of the seeding decision while maintaining the same number of cases-and the resulting p value is the proportion of test statistics that exceed that found by the experiment. In practice, it is unrealistic to calculate all possible values of the test statistic; therefore, the seeding decision is instead permuted a sufficiently large number of times (in our case, 100 000). For statistical significance, the p value must be less than the chosen significance level.

c. Cases required for statistical significance
A key element of the statistical design for the WWMPP is the determination of the required sample size to achieve statistical significance. The test had to be chosen such that useful results could be achieved within the expected sample sizes obtainable in a 5-6-yr project. Sample size is estimated from 1) the desired significance and power for the statistical test, 2) the expected difference in precipitation between the seeded and unseeded cases, 3) the correlations among the precipitation amounts at the target and control gauge sites, 4) the precipitation variance at each of the gauge sites, and 5) the experimental design applied.

1) SIGNIFICANCE AND POWER
As mentioned in section 4a, the one-tailed significance level chosen for the statistical test of the WWMPP is 5%. This level indicates that if the null hypothesis is rejected, there is less than a 5% probability that this conclusion was reached by pure chance (i.e., because of random variations in the samples) rather than as a result of a treatment effect. The power of the test is chosen to be 80%, implying there is an 80% chance that a change of the assumed magnitude will be detected. As in most scientific experiments, particularly those in the weather modification field, the emphasis is placed on not making an incorrect positive conclusion, while ensuring that enough samples are taken to detect a significant change.

2) EXPECTED INCREASE IN PRECIPITATION
A number of studies involving both randomized experiments and evaluations of operational programs in similar winter environments to the WWMPP have indicated seasonal increases of precipitation of 5%-15% (Weather Modification Association 2011). For example, these include the Lake Almanor experiment (Mooney and Lunn 1969;Warburton et al. 1995), the Climax experiments (e.g., Grant 1986), the Bridger Range Experiment Heimbach 1983, 1988), and the operational Utah program (Griffith et al. 2009). However, results from only select cases seeded during randomized experiments, which occurred under conditions affecting only a fraction of the seasonal snowfall, have at times shown substantially larger increases. Recently, after the WWMPP design was established, results of the Snowy Mountain experiment in Australia were published that showed an increase of 14% when tested on the covariate of seeding generator hours greater than 45, signifying well-seeded cases (Manton and Warren 2011). Therefore, for the sake of estimating sample sizes that would be required to reach a definite conclusion, reasonable precipitation increases because of seeding are estimated to range between 10% and 15%.

3) CORRELATIONS BETWEEN GAUGE SITES AND VARIANCE AT EACH GAUGE SITE
The effectiveness of the crossover design is contingent on the correlation between the two target areas and between the local control and target areas (those in the same mountain range). Ideally, high-resolution gauge data at each of the sites would have been collected over several winter seasons to determine the correlations as well as the precipitation variance; however, this was not possible for the current program. Without these data, daily snow water equivalent (SWE) accumulations measured at SNOTEL sites were used to estimate the correlations between the two target sites (the two mountain ranges) and between control and target sites in the same range. The SNOTEL data were processed similarly to those used in the study of Ikeda et al. (2010).
Comparing the Old Battle SNOTEL, which is in the target area of the Sierra Madres, with the Brooklyn Lake SNOTEL, which is in the target area of the Medicine Bows, shows that the target areas are correlated at about 0.5 (as mentioned in section 4a.). Correlations between controls and targets are more difficult to establish with SNOTEL data. In the Sierra Madres, the WWMPP control gauge sites are nearly collocated with SNOTEL sites (see Fig. 4). However, in the Medicine Bows, there are no SNOTEL sites representative of the control gauge sites. They are either quite far from the control site or at a significantly different elevation. Nonetheless, correlation values were calculated for all SNOTEL sites in each range in comparison with the target SNOTEL and are graphically represented in Fig. 9. Overall, there is a slight trend toward a decrease in correlations with distance, but it is quite variable. Obviously, some of that variability is due to differences in elevation, which is also an important factor in snowfall distribution. Using the SNOTELs either near the control sites in each range or interpolated to the control sites suggests that target-control correlations are roughly 0.5-0.7. These values are consistent with a study using monthly totals from a precipitation gauge network in the near vicinity of the target area in the Medicine Bows (Choi and Tung 1990).
Another caveat with this approach is that the SNOTEL dataset of daily precipitation amounts could be quite different than one with 4-h amounts under matched conditions in both ranges. It is likely that the correlations (both target range to target range and target to control in each range) would be higher during experimental conditions than from those based on daily data.
The variance used in the sample size calculations is based on 104 days of high-resolution precipitation data collected at the GLEES site during the 2006/07 winter season (prior to the start of the randomized seeding experiment). These data were aggregated into 4-h bins and the relative variance was calculated using the nonzero values. The estimated relative variance is 1.14.

4) SAMPLE SIZE ESTIMATE
The following equation can be used to estimate sample size n RRR for the root regression ratio design (Gabriel 1999): where s 2 is the relative variance at the gauges, t is the correlation between the target sites, k is the correlation between the target site and the nearby control (same FIG. 9. Color-coded correlation of daily SWE between the SNOTEL sites in each range and the reference SNOTEL site in each target area, indicated by the brown circled crosses: Old Battle in (a) the Sierra Madre Range, and Brooklyn Lake in (b) the Medicine Bow Range. WWMPP control gauge sites are denoted by the black diagonal crosses. Distance scale is applicable to both panels. SNOTEL data from 28 seasons  were used for the correlation calculations.
FEBRUARY 2014 mountain range), g is the correlation between the target site and the far control (opposite mountain range), Q a and Q b are the ath and bth quantiles of the normal distribution, and d is the proportion increase in precipitation. The Q a and Q b values are based on desired statistical properties, where a and b are the significance level and the power, respectively. For the WWMPP, a 5 0.95, Q a 5 1.645, b 5 0.80, and Q b 5 0.842. Equation (6) is based on a schema that has additional assumptions: correlations between the target sites in each range and between the control sites in each range are the same (and equal to t), the correlations between the target site in each range and the control site(s) in the same range are all the same (and equal to k), and the correlations between the target site in one range and the control site(s) in the other range are the same (and equal to g, estimated to be 0.5 for the WWMPP).
Using Eq. (6) with the estimates and assumptions described above, the number of samples needed to detect a 10% (15%) change in precipitation amount is 776 (361). However, if we assume that the experimental design, which matches meteorological conditions in each range, increases the correlation between the targets to 0.6 and the correlation between the target and nearby control to 0.8, then the required sample size decreases to 272 (126) for detecting a 10% (15%) change. Clearly, the correlations between the target sites and between the target and nearby control sites have a large effect on the estimated sample sizes.
The number of cases that might be expected during a normal winter season is difficult to estimate given the lack of prior data appropriate for assessing the seeding criteria. As with the correlations, a coarse estimate can be obtained using precipitation from historical SNOTEL observations (1987)(1988)(1989)(1990)(1991)(1992)(1993)(1994)(1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002)(2003)(2004)(2005)(2006) in this case) as a proxy for seeding conditions. The average number of days in one season with the minimum resolved amount of precipitation (.0.25 mm) in both target areas was 124. This is an upper limit of potential cases since the RSE time period, winds and temperature thresholds were not considered. Another method, based on 3½ months of only one season (2006/07), used archived WRF model output to estimate cases. Cloud liquid water over both ranges (roughly of similar magnitude and extent) was required, 258C (at 700 hPa) was used for the temperature threshold-prior to establishing the 288C threshold-and perfect forecasting (since the analysis was done in hindsight) was assumed for the start of a 4-h case period. Extrapolating to a whole season (5 months) yielded an estimate of 65-70 cases per season for the 2006/07 conditions, which was a season with close to average snowfall. Although this is about half of the SNOTEL estimate, it is likely an overestimate of expected cases per season. Nonetheless, this estimate was used to conclude that a 5-yr experimental period should detect a change of 15% and probably a 10% change.

5) COMMENTS ON SAMPLE SIZE ESTIMATE
Since our estimates of sample size were based on variance and correlations from data not ideally suited to the experimental design, refining the estimates using precipitation data collected during the RSE can help verify our a priori estimates of sample size needed for statistical significance. The caveat is that statistically this is not strictly proper since the data are not independent of our study. However, the refined estimates are used only to assess our initial estimates, and do not affect the design, operations, or results of the RSE in any way. One of the largest changes evident in the precipitation data is the variance measured over the RSE 4-h periods: 0.63-0.66 compared to 1.14 from the earlier measurements, which were partitioned over all precipitation periods. This alone results in about a 43% reduction in sample size. The correlations between target sites, and between target and controls, are close to the range of estimates used initially. From the RSE cases, the correlation value between target to target is 0.62, control to control (opposite ranges) is 0.50-0.72, target to near controls is 0.54-0.71, and target to far controls is 0.44-0.55. The sample size results from using RSE data show lower estimates, as would be expected from the more appropriate partitioning of the precipitation data. An RSEderived sample size is 236 (110) for detecting a 10% (15%) change.
Likewise, the number of cases expected per season can be better established from the actual cases selected. Following four seasons of operations, 25-30 cases per season were obtained per season. This is considerably less than the 65-70 cases originally estimated, but the decrease is reasonable given the temperature threshold difference from that used in the earlier analysis (258C instead of the RSE threshold of 288C) and the more stringent decision process (Fig. 2) of forecasting similar SLW cloudy conditions over both ranges. These improved estimates of cases needed and cases expected per season plus the changes and instrument deployments made prior to starting the RSE led to the WWMPP funding extension by the Wyoming legislature. An experimental period of six years should provide enough cases to test if an increase in SWE as large as 15% is observed, and may provide enough cases for a test of a change as small as 10%. As shown in the previous section, far fewer cases are required to detect a 15% than a 10% change. From Eq. (6), one sees that the sample size varies linearly as a function of [1/ln(1 1 d)] 2 , where d is the seeding effect represented as a proportion. This result emphasizes the importance of applying restrictive criteria in the process of selecting cases to ensure that seeded cases are likely to be successful, as opposed to more liberal criteria that would result in more cases but also dilute the results because of either ineffective seeding application or poor targeting.

Summary
An overview of the Wyoming Weather Modification Pilot Project has been presented. The WWMPP, a state-funded program, is designed to evaluate the effectiveness of cloud seeding with silver iodide to enhance snowfall from winter orographic storms in the Medicine Bow and Sierra Madre Ranges of southcentral Wyoming. The statistical evaluation is based on a randomized crossover design for the two barriers-an efficient choice given the proximity of the mountain ranges and the funding period of the experiment. The criteria for case selection follow the long-established conceptual model of ground-based seeding of winter orographic storms. The facilities needed for operations and evaluation include a sounding unit, microwave radiometers, ground-based seeding generators, and precipitation gauges located in target areas as well as control areas.
The response variable of the randomized seeding experiment is the 4-h accumulation of precipitation. The test statistic for the WWMPP design is the root regression ratio, which has been described in Gabriel (1999) and outlined in more detail here. Estimates of the number of cases needed for statistical significance using data collected prior to the experiment were presented and suggested that changes in precipitation of 15% (and possibly 10%) should be detectable in a 5-6-yr program. Updated estimates using more appropriate but not independent data give a clearer suggestion that changes greater than about 10% should be detectable based on the design parameters and operational procedures. These estimates are highly dependent on the expected effect of seeding, on the correlations between the two target barriers and between the target and control sites, and on the variance of the response variable, precipitation.
While the statistical design of the RSE was the main focus of this paper, a wide range of additional studies have been and continue to be performed as part of the WWMPP, including aircraft observations of aerosol concentrations and cloud physics parameters, trace chemistry analysis of snow and water samples, plume detection, streamflow response, and numerical modeling studies.