A participatory framework for developing public participation GIS solutions to improve resource management systems

ABSTRACT Participatory approaches elicit information from multiple stakeholders while planning and implementing resource management systems. Such elicited information is often associated with significant variability. Public participation geographical information science (GIS) (PP-GIS) solutions can reduce this variability by helping stakeholders to measure the factors involved and provide the elicited information. We propose a ‘Quality Function Deployment’-based participatory framework for developing such PP-GIS solutions. It is demonstrated using a case study to enhance an existing PP-GIS into a solution for rainwater harvesting systems in Indian villages. The novelty of the proposed framework is that it identifies metrics and carries out comparative analysis of three existing solutions: participatory rural appraisal, participatory mapping and PP-GIS. In the case study, PP-GIS scored less than participatory mapping as it scored less on usability and affordability. To improve PP-GIS in these aspects, an easy-to-use mobile and web based, free and open source PP-GIS solution, Watershed GIS, was developed. It scored better than the three existing solutions and its usage resulted in substantial reduction of variability in criteria values and thus better ranking of alternatives, with the average coefficient of variation decreasing from 0.12 to 0.05.


Introduction
The use of geographical information science (GIS) in projects that involve community participation has increased substantially over the years (Craig et al. 2002), and it has become 'vitally embedded in broader transformations of science, society, and culture' CONTACT Nagesh Kolagani nagesh.k@iiits.in IIIT Chittoor, Sri City, AP 517588, India (Formerly Ph.D. scholar, Department of Engineering Design, IIT Madras, India) The underlying research materials for this article can be accessed at http://dx.doi.org/10.1080/13658816.2016. 1206202: 1. Quantifiable metrics identified by experts for measuring requirements of community stakeholders and expert stakeholders. 2. Relationship between requirements and quantifiable metrics. 3. Metric-based scores of existing solutions, metric-wise targets and metric-based evaluation of various iterations of Watershed GIS. 4. Stakeholder-perceived scores of existing solutions by the community stakeholders and the expert stakeholders. (Pickles 1995). However, despite being more precise and powerful, it can only augment cognitive abilities of humans rather than replace them (Goodchild 2006). Pickles (1991) also points out the potential anti-democratic nature of GIS due to its differential accessibility and other aspects. To address these concerns, a distinct participatory method known as public participation GIS (PP-GIS) has evolved (Sieber 2006). It promotes more inclusive community participation and helps communities climb the participation ladder (Craig et al. 2002, Weiner andHarris 2003). PP-GIS makes it possible to incorporate local knowledge into GIS and use it in participatory planning (Craig et al. 2002). When planning and implementing resource management systems, a PP-GIS solution can help stakeholders gather information with less variability by measuring the factors involved. Here, we propose a novel participatory framework for developing such a PP-GIS solution. As the ultimate users of the PP-GIS solution, the stakeholders are the correct persons to identify the challenges they face in accessing such a solution and what their requirements are (Elwood 2008). We demonstrate the proposed framework by developing a novel PP-GIS solution for watershed management systems in Indian villages.
In rural areas of developing countries, the performance of resource management systems, such as groundwater usage systems (Nagaraj et al. 2000), drinking water supply and sanitation systems (Baye et al. 2012) and land use systems (Lippe et al. 2011), is often poor in terms of meeting stakeholder requirements. Including stakeholders in planning and implementation ensures better capture of stakeholder requirements and improved performance of these systems (Whittington et al. 2009, Sheer et al. 2013, Voinov et al. 2016.
During planning and implementation of resource management systems, stakeholders need to analyse multiple alternatives. A quantitative model, which ranks these alternatives by evaluating their efficiency, will facilitate such analysis. The efficiency is a function of multiple criteria that captures both technical and social requirements of stakeholders (Kolagani et al. 2015). Such a model will help stakeholders analyse 'what-if' scenarios (Labiosa et al. 2013), dialogue and collaborate in evaluating alternatives (Hossard et al. 2013, Sahin et al. 2016 and select through consensus the best alternative. The alternatives are evaluated by combining criteria to compute their efficiencies using one of the several multiple criteria analysis models that are available (Figueira et al. 2005). These models are calibrated by involving stakeholders in identifying and estimating their parameters (Rykiel 1996, Hewitt et al. 2014. We use weighted summation model shown in Equation (1) for its simplicity (Howard 1991, Hajkowicz andHiggins 2008): where ɳ evaluated is the evaluated efficiency, w i is the relative importance of ith criterion and x i is the value of ith criterion. Kolagani et al. (2015) propose a participatory model calibration framework where stakeholders identify criteria and derive corresponding weights to construct the weighted summation model. The stakeholders also provide values of each criterion for each alternative. Various participatory methods, such as participatory rural appraisal (PRA) (Mukherjee 2002, Chambers 2008 and participatory mapping (PM) (Chambers 2005), are used to gather such information. While each criterion is objective in nature and has unique value for each alternative, including multiple stakeholders will generate conflicting information (Lippe et al. 2011, Krueger et al. 2012. To reduce the variability among multiple stakeholders, Kolagani et al. (2015) propose decomposing each criterion into factors that govern it so that stakeholders can measure them. For example, a water inflow criterion for small rainwater harvesting (RWH) systems can be decomposed into factors such as extent of its catchment area and its subareas with similar land use, soil and slope. Stakeholders can visit each system to map and measure its catchment area and other factors using GIS.
While PP-GIS facilitates such use of GIS by stakeholders (Nyerges et al. 2002, Jankowski 2006), many of them experience difficulties in using it (Elwood 2008). To make PP-GIS more accessible, stakeholders need to be trained in using it as most of them are not familiar with computers. Arciniegas et al. (2013) point out the need to keep at low levels the cognitive effort required by the stakeholders to process large volume of information provided by the PP-GIS. Otherwise, it can result in marginalization and alienation of the stakeholders (Chambers 2005) and stifle their creativity and participation (Jankowski and Nyerges 2001). Hence, there is a need to develop an easy-to-use PP-GIS solution that requires less training and cognitive effort. The rapid spread of mobile phones with easy-to-use touch screen interfaces and web technologies such as CyberGIS (Wang et al. 2013) is making development of such PP-GIS solutions possible.
Developing such PP-GIS solutions requires a framework to carry out detailed requirement analysis with stakeholders and to develop a solution that meets their requirements (Papathanasiou and Kenward 2014). We propose a 'Quality Function Deployment'-based participatory framework for developing PP-GIS solutions (Akao andMazur 2003, Zaim et al. 2014). As the ultimate users of the PP-GIS solution, stakeholders are the correct persons to identify such requirements. To help in gathering these requirements, the stakeholders are asked to compare existing PP-GIS with other existing solutions (PRA and PM) in terms of how well these solutions meet their requirements. The gaps in the extent to which existing solutions meet stakeholder requirements are used while determining targets for the new solution. The novelty of our framework is that it develops metrics to measure how well each solution meets each stakeholder requirement and uses them to compare PRA, PM and PP-GIS and, hence, develop an enhanced PP-GIS solution that addresses the stakeholder requirements.
The remainder of the paper is organized as follows. First, we explain the proposed framework involving two stages: requirement analysis and PP-GIS solution development. It is followed by a section on application of the framework to a case study for development of a PP-GIS solution to help stakeholders in planning and implementation of RWH systems in Indian villages. A subsequent section discusses the results of the case study.

Proposed framework
PP-GIS solution is used to reduce variability in criterion values of systems obtained from stakeholders. We propose a participatory framework for developing such a PP-GIS solution. While stakeholders identify criteria and their weights in the participatory model calibration framework (Kolagani et al. 2015), the proposed framework gathers from stakeholders their requirements for estimating criteria values. The framework consists of two stages ( Figure 1): the requirement analysis stage and the PP-GIS solution development stage. The key steps of these two stages are explained in detail below.
Step 1 is to identify stakeholder requirements and their weights. In the 'Quality Function Deployment' literature, this is referred to as listening to the 'voice of the customer' (Bossert 1991). While developing a PP-GIS solution, these requirements are best identified from stakeholders who are knowledgeable about these systems (Shrestha et al. 2004). Relative priority of these requirements is gathered from them as weights to help in identifying high priority requirements (Liu et al. 1998) using the 'Analytic Hierarchy Process' method (Saaty 1980).
Step 2 comprises identifying metrics for each requirement. A team of experts identify one or more technical metrics for each requirement to measure how well a solution, existing or new, meets that requirement. For example, for the qualitative stakeholder requirement that says: 'we need training to use the solution', we will need quantitative metrics to measure how easy the training manuals are to use. When a requirement is measured by more than one metric, the relative weight of each metric in measuring that requirement is evaluated using the 'Analytic Hierarchy Process' method. The overall weight of each metric is calculated using Equation (2): where m ji is the weight of jth metric in measuring ith requirement, r i is the weight of ith requirement, n is the total number of requirements, and M j is the overall weight of jth metric.
Step 3 is to identify existing solutions that stakeholders currently use and evaluate them using the metrics developed. An overall weighted score on a 0-1 scale is evaluated for each existing solution [Equation (3)]:  ( 3) where s kj is the score of kth solution for jth metric, M j is overall weight of jth metric computed using Equation (2), p is number of metrics, and S metric-based,k is metric-based score of kth solution. These solutions are also evaluated by stakeholders based on their perception of how well these solutions meet each of their requirements. An overall weighted score on a 0-1 scale is computed for each existing solution using Equation (4): where s ki is the score of kth solution for ith requirement, r i is the weight of ith requirement, and S stakeholder-perceived,k is the stakeholder-perceived score of kth solution.
The metric-based score of existing solutions are compared with their stakeholderperceived scores and if they do not match, the metrics are refined. If necessary, requirements are also refined (regions in Figure 1 bound by dashed lines). The novelty of the proposed framework is that it treats the evaluation scores of existing solutions perceived by the stakeholders as the standard since they are the ultimate users of these solutions. The metrics (and even the requirements) are refined iteratively until metricbased scores and stakeholder-perceived scores match.
Step 4 is to determine targets for the new PP-GIS solution development stage based on how well existing solutions meet stakeholder requirements.
Step 5 is to develop a new solution to meet these targets. The new solution is evaluated using metrics and is refined iteratively until its metric-based scores match target scores.
Step 6 comprises testing the new solution for reduction of variability while obtaining criteria values from stakeholders. Variability in criteria values is measured using coefficient of variation. This also results in reduction of variability while evaluating efficiency of each system. This in turn leads to better ranking of the alternatives. For deciding on how good these rankings were, Kolagani et al. (2015) used the Spearman's rank correlation coefficient of these rankings with the rankings of alternatives based on stakeholderperceived efficiencies. While median can be used to aggregate efficiencies evaluated by multiple ESH, the true advantage of reducing variability lies in a frequency perspective of the rankings (Rosenbloom 1997, Lafleur 2011. Hence, we analyse the frequencies with which each alternative is assigned various ranks.

Application of framework to a case study
The proposed framework was demonstrated using a case study to develop a PP-GIS solution, Watershed GIS, for watershed management systems (Johannes et al. 2002, Miller et al. 2004 in Indian villages (Adinarayana et al. 2004). Since the 1990s, many villages in India (Nagaraj et al. 2000, Kumar et al. 2014) and in several other developing countries (Feuillette et al. 2003) have been suffering from a severe water crisis due to overexploitation of water resources. To address this crisis, the Government was implementing an integrated watershed management program (National Rainfed Area Authority 2011) to construct several small RWH systems with the participation of stakeholders and increase availability of water. Watershed GIS helped the stakeholders participate effectively in planning and implementation of these systems.
For demonstrating Watershed GIS, E. Palaguttapalli village in Chittoor district of Andhra Pradesh state of India was selected as the case study village (13°32′26.4″ N, 79°03′51.8″ E, see Figure 2). It is a representative village in terms of its small RWH systems and the socioeconomic and educational background of its stakeholders. It has 240 stakeholders belonging to four categories of farmers (Agriculture Census Division 2012). It is primarily an agricultural village spreading across an area of approximately 500 ha. At the time of this research, close to half of the area was cultivated and approximately 50 ha of the cultivated area was irrigated for cultivating commercial crops such as Sugarcane and Mango.
To develop Watershed GIS using the proposed framework, as part of step 1, two types of stakeholders were identified: expert stakeholders (ESH) and common stakeholders (CSH). Stakeholders who had expertise in gathering criteria values of resource management systems were referred to as ESH. The other stakeholders who used Watershed GIS to analyse alternatives were referred to as CSH. To ensure well thought out answers (Shrestha et al. 2004), only stakeholders who had detailed practical knowledge of these systems were used. We treated stakeholders who had high school education and had been involved in planning and implementation of at least 10 such systems as ESH. As knowledgeable CSH and ESH are few in a village, a small sample size is common (Harrison and Qureshi 2000) and sufficient (Griffin and Hauser 1993). The ESH used Watershed GIS for gathering information about the physical and social criteria values of each alternative system (Moody and van Ast 2012, Panagopoulos et al. 2012, Chowdary et al. 2013, Mahmoud et al. 2014. The CSH used it to view the information gathered by the ESH. Out of 240 CSH in the case study village, 60 had practical knowledge of RWH systems. From among them, 20 were selected using stratified sampling from each category of farmers (Table 1) to ensure diverse viewpoints were captured. Similarly, 10 ESH were selected from among 20 present in the case study village. Requirements of CSH and ESH were identified using two participatory methods: key informant interviews and focus group discussions (Griffin andHauser 1993, Jaraiedi andRitz 1994).
Weights of these requirements were gathered from the CSH and the ESH using the 'Analytic Hierarchy Process' method. 'Expert Choice' software (Forman et al. 1985) was used for this purpose. It involved carrying out pair-wise comparison of requirements using a questionnaire. The questionnaire was administered to multiple stakeholders to avoid individual biases. Questionnaire response from each stakeholder gave rise to a set of weights, one for each requirement. As the number of stakeholders surveyed was small, to account for outliers, median (Kolagani et al. 2015) was preferred to mean (Tian et al. 2013).
As part of step 2, for each requirement of the CSH and the ESH, one or more metrics were identified by a team of three experts drawn from different areas of expertise: academic research, software development and social work.
The next step was to evaluate the three existing solutions that stakeholders were using currently: PRA, PM and PP-GIS (Table 2). For example, while using PRA to estimate catchment area of a system, stakeholders undertook a 'transect walk' to the location of the system and estimated the criterion values through individual and group discussions. While using PM, they demarcated the catchment area of the system on a paper map and measured it approximately. While using PP-GIS, they visited the system and walked along the boundary of its catchment area; the PP-GIS calculated exact extent of the catchment area automatically. Metric-based evaluations of these three existing solutions were carried out using usability trials. Stakeholder-perceived evaluations were carried • Ease-of-use (Comair et al. 2014) out using focus group discussions as it helped stakeholders to discuss and compare solutions critically and achieve consensus.
Step 4 comprised determining targets for Watershed GIS. To set realistic targets, it is useful to know about correlations between metrics. For example, negative correlation between metrics, such as 'measurement error' and 'training period' represents a tradeoff and both cannot be minimized simultaneously. Hence, the metrics were compared with each other to populate a correlation matrix Clausing 1988, Govers 1996). Results from evaluation of existing solutions, together with correlation matrix, were used to determine targets for Watershed GIS.
Next step was to develop and evaluate Watershed GIS. The first iteration of Watershed GIS was developed as a Python plug-in program to the open source Quantum GIS (QGIS) to increase its usability and meet the targets. Its usage involved the following steps: 1. The ESH visited each RWH system and collected: location, photographs, audio recording of interviews, and physical and social information as fields in one record of a tablet-based questionnaire. 2. 'Watershed GIS' processed each record in the questionnaire and added a RWH system to the GIS map at appropriate location tagging it with its information. 3. The CSH viewed this GIS map of all RWH systems in their village, searched for desired RWH systems and viewed their detailed information.
Step 6 comprised testing the final iteration of Watershed GIS for its ability to facilitate reduction of variability while obtaining criteria values from the ESH. It was also tested for reduction of variability in evaluating efficiency and its effect on ranking of alternatives. For this purpose, 10 RWH systems were selected from the case study village. For each RWH system, each ESH was asked to provide an estimate for following four criteria: (1) water inflows, (2) site suitability, (3) social equity, and (4) spatial equity.
Obtaining criteria values was done by the ESH twice: once without using Watershed GIS and once using it. To obtain the value of 'water inflows' criteria without using Watershed GIS, the ESH demarcated catchment area of each RWH system on a paper map based on their knowledge. Subsequently, they divided the catchment area into subareas that had similar land use, soil type and slope categories. They calculated the approximate extent of each of these subareas and the catchment area and estimated a value for the 'water inflows' criterion using a set of rules made available to them. For repeating the same exercise using Watershed GIS, they visited each RWH system and walked along the boundary of its catchment area and various subareas in the field to map them. A map of these areas was displayed by the Watershed GIS and their exact extents were calculated. The ESH estimated from these extents a value for 'water inflows' criterion using the same rules as described earlier. Other criteria were estimated similarly.

Results and discussion
A total of eight primary requirements were identified for the CSH and five primary requirements for the ESH (Table 3). For the CSH, 'trustworthiness of source of information' was the most important requirement, with a weight of 19%. This was in agreement with what a focus group participant had reported: 'We cannot believe what outside agencies tell us. We believe only in information reported by our own people'. 'Privacy while accessing information' was one of the next two important requirements. Another participant from lower socioeconomic strata had said: 'Village elders don't want us to know any details. They feel we will start questioning them if we get to know too many details. So, we should be able to access information without their knowledge'. 'Availability of latest information' was the other important requirement.
Two requirements, 'detailed information' and 'viewing information', were ranked as least important by the CSH. They pointed out that due to their lower educational and socioeconomic status, they were traditionally denied access to information. Hence, they wanted less importance to be given to content of information and ways to view it than to getting access to it. However, the ESH felt they needed to gather good information, since they were paid for it. Hence, they ranked 'previewing gathered information' as the most important requirement and 'gathering detailed information' as the third most important.
In addition, 6 secondary and 10 tertiary requirements were identified for the CSH under their 'detailed information' primary requirement. These requirements and their weights are shown in Figure 3 and explained in detail in the Appendix. These include two social criteria, 'spatial equity' and 'social equity', and two technical criteria, 'water inflows' and 'site suitability' (see also Kolagani et al. 2015).
A total of 33 metrics were identified for the 19 requirements of the stakeholders. A list of these metrics, methods to measure them and relationship between metrics and requirements are available in supplementary materials 1 and 2. Table 4 shows these details for the 'availability of technical support' requirement of the CSH.
Metric-based and stakeholder-perceived scores of the three existing solutions, PRA, PM and PP-GIS, are given in supplementary materials 3 and 4 and are also summarized in Table 5. While PM scored better than PRA, PP-GIS scored less than PM. Given the unique advantages that PP-GIS offers in terms of improved visualization and usability, this presents an opportunity for improving PP-GIS.
The metrics were compared and a correlation matrix was populated. A value of (−9) indicates strong negative correlation. For example, for ESH the metric 'educational background needed' conflicted strongly with the metric 'training period' since stakeholders with less computer education needed more training (Figure 4). Targets were determined for Watershed GIS as shown in supplementary material 3. The overall target scores for the CSH and the ESH were 0.58 and at 0.68, respectively. To meet these targets, Watershed GIS was enhanced iteratively as described in the following: Iteration 1: Metric-based evaluation of Watershed GIS iteration 1 was carried out and the metric-wise scores are available in supplementary material 3. Overall weighted scores of Watershed GIS iteration 1 were 0.57 for the CSH and 0.54 for the ESH.
Iteration 2: Since these scores were less than the target scores, the Watershed GIS was further enhanced during a second iteration. It was noticed during usability trials that one   important mistake ESH were often making was providing wrong references to location information in questionnaires. Preparation of GIS map was being done in one step without visual feedback. Therefore, ESH had to wait till all RWH systems were added before they could intervene and correct mistakes. To avoid this problem, during second iteration, a graphical user interface was added to the Watershed GIS to add each RWH system individually. Watershed GIS iteration 2 was evaluated again, and its overall scores improved to 0.64 for the CSH and 0.65 for the ESH. Iteration 3: Watershed GIS was further enhanced during a third iteration. A new mobile application was developed so that collection of all information about each RWH system could be done by one device: a mobile phone. The mobile application compressed information relating to each RWH system into an encrypted zip file and uploaded it to a web application of Watershed GIS for subsequent processing. It reduced mistakes by ESH, data corruption and tampering associated with use of multiple devices.
It was also noticed during usability trials that while the CSH wanted to view specific kinds of maps emphasizing certain aspects, such as a water map showing all working wells and irrigated farms, they were unable to build the required spatial queries without substantial help from the ESH. Hence, during iteration 3, a drop down list containing several predefined spatial queries was also provided so that the CSH could select and view desired maps without help from the ESH.
The overall score for use of Watershed GIS iteration 3 by ESH improved to 0.82, which was higher than the target score of 0.68 ( Figure 5). For the CSH, its overall score improved to 0.69, which was more than the target score of 0.58. As the targets were met, no further iterations were carried out.
Use of Watershed GIS to measure criteria values lead to reduction of variability among values obtained from different ESH. For 'social equity' criterion, average coefficient of variation across 10 RWH systems decreased from 0.44 to 0.10 with use of Watershed GIS (Figure 6). For 'spatial equity' criterion, the average coefficient of variation decreased from 0.36 to 0.12. However, for RWH systems 3 and 4, it was already low at 0.15 and 0.12, respectively, without use of Watershed GIS and did not decrease any further with its use. For both 'water inflows' and 'site suitability' criteria, average coefficient of variation was already low at an average of 0.17 even without use of Watershed GIS. This is due to the technical nature of these two  criteria. It further decreased slightly to 0.11 for both criteria with use of Watershed GIS.
Use of Watershed GIS also resulted in reduction of variability while evaluating efficiency of each RWH system. The coefficient of variation decreased from an average of 0.12 to an average of 0.05 with use of Watershed GIS (Figure 7). This resulted in better ranking of the alternatives. When median was used to aggregate efficiencies evaluated by multiple ESH, use of Watershed GIS resulted in improvement of rankings from 91% to 94%. When using the frequency approach (Table 6), use of Watershed GIS resulted in an improvement of the correlation of metric-based rankings with stakeholder-perceived rankings from 84% to 90%. Rankings perceived by the stakeholders are treated as the reference, since they know these systems well and are their ultimate users (Kolagani et al. 2015).

Discussion and conclusions
A participatory framework for developing a PP-GIS solution for resource management systems was proposed and demonstrated using a case study. As part of the case study, stakeholder requirements and associated technical metrics were identified for a PP-GIS solution to plan and implement small RWH systems in Indian villages. A PP-GIS solution was developed and enhanced iteratively into a Watershed GIS using Android mobile platform and Quantum GIS software. It was made available to the stakeholders as a free and open source software.
As can be seen from the results, Watershed GIS was able to meet the stakeholder requirements well. By incorporating improved usability as a requirement, Watershed GIS  was able to overcome many of the difficulties stakeholders experience in using a PP-GIS solution (Elwood 2008). Through improved visualization of information, cognitive effort required from stakeholders was kept at low levels (Arciniegas et al. 2013). It resulted in significant reduction of variability while gathering criteria values from multiple ESH and evaluating efficiencies of RWH systems leading to better ranking of alternatives. Projected benefits of the Watershed GIS include bringing the stakeholders together, empowering them through better visualization and understanding and facilitating their participation in the planning and implementation process. This is expected to lead to use of locally relevant social criteria in the planning process, which in turn may create a sense of ownership among the stakeholders and ensure long term maintenance and sustainability of the RWH systems. The proposed participatory framework can also be used for developing solutions for natural and other resource management systems where stakeholders are facing difficulties in using existing PP-GIS solutions.
Future work will involve verification of how well such a PP-GIS solution can help stakeholders in these directions and subsequently making further improvement of it based on feedback from the stakeholders. A limitation of the case study was the presence of uncertainties in various types of information elicited from the stakeholders. Probabilistic approaches will be needed to account for these uncertainties. It will also need multiple case studies involving more number of stakeholders.