Sensitivity analysis and retrieval of optimum SLEUTH model parameters

Abstract The Cellular Automata (CA) based SLEUTH model has emerged as a widely applied model to many cities for land use land cover (LULC) change and urban growth modelling due to its simplicity, robustness, and ease of implementation. The present study employed a rigorous sensitivity testing of self-modifying constants, Monte Carlo runs and critical slope to determine their influence on model calibration performance. Calibration performance has been examined in terms of statistical measures i.e., urban area, clusters, edges, mean cluster size, and cluster radius, best model fitness measure (i.e., Optimal SLEUTH Metrics (OSM)), overall accuracy percentage and hit-miss-false alarm method have been used. The sensitivity analysis reveals the optimum values for self-modifying parameters as {1.3, 0.10, 0.90, and 1.25} for boom, bust, critical low and critical high respectively; Monte Carlo runs as sixty (60) and critical slope as 15 to simulate the urban growth of the study area.


Introduction
The urban expansion at the cost of other land use/land cover (LULC) has become a local as well as global issue of concern. As the urban population increases, the housing and amenity requirements intensify (Abdullahi et al. 2014;Al-Sharif and Pradhan 2015). Municipality and planning departments strive to fulfil the developmental demands simultaneously considering the sustainability of the natural resource and environment through optimum developmental scenario generation using urban growth simulation models. Thus, enhancing the efficiency and accuracy of the models is necessarily an important requirement. In the last few decades, the Cellular Automata (CA) has been extensively used technique in modelling urbanization which exhibits self-organizing nature and spatial complexity (Batty et al. 1999;Ilyassova et al. 2021). The CA based SLEUTH is scalable and consistently used urban growth model in the recent past. The SLEUTH model can yield higher accuracy by utilizing minimum number of input datasets (Jayasinghe et al. 2021). SLEUTH model refers to its input data layers i.e., Slope, Land Use, Exclusion, Urban, Transportation and Hillshade. The SLEUTH model incorporates geographical characteristics, proximity variables, transportation networks, prohibited areas, neighbourhood characteristics and urban dynamics in the simulation process (Clarke-Lauer and Clarke 2011;Saxena and Jat 2020).
The SLEUTH simulates urban growth using four transitions rules; spontaneous, models the new development in undeveloped areas especially at the outskirts; new spreading center models the development around isolated built-up units which are suitable and among desirable locations for new urban spreading centers; edge/organic growth prompt further expansion of already urbanized areas in terms of spread, and road influenced growth, is for the development along roadside areas due to increased accessibility to the facilities and services. These urban growth/transition rules are set up by five growth coefficients; diffusion is responsible for spontaneous growth, breed, determines the locational probability of newly urbanized pixels from the previous phase to further continue spreading, spread determines the clustered or edged urbanization around existing urbanization, slope resistance is a topographic suitability criterion for each growth types, and road gravity controls the direction of urbanization as a function of varying road accessibility. In addition to these, the self-modifying parameters i.e., boom, bust, critical low and critical high, the Monte Carlo (MC) runs, critical slope, cellular neighbourhood, game of life rules and diffusive value parameter are other important model (SLEUTH) parameters, which significantly affect growth simulation process of the model. Some of the parameter/constant values are set as model default settings (e.g., self-modifying parameters, cellular neighbourhood which affects game of life rule, diffusive value parameter which affects diffusion growth coefficient, and game of life rules) and few i.e., MC runs and critical slope are decided by the individual user based on the information available in the literature.
Over the years several improvements have been made to the SLEUTH model and its sensitivity was investigated for temporal scale in terms of length, frequency, irregularity of the spacing of time-slices used both in input data, output and level of aggregation of LULC classes (Candau 2002;Herold et al. 2005;Clarke 2008b). The sensitivity analysis approach helps in determining the contribution of model input parameters or model coefficients in model outcome uncertainties. Such information may help decision-makers in knowing about the consequences of possible errors in the model results. Despite being popular, the SLEUTH model requires further improvements to simulate LULC change and urban growth more realistically for heterogeneous urban areas of different socio-economic and geographical settings (Jantz et al. 2010;Clarke-Lauer and Clarke 2011).
In addition to the five growth coefficients, the SLEUTH model has another set of parameters named self-modification parameters, which are intended to simulate urban growth more realistically by allowing the growth coefficients to change during the model runs to adapt non-linearity of urbanization processes (Clarke et al. 1997). The SLEUTH model is stochastic and utilizes the Monte Carlo (MC) method to produce multiple urban growth simulated outcomes for each unique set of growth coefficients. Model best-fit statistics are averaged over the MC trials (Goldstein et al. 2005). To obtain spatial variability and computational efficiency while maintaining the rigorous calibration procedure, it is essential to perform model sensitivity testing for the number of MC simulation runs. The critical slope parameter is used to consider topographical characteristics in urban growth simulations. The critical slope is a threshold value above which urban development can be discouraged. By modifying this value, the model can regulate growth on the land having a different topographic slope (Jantz et al. 2010). An appropriate critical slope value can be best identified by rigorous sensitivity analysis (Jantz and Goetz 2005). However, still as suggested by many authors, model sensitivity to above-mentioned constants/parameters have not been tested and model performance can be improved through the use of the appropriate value of these model constants for different socio-economic conditions. Thus, the present study is aimed to test the sensitivity of SLEUTH model for important model parameters/constants i.e., self-modifying constants, Monte Carlo runs, and critical slope to arrive at their optimum values for better model calibration and overall model performance.

Study area and data used
The Pushkar region lies at the outskirt of Ajmer city in Rajasthan (India) and its population is around 21,626 (as per 2011 census). Pushkar is situated in the west side of Taragarh & Madar Hills. It is along the Aravalli Mountain ranges at an average elevation of 486.0 meters above MSL. Pushkar is a place of religious and tourist attraction. It has experienced a lot of urban growth especially fragmented and scattered ones due to the trade and industrial development in the recent past. Pushkar has heterogeneous LULC along with distinct topographical features. The study area lies in the extent of 74 32 0 30 0 'N to 74 35 0 30 0 ' E ( Figure 1).
The present study utilizes merged product of multi-spectral LISS III data þ panchromatic data and LISS IV sensor data of a finer spatial resolution i.e., 5 m for the years 1997, 2000, 2004, 2008, 2013, and 2015 for extraction of LULC information which is further used for parameterization of the SLEUTH. Survey of India (SOI) toposheet of 1:50,000 scale, google earth images, Digital Elevation Model (DEM) of 1 m resolution have also been utilized for conducting the present research.

Methodology
The overall methodology includes extraction of historical LULC information for different years (i.e., 1997, 2000, 2004, 2008, 2013 and 2015) and preparation of thematic layers in GIS for the parameterization of the SLEUTH. The standard methodology has been used for image classification like signature selection, signature evaluation, feature selection, training of classification algorithm, and accuracy assessment. More than 85% accuracy was achieved for individual classified satellite images which is satisfactory for LULC applications (Herold et al. 2005;Musa et al. 2017). The transportation network i.e., road layers are prepared in GIS by performing on-screen digitization using high-resolution Google Earth images of respective years i.e., 1997, 2000, 2004, 2008, 2013, and 2015. The slope and hillshade layer are prepared from DEM. Exclusion layer was prepared in GIS by digitizing important areas that should be prohibited from urbanization like water body, dense forest, historical monuments, recreational parks, etc. The input dataset prepared for the parameterization of the SLEUTH model are presented in Figure 2.
Furthermore, model was calibrated using a multi-phase approach and Genetic Algorithm based calibration method. The model calibration performance is assessed based on the goodness of fit metrics and model simulation/prediction capability has been examined in terms of spatial and statistical measures obtained from simulated results corresponding to different values of selected model parameters and the same statistics calculated from the reference LULC map of that particular year.
The SLEUTH model sensitivity for the selected parameters was examined for a range of values of each parameter iteratively, decided as default constant value ± 50% because model parameters will pertain to individual roles which otherwise may alter. The complete methodology adopted for the study is presented in Figure 3. The model parameterized for various settings is calibrated in three subsequent phases i.e., coarse, fine, and final. Further, urban growth has been simulated for a range of values of each parameter, changing one parameter at a time iteratively. Each model calibration outcome was examined based on the goodness of fit metrics i.e., OSM, total fitness, and standard deviation. The best performing calibration phase produced optimized growth coefficients which were utilized for the final simulation of urban growth. The nearness of simulated results with reference values for each parameter value in different model runs was determined in terms of accuracy and spatial & statistical measures. Reference urban area was obtained from the two different reference datasets i.e., seed urban area of the year 2015 obtained from classified image and urban area for the year 2016 and 2017 digitized from high-resolution image of the same year.

Sensitivity analysis of self-modifying parameters, Monte Carlo runs, and critical slope
With varying step values for individual coefficient range, the SLEUTH is calibrated with all possible values of the individual parameter iteratively by keeping other model constants/parameters the same. Different self-modifying parametric settings i.e., boom, 0.5, 0.7, 0.9, 1.1, 1.3 and 1.5; bust, 0.05, 0.06, 0.08, 0.10, 0.12, 0.35, 0.65, 0.95, 1.25 and 1.35; critical low, 0.50, 0.70, 0.75, 0.80, 0.90, 1.0, 1.25 and 1.50; and critical high, 0.65, 0.90, 0.95, 1.05, 1.20, 1.25, 1.35, 1.50, 1.55, 1.75 and 1.95 have been tested (Table 1). For thirtyfive (35) combinations of parameters model was calibrated independently and individual calibration itself has passed through the three phases. So, in total 105 calibration runs have been performed to arrive at optimum self-modifying parameter settings corresponding to the improved performance of the model. Model sensitivity was tested for a range of MC runs i.e., 10 to 300; 10 À 100 with an equal interval of 10 and three simulations with 150, 200, and 300 MC iterations. The model has been calibrated for each MC set independently and the change in model performance was analyzed to arrive at optimum MC run (Table 1).
The critical slope with a default value of 15 remains constant throughout the simulation process of the original SLEUTH model. The modification in critical slope value may influence the possibility of urbanization at locations having different slopes. In this study, the sensitivity analysis has been performed for a range of critical slope values i.e., from 1 to 29 with a step value of 2 (Table 1). The model response is simulated for a total of 15 different values of critical slope and relative change in model-simulated growth was analyzed based on the above discussed metrics and methods.

Sleuth-Default versus SLEUTH-sensitivity
The model parameterized with the optimum parameters and with the default settings were termed as SLEUTH-Sensitivity and SLEUTH-Default, respectively. Both the versions of the SLEUTH were analyzed through above-discussed landscape metrics and allocation disagreement and position disagreement based hit-miss-false alarms based metrics like sensitivity, specificity, precision. overall accuracy, Total Operating Characteristic (TOC) and Mathews Correlation Coefficient (MCC). Hit, miss, false alarms, and null successes are the components of correctness and errors. The Hits (H) indicate the correct simulation of the urban locations, Misses (M) are the urban locations that are not simulated by the model, False alarms (F) indicate falsely simulated urban growth by the model, and true negative indicates non-urban locations truly find out by the model. The sensitivity represents the correctly simulated urban areas i.e., true positive rate, specificity is the correct simulation of non-urban areas i.e., true negative rate. The TOC curve is used to an effective representation of denoting complete information about the contingency matrix for all thresholds (Pontius and Millones 2011;Pontius and Si 2014;Stehman and Foody 2019). At any threshold, one can determine the actual numbers of hits, misses, false alarms, and true negatives. The MCC which varies between À1 and þ1, is useful to assess the quality of model simulation results. The MCC value approaching þ1 is the representation of perfect simulation, while À1 is an inaccurate or total disagreement between the actual and model simulation. while, '0 0 MCC value indicates a random model simulation (eq ð5ÞÞ (Kantakumar et al. 2019(Kantakumar et al. , 2020

Results
The current section summarizes the sensitivity testing results which were aimed to improve the performance of the SLEUTH model. The SLEUTH-Default (with default parameter settings) model calibration took 6 hours 53 minutes to complete on a 64-bit Windows 7 operating system with Intel (R) Xeon (R) CPU E5-2699 v3 @ 2.30 GHz processor. The time elapsed in the calibration process is significantly reduced by using the Genetic Algorithm (GA) as compared to the brute force method in SLEUTH (Herold et al. 2005). The computational efficiency of GA based SLEUTH model motivated us to perform repeated calibration (in three phases) which improved the model fitness in terms of OSM measure successively; 0.258, 0.263, and 0.276 (Table 2). The refined growth coefficient values obtained from the final phase calibration with optimum model parameters obtained from sensitivity i.e., 49, 45, 25, 68, and 46 for diffusive, breed, spread, slope resistance, and road gravity growth coefficients, respectively were used for simulating the urban growth up to the year 2040.

Sensitivity analysis of constants and parameters
To optimize the discussion on hundreds of calibration results, only best-attained coefficient sets against the best OSM (Table 3) have been discussed. Model fitness is observed between 0.08 to 0.32 for different values of boom constant. For different values of the bust, critical low and critical high the best OSM value is found to be varying between 0.26 to 0.29, 0.27 to 0.34, and 0.26 to 0.29, respectively. Similarly, for different settings of MC runs and Critical slope, the OSM has been found to vary from 0.27 À 0.30 and 0.25 À 0.35, respectively. No clear trend for growth coefficients was observed by increasing or decreasing the model constants/parameters due to the stochasticity involved in the SLEUTH model functionality to represent the random aspect of urbanization. However, it has given an idea of the best performing set of values at which the model imitates urban growth more realistically as compared to the default constant values for different socio-economic conditions. The optimum values of self-modifying parameters i.e., boom, bust, critical low, and critical high are obtained as 1.30, 0.10-0.20, 0.90 and 1.25 as compared to the default values i.e., 1.01, 0.09, 0.97 and 1.3, respectively. Also, MC 60 runs and critical slope of 15 were found the most optimum ones (Table 3) Such an investigation has led to better clarity of the optimum self-modifying constants values. With finer resolution data, very small size built-up areas and growth at edges are well captured. Also, urban clusters may not be correctly captured when compared with the manual onscreen digitization of urban patches. However, it is legitimate to find the differences in computed spatial and statistical measures from the two i.e., reference urban area (0.5 m resolution data) and simulated urban area (5.0 m resolution). However, a relative comparison of spatial and statistical measures between reference and simulated model outcomes has been found useful to decide the optimum values of the self-modifying constants. The SLEUTH calibration performed at 60 MC iteration and critical slope 15 is found to be more consistent, and goodness of fit landscape metrics i.e., compare, pop, LeeSallee, edge, cluster radius calculated from the simulated urban growth is closer to what is calculated from the reference urban area (Figure A.9-A.11). While comparing simulated and actual urban growth in terms of accuracy percentage using high-resolution images obtained from Google Earth for the years 2016 and 2017, as a reference map, the SLEUTH model calibration found to be more accurate at the boom 1.3, bust 0.10, critical high 1.25, critical low 0.90 ( Figure A.12), MC runs 60 and critical slope 15 (Table B.2).

Performance evaluation of SLEUTH-default and SLEUTH-sensitivity
The SLEUTH model would be more accurate to produce realistic urban growth corresponding to the optimum values of self-modifying constants, MC runs and critical slope.  To further establish this fact, the SLEUTH model has been tested for the two set of constant/parameters values i.e., SLEUTH with optimum values achieved through model sensitivity i.e., SLEUTH-Sensitivity and the model with default settings i.e., SLEUTH-Default. The best fit growth coefficients were achieved from SLEUTH-Sensitivity f2,1,27,12,17g with improved model fitness i.e., OSM value of 0.30 as compared to the SLEUTH-Default f49, 45,25,68,46g with 0.28 OSM value. The base statistics is the reference criteria which is computed form the model input dataset prepared from the remote sensing data.
The total urban area, edges, clusters Xmean, Ymean, radius and mean cluster size are more accurately captured from the SLEUTH-Sensitivity as compared to the SLEUTH-Default (Table 4). The present study revealed that the optimum constants/parameter values obtained from SLEUTH-Sensitivity have produced improved urban growth having correct hits over the years as compared to the SLEUTH-Default (Table 5) model. Higher OSM, lesser misses and slightly higher false alarms are observed for SLEUTH-Sensitivity.
The SLEUTH-Sensitivity is found to be more sensitive (0.9) to capture the urban areas well as compared to the SLEUTH-Default (0.75). Although, the specificity of both the models is found to be similar (i.e., 0.76) which means both the models are equally good at capturing non-urban areas. SLEUTH-Default is not good at precision i.e., 0.33 as compared to the SLEUTH-Sensitivity i.e., 0.71 which indicates poor positive predictiveness by the SLEUTH-Default. With 82% accuracy SLEUTH-Sensitivity is more efficient than SLEUTH-Default (i.e., 76%) ( Table 6). The MCC values also indicate that SLEUTH-Sensitivity is more efficient in simulating urban and non-urban areas well along with fewer false alarms and false negatives than SLEUTH-Default. The TOC curve at all threshold values is giving a better no. of true positives and true negatives along with fewer false alarms and false negatives for SLEUTH-Sensitivity than SLEUTH-Default ( Figure 4). Also, the AUC is indicating better accuracy of SLEUTH-Sensitivity in capturing urban and non-urban areas as compared to the SLEUTH-Default (Table 6). Moreover, SLEUTH-Sensitivity is found more accurately capturing the time series urban growth as compared to the SLEUTH-Default ( Figure 5).

Discussion
Sensitivity testing helped in critically understanding the effect of self-modifying parameters, Monte Carlo iterations, and Critical Slope value on SLEUTH performance and in arriving at their optimum values for which the model would be simulating urban growth  satisfactorily and in agreement with actual urban growth. The boom 1.3, bust 0.10, critical low 0.90, and critical high 1.25 are found more efficient in capturing correct urban growth, and different urban growth forms as compared to the default constant values. However, the choice of parameter selection may vary based on urban development practices, socio-economic and topographical characteristics. Also, all types of urban forms may not be accurately captured for a set of constant values. Generally, one type of urban practice prevails for similar geographical settings at a point in time e.g., in developing countries fragmented type of growth prevails. Thus, while performing sensitivity analysis targeting only prevailing urban forms at a time would be much beneficial rather focusing on all. In that situation, different sets of parameters can be achieved for different urban forms using sensitivity testing. The present study revealed that model calibration is not affected notably by the number of MC iterations. The calibration performed at different MC iterations does not lead to a significant trend in terms of different goodness of fit metrics and other statistical measures however, sensitivity analysis is useful in deciding the suitable MC runs for a specific modelling application which leads to a better model prediction accuracy and efficiency. Generally, 100 MC runs were assumed for better accuracy to calibrate the model however, in the present study MC 60 is found more efficient and accurate for model calibration which will also reduce lots of computational overhead. The critical slope values influence modelling performance and its optimum value may be different for an area having different socio-economic, construction practices, and topographical characteristics. The study reveals that the sensitivity analysis helps in improving the model performance and is well supported with the spatial and statistical measures, the goodness of fit metrics, accuracy percentage, hit-miss-false alarm methods, sensitivity, specificity, precision, MCC, and TOC. The present study reveals that it would be of increasing efficiency of the model to perform sensitivity analysis of SLEUTH model parameters well before applying the model to any study area. The study will be helpful for researchers, modelers, and planners to identify the best suitable set of model constant/parameters to accurately model and plan urban growth. Since genetic algorithm based SLEUTH has better computational efficiency so, it would be easier for one to evaluate as many numbers of calibrations for reaching out to the best model parametric settings which will eventually lead to improved model accuracy.

Limitations and future recommendations of the study
Optimum values of model constants/parameters obtained from sensitivity testing may be suitable only for the urban growth forms in similar socio-economic conditions as of Pushkar town. The set of optimum values is needed to be tested for urban areas having different socio-economic conditions. Various other important model parameters e.g., game of life rules, cellular neighborhood, and diffusive value parameter have a significant contribution in model calibration and can be in the list of future scope of sensitivity testing.

Conclusion
The Cellular Automata based SLEUTH model parameters/constants e.g., self-modification (boom, bust, critical low, and critical high), number of MC iterations, and the critical slope are sensitive to different socio-economic and geographical settings. In the present study, an optimum set of model parameters was determined at which model can capture the urban area as well as different urban forms and growth more realistically than the default values. The study revealed that there is a significant influence of different model parameters on model performances. There is no definite trend in model coefficients that have been found during sensitivity analysis because of the possible stochastic process involved in urbanization and urban growth simulation. The optimum values for self-modifying parameters such as for boom, bust, critical low, and critical high have been obtained as 1.3, 0.10, 0.90, and 1.25, respectively. The optimum value of Monte Carlo runs is sixty (60). A range of 15-19 for the critical slope was observed as optimal. The SLEUTH with optimum parameters i.e., SLEUTH-Sensitivity has achieved better model outcomes than SLEUTH-Default. The improved statistical measures of SLEUTH-Sensitivity fmodel fitness i.e., OSM (0.30), sensitivity (0.9), specificity (0.76), precision (0.71), overall accuracy (82%), TOC based AUC (0.83), and MCC (0.7)g than SLEUTH-Default fmodel fitness i.e., OSM (0.28), sensitivity (0.75), specificity (0.76), precision (0.33), overall accuracy (76%), TOC based AUC (0.62), and MCC (0.38)g has improved the model efficiency and reliability. Thus, present study recommends to include sensitivity analysis of model parameters as a part of modelling to improve the model performance.

Data Availability statement
The raw data can be obtained from the USGS Earth Explorer (https://earthexplorer.usgs. gov/). The other data that support the findings of this study are provided as supplemental data.

Disclosure statement
No potential conflict of interest was reported by the author(s).