Bent-cable quantile regression model

Abstract This article considers a bent-cable quantile regression model that comprises two linear segments but is smoothly jointed by a quadratic bend. This model is very flexible to allow the relationship between the response variable and a covariate of interest to change gradually or abruptly across a change point value in the covariate. However, due to the non-differentiability of the objective function in quantile regression, it is challenge to estimate the unknown parameters. Our work aims to develop a gradient-search algorithm to obtain the estimators of the regression coefficients and the change point location. We establish the asymptotic properties of proposed estimators by using the modern empirical processes theory. Monte Carlo simulation studies and an economic empirical application illustrate the good performance of our procedures.


Introduction
Change-point phenomenon is commonly encountered in many disciplines, such as environmental sciences, economics, finance and medicine. For example, the GDP per capita initial grows slowly in linear fashion as the quality of electricity supply rises, but suddenly boosts by a linear way with the quality of electricity supply gains after a given point. From a statistical view of point, continuous piecewise linear regression (or segmented regression) model is a sub-class of changepoint model subject to the requirement that the piecewise linear regression function is continuous everywhere. Numerous work on estimating and testing for the change point have been extensively investigated in continuous piecewise linear regression models, see Quandt (1958), Hinkley (1969), Feder (1975), Lerman (1980), Chappell (1989), Muggeo (2003) and Hansen (2017), and among others. The continuous piecewise linear regression model is appealing due to its simple structure and its simple interpretation. However, its loss function is non-differentiable with respect to the change point parameter, leading to considerable challenges in estimating the parameters, and hence it is difficult to derive their asymptotic properties. Another limitation is that the continuous piecewise linear regression model is not always realistic when relationship between the response variable and a covariate of interest exhibits a gradual transitional change rather than an abrupt change. As an illustration, the GDP per capita initially rises in linear fashion and goes through a curved transition phase, then followed by a linear decreasing trend on the quality of electricity supply (Figure 2). Therefore, it is desirable to formulate a formal parametric model that is flexible enough to handle gradual transitions.
To this end, Chiu, Lockhart, and Routledge (2006) proposed a bent-cable regression model, which consists of two linear curves, and is jointed smoothly by a quadratic bend to represent the abrupt with unknown width or gradual transition. The bent cable model can be regarded as an extension to the widely used segmented regression with its simple structure retained. In fact, an extremely sharp bend reduces the bent cable close to a continuous piecewise linear regression model. As a result, bent cable model is flexible enough to handle change-point phenomena, which does not need some unreasonable a priori assumption of abruptness. Moreover, the loss function of bent-cable regression model is differentiable with respect to all parameters, which is convenient to implementation. Chiu, Lockhart, and Routledge (2006) also developed the asymptotic properties of the bent-cable regression model based on least squares estimation under some nonstandard regularity conditions for independent and identically distributed (IID) data. Chiu and Lockhart (2010) extended the former work to account for bent-cable mean autoregressive model.
Although the bent-cable mean regression model could be reasonable to characterize changepoint phenomena, there are some limiting results for bent-cable regression model based on least squares estimation. The main disadvantages are that the model might be unreliable when the data is not normally distributed. Moreover, it is well-known that the estimators based on least squares criterion are sensitivity to outliers. Therefore, it is desirable to consider change point effects on conditional quantile functions, which can merit the appealing features of the quantile regression (Koenker and Bassett 1978), i.e., they are flexible for capturing the heteroscedasticity and robust to outliers in response variables. There are rich literature on estimate methods and test procedures developed for the structural change in conditional quantile with unknown timing (Qu 2008;Su and Xiao 2008;Oka and Qu 2011), and quantile regression with a change point due to a covariate threshold (Caner 2002;Kato 2009;Olmo 2011, Galvao et al. 2014;Lee, Seo, and Shin 2011;Zhang, Wang, and Zhu 2014), among others. Nevertheless, all the aforementioned models are not suitable for modeling a continuous quantile regression at the change point location. Li et al. (2011) proposed a bent line quantile regression model, where the response variable is piecewise linear but still continuous in covariates. To our limited knowledge, however, there is still inadequate study on bent-cable regression model in quantile framework.
The preceding discussion motivates us to consider a bent-cable quantile regression model. Compared to the bent line quantile regression model (Li et al. 2011), our bent-cable quantile regression model is flexible enough to capture the transition phenomena. Our goal in this article is to develop a simple and feasible estimating procedure for the bent-cable quantile regression. Although the regression function of bent-cable quantile regression model is differentiable with respect to all parameters, the loss function is non-differentiable over the real space. As a result, it is of great challenge to estimate parameters for bent-cable quantile regression. Inspired by the work of Bottai, Orsini, and Geraci (2015), we develop an effective and feasible gradient-search algorithm to overcome this problem. This algorithm is based on the gradient of loss function and generates a finite sequence of parameter values along with the decreases direction of the loss function. Furthermore, by using the modern empirical processes theory, we derive the asymptotic normal distribution for all the regression coefficients and the location of change point. Finally, we evaluate the performance of the proposed estimating method in Monte Carlo Studies and an interesting economical application.
The remainder of this article is organized as follows. In Sec. 2, we introduce the bent-cable quantile regression model, and develop a gradient search estimating method. We also derive the asymptotic properties of the proposed estimator. Section 3 investigates the finite sample performance of the proposed method via Monte Carlo experiments. Section 4 reports an economic empirical application to the bent-cable relationship between the GDP per capita and the quality of electricity supply. Section 5 concludes the article. All the technical proof and simulation results are presented in Appendix.

Model and method
The bent-cable mean regression model proposed by Chiu, Lockhart, and Routledge (2006) takes the form where Y is the response, X is a covariate of interest, and qðX, n, cÞ is the basic bent cable, qðX, n, cÞ ¼ ðX À n þ cÞ 2 4c IfjX À nj cg þ ðX À nÞIfX À n > cg: Here, IðÁÞ is an indicator function, c > 0 is the half-width of bend, and n is the change point and ðb 0 , b 1 , b 2 Þ are regression coefficients. Model (1) provides a flexible tool to analyze change-point data that exhibit a transition between two approximately linear phases. It quantifies the change between the two linear phases by a quadratic function with unknown bend parameter c > 0: Specifically, b 1 and b 1 þ b 2 are the slopes of the two linear phase, respectively. It is worth noting that model (1) includes the segmented regression model as a limiting case. In fact, when c ¼ 0, model (1) reduces to a classic piecewise-linear regression model, which is also called bent line model. Therefore, the bent-cable model is more flexible for generalizing the piecewise-linear model with a simple structure exhibiting a change. Chiu, Lockhart, and Routledge (2006) proposed an estimator of ðb 0 , b 1 , b 2 , c, nÞ, and developed the asymptotic properties of estimators for model (1) based on least squares criterion. However, there are some limitations for bent-cable regression model based on mean regression. Their model is not only sensitive to outliers, but also could be less efficient by heteroscedasticity. In addition, the bent-cable mean regression model cannot provide complete picture of the condition distribution of the response variable at given covariates. For these issues, we formulate an alternative model, the bent-cable quantile regression model, where Q s ðYjX, ZÞ is the sth quantile of response variable Y given covariates X and Z, X is a scalar variable with a possible change point, Z is a p-dimensional vector of covariates, and qðX, n, cÞ is the same to that in model (1). As discussed before, the bent-cable quantile regression model is flexible for generalizing the bent line quantile regression proposed by Li et al. (2011). Throughout the article, we denote h ¼ ðb 0 , b 1 , b 2 , b > 3 , c, nÞ > to be the unknown parameters, and omit the subscript s for ease of notation. For n independent and identically distributed observations fY i , X i , Z > i g n i¼1 from the population ðY, X, Z > Þ, an natural way to estimate parameters of h in model (2) is to minimize the following loss function, where q s ðuÞ ¼ uðs À Ifu < 0gÞ is the check function and gðX i , Due to the non-differentiability of the check function at origin, the common gradient-based optimization methods (e.g., Newton-Raphson algorithm), could be not applicable. Thus, the minimization of the loss function (3) is not a trivial problem. Consequently, it is not straightforward to make statistical inference about the parameters h: Therefore, it is desirable to develop a new and computational efficient method to estimate all parameters in bent-cable quantile regression model.
Inspired by the work of Bottai, Orsini, and Geraci (2015), we propose a gradient-search algorithm to estimate h in the bent-cable quantile regression model. To proceed, we define the subgradient of the loss function (3), where @gðX, Z; hÞ @h ¼ 1, X, qðX, n, cÞ, Z > , b 2 @qðX, n, cÞ @c , b 2 @qðX, n, cÞ @n ! , @qðX, n, cÞ @c ¼ c 2 À ðX À nÞ 2 4c 2 IfjX À nj cg, @qðX, n, cÞ @n ¼ X À ðn À cÞ 2c IfjX À nj cg À IfX > n þ cg: For a current estimate value, one can obtain a new estimate value in the direction of ÀsðhÞ where the loss function is smaller. One can repeat this iteration until the all estimators are convergence. The proposed gradient-search algorithm for bent-cable quantile regression model can be summarized in Algorithm (1). Some parameters are required in Algorithm (1), the maximum number of iteration is Kmax, the tolerance > 0, the factor a 2 ð0, 1Þ for shortening the step-length and the factor b > 1 for expanding the step length. Letĥ be the proposed estimator of h obtained by Algorithm (1).
Input the initial values h 0 , the initial step-length d 0 and set k ¼ 0 Return h kþ1 ; break end if end while

Asymptotic properties
In this section, we establish the large sample properties for the proposed estimatorsĥ: For simplicity, let m h ¼ q s ðY À gðX, Z; hÞÞ and w s ðuÞ ¼ s À Iðu < 0Þ: Then the first derivative of function m h is hÞ @h w s ðY À gðX, Z; hÞÞ: To proceed, we use some notations of the empirical process for m h : We define L n, s ðhÞ ¼ P n m h and L s ðhÞ ¼ Pm h , where Pf denotes the expectation Ef ðXÞ ¼ Ð f dP and P n f denotes 1 n P n i¼1 f ðX i Þ: In the Appendix A, it can be verified that L s ðhÞ admits a second-order Taylor expansion at true value h 0 and its Hessian matrix is where f Y ðgðX, Z; h 0 ÞjX, ZÞ is the conditional density function of Y given ðX, ZÞ: Define the matrix We can establish the consistency by Theorem 2.1 and Lemma 2.9 in Newey and McFadden (1994), and the asymptotic normality by Theorem 5.23 in Van der Vaart (2000). In practice, the asymptotic covariance matrix can be estimated byV À1 n ðĥÞR n ðĥÞV À1 n ðĥÞ, wherê hjX, ZÞ, such as the kernel estimator. However, the plug-in version of the asymptotic covariance matrix is computationally complicated. Fortunately, Theorem 2.2 guarantees that the bootstrap method (Efron and Tibshirani 1994) is valid for the proposed estimator. From the experience of our simulation studies in Sec. 3, it works well for estimating the asymptotic covariance matrix by the bootstrap method.

Numerical studies
In this section, we conduct Monte Carlo simulations to evaluate the performance of the proposed method with finite sample size. We consider the following two scenarios, where X is generated from a uniform distribution UðÀ1, 4Þ, Z is generated from a binomial distribution Bð1, 0:5Þ, and the sth quantile of e is 0. We set e ¼ẽ À Q s ðẽÞ, where Q s ðẽÞ is the sth quantile ofẽ: For each scenario, we consider three types ofẽ : (1)ẽ $ Nð0, 1Þ, (2)ẽ $ t 3 , (3)ẽ $ 0:9Nð0, 1Þ þ 0:1t 3 , where N(0, 1) is the standard normal distribution, and t 3 is the student distribution with three degrees of freedom. The true parameters are set as h ¼ ðb 0 , b 1 , b 2 , b 3 , c, nÞ > ¼ ð2, À 1, 3, 1, 0:2, 2Þ > : For each scenario, we set the sample size n ¼ 200 and repeat 1000 times at the quantile s ¼ 0:1, 0:3, 0:5, 0:7, 0:9, respectively. Tables A1-A3 in Supplementary materials report the detailed simulation results, including the estimation bias ("Bias"), the empirical standard error ("SD"), the average estimated standard error ("ESE") and the 95% coverage probability ("CP"). From these tables, all biases are close to zero, which means that the proposed estimators are asymptotically consistent. Besides, the ESEs are quite close to SDs for all parameters. Moreover, the CPs are close to the nominal level 95% except that at the extreme quantile levels (e.g., s ¼ 0:1, 0:9). This may be due to the few observations at extreme quantile levels. For an intuitive exhibition of the simulation results, we plot the average of the estimate values and the average of 95% asymptotic confidence intervals ("CI") for all parameters at different quantile levels s ¼ 0:1, 0:3, 0:5, 0:7, 0:9 in Figure 1. These plots show that our proposed estimator performs well.
In summary, the proposed estimator by the gradient-search algorithm is efficient and feasible for bent-cable quantile regression model, and has good finite sample performance.

Application
We apply our model to analyze a dataset of the GDP per capita and the quality of electricity supply. This data have 140 countries and regions in 2014, which can obtain from The Global Competitiveness Report 2015 À 2016 (Schwab 2015). The GDP per capita (US $) is an economic indicator which used to describe the economic growth of a nation or region. The quality of electricity supply is an indicator without unit which is used to measure how reliable the electricity supply in a country or region. It takes a value from one to seven, ranging from the worst quality of electricity supply (one indicates extremely unreliable electricity supply) to the best quality of electricity supply (seven indicates extremely reliable supply). Figure 2 depicts the non-linear relationship between GDP per capita and the quality of electricity supply. Specially, the GDP per capita increases slowly as quality of electricity supply rises, Table 1. The estimated parameters and their standard errors (listed in parentheses) for GDP per capita and quality of electricity supply data. For comparison, the fitted parameters by the bent-cable mean regression in Chiu, Lockhart, and Routledge (2006) are presented as Chiu.  and then rapidly boosts with the quality of electricity supply gains after a change point. Thus, we can use the bent-cable quantile regression model to analyze this data, where Y i is the ith country's GDP per capita, X i is the ith country's quality of electricity supply, and h ¼ (b 0 , b 1 , b 2 ) are unknown regression coefficients, c is the unknown bandwidth parameter in qðÁÞ and n is the unknown change point location. We estimate all parameters by proposed gradient-search algorithm at five quantile levels s ¼ 0:1, 0:3, 0:5, 0:7, 0:9, and the estimations of change point are 5. 794, 5.687, 5.683, 5.572, 5.849, respectively. Figure 2 shows the trend of bent-cable quantile regression function at different quantile levels. Given a quantile level, the quality of electricity supply has little positive impact on GDP per capita when quality of electricity supply smaller than the change point, while the quality of electricity supply has a strong positive impact on GDP per capita when quality of electricity supply larger than the change point. This suggests that the government officials should pay attention to the development of quality of electricity supply, which may appropriately improve the GDP per capita. Note that the change point locations are somewhat different at different quantile levels. For example, the change point location is 5.794 at s ¼ 0:1, while it is 5.683 at s ¼ 0:5: Therefore, countries or regions with different economic development need to pay attention to different critical values of quality of electricity supply. From Table  1, it shows that there is a clearly positive relationship between the GDP per capita and the quality of electricity supply. Furthermore, it also illustrates that the GDP per capita can be improved by a high quality of electricity supply, especially when the quality of electricity supply is more than 5.7.
For comparison, we also fit the data by the bent-cable mean regression model proposed by Chiu, Lockhart, and Routledge (2006). The fitted line is close to the bent-cable median regression.

Concluding remarks
In this work, we propose a bent-cable quantile regression model, which can be flexible to handle change-point phenomenon in data. We develop a gradient-search algorithm for to estimate all parameters in model and establish the asymptotic properties of the proposed estimator based on the modern empirical processes theory. Simulation studies demonstrate the good finite sample performances of the proposed estimator.
Our work may be extended in several ways. The errors in our model are assumed to be uncorrelated. However, the autocorrelated errors are common in change point problems, similar to the work in Chiu and Lockhart (2010). Therefore, it would be worthwhile to extend our model to autocorrelated errors. Another interesting extension is to consider multiple gradual or abrupt transitions in quantile regression model. A careful investigation on this direction is also warranted in the further study.

A Technical Proofs
We provide the regularity conditions. (A1) The true parameter h 2 H, where H is a compact subset of R pþ5 : (A2) For given s, h 0 ¼ argmin h2H L s ðhÞ exists and is unique, and the minimization of the objective function at true parameter is sufficient, L n, s ðĥÞ L s ðh 0 Þ À o p ðn À1 Þ: (A3) The density function f X ðxÞ of X is continuous and uniformly integrable. The conditional distribution function F Y ðyjX, ZÞ of Y has a continuous density function f Y ðyjX, ZÞ, and f Y ðyjX, ZÞ is uniformly integrable in U ¼ fy : 0 < F Y ðyjX, ZÞ < 1g: (A4) E sup h2H j @ @h gðX, Z; hÞj < 1: (A5) Given b 2 6 ¼ 0, the Hessian matrix Vðh 0 Þ is nonsingular Lemma A.1. Under the regularity conditions (A1)-(A4), sup h2H jL n, s ðhÞ À L s ðhÞj ! p 0, as n ! 1: Proof of Lemma A.1. By the law of large numbers, for any h 2 H, we have It implies that L s ðhÞ is the limit of the loss function L n, s ðhÞ: It is easy to show that L s ðhÞ is continuous with respect to h: Indeed, by the law of iterated expectations, we obtain Since gðX, Z; hÞ is continuous and the regularity condition (A3) holds, L s ðhÞ is continuous. It is sufficient to show the Lipschitz condition for L n, s ðhÞ in probability. Forh, h 2 H, the difference of loss function is jL n, s ðhÞ À L n, s ðhÞj ¼ By Knight's identity (Knight 1998), we have jL n, s ðhÞ À L n, s ðhÞj ¼ Further, we can obtain jL n, s ðhÞ À L n, s ðhÞj 1 n  (2000) for the asymptotic normality of M-estimators under the following conditions, (ii) m h is differentiable at h 0 in probability and such that, for every h 1 and h 2 in a neighborhood of h 0 and a measurable function _ g h with E_ g h < 1, jm h 1 À m h 2 j _ g h jjh 1 À h 2 j, (iii) L s ðhÞ ¼ Pm h admits a second-order Taylor expansion at a point of maximum h 0 with nonsingular symmetric second derivative matrix.