Calendar-based age replacement policy with dependent renewal cycles

ABSTRACT In this article, we introduce an age-based replacement policy in which the preventive replacements are restricted to specific calendar times. Under the new policy, the assets are renewed at failure or if their ages are greater than or equal to a replacement age at given calendar times, whichever occurs first. This policy is logistically applicable in industries such as utilities where there are large and geographically diverse populations of deteriorating assets with different installation times. Since preventive replacements are performed at fixed times, the renewal cycles are dependent random variables. Therefore, the classic renewal reward theorem cannot be directly applied. Using the theory of Markov chains with general state space and a suitably defined ergodic measure, we analyze the problem to find the optimal replacement age, minimizing the long-run expected cost per time unit. We further find the limiting distributions of the backward and forward recurrence times for this policy and show how our ergodic measure can be used to analyze more complicated policies. Finally, using a real data set of utility wood poles’ maintenance records, we numerically illustrate some of our results including the importance of defining an appropriate ergodic measure in reducing the computational expense.


Introduction
In today's global economy, the net earning for many manufacturing and service industries can be achieved only by efficiently utilizing resources at a lower cost (Giessing, 2008). Although material cost, labor cost, and the other legacy costs are difficult to affect, maintenance cost is one of the expenses that an industry can decisively control. Maintenance cost is dependent on the maintenance policy, a mapping from the system states (breakdown, age, etc.) to maintenance actions (inspection, repair, replacement; Archibald and Dekker (1996)).
The classic maintenance policies in the reliability and maintenance literature are the age-based replacement policy (Barlow and Hunter, 1960) and the block replacement policy (Barlow and Proschan, 1965). In the former, a component is replaced with a new one at failure or at age t p , whichever occurs first. In the latter, the component is replaced at failure and at equally spaced time points independent of the failure history of the component. Both policies are extensively studied in the literature, assuming more general models and cost structures. The literature on maintenance policies is very extensive and we review a few examples of existing models here. A comprehensive review of the literature on maintenance policies can be found in the works by Wang (2002), Nakagawa (2008Nakagawa ( , 2014, and Lai et al. (2010).
One example of extending the age-based replacement policy is the work by Frickenstein and Whitaker (2003) where the age is measured in two timescales. Under this policy, a component is replaced if its usage path crosses the boundary of a two-dimensional region or at failure. The goal is to find the optimal two-dimensional region, minimizing the long-run expected cost per time unit. Chien et al. (2010) expanded the age-based CONTACT Maliheh Aramon Bajestani maramon@mie.utoronto.ca Supplemental data for this article can be accessed at www.tandfonline.com/uiie. policy where the decision to replace or minimally repair the component at failure depends on the entire repair cost history information up to the failure time. In another extension, Shafiee and Finkelstein (2015) proposed an age-based group policy for a multi-unit series system, taking into account the economic dependence among the components. Their numerical study on wind turbine bearings showed that the group maintenance policy yields a lower cost compared with the case where each component is maintained individually. In another work, Shafiee et al. (2016) studied an age-and usage-dependent maintenance policy for railway tracks. Some other recent developments of the agebased replacement policy include considering multi-attribute objective function (Wijaya et al., 2012); modeling the degradation process as a non-homogenous continuous-time Markov chain (Sheu and Zhang, 2013;Sheu et al., 2015); developing an age-based replacement policy for a multiple component system where each component experiences both soft and hard failure processes that are mutually competing and system dependent (Song et al., 2014); and studying a combined process-age and demand-initiated preventive maintenance policy for a standby safety unit with arbitrary time to failure and time-to-repair distributions (Vaurio, 2015).
In the age-based replacement policy and its extensions, the replacement age is measured from the time of the last replacement or the last repair. The preventive replacements cannot be planned in advance and the maintenance logistics, including the maintenance labor and budget, should be available at any calendar time to replace a component reaching its replacement age. Therefore, implementing the age-based replacement policy and its generalizations is not logistically reasonable in Copyright ©  "IIE" asset-intensive industries such as utilities. In these industries, there are large and geographically diverse populations of deteriorating assets with different installation times where it is not possible to freely choose a group of assets for maintenance. In this article, we introduce a practical alternative policy to restrict the preventive replacements to specific checking time points that are scheduled independent of the failure history of the assets. In this policy, the assets are replaced with new ones at failure or if their ages are greater than or equal to the replacement age at some fixed checking point, whichever occurs first. We refer to the alternative policy as the calendar-based age replacement policy.
Our calendar-based age replacement policy is motivated by the use of several thousands of utility wood poles in the distribution system of a Canadian electricity distributor. The current maintenance strategy of the company is reactive; the poles are replaced with new ones when they fail. As the stock of poles ages, a large number of failures might occur, yielding an unexpected increase in the demand for maintenance resources. To avoid such a situation, the distributor has decided to embark on a preventive replacement program. The company already has treatment schedules for poles installed in different locations. Therefore, using the calendar-based age replacement policy where poles can be preventively replaced at already fixed scheduled treatment points is practically justifiable on the grounds of administrative and logistical feasibility for the company.
The primary difference in the calendar-based age replacement policy from the age-based replacement policy is that the checking points schedule is based on fixed calendar times, not on the time since previous replacement (renewal). Therefore, the renewal cycles (the time between consecutive replacements) are dependent random variables. In other words, the cost and the length of a renewal cycle depend on whether the previous renewal was at a checking point or at failure. The dependency of the renewal cycles under a fixed schedule was introduced in Wang et al. (2010) and Wang and Banjevic (2012). Wang et al. (2010) studied a multi-component delay time model where the inspections are scheduled based on a fixed plan for the whole system. They defined the time-to-failure since the previous inspection as forward time. Assuming that the limiting distribution of the forward time exists, they derived the unconditional expected cost and unconditional expected length of the renewal cycle. In their subsequent work (Wang and Banjevic, 2012), they proved that the limiting distribution of the forward time exists for the delay time model with fixed inspection schedule. Using the classic renewal reward theorem, Wang et al. (2010) heuristically assumed that the expected cost per time unit for an infinite time span equals the unconditional expected cost per one renewal cycle divided by the unconditional expected length of one cycle (Barlow and Proschan, 1996). However, since the renewal cycles are dependent under a fixed inspection program, the renewal reward theorem cannot be directly applied.
Our calendar-based age replacement policy is also a generalization of the modified block replacement policy introduced by Berg and Epstein (1976). The modified block replacement policy adjusts the block replacement policy by not replacing the components with ages equal to or less than a given threshold at scheduled replacement points. Berg and Epstein (1976) assumed that the threshold age is less than the length of the scheduled replacement intervals. The calendar-based age replacement policy is different from the modified block replacement policy in that the threshold replacement age can take any positive value, not only values less than the length of the scheduled checking intervals. In the modified block replacement policy, the replacement points are no longer independent renewal points. To find the expected cost per time unit for an infinite time horizon, Berg and Epstein (1976) heuristically assumed that the age of the asset at the planned replacement points has a stationary distribution and derived the expected cost per time unit for an infinite time horizon as if the replacement points are independent renewal points. Subsequent papers on the modified block replacement policy (Archibald and Dekker, 1996;Scarf and Deara, 2003;Li, 2005) also took the heuristic approach of treating the dependent replacement points as independent renewal points.
Renewal theory is a common approach for analyzing the performance of different maintenance policies in the literature. However, many authors have applied the renewal theory heuristically, ignoring that the independency of renewal cycles is a necessity. Although applying the renewal reward theory heuristically might result in accurate solutions, it does not provide a firm theoretical basis for the analysis. In this article, we present the "formal" analysis of the calendar-based age replacement policy through using the theory of Markov chains with a general state space and distinguishing between dependent renewal cycles and independent regeneration cycles. A regeneration cycle corresponds to the time between two successive renewals at fixed checking points. Specifically, using the ergodicity of a Markov chain with a general state space, we prove that the regeneration cycles have a finite expected length and show that the expected cost per time unit for an infinite time span equals the expected cost per regeneration cycle divided by the expected length of one regeneration cycle. We finally show how the expected cost and length of a regeneration cycle can be calculated using the expected cost and length of one renewal cycle. The expectation is calculated with respect to a suitably defined ergodic probability measure. The Markov renewal stochastic technique with a general state space that we use in this article has been used by some authors in the maintenance literature for the analytical analysis of maintenance policies that could only be analyzed through simulation (Grall et al., 2002;Dieulle et al., 2003;Castanier et al., 2005;Meier-Hirmer et al., 2009;Zhou et al., 2013;Mercier and Pham, 2014). Similar to these works, our article is in the direction of showing that the Markov renewal stochastic technique is more powerful than the classic renewal technique. Furthermore, we show that defining the appropriate ergodic measure is essential to creating a reasonable computational expense.
The main contribution of this article is the formal analysis of the calendar-based age replacement policy using a novel ergodic measure which, although less intuitive, provides three main advantages: First, the calculation of the expected cost per time unit is significantly faster, as it does not include renewal function calculation. It is worth mentioning that the formula that Berg and Epstein (1976) heuristically developed for the special case of the calendar-based age replacement policy and the formulas that Dekker and Smeitink (1991) and Scarf and Deara (2003) developed to extend the modified block replacement policy include the renewal function. Second, it allows us to easily extend our results, analyzing more complicated policies where the solutions of heuristic analysis would be difficult to justify. Specifically, in online Appendix D, we show how the Markov renewal theory can be used to analyze several policies for which Scarf and Deara (2003) stated that the long-term cost per time unit expressions are not available. Third, it allows calculating other useful operating characteristics of the calendar-based age replacement policy including the limiting distributions of the backward and the forward times.
The rest of this article is organized as follows. We model the problem using a suitably defined Markov chain in Section 2 and prove its ergodicity in Section 3. The expected cost per time unit for an infinite time span and the limiting distributions of the backward and the forward recurrence times are derived in Sections 4 and 5, respectively. The modified block replacement policy, a special case of the calendar-based age policy is discussed in Section 6. In Section 7, we provide a numerical study and Section 8 constitutes a conclusion. Some of the proofs, the detailed analysis, and several generalizations of the calendarbased age policy are provided in the online appendices.

Problem formulation using a Markov chain
We consider a one-component system, failing according to the probability density function f (z). The corresponding cumulative distribution and reliability functions of the component are denoted by F (z) and R(z), respectively. To avoid technical difficulties, we assume R(z) > 0, z > 0. According to the calendar-based age replacement policy, the age of the component is checked at equidistant calendar times , 2 , . . . , and is renewed if its age is greater than or equal to t p or at failure, whichever occurs first. We refer to fixed calendar times as checking points in the rest of this article. There are two types of renewals: renewal at failure and renewal at checking point. The former is a failure renewal and the latter is a preventive renewal. We assume that the failures are self-announcing and the times to check the age (whether on the spot or from a database) and perform renewals are negligible. The goal is to find the optimal replacement age t * p such that the expected cost per time unit for an infinite time span is minimized. The total cost consists of three types of costs: c s , c f , and c p , denoting the cost of checking the age, the failure renewal cost, and the preventive renewal cost, respectively. The cost of checking the age is a function of the fleet size, including the transportation cost. It can be negligible if the age is measured in calendar time scale and there is a computerized system recording the installation time and the failure history of the components. On the other hand, it can be high if the age is measured in deterioration scale and a site visit and specific measurements and equipments are required to retrieve information on the deterioration levels of the components. In the context of our example, the poles, regardless of their ages, are visited for treatment which contributes to the c s cost.
A sample path of the failure and preventive replacements is shown in Fig. 1, where L n and C n , n = 0, 1, show the random length and cost of the n + 1th renewal cycle. Due to the fixed schedule of preventive replacement intervals, the probability that the next renewal is at failure or at a checking point depends on whether the previous renewal was a failure or a preventive one. Therefore, the sequence of cycle lengths is not a renewal process. To formally formulate the calendar-based age replacement policy, our idea is to construct a renewal process by defining a suitable Markov chain.
Let {X n : n = 0, 1, 2, . . .} be a homogeneous Markov chain on a measurable space (S, S ) with transition function P(x, ) and S is a Borel field on S. In the calendar-based age replacement policy, we define X n as the nth forward time being equal to the time between the previous checking point and the nth renewal (we can define X n to represent the age at the beginning of the nth fixed interval (see Section 6 for more details); this definition, however, will result is an expression that is computationally expensive). If the nth renewal occurs at one of the checking points, X n = 0; otherwise, 0 < X n < . Without loss of generality, we assume that the fixed calendar-based checking points schedule is started before the installation of the component. A possible sample path of X n random variables is shown in Fig. 1. As illustrated, X 0 = x 0 equals the time between the installation time and the previous checking point. The first renewal occurs at failure with X 1 = x 1 ; the next checking point is then in − x 1 time units. The second renewal occurs at the calendar time 5 with X 2 = 0; the next checking point is therefore in time units. The Markov chain has a general state space, but since the age of the component might be greater than or equal to t p at some checking point, there is a positive probability that the chain's state will be zero. The one-step transition kernel of the Markov chain is given below Note that in the rest of this article, 1(A) denotes the indicator function being equal to one if the event A is true and zero otherwise, and n(x) denotes the smallest integer greater than or equal to ( As shown above, the one-step transition kernel of the Markov chain depends on the replacement threshold age t p and the replacement interval length . For convenience, we suppress t p and from the notation of the transition kernels in the rest of this article. It is clear that P(x, S) = p(x, 0) + 0 p(x, y)dy = 1, x ∈ S. Furthermore, let us assume that conditioned on a realization of {X n = x n , n ≥ 0}, there are two sequences of independent and non-negative random variables {L n } and {C n } such that the distributions of {L n } and {C n } depend only on x n . In other words, . In our problem, L n and C n are the random length and the random cost associated with the n + 1th renewal cycle, respectively (see Fig. 1). Let Assuming the cost of each renewal cycle is paid at the end of the cycle, the total cost up to time t equals (1) In our motivating example in the context of electricity distributor, the wood poles should be treated using special preservation programs such as chromated copper arsenate and pentachlorophenol (Penta), regardless of their ages, in intervals of 10 years. Therefore, it is reasonable for the company to restrict the preventive replacements to these already fixed points. However, in some applications, if there are no exogenous factors determining the length of the fixed intervals, can also be considered as a decision variable and the optimization problem then changes to finding It is intuitively clear whenever a renewal occurs at a checking point, the Markov chain restarts itself. In other words, the preventive renewal is the regeneration point. Formally, let N 1 = inf{n : n ≥ 1, X n = 0} be a random variable denoting the number of renewals until the first preventive renewal and T 1 = N 1 −1 n=0 L n be the total time until the first preventive renewal. For example, in Fig. 1, N 1 = 2, and T 1 = L 0 + L 1 , respectively. If such a T 1 exists, since the chain starts over again, there should be a sequence of such regeneration times denoted as T 2 , T 3 , . . . , yielding independent cycles. Let T k = N k −1 n=N k−1 L n , k ≥ 2 be the kth regeneration time where N k is the number of renewals until the kth preventive renewal. The sequence of regeneration times, {T k : k ≥ 1}, is a delayed renewal process, as T 1 has a different distribution than T k , k ≥ 2. Therefore, using the delayed renewal reward theorem (Barlow and Proschan, 1996), the expected cost per time unit for an infinite time span equals: The last equality follows since the first regeneration time has the same distribution as the subsequent regeneration times conditioned on X 0 = 0.
To use the above result, we need to prove that the first regeneration time exists and will occur infinitely often. In other words, we need to prove that the Markov chain {X n } is ergodic. In the following sections, we first prove the ergodicity of the Markov chain and then calculate E

Ergodicity of the Markov chain
Let us define the nth step transition kernel by This gives us the following equations: If the Markov chain is ergodic-i.e., lim n→∞ p (n) (x, y) = p(y) exists-it is independent of the initial state, and p(0) + 0 p(y)dy = 1 (p(y) is a proper distribution). We will then have To prove the ergodicity of our Markov chain {X n } with a general state space, it suffices to show that the chain is recurrent and has an invariant probability measure .

Recurrent Markov chain
According to , a Markov chain {X n } is (A, λ, ϕ, n 0 ) recurrent if a set A ∈ S , a probability measure ϕ on A, a λ > 0 and an integer n 0 exist, such that An obvious candidate for the regeneration set A in our Markov chain {X n } is x = 0, as the component's age eventually will be greater than or equal to the replacement age t p at some checking point and a preventive renewal will consequently occur. Considering A = {0}, ϕ(0) = 1, n 0 = 1, and 0 < λ ≤ R(t p + ), Lemma 1 proves that our chain {X n } is recurrent.
Proof. See online Appendix A.

Invariant probability measure
Based on Theorem 2.1 of , is an invariant and unique up to a multiplicative constant measure for any recurrent Markov chain {X n } where N 1 (see Section 2) is a random integer denoting the number of renewals until the chain enters the regeneration set for the first time. Furthermore, Corollary 2.2 of  states an invariant probability measure Lemma 2 shows that π (.) exists in our problem.

Lemma 2. In our recurrent Markov chain
Proof. See online Appendix A.
Following the same reasoning as in Theorem 1 of Wang and Banjevic (2012), it can be shown that π (.) in our problem is absolutely continuous in (0, ) and has a probability density function satisfying the integral equation (3). Furthermore, it can be shown that ν(0) = 1 and ν(E) . Since the proofs of the preceding statements are the straightforward modification of Theorem 1 of Wang and Banjevic (2012), we do not include them here. Equation (3) can be simplified as below, where g(0) = 1: To solve Equation (4), we use the successive approximation procedure (Pogorzelski, 1966;Keffer, 1999), where for y > 0: The procedure repeats until |g i (y) − g i−1 (y)| ≤ , ∀y ∈ (0, ). After solving Equation (4), we have p(0) = 1/[1 + 0 g(y)dy] and p(y) = g(y)p(0), 0 < y ∈ . We use = 0.001 in our numerical study in Section 7. Another method of solving Equation (4) is to produce its equivalent system of linear equations by discretizing the state space and applying the quadrature rules (Press et al., 2007).

Expected cost per time unit for an infinite time span
Using the ergodicity of the Markov chain, we can calculate the expected cost per time unit as shown in Theorem 1.

Theorem 1. The expected cost per time unit for an infinite time span equals:
Proof. See online Appendix A.
It is interesting to note that the expected cost and the expected length of one regeneration cycle respec- i.e., the above numerator and denominator divided by p(0) (see details in the proof of Theorem 1).
Based on Theorem 1, to calculate the expected cost per time unit, we need to know the ergodic measure for the Markov chain {X n } and the expected cost and length of one renewal cycle where the expectation is calculated with respect to the ergodic measure. The calculation of the former is discussed in Section 3.2. We discuss the calculation of the latter below.

The expected cost and length of one renewal cycle
As shown in Fig. 1, C 0 and L 0 represent the cost and the length of the first renewal cycle, respectively. We introduce the following notation to calculate E[C 0 |X 0 = y] and E[L 0 |X 0 = y]: Z time to first failure; A n an event that the the first renewal is at failure and occurs in the nth interval: the first component's failure occurs in (n − 1) − y, n − y before its age reaches t p ; B n an event that the first renewal is at the nth checking point: the age of the component is greater than or equal to t p at the nth checking point before failure.
We now have Therefore, where ((n − 1)c s + c f ) and (nc s + c p ) are the total costs if the first renewal is due to a failure in the nth interval or due to reaching the replacement age t p at the nth checking point. The above equation can be simplified as below: We also have where the first term is the expected length if the first failure occurs in ((n − 1) − y, n − y) before reaching the replacement age t p and the second term is the expected length if the first renewal is at the nth checking point. Simplifying the above equation results in the following: Let It is worth mentioning that not only the expected cost per one renewal cycle and the mean time of a renewal cycle functions of t p , but the ergodic probability measure p(.) is also a function of t p . Therefore, it is challenging to find the optimal replacement age t * p analytically. Since Proposition 1 below shows the function G(t p )/H(t p ) is bounded above, the optimal replacement age t * p can be found numerically.

Proof. See online Appendix A.
Proposition 2. If t p < (modified block replacement policy), the expected cost per time unit (Equation (5)) reduces to Proof. See online Appendix A.
If the time-to-failure has an exponential or an Erlang distribution, the ergodic measure g() can be found analytically (see online Appendix B for details). If the cost of checking the age is negligible, it can be proved, partially with help of Proposition 2, that our calendar-based age replacement policy reduces to the classic age-based replacement policy and block replacement policy for → 0 and t p → 0, respectively.

Limiting distributions of backward and forward times of the calendar-based age policy
In this section, we discuss the limiting distributions of backward and forward times for the calendar-based age replacement policy.
Assuming that t is a current time, the time between the latest renewal before t and time t represents the current age of the component, and the time between t and the next renewal immediately after t represents the remaining life of the component. The former is called the backward recurrence time and the latter the forward recurrence time. These two quantities have wide applications in reliability theory (Polatioglu and Sahin, 1998;Stadje, 2003;Tortorella, 2015). For example, the backward time and forward time distributions, in respective terms, provide information on the current age and remaining useful life of the components in the fleet. The recurrence time distributions can be used to find the expected number of failures before the next checking point and the budget for failure renewals' logistics (see Section 7).

Backward time
Let us consider a continuous time, two-dimensional process where the chain {X i } stays a random length of time in each state, as below : As discussed in Section 2, V i = i−1 n=0 L n , i ≥ 1, V 0 = 0, and T 1 = N 1 −1 n=0 L n where N 1 is the number of renewals until the first regeneration point (see Section 2). Let K(u) = P{T 1 ≤ u|X 0 = 0}; it is clear that K(.) is lattice since ∞ n=0 P{T 1 = n |X 0 = 0} = 1. Furthermore, let A(t ) and A(t ), defined below, denote the backward time and its distribution at time t: By the regeneration property of N 1 (Corollary 2.1 of ) and the strong Markov property of {W (t ), t ≥ 0}, we have As already mentioned, K(.) is lattice and τ = E[T 1 |X 0 = 0] = E[ N 1 −1 n=0 L n |X 0 = 0] < ∞ for a fixed t p (see proof of Proposition 1 in online Appendix A). Therefore, based on Feller's theorem (Feller, 1971, p. 363), the limiting distribution of the backward recurrence time is As shown in Equation (9), the limiting distribution of the backward time is a function of y, the time that has passed since the previous checking point. To understand the dependency of the limiting distribution of the backward time on y, let us assume that at the current time, we know that y time units have passed since the previous checking point; the future checks will therefore occur in the next − y, 2 − y, . . ., time units. Since the time until future checks depends on y, the dependency of the limiting distribution of the backward time on y is understandable.
To calculate the limiting distribution of the backward time, we need to find τ and a(t ). As discussed earlier, Using the same procedure as in the proof of Theorem 1 in Appendix A, we have Since V 0 = 0 and V 1 = L 0 , then: Based on Equation (10), the calculation of a(t ) reduces to finding P{L 0 > t|X 0 = y}. Let Q(y, E) = P{L 0 ∈ E|X 0 = y} = E q(y, u)du, E ⊂ (0, n(y) − y]. We have q(y, u) = f (u), u ∈ (0, n(y) − y) and q(y, n(y) − y) = R(n(y) − y).
It is worth mentioning that to apply Feller's theorem, another technical condition is necessary: a(t ) should be directly Riemann integrable. Equation (10) shows that a(t ) is a monotonically decreasing function for t ≥ w; therefore, it is directly Riemann integrable.

Forward time
As shown, for a given y, the distributions of the backward and forward times of the calendar-based age replacement policy do not have the same asymptotic behavior. In other words

Modified block replacement policy
As mentioned in Section 1, our calendar-based age replacement policy is the modified block replacement policy when t p < . To analyze the modified block replacement policy, Berg and Epstein (1976) defined x as the age of the component at times + , (2 ) + , (3 ) + , . . . (just after checking points) where 0 ≤ x ≤ t p . They heuristically assumed that the age of the component has a stationary distributionf (x) with a positive mass at x = 0. That is,f (0) + t p 0f (y)dy = 1. They heuristically stated that the expected cost per time unit for an infinite time horizon can be calculated using Equation (11) as if the fixed points are the renewal points: where E y [M y ( )] = t p 0f (y)M y ( )dy +f (0)M 0 ( ) and M y ( ) is the expected number of failure renewals in time units given the age of the component is y at the beginning of the interval. By defining a Markov chain of the ages at fixed times + , (2 ) + , (3 ) + , . . . , we formally derive Equation (11) in online Appendix C that is applicable for all t p > 0. However, one of our contributions in this article is to use a different Markov chain to formally find the expected cost per time unit. We use the Markov chain of forward times, which is less intuitive than the Markov chain of the ages. However, it results in Equation (5), not including the renewal function. Therefore, its numerical computation is more accurate and is significantly faster (see Section 7 for computational results). Furthermore, the Markov chain of the forward times can be used to analyze more complex policies. In online Appendix D, we discuss several variations of the calendar-based age replacement policy where the result of Theorem 1 can be generalized. We also show how the Markov renewal technique can be used to analyze the grouped modified block replacement policy where Scarf and Deara (2003) stated that there is no closed-form expression for its cost per time unit.

Numerical study
In this section, we use a real example to first find the optimal replacement age, compare the performance of the calendar-based age replacement policy with two other age-based replacement policies, and to calculate the limiting distribution of the backward time. We then use two different ergodic measures to compare the accuracy and computation time of calculating the cost per time unit of the calendar-based age replacement policy.
Example We use a real example in this article: utility wood poles in the distribution system of a Canadian electricity distributor with = 10 years. To find the distribution of the timeto-failure, we analyzed a data set containing 60 363 maintenance histories collected between 2003 and 2013, inclusive: 514 records are failure events and the rest are right-censored events. We assumed that the time to failure follows a Weibull distribution with parameter θ = (β, η) where β and η are the shape and scale parameters, respectively. Using the "eha" package in R statistical software (Broström, 2012), the maximum likelihood estimateŝ β = 5.1 andη = 114.33 years were obtained. Table 1 (the third and fourth columns) shows the optimal replacement age, t * p years, and the optimal long-run expected cost per year, γ * , of the calendar-based age replacement policy for different cost parameters. The preventive renewal cost c p was set at $7000; this approximately equals the cost of buying a new pole and its installation. Since we did not have precise estimates of the failure renewal cost c f and the cost of checking the age c s , the results are presented for different ratios of the failure renewal cost to the preventive renewal cost and three different costs of checking the age. The optimal replacement age was calculated in multiples of a year. First, for an initial t p = t 0 , we solved Equation (4) to find the ergodic measure p(y) and calculated the expected cost per time unit (Equation (5)) for the given t 0 using Equations (6) and (7) and numerical integration (Simpson's rule was used for the numerical integration). We then increased the replacement age by one unit and calculated Equation (5) for t p = t 0 + 1. We repeated this procedure until the expected cost per time unit was greater than the previously calculated expected cost per time unit. The replacement age corresponding to the last expected cost per time unit is the optimal replacement age. The obvious candidate for t 0 was one. However, to reduce the search space, we initially followed the above procedure in increments of 10 years to determine an interval containing the optimal replacement age and then repeated the procedure in increments of 1 year to find t * p . As shown in Table 1, we observe that for a given ratio of the failure renewal cost to the preventive renewal cost, the optimal replacement ages, calculated in multiples of a year, are equal for three different costs of checking the age. However, the optimal replacement age is not in theory independent of the cost of checking the age. The investigation of the long-run expected Table . The optimal replacement age, t * p years, and the optimal long-run expected cost per year, γ * , of the calendar-based age replacement policy, the restricted agebased replacement policy, and the classic age-based policy for different ratios of cost parameters when c p = $7000. Another useful operating characteristic of the calendarbased age replacement policy is the value of the forward time ergodic measure at zero. It represents the percentage of the renewals that are preventive replacements and are performed at checking points. For example, if c p = 7000, c f = 14 000, c s = 0, and t p = 82, p(0) is then 0.76, meaning that in a given time interval, 76% of the total renewals are performed at checking points and 24% at failure. Therefore, the ratio of preventive renewals to failure renewals is 0.76/0.24 = 3.16. This information can be used to approximately determine the expected number of preventive and corrective replacements between two subsequent checking points in a fleet of components (see the paragraph on limiting distribution of the backward time below).

Comparison of the calendar-based age policy and two other
age-based policies, if they are feasible If the checking points are scheduled time units after a renewal, not based on a calender plan, the calendar-based age replacement policy reduces to the classic age-based replacement policy (Barlow and Hunter, 1960) with one difference. As preventive replacements are still restricted to checking points, the optimal t * p is a multiple of . We refer to this policy as the restricted age-based replacement policy where In the preceding equation, c a p and c a f represent the cost of preventively replacing the pole and the cost of replacing the pole at failure in the classic and restricted age-based replacement policies, respectively.
As we already mentioned, the implementation of the calendar-based age replacement policy is logistically easier than the age-based policies. Assuming a situation where both types of policies are logistically feasible for the decision-maker, we compared the calendar-based age replacement policy with the restricted age-based replacement policy and the classic agebased replacement policy. We assumed that c p = $7000 and the cost of replacing a pole at failure in both the age-based policies (classical and restricted) is equal to the cost of renewal at failure in the calendar-based age policy; i.e., c a f = c f . The optimal replacement age and the optimal long-run expected cost per year for the restricted age-based policy and the classic age-based policy are shown in Table 1 (the seventh to 10th columns) for different ratios of c f /c p and c a p /c p . As expected, the classic age-based policy always results in lower long-run expected cost per year than the restricted age-based policy.
The results in Table 1 show that restricted and classical age-based replacement policies result in a bigger optimal replacement age than the calendar-based age replacement policy for all c f /c p , c a p /c p , and c s values. This observation implies that the decision maker is more cautious by using the calendarbased age replacement policy, as she should wait at least until the next checking point to perform preventive replacement.
The results further indicate that the calendar-based age replacement policy is not always optimal when c a p /c p = 1 for all c f /c p , as it results in a slightly bigger long-run expected cost per year. For example, if c a p /c p = 1 and c f /c p = 2, γ * = 102.2260 for the restricted age-based policy and γ * = 102.0336 for the classic age-based policy, whereas γ * = 102.2287 for the calendar-based age policy. However, if the preventive renewal cost is lower than the cost of preventive replacement-i.e., c a p /c p > 1-the long-run expected cost per year of the calendarbased age policy is lower than the long-run cost of both agebased policies for all three c s values, making it the optimal policy. For example, when c a p /c p = 2 and c f /c p = 3, γ * = 180.8895 for both age-based policies, whereas for the calendar-based age policy γ * is lower (γ * = 116.5696, 126.5683, and 166.5631 $/year for c s = 0, 100, and 500, respectively). Since the calendar-based age replacement policy is defined for a fleet of components, it allows the replacements of several poles at the same time. Therefore, due to shared logistics that can be planned long in advance, it is reasonable to have a lower preventive renewal cost per pole than the cost of preventive replacement of a single pole in the age-based policies. It is clear that for c a p /c p < 1, the age-based policies would results in a smaller long-run expected cost per year than the calendar-based age replacement policies. However, due to the logistical superiority of the calendar-based age policy, it is more reasonable to assume that c a p /c p > 1 in the real world. Limiting distribution of the backward time The limiting distribution of the backward time (Equation (9)), lim n→∞ P{A(n + y) ≥ w|X 0 = 0}, is shown in Table 2 for different values of y and w > 0 when t p = 82 years. The limiting distribution of the backward time provides information on the current age of the components in the fleet. For example, as shown, the backward limiting distribution values are 0.9090 and 0.5423 for (w = 10, y = 0) and (w = 40, y = 3), respectively. This means that in the long run, the ages of 91% of the poles are greater than or equal to 10 years at checking points (y = 0) and the age of 54% of the poles are greater than or equal to 40 years when 3 years have passed since previous checking point (y = 3). Note that the values of the backward limiting distributions are zero for all y when w > 92 since a pole's maximum age is 92 years. The backward distribution at the beginning of each year can be used to calculate the expected number of failures in the following year and consequently plan for the logistics of failure renewals. For   Table 2 for y = 5 and w > 0 can be used to calculate the expected number of failures in the next year, which equals 176.
Based on Table 2, we have lim n→∞ P{A(n + y) ≥ w|X 0 = 0} = 0.0053 for y = 0 and w = 82, meaning that 0.53% of the poles have reached the replacement age at checking points. We have already mentioned that in the steady state, the ratio of the preventive renewals to failure renewals has a value that almost three in any given interval. Therefore, we can conclude that the percentage of the poles that fail between two successive checking points is 0.53/3 = 0.17%. Table 2 further shows the value of the limiting backward distribution is different for different values of time since previous checking point y. We also observe that for w = k , k = 1, . . . , 8, the limiting distribution is decreasing in y; however, for w = k , we do not see the decreasing pattern. More general analysis of the structure of the limiting backward distribution would be difficult.

Different ergodic measures
We have previously discussed that both forward time and age ergodic measures can be used to analytically analyze the calendar-based age replacement policy, though they result in different expressions for the long-run expected cost per time unit. Equations (5) and (11) were derived to calculated the expected cost per time unit if the Markov chain of forward times and ages are used, respectively.
In Table 3, we report the long-run expected cost per year and the computation time of different ergodic measures and simulation for two different scenarios. In the first scenario, the timeto-failure had a Weibull distribution with shape parameter = 5 and scale = 2 years, = 1, and t p = 0.1. We chose 0.0005 as the length of each subinterval for numerical integration. In the second scenario, the data of the wood pole example were used with t p = 82 years and the length of each subinterval for numerical integration was 0.01. In both scenarios, c p = 7000, c f = 14 000, c s = 0 and the number of simulation runs is 1000. As shown in Table 3, in the first scenario, the forward time measure (Equation (5)) calculated the long-run expected cost per year two orders of magnitude faster than the age measure (Equation (11)). As and t p increase, the difference between the computation times of the forward time and the age measures significantly increase. In the second scenario, the computation time of the forward time measure was 102 seconds, whereas the age measure timed out after 15 hours. The solutions of the forward time measure in both cases were accurate as they are very close to the simulation results.

Conclusion
Motivated by the use of utility wood poles in the distribution network of a Canadian electricity distributor, we introduced the calendar-based age replacement policy, in which the preventive replacements are restricted to fixed checking points. The schedule of checking points is based on calendar times and is independent of the failure history of the components. Under this policy, the component is replaced with a new one at failure or if its age is greater than or equal to the replacement age at some checking point, whichever occurs first.
Since the preventive replacement schedule is based on fixed times, not on the time since the previous renewal, the cost and the length of renewal cycles are dependent random variables and the classic renewal reward theorem cannot be directly applied. To formally analyze the calendar-based age replacement policy, we defined the Markov chain of forward times with a general state space and proved its ergodicity. We further showed that the long-run expected cost per time unit can be found using the expected cost and length of one renewal cycle where the expectation is calculated with respect to the forward time ergodic measure. Using the forward time ergodic measure, we also determined the limiting distributions of the backward and forward times of the calendar-based age replacement policy, showing that they have different asymptotic behaviors and their distributions depend on the time since the previous checking point.
We also showed that another Markov chain, namely, the Markov chain of the ages at fixed points, can be used to formally derive the long-run expected cost per time unit of the calendar-based age replacement policy. However, the derived expression requires the calculation of a renewal function that is computationally expensive. Our computational results based on a real maintenance data set of utility wood poles show that the computation time to calculate the long-run expected cost per time unit is significantly lower if we use the expression from the forward time ergodic measure. Finally, we used the theory of a Markov chain with general state space and the forward time ergodic measure to formally analyze more complicated policies, for which analytic expressions of the long-run expected cost per time unit are not available in the literature.
One of our main assumptions used in the analysis of the calendar-based age replacement policy and all of the other generalized policies is that the schedule of checking points is independent of the failure history of the components. Relaxing this assumption where the history of failures can update the schedule of checking points is an interesting topic for future research. We have also assumed that there is no constraint on the number of components to be replaced at each checking point. However, the maintenance resources are usually limited. Taking into account the limited availability of maintenance resources is also a challenging topic to pursue in the future. It requires a new decision to prioritize the components reaching the replacement age for preventive replacement. In online Appendix D, we discuss how the ergodic measure developed in this article can be generalized to analyze several more complicated policies. However, numerical calculation of the ergodic measure for generalized policies would be a challenge that can be studied in the future. Simulation models can also be developed to investigate the efficiency of the numerical calculation in detail.

Notes on contributors
Maliheh Aramon Bajestani received her Ph.D. in Operations Research from the Department of Mechanical & Industrial Engineering at the University of Toronto. Immediately following the completion of her doctorate, she began working at the Centre for Maintenance Optimization and Reliability Engineering (C-MORE) as a postdoctoral fellow. Her project involved the development of a new age-based maintenance policy with dependent renewal cycles (published in this article). She is now employed as an Applied Researcher at 1QBit where she is developing algorithms for solving hard optimization problems by quantum computers.
Dragan Banjevic holds a B.Sc., M.Sc., and Ph.D. in Statistics from the University of Belgrade, Serbia. He is a Senior Research Associate and Project Director in the C-MORE Laboratory at the Department of Mechanical & Industrial Engineering, University of Toronto. His research interests include reliability theory, survival analysis, sample surveys, waiting times for patterns in Markov chains, combinatorial probability, testing of randomness, and algorithmic foundations of probability. He has published several papers in international journals and international conference proceedings.