Kriging-Weighted Laplacian Kernels

—Laplacian kernels, which are widely used as sharp- ening filters in image processing, are isotropic and tend to over-highlight fine details with a sharp discontinuity in images. To address this issue, this paper introduces a method that integrates anisotropic averaging with the Laplacian kernels. The proposed method can also be useful as a new type of image convolution for designing convolutional neural networks.


I. INTRODUCTION
Image convolution, which is the process of adding values of a pixel and its neighbors weighted by a filter kernel to every pixel, is not only a general well-adopted procedure in image processing but also plays a critical role in the design of convolutional neural networks [1], [2] and most widely used in deep learning [3], [4]. The purpose of convolution is to modify the spatial characteristics of an image, resulting in, such as smoothing, sharpening, enhancing, or edge highlighting. In general, the convolution of a kernel k and an image f (x, y) can be expressed as g(x, y) = k ⊛ f (x, y) where g(x, y) is the output image, and ⊛ denotes the convolution operator. Laplacian kernels are widely applied for image sharpening. The Laplacian operator, which takes partial derivatives along the two spatial axes of an image f (x, y), is defined as [5] which highlights sharp intensity transitions and reduces the effect of regions having slowly varying gray levels, resulting in the following 3 × 3 Laplacian filters where the left matrix is based on Eq. 2, and the right is an extension of Eq. 2 to include the diagonal elements.
To rectify the effect of featureless image background while sharpness is still reserved can be obtained by subtracting the Laplacian image from the original, giving [5] Because L 1 and L 2 are derived from the second derivative of an image, which highlights regions of rapid intensity change, both kernels are sensitive to noise. To address this issue, it is suggested to Gaussian-smooth the image before applying the Laplacian. This procedure is known as the Laplacian of Gaussian (LoG), whose function is expressed as where σ is the standard deviation, All L 1 , L 2 , and LoG kernels yield isotropic results. Because an isotropic operator applies the rate of change equally in all directions of an image, it carries no particular sensitivity or bias toward different spatial orientations. An anisotropic topography has directional attributes that may be different at all points or directions of an image. Effort was spent on addressing this issue by minimizing the overall anisotropy produced by Laplacian kernels [7], or using local Laplacian kernels [8], [9].
As an attempt to consider anisotropic properties of an image convolved with Laplacian kernels, this paper introduces the use of kriging weights as a secondary kernel whose derivation is based on the statistical properties of an anisotropic semivariogram. Unlike having fixed values of coefficients for a kernel, the kriging weights vary according to changes in image semivariograms, giving different effects of convolution in images of different spatial statistics.

II. KRIGING-WEIGHTED LAPLACIAN OPERATORS
Let z(a) be a random function of spatial location a, the ordinary kriging of geostatistics holds the following intrinsic hypothesis [10]: where E[·], µ, and γ(·) denote expected value, mean, and semivariogram of the random function, respectively; and a and a + h are any two spatial locations separated by distance h.
Let z(a i ) and z(a j ) be the values of two data points at two spatial locations a i and a j , respectively. It was shown that [11] To derive kriging weights in the setting of the Laplacian operators, let a 0 be the location of f (x, y), , and a 8 of f (x + 1, y + 1), which are shown in Fig. 1. The value of f (a 0 ) can be predicted using the best linear unbiased estimator of ordinary kriging as [11] f where w i , i = 1, . . . , 8 are kriging weights that can be optimally determined by minimizing the estimation variance σ 2 (a 0 ): Using Eq. 7, the estimate is done via the following constrained optimization which is subject to Based on the following proof [11] Var Eq. 9 can be rewritten as (11) Using Eq. 6, The Lagrangian function for ordinary kriging estimate of f (a 0 ) can be constructed as where λ is a Lagrange multiplier. Let γ(a i − a j ) now be denoted as γ i,j . To minimize the variance of estimation, all first derivatives of the Lagrangian function with respect to w i , i = 1, . . . , 8, and λ must be set to zero, which are The above Lagrangian method results in the determination of unique kriging weights by solving the following set of simultaneous equations: To consider anisotropy in the semivariogram model, the experimental semivariogram measured in the horizontal direction of an image can be expressed as where N (∆x) is the number of pixel pairs separated by ∆x. Likewise, for the vertical direction, the experimental semivariogram can be defined as where N (∆y) is the number of pixel pairs separated by ∆y. For a matrix, the main diagonal is the line of elements running from the top-left to bottom-right corners; whereas the antidiagonal is the line of elements running from the toprightt to bottom-left corners. Let ∆D + and ∆D − indicate (∆x, ∆y) measured in the directions of the main diagonal and antidiagonal of an image, respectively. The experimental semivariograms measured in the main diagonal and antidiagonal directions can be obtained, respectively, as where N (∆D + ) is the number of pixel pairs separated by (∆x, ∆y) in the main-diagonal direction, and γ 1,4 (∆y) γ 1,5 (∆D + ) γ 1,6 (∆y) γ 1,7 (∆D + ) γ 1,8 (∆D + ) 1 0 γ 2,3 (∆x) γ 2,4 (∆D − ) γ 2,5 (∆D + ) γ 2,6 (∆D − ) γ 2,7 (∆y)
where N (∆D − ) is the number of pixel pairs separated by (∆x, ∆y) in the antidiagonal direction.
Using the form of Eq. 15, the kriging weights can be determined by solving for w in the following ordinary kriging system represented in matrix form as where A, w, and b are given in Table I. It can be seen that A and b contain terms of statistical distances that incorporate anisotropy, spatial correlation, and clustering information; and therefore provide the set of weights for estimation according to particular pattern of spatial continuity [12]. As a result, the 3 × 3 kriging-weighted kernel for Laplacian filter L 2 is of variable coefficients, depending on the anisotropic semivariogram of an image, where, being similar to the Laplacian kernel, the weight at the kernel center is set to -1 to impose that the sum of all elements of the kernel is zero so that the convolution of a homogeneous region is zero. Likewise, the kriging-weighted kernel for Laplacian filter L 1 can be constructed by considering only f (a 2 ), f (a 4 ), f (a 5 ), and f (a 7 ) for the estimate of f (a 0 ) to compute corresponding w * 1 , w * 2 , w * 3 , and w * 4 , respectively, 4 i=1 w * i = 1; whereas weights for the diagonal elements are set to zero. The corresponding kriging system yields A * , w * , and b * of size 5 × 5, 5 × 1, and 5 × 1, respectively. Thus, kriging-weighted kernels for L 1 and L 2 are defined, respectively, as Because convolution is commutative: α ⊛ β = β ⊛ α, and associative: α ⊛ (β ⊛ κ) = (α ⊛ β) ⊛ κ, where α, β, and κ are kernels, multistage filtering therefore can be performed in a single operation. Instead of having an image convolved with a Laplacian kernel first, then the result convolved with the kriging-weighted kernel, the convolution of an image with the kriging-weighted Laplacian kernel can be carried out as where ω i is a hybrid filter or kriging-weighted Laplacian kernel that is defined as III. RESULTS Fig. 2 shows the 225 × 225 "Lena" (a), 195 × 259 "bridge" (e), and 182 × 278 "nature" (i) images used to illustrate the performance of the kriging-weighted Laplacian kernels. For the the LoG, σ = 0.5 and the filter size is 5 × 5. Fig. 2 also shows images filtered by Laplacian L 2 , LoG, and krigingweighted Laplacian ω 2 kernels. For the "Lena" image, it can be seen that while both L 2 and LoG sharpened the image but also highlighted pixels of high changes in intensity, yielding undesirable visual effects such as pixels of the eyes and hair; the ω 2 kernel provided a sharpened image with a much better visual appearance. Similar observations can be made for the "bridge" image, particularly with the suspension cables. For the "nature" image, the L 2 kernel yielded most dominantly noise-sensitive results, while the ω 2 kernel could balance both sharpness and noise suppression.
A measure of sharpness in grayscale images using gradients [13], and the perception-based image quality evaluator (PIQE) [14], which is a no-reference image-quality metric, were used to evaluate both sharpness and quality of the images filtered by the Laplacian (L 1 and L 2 ), LoG, and krigingweighted Laplacian (ω 1 and ω 2 ) filters. The PIQE score is inversely correlated to the perceptual quality of an image in the range [0, 100]. A lower score indicates higher perceptual image quality. Here, the absolute relative difference in quality between the original and a filtered image is defined as where f and f * are original and filtered images, respectively. Table II shows the measure of sharpness and |∆ r | of the three original and corresponding images filtered by the Laplacian (L 1 and L 2 ), LoG, and kriging-weighted Laplacian (ω 1 and ω 2 ) kernels. All filtered images increased the sharpness of the three original images. While being able to increase sharpness in images, both ω 1 and ω 2 provided results that have the most similar naturally visual appearance to the original images by avoiding over-highlighting fine details with a sharp discontinuity. Particularly, as seen in Fig. 2 (b) and (j) and corresponding results shown in Table II, L2 yielded highest measures of image sharpness but resulted in most unnatural visual expression. The LoG filter highlights the fine details to a lesser degree than the Laplacian filters.

IV. CONCLUDING REMARKS
A geostatistical approach for incorporating anisoptropic information into the isotropic Laplacian filters has been presented. Results obtained from the examples illustrate the effective performance of the kriging-weighted Laplacian kernels. Kriging-weighted kernels of difference sizes can be derived for convolving with other types of filters, and used as an adaptive method by considering local directional semivariograms in an image. Furthermore, kriging-weighted kernels can be applied as a stand-alone filters for image smoothing and enhancement.
Code availability: MATLAB codes implemented in this study are available at: https://sites.google.com/view/ tuan-d-pham/codes, under the name "Kriging-weighted Laplacian kernels".