Sobel Operator

The Sobel operator approximates the gradient (the rate and direction of change) of the intensity of an image. In other words, it detects edges.

The operator is separable, which means that you can compute the one-dimensional gradients along xxx and yyy separately by convolving a given 3×3 kernel with the image, and then combine them. The kernel for xxx is

Kx=[−1, 0, +1,-2, 0, +2,-1, 0, +1]\n\mathbf{K}_{x} = {\begin{bmatrix}-1 & 0 & +1 \-2 & 0 & +2 \-1 & 0 & +1\end{bmatrix}}, \nKx=⎣⎢⎡​−1−2−1​000​+1+2+1​⎦⎥⎤​,

and the kernel for yyy is

Ky=[−1, −2, −1, 0, 0, 0, +1, +2, +1]\n\mathbf{K}_{y} = {\begin{bmatrix}-1 & -2 & -1 \\0 & 0 & 0\ +1 & +2 & +1\end{bmatrix}}.\nKy=⎣⎢⎡​−10+1​−20+2​−10+1​⎦⎥⎤​.

To apply the convolution operator ∗, for each pixel a_{i,j} in the image A, center the 3×3 kernel K around the pixel, multiply each element of the kernel by its corresponding pixel value, and then sum.

The gradient along xxx at ⟨i,j⟩ is thus

g_x(i,j)=[Kx \ast \begin{bmatrix} a_{i−1,j−1} & a_{i,j−1} & a_{i+1,j−1} \ a_{i−1,j} & a_{i,j} & a_{i+1,j} \ a_{i−1,j+1} & a_{i,j+1} & a_{i+1,j+1} \end{bmatrix}].

g_{x(i,j)} = {\begin{bmatrix}-1 & 0 & +1 \-2 & 0 & +2 \-1 & 0 & +1\end{bmatrix}} * {\begin{bmatrix}a_{i−1,j−1}&a_{i,j−1}&a_{i+1,j−1} \ a_{i−1,j}&a_{i,j}&a_{i+1,j} \ a_{i−1,j+1}& a_{i,j+1}& a_{i+1,j+1} \end{bmatrix}} ,

and the gradient along yyy at ⟨i,j⟩ is

g_y(i,j)=[Ky \ast \begin{bmatrix} a_{i−1,j−1} & a_{i,j−1} & a_{i+1,j−1} \ a_{i−1,j} & a_{i,j} & a_{i+1,j} \ a_{i−1,j+1} & a_{i,j+1} & a_{i+1,j+1} \end{bmatrix}].

g_{y(i,j)} = {\begin{bmatrix}-1 & -2 & -1 \-2 & 0 & 2 \+1 & +2 & +1\end{bmatrix}} * {\begin{bmatrix}a_{i−1,j−1}&a_{i,j−1}& a_{i+1,j−1} \ a_{i−1,j}&a_{i,j}&a_{i+1,j} \ a_{i−1,j+1}&a_{i,j+1}& a_{i+1,j+1} \end{bmatrix}} ,

or in standard form,

g_x(i,j)=−a_{i−1,j−1}+a_{i+1,j−1}−2a_{i−1,j}+2a_{i+1,j}−a_{i−1,j+1}+a_{i+1,j+1},\

g_y(i,j)=−a_{i−1,j−1}−2a_{i,j−1}−a_{i+1,j−1}+a_{i−1,j+1}+2a_{i,j+1}+a_{i+1,j+1}.

In code, we can represent these kernels as 9-element array of numbers, where k_{i,j} \in K is the 3j + i element of the array.

Kx = Array(9) [-1, 0, 1, -2, 0, 2, -1, 0, 1] Ky = Array(9) [-1, -2, -1, 0, 0, 0, 1, 2, 1]

To start, let’s compute the gradient for each dimension and each channel (red, green and blue) separately. There are faster ways to do this, but for now, for each pixel, we’ll iterate over each element in the kernel. And since the gradient covers the interval [-255,+255], it can be negative, so we use δx+255/2 for display. This shows gray in areas of constant intensity.

The gradient in xxx highlights the leading edge of the surfboard and arms, as well as the frothy waves.

Meanwhile the gradient in yyy highlights the sides of the surfboard and the waves.

To compute the magnitude of the gradient (shown above in the first image), we compute the length of the vector ⟨Gx,Gy⟩, which is

∣G∣=|\mathbf{G}| = \sqrt{{G_{x}^2 + G_{y}^2}}.

In JavaScript, this can be computed as Math.hypot.

What about grayscale edge detection? For many applications we don’t want to compute red, green and blue edges separately. For grayscale, we first need to know a little about color; we can’t simply average the RGB channel values before applying the Sobel operator.

Humans do not perceive intensity linearly in [0,255]—those values are intended for display hardware—so we must linearize before averaging. The sRGB specification (and CSS Color Level 4) specifies how to convert between sRGB in [0,255] and “uncompanded” linear light in [0,1].

The sRGB specification also says how to convert linear RGB values to luminosity Y:

Y=0.2126R+0.7152G+0.0722B.

Now we have an accurate grayscale.

The grayscale gradient in xxx, again normalized to [0,255]:

The grayscale gradient in yyy:

And the magnitude of the gradient ∣G∣,

Since the Sobel operator computes the gradient G and not merely its magnitude, we can also visualize the direction of the gradient,

Θ=atan(Gy/Gx).

We can even visualize the direction of the gradient in RGB, and its magnitude in alpha, resulting in some beautiful rainbow tendrils.

Appendix

Photo: Kiril Dobrev/Pixabay. This notebook was inspired by a tutorial on seam carving by Avik Das, a topic I hope to visit in the near future.