Python与FPGA——图像锐化

文章目录

• 前言
• 一、图像锐化
• 二、Python robert锐化
• 三、Python sobel锐化
• 四、Python laplacian锐化
• 五、FPGA sobel锐化
• ���结

  ---

  前言

    在增强图像之前一般会先对图像进行平滑处理以减少或消除噪声，图像的能量主要集中在低频部分，而噪声和图像边缘信息的能量主要集中在高频部分。因此，平滑处理会使原始的图像边缘和轮廓变得模糊。为了减少不利效果的影响，需要利用图像锐化技术。

  ---

  一、图像锐化

    图像锐化其实就是使用robert，sobel，laplacian这些人发明的窗口，进行图像的处理。图像锐化过程和sobel边缘检测的过程类似，可以移步至《Python与FPGA——sobel边缘检测》课程，一探究竟。

  一阶微分的边缘检测

    图像f（x, y）在像素（x, y）梯度的定义为

  G = ∂ f ∂ x + ∂ f ∂ y G = \frac{\partial f}{\partial x} + \frac{\partial f}{\partial y} G=∂x∂f​+∂y∂f​

  也可以用差分来替代微分，即

  ∂ f ∂ x = f ( i + 1 , j ) − f ( i , j ) \frac{\partial f}{\partial x} = f(i + 1, j) - f(i, j) ∂x∂f​=f(i+1,j)−f(i,j)

  ∂ f ∂ y = f ( i , j + 1 ) − f ( i , j ) \frac{\partial f}{\partial y} = f(i, j + 1) - f(i, j) ∂y∂f​=f(i,j+1)−f(i,j)

  梯度的幅值即模值，为

  ∣ G ∣ = ( ∂ f ∂ x ) 2 + ( ∂ f ∂ y ) 2 = [ f ( i + 1 , j ) − f ( i , j ) ] 2 + [ f ( i , j ) − f ( i , j ) ] 2 |G| = \sqrt{(\frac{\partial f}{\partial x})^2 + (\frac{\partial f}{\partial y})^2} = \sqrt{[f(i + 1, j) - f(i, j)]^2 + [f(i, j ) - f(i, j)]^2} ∣G∣=(∂x∂f​)2+(∂y∂f​)2 ​=[f(i+1,j)−f(i,j)]2+[f(i,j)−f(i,j)]2 ​

  梯度方向为

  θ = a r c t a n ( ∂ f ∂ y / ∂ f ∂ x ) = a r c t a n [ f ( i , j + 1 ) − f ( i , j ) f ( i + 1 , j ) − f ( i , j ) ] \theta = arctan(\frac{\partial f}{\partial y}/\frac{\partial f}{\partial x}) = arctan[\frac{f(i, j + 1) - f(i, j)}{f(i + 1, j) - f(i, j)}] θ=arctan(∂y∂f​/∂x∂f​)=arctan[f(i+1,j)−f(i,j)f(i,j+1)−f(i,j)​]

  图像f(i, j)处的梯度g为

  g ( i , j ) = G [ f ( i , j ) ] g(i, j) = G[f(i, j)] g(i,j)=G[f(i,j)]

  使用 g ( i , j ) g(i, j) g(i,j)去替代原来的像素。

    一阶导算子有robert算子，perwitt算子，sobel算子。

  1. Roberts算子

  G x = [ 1 0 0 − 1 ] G y = [ 0 − 1 1 0 ] G_x = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix} \quad\quad\quad G_y = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix} Gx​=[10​0−1​]Gy​=[01​−10​]

  2. Prewitt算子

  G x = [ − 1 0 1 − 1 0 1 − 1 0 1 ] G y = [ − 1 − 1 − 1 0 0 0 1 1 1 ] G_x = \begin{bmatrix} -1 & 0 & 1\\ -1 & 0 & 1\\ -1 & 0 & 1 \end{bmatrix} \quad\quad\quad G_y = \begin{bmatrix} -1 & -1 & -1\\ 0 & 0 & 0\\ 1 & 1 & 1 \end{bmatrix} Gx​=⎣⎡​−1−1−1​000​111​⎦⎤​Gy​=⎣⎡​−101​−101​−101​⎦⎤​

  3. Sobel算子

  G x = [ − 1 0 + 1 − 2 0 + 2 − 1 0 + 1 ] G y = [ + 1 + 2 + 1 0 0 0 − 1 − 2 1 ] G_x = \begin{bmatrix} -1 & 0 & +1\\ -2 & 0 & +2\\ -1 & 0 & +1 \end{bmatrix} \quad\quad\quad G_y = \begin{bmatrix} +1 & +2 & +1\\ 0 & 0 & 0\\ -1 & -2 & 1 \end{bmatrix} Gx​=⎣⎡​−1−2−1​000​+1+2+1​⎦⎤​Gy​=⎣⎡​+10−1​+20−2​+101​⎦⎤​

  二阶微分的边缘检测

    二阶微分公式用差分法，推理如下

  ∂ 2 f ∂ x 2 = 2 f ( x , y ) − f ( x − 1 , y ) − f ( x + 1 , y ) \frac{\partial^2 f}{\partial x^2}=2f(x,y)-f(x-1,y)-f(x+1, y) ∂x2∂2f​=2f(x,y)−f(x−1,y)−f(x+1,y)

  ∂ 2 f ∂ y 2 = 2 f ( x , y ) − f ( x , y − 1 ) − f ( x , y + 1 ) \frac{\partial^2 f}{\partial y^2}=2f(x,y)-f(x,y-1)-f(x, y+1) ∂y2∂2f​=2f(x,y)−f(x,y−1)−f(x,y+1)

  ▽ 2 f = 4 f ( x , y ) − [ f ( x − 1 , y ) + f ( x , y − 1 ) + f ( x , y + 1 ) + f ( x + 1 , y ) ] \triangledown^2f=4f(x,y)-[f(x-1,y)+f(x,y-1)+f(x,y+1)+f(x+1,y)] ▽2f=4f(x,y)−[f(x−1,y)+f(x,y−1)+f(x,y+1)+f(x+1,y)]

  符合二阶微分的算子是laplacian。

  G x = [ 0 − 1 0 − 1 4 − 1 0 − 1 0 ] G y = [ − 1 − 1 − 1 − 1 8 − 1 − 1 − 1 − 1 ] G_x = \begin{bmatrix} 0 & -1 & 0\\ -1 & 4 & -1\\ 0 & -1 & 0 \end{bmatrix} \quad\quad\quad G_y = \begin{bmatrix} -1 & -1 & -1\\ -1 & 8 & -1\\ -1 & -1 & -1 \end{bmatrix} Gx​=⎣⎡​0−10​−14−1​0−10​⎦⎤​Gy​=⎣⎡​−1−1−1​−18−1​−1−1−1​⎦⎤​

  二、Python robert锐化

  import numpy as npimport matplotlib.pyplot as pltdef image_gray(image):    gray = np.dot(image[:, :, ...], [0.299, 0.587, 0.114])#等同0.299 * image[:, :, 0] + 0.587 * image[:, :, 1] + 0.114 * image[:, :, 2]    return gray.astype(np.uint8)    def robert_sharpen(image, gx, gy):    h, w = image.shape    n, n = gx.shape    filtered_image = np.zeros((h, w))    m = int(n / 2)    for i in range(m, h - m):        for j in range(m, w - m):               gx_value = np.sum(np.multiply(gx, image[i - m: i + m, j - m: j + m]))            gy_value = np.sum(np.multiply(gy, image[i - m: i + m, j - m: j + m]))            gxy_value = np.sqrt(gx_value ** 2 + gy_value ** 2)            filtered_image[i, j] = gxy_value    return filtered_image.astype(np.uint8)    img = plt.imread("lenna.png")img = img * 255#图像是[0-1]--->[0-255],确认一下自己的图像是[0-1]还是[0-255]img = img.astype(np.uint8)gx = np.array([[1, 0],               [0, -1]])gy = np.array([[0, 1],               [-1, 0]])gray = image_gray(img)robert_image = robert_sharpen(gray, gx, gy)fig = plt.figure(figsize=(10, 6))ax = plt.subplot(1, 2, 1)ax.set_title("raw image")ax.set_xlabel("width")ax.set_ylabel("height")plt.imshow(gray, cmap="gray")ax = plt.subplot(1, 2, 2)ax.set_title("robert image")ax.set_xlabel("width")ax.set_ylabel("height")plt.imshow(robert_image, cmap="gray")

  三、Python sobel锐化

  import numpy as npimport matplotlib.pyplot as pltdef image_gray(image):    gray = np.dot(image[:, :, ...], [0.299, 0.587, 0.114])#等同0.299 * image[:, :, 0] + 0.587 * image[:, :, 1] + 0.114 * image[:, :, 2]    return gray.astype(np.uint8)    def sobel_sharpen(image, gx, gy):    h, w = image.shape    n, n = gx.shape    filtered_image = np.zeros((h, w))    m = int((n-1) / 2)    for i in range(m, h - m):        for j in range(m, w - m):               gx_value = np.sum(np.multiply(gx, image[i - m: i + m + 1, j - m: j + m + 1]))            gy_value = np.sum(np.multiply(gy, image[i - m: i + m + 1, j - m: j + m + 1]))            gxy_value = np.sqrt(gx_value ** 2 + gy_value ** 2)            filtered_image[i, j] = gxy_value    return filtered_image.astype(np.uint8)    img = plt.imread("lenna.png")img = img * 255#图像是[0-1]--->[0-255],确认一下自己的图像是[0-1]还是[0-255]img = img.astype(np.uint8)gx = np.array([[-1, 0, 1],               [-2, 0, 2],               [-1, 0, 1]])gy = np.array([[-1, -2, -1],               [0, 0, 0],               [1, 2, 1]])gray = image_gray(img)sobel_image = sobel_sharpen(gray, gx, gy)fig = plt.figure(figsize=(10, 6))ax = plt.subplot(1, 2, 1)ax.set_title("raw image")ax.set_xlabel("width")ax.set_ylabel("height")plt.imshow(gray, cmap="gray")ax = plt.subplot(1, 2, 2)ax.set_title("sobel image")ax.set_xlabel("width")ax.set_ylabel("height")plt.imshow(sobel_image, cmap="gray")

  四、Python laplacian锐化

  import numpy as npimport matplotlib.pyplot as pltdef image_gray(image):    gray = np.dot(image[:, :, ...], [0.299, 0.587, 0.114])#等同0.299 * image[:, :, 0] + 0.587 * image[:, :, 1] + 0.114 * image[:, :, 2]    return gray.astype(np.uint8)    def laplacian_sharpen(image, gx, gy):    h, w = image.shape    n, n = gx.shape    filtered_image = np.zeros((h, w))    m = int((n-1) / 2)    for i in range(m, h - m):        for j in range(m, w - m):               gx_value = np.sum(np.multiply(gx, image[i - m: i + m + 1, j - m: j + m + 1]))            gy_value = np.sum(np.multiply(gy, image[i - m: i + m + 1, j - m: j + m + 1]))            gxy_value = np.sqrt(gx_value ** 2 + gy_value ** 2)            filtered_image[i, j] = gxy_value    return filtered_image.astype(np.uint8)    img = plt.imread("lenna.png")img = img * 255#图像是[0-1]--->[0-255],确认一下自己的图像是[0-1]还是[0-255]img = img.astype(np.uint8)gx = np.array([[0, -1, 0],               [-1, 4, -1],               [0, -1, 0]])gy = np.array([[-1, -1, -1],               [-1, 8, -1],               [-1, -1, -1]])gray = image_gray(img)sobel_image = sobel_sharpen(gray, gx, gy)fig = plt.figure(figsize=(10, 6))ax = plt.subplot(1, 2, 1)ax.set_title("raw image")ax.set_xlabel("width")ax.set_ylabel("height")plt.imshow(gray, cmap="gray")ax = plt.subplot(1, 2, 2)ax.set_title("sobel image")ax.set_xlabel("width")ax.set_ylabel("height")plt.imshow(sobel_image, cmap="gray")

  五、FPGA sobel锐化

  //3*3图像//P11   P12   P13//P21   P22   P23//P31   P32   P33//Gx算子//-1     0     1//-2     0     2//-1     0     1//Gx = -P11 + P13 - 2*P21 + 2*P23 - P31 + P33//Gx = (P13 - P11) + 2*(P23 - P21) + (P33 - P31)//Gy算子//1      2     1//0      0     0//-1     -2    -1//Gy = P11 + 2*P12 + P13 - P31 - 2*P32 - P33//Gy = (P11 - P31) + 2*(P12 - P32) + (P13 - P33)module  ycbcr_sobel_sharpen(inputwiresys_clk,//系统时钟，频率为50MHZinputwiresys_rst_n,//系统复位，低电平有效inputwirergb_valid,//RGB565图像显示有效信号inputwire[7:0]y_data,//Y分量inputwire[11:0]pixel_x,//有效显示区域横坐标inputwire[11:0]pixel_y,//有效显示区域纵坐标outputreg[15:0]sobel_data//Sobel算法处理后的图像数据);regy_valid;//Y分量有效信号//shift ramwire[7:0]data_row1;//移位寄存器第一行数据wire[7:0]data_row2;//移位寄存器第二行数据wire[7:0]data_row3;//移位寄存器第三行数据//3*3像素数据，左上角至右下角共9个数据reg[7:0]p11;//3*3第1个像素数据reg[7:0]p12;//3*3第2个像素数据reg[7:0]p13;//3*3第3个像素数据reg[7:0]p21;//3*3第4个像素数据reg[7:0]p22;//3*3第5个像素数据reg[7:0]p23;//3*3第6个像素数据reg[7:0]p31;//3*3第7个像素数据reg[7:0]p32;//3*3第8个像素数据reg[7:0]p33;//3*3第9个像素数据//Sobel算子wire[15:0]Gx;//水平梯度值wire[15:0]Gy;//数值梯度值wire[7:0]Gxy;//总体梯度值assign  data_row3 = y_data  ;assign  Gx = (p13 - p11) + 2*(p23 - p21) + (p33 - p31)  ;assign  Gy = (p11 - p31) + 2*(p12 - p32) + (p13 - p33)  ;//设定第一行、第二行，第一列、第二列显示全白色always@(*)if((pixel_y == 12'd0)||(pixel_y == 12'd1)||(pixel_x == 12'd2)||(pixel_x == 12'd3))sobel_data = 16'hffff  ;elsesobel_data = {Gxy[7:3],Gxy[7:2],Gxy[7:3]}  ;//锐化核心代码always@(posedge sys_clk or negedge sys_rst_n)if(sys_rst_n == 1'b0)y_valid  p11,p12,p13}