行业知识

四、多特征的线性回归模型

2026-11-13 07:18:09 来源：互联网转载

三、梯度下降法-CSDN博客文章浏览阅读1.1k次，点赞39次，收藏19次。本文主要介绍了机器学习中的线性回归模型使用梯度下降法寻找最优的w和b，并且对学习率在梯度下降法中的作用做一定示例演示，还对官方的梯度下降法演示lab进行整理归纳讲解，从多维度学习理解梯度下降法在线性回归模型中的应用过程。https://blog.csdn.net/hehe_soft_engineer/article/details/139380896

一、问题背景

房屋价格并不是一个变量决定的，而是由多个因素共同作用决定的

二、多特征(n个特征)变量的线性回归表示方法

三、向量化

在python中如何表示并计算多特征线性回归模型(使用numpy工具包)

1.向量表示

# 向量表示w = np.array([1.0,2.5,-3.3])b = 4x = np.array([10,20,30])

在python中数组下标是从0开始的，也就是向量w中的w1在向量中用w[0]表示。

2.两种计算方式(第二种效率更高)

# 一般计算方式fw,b(x)f = 0for j in range(0,n):    f = f + w[j] * x[j]f = f + b# 其中 range(0,n) -> j = 0,1,2,…,n-1

# 直接用numpy库dot点乘函数计算fw,b(x)f = np.dot(w,x) + b

3.python中的向量化实验(C1_W2_Lab01_Python_Numpy_Vectorization_Soln)

(1)向量创建的各种示例

# NumPy routines which allocate memory and fill arrays with valuea = np.zeros(4);                print(f"np.zeros(4) :   a = {a}, a shape = {a.shape}, a data type = {a.dtype}")a = np.zeros((4,));             print(f"np.zeros(4,) :  a = {a}, a shape = {a.shape}, a data type = {a.dtype}")a = np.random.random_sample(4); print(f"np.random.random_sample(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}")

其中a.shape表示a的维度，(4,) 表示有4个元素的一维向量

# NumPy routines which allocate memory and fill arrays with value but do not accept shape as input argumenta = np.arange(4.);              print(f"np.arange(4.):     a = {a}, a shape = {a.shape}, a data type = {a.dtype}")a = np.random.rand(4);          print(f"np.random.rand(4): a = {a}, a shape = {a.shape}, a data type = {a.dtype}")

也可以手动指定元素：

# NumPy routines which allocate memory and fill with user specified valuesa = np.array([5,4,3,2]);  print(f"np.array([5,4,3,2]):  a = {a},     a shape = {a.shape}, a data type = {a.dtype}")a = np.array([5.,4,3,2]); print(f"np.array([5.,4,3,2]): a = {a}, a shape = {a.shape}, a data type = {a.dtype}")

(2)向量操作

①索引

#vector indexing operations on 1-D vectorsa = np.arange(10)print(a)#access an elementprint(f"a[2].shape: {a[2].shape} a[2]  = {a[2]}, Accessing an element returns a scalar")# access the last element, negative indexes count from the endprint(f"a[-1] = {a[-1]}")#indexs must be within the range of the vector or they will produce and errortry:    c = a[10]except Exception as e:    print("The error message you'll see is:")    print(e)

②切片Slicing

#vector slicing operationsa = np.arange(10)print(f"a         = {a}")#access 5 consecutive elements (start:stop:step)c = a[2:7:1];     print("a[2:7:1] = ", c)# access 3 elements separated by twoc = a[2:7:2];     print("a[2:7:2] = ", c)# access all elements index 3 and abovec = a[3:];        print("a[3:]    = ", c)# access all elements below index 3c = a[:3];        print("a[:3]    = ", c)# access all elementsc = a[:];         print("a[:]     = ", c)

③对向量进行符号运算

a = np.array([1,2,3,4])print(f"a             : {a}")# negate elements of ab = -aprint(f"b = -a        : {b}")# sum all elements of a, returns a scalarb = np.sum(a)print(f"b = np.sum(a) : {b}")b = np.mean(a)print(f"b = np.mean(a): {b}")b = a**2print(f"b = a**2      : {b}")

④对两个向量进行加减操作

# 相同size可以正常执行a = np.array([ 1, 2, 3, 4])b = np.array([-1,-2, 3, 4])print(f"Binary operators work element wise: {a + b}")

# 不同size不能正常执行，报告错误#try a mismatched vector operationc = np.array([1, 2])try:    d = a + cexcept Exception as e:    print("The error message you'll see is:")    print(e)

⑤标量和向量的运算

a = np.array([1, 2, 3, 4])# multiply a by a scalarb = 5 * aprint(f"b = 5 * a : {b}")

⑥向量之间的点积运算

# test 1-Da = np.array([1, 2, 3, 4])b = np.array([-1, 4, 3, 2])c = np.dot(a, b)print(f"NumPy 1-D np.dot(a, b) = {c}, np.dot(a, b).shape = {c.shape} ")c = np.dot(b, a)print(f"NumPy 1-D np.dot(b, a) = {c}, np.dot(a, b).shape = {c.shape} ")

⑦ 比较自己实现和numpy库实现的点积计算速度

#自己实现的点积计算def my_dot(a, b):    """   Compute the dot product of two vectors    Args:      a (ndarray (n,)):  input vector      b (ndarray (n,)):  input vector with same dimension as a        Returns:      x (scalar):    """    x=0    for i in range(a.shape[0]):        x = x + a[i] * b[i]    return x

上面函数和numpy.dot()比较：

np.random.seed(1)a = np.random.rand(10000000)  # very large arraysb = np.random.rand(10000000)tic = time.time()  # capture start timec = np.dot(a, b)toc = time.time()  # capture end timeprint(f"np.dot(a, b) =  {c:.4f}")print(f"Vectorized version duration: {1000*(toc-tic):.4f} ms ")tic = time.time()  # capture start timec = my_dot(a,b)toc = time.time()  # capture end timeprint(f"my_dot(a, b) =  {c:.4f}")print(f"loop version duration: {1000*(toc-tic):.4f} ms ")del(a);del(b)  #remove these big arrays from memory

可以看到numpy计算效率高很多，这是因为NumPy更好地利用了底层硬件的数据并行性。GPU和现代CPU实现了单指令多数据(SIMD)管道，允许并行发出多个操作。这在机器学习中至关重要，因为数据集通常非常大。

(3)矩阵的创建

a = np.zeros((3, 5))                                       print(f"a shape = {a.shape}, a = \n{a}")                     a = np.zeros((2, 1))                    print(f"a shape = {a.shape}, a = {a}")a = np.random.random_sample((1, 1))  print(f"a shape = {a.shape}, a = {a}")

也可以手动指定矩阵元素

# NumPy routines which allocate memory and fill with user specified valuesa = np.array([[5], [4], [3]]);   print(f" a shape = {a.shape}, np.array: a = \n{a}")a = np.array([[5],   # One can also              [4],   # separate values              [3]]); #into separate rowsprint(f" a shape = {a.shape}, np.array: a = \n{a}")

(4)矩阵操作

①索引[row, column]定位

#vector indexing operations on matricesa = np.arange(6).reshape(-1, 2)   #reshape is a convenient way to create matricesprint(f"a.shape: {a.shape}, \na= {a}")#access an elementprint(f"\na[2,0].shape:   {a[2, 0].shape}, a[2,0] = {a[2, 0]},     type(a[2,0]) = {type(a[2, 0])} Accessing an element returns a scalar\n")#access a rowprint(f"a[2].shape:   {a[2].shape}, a[2]   = {a[2]}, type(a[2])   = {type(a[2])}")

②矩阵切片

#vector 2-D slicing operationsa = np.arange(20).reshape(-1, 10)print(f"a = \n{a}")#access 5 consecutive elements (start:stop:step)print("a[0, 2:7:1] = ", a[0, 2:7:1], ",  a[0, 2:7:1].shape =", a[0, 2:7:1].shape, "a 1-D array")#access 5 consecutive elements (start:stop:step) in two rowsprint("a[:, 2:7:1] = \n", a[:, 2:7:1], ",  a[:, 2:7:1].shape =", a[:, 2:7:1].shape, "a 2-D array")# access all elementsprint("a[:,:] = \n", a[:,:], ",  a[:,:].shape =", a[:,:].shape)# access all elements in one row (very common usage)print("a[1,:] = ", a[1,:], ",  a[1,:].shape =", a[1,:].shape, "a 1-D array")# same asprint("a[1]   = ", a[1],   ",  a[1].shape   =", a[1].shape, "a 1-D array")

五、多特征线性回归模型

1.之前使用设置变量方法来表示 wi 和向量表示法对比

可以看到我们使用向量表示法后多大的维度只需要用一个向量表示，而不需要设置一堆变量然后去更新。

2.单特征梯度下降和多特征梯度下降对比

3.多特征线性回归模型实验(C1_W2_Lab02_Multiple_Variable_Soln)

(1).背景及提供的数据集

你将使用房价预测的例子。训练数据集包含三个示例(三行数据)，具有四个特征(大小、卧室、楼层和年龄)，如下表所示。请注意，与之前的实验不同，这里的尺寸是平方英尺，而不是1000平方英尺。这导致了一个问题，你将在下一个实验中解决!

# 向量化X_train = np.array([[2104, 5, 1, 45], [1416, 3, 2, 40], [852, 2, 1, 35]])y_train = np.array([460, 232, 178])

(2).参数初始化

b_init = 785.1811367994083w_init = np.array([ 0.39133535, 18.75376741, -53.36032453, -26.42131618])print(f"w_init shape: {w_init.shape}, b_init type: {type(b_init)}")

初始化一个四维向量，对应于数据集的四个维度

(3).实现多维度线性回归模型代价计算

# 实现 (3) 公式的计算def compute_cost(X, y, w, b):     """    compute cost    Args:      X (ndarray (m,n)): Data, m examples with n features      y (ndarray (m,)) : target values      w (ndarray (n,)) : model parameters        b (scalar)       : model parameter          Returns:      cost (scalar): cost    """    m = X.shape[0]    cost = 0.0    for i in range(m):                                        f_wb_i = np.dot(X[i], w) + b           #(n,)(n,) = scalar (see np.dot)        cost = cost + (f_wb_i - y[i])**2       #scalar    cost = cost / (2 * m)                      #scalar        return cost

测试用初始值w_init和b_init计算代价值

# Compute and display cost using our pre-chosen optimal parameters. cost = compute_cost(X_train, y_train, w_init, b_init)print(f'Cost at optimal w : {cost}')

(4).实现多变量线性回归模型梯度计算

# 实现 (6) (7) 公式的计算def compute_gradient(X, y, w, b):     """    Computes the gradient for linear regression     Args:      X (ndarray (m,n)): Data, m examples with n features      y (ndarray (m,)) : target values      w (ndarray (n,)) : model parameters        b (scalar)       : model parameter          Returns:      dj_dw (ndarray (n,)): The gradient of the cost w.r.t. the parameters w.       dj_db (scalar):       The gradient of the cost w.r.t. the parameter b.     """    m,n = X.shape           #(number of examples, number of features)    dj_dw = np.zeros((n,))    dj_db = 0.    for i in range(m):                                     err = (np.dot(X[i], w) + b) - y[i]           for j in range(n):                                     dj_dw[j] = dj_dw[j] + err * X[i, j]            dj_db = dj_db + err                            dj_dw = dj_dw / m                                    dj_db = dj_db / m                                            return dj_db, dj_dw

测试一下初始情况下的梯度

#Compute and display gradient tmp_dj_db, tmp_dj_dw = compute_gradient(X_train, y_train, w_init, b_init)print(f'dj_db at initial w,b: {tmp_dj_db}')print(f'dj_dw at initial w,b: \n {tmp_dj_dw}')

(5).将上面步骤(3)(4)实现的代码代入梯度下降算法的上面公式(5)中

def gradient_descent(X, y, w_in, b_in, cost_function, gradient_function, alpha, num_iters):     """    Performs batch gradient descent to learn theta. Updates theta by taking     num_iters gradient steps with learning rate alpha        Args:      X (ndarray (m,n))   : Data, m examples with n features      y (ndarray (m,))    : target values      w_in (ndarray (n,)) : initial model parameters        b_in (scalar)       : initial model parameter      cost_function       : function to compute cost      gradient_function   : function to compute the gradient      alpha (float)       : Learning rate      num_iters (int)     : number of iterations to run gradient descent          Returns:      w (ndarray (n,)) : Updated values of parameters       b (scalar)       : Updated value of parameter       """        # An array to store cost J and w's at each iteration primarily for graphing later    J_history = []    w = copy.deepcopy(w_in)  #avoid modifying global w within function    b = b_in        for i in range(num_iters):        # Calculate the gradient and update the parameters        dj_db,dj_dw = gradient_function(X, y, w, b)   ##None        # Update Parameters using w, b, alpha and gradient        w = w - alpha * dj_dw               ##None        b = b - alpha * dj_db               ##None              # Save cost J at each iteration        if i

上一篇：揭秘灰度打印，深度探索技术与应用的奥秘

下一篇：服务器内网怎么进?