白话机器学习的数学——回归——作为一次函数实现

预测函数

\[ f_\theta(x) = \theta_0 + \theta_1x \]

目标函数

\[ E(\theta) = \frac 12\sum_{i=1}^n(y^{(i)} - f_\theta(x^{(i)}))^2 \]

标准化

\(\mu\) 是训练数据的平均值，\(\sigma\) 是标准差。

\[ z^{(i)} = \frac{x^{(i)} - \mu}{\sigma} \]

更新表达式

\[ \theta_0 := \theta_0 - \eta\sum_{i=1}^n(f_\theta(x^{(i)}) - y^{(i)}) \]

\[ \theta_1 := \theta_1 - \eta\sum_{i=1}^n(f_\theta(x^{(i)}) - y^{(i)})x^{(i)} \]

代码示例

import numpy as np
import matplotlib.pyplot as plt

# 训练数据
train = np.loadtxt('click.csv', delimiter=',', dtype='int', skiprows=1)
train_x = train[:, 0]
train_y = train[:, 1]

# 参数初始化
theta0 = np.random.rand()
theta1 = np.random.rand()

# 预测函数
def f(x):
    return theta0 + theta1 * x

# 目标函数
def E(x, y):
    return 0.5 * np.sum((y - f(x)) ** 2)

# 标准化
mu = train_x.mean()
sigma = train_x.std()
def standardize(x):
    return (x - mu) / sigma
train_z = standardize(train_x)

# 学习率
ETA = 1e-3
# 误差的差值
diff = 1
# 更新次数
count = 0
# 直到误差的差值小于 0.01 为止，重复参数更新
error = E(train_z, train_y)
while diff > 1e-2:
    # 更新结果保存到临时变量
    tmp_theta0 = theta0 - ETA * np.sum((f(train_z) - train_y))
    tmp_theta1 = theta1 - ETA * np.sum((f(train_z) - train_y) * train_z)
    # 更新参数
    theta0 = tmp_theta0
    theta1 = tmp_theta1
    # 计算与上一次误差的差值
    current_error = E(train_z, train_y)
    diff = error - current_error
    error = current_error
    # 输出日志
    count += 1
    log = '第 {} 次 : theta0 = {:.3f}, theta1 = {:.3f}, 差值 = {:.4f}'
    print(log.format(count, theta0, theta1, diff))

# 绘图确认
x = np.linspace(-3, 3, 100)
plt.plot(train_z, train_y, 'o')
plt.plot(x, f(x))
plt.show()