# SVR

## 问题构建

构建一个宽度为2ε的间隔带，若训练**样本落入此间隔带，则认为是被预测正确的**。图像：

![img](https://img-blog.csdnimg.cn/20190221092020343.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

因此SVR问题可转化为：

$$min\_{w,b}\frac{1}{2}||w||^2+C \sum\_{i=1}^m L(f(x\_i)-y\_i)$$

其中L是误差计算函数，对小于$$\epsilon$$的误差为0：

$$
\begin{aligned}
L(z) =
\begin{cases}
0,|z|<\epsilon\\
|z|-\epsilon,other
\end{cases}
\end{aligned}
$$

同样是数字，**引入松弛变量替代上下两边的损失值**，（间隔带两侧的松弛程度可有所不同），则变成：

$$min\_{w,b,\xi\_i,\hat{\xi\_i}}\frac{1}{2}||w||^2+C \sum\_{i=1}^m (\xi\_i+ \hat{\xi\_i})$$

约束（即预测值与真实的差$$\le\epsilon + \xi$$，也就是要小于，间隔区间外+损失值$$\xi$$部分那个段位，再超就违反推理了）：

$$
\begin{aligned}
s.t... \&f(x\_i)-y\_i \le \epsilon+\xi\_i \\
\&y\_i-f(x\_i) \le \epsilon +\hat{\xi\_i} \\
&\xi\_i \ge 0,\hat{\xi\_i} \ge 0,i=1,2,...,m
\end{aligned}
$$

引入拉格朗日乘子$$\hat{\alpha},\alpha$$，可得拉格朗日函数：

$$
\begin{aligned}
\frac{1}{2}||w||^2+C \sum\_{i=1}^m (\xi\_i+ \hat{\xi\_i}) + \sum\_{i=1}^m \alpha\_i(f(x\_i)-y\_i-\epsilon -\xi\_i)\\
+\sum\_{i=1}^m \hat{\alpha\_i}(y\_i-f(x\_i)-\epsilon -\hat{\xi\_i})-\sum\_{i=1}^m\mu\_{i1}\xi\_i-\sum\_{i=1}^m\mu\_{i2}\hat{\xi\_i}
\end{aligned}
$$

存在不等式约束，所以上式**还有KKT条件等约束（参考前面的SVM，后面也有）**，同样的求导，然后对偶（参考SVM啦）。

求导：

$$
\begin{aligned}
\&w=\sum\_{i=1}^m(\hat{\alpha\_i}-\alpha\_i)x\_i\\
&0=\sum\_{i=1}^m(\hat{\alpha\_i}-\alpha\_i)\\
\&C=\alpha\_i+\mu\_i\\
\&C=\hat{\alpha\_i}+\hat{\mu\_i}
\end{aligned}
$$

把上边的式子带入，即可求得SVR的对偶问题（和SVM分类很像！）：

$$
\begin{aligned}
\&max\_{\alpha,\hat{\alpha}}\sum\_{i=1}^my\_i(\hat{\alpha\_i}-\alpha\_i)-\epsilon(\hat{\alpha\_i}+\alpha\_i)-\frac{1}{2}\sum\_{i=1}^m\sum\_{j=1}^m(\hat{\alpha\_i}-\alpha\_i)(\hat{\alpha\_j}-\alpha\_j)x\_i^Tx\_j\\
\&s.t....\sum\_{i=1}^m(\hat{\alpha\_i}-\alpha\_i)=0\\
&0 \le\alpha\_i,\hat{\alpha\_i} \le C
\end{aligned}
$$

KKT条件：

$$
\begin{cases}
\alpha\_i(f(x\_i)-y\_i-\epsilon-\xi\_i)=0\\
\hat{\alpha\_i}(y\_i-f(x\_i)-\epsilon -\hat{\xi\_i})=0\\
\alpha\_i \hat{\alpha\_i}=0,\xi\_i \hat{\xi\_i}=0\\
(C-\alpha\_i)\xi\_i=0\\
(C-\hat{\alpha\_i})\hat{\xi\_i}=0
\end{cases}
$$

如果求出a的话最后，可得SVR的模型（和SMO一样求）：

$$
\begin{aligned}
\&f(x)=\sum\_{i=1}^m (\hat{\alpha\_i}-\alpha\_i)x\_i^Tx+b\\
\&b=y\_i+\epsilon-\sum\_{i=1}^m (\hat{\alpha\_i}-\alpha\_i)x\_i^Tx
\end{aligned}
$$


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://im-qianuxn.gitbook.io/pytorch/ji-suan-ji/ml/7-zhi-chi-xiang-liang/svr.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.