# 核函数

现实中可能有些不存在线性的可分超平面，但是可能映射到更高维度可能就可分了，有证明显示，如果原始空间维度有限，那么一定存在高维特征空间使样本可分。

$$
\begin{aligned}
\&f(x)=w^Tx+b\\
\&x \to \phi(x)\\
\&f(x)=w^T\phi(x)+b
\end{aligned}
$$

这样对x的映射关系，可以直接用之前推导的所有公式里：

$$
\begin{aligned}
\&min\_{w,b}\frac{1}{2}||w||^2\\
\&s.t. y\_i(w^T\phi(x\_i)+b) \ge 1,i=1,2,..,N
\end{aligned}
$$

**对偶形式映射：**

$$
\begin{aligned}
& max\_{\alpha}\sum\_{i=1}^{N}\alpha\_i-\frac{1}{2}\sum\_{i=1}^{N}\sum\_{j=1}^{N}\alpha\_i\alpha\_jy\_iy\_j\phi(x\_i)^T\phi(x\_j)\\
& s.t. ......
\end{aligned}
$$

这种映射我们并不知道具体是如何的，因此也不知如何去计算了，所以这里就设想出来核函数的概念了：

$$
k(x\_i,x\_j)=\phi(x\_i)^T  \phi(x\_j)
$$

```python
    def kernel(self, x1, x2):
        if self._kernel == 'linear':
            return sum([x1[k] * x2[k] for k in range(self.n)])

        elif self._kernel == 'poly':
            return (sum([x1[k] * x2[k] for k in range(self.n)]) + 1) ** 2

        return 0
```

假设原来的这种内积映射，是等价于某个函数k(.,.)计算的结果。问题就变成了：

$$
\begin{aligned}
& max\_{\alpha}\sum\_{i=1}^{N}\alpha\_i-\frac{1}{2}\sum\_{i=1}^{N}\sum\_{j=1}^{N}\alpha\_i\alpha\_jy\_iy\_jk(x\_i,x\_j)\\
& s.t. ......
\end{aligned}
$$

**模型变成了：**

$$
\begin{aligned}
f(x) &=w^T\phi(x)+b=\sum\_{i=1}^{N}\alpha\_iy\_i\phi(x\_i)^T\phi(x)+b\\
&=\sum\_{i=1}^{N}\alpha\_iy\_i k(x\_i,x)+b\\
\end{aligned}
$$

**核函数性质：**

k是核函数，当且仅当核矩阵K总是半正定

**常见核函数列表：**

线性核、多项式核、高斯核、拉普拉斯核、sigmoid核

$$
\begin{aligned}
&(1):K(X\_i,x\_j)=x\_i^Tx\_j\\
&(2):K(X\_i,x\_j)=(x\_i^Tx\_j)^d \\
&(3):K(X\_i,x\_j)=exp(-\frac{||x\_i-x\_j||^2}{2\sigma^2}),\sigma > 0\\
&(4):K(X\_i,x\_j)=exp(-\frac{||x\_i-x\_j||}{\sigma}),\sigma > 0 \\
&(5):K(X\_i,x\_j)=tanh(\beta x\_i^Tx\_j+\theta),\beta >0,\theta <0
\end{aligned}
$$

此外，核函数的线性组合还是核函数


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://im-qianuxn.gitbook.io/pytorch/ji-suan-ji/ml/7-zhi-chi-xiang-liang/svm-he-han-shu.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.