# 收缩自动编码器

## 推导

Contractive Auto-Encoders，简称CAE，是一种正则自编码器

训练时在损失函数中**加上正则化项**，使得编码器函数的导数尽可能小

$$L\_{CAE}(\theta)=\sum\_{x \in D\_n}(L(x,g(h(x)))+\lambda ||L\_h(x)||^2)$$

正则化项是编码器函数导数的二范数平方

$$||L\_h(x)||^2=\sum\_{i,j}(\frac{\partial h\_j(x)}{\partial x\_i})^2$$

## 作用

​ 收缩惩罚迫使自编码器学习到的所有映射**对于输入的梯度都很小（因为你迫使它尽可能小啊）**，而重构误差迫使自编码器学习一个恒等映射，保留完整的信息（编码器的特性）。

​ 两种推动力冲击下，使得大部分映射对于输入的梯度都很小，而只有少部分的大梯度（这些迫使也没有，是真的很有用的特征）。

​ 这样在输入具有小扰动时，小梯度会减小这些扰动，达到增加自编码器对**输入附近小扰动的鲁棒性**。


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://im-qianuxn.gitbook.io/pytorch/ji-suan-ji/shen-du-xue-xi-li-lun/01/04.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.