# 高斯混合GMM

1）N(均值,方差)为高斯分布，假设有k个类型，wj为第j个分布的权重（j=1,..,k;∑wj=1），则GMM中的概率的概率分布Qi可求（上EM步骤那样，先随机假设出参数。如，分布权重：多少概率属于男，属于女；分布参数：方差、均值）

记Qi = qij，表示样本 xi 属于类 zj 的混合概率（由男女混合分布加成得到，这个概率作为最终类别概率）：

![img](https://img-blog.csdnimg.cn/20190301204249965.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

例：

![img](https://img-blog.csdnimg.cn/20190228172153484.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

2）极大似然。通过1）我们已经可以知道每个样本的类别标记了（如上例，男:女=0.5625:0.4375，当前样本标记设为男。）

![img](https://img-blog.csdnimg.cn/20190228171905103.png)

所以可以执行EM的M步极大似然了。展开原式子，替换成高斯分布的形式，方便极大似然的求导：

![img](https://img-blog.csdnimg.cn/20190228204428413.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

3）对均值uj,j=1,...,k求导，得到u的更新形式：

![img](https://img-blog.csdnimg.cn/20190228210553461.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

以下推导需要两个知识：

矩阵求导：<https://blog.csdn.net/jiang425776024/article/details/87388015>

多元高斯模型求导：<https://blog.csdn.net/SZU_Hadooper/article/details/78090348>

4）同理，∑的更新形式可求：

![img](https://img-blog.csdnimg.cn/20190228222714213.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

5）最后是wj,j=1,..,k的更新：

根据约束：

![img](https://img-blog.csdnimg.cn/20190228173626315.png)

可以构造拉格朗日形式，因为L式子中其它两个不含有w，所以两个∑∑里面直接写成![\large q\_{ji} ln\_{wj}](https://private.codecogs.com/gif.latex?%5Clarge%20q_%7Bji%7D%20ln_%7Bwj%7D)了：

![img](https://img-blog.csdnimg.cn/20190301204404401.png)

求导：

![img](https://img-blog.csdnimg.cn/20190301205420974.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

（纯原创，公式推得很辛苦，麻烦转载刷刷流量，就这点需求了）

6）最终，高斯混合模型的更新方式已全部得出，可以直接按照结论进行更新（第一次随机假设出的那些）参数了。

1.首先初始化μ,∑,w

2.E步，根据模型参数的当前估计值，计算第i个样本来自第j个高斯分布的概率：

![img](https://img-blog.csdnimg.cn/20190301204249965.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)

3.M步，用极大似然推导公式进行μ,∑,w参数更新：

![img](https://img-blog.csdnimg.cn/2019030121013443.png)

![img](https://img-blog.csdnimg.cn/20190301210145307.png)![img](https://img-blog.csdnimg.cn/20190301210215587.png)

4.参数μ,∑,w几乎不再变化时，样本属于各个类的概率可求，其中最终标记可设为概率最大对应的类。

![img](https://img-blog.csdnimg.cn/20190228171905103.png)

## sklearn 使用

sklearn Gaussian mixture models API:<https://scikit-learn.org/stable/modules/mixture.html>

参数介绍：<https://scikit-learn.org/stable/modules/generated/sklearn.mixture.GaussianMixture.html#sklearn.mixture.GaussianMixture>

```python
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm
from sklearn import mixture

n_samples = 300

n_features = 2
# generate random sample, two components
np.random.seed(0)

# 数据1：generate spherical球形 data centered on (20, 20)
shifted_gaussian = np.random.randn(n_samples, n_features) + np.array([20, 20])

# 数据2：生成零中心拉伸高斯数据,C为矩阵变换用于拉伸数据,中心(0,0)
C = np.array([[0., -0.7], [3.5, .7]])
stretched_gaussian = np.dot(np.random.randn(n_samples, n_features), C)

# 合并数据，竖直
X_train = np.vstack([shifted_gaussian, stretched_gaussian])

'''
fit a Gaussian Mixture Model with two components
covariance_type={‘full’ (default), ‘tied’, ‘diag’, ‘spherical’}控制这每个簇的形状自由度。
默认设置是convariance_type=’diag’,意思是簇在每个维度的尺寸都可以单独设置，但椭圆边界的主轴要与坐标轴平行。
covariance_type=’spherical’时模型通过约束簇的形状，让所有维度相等。这样得到的聚类结果和k-means聚类的特征是相似的，虽然两者并不完全相同。
covariance_type=’full’时，该模型允许每个簇在任意方向上用椭圆建模。
'''
clf = mixture.GaussianMixture(n_components=2, covariance_type='full')
clf.fit(X_train)
plt.scatter(X_train[:, 0], X_train[:, 1], 0.5)

x = np.linspace(-20., 30.)
y = np.linspace(-20., 30.)
X, Y = np.meshgrid(x, y)
XX = np.array([X.ravel(), Y.ravel()]).T
# 预测簇类别
pz = clf.predict(XX).reshape(X.shape)
# 簇概率
prbz = clf.predict_proba(XX).round(5)
print('簇的中点', clf.means_)
print('簇的协方差', clf.covariances_)
# 计算每个样本的加权对数概率。
Z = clf.score_samples(XX)
Z = -Z.reshape(X.shape)
CS = plt.contour(X, Y, Z, norm=LogNorm(vmin=1.0, vmax=1000.0),
                 levels=np.logspace(0, 3, 10))
CB = plt.colorbar(CS, shrink=0.8, extend='both')

plt.title('Negative log-likelihood predicted by a GMM')
plt.axis('tight')
plt.show()

#
簇的中点 [[19.91453549 19.97556345]
 [-0.13607006 -0.07059606]]
```

![img](https://img-blog.csdnimg.cn/2019030317022081.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYW5nNDI1Nzc2MDI0,size_16,color_FFFFFF,t_70)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://im-qianuxn.gitbook.io/pytorch/ji-suan-ji/ml/cluster/gmm.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.