python进行主成分、因子分析

您所在的位置:网站首页 Python计算得分代码 python进行主成分、因子分析

python进行主成分、因子分析

2023-11-16 22:15| 来源: 网络整理| 查看: 265

首先这玩意是干啥的?在什么情况下需要用到主成分因子分析? 首先是我们无法特别明确的界定一个事情的好与坏,如果能够明确界定那就直接上二分类了嘛。就是无法特别明确界定好坏,但是从业务上也知道这些指标对于我们想要评价的这个事情的好坏有着正向或者是负向的关系,影响的维度比较多,并且也无法确定每个维度的权重,我们需要通过这些指标来得出一个事情的综合好坏。这个时候就适合用此方法。先把这一堆指标降维融合、精简到几个指标,再通过这几个合成的核心指标来计算综合得分,进而评价样本水平。

import numpy as np import pandas as pd import matplotlib.pyplot as plt import matplotlib as mpl #画图中的显示中文 mpl.rcParams['font.sans-serif'] = ['simHei'] mpl.rcParams['axes.unicode_minus'] = False from factor_analyzer import FactorAnalyzer path = 'E:/data/'

需要用到factor_analyzer这个因子分析的包,自己去pypi搜索下载下来本地pip install一下就行

1.准备数据集 data = pd.read_excel(path+'factor2.xlsx') #去掉前两列索引变量 df_model = data.iloc[:,2:] 2.数据集归一化(这里需要注意,你的变量如果跟想要评价的结果都是正相关的则没问题,如果有负相关的,需要单独(x.mean()-x)/x.std()处理成正相关) #归一化处理(x-μ)/std df_model = df_model.apply(lambda x:(x-x.mean())/x.std()) 3.充分性检验和相关性检验(充分性检验要求P值接近于0,拒绝原假设。相关性检验要求KMO值大于0.6,也就是说变量之间有不错的相关性,这样比较适合做主成分因子分析,效果会好些,但小于0.6也不是不能做,干就完了)想了解具体原理的去百度补一下相关知识,这里就直接实操了。 #充分性检测 print('巴特利球形度检验') from factor_analyzer.factor_analyzer import calculate_bartlett_sphericity chi_square_value,p_value=calculate_bartlett_sphericity(df_model) print('卡方值:',chi_square_value,'P值', p_value) #相关性检验kmo要大于0.6 from factor_analyzer.factor_analyzer import calculate_kmo kmo_all,kmo_model=calculate_kmo(df_model) print('KMO检验:',kmo_model) 巴特利球形度检验 卡方值: 1998.9543401389722 P值 4.6797864528632136e-279 KMO检验: 0.36023480281173537 4.做碎石图,确定因子个数(参数:总共18个变量,选择最大方差旋转,用主成分分析法) #查看相关矩阵特征值 fa = FactorAnalyzer(18, rotation='varimax',method='principal',impute='mean') fa.fit(df_model) ev, v = fa.get_eigenvalues() print('相关矩阵特征值:',ev) #Create scree plot using matplotlib plt.figure(figsize=(8, 6.5)) plt.scatter(range(1,df_model.shape[1]+1),ev) plt.plot(range(1,df_model.shape[1]+1),ev) plt.title('碎石图',fontdict={'weight':'normal','size': 25}) plt.xlabel('因子',fontdict={'weight':'normal','size': 15}) plt.ylabel('特征值',fontdict={'weight':'normal','size': 15}) plt.grid() # plt.savefig('E:/suishitu.jpg') plt.show() #确定因子个数 n_factors = sum(ev>1) 相关矩阵特征值: [ 7.62901992e+00 4.74639274e+00 3.12047262e+00 1.51961304e+00 1.32788180e+00 1.06456980e+00 6.97386976e-01 5.70642701e-01 4.55960747e-01 2.33053230e-01 2.21733037e-01 1.23554665e-01 1.10435722e-01 6.05144403e-02 5.11050136e-02 3.11355312e-02 2.13299687e-02 1.13465714e-02 3.17478868e-03 6.76688352e-04 3.82492968e-13 -1.38557321e-16]

在这里插入图片描述

我们需要通过碎石图来判断主成分提取的因子个数,提取特征值大于1的因子,因此取6个因子;当然也不是非得卡死1,也得看尽量保证选择的因子能够让累计方差贡献率在80%以上,也就是说,我主成分降维后,优选出来的这几个变量能够表达原来全部变量的80%以上的信息。

5.根据确定的因子个数进行因子分析,获取方差贡献率和因子得分 #取旋转后的结果 fa2 = FactorAnalyzer(n_factors,rotation='varimax',method='principal') fa2.fit(df_model) #给出贡献率 var = fa2.get_factor_variance() #计算因子得分 fa2_score = fa2.transform(df_model) #得分表 column_list = ['fac'+str(i) for i in np.arange(n_factors)+1] fa_score = pd.DataFrame(fa2_score,columns=column_list) for col in fa_score.columns: data[col] = fa_score[col] print("\n各因子得分:\n",fa_score) #方差贡献表 df_fv = pd.DataFrame() df_fv['因子'] = column_list df_fv['方差贡献'] = var[1] df_fv['累计方差贡献'] = var[2] df_fv['累计方差贡献占比'] = var[1]/var[1].sum() print("\n方差贡献表:\n",df_fv) 各因子得分: fac1 fac2 fac3 fac4 fac5 fac6 0 3.020768 0.946129 -0.025456 -0.501881 -0.123309 -1.606739 1 0.042370 0.816350 4.534827 -0.457782 -0.716573 0.515549 2 1.135427 -1.261536 0.796900 1.346283 -0.220619 0.658194 3 0.174743 3.472442 -1.039661 2.809848 0.026430 0.291234 4 1.692862 -0.787293 -0.292800 -0.252617 -0.136059 0.575656 5 1.124442 -0.620196 0.184931 1.018929 -0.479719 0.159320 6 0.587122 0.313125 0.308073 -0.541384 -0.803461 -0.765615 7 1.013917 0.051861 -0.429942 -0.523109 0.113005 -0.970626 8 0.471471 -0.927617 0.374838 0.795526 -0.221395 -0.178835 9 -0.929839 2.896279 0.464677 -1.178917 0.949932 -0.158012 10 0.962930 -0.075328 -1.182600 -0.753314 1.357642 1.205855 11 0.150320 -0.236009 0.408610 -0.596938 1.454235 -0.954688 12 0.744414 -0.187360 -0.618946 -0.406181 -0.978367 -0.061651 13 0.325193 -0.007041 -0.349952 -0.627876 0.163814 -0.720473 14 -0.600366 -0.538897 0.550342 0.910347 4.057739 0.556806 15 0.295682 -0.645510 -0.328384 -0.498560 1.253422 -0.028872 16 -0.141300 0.281250 0.166408 -0.562846 -0.717198 3.566357 17 -0.542184 -0.568310 -0.394864 2.499605 -0.737614 -0.318965 18 -0.215805 -0.459366 -0.693509 -0.596361 -0.295262 1.895700 19 -0.341953 -0.248086 -0.569984 -0.727143 -0.103604 0.236886 20 -0.705395 -0.440974 0.180845 -0.536279 -0.391072 -0.442299 21 -1.100028 -0.988483 0.764337 1.620859 -0.149596 -0.917870 22 -0.826027 -0.203597 -0.508525 0.458925 -0.673198 0.594257 23 -0.849823 -0.163812 -0.349875 0.267730 -0.519616 -0.491487 24 -0.726936 0.527939 -0.674579 -0.635639 -1.037275 0.167674 25 -1.486723 -0.427352 0.318366 -0.243117 -0.014671 -1.237629 26 -1.056571 -0.205819 -0.577365 -0.756063 -0.183059 -0.580182 27 -1.109306 0.130168 -0.716755 -0.739503 -0.575652 -0.539539 28 -1.109405 -0.442956 -0.299960 -0.592541 -0.298901 -0.450002 方差贡献表: 因子 方差贡献 累计方差贡献 累计方差贡献占比 0 fac1 0.263248 0.263248 0.298407 1 fac2 0.174302 0.437551 0.197581 2 fac3 0.151663 0.589214 0.171919 3 fac4 0.142233 0.731447 0.161229 4 fac5 0.084328 0.815776 0.095591 5 fac6 0.066404 0.882180 0.075273 6.计算综合得分及排名(综合得分=累计方差贡献占比1 * 因子得分1 + 累计方差贡献占比2 * 因子得分2 + …)

为什么用累计方差贡献占比来当各个因子的权重?在这之后的某一天我突然想明白了,方差代表着变量中各个样本的差异,方差越大就代表差异越大,也就意味着这个大方差的变量能够很好的区分出各个样本的差距,想的极端一点,假设某个方差趋近于0的变量,就说明样本之间无明显差异,那这个变量不就相当于是废的么。如此看来,越能更好区分样本差异的变量,给的权重就越高确实挺科学。

data['factor_score'] = ((var[1]/var[1].sum())*fa2_score).sum(axis=1) data = data.sort_values(by='factor_score',ascending=False).reset_index(drop=True) data['rank'] = data.index+1 data.iloc[:,-8:] fac1fac2fac3fac4fac5fac6factor_scorerank00.1747433.472442-1.0396612.8098480.0264300.2912341.036976113.0207680.946129-0.025456-0.501881-0.123309-1.6067390.870330220.0423700.8163504.534827-0.457782-0.7165730.5155490.850062331.135427-1.2615360.7969001.346283-0.2206190.6581940.47208044-0.600366-0.5388970.5503420.9103474.0577390.5568060.385555551.124442-0.6201960.1849311.018929-0.4797190.1593200.375212661.692862-0.787293-0.292800-0.252617-0.1360590.5756560.28886577-0.9298392.8962790.464677-1.1789170.949932-0.1580120.263502880.962930-0.075328-1.182600-0.7533141.3576421.2058550.16824099-0.1413000.2812500.166408-0.562846-0.7171983.5663570.15115810100.471471-0.9276170.3748380.795526-0.221395-0.1788350.11548911111.0139170.051861-0.429942-0.5231090.113005-0.9706260.09229112120.5871220.3131250.308073-0.541384-0.803461-0.7656150.06831113130.150320-0.2360090.408610-0.5969381.454235-0.9546880.0393791414-0.542184-0.568310-0.3948642.499605-0.737614-0.318965-0.03347215150.295682-0.645510-0.328384-0.4985601.253422-0.028872-0.05850316160.744414-0.187360-0.618946-0.406181-0.978367-0.061651-0.08494117170.325193-0.007041-0.349952-0.6278760.163814-0.720473-0.1043191818-1.100028-0.9884830.7643371.620859-0.149596-0.917870-0.2142181919-0.215805-0.459366-0.693509-0.596361-0.2952621.895700-0.2560682020-0.826027-0.203597-0.5085250.458925-0.6731980.594257-0.3197722121-0.341953-0.248086-0.569984-0.727143-0.1036040.236886-0.3583592222-0.849823-0.163812-0.3498750.267730-0.519616-0.491487-0.3896092323-0.7269360.527939-0.674579-0.635639-1.0372750.167674-0.4176012424-0.705395-0.4409740.180845-0.536279-0.391072-0.442299-0.4236722525-1.486723-0.4273520.318366-0.243117-0.014671-1.237629-0.6071112626-1.109405-0.442956-0.299960-0.592541-0.298901-0.450002-0.6281232727-1.056571-0.205819-0.577365-0.756063-0.183059-0.580182-0.6382842828-1.1093060.130168-0.716755-0.739503-0.575652-0.539539-0.64339929


【本文地址】


今日新闻


推荐新闻


CopyRight 2018-2019 办公设备维修网 版权所有 豫ICP备15022753号-3