机器学习特征选择方法总结
共 8425字,需浏览 17分钟
·
2020-09-15 19:04
收集的数据格式不对(如 SQL 数据库、JSON、CSV 等)
缺失值和异常值
标准化
减少数据集中存在的固有噪声(部分存储数据可能已损坏)
数据集中的某些功能可能无法收集任何信息以供分析
提高精度
降低过拟合风险
加快训练速度
改进数据可视化
增加我们模型的可解释性
![](https://filescdn.proginn.com/91c42e667e450a31e4a592f70ab66936/9126227364ffd3905d8d92039731e6eb.webp)
X = df.drop(['class'], axis = 1)
Y = df['class']
X = pd.get_dummies(X, prefix_sep='_')
Y = LabelEncoder().fit_transform(Y)
X2 = StandardScaler().fit_transform(X)
X_Train, X_Test, Y_Train, Y_Test = train_test_split(X2, Y, test_size = 0.30, random_state = 101)
start = time.process_time()
trainedforest = RandomForestClassifier(n_estimators=700).fit(X_Train,Y_Train)
print(time.process_time() - start)
predictionforest = trainedforest.predict(X_Test)
print(confusion_matrix(Y_Test,predictionforest))
print(classification_report(Y_Test,predictionforest))
figure(num=None, figsize=(20, 22), dpi=80, facecolor='w', edgecolor='k')
feat_importances = pd.Series(trainedforest.feature_importances_, index= X.columns)
feat_importances.nlargest(7).plot(kind='barh')
![](https://filescdn.proginn.com/a9ae4aedf88ec5a8b8b0de7ca9435d43/9b4274abe92100c7b01f605e94c173f9.webp)
X_Reduced = X[['odor_n','odor_f', 'gill-size_n','gill-size_b']]
X_Reduced = StandardScaler().fit_transform(X_Reduced)
X_Train2, X_Test2, Y_Train2, Y_Test2 = train_test_split(X_Reduced, Y, test_size = 0.30, random_state = 101)
start = time.process_time()
trainedforest = RandomForestClassifier(n_estimators=700).fit(X_Train2,Y_Train2)
print(time.process_time() - start)
predictionforest = trainedforest.predict(X_Test2)
print(confusion_matrix(Y_Test2,predictionforest))
print(classification_report(Y_Test2,predictionforest))
![](https://filescdn.proginn.com/2f87e960fa93f824f543a6804b836f23/4dc136eeb8fdd58a4a8bb6701c58c65c.webp)
start = time.process_time()
trainedtree = tree.DecisionTreeClassifier().fit(X_Train, Y_Train)
print(time.process_time() - start)
predictionstree = trainedtree.predict(X_Test)
print(confusion_matrix(Y_Test,predictionstree))
print(classification_report(Y_Test,predictionstree))
![](https://filescdn.proginn.com/4574576eec93b5bbceab479dde0f3354/fee037cfc2a2856a771a8b2e6a0c1ce2.webp)
import graphviz
from sklearn.tree import DecisionTreeClassifier, export_graphviz
data = export_graphviz(trainedtree,out_file=None,feature_names= X.columns,
class_names=['edible', 'poisonous'],
filled=True, rounded=True,
max_depth=2,
special_characters=True)
graph = graphviz.Source(data)
graph
![](https://filescdn.proginn.com/0f9b619a4921de4a4e1650cb07ac91ec/d975f1c65a8da2b5022f810e42e20de5.webp)
from sklearn.feature_selection import RFE
model = RandomForestClassifier(n_estimators=700)
rfe = RFE(model, 4)
start = time.process_time()
RFE_X_Train = rfe.fit_transform(X_Train,Y_Train)
RFE_X_Test = rfe.transform(X_Test)
rfe = rfe.fit(RFE_X_Train,Y_Train)
print(time.process_time() - start)
print("Overall Accuracy using RFE: ", rfe.score(RFE_X_Test,Y_Test))
![](https://filescdn.proginn.com/7a0839933c4940ca4a8d29e65cb2b5ff/a932b1d745ec327999ea941f5fd18d11.webp)
SelecFromModel
from sklearn.ensemble import ExtraTreesClassifier
from sklearn.feature_selection import SelectFromModel
model = ExtraTreesClassifier()
start = time.process_time()
model = model.fit(X_Train,Y_Train)
model = SelectFromModel(model, prefit=True)
print(time.process_time() - start)
Selected_X = model.transform(X_Train)
start = time.process_time()
trainedforest = RandomForestClassifier(n_estimators=700).fit(Selected_X, Y_Train)
print(time.process_time() - start)
Selected_X_Test = model.transform(X_Test)
predictionforest = trainedforest.predict(Selected_X_Test)
print(confusion_matrix(Y_Test,predictionforest))
print(classification_report(Y_Test,predictionforest))
如果两个特征之间的相关性为 0,则意味着更改这两个特征中的任何一个都不会影响另一个。
如果两个特征之间的相关性大于 0,这意味着增加一个特征中的值也会增加另一个特征中的值(相关系数越接近 1,两个不同特征之间的这种联系就越强)。
如果两个特征之间的相关性小于 0,这意味着增加一个特征中的值将使减少另一个特征中的值(相关性系数越接近-1,两个不同特征之间的这种关系将越强)。
Numeric_df = pd.DataFrame(X)
Numeric_df['Y'] = Y
corr= Numeric_df.corr()
corr_y = abs(corr["Y"])
highest_corr = corr_y[corr_y >0.5]
highest_corr.sort_values(ascending=True)
![](https://filescdn.proginn.com/d5c64b8674e1babdf283549613830f13/eeb5bcc82a86ff0a5fc8967647dfde1d.webp)
figure(num=None, figsize=(12, 10), dpi=80, facecolor='w', edgecolor='k')
corr2 = Numeric_df[['bruises_f' , 'bruises_t' , 'gill-color_b' , 'gill-size_b' , 'gill-size_n' , 'ring-type_p' , 'stalk-surface-below-ring_k' , 'stalk-surface-above-ring_k' , 'odor_f', 'odor_n']].corr()
sns.heatmap(corr2, annot=True, fmt=".2g")
![](https://filescdn.proginn.com/ec772e3bdc941556e4862157f0f59ee4/9454f2b4bcf71fe4767703d730797f67.webp)
单变量选择
Classification = chi2, f_classif, mutual_info_classif
Regression = f_regression, mutual_info_regression
from sklearn.feature_selection import SelectKBest
from sklearn.feature_selection import chi2
min_max_scaler = preprocessing.MinMaxScaler()
Scaled_X = min_max_scaler.fit_transform(X2)
X_new = SelectKBest(chi2, k=2).fit_transform(Scaled_X, Y)
X_Train3, X_Test3, Y_Train3, Y_Test3 = train_test_split(X_new, Y, test_size = 0.30, random_state = 101)
start = time.process_time()
trainedforest = RandomForestClassifier(n_estimators=700).fit(X_Train3,Y_Train3)
print(time.process_time() - start)
predictionforest = trainedforest.predict(X_Test3)
print(confusion_matrix(Y_Test3,predictionforest))
print(classification_report(Y_Test3,predictionforest))
![](https://filescdn.proginn.com/204dbdd84cf6c4c913ce8ba8caea82cb/676790757b2a9859c78a72925c630ca6.webp)
from sklearn.linear_model import LassoCV
regr = LassoCV(cv=5, random_state=101)
regr.fit(X_Train,Y_Train)
print("LassoCV Best Alpha Scored: ", regr.alpha_)
print("LassoCV Model Accuracy: ", regr.score(X_Test, Y_Test))
model_coef = pd.Series(regr.coef_, index = list(X.columns[:-1]))
print("Variables Eliminated: ", str(sum(model_coef == 0)))
print("Variables Kept: ", str(sum(model_coef != 0)))
![](https://filescdn.proginn.com/c05d99b776d3a13f0e0a65f96fb82a03/d5551dce2cd8cc0dbcf67f9cb7076b1e.webp)
figure(num=None, figsize=(12, 10), dpi=80, facecolor='w', edgecolor='k')
top_coef = model_coef.sort_values()
top_coef[top_coef != 0].plot(kind = "barh")
plt.title("Most Important Features Identified using Lasso (!0)")
![](https://filescdn.proginn.com/136741a93e95c4d431a92d767f76efc9/ff5e38fc5fdf37f73d429dad5c1aecb2.webp)
9.9元秒杀【特征工程与模型优化特训】,两大实战项目,学习多种优化方法 掌握比赛上分利器。优秀学员还可获得1V1简历优化及内推名额!在售价199.9元,限时9.9元秒!
课程详情如下???
参与方式:
扫描上方海报二维码
回复“7”
课程从数据采集到数据处理、到特征选择、再到模型调优,带你掌握一套完整的机器学习流程,对于不同类型的数据,不同场景下的问题,选择合适的特征工程方法和模型优化方法进行处理尤为重要。
本次课程还会提供两个经典的 Kaggle 比赛案例和详细的模型融合模板,带你更容易地理解机器学习,掌握比赛上分利器。
五大课程特色
参与方式:
扫描上方海报二维码
回复“7”
戳↓↓“阅读原文”查看课程详情!(机器学习集训营预习课之一)