Advertisement

采用CountVectorizer和TfidfVectorizer方法处理fetch_20newsgroups数据集,并比较不同情况下的分类效果(精确度作为评估指标)。

阅读量:

from sklearn.datasets import fetch_20newsgroups
from sklearn.cross_validation import train_test_split
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import classification_report
from sklearn.feature_extraction.text import TfidfVectorizer

#1.使用为去掉停用词的DictVectorizer对20newsgroup进行分类
news = fetch_20newsgroups(subset='all')
#对数据进行分割
X_train,X_test,y_train,y_test = train_test_split(news.data,news.target,test_size=0.25,random_state=33)
#采用默认配置对CountVectorizer进行初始化
count_vec = CountVectorizer()

全部评论 (0)

还没有任何评论哟~