PySpark uses Word2Vector to calculate the similarity between word vectors.
发布时间
阅读量：
阅读量
1. 计算 list 中各元素的相似度
输入数据采用DataFrame格式进行呈现：
+--------------------+
|sentence|
+--------------------+
|[a, b, a, b, a, b...|
|[a, b, a, b, a, b...|
+--------------------+
PySpark代码的实现方式：
from pyspark.sql import SQLContext
from pyspark.context import SparkContext
from pyspark.ml.feature import Word2Vec
from pyspark.sql.functions import format_number as fmt
sc = SparkContext("local", "WordCount") # 初始化配置
sqlContext = SQLContext(sc)
sent = ("a b " * 100 + "a c " * 10).split(" ")全部评论 (0)
还没有任何评论哟~