项目一报错：bigger than spark.driver.maxResultSize
发布时间
阅读量：
阅读量
七、spark集群报错：bigger than spark.driver.maxResultSize
难点
错误代码
20/09/15 15:21:32 ERROR scheduler.TaskSetManager: Total size of serialized results of 423 tasks (4.0 GB) is bigger than spark.driver.maxResultSize (4.0 GB)
Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Total size of serialized results of 423 tasks (4.0 GB) is bigger than spark.driver.maxResultSize (4.0 GB)
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1499)
at org全部评论 (0)
还没有任何评论哟~