Spark: Simple Page Rank val iters = 10 val lines = sc.textFile("/Users/yyuan/jyuan/src/apache/spark/...

Please Visit: http://ift.tt/1ajReyV

Spark: Simple Page Rank
val iters = 10
val lines = sc.textFile("/Users/yyuan/jyuan/src/apache/spark/1.3.0/spark-1.3.0/graphx/data/followers.txt", 1)
val links = lines.map{ s =>
  val parts = s.split("\\s+")
  (parts(0), parts(1))
}.distinct().groupByKey().cache()
var ranks = links.mapValues(v => 1.0) //init value

for (i <- 1 to iters) {
  val contribs = links.join(ranks).values.flatMap{ case (urls, rank) =>
    val size = urls.size
    urls.map(url => (url, rank / size))
  }
  ranks = contribs.reduceByKey(_ + _).mapValues(0.15 + 0.85 * _)
}

val output = ranks.sortBy( x=>x._2,false).collect()
output.foreach(tup => println(tup._1 + " has rank: " + tup._2 + "."))
http://ift.tt/1Omffix

spark/SparkPageRank.scala at master · apache/spark · GitHub

from Public RSS-Feed of Jeffery yuan. Created with the PIXELMECHANICS 'GPlusRSS-Webtool' at http://gplusrss.com http://ift.tt/1Omfie9
via LifeLong Community