Logistic Regression

Spark Examples

Logistic Regression

This is an iterative machine learning algorithm that seeks to find the best hyperplane that separates two sets of points in a multi-dimensional feature space. It can be used to classify messages into spam vs non-spam, for example. Because the algorithm applies the same MapReduce operation repeatedly to the same dataset, it benefits greatly from caching the input data in RAM across iterations.

val points = spark.textFile(...).map(parsePoint).cache()

var w = Vector.random(D) // current separating plane

for (i <- 1 to ITERATIONS) {

  val gradient = points.map(p =>

    (1 / (1 + exp(-p.y*(w dot p.x))) - 1) * p.y * p.x

  ).reduce(_ + _)

  w -= gradient

}

println("Final separating plane: " + w)

欢迎分享，转载请注明来源：内存溢出

原文地址: http://outofmemory.cn/zaji/2091790.html