Spark mllib.stat.Statistics - kolmogorovSmirnovTest CDF - scala

Question

I am looking through the example HypothesisTestingKolmogorovSmirnovTestExample.scala for spark and can't seem to figure out the CDF aspect.
Their example:
import org.apache.spark.{SparkConf, SparkContext}
import org.apache.spark.mllib.stat.Statistics
import org.apache.spark.rdd.RDD
val data: RDD[Double] = sc.parallelize(Seq(0.1, 0.15, 0.2, 0.3, 0.25)) // an RDD of sample data
val myCDF = Map(0.1 -> 0.2, 0.15 -> 0.6, 0.2 -> 0.05, 0.3 -> 0.05, 0.25 -> 0.1)
val testResult2 = Statistics.kolmogorovSmirnovTest(data, myCDF)
println(testResult2)
This returns:
Very strong presumption against null hypothesis: Sample follows theoretical distribution.
This makes sense - what doesn't is when I try to have it not reject the Null:
val data: RDD[Double] = sc.parallelize(Seq(0.1, 0.15, 0.2, 0.3, 0.25)) // an RDD of sample data
val myCDF = Map(0.1 -> 0.1, 0.15 -> 0.15, 0.2 -> 0.2, 0.3 -> 0.3, 0.25 -> 0.25) //CDF matching the data distribution
val testResult2 = Statistics.kolmogorovSmirnovTest(data, myCDF)
println(testResult2)
This ALSO returns:
Very strong presumption against null hypothesis: Sample follows theoretical distribution.
What gives? The CDF and the data are the exact same distribution, are they not? Why would the Null be rejected? What am I assuming/doing wrong?

Answer 1

What is the scenario, you can use the KS test:
KS Test is one of the Goodness-fit-Test to be executed after the fit distribution for the data.
this test will tell you whether the identified distribution for the data is correct or not. we need to validate this with the p-value.
if the p value > 0.05 then the distribution you set for the data is fine. the p value is < 0.05 then you need fit data with the different distribution.
Rejecting Null means, p value is < 0.05: Data Not fit for the given distribution