I am trying to fill in blank values using R just as I would using excel to fill blank cells in a series with linear values. I am trying to get R to fill in the med_age for the dates between 1940 and 1950, 1950 and 1960 in a linear fashion based on the med_age above and below the missing values.
obs = seq(1,79,1)
date = (seq(1940,2018,1))
med_age_data = data.frame(obs, date)
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "1940", "29")
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "1950", "30.2")
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "1960", "29.5")
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "1970", "30")
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "1980", "30")
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "1990", "32.9")
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "2000", "35.3")
med_age_data$med_age <- replace(med_age_data$med_age, med_age_data$date == "2010", "37.2")
Related
for(i <- 10 to 28) {
if(i % 7 == 0 && i % 5 != 0) {
println(s"$i")
}
}
Output will be :
14
21
28
Need output like: 14,21,28.....
can anyone help us writing the code in Scala.
You can do this w/o foreach as below
val result = (10 to 28).filter(i => (i % 7 == 0 && i % 5 != 0 ) ).mkString(",")
println(result)
Considering x,p,r are evaluated from the previous expressions , what is happening underneath , <- and then after =
val a = for{
x <- y
p = q (x)
r <- s (p)
} yield (something(p.something, r.something))
The <- is equivalent (syntactic sugar) to .flatMap call, while = is equivalent to val x = (and yield kind of final .map).
So the code is equivalent to:
val a = y.flatMap { x => // first <-
val p = q (x)
s(p).map { r => // 2nd <- + yield
something(p.something, r.something)
}
}
Below is an example of code which will generate Correlation Matrix but I need to add column header and row header in front and top of matrix.
For example in the above matrix amber coloured objects are the labels which i need to add to the blue coloured data generated by Correlation matrix whose code i have attached below.
In Scala breeze is there a way to add labels to matrix ? The problem is DenseMatrix is Double and labels are character so i am not able to add any char label to matrix object.
def getCorMatrix(c :String,d :String,n :Int) :breeze.linalg.DenseMatrix[Double] = {
CorMatrixlogger.info("Inside generating Correlation Matrix")
val query = MongoDBObject("RunDate" -> d) ++ ("Country" -> c)
CorMatrixlogger.info("Query Object created for {}", c)
val dbObject = for (d <- price.find(query)) yield(d)
val objectReader = (dbObject map {x => objectRead(x)}).toList
val fetchAssetData = objectReader map {x => x.Symbol} map { x=> assetStats(x,n) } filterNot {x => x.length < n-1}
CorMatrixlogger.info("Asset Data fetched")
val excessReturnMatrix = DenseMatrix((for(i <- fetchAssetData) yield i.excessReturn).map(_.toArray):_*)
CorMatrixlogger.info("Excess Return matrix generated")
val transposeExcessreturnMatrix = excessReturnMatrix.t
val vcvMatrix = breeze.numerics.rint(((excessReturnMatrix * transposeExcessreturnMatrix):/ (n-1).toDouble ) :* 1000000.0) :/ 1000000.0
CorMatrixlogger.info("VcV Matrix Generated")
val transposeStDevVector = DenseMatrix(for (i <- fetchAssetData ) yield i.sigma)
val stDevVector = transposeStDevVector.t
val stDevMatrix = breeze.numerics.rint(( stDevVector * transposeStDevVector) :* 1000000.0) :/ 1000000.0
CorMatrixlogger.info("Correlation Matrix Generated")
lowerTriangular(breeze.numerics.rint((vcvMatrix :/ stDevMatrix) :* 10000.0) :/ 10000.0)
}
Edit
Thanks David. Your solution really worked well for me.
val ma = DenseMatrix((1.0,2.0,3.0), (3.0,4.0,5.0),(6.0,7.0,8.0))
val im = DenseMatrix.tabulate(ma.rows,ma.cols)(ma(_,_).toString)
val head = DenseVector("a","b","c")
val thead = head.t
val withHeader:DenseMatrix[String] = DenseMatrix.tabulate(im.rows+1, im.cols+1) { (i, j) =>
if (i == 0 && j == 0) " "
else if (i == 0) head(j -1)
else if (j == 0 ) thead (i -1)
else im(i-1,j-1)
} //> withHeader : breeze.linalg.DenseMatrix[String] = a b c
//| a 1.0 2.0 3.0
//| b 3.0 4.0 5.0
//| c 6.0 7.0 8.0
There's nothing built in, sadly. You could do something like
val withHeader:DenseMatrix[Any] = DenseMatrix.tabulate(n+1, m+1){ (i, j) =>
if (i == 0 && j == 0) ""
else if (i == 0) colHeaders(j - 1)
else if (j == 0) rowHeaders(i - 1)
else orig(i - 1, j - 1)
}
You lose all typing information that way, of course, but if you just need to toString something, it's probably the quickest way in current Breeze.
What is the best way to convert this code which uses memoization into proper Scala using cases and functional programming?
def uniquePathsMemoization(n:Int, m:Int, row:Int, col:Int, seen:Array[Array[Int]]):Int = {
if (row == m && col == n) 1
if (row > m || col > n) 0
if (seen(row+1)(col) == -1) seen(row+1)(col) = uniquePathsMemoization(n, m, row + 1, col, seen)
if (seen(row)(col + 1) == -1 ) seen(row)(col) = uniquePathsMemoization(n,m, row, col + 1, seen)
seen(row+1)(col) + seen(row)(col + 1)
}
This is a modified version of your code that uses match and case
def uniquePathsMemoization(n:Int, m:Int, row:Int, col:Int, seen:Array[Array[Int]]):Int = (row,col) match{
case (row,col) if row == m && col == n =>
1
case (row,col) if row > m || col > n =>
0
case (row,col) =>
if (seen(row+1)(col) == -1) seen(row+1)(col) = uniquePathsMemoization(n, m, row + 1, col, seen)
if (seen(row)(col + 1) == -1 ) seen(row)(col) = uniquePathsMemoization(n,m, row, col + 1, seen)
seen(row+1)(col) + seen(row)(col + 1)
}
It is not easy to convert this code to a pure functional version, due to the state stored in the seen array. But this state can be hidden for the rest of the application, using a function decorator:
def uniquePathsMemoizationGenerator( maxRows: Int, maxCols:Int ) : (Int,Int,Int,Int) => Int = {
def uniquePathsMemoization(n:Int, m:Int, row:Int, col:Int, seen:Array[Array[Int]]):Int = (row,col) match{
case (row,col) if row == m && col == n =>
1
case (row,col) if row > m || col > n =>
0
case (row,col) =>
if (seen(row+1)(col) == -1) seen(row+1)(col) = uniquePathsMemoization(n, m, row + 1, col, seen)
if (seen(row)(col + 1) == -1 ) seen(row)(col) = uniquePathsMemoization(n,m, row, col + 1, seen)
seen(row+1)(col) + seen(row)(col + 1)
}
val seen = Array.fill(maxRows,maxCols)(-1)
uniquePathsMemoization(_,_,_,_,seen)
}
val maxRows = ???
val maxCols = ???
val uniquePaths = uniquePathsMemoizationGenerator( maxRows, maxCols )
// Use uniquePaths from this point, instead of uniquePathsMemoization
What is the difference (if any) between two code fragments below?
Example from Ch7 of Programming i Scala
def grep(pattern: String) =
for (
file <- filesHere
if file.getName.endsWith(".scala");
line <- fileLines(file)
if line.trim.matches(pattern)
) println(file + ": " + line.trim)
and this one
def grep2(pattern: String) =
for (
file <- filesHere
if file.getName.endsWith(".scala")
) for (
line <- fileLines(file)
if line.trim.matches(pattern)
) println(file + ": " + line.trim)
Or
for (i <- 1 to 2)
for (j <- 1 to 2)
println(i, j)
and
for (
i <- 1 to 2;
j <- 1 to 2
) println(i, j)
In this case there is no difference. However when using yield there is:
for (
i <- 1 to 2;
j <- 1 to 2
) yield (i, j)
Will give you a sequence containing (1,1), (1,2), (2,1) and (2,2).
for (i <- 1 to 2)
for (j <- 1 to 2)
yield (i, j)
Will give you nothing, because it generates the sequence (i,1), (i,2) on each iteration and then throws it away.
Sometimes it is also useful to output a multi dimensional collection (for example a matrix of table):
for (i <- 1 to 2) yield for (j <- 1 to 2) yield (i, j)
Will return:
Vector(Vector((1,1), (1,2)), Vector((2,1), (2,2)))