I'm writing a Play 2.3.2 application in Scala.
I use reactivemongo as driver for my MongoDB database.
I've a collection named "recommendation.tagsSimilarity", that contain the value of the similarity between my tags, where a tag is in the form :"category:attribute".
An example of a document is like the following:
{
"_id" : ObjectId("5440ec6e4165e71ac4b53a71"),
"id" : "10912199495810912197116116114-10912199581091219711611611450",
"tag1" : "10912199495810912197116116114",
"tag1Name" : "myc1:myattr",
"tag2" : "10912199581091219711611611450",
"tag2Name" : "myc:myattr2",
"eq" : 0
}
A doment represents an element of a matrix of nxn dimensions, where n is the number of tags saved.
Now I've created a collection named "recommendation.correlation" on which i save the correlation between a "category" and a tag.
For do that I'm writing a method that iterate on the elements of the TagSimilarity as a matrix.
def calculateCorrelation: Future[Boolean] = {
def calculate(category: String, tag: String): Future[(Double, Double)] = {//calculate the correlation and return the tuple value
}
play.Logger.debug("Start Correlation")
Similarity.all.toList flatMap { tagsMatch =>
for(i <- tagsMatch) {
val category = i.tag1Name.split(":")(0) // get the tag category
for(j <- tagsMatch) {
val productName = j.tag2Name //obtain the product tag
calculate(category, productName) flatMap {value =>
val correlation = Correlation(category, productName, value._1, value._2) //create the correlation object
val query = Json.obj("category" -> category, "attribute" -> productName)
Correlations.update(query, correlation, upsert = true) flatMap{status => status match {
case LastError(ok, _, _, _, _, _, _) => Future{true}
case _ => Future{false}
}}
}
}
}
}
}
But the compiler gives me the following error:
[error] /Users/alberto/git/bdrim/modules/recommendation-system/app/recommendationsystem/algorithms/Pearson.scala:313: type mismatch;
[error] found : Unit
[error] required: scala.concurrent.Future[Boolean]
[error] for(i <- tagsMatch) {
[error] ^
[error] /Users/alberto/git/bdrim/modules/recommendation-system/app/recommendationsystem/algorithms/Pearson.scala:313: type mismatch;
[error] found : Unit
[error] required: scala.concurrent.Future[Boolean]
[error] for(i <- tagsMatch) {
[error] ^
[error] one error found
What's wrong?? I can't understand why the for statement don't return nothing.
In addition to I want to ask why i can't write the code in a for comprehension in Scala for iterate two times on the list.
You forgot to use yield with for:
for(i <- tagsMatch) { ... } gets translated to a foreach instruction.
Using for(i <- tagsMatch) yield { ... } it will actually translate to map/flatMap and yield a result (remember to use it on both of your fors).
Related
Hi I am trying to execute a for comprehension like
(for {
player <- playerRepository.findById(playerId) // findById returns EitherT[Future, String, Player]
teamOpt <- teamRepository.findByPlayer(playerId) // findByPlayer returns EitherT[Future, String, Option[Team]]
playedMatches <- teamOpt.map(team => playedMatchesRepository.findByTeamId(team.id)) // findByTeamId returns EitherT[Future, String, Seq[PlayedMatches]]
} yield (
player,
teamOpt,
playedMatches
)).fold(
error => {
logger.error(s"all error: $error")
Left(error)
},
tuple => {
logger.debug(s"get success -> $tuple")
Right(playerDetailResponse(tuple._1, tuple._2, tuple._3))
}
)
I can not get a corret structure for
playedMatches <- teamOpt.map(team => playedMatchesRepository.findByTeamId(team.id))
I am getting the following error when I compile the project
[error] /Users/agusgambina/code/soccer/app/services/impl/PlayerServiceImpl.scala:28:17: type mismatch;
[error] found : Option[(models.Player, Option[models.Team], cats.data.EitherT[scala.concurrent.Future,String,Seq[models.PlayedMatches]])]
[error] required: cats.data.EitherT[scala.concurrent.Future,?,?]
[error] playedMatches <- teamOpt.map(team => playedMatchesRepository.findByTeamId(team.id))
[error] ^
[error] one error found
I tried to wrap
playedMatches <- teamOpt.map(team => playedMatchesRepository.findByTeamId(team.id)) // findByTeamId returns EitherT[Future, String, Seq[PlayedMatches]]
in here, you are getting an Option[EitherT[Future, String, Seq[PlayedMatches]]] which doesn't compose with the EitherT[Future, String, ???] you are using as Monad for the for comprehension.
one option you have is to actually use a fold on teamOpt.
teamOpt.fold(EitherT(Future.successful(Left("Team not Found"): Either[String, Team]))){ team => playedMatchesRepository.findByTeamId(team.id) }
This way you unwrap the Option with the error case if is empty or the success case if non-empty. (create a function that takes the teamOPt as parameter and the for-comprehension will look much better)
Hope it helps
update
In case of the empty case be successful, and be happy returning an empty sequence:
teamOpt.fold(
EitherT(Future.successful(Right(Seq()): Either[String, Seq[PlayedMatches]))
){ team =>
playedMatchesRepository.findByTeamId(team.id)
}
Trying to create multiple dataframes in a single foreach, using spark, as below
I get values delivery and click out of row.getAs("type"), when I try to print them.
val check = eachrec.foreach(recrd => recrd.map(row => {
row.getAs("type") match {
case "delivery" => val delivery_data = delivery(row.get(0).toString,row.get(1).toString)
case "click" => val click_data = delivery(row.get(0).toString,row.get(1).toString)
case _ => "not sure if this impacts"
}})
)
but getting below error:
Error:(41, 14) type mismatch; found : String("delivery") required: Nothing
case "delivery" => val delivery_data = delivery(row.get(0).toString,row.get(1).toString)
^
My plan is to create dataframe using todf() once I create these individual delivery objects referenced by delivery_data and click_data by:
delivery_data.toDF() and click_data.toDF().
Please provide any clue regarding the error above (in match case).
How can I create two df's using todf() in val check?
val declarations make your first 2 cases return type to be unit, but in the third case you return a String
for instance, here the z type was inferred by the compiler, Unit:
def x = {
val z: Unit = 3 match {
case 2 => val a = 2
case _ => val b = 3
}
}
I think you need to cast this match clause to String.
row.getAs("type").toString
I need to implement a class that basically holds a key and a value pair where the value is a sequence of tuples. This tuple contains an object SynthesizedMetricTag, and a type parameter A with a Numeric context bound, so effectively a sequence of k/v where v needs to be a number.
case class Cohort[A : Numeric](index:Any,values:Seq[(SynthesizedMetricTag,A)])
The problem comes when I have to implement a function that merges two instances of this class. More specific, the problem comes when I have to merge a seq of type A with a seq of type B. Both types have a Numeric context bound, so the idea is that in the end I get a Cohort[C] that comforms to the context bound and merges all the K/V pairs of the sequences of type A and B without repeating any key.
case class Cohort[A : Numeric](index:Any,values:Seq[(SynthesizedMetricTag,A)]) {
def merge[B : Numeric, C:Numeric](that:Cohort[B]):Cohort[C] =
if(this.index != that.index) throw new Exception("Unable to merge Cohorts. Criteria is not the same")
else {
val b = new ArrayBuffer[(SynthesizedMetricTag,C)]()
val seen = new mutable.HashSet[SynthesizedMetricTag]()
for (x <- this.values; y <- that.values){
if(!seen(x._1)){
b+= x
seen += x._1
}
if(!seen(y._1)){
b+= y
seen += y._1
}
}
Cohort(this.index,b.toSeq)
}
}
Of course this code throws the following errors:
[error] /Users/ernestrc/dev/everreach/operations-api/src/main/scala/everreach/operations/model/Cohort.scala:14: type mismatch;
[error] found : (everreach.operations.model.SynthesizedMetricTag, A)
[error] required: (everreach.operations.model.SynthesizedMetricTag, C)
[error] b+= x
[error] ^
[error] /Users/ernestrc/dev/everreach/operations-api/src/main/scala/everreach/operations/model/Cohort.scala:18: type mismatch;
[error] found : (everreach.operations.model.SynthesizedMetricTag, B)
[error] required: (everreach.operations.model.SynthesizedMetricTag, C)
[error] b+= y
So I've tried the following:
case class Cohort[A : Numeric](index:Any,values:Seq[(SynthesizedMetricTag,A)]) {
def merge[B : Numeric, C:Numeric](that:Cohort[B]):Cohort[C] =
if(this.index != that.index) throw new Exception("Unable to merge Cohorts. Criteria is not the same")
else {
val b = new ArrayBuffer[(SynthesizedMetricTag,C)]()
val seen = new mutable.HashSet[SynthesizedMetricTag]()
for (x <- this.values; y <- that.values){
if(!seen(x._1)){
b+= x.asInstanceOf[(SynthesizedMetricTag,C)]
seen += x._1
}
if(!seen(y._1)){
b+= y.asInstanceOf[(SynthesizedMetricTag,C)]
seen += y._1
}
}
Cohort(this.index,b.toSeq)
}
}
And it compiles, but I'm sure this is not the idiomatic way of doing this. Do you guys know how to solve this problem ?
EDIT
Alexey made a good point:
What exactly do you want to happen if e.g. A is Double, B is Long, and
C is Byte?
Well C needs to be passed as a type parameter because otherwise the compiler doesn't know what C is, so I would like A and B to be converted to C. But what I really want to happen is to never lose information. So If for example, A is an Int and B is a Float, I want A to be converted to Float and merge both A and B into a collection C(Float).
Of course this hierarchy does not exist, does it? That's why I need to abstract the result type into C and manually pass it as a type parameter.
I would just define the method with an implicit evidence from types A and B to type C
case class Cohort[A <: Numeric](index:Any,values:Seq[(SynthesizedMetricTag,A)]) {
def merge[B <: Numeric, C](that:Cohort[B])(implicit ev1:A=>C, ev2:B=>C):Cohort[C] =
if(this.index != that.index) throw new Exception("Unable to merge Cohorts. Criteria is not the same")
else {
val b = new ArrayBuffer[(SynthesizedMetricTag,C)]()
val seen = new mutable.HashSet[SynthesizedMetricTag]()
for (x <- this.values; y <- that.values){
if(!seen(x._1)){
b += x._1 -> ev1(x._2)
seen += x._1
}
if(!seen(y._1)){
b+= y._1 -> ev2(y._2)
seen += y._1
}
}
Cohort(this.index,b.toSeq)
}
}
I have a function that is attempting to build a JSON object containing a representation of multiple tuples that are stored in a Seq[A], where A is a (Loop, Option[User]). My code looks like so:
def loops = Action {
val page : Int = 1
val orderBy : Int = 1
val filter : String = ""
val jsonifyLoops : (Loop, Option[User]) => Map[String, String] = {
case (loop,user) =>
Map(
"name" -> loop.name,
"created_at" -> loop.createdAt.map(dateFormat.format).getOrElse(""),
"deleted_at" -> loop.deletedAt.map(dateFormat.format).getOrElse(""),
"user_name" -> user.map(_.name).getOrElse("")
)
}
Ok(toJson(Map(
"loops" -> toJson(
Loop.list( page = page, orderBy = orderBy, filter = ("%"+filter+"%") )
.items.map( jsonifyLoops )
)
)
))
}
Loops.list produces a Page[A], from the helper class below:
case class Page[A](items: Seq[A], page: Int, offset: Long, total: Long) {
lazy val prev = Option(page - 1).filter(_ >= 0)
lazy val next = Option(page + 1).filter(_ => (offset + items.size) < total)
}
Thus, Loops.list(...).items should get me a Seq[(Loop, Option[User])], onto which I should be able to apply a map function. I've defined my jsonifyLoops function to have what I think is the appropriate prototype, but I must be doing something wrong, because the compiler throws me the following error:
[error] [...] Application.scala:42: type mismatch;
[error] found : (models.Loop, Option[models.User]) => Map[String,String]
[error] required: (models.Loop, Option[models.User]) => ?
[error] .items.map( jsonifyLoops )
[error] ^
What am I doing wrong?
Your function jsonifyLoops takes two arguments: a Loop and an Option[User]. However, the members of items are tuples of the type (Loop, Option[User]), and thus items.map requires as an argument a function of one argument accepting that tuple. So, you need to convert jsonifyLoops from a binary function to a unary function that takes a pair of arguments; Function2#tupled will do this for you:
scala> :t jsonifyLoops
(Loop, Option[User]) => Map[String,String]
scala> :t jsonifyLoops.tupled
((Loop, Option[User])) => Map[String,String]
You'd use it like this:
Loop.list(page = page, orderBy = orderBy, filter = ("%"+filter+"%"))
.items.map(jsonifyLoops.tupled)
You need to add a default case to your pattern matching within jasonifyLoops.
In absence of a default case, if your case statement fails you return a Unit.
So something like this should work:
val jsonifyLoops : (Loop, Option[User]) => Map[String, String] = {
case (loop,user) =>
Map(
"name" -> loop.name,
"created_at" -> loop.createdAt.map(dateFormat.format).getOrElse(""),
"deleted_at" -> loop.deletedAt.map(dateFormat.format).getOrElse(""),
"user_name" -> user.map(_.name).getOrElse("")
)
case _ => Map[String, String]()
}
This just says that if the input does not match, return an empty Map. However, you should replace this with whatever handling you want to do for the default case.
I am getting a type mismatch compile error for the following code:
case class MyClass(name: String)
def getMyClass(id : String) = {
//For now ignore the id field
Some(Seq(MyClass("test1"), MyClass("test2"), MyClass("test3"), MyClass("test4"), MyClass("test5")))
}
def getHeader() = {
Map(
"n" -> List(Map("s"->"t"), Map("s"->"t"), Map("s"->"t")),
"o" -> List(Map("s"->"t"), Map("s"->"t"), Map("s"->"t")),
"id" -> "12345"
)
}
def castToString(any: Option[Any]): Option[String] = {
any match {
case Some(value: String) => Some(value)
case _ => None
}
}
val h = getHeader()
for{
id <- castToString(h.get("id")) //I hate I have to do this but the map is a Map[String,Any]
m <- getMyClass(id) //This strips the Some from the Some(Seq[MyClass])
item <- m //XXXXXXXX Compile errors
oList <- h.get("o")
nList <- h.get("n")
} yield {
(oList, nList, item)
}
The error is:
C:\temp\s.scala:28: error: type mismatch;
found : Seq[(java.lang.Object, java.lang.Object, this.MyClass)]
required: Option[?]
item <- m
^
But m is of type Seq[MyClass]. I am trying to iterate through the list and set item
You can't mix container types in this way, specifically given the signature of Option.flatMap (to which this expression is desugared - see the comment by pst). However, there's a pretty easy solution:
for{
id <- castToString(h.get("id")).toSeq
m <- getMyClass(id).toSeq
oList <- h.get("o")
nList <- h.get("n")
} yield {
(oList, nList, item)
}
A better explanation of why the code you mentioned doesnt work can be found here:
What is Scala's yield?
You could change your code to the one Kris posted.