Why are these scalacheck recursive generators not equivalent? - scala

While trying to learn the ScalaCheck tool, I wrote two versions of a Map generator (I know there is one of these built in, but this was an exercise).
It seems that genMap0 and genMap00 should be equivalent, and genMap00 is bit cleaner, but in fact genMap0 works, but genMap00 fails miserably.
The yield is adorned with a println that can be turned on to see what happening (just edit the speak method), but even with this information I cannot say I really understand why the difference. This makes me think that another generator I try to write may also be flawed.
Can someone give a nice explanation of what is different between genMap0 and genMap00?
import org.scalacheck._
import Arbitrary._
import Gen._
import Prop._
def speak(message: String): Unit = if (false) println(message)
lazy val genMap0: Gen[Map[Int, Int]] = for {
k <- arbitrary[Int]
v <- arbitrary[Int]
b <- arbitrary[Boolean]
m <- if (b) value(Map.empty[Int, Int]) else genMap0
} yield if (b) {
speak("false"); m
} else {
speak("true"); m.updated(k, v)
}
lazy val genMap00: Gen[Map[Int, Int]] = for {
k <- arbitrary[Int]
v <- arbitrary[Int]
m <- oneOf(Map.empty[Int, Int], genMap00)
} yield if (m.isEmpty) {
speak("empty:" + m); m
} else {
speak("not empty:" + m); m.updated(k, v)
}
val n = 5
for (i <- 1 to n; m <- genMap0.sample) println(m)
println("--------------")
for (i <- 1 to n; m <- genMap00.sample) println(m)
This is the output (genMap00 always generates the empty map):
scala -cp scalacheck_2.10-1.10.1.jar
...
// Exiting paste mode, now interpreting.
Map()
Map(1 -> 1, 1530546613 -> -1889740266, -187647534 -> 0)
Map()
Map(-1 -> 2039603804)
Map(646468221 -> 1)
--------------
Map()
Map()
Map()
Map()
Map()

The problem the recursive generation always starts with an empty map, so gen00 always ends up with a generator that produces an empty map. The problem is the empty condition is also being used to detect termination.
This is fixed by gen000:
lazy val genMap000: Gen[Map[Int, Int]] = for {
k <- arbitrary[Int]
v <- arbitrary[Int]
m <- oneOf(None, genMap000.map(g => Some(g)))
} yield (for (x <- m) yield x.updated(k, v)).getOrElse(Map())
This uses an intermediate Option[Map], with the None state indicating termination.
Using the explicit Boolean generator appears to be cleaner.

Related

List to multiple anonymous/underscore parameters in for-comprehension

I'm kind of new to Scala/functional so I'm not yet able to use technical language.
I'm experiencing problems with a for-comprehension
val queries =
for {
_ <- createBanco
_ <- createBancoMedio
bankInsertions <- Update[Banco](insertStr).updateMany(NonEmptyList.fromList(createBankList(1, maxBanks)).get)
mediumInsertions <- Update[BancoMedio](mediumInsert).updateMany(NonEmptyList.fromList(mediumList).get)
bankCount <- BancoStatements.getCount().unique
bankGetIds <- BancoStatements.getIds(0, maxBanks).to[List]
bankSome <- BancoStatements.getSome(halfBanks).to[List]
} yield (bankCount, bankGetIds, bankSome)
//Execute database queries, saves them on tuple
val transactionResults : (Int, List[String], List[Banco]) =
queries.transact(h2Transactor).unsafeRunSync()
I'm trying to refactor the _ <- createBanco & _ <- createBancoMedio, which are both a ConnectionIO[Int] object.
Id like to convert those to a single List(createBanco, createBancoMedio) and then execute transact.
However, i'd be altering the return type of the for-comprehension by doing that. I'd like to know if there is any way on doing that without affecting the for output value
Basically, treat the list as if I was writing multiple anonymous parameters manually.
You can use .sequence to turn a List[G[A]] into a G[List[A]] if G has an Applicative instance, which ConnectionIO does:
val queries =
for {
_ <- List(createBanco, createBancoMedio).sequence
...
Just solved it, did another for comprehension for the List
val createList = for {
m <- createBancoMedio
b <- createBanco
} yield List(b, m)
val queries =
for {
_ <- createList ....
This way i had a ConnectionIO[List[Int]]

Cats Writer Vector is empty

I wrote this simple program in my attempt to learn how Cats Writer works
import cats.data.Writer
import cats.syntax.applicative._
import cats.syntax.writer._
import cats.instances.vector._
object WriterTest extends App {
type Logged2[A] = Writer[Vector[String], A]
Vector("started the program").tell
val output1 = calculate1(10)
val foo = new Foo()
val output2 = foo.calculate2(20)
val (log, sum) = (output1 + output2).pure[Logged2].run
println(log)
println(sum)
def calculate1(x : Int) : Int = {
Vector("came inside calculate1").tell
val output = 10 + x
Vector(s"Calculated value ${output}").tell
output
}
}
class Foo {
def calculate2(x: Int) : Int = {
Vector("came inside calculate 2").tell
val output = 10 + x
Vector(s"calculated ${output}").tell
output
}
}
The program works and the output is
> run-main WriterTest
[info] Compiling 1 Scala source to /Users/Cats/target/scala-2.11/classes...
[info] Running WriterTest
Vector()
50
[success] Total time: 1 s, completed Jan 21, 2017 8:14:19 AM
But why is the vector empty? Shouldn't it contain all the strings on which I used the "tell" method?
When you call tell on your Vectors, each time you create a Writer[Vector[String], Unit]. However, you never actually do anything with your Writers, you just discard them. Further, you call pure to create your final Writer, which simply creates a Writer with an empty Vector. You have to combine the writers together in a chain that carries your value and message around.
type Logged[A] = Writer[Vector[String], A]
val (log, sum) = (for {
_ <- Vector("started the program").tell
output1 <- calculate1(10)
foo = new Foo()
output2 <- foo.calculate2(20)
} yield output1 + output2).run
def calculate1(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate1").tell
output = 10 + x
_ <- Vector(s"Calculated value ${output}").tell
} yield output
class Foo {
def calculate2(x: Int): Logged[Int] = for {
_ <- Vector("came inside calculate2").tell
output = 10 + x
_ <- Vector(s"calculated ${output}").tell
} yield output
}
Note the use of for notation. The definition of calculate1 is really
def calculate1(x: Int): Logged[Int] = Vector("came inside calculate1").tell.flatMap { _ =>
val output = 10 + x
Vector(s"calculated ${output}").tell.map { _ => output }
}
flatMap is the monadic bind operation, which means it understands how to take two monadic values (in this case Writer) and join them together to get a new one. In this case, it makes a Writer containing the concatenation of the logs and the value of the one on the right.
Note how there are no side effects. There is no global state by which Writer can remember all your calls to tell. You instead make many Writers and join them together with flatMap to get one big one at the end.
The problem with your example code is that you're not using the result of the tell method.
If you take a look at its signature, you'll see this:
final class WriterIdSyntax[A](val a: A) extends AnyVal {
def tell: Writer[A, Unit] = Writer(a, ())
}
it is clear that tell returns a Writer[A, Unit] result which is immediately discarded because you didn't assign it to a value.
The proper way to use a Writer (and any monad in Scala) is through its flatMap method. It would look similar to this:
println(
Vector("started the program").tell.flatMap { _ =>
15.pure[Logged2].flatMap { i =>
Writer(Vector("ended program"), i)
}
}
)
The code above, when executed will give you this:
WriterT((Vector(started the program, ended program),15))
As you can see, both messages and the int are stored in the result.
Now this is a bit ugly, and Scala actually provides a better way to do this: for-comprehensions. For-comprehension are a bit of syntactic sugar that allows us to write the same code in this way:
println(
for {
_ <- Vector("started the program").tell
i <- 15.pure[Logged2]
_ <- Vector("ended program").tell
} yield i
)
Now going back to your example, what I would recommend is for you to change the return type of compute1 and compute2 to be Writer[Vector[String], Int] and then try to make your application compile using what I wrote above.

Future composition in Scala with chunked response

I think I have understanding of how future composition works but I am confused how to invoke the next future on chunk of response from first future.
Say the first future returns a list of integer and list is huge. I want to apply some function to that list with 2 elements at a time. How do I do that?
This example summarizes my dilemma:
val a = Future(List(1,2,3,4,5,6))
def f(a: List[Int]) = Future(a map (_ + 2))
val res = for {
list <- a
chunked <- list.grouped(2).toList
} yield f(chunked)
<console>:14: error: type mismatch;
found : List[scala.concurrent.Future[List[Int]]]
required: scala.concurrent.Future[?]
chunked <- list.grouped(2).toList
^
The return type has to be Future[?] so I can fix it by moving second future to yield part:
val res = for {
list <- a
} yield {
val temp = for {
chunked <- list.grouped(2).toList
} yield f(chunked)
Future.sequence(temp)
}
I feel it loses its elegance now, since it becomes nested (see two for comprehensions instead of one in the first approach). Is there a better way to achieve the same?
Consider
a.map { _.grouped(2).toList }.flatMap { Future.traverse(_)(f) }
Or, if you are set on only using for comprehension for some reason, here is how, without "cheating" :)
for {
b <- a
c <- Future.traverse(b.grouped(2).toList)(f)
} yield c
Edit in response to the comment It's not really that hard to add more processing to your chunked list if needed:
for {
b <- a
chunks = b.grouped(2).toList
processedChunks = processChunks(chunks)
c <- Future.traverse(processedChunks)
} yield c
Or, without for comprehension:
a
.map { _.grouped(2).toList }
.map(processChunks)
.flatMap { Future.traverse(_)(f) }
You cannot mix Future with List in a for-comprehension. All involved objects have to be of the same type. Also, in your working example, your result value res is of type Future[Future[List[List[Int]]]], which is probably not what you want.
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
a: scala.concurrent.Future[List[Int]] = scala.concurrent.impl.Promise$DefaultPromise#3bd3cdc8
f: (a: List[Int])scala.concurrent.Future[List[Int]]
scala> val b: Future[List[List[Int]]] = a.map(list => list.grouped(2).toList)
b: scala.concurrent.Future[List[List[Int]]] = scala.concurrent.impl.Promise$DefaultPromise#74db196c
scala> val res: Future[List[List[Int]]] = b.flatMap(lists => Future.sequence(lists.map(f)))
res: scala.concurrent.Future[List[List[Int]]] = scala.concurrent.impl.Promise$DefaultPromise#28f9873c
With for-comprehension
for {
b ← a.map(list ⇒ list.grouped( 2 ).toList)
res ← Future.sequence(b.map(f))
} yield res

Future[Option[Boolean]] in a for comprehension.. Simple right?

Suppose I have:
val res:Future[Option[Boolean]] = Future(Some(true))
and I can do:
res.map(opt => opt.map(r => print(!r)))
I guess the for comprehension for this would be:
for {
opt <- res
r <- opt
} yield (print(!r))
but this does not work! I get an error ie:
error: type mismatch;
found : Option[Unit]
required: scala.concurrent.Future[?]
r <- opt
How do I use a Future[Option[Boolean]] in a for comprehension to extract or convert the Boolean?
Note: this is a simplification of the problem I have at the moment with many Future[Option[Boolean]] variables that I would like to use together in a for comprehension.
A for-comprehension really makes this look like it should all work, doesn't it? But let's think about what you're asking for.
First, note that for un-nests:
for {xs <- List(List(5)); x <- xs} yield x
produces
List(5)
Now, without even getting into the type signatures or desugaring, we can think about replacing List with some arbitrary type T, and we'll call the contained type A:
for { xs <- T(T(a: A)); x <- xs } yield x
and we should get a
T[A]
back (presumably the one we put in, but the types don't actually promise us that).
Okay, but what about
for { xs <- T(U(a: A)); x <- xs } yield x
? This is strictly more general than the case where the two things have the same nesting. Well, if T and U both have a common supertype S, then we can just view the whole thing as S(S(a: A)), so we'll at least get an S back. But what about in the general case?
The bottom line is that it depends. For example, let's consider the case where T=Future, U=Option. We have the following possibilities:
Success(Some(a))
Success(None)
Failure(t: Throwable)
Now, can we come up with any coherent policy for unwrapping? If we unwrap into a Future, then what A do you use for the Success(None) case? You don't have one available to return. Likewise, if you try to vanquish the outer Future, how do you know, without explicitly stating it somehow to the compiler, that Failure should be mapped to None (if indeed it should--maybe it should go to a default!).
So the bottom line is that you just can't do this correctly in general without specifying what is supposed to happen for every pair T[U[_]]. (I encourage the interested reader to peruse tutorials on monads and monad transformers.)
There is a way out, though: if you can explicitly turn your U into a T, or your T into your U, you can take advantage of the unwrapping capability. It's pretty easy to turn an Option into a Future , so the easiest solution is
for { opt <- res; r <- Future(opt.get) } yield r
(where just let the exception get thrown on none.get). Alternatively, you can turn the Future into an Option with the slightly ugly
for { opt <- res.value.flatMap(_.toOption); r <- opt } yield r
Equivalent code for
for {
opt <- res
r <- opt
} yield (print(!r))
is not
res.map(opt => opt.map(r => print(!r)))
but
res.flatMap(opt => opt.map(r => print(!r)))
And it makes no sense in this case.
For the chain of maps you could use nested for-comprehensions
for { opt <- res }
for { r <- opt }
print(!r)
But map looks better.
Well, for comprehensions are deceiving in the way they look. Your for-comprehension expands to:
res.flatMap(opt => opt.map(r => print(!r))
Which is obviously wrong as flatMap expects a return type of Future[T] and you are providing Option[Unit]
Though sometimes, for code tidiness you would want to have a single for loop with many such expressions. In those cases you could:
scala> implicit def toFuture[T](t: => T):Future[T] = {
| val p = Promise[T]()
| p.tryComplete(Try(t))
| p.future
| }
scala> for {
| opt <- res
| r <- opt
| } yield {print(!r)}
false
The above produces:
res.flatMap[Option[Unit]](((opt: Option[Boolean]) =>
toFuture[Option[Unit]](opt.map[Unit](((r: Boolean) => print(!r))))))
Edit: Though you need to take all the pain if you are using yield. If you do not wish to use for comprehension as an expression, then you can do as you desired:
scala> val i = Future(Some(true))
i: scala.concurrent.Future[Some[Boolean]] = scala.concurrent.impl.Promise$DefaultPromise#6b24a494
scala> val j = Option(1)
j: Option[Int] = Some(1)
scala> val k = Right(1).right
k: scala.util.Either.RightProjection[Nothing,Int] = RightProjection(Right(1))
scala>
| for{
| x <- i
| y <- j
| z <- k
| }{
| println(i,j,k)
| }
(scala.concurrent.impl.Promise$DefaultPromise#6b24a494,Some(1),RightProjection(Right(1)))
This way no implicit is required. As compiler uses foreach at every junction. -Xprint:typer gives:
i.foreach[Unit](((x: Option[Boolean]) =>
j.foreach[Any](((y: Int) =>
k.foreach[Unit](((z: Int) => println(scala.this.Tuple3.apply[scala.concurrent.Future[Option[Boolean]], Option[Int], Either.RightProjection[Nothing,Int]](i, j, k))))))))
}

Scala Option return type

I am newbie in Scala programming world but loving it. Recently I have started porting my research App into Scala and one of thing I am still struggling is the return keyword. For example in below code
def readDocument(dbobj:MongoDBObject) = Option[ContainerMetaData]
{
for(a <- dbobj.getAs[String]("classname");
b <- dbobj.getAs[Long]("id");
c <- dbobj.getAs[Long]("version");
d <- dbobj.getAs[String]("description");
e <- dbobj.getAs[String]("name");
f <- dbobj.getAs[String]("tag");
g <- dbobj.getAs[Int]("containertype");
h <- dbobj.getAs[Date]("createddate")
)
{
val ctype = ContainerType(g)
val jodadt = new DateTime(h)
val data = new ContainerMetaData(a,b,c,d,e,f,ctype,jodadt)
Some(data)
}
None
}
In above code I get the error message:
type mismatch; found : None.type required: om.domain.ContainerMetaData
So if I remove the explicit return type the code works but then without explicit return keyword I am not able to terminate my code at Some(data).
def readDocument(dbobj:MongoDBObject)=
{
for(a <- dbobj.getAs[String]("classname");
b <- dbobj.getAs[Long]("id");
c <- dbobj.getAs[Long]("version");
d <- dbobj.getAs[String]("description");
e <- dbobj.getAs[String]("name");
f <- dbobj.getAs[String]("tag");
g <- dbobj.getAs[Int]("containertype");
h <- dbobj.getAs[Date]("createddate")
)
{
val ctype = ContainerType(g)
val jodadt = new DateTime(h)
val data = new ContainerMetaData(a,b,c,d,e,f,ctype,jodadt)
Some(data)
}
None
}
And if add a return keyword then compiler complains
method `readDocument` has return statement; needs result tye
Few more additional info, this is the trait I am extending
trait MongoDAOSerializer[T] {
def createDocument(content:T) : DBObject
def readDocument(db:MongoDBObject) : Option[T]
}
The problem is, that you are missing the yield keyword in the for-comprehension. And also the None at the end is unnecessary, as the for-comprehension will yield None, if one of the values is missing and also the explicit creation of a Some in the comprehension is not needed, as it will create an Option anyway. Your code hase to look like this (not tested)
def readDocument(dbobj: MongoDBObject): Option[ContainerMetaData] = {
for {
a <- dbobj.getAs[String]("classname")
b <- dbobj.getAs[Long]("id")
c <- dbobj.getAs[Long]("version")
d <- dbobj.getAs[String]("description")
e <- dbobj.getAs[String]("name")
f <- dbobj.getAs[String]("tag")
g <- dbobj.getAs[Int]("containertype")
h <- dbobj.getAs[Date]("createddate")
} yield {
val ctype = ContainerType(g)
val jodadt = new DateTime(h)
new ContainerMetaData(a,b,c,d,e,f,ctype,jodadt)
}
}