I'm playing around with queue to learn.
Although reproducing this example from the doc
import cats.effect.IO
import cats.effect.std.Queue
import cats.effect.unsafe.implicits.global
import cats.syntax.all._
import fs2.Stream
val program = for {
queue <- Queue.unbounded[IO, Option[Int]]
streamFromQueue = Stream.fromQueueNoneTerminated(queue) // Stream is terminated when None is returned from queue
_ <- Seq(Some(1), Some(2), Some(3), None).map(queue.offer).sequence
result <- streamFromQueue.compile.toList
} yield result
program.unsafeRunSync()
Was ok. I tried to twist it a little and got surprised by an error that i can't explain.
(for {
queue <- Queue.unbounded[IO, Option[Int]]
_ <-
Stream
.emits((1 to 4 toList))
.covary[IO]
.map(Option[Int])
//.metered(1 second)
//.onFinalize(IO.println("Stream 1 is done"))
.enqueueNoneTerminated(queue)
.compile
.drain
_ <-
Stream
.fromQueueNoneTerminated(queue)
.onFinalize(IO.println("Stream 2 is done"))
.map(int => int * 10)
.evalTap(IO.println(_))
.compile
.drain
} yield ()).unsafeRunSync()
I get the following error
[error] fs2App.scala:40:32: type mismatch;
[error] found : cats.effect.std.Queue[cats.effect.IO,Option[Int]]
[error] required: cats.effect.std.Queue[[x]cats.effect.IO[x],Option[Any]]
[error] Note: Option[Int] <: Option[Any], but class Queue is invariant in type A.
[error] You may wish to define A as +A instead. (SLS 4.5)
[error] .enqueueNoneTerminated(queue)
[error] ^
[error] one error found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 4 s, completed 4 Nov 2022, 18:37:22
Not sure how to use that method enqueueNoneTerminated
Any suggestions ?
Related
At the next site:
https://typelevel.org/cats/datatypes/either.html
it is presented:
object EitherStyle {
def parse(s: String): Either[Exception, Int] =
if (s.matches("-?[0-9]+")) Either.right(s.toInt)
else Either.left(new NumberFormatException(s"${s} is not a valid integer."))
def reciprocal(i: Int): Either[Exception, Double] =
if (i == 0) Either.left(new IllegalArgumentException("Cannot take reciprocal of 0."))
else Either.right(1.0 / i)
def stringify(d: Double): String = d.toString
}
Yet, I am getting the error:
[error] /application/learningSBT/hello-world/src/main/scala/Main.scala:16:39: value right is not a member of object scala.util.Either
[error] if (s.matches("-?[0-9]+")) Either.right(s.toInt)
[error] ^
[error] /application/learningSBT/hello-world/src/main/scala/Main.scala:17:17: value left is not a member of object scala.util.Either
[error] else Either.left(new NumberFormatException(s"${s} is not a valid integer."))
[error] ^
[error] /application/learningSBT/hello-world/src/main/scala/Main.scala:21:14: value left is not a member of object scala.util.Either
[error] Either.left(new IllegalArgumentException("Cannot take reciprocal of 0."))
[error] ^
[error] /application/learningSBT/hello-world/src/main/scala/Main.scala:22:17: value right is not a member of object scala.util.Either
[error] else Either.right(1.0 / i)
[error] ^
[error] four errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 2 s, completed Feb 12, 2020 10:25:02 AM
However, when I replaced Either.right with Right and Either.left with Left I got this code compiling:
object EitherStyle {
def parse(s: String): Either[Exception, Int] =
if (s.matches("-?[0-9]+")) Right(s.toInt)
else Left(new NumberFormatException(s"${s} is not a valid integer."))
def reciprocal(i: Int): Either[Exception, Double] =
if (i == 0)
Left(new IllegalArgumentException("Cannot take reciprocal of 0."))
else Right(1.0 / i)
def stringify(d: Double): String = d.toString
}
So, I wonder what makes this to happen.
This is an extension of cats to the standard Either object.
Import cats.syntax.either._ for this to work.
Running the sbt test, I've got the following error message:
error] java.lang.NumberFormatException: For input string: "505c621128f97f31c5870f2a9e2d274fa432bd0e"
[error] at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
[error] at java.lang.Long.parseLong(Long.java:589)
[error] at java.lang.Long.parseLong(Long.java:631)
[error] at scala.collection.immutable.StringLike.toLong(StringLike.scala:305)
[error] at scala.collection.immutable.StringLike.toLong$(StringLike.scala:305)
[error] at scala.collection.immutable.StringOps.toLong(StringOps.scala:29)
[error] at sbt.TestStatus$.$anonfun$read$1(TestStatusReporter.scala:42)
[error] at sbt.TestStatus$.$anonfun$read$1$adapted(TestStatusReporter.scala:42)
[error] at scala.collection.Iterator.foreach(Iterator.scala:937)
[error] at scala.collection.Iterator.foreach$(Iterator.scala:937)
[error] at scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
[error] at sbt.TestStatus$.read(TestStatusReporter.scala:42)
[error] at sbt.TestStatusReporter.succeeded$lzycompute(TestStatusReporter.scala:20)
[error] at sbt.TestStatusReporter.succeeded(TestStatusReporter.scala:20)
[error] at sbt.TestStatusReporter.doComplete(TestStatusReporter.scala:31)
[error] at sbt.TestFramework$.$anonfun$createTestTasks$7(TestFramework.scala:240)
[error] at sbt.TestFramework$.$anonfun$createTestTasks$7$adapted(TestFramework.scala:240)
[error] at sbt.TestFramework$.$anonfun$safeForeach$1(TestFramework.scala:150)
[error] at sbt.TestFramework$.$anonfun$safeForeach$1$adapted(TestFramework.scala:149)
[error] at scala.collection.Iterator.foreach(Iterator.scala:937)
[error] at scala.collection.Iterator.foreach$(Iterator.scala:937)
[error] at scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
[error] at scala.collection.IterableLike.foreach(IterableLike.scala:70)
[error] at scala.collection.IterableLike.foreach$(IterableLike.scala:69)
[error] at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
[error] at sbt.TestFramework$.safeForeach(TestFramework.scala:149)
[error] at sbt.TestFramework$.$anonfun$createTestTasks$1(TestFramework.scala:226)
[error] at sbt.Tests$.$anonfun$testTask$1(Tests.scala:231)
[error] at sbt.Tests$.$anonfun$testTask$1$adapted(Tests.scala:231)
[error] at sbt.std.TaskExtra$$anon$1.$anonfun$fork$2(TaskExtra.scala:110)
[error] at sbt.std.Transform$$anon$3.$anonfun$apply$2(System.scala:46)
[error] at sbt.std.Transform$$anon$4.work(System.scala:67)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:269)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:16)
[error] at sbt.Execute.work(Execute.scala:278)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:269)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:178)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:37)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[error] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[error] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[error] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[error] at java.lang.Thread.run(Thread.java:748)
[error] java.lang.NumberFormatException: For input string: "505c621128f97f31c5870f2a9e2d274fa432bd0e"
[info] ScalaTest
[info] Run completed in 522 milliseconds.
[info] Total number of tests run: 3
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 3, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 3, Failed 0, Errors 0, Passed 3
As you can see, all test passed. What is wrong? Hint, I am using Intellij.
Update
Here is the code:
import atto._
import Atto._
import cats._
import cats.implicits._
sealed trait PcpPair
case class PcpHead(key: String, value: String) extends PcpPair
case class PcpFieldValue(field: String, value: String) extends PcpPair
case class Pcp(head: List[PcpHead], fv: List[PcpFieldValue], body: String)
object PcpProtocol {
implicit val pcpProtocol: Protocol[Pcp] = new Protocol[Pcp] {
override def encode(text: String): ProtocolResult[Pcp] =
doc
.parseOnly(text)
.either
.flatMap { t =>
isValidPcp(t._1) match {
case true => Right(t)
case false => Left("It is not a valid PCP protocol.")
}
}
.map(t => Pcp(filterPcpHead(t._1), filterFieldValue(t._1), t._2))
override def decode(msg: Pcp): String = ???
}
private val PcpValidity = "pcp-"
private val key = stringOf(letter | char('-'))
private val value = stringOf(notChar('\n'))
private val kv = (key <~ char(':')) ~ value
private val kvs = sepBy(kv, char('\n'))
private val doc = (kvs <~ string("\n\n")) ~ takeText
private val isValidPcp: List[(String, String)] => Boolean = textList =>
textList
.map(kv => kv._1.startsWith(PcpValidity))
.foldLeft(true)(_ || _)
private val filterPcpHead: List[(String, String)] => List[PcpHead] = textList =>
textList
.filter(text => text._1.contains(PcpValidity))
.map(text => PcpHead(text._1, text._2))
private val filterFieldValue: List[(String, String)] => List[PcpFieldValue] = textList =>
textList
.filter(text => !text._1.contains(PcpValidity))
.map(text => PcpFieldValue(text._1, text._2))
private val decodeHead: List[PcpHead] => String = heads =>
heads.foldLeft("") { (acc, value) =>
acc |+| value.key |+| ":" |+| value.value |+| "\n"
}
private val decodePair: List[PcpPair] => ((String, PcpPair) => String) => String = pcpList => fnPcp =>
pcpList.foldLeft("")(fnPcp)
}
and here is the test:
class PcpParserSpec extends FunSpec with Matchers {
val valid =
"""pcp-action:MESSAGE
|pcp-channel:apc\:///
|pcp-body-type:text
|PUBLICKEY:THISPK
|TOPIC:SEND
|
|Hello Foo""".stripMargin
describe("Receive message from SAP server") {
it("should contains pcp-channel:apc") {
Protocol.encode(valid) should be('right)
}
it("should be separated by head and body") {
Protocol.encode(valid) match {
case Right(value) => assert(value.head.length > 0)
case Left(text) => assert(text.length > 0)
}
}
describe("The body of the message") {
it("should contains Hello Foo") {
Protocol.encode(valid) match {
case Right(value) => assert(value.body == "Hello Foo")
case Left(text) => assert(text.length > 0)
}
}
}
}
describe("Send message to SAP") {
it("should encode appropriate PCP protocol") {
}
}
}
What you'll probably find is that target/streams/test/test/\$global/streams/succeeded_tests (in your project directory) is getting garbled. (Mine was an encoded mess).
You should be getting something like
#Successful Tests
#Thu Apr 04 07:54:33 NZDT 2019
MyTest=1554317673393
The number it's trying to read is after the =.
The confusing bit (to me) is that it's outputting the summary - mine didn't when it failed parsing.
Before you run the following, could you share (some of) the contents of succeeded_tests (above) in a comment here (I'm curious to see these failures).
sbt clean test should fix you up...
I am trying to build an application using FS2 (0.10.0). I've taken this example from the documentation:
import fs2._
// import fs2._
import fs2.async
// import fs2.async
import scala.concurrent.ExecutionContext
// import scala.concurrent.ExecutionContext
import cats.effect.{ Effect, IO }
// import cats.effect.{Effect, IO}
type Row = List[String]
// defined type alias Row
trait CSVHandle {
def withRows(cb: Either[Throwable,Row] => Unit): Unit
}
// defined trait CSVHandle
def rows[F[_]](h: CSVHandle)(implicit F: Effect[F], ec: ExecutionContext): Stream[F,Row] =
for {
q <- Stream.eval(async.unboundedQueue[F,Either[Throwable,Row]])
_ <- Stream.suspend { h.withRows { e => async.unsafeRunAsync(q.enqueue1(e))(_ => IO.unit) }; Stream.emit(()) }
row <- q.dequeue.rethrow
} yield row
// rows: [F[_]](h: CSVHandle)(implicit F: cats.effect.Effect[F], implicit ec: scala.concurrent.ExecutionContext)fs2.Stream[F,Row]
But it fails on compilation with:
type mismatch;
[error] found : fs2.Stream[F,Row]
[error] required: fs2.Stream[fs2.Pure,?]
[error] Expanded types:
[error] found : fs2.Stream[F,List[String]]
[error] required: fs2.Stream[fs2.Pure,?]"
[error] row <- q.dequeue.rethrow
I'm afraid I'm stuck and I don't understand why is that happening. Any idea?
As per this Gitter conversation there is an error in the example. Use instead:
def rows[F[_]](h: CSVHandle)(implicit F: Effect[F], ec: ExecutionContext): Stream[F,Row] =
for {
q <- Stream.eval(async.unboundedQueue[F,Either[Throwable,Row]])
_ <- Stream.eval { F.delay(h.withRows(e => async.unsafeRunAsync(q.enqueue1(e))(_ => IO.unit))) }
row <- q.dequeue.rethrow
} yield row
I was playing around a bit with macros and I thought writing a json type provider would be a good start to get a deeper understanding of how all this works, but I hit a weird error that I can't seem to be able to figure out myself. Code is available on GitHub if you want to take a look at the whole stuff: https://github.com/qwe2/json-typeprovider/.
The problematic part is, I tried to make it as typesafe as I can, meaning I wanted to implement json arrays in such a way, that indexing into them would return the correct type (as a subsequent macro invocation). relevant methods of the code:
json to Tree method:
def jsonToTpe(value: JValue): Option[Tree] = value match {
case JNothing => None
case JNull => None
case JString(s) => Some(q"$s")
case JDouble(d) => Some(q"$d")
case JDecimal(d) => Some(q"scala.BigDecimal(${d.toString})")
case JInt(i) => Some(q"scala.BigInt(${i.toByteArray})")
case JLong(l) => Some(q"$l")
case JBool(b) => Some(q"$b")
case JArray(arr) =>
val arrTree = arr.flatMap(jsonToTpe)
val clsName = c.freshName[TypeName](TypeName("harraycls"))
val hArray =
q"""
class $clsName {
#_root_.com.example.json.provider.body(scala.Array[Any](..$arrTree))
def apply(i: Int): Any = macro _root_.com.example.json.provider.DelegatedMacros.arrApply_impl
#_root_.com.example.json.provider.body(scala.Array[Any](..$arrTree))
def toArray: scala.Array[Any] = macro _root_.com.example.json.provider.DelegatedMacros.selectField_impl
}
new $clsName {}
"""
Some(hArray)
case JSet(set) => Some(q"scala.Set(..${set.flatMap(jsonToTpe)})")
case JObject(fields) =>
val fs = fields.flatMap { case (k, v) =>
jsonToTpe(v).map(v => q"""
#_root_.com.example.json.provider.body($v) def ${TermName(k)}: Any =
macro _root_.com.example.json.provider.DelegatedMacros.selectField_impl""")
}
val clsName = c.freshName[TypeName](TypeName("jsoncls"))
Some(q"""
class $clsName {
..$fs
}
new $clsName {}
""")
}
reading the annotation:
class body(tree: Any) extends StaticAnnotation
def arrApply_impl(c: whitebox.Context)(i: c.Expr[Int]): c.Tree = {
import c.universe._
def bail(msg: String): Nothing = {
c.abort(c.enclosingPosition, msg)
}
def error(msg: String): Unit = {
c.error(c.enclosingPosition, msg)
}
val arrValue = selectField_impl(c)
val arrElems = arrValue match {
case q"scala.Array.apply[$tpe](..$elems)($cls)" => elems
case _ => bail("arr needs to be an array of constants")
}
val idx = i.tree match {
case Literal(Constant(ix: Int)) => ix
case _ => bail(s"i needs to be a constant Int, got ${showRaw(i.tree)}")
}
arrElems(idx)
}
def selectField_impl(c: whitebox.Context) : c.Tree = {
c.macroApplication.symbol.annotations.filter(
_.tree.tpe <:< c.typeOf[body]
).head.tree.children.tail.head
}
As you can see the way I tried to do it was, basically shove the actual array into a static annotation and when indexing it, I would dispatch that to another macro that can figure out the type. I got the idea from reading about vampire methods.
This is the json I'm trying to parse:
[
{"id": 1},
{"id": 2}
]
And this is how I invoke it:
val tpe3 = TypeProvider("arrayofobj.json")
println(tpe3.toArray.mkString(", "))
Reading an array of ints or an object of primitive fields works as expected but an array of objects throws a stackoverflow during compilation:
[error] /home/isti/projects/json-typeprovider/core/src/main/scala/com/example/Hello.scala:7:14: Internal error: unable to find the outer accessor symbol of object Hello
[error] object Hello extends App {
[error] ^
[error] ## Exception when compiling 1 sources to /home/isti/projects/json-typeprovider/core/target/scala-2.12/classes
[error] null
[error] java.lang.String.valueOf(String.java:2994)
[error] scala.collection.mutable.StringBuilder.append(StringBuilder.scala:200)
[error] scala.collection.TraversableOnce.$anonfun$addString$1(TraversableOnce.scala:359)
[error] scala.collection.immutable.List.foreach(List.scala:389)
[error] scala.collection.TraversableOnce.addString(TraversableOnce.scala:357)
[error] scala.collection.TraversableOnce.addString$(TraversableOnce.scala:353)
[error] scala.collection.AbstractTraversable.addString(Traversable.scala:104)
[error] scala.collection.TraversableOnce.mkString(TraversableOnce.scala:323)
[error] scala.collection.TraversableOnce.mkString$(TraversableOnce.scala:322)
[error] scala.collection.AbstractTraversable.mkString(Traversable.scala:104)
[error] scala.collection.TraversableOnce.mkString(TraversableOnce.scala:325)
[error] scala.collection.TraversableOnce.mkString$(TraversableOnce.scala:325)
[error] scala.collection.AbstractTraversable.mkString(Traversable.scala:104)
[error] scala.collection.TraversableOnce.mkString(TraversableOnce.scala:327)
[error] scala.collection.TraversableOnce.mkString$(TraversableOnce.scala:327)
[error] scala.collection.AbstractTraversable.mkString(Traversable.scala:104)
[error] xsbt.DelegatingReporter$.makePosition$1(DelegatingReporter.scala:89)
[error] xsbt.DelegatingReporter$.convert(DelegatingReporter.scala:94)
[error] xsbt.DelegatingReporter.info0(DelegatingReporter.scala:125)
[error] xsbt.DelegatingReporter.info0(DelegatingReporter.scala:102)
[error] scala.reflect.internal.Reporter.error(Reporting.scala:84)
[error] scala.reflect.internal.Reporting.globalError(Reporting.scala:69)
[error] scala.reflect.internal.Reporting.globalError$(Reporting.scala:69)
[error] scala.reflect.internal.SymbolTable.globalError(SymbolTable.scala:16)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerSelect(ExplicitOuter.scala:235)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
[error] scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267)
Edit: that's only the top of the stacktrace, there are a lot more of scala.tools.nsc.transform.ExplicitOuter$OuterPathTransformer.outerPath(ExplicitOuter.scala:267).
I am trying to get this query right but getting errors. First, searchUser returns a sequence of matching UserEntries that contains unique id for user and for each of the userId, there is a second query obtains some other user info + address from another table.
Code:
def searchUsers(pattern: String) = auth.SecuredAction.async {
implicit request =>
usersService.searchUser(pattern) flatMap { usrList =>
for {
u <- usrList
ui <- usersService.getUsersWithAddress(u.id)
} yield {
Ok(views.html.UserList(ui))
}
}
}
Signatures for the APIs used:
def searchUser(pattern: String): Future[Seq[UserEntry]] = ...
def getUsersWithAddress(userId: Long): Future[Seq[(UserProfile, Seq[String])]] = ...
Error:
[error] modules/www/app/controllers/Dashboard.scala:68: type mismatch;
[error] found : scala.concurrent.Future[play.api.mvc.Result]
[error] required: scala.collection.GenTraversableOnce[?]
[error] ui <- usersService.getUsersWithAddress(u.id)
[error] ^
[error] one error found
[error] (www/compile:compileIncremental) Compilation failed
If I comment out line "u <- usrList" and hardcode a userid for the next line like "ui <- usersService.getUsersWithAddress(1L)" it works. Any idea what I am missing?
When you use multiple generators in a for comprehension, the monads have to be of the same type. E.g. you can't:
scala> for{ x <- Some("hi"); y <- List(1,2,3) } yield (x,y)
<console>:11: error: type mismatch;
found : List[(String, Int)]
required: Option[?]
for{ x <- Some("hi"); y <- List(1,2,3) } yield (x,y)
^
What you can do is convert one or the other to match the correct type. For the above example, that would be:
scala> for{ x <- Some("hi").toSeq; y <- List(1,2,3) } yield (x,y)
res2: Seq[(String, Int)] = List((hi,1), (hi,2), (hi,3))
In your particular case, one of your generators is a GenTraversableOnce, and the other is a Future. You can probably use Future.successful(theList) to get two futures. See the answer here for example:
Unable to use for comprehension to map over List within Future
Based on #Brian's answer arrived at a solution.. following worked (could not enter formatter block of code as comment - so adding as answer):
usersService.searchUser(pattern) flatMap { usrList =>
val q = for {
u <- usrList
} yield (usersService.getUserWithAddress(u.id))
val r = Future.sequence(q)
r map { ps =>
Ok(views.html.UserList(ps))
}
}
For comprehension accumulated the Futures and then the sequence was flattened and then mapped. Hope this is how it is done!
Note: also I had to change the signature of getUserWithAddress to X instead of Seq[X]