I wanted to use Alpakka for handling S3 upload and download with Akka Steams. However, I got stuck with using Source produced by S3Client within Akka Http routes. The error message I get is:
[error] found : akka.stream.scaladsl.Source[akka.util.ByteString,_$1] where type _$1
[error] required: akka.http.scaladsl.marshalling.ToResponseMarshallable
[error] complete(source)
I assume that it is some annoying trivial thing, like missing implicit import, but I was not able to pinpoint what I am missing.
I've created some minimal example to illustrate the issue:
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.server.Directives._
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.Source
import akka.util.ByteString
import scala.concurrent.ExecutionContext
class Test {
implicit val actorSystem: ActorSystem = ActorSystem()
implicit val materializer: ActorMaterializer = ActorMaterializer()
implicit val executionContext: ExecutionContext = actorSystem.dispatcher
val route = (path("test") & get) {
def source: Source[ByteString, _] = ??? // just assume that I am able to get that value
complete(source) // here error happens
}
Http().bindAndHandle(route, "localhost", 8000)
}
Do you have some suggestions, what can I try? I am using
libraryDependencies += "com.typesafe.akka"%% "akka-http" % "10.0.5"
You need to create an HttpEntity from the source, and give it a content-type.
complete(HttpEntity(ContentTypes.`application/json`, source))
Related
I'm looking at the akka streams quickstart tutorial and I wanted to understand how a piece of code works. The following code from the example prints out the values from 1 to 100 to the console:
import akka.stream.scaladsl._
import akka.{Done, NotUsed}
import akka.actor.ActorSystem
object Akka_Streams extends App{
implicit val system: ActorSystem = ActorSystem("QuickStart")
val source: Source[Int, NotUsed] = Source(1 to 100)
source.runForeach(i => println(i))
}
What I don't understand is, when I change the code to the following and remove the implicit, the code no longer works. I get a type mismatch error (shown below the following code):
object Akka_Streams extends App{
val system: ActorSystem = ActorSystem("QuickStart")
val source: Source[Int, NotUsed] = Source(1 to 100)
source.runForeach(i => println(i))(system)
}
Error:
type mismatch;
found : akka.actor.ActorSystem
required: akka.stream.Materializer
source.runForeach(i => println(i))(system)
Why did this work before but not now? the source.runForeach method takes a Materalizer type so I wonder why this was working at all to begin with? From what I can see, an ActorSystem is not a Materalizer or a sub-type of it so I'm confused.
It is related to how Scala compiler converts ActorSystem to a Materializer
It is done via implicit conversions with following method
/**
* Implicitly provides the system wide materializer from a classic or typed `ActorSystem`
*/
implicit def matFromSystem(implicit provider: ClassicActorSystemProvider): Materializer =
SystemMaterializer(provider.classicSystem).materializer
It requires parameter provider to be implicit.
So having implicit key is allowing the compiler to take an implicit actor system when it needs an instance of a Materializer, and such conversion is done without any need to explicitly define a materializer in the scope.
I am following the instructions on the cats IO website to run a sequence of effects in parallel:
My code looks like:
val maybeNonEmptyList: Option[NonEmptyList[Urls]] = NonEmptyList.fromList(urls)
val maybeDownloads: Option[IO[NonEmptyList[Either[Error, Files]]]] = maybeNonEmptyList map { urls =>
urls.parTraverse(url => downloader(url))
}
But I get a compile time error saying:
value parTraverse is not a member of cats.data.NonEmptyList[Urls]
[error] urls.parTraverse(url => downloader(url))
I have imported the following:
import cats.data.{EitherT, NonEmptyList}
import cats.effect.{ContextShift, IO, Timer}
import cats.implicits._
import cats.syntax.parallel._
and also i have the following implicits:
implicit val cs: ContextShift[IO] = IO.contextShift(ExecutionContext.global)
implicit val timer: Timer[IO] = IO.timer(ExecutionContext.global)
Why do i still get the problem?
This is caused because the implicit enrichment is being imported twice, so it becomes ambiguous
import cats.implicits._
import cats.syntax.parallel._
As of recent versions of cats, implicits imports are never required, only syntax
The recommended pattern is to do only import cats.syntax.all._
I have a scala project that uses akka. I want the execution context to be available throughout the project. So I've created a package object like this:
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import com.datastax.driver.core.Cluster
package object connector {
implicit val system = ActorSystem()
implicit val mat = ActorMaterializer()
implicit val executionContext = executionContext
implicit val session = Cluster
.builder
.addContactPoints("localhost")
.withPort(9042)
.build()
.connect()
}
In the same package I have this file:
import akka.stream.alpakka.cassandra.scaladsl.CassandraSource
import akka.stream.scaladsl.Sink
import com.datastax.driver.core.{Row, Session, SimpleStatement}
import scala.collection.immutable
import scala.concurrent.Future
object CassandraService {
def selectFromCassandra()() = {
val statement = new SimpleStatement(s"SELECT * FROM animals.alpakka").setFetchSize(20)
val rows: Future[immutable.Seq[Row]] = CassandraSource(statement).runWith(Sink.seq)
rows.map{item =>
print(item)
}
}
}
However I am getting the compiler error that no execution context or session can be found. My understanding of the package keyword was that everything in that object will be available throughout the package. But that does not seem work. Grateful if this could be explained to me!
Your implementation must be something like this, and hope it helps.
package.scala
package com.app.akka
package object connector {
// Do some codes here..
}
CassandraService.scala
package com.app.akka
import com.app.akka.connector._
object CassandraService {
def selectFromCassandra() = {
// Do some codes here..
}
}
You have two issue with your current code.
When you compile your package object connector it is throwing below error
Error:(14, 35) recursive value executionContext needs type
implicit val executionContext = executionContext
Issue is with implicit val executionContext = executionContext line
Solution for this issue would be as below.
implicit val executionContext = ExecutionContext
When we compile CassandraService then it is throwing error as below
Error:(17, 13) Cannot find an implicit ExecutionContext. You might pass
an (implicit ec: ExecutionContext) parameter to your method
or import scala.concurrent.ExecutionContext.Implicits.global.
rows.map{item =>
Error clearly say that either we need to pass ExecutionContext as implicit parameter or import scala.concurrent.ExecutionContext.Implicits.global. In my system both issues are resolved and its compiled successfully. I have attached code for your reference.
package com.apache.scala
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import com.datastax.driver.core.Cluster
import scala.concurrent.ExecutionContext
package object connector {
implicit val system = ActorSystem()
implicit val mat = ActorMaterializer()
implicit val executionContext = ExecutionContext
implicit val session = Cluster
.builder
.addContactPoints("localhost")
.withPort(9042)
.build()
.connect()
}
package com.apache.scala.connector
import akka.stream.alpakka.cassandra.scaladsl.CassandraSource
import akka.stream.scaladsl.Sink
import com.datastax.driver.core.{Row, SimpleStatement}
import scala.collection.immutable
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
object CassandraService {
def selectFromCassandra() = {
val statement = new SimpleStatement(s"SELECT * FROM animals.alpakka").setFetchSize(20)
val rows: Future[immutable.Seq[Row]] = CassandraSource(statement).runWith(Sink.seq)
rows.map{item =>
print(item)
}
}
}
I am playing with Akka Http client side. I have created a simple request but how can I unmarshal the respose? in the server side it is easy to use circe to marshal the response, but I have difficulties in the client side
import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.Uri.Query
import akka.http.scaladsl.model._
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.stream.Materializer
import scala.concurrent.ExecutionContext
class QuestionsFetcher {
import de.heikoseeberger.akkahttpcirce.CirceSupport._
import io.circe.generic.auto._
val baseUrl = "https://somewhere.com"
def fetch(tag: String)(implicit ac: ActorSystem, materializer: Materializer) = {
implicit val ec: ExecutionContext = ac.dispatcher
val fromDate = DateTime.now.minus(1000 * 60 * 60 * 24)
val uri = Uri(baseUrl).withQuery(Query("order"->"desc"))
val request = HttpRequest(HttpMethods.GET, uri)
Http().singleRequest(request)
.map(r => Unmarshal(r.entity.withContentType(ContentTypes.`application/json`)).to[Items])
}
}
When running the code I am getting
ErrorFuture(io.circe.ParsingFailure: expected json value got (line 1, column 1))
the content was gziped - so after running the decoder flow
it was ok
I am trying to use scala Futures to implement a threaded bulk get from a network service key/value store.
roughly
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration._
def bulkGet(keys: List[String])
val listFut = keys.map( future{ "network get request here" } )
val values = Future.sequence(listFut)
Await.result(values, Duration(10, SECONDS))
gives me a compile error
[info] Compiling 1 Scala source to .../target/scala-2.10/classes...
[error] .... type mismatch;
[error] found : scala.concurrent.Future[List[String]]
[error] required: scala.concurrent.Awaitable[scala.concurrent.Future[List[String]]]
[error] Await.result(values, Duration(10, SECONDS))
^
what am I doing wrong.
I am following the docs re: how to block on a result
http://docs.scala-lang.org/overviews/core/futures.html
Is a scala.concurrent.Future not by definition Awaitable? How do I coerce it to be?
If I fix the syntax in your example code (by putting the body of the def into a block, and replacing future{ "network get request here" } with _ => Future{ "network get request here" }), this compiles and works. The problem is in some other part of the code.
Works for me.
$ scala
Welcome to Scala version 2.10.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.6.0_65).
Type in expressions to have them evaluated.
Type :help for more information.
scala> :paste
// Entering paste mode (ctrl-D to finish)
import scala.concurrent.{Future, Await}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.{Duration, SECONDS}
def bulkGet(keys: List[String]) = {
val listFut = keys.map(_ => Future("network get request here"))
val values = Future.sequence(listFut)
Await.result(values, Duration(10, SECONDS))
}
// Exiting paste mode, now interpreting.
import scala.concurrent.{Future, Await}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.{Duration, SECONDS}
bulkGet: (keys: List[String])List[String]
scala> bulkGet(List("foo", "bar", "baz"))
res0: List[String] = List(network get request here, network get request here, network get request here)