I'm implementing a Vault client in Scala using Http4s client.
And I'm now starting to write integration tests. So far I have this:
abstract class Utils extends AsyncWordSpec with Matchers {
implicit override def executionContext = ExecutionContext.global
implicit val timer: Timer[IO] = IO.timer(executionContext)
implicit val cs: ContextShift[IO] = IO.contextShift(executionContext)
val vaultUri = Uri.unsafeFromString(Properties.envOrElse("VAULT_ADDR", throw IllegalArgumentException))
val vaultToken = Properties.envOrElse("VAULT_TOKEN", throw IllegalArgumentException)
val clientResource = BlazeClientBuilder[IO](global)
.withCheckEndpointAuthentication(false)
.resource
def usingClient[T](f: VaultClient[IO] => IO[Assertion]): Future[Assertion] = {
clientResource.use { implicit client =>
f(new VaultClient[IO](vaultUri, vaultToken))
}.unsafeToFuture()
}
}
Then my tests look like this (just showing one test):
class SysSpec extends Utils {
"The auth endpoint" should {
"successfully mount an authentication method" in {
usingClient { client =>
for {
result <- client.sys.auth.create("test", AuthMethod(
"approle", "some nice description", config = TuneOptions(defaultLeaseTtl = 60.minutes)
))
} yield result should be (())
}
}
}
}
This approach works, however it doesn't feel right. For each test I'm opening the connection (clientResource.use) and recreating the VaultClient.
Is there a way for me to reuse the same connection and client for all the tests in SysSpec.
Please note these are integration tests and not unit tests.
This is the best I could come up with.
abstract class Utils extends AsyncWordSpec with Matchers with BeforeAndAfterAll {
implicit override def executionContext = ExecutionContext.global
implicit val timer: Timer[IO] = IO.timer(executionContext)
implicit val cs: ContextShift[IO] = IO.contextShift(executionContext)
val (httpClient, finalizer) = BlazeClientBuilder[IO](global)
.withCheckEndpointAuthentication(false)
.resource.allocated.unsafeRunSync()
override protected def afterAll(): Unit = finalizer.unsafeRunSync()
private implicit val c = httpClient
val client = new VaultClient[IO](uri"http://[::1]:8200", "the root token fetched from somewhere")
}
Then the tests just use the client directly:
class SysSpec extends Utils {
"The auth endpoint" should {
"successfully mount an authentication method" in {
client.sys.auth.create("test", AuthMethod(
"approle", "some nice description",
config = TuneOptions(defaultLeaseTtl = 60.minutes))
).map(_ shouldBe ()).unsafeToFuture()
}
}
}
My two main problems with this approach are the two unsafeRunSyncs in the code. The first one is to create the client and the second one to clean the resource. However it is a much better approach then repeatedly creating and destroy the client.
I would also like not to use the unsafeToFuture but that would require ScalaTest to support Cats-Effect directly.
Related
I'm building a toy project to learn Scala 3 and i'm stuck in one problem, first of all i'm following the tagless-final approach using cats-effect, the approach is working as expected except for the entity serialization, when i try to create a route using akka-http i have the following problem:
def routes: Route = pathPrefix("security") {
(path("auth") & post) {
entity(as[LoginUserByCredentialsCommand]) {
(command: LoginUserByCredentialsCommand) =>
complete {
login(command)
}
}
}}
F[
Either[com.moralyzr.magickr.security.core.errors.AuthError,
com.moralyzr.magickr.security.core.types.TokenType.Token
]
]
Required: akka.http.scaladsl.marshalling.ToResponseMarshallable
where: F is a type in class SecurityApi with bounds <: [_] =>> Any
For what i understood, akka-http does not know how to serialize the F highly-kinded type, by searching a little bit i found the following solution, it consists of creating an implicit called marshallable to show the akka-http how to serialize the type, however when i implement it i get a StackOverflow error :(
import akka.http.scaladsl.marshalling.ToResponseMarshaller
import cats.effect.IO
trait Marshallable[F[_]]:
def marshaller[A: ToResponseMarshaller]: ToResponseMarshaller[F[A]]
object Marshallable:
implicit def marshaller[F[_], A : ToResponseMarshaller](implicit M: Marshallable[F]): ToResponseMarshaller[F[A]] =
M.marshaller
given ioMarshaller: Marshallable[IO] with
def marshaller[A: ToResponseMarshaller] = implicitly
I'm really stuck right now, does anyone have an idea on how can i fix this problem? The complete code can be found here
Edit: This is the login code
For clarity, here are the class that instantiate the security api and the security api itself
object Magickr extends IOApp:
override def run(args: List[String]): IO[ExitCode] =
val server = for {
// Actors
actorsSystem <- ActorsSystemResource[IO]()
streamMaterializer <- AkkaMaterializerResource[IO](actorsSystem)
// Configs
configs <- Resource.eval(MagickrConfigs.makeConfigs[IO]())
httpConfigs = AkkaHttpConfig[IO](configs)
databaseConfigs = DatabaseConfig[IO](configs)
flywayConfigs = FlywayConfig[IO](configs)
jwtConfig = JwtConfig[IO](configs)
// Interpreters
jwtManager = JwtBuilder[IO](jwtConfig)
authentication = InternalAuthentication[IO](
passwordValidationAlgebra = new SecurityValidationsInterpreter(),
jwtManager = jwtManager
)
// Database
_ <- Resource.eval(
DbMigrations.migrate[IO](flywayConfigs, databaseConfigs)
)
transactor <- DatabaseConnection.makeTransactor[IO](databaseConfigs)
userRepository = UserRepository[IO](transactor)
// Services
securityManagement = SecurityManagement[IO](
findUser = userRepository,
authentication = authentication
)
// Api
secApi = new SecurityApi[IO](securityManagement)
routes = pathPrefix("api") {
secApi.routes()
}
akkaHttp <- AkkaHttpResource.makeHttpServer[IO](
akkaHttpConfig = httpConfigs,
routes = routes,
actorSystem = actorsSystem,
materializer = streamMaterializer
)
} yield (actorsSystem)
return server.useForever
And
class SecurityApi[F[_]: Async](
private val securityManagement: SecurityManagement[F]
) extends LoginUserByCredentials[F]
with SecurityProtocols:
def routes()(using marshaller: Marshallable[F]): Route = pathPrefix("security") {
(path("auth") & post) {
entity(as[LoginUserByCredentialsCommand]) {
(command: LoginUserByCredentialsCommand) =>
complete {
login(command)
}
}
}
}
override def login(
command: LoginUserByCredentialsCommand
): F[Either[AuthError, Token]] =
securityManagement.loginWithCredentials(command = command).value
================= EDIT 2 =========================================
With the insight provided by Luis Miguel, it makes a clearer sense that i need to unwrap the IO into a Future at the Marshaller level, something like this:
def ioToResponseMarshaller[A: ToResponseMarshaller](
M: Marshallable[IO]
): ToResponseMarshaller[IO[A]] =
Marshaller.futureMarshaller.compose(M.entity.unsafeToFuture())
However, i have this problem:
Found: cats.effect.unsafe.IORuntime => scala.concurrent.Future[A]
Required: cats.effect.IO[A] => scala.concurrent.Future[A]
I think i'm close! Is there a way to unwrap the IO keeping the IO type?
I managed to make it work! Thanks to #luismiguel insight, the problem was that the Akka HTTP Marshaller was not able to deal with Cats-Effect IO monad, so the solution was an implementation who unwraps the IO monad using the unsafeToFuture inside the marshaller, that way i was able to keep the Tagless-Final style from point to point, here's the solution:
This implicit fetches the internal marshaller for the type
import akka.http.scaladsl.marshalling.ToResponseMarshaller
import cats.effect.IO
trait Marshallable[F[_]]:
def marshaller[A: ToResponseMarshaller]: ToResponseMarshaller[F[A]]
object Marshallable:
implicit def marshaller[F[_], A: ToResponseMarshaller](implicit
M: Marshallable[F]
): ToResponseMarshaller[F[A]] = M.marshaller
given ioMarshallable: Marshallable[IO] with
def marshaller[A: ToResponseMarshaller] = CatsEffectsMarshallers.ioMarshaller
This one unwraps the IO monad and flatMaps the marshaller using a future, which akka-http knows how to deal with.
import akka.http.scaladsl.marshalling.{
LowPriorityToResponseMarshallerImplicits,
Marshaller,
ToResponseMarshaller
}
import cats.effect.IO
import cats.effect.unsafe.implicits.global
trait CatsEffectsMarshallers extends LowPriorityToResponseMarshallerImplicits:
implicit def ioMarshaller[A](implicit
m: ToResponseMarshaller[A]
): ToResponseMarshaller[IO[A]] =
Marshaller(implicit ec => _.unsafeToFuture().flatMap(m(_)))
object CatsEffectsMarshallers extends CatsEffectsMarshallers
I need to start two Finagle ListeningServers at once because I have to implement two different traits that extend ListeningServer.
/* A simplified example to give you an idea of what I'm trying to do */
trait FirstListeningServer extends ListeningServer {
def buildFirstServer() = ???
def main(): Unit = {
val server = buildFirstServer()
closeOnExit(server)
Await.ready(server)
}
}
trait SecondListeningServer extends ListeningServer {
def buildSecondServer() = ???
def main(): Unit = {
val server = buildSecondServer()
closeOnExit(server)
Await.ready(server)
}
}
Basically, each ListeningServer is a com.twitter.util.Awaitable and whenever I have to instantiate a new ListeningServer I use Await.ready(myListeningServer).
class MyServer extends FirstListeningServer with SecondListeningServer {
override def main(): Unit = {
val firstServer = buildFirstServer()
closeOnExit(firstServer)
val secondServer = buildSecondServer()
closeOnExit(secondServer)
Await.all(firstServer, secondServer)
}
}
Now I'm not sure if using Await.all() is the right choice in order to start several ListeningServers concurrently. I would have used com.twitter.util.Future.collect() but I have two Awaitables.
def all(awaitables: Awaitable[_]*): Unit
Returns after all actions have completed.
I'm using Scala 2.12 and Twitter 20.3.0.
I want to test the following function:
def curl(host: String, attempt: Int = 200): ZIO[Loggings with Clock, Throwable, Unit]
If the environment would just use standard ZIO environments, like Console with Clock, the test would work out of the box:
testM("curl on valid URL") {
(for {
r <- composer.curl("https://google.com")
} yield
assert(r, isUnit))
}
The Test environment would be provided by zio-test.
So the question is, how to extend the TestEnvironment with my Loggings module?
Note that this answer is for RC17 and will change significantly in RC18. You're right that as in other cases of composing environments we need to implement a function to build our total environment from the modules we have. Spec has several combinators built in such as provideManaged to do this so you don't need to do it within your test itself. All of these have "normal" variants that will provide a separate copy of the environment to each test in a suite and "shared" variants that will create one copy of the environment for the entire suite when it is a resource that is expensive to create like a Kafka service.
You can see an example below of using provideSomeManaged to provide an environment that extends the test environment to a test.
In RC18 there will be a variety of other provide variants equivalent to those on ZIO as well as a new concept of layers to make it much easier to build composed environments for ZIO applications.
import zio._
import zio.clock._
import zio.test._
import zio.test.environment._
import ExampleSpecUtil._
object ExampleSpec
extends DefaultRunnableSpec(
suite("ExampleSpec")(
testM("My Test") {
for {
time <- clock.nanoTime
_ <- Logging.logLine(
s"The TestClock says the current time is $time"
)
} yield assertCompletes
}
).provideSomeManaged(testClockWithLogging)
)
object ExampleSpecUtil {
trait Logging {
def logging: Logging.Service
}
object Logging {
trait Service {
def logLine(line: String): UIO[Unit]
}
object Live extends Logging {
val logging: Logging.Service =
new Logging.Service {
def logLine(line: String): UIO[Unit] =
UIO(println(line))
}
}
def logLine(line: String): URIO[Logging, Unit] =
URIO.accessM(_.logging.logLine(line))
}
val testClockWithLogging
: ZManaged[TestEnvironment, Nothing, TestClock with Logging] =
ZIO
.access[TestEnvironment] { testEnvironment =>
new TestClock with Logging {
val clock = testEnvironment.clock
val logging = Logging.Live.logging
val scheduler = testEnvironment.scheduler
}
}
.toManaged_
}
This is what I came up:
testM("curl on valid URL") {
(for {
r <- composer.curl("https://google.com")
} yield
assert(r, isUnit))
.provideSome[TestEnvironment](env => new Loggings.ConsoleLogger
with TestClock {
override val clock: TestClock.Service[Any] = env.clock
override val scheduler: TestClock.Service[Any] = env.scheduler
override val console: TestLogger.Service[Any] = MyLogger()
})
}
Using the TestEnvironment with provideSome to setup my environment.
I want to inject Configuration instance in one of my testing classes, I extend my test class with ConfiguredApp and injected the Configuration, it looks like this:
#DoNotDiscover()
class MyApiServiceSpec extends FreeSpec with ScalaFutures with ConfiguredApp {
implicit val formats = DefaultFormats
implicit val exec = global
lazy val configuration = app.injector.instanceOf[Configuration]
"Global test" - {
"testcase 1" in {
Server.withRouter() {
case GET(p"/get/data") => Action { request =>
Results.Ok()
}
} { implicit port =>
WsTestClient.withClient { implicit client =>
val service = new MyApiService {
override def config: Configuration = configuration
override val ws: WSClient = client
}
whenReady(service.getData()) { res =>
//i will test stuff here
}
}
}
}
}
}
(MyApiService is a trait)
Exception encountered when invoking run on a nested suite -
ConfiguredApp needs an Application value associated with key
"org.scalatestplus.play.app" in the config map. Did you forget to
annotate a nested suite with #DoNotDiscover?
java.lang.IllegalArgumentException: ConfiguredApp needs an Application
value associated with key "org.scalatestplus.play.app" in the config
map. Did you forget to annotate a nested suite with #DoNotDiscover?
someone have an idea why is that...?
thanks!333333
My answer is not answer to current question, but I want give some advice. If you want to write unit tests for controllers or some service, I would suggest to use a PlaySpec. In order to inject custom configuration for testing environment:
class MyControllerSpec extends PlaySpec with OneAppPerSuite {
val myConfigFile = new File("app/test/conf/application_test.conf")
val parsedConfig = ConfigFactory.parseFile(myConfigFile)
val configuration = ConfigFactory.load(parsedConfig)
implicit override lazy val app: Application = new GuiceApplicationBuilder()
.overrides(bind[Configuration].toInstance(Configuration(configuration)))
.build()
"MyController #index" should {
"should be open" in {
val result = route(app, FakeRequest(GET, controllers.routes.MyController.index().url)).get
status(result) mustBe OK
}
}
}
It seems that you tried to run this test alone. But with a ConfiguredAppyou must run this test with a Suite, like
class AcceptanceSpecSuite extends PlaySpec with GuiceOneAppPerSuite {
override def nestedSuites = Vector(new MyApiServiceSpec)
}
The injection looks ok.
Let say I have an actor called TestedActor wich is able to save an Int value and send it back as follow:
class TestedActor extends Actor {
override def receive = receive(0)
def receive(number: Int): Receive = {
case new_number: Int => context.become(receive(new_number))
case ("get", ref: ActorRef) => ref ! number
}
}
In my test, I would like to be able to get this Integer and test it.
So i've been thinking about creating something like:
class ActorsSpecs extends FlatSpec with Matchers {
case class TestingPositive(testedActor: ActorRef) extends Actor {
override def receive = {
case number: Int => checkNumber(number)
case "get" => testedActor ! ("get", self)
}
def checkNumber(number: Int) = {
number should be > 0
}
}
implicit val system = ActorSystem("akka-stream")
implicit val flowMaterializer = ActorMaterializer()
val testedActor = system.actorOf(Props[TestedActor], name = "testedActor")
val testingActor = system.actorOf(Props(new TestingPositive(testedActor)), name = "testingActor")
testingActor ! "get"
}
This way, i'm able to create this TestingPositive actor, to get the number in the TestedActor and test it in checkNumber.
It seems to be working well, my problem is :
When the test fail, it raise an exception in the actor thread, I can see what went wrong in the console, but it is still saying that all my tests succeeded. Because (I think) the main thread is not aware of this failure.
Does someone knows an easier way than all of this TestingActor stuff?
Or any solution to tell the main thread that it failed?
Thank you
Take a look at using TestKit docs here. You can write a much simpler test for your actor. See how you like this test:
import akka.actor.{Props, ActorSystem}
import akka.testkit.{TestProbe, TestKit}
import org.scalatest.{BeforeAndAfterAll, FlatSpecLike, ShouldMatchers}
class ActorSpecs extends TestKit(ActorSystem("TestSystem"))
with FlatSpecLike
with ShouldMatchers
with BeforeAndAfterAll {
override def afterAll = {
TestKit.shutdownActorSystem(system)
}
def fixtures = new {
val caller = TestProbe()
val actorUnderTest = system.actorOf(Props[TestedActor], name = "testedActor")
}
"The TestedActor" should "pass a good test" in {
val f = fixtures; import f._
caller.send(actorUnderTest, 42)
caller.send(actorUnderTest, ("get", caller.ref))
caller.expectMsg(42)
}
"The TestedActor" should "fail a bad test" in {
val f = fixtures; import f._
caller.send(actorUnderTest, 42)
caller.send(actorUnderTest, ("get", caller.ref))
caller.expectMsg("this won't work")
}
}
Also, you should know about sender. While your get certainly works, a cleaner approach might be to reply to the sending actor:
def receive(number: Int): Receive = {
case new_number: Int => context.become(receive(new_number))
case "get" => sender ! number
}
And the test becomes:
"The TestedActor" should "pass a good test" in {
val f = fixtures; import f._
caller.send(actorUnderTest, 42)
caller.send(actorUnderTest, "get")
caller.expectMsg(42)
}
And finally, I'll shamelessly plug my recent blog post about maintaining an akka code base with my team. I feel morally obligated to give a new hAkker an opportunity to read it. :)