I'm trying to concatenate a vector into a string in order to insert it into a get call.
Previously in my code I use the following:
.exec(http("request_11")
.get("/api/data/users/${userId}")
.headers(headers_3)
.check(jsonPath("$..users[0].links.orgRoles[*]").findAll.saveAs("orgRoles")))
Which gives me an orgRoles session variable of:
Vector(b2b5fb81-4025-48a5-9890-a6ec7d64f317, 744db047-1477-4bb6-9c55-70071ce41cb8, 423b0b07-7bfa-416a-9b95-cc1d75d7d781)
I'd then like to use this concatenated session variable like so:
.exec(http("request_50")
.get("/api/data/orgRoles/${orgRoles}")
Which would, in effect, do the following get call:
.exec(http("request_50")
.get("/api/data/orgRoles/b2b5fb81-4025-48a5-9890-a6ec7d64f317,744db047-1477-4bb6-9c55-70071ce41cb8,423b0b07-7bfa-416a-9b95-cc1d75d7d781")
.headers(headers_3))
I'm brand new to Gatling and Scala, and have done a ton of research. I found the following answer which offers that
session => session("itemIds").validate[Seq[String]].map(_.mkString(","))
is the right approach. In turn, I've tried to manipulate the session variable with the following:
...
.exec { session =>
session("orgRoles").map(_.mkString(","))
}
.exec(http("request_50")
.get("/api/data/orgRoles/${orgRoles}")
.headers(headers_3))
...
Yet end up with the error:
value mkString is not a member of io.gatling.core.session.SessionAttribute
Any suggestions on moving forward?
The error you are getting is because by:
session("orgRoles")
you are calling Session.apply method at session instance (for more see source code). The apply method returns SessionAttribute instance, which has following methods:
// Use when you are certain that key exists in session, otherwise you'll get exception which you has to handle
def as[T: NotNothing]: T
// Use when you want to handle also not existing key and you can react to `None` case
def asOption[T: TypeCaster: ClassTag: NotNothing]: Option[T]
// Use when you want to code against success path and use combinators provided by `Validation`
def validate[T: TypeCaster: ClassTag: NotNothing]: Validation[T]
Each servers different purpose (see comments). All above mentioned methods of SessionAttribute are just getters from session. Their only purpose is to provide you a value of the key stored in session in any form.
Only after getting the value you can do anything with it. The safest bet is to use validate method which provides common combinators (used in all Scala collections) map and flatMap to manipulate the value when it exist. You can find more about Validation concept here. Hence your code should look like:
// Just getting from session and concatenating
session => session("orgRoles").validate[Seq[String]].map( _.mkString(",") )
// also storing back to session
session => session.set("orgRoles", session("orgRoles").validate[Seq[String]].map( _.mkString(",") ))
This is also what you already wrote above. But you are already using check DSL method and thus you can use transform() directly in your first call as:
.exec(
http("request_11")
.get("/api/data/users/${userId}")
.headers(headers_3)
.check(
jsonPath("$..users[0].links.orgRoles[*]")
.findAll
.transform( _.mkString(",") )
.saveAs("orgRoles")
)
)
It's straightforward, easier to follow and there is no temporary save to Session. Even the function which you put to transform is easier, while it is defined as A => B thus plain transformation. Check-out more about transforming here.
Have you tried calling the toString or maybe groupHierarchy?
session => session("itemIds").validate[Seq[String]].map(_.toString.mkString(","))
session => session("itemIds").validate[Seq[String]].map(_.groupHierarchy.mkString(","))
take a look at: https://github.com/gatling/gatling/blob/master/gatling-core/src/main/scala/io/gatling/core/session/Session.scala
I ended up solving it with this:
.exec(session =>
session.set("orgRolesConcat", session("orgRoles").as[Seq[String]].mkString(",")))
Related
I'm using Mockito with ScalaTest. Consider this simplified example.
Model case class:
case class Batch(batchId: Long,
timestamp: Option[LocalDateTime] = Some(LocalDateTime.now),
invoicesReceived: Option[Int])
In my test I'm mocking a class called BatchRepository which has a method with this signature:
def create(conn: Connection, batch: Batch): Long
Relevant bit of test code:
verify(batchRepository, times(1)).create(anyObject(),
Batch(anyLong(), anyObject(), Matchers.eq(Some(1))))
)
The beef is: I'd like to verify that the code under test calls the mocked repository method with whatever Connection and a Batch instance with whatever id and timestamp, but invoicesReceived being exactly Some(1).
Using Mockito, is this possible at all, and if so, how?
The production code creates a new Batch which sets the timestamp to current moment, so I think it's pretty much impossible to create a real Batch object in the test for the verify() call with the exact same timestamp. So at least for the timestamp I'd need anyObject().
I tried many variations, like wrapping the whole Batch in Matchers.eq(), but I haven't found anything that works:
Invalid use of argument matchers! 2 matchers expected, 4 recorded [...]
I'd be happy to hear I'm using matchers all wrong if there turns out to be some alternative way to use Mockito to test what I want. 🙂
(I was having hard time writing a good name for this question; please edit or leave a comment if you understand what I'm asking and can express it more succinctly.)
The problem is you are trying to verify two calls at once - create and Batch.apply. Can't do that.
One way to do what you want is ArgumentCaptor:
val captor = ArgumentCaptor.forClass(classOf[Batch])
verify(batchRepository).create(any(), captor.capture)
captor.getValue should matchPattern {
case Batch(_, _, Some(1)) =>
}
// or just `captor.getValue.infoReceived shouldBe Some(1)`
Have you tried mockito-scala? it supports partial functions as matchers so you could just write
verify(batchRepository, times(1)).create(any, argMatching({case Batch(_, _, Some(1)) => }))
Or even better if you use the idiomatic syntax
batchRepository.create(*, argMatching({case Batch(_, _, Some(1)) => })) was called
is there a way to save the result of a slick query into a new object?
This is my slick result, there is only one "object" in the list
val result: Future[Seq[ProcessTemplatesModel]] = db.run(action)
The result should be mapped on ProcessTemplatesModel because I want to access the values like this
process.title
Is this possible?
Thanks
TL;DR: you should keep the context as long as you can.
Future denotes the fact that the value will be given at some time in the future (this is what I call some context for the value).
The bad way to use it would be to block your thread, until such value is found, and then work with it.
A better way is to tell your program: "Once the value is found (whenever that is), do something with it". That's a continuation, or call-back, and is implemented with map and flatMap in scala.
Seq is another context for your value. It means that you actually have different possible values. If you want to make sure that you have at most one value, you can always do seq.headOption to switch context from Seq to Option.
The bad way to use it would be to take the first value without bothering checking if it exists or not.
A better way is to tell your program: "No matter how many values you have, do this for each of them".
Now, how do you work in context? You use the Functor and/or Monad operators: map, flatMap.
For instance, if you want to apply a function convertToSomethingElse to each element of your context, just do
result.map(list => list.map(process => convertToSomethingElse(process))
And you'll get a Future[Seq[SomethingElse]].
Another example, if you want to save the result somewhere else, you'll probably have some IO, or database operations, which may take some time, and possibly fail. We will assume you have a function save(entity: ProcessTemplateModel): Future[Boolean] that allows you to save one of your models. The fact that the function will take some time (and that it will be started in another thread) and possibly fail is visible in the return type Future[Boolean] (Boolean is not important here, it's the fact that we have again the Future context that matters).
Here, you will have to do (assuming you just want to save the first element in your list):
val savedFirstResult: Future[Option[ProcessTemplatesModel]] = result.flatMap {list =>
Future.traverse(list.headOption){ process => //traverse will switch the Future and Option contexts
save(process)
}
}
So as you can see, we can do most of what we want by staying inside the contexts that are returned by Slick. You shouldn't want to get outside of them because
most of the time, there's no need to, when you have map to use inside context some function for values outside context
extracting methods are most of the time unsafe: Option#get throws an exception if no element is in the Option, Await.result(future, duration) may block all computations or throw exceptions
responses in Play! can be given as Futures in a controller, using Action.async
I've wrote a class with some functions that does HTTP calls and returns a Future[String]. I use those functions inside a method that I need to write some tests:
def score(rawEvent: Json) = {
httpService
.get("name", formatJsonAttribute(rawEvent.name))
.onComplete { op =>
op.map { json =>
//What must be tested
}
}
}
The function onComplete doesn't have a return type - it returns Unit. How can I replace that onComplete to make my function return something to be tested?
I completely agree with #Michal, that you should always prefer map to onComplete with Futures. However I'd like to point out that, as you said yourself, what you wish to test is not the HTTP call itself (which relies on an HTTP client you probably don't need to test, a response from a server on which you may have no control, ...), but what you do with its answer.
So why not write a test, not on the function score, but on the function you wrote in your onComplete (or map, if you decided to change it)?
That way you will be able to test it with precise values for json, that you may wish to define as the result you will get from the server, but that you can control completely (for instance, you could test border cases without forcing your server to give unusual responses).
Testing that the two (HTTP call and callback function) sit well together is not a unit-test question, but an integration-test question, and should be done only once you know that your function does what is expected of it.
At that time, you will effectively need to check the value of a Future, in which case, you can use Await.result as #Michal suggested, or use the relevant constructs that your test framework gives. For instance, scalatest has an AsyncTestSuite trait for this kind of issue.
Use map instead of onComplete. It will also provide you with resolved value inside mapping function. The return type of score function will be Future[T] where T will be the result type of your processing.
In the tests you can use scala.concurrent.Await.result() function.
I have been using the mapWithState API in Spark Streaming, but 2 things are not clear about the StateSpec.function:
Let's say my function is:
def trackStateForKey(batchTime: Time,
key: Long,
newValue: Option[JobData],
currentState: State[JobData]): Option[(Long, JobData)]
Why is the new value an Option[T] type? As far as I've seen, it was always defined for me, and since the method is supposed to be called with a new state, I don't really see the point why it could be optional.
What does the return value mean? I tried to find some pointers in the documentations and source code, but none of them describe what it is used for. Since I'm modifying the state of a key using state.remove() and state.update(), why would I have to do the same with return values?
In my current implementation I return None if I remove the key, and Some(newState) if I update it, but I'm not sure if that is correct.
Why is the new value an Option[T] type? As far as I've seen, it was
always defined for me, and since the method is supposed to be called
with a new state, I don't really see the point why it could be
optional.
It is an Option[T] for the reason that if you set a timeout using StateSpec.timeout, e.g:
StateSpec.function(spec _).timeout(Milliseconds(5000))
then the value passed in once the function times out will be None and the isTimingOut method on State[T] will yield true. This makes sense, because a timeout of the state doesn't mean that a new value has arrived for the specified key, and generally safer to use than passing null for T (which wouldn't work for primitives anyway) as you expect the user to safely operate on an Option[T].
You can see that in the Sparks implementation:
// Get the timed out state records, call the mapping function on each and collect the
// data returned
if (removeTimedoutData && timeoutThresholdTime.isDefined) {
newStateMap.getByTime(timeoutThresholdTime.get).foreach { case (key, state, _) =>
wrappedState.wrapTimingOutState(state)
val returned = mappingFunction(batchTime, key, None, wrappedState) // <-- This.
mappedData ++= returned
newStateMap.remove(key)
}
}
What does the return value mean? I tried to find some pointers in the
documentations and source code, but none of them describe what it is
used for. Since I'm modifying the state of a key using state.remove()
and state.update(), why would I have to do the same with return
values?
The return value is a way to pass intermediate state along the spark graph. For example, assume that I want to update my state but also perform some operation in my pipeline with the intermediate data, e.g:
dStream
.mapWithState(stateSpec)
.map(optionIntermediateResult.map(_ * 2))
.foreachRDD( /* other stuff */)
That return value is exactly what allows me to continue operating on said data. If you don't care for the intermediate result and only want the complete state, then outputting None is perfectly fine.
Edit:
I've written a blog post (following this question) which attempts to give an in-depth explanation to the API.
I'm exploring the Play Framework and have run in to a bit of a corner.
After making an API call to Google Analytics with the WS library, I receive a Future[Response] object. After digesting that Response object, I get the data I really care about, but because it's wrapped in a Future, I'm having some trouble writing it to the browser.
OK( gaApiString )
This gives me an error that reads:
Cannot write an instance of scala.concurrent.Future[String] to HTTP
response. Try to define a Writeable[scala.concurrent.Future[String]]
I'm having some trouble finding & understanding how to use a Writable object. Little help?
You need to map the Future to a Future[Result], passing it to Action.async.
def test = Action.async {
val wsResult: Future[String] = ...
wsResult.map { gaApiString =>
Ok(gaApiString)
}
}
If gaApiString is actually List[String], then it depends on what you want to do with it. Displaying it as a comma-separated list you could just change it to Ok(gaApiString.mkString(",")). The key here is mapping the Future to manipulate the value after it's completed.