To set the context, I am trying to do this in Flutter.
For example,
I have a test that passes, "if I set my mock to say 'no network connection' and expect 'NetworkUnavailable' to result."
Then, I thought to write next test that "if network is available, result could be anything except 'NetworkUnavailable'"
I am struggling to setup an expect matcher for that. Does Mockito has something for this, something like AnyExcept([matcher])?
There is an isNot Matcher that you can combine with other Matchers. So, for example, you should be able to do something like: expect(valueToTest, isNot(unwantedValue)); or expect(valueToTest, isNot(isIn([unwantedValue1, unwantedValue2])));
If that doesn't do quite what you want, you also can use the predicate to easily create your own Matcher from a boolean function.
(You might not have found these if you were searching the Mockito documentation
because they're part of package:matcher (normally included as part of package:test); they're used for unit tests in general, not just for mocks.)
Related
I am going through the coursera functional programming and have an assignment where the scalatest is written using FunSuite and Checkers.
This test framework is new to me but I have some basic idea of using assertion, as I have developed pigunit for an user defined function using assert.
As google didn't give me clear usage of Checkers and how it is different from assert, could anyone clarify where Checkers can be used and why not assert be used.
Thanks
As you know, an assertion is a way of testing that a certain condition holds. These are pretty simple in ScalaTest, as you only need to use assert. For example:
assert(List(1, 2, 3).length == 3)
"Checkers," or, as they are more often called, properties, are a bit different. They are a way to assert that a condition holds for all possible inputs instead of for a single case. For example, here is a property that tests that a list always has a nonnegative length:
check((ls: List[Int]) => ls.length >= 0)
At this point, ScalaTest defers to ScalaCheck to do the heavy lifting. ScalaCheck generates random values for ls in an effort to find one that fails the test. This concept is called property-based testing. You can read more about how to use it in ScalaTest here.
Im working with scalatest and scalacheck, alsso working with FeatureSpec.
I have a generator class that generate object for me that looks something like this:
object InvoiceGen {
def myObj = for {
country <- Gen.oneOf(Seq("France", "Germany", "United Kingdom", "Austria"))
type <- Gen.oneOf(Seq("Communication", "Restaurants", "Parking"))
amount <- Gen.choose(100, 4999)
number <- Gen.choose(1, 10000)
valid <- Arbitrary.arbitrary[Boolean]
} yield SomeObject(country, type, "1/1/2014", amount,number.toString, 35, "something", documentTypeValid, valid, "")
Now, I have the testing class which works with FeatureSpec and everything that I need to run the tests.
In this class I have scenarios, and in each scenario I want to generate a different object.
The thing is from what I understand is that to generate object is better to use forAll func, but for all will not sure to bring you an object so you can add minSuccessful(1) to make sure you get at list 1 obj....
I did it like this and it works:
scenario("some scenario") {
forAll(MyGen.myObj, minSuccessful(1)) { someObject =>
Given("A connection to the system")
loginActions shouldBe 'Connected
When("something")
//blabla
Then("something should happened")
//blabla
}
}
but im not sure exactly what it means.
What I want is to generate an invoice each scenario and do some actions on it...
im not sure why i care if the generation work or didnt work...i just want a generated object to work with.
TL;DR: To get one object, and only one, use myObj.sample.get. Unless your generator is doing something fancy that's perfectly safe and won't blow up.
I presume that your intention is to run some kind of integration/acceptance test with some randomly generated domain object—in other words (ab-)use scalacheck as a simple data generator—and you hope that minSuccessful(1) would ensure that the test only runs once.
Be aware that this is not the case!. scalacheck will run your test multiple times if it fails, to try and shrink the input data to a minimal counterexample.
If you'd like to ensure that your test runs only once you must use sample.
However, if running the test multiple times is fine, prefer minSuccessful(1) to "succeed fast" but still profit from minimized counterexamples in case the test fails.
Gen.sample returns an option because generators can fail:
ScalaCheck generators can fail, for instance if you're adding a filter (listingGen.suchThat(...)), and that failure is modeled with the Option type.
But:
[…] if you're sure that your generator never will fail, you can simply call Option.get like you do in your example above. Or you can use Option.getOrElse to replace None with a default value.
Generally if your generator is simple, i.e. does not use generators that could fail and does not use any filters on its own, it's perfectly safe to just call .get on the option returned by .sample. I've been doing that in the past and never had problems with it. If your generators frequently return None from .sample they'd likely make scalacheck fail to successfully generate values as well.
If all that you want is a single object use Gen.sample.get.
minSuccessful has a very different meaning: It's the minimal number of successful tests that scalacheck runs—which by no means implies
that scalacheck takes only a single value out of the generator, or
that the test runs only once.
With minSuccessful(1) scalacheck wants one successful test. It'll take samples out of the generator until the test runs at least once—i.e. if you filter the generated values with whenever in your test body scalacheck will take samples as long as whenever discards them.
If the test passes scalacheck is happy and won't run the test a second time.
However if the test fails scalacheck will try and produce a minimal example to fail the test. It'll shrink the input data and run the test as long as it fails and then provides you with the minimized counter example rather than the actual input that triggered the initial failure.
That's an important property of property testing as it helps you to discover bugs: The original data is frequently too large to lend itself for debugging. Minimizing it helps you discover the piece of input data that actually triggers the failure, i.e. corner cases like empty strings that you didn't think of.
I think the way you want to use Scalacheck (generate only one object and execute the test for it) defeats the purpose of property-based testing. Let me explain a bit in detail:
In classical unit-testing, you would generate your system under test, be it an object or a system of dependent objects, with some fixed data. This could e.g. be strings like "foo" and "bar" or, if you needed a name, you would use something like "John Doe". For integers and other data, you can also randomly choose some values.
The main advantage is that these are "plain" values—you can directly see them in the code and correlate them with the output of a failed test. The big disadvantage is that the tests will only ever run with the values you specified, which in turn means that your code is also only tested with these values.
In contrast, property-based testing allows you to just describe how the data should look like (e.g. "a positive integer", "a string of maximum 20 characters"). The testing framework will then—with the help of generators—generate a number of matching objects and execute the test for all of them. This way, you can be more sure that your code will actually be correct for different inputs, which after all is the purpose of testing: to check if your code does what it should for the possible inputs.
I never really worked with Scalacheck, but a colleague explained it to me that it also tries to cover edge-cases, e.g. putting in a 0 and MAX_INT for a positive integer, or an empty string for the aforementioned string with max. 20 characters.
So, to sum it up: Running a property-based test only once for one generic object is the wrong thing to do. Instead, once you have the generator infrastructure in place, embrace the advantage you then have and let your code be checked a lot more times!
In previous versions of Elastic4s you could do something like
val argument1: ArgumentCapture[DeleteIndexDefinition] = ???
verify(client).execute(argument1.capture())
assert(argument1 == ???)
val argument2: ArgumentCapture[IndexDefinition] = ???
verify(client, times(2)).execute(argument2.capture())
assert(argument2 == ???)
after several executions in your test (i.e. one DeleteIndexDefinition, followed of two IndexDefinition). And each verify would be matched against its type.
However, Elastic4s now takes an implicit parameter in its client.execute method. The parameter is of type Executable[T,R], which means you now need something like
val argument1: ArgumentCapture[DeleteIndexDefinition] = ???
verify(client).execute(argument1.capture())(any[Executable[DeleteIndexDefinition,R]])
assert(argument1 == ???)
val argument2: ArgumentCapture[IndexDefinition] = ???
verify(client, times(2)).execute(argument2.capture())(any[Executable[IndexDefinition,R]])
assert(argument2 == ???)
After doing that, I was getting an error. Mockito is considering both three client.execute in the first verify. Yes, even if the first parameter is of a different type.
That's because the implicit(the second parameter) has, after type erasure, the same type Executable.
So the asertions were failing. How to test in this setup?
The approach now taken in elastic4s to encapsulate the logic for executing each request type is one using typeclasses. This is why the implicit now exists. It help modularize each request type, and avoids the God class anti-pattern that was starting to creep into the ElasticClient class.
Two things I can think of that might help you:
What you already posted up, using Mockito and passing in the implicit as another matcher. This is how you can mock a method using implicits in general.
Not use mockito, but spool up a local embedded node, and try it against real data. This is my preferred approach when I write elasticsearch code. The advantages are that you're testing real queries against the real server, so not only checking that they are invoked, but that they actually work. (Some people might consider this an integration test, but whatever I don't agree, it all runs inside a single self contained test with no outside deps).
The latest release of elastic4s even include a testkit that makes it really easy to get the embedded node. You can look at almost any of the unit tests to give you an idea how to use it.
My solution was to create one verify with a generic type. It took me a while to realise that even if there is no common type, you always have AnyRef.
So, something like this works
val objs: ArgumentCaptor[AnyRef] = ArgumentCaptor.forClass(classOf[AnyRef])
verify(client, times(3)).execute(objs.capture())(any())
val values = objs.getAllValues
assert(values.get(0).isInstanceOf[DeleteIndexDefinition])
assert(values.get(1).isInstanceOf[IndexDefinition])
assert(values.get(2).isInstanceOf[IndexDefinition])
I've created both the question and the answer. But I'll consider other answers.
I'm migrating my karma-ng-scenario tests suite to protractor. I would like to do something like
// karma-ng-scenario
expect(element('legend').text()).not().toBe("LOGIN_CONNECT");
in the protractor way. But it seems there isn't a not() function.
I'm using angular-translate to bind the LONGIN_CONNECT string into multiple languages and I want to test if the string is translated.
More globally, is there a a way test if something is different ? ... don't have a class, don't exists on the page, is not selected, ...
It is definitely worth looking at the API docs. I have these open pretty much all the time.
There are lots of Web Driver functions you can use like isEnabled(), isDisplayed(), isSelected() etc. Protractor uses Jasmine syntax so you can use '.toBe(false)' to assert things are false.
To check for classes, you can do something like:
expect(myElement.getAttribute('class')).toContain('my-class-name');
To compare strings and assert that they do NOT match you could use .not. Jasmine docs
say:
Every matcher's criteria can be inverted by prepending .not:
expect(x).not.toEqual(y); compares objects or primitives x and y and
passes if they are not equivalent
You can use something like:
expect(model.getText()).not.toContain('abcdef');
There is a .not property nowadays.
I'm using the following to check for NOT matching:
expect(element('legend').text() === "LOGIN_CONNECT").toBe(false);
I'm trying to test-drive some Scala code using Specs2 and Mockito. I'm relatively new to all three, and having difficulty with the mocked methods returning null.
In the following (transcribed with some name changes)
"My Component's process(File)" should {
"pass file to Parser" in new modules {
val file = mock[File]
myComponent.process(file)
there was one(mockParser).parse(file)
}
"pass parse result to Translator" in new modules {
val file = mock[File]
val myType1 = mock[MyType1]
mockParser.parse(file) returns (Some(myType1))
myComponent.process(file)
there was one(mockTranslator).translate(myType1)
}
}
The "pass file to Parser" works until I add the translator call in the SUT, and then dies because the mockParser.parse method has returned a null, which the translator code can't take.
Similarly, the "pass parse result to Translator" passes until I try to use the translation result in the SUT.
The real code for both of these methods can never return null, but I don't know how to tell Mockito to make the expectations return usable results.
I can of course work around this by putting null checks in the SUT, but I'd rather not, as I'm making sure to never return nulls and instead using Option, None and Some.
Pointers to a good Scala/Specs2/Mockito tutorial would be wonderful, as would a simple example of how to change a line like
there was one(mockParser).parse(file)
to make it return something that allows continued execution in the SUT when it doesn't deal with nulls.
Flailing about trying to figure this out, I have tried changing that line to
there was one(mockParser).parse(file) returns myResult
with a value for myResult that is of the type I want returned. That gave me a compile error as it expects to find a MatchResult there rather than my return type.
If it matters, I'm using Scala 2.9.0.
If you don't have seen it, you can look the mock expectation page of the specs2 documentation.
In your code, the stub should be mockParser.parse(file) returns myResult
Edited after Don's edit:
There was a misunderstanding. The way you do it in your second example is the good one and you should do exactly the same in the first test:
val file = mock[File]
val myType1 = mock[MyType1]
mockParser.parse(file) returns (Some(myType1))
myComponent.process(file)
there was one(mockParser).parse(file)
The idea of unit testing with mock is always the same: explain how your mocks work (stubbing), execute, verify.
That should answer the question, now a personal advice:
Most of the time, except if you want to verify some algorithmic behavior (stop on first success, process a list in reverse order) you should not test expectation in your unit tests.
In your example, the process method should "translate things", thus your unit tests should focus on it: mock your parsers and translators, stub them and only check the result of the whole process. It's less fine grain but the goal of a unit test is not to check every step of a method. If you want to change the implementation, you should not have to modify a bunch of unit tests that verify each line of the method.
I have managed to solve this, though there may be a better solution, so I'm going to post my own answer, but not accept it immediately.
What I needed to do was supply a sensible default return value for the mock, in the form of an org.mockito.stubbing.Answer<T> with T being the return type.
I was able to do this with the following mock setup:
val defaultParseResult = new Answer[Option[MyType1]] {
def answer(p1: InvocationOnMock): Option[MyType1] = None
}
val mockParser = org.mockito.Mockito.mock(implicitly[ClassManifest[Parser]].erasure,
defaultParseResult).asInstanceOf[Parser]
after a bit of browsing of the source for the org.specs2.mock.Mockito trait and things it calls.
And now, instead of returning null, the parse returns None when not stubbed (including when it's expected as in the first test), which allows the test to pass with this value being used in the code under test.
I will likely make a test support method hiding the mess in the mockParser assignment, and letting me do the same for various return types, as I'm going to need the same capability with several return types just in this set of tests.
I couldn't locate support for a shorter way of doing this in org.specs2.mock.Mockito, but perhaps this will inspire Eric to add such. Nice to have the author in the conversation...
Edit
On further perusal of source, it occurred to me that I should be able to just call the method
def mock[T, A](implicit m: ClassManifest[T], a: org.mockito.stubbing.Answer[A]): T = org.mockito.Mockito.mock(implicitly[ClassManifest[T]].erasure, a).asInstanceOf[T]
defined in org.specs2.mock.MockitoMocker, which was in fact the inspiration for my solution above. But I can't figure out the call. mock is rather overloaded, and all my attempts seem to end up invoking a different version and not liking my parameters.
So it looks like Eric has already included support for this, but I don't understand how to get to it.
Update
I have defined a trait containing the following:
def mock[T, A](implicit m: ClassManifest[T], default: A): T = {
org.mockito.Mockito.mock(
implicitly[ClassManifest[T]].erasure,
new Answer[A] {
def answer(p1: InvocationOnMock): A = default
}).asInstanceOf[T]
}
and now by using that trait I can setup my mock as
implicit val defaultParseResult = None
val mockParser = mock[Parser,Option[MyType1]]
I don't after all need more usages of this in this particular test, as supplying a usable value for this makes all my tests work without null checks in the code under test. But it might be needed in other tests.
I'd still be interested in how to handle this issue without adding this trait.
Without the full it's difficult to say but can you please check that the method you're trying to mock is not a final method? Because in that case Mockito won't be able to mock it and will return null.
Another piece of advice, when something doesn't work, is to rewrite the code with Mockito in a standard JUnit test. Then, if it fails, your question might be best answered by someone on the Mockito mailing list.