New to scala futures I try to call a web service like
wsClient.url(baseUrl + url).withHeaders("Content-Type" -> "application/json").post(dataForR).flatMap(parseOutlierResponse)
using ply-ws library
I validate & map the response as follows https://gist.github.com/geoHeil/943a18d43279762ad4cdfe9aa2e40770
The main thing is:
Await.result(callAMethodCallingTheFirstSnippet, 5.minutes)
Strangely this works just fine in the repl. However if run via sbt run I get a NullPointer Exception. I already verified the JSON response manually. It validates like a breeze. Even the mapping works great. However, there must be a problem with the futures I am using. But I am not sure what is wrong. It seems like the flatMap method is called before there already is a result.
Interestingly if I do not await the result there is no null-pointer exception, but the parsed result is displayed correctly (however, the program does not exit). But there, where I really use this code, I somehow need to await the successful completion to further deal with it.
Below you will find an illustration of the problem
I do not see any major concern with your code! I made a small test with the following code bit and it seems to be working perfectly, both in the REPL and when using sbt run:
WS.clientUrl(s"http://$hostName/api/put").withHeaders(jsonHeaders: _*).post(body).map { r =>
if (r.status >= 400)
logger.warn(s"Invalid http response status: ${r.status} \n ${r.body}")
else
logger.debug(s"Successfully persisted data. Http response ${r.status}")
}
After more and more debugging I found that some implicits were in the wrong scope and the order of dependent case-classes was wrong. After moving them into the correct scope (the method performing the request) the null-pointer exception is fixed.
I could only find the "real" error after changing from flatmap to map which I find very strange. However, now both methods work fine.
Related
I have been getting very frustrated with Dart as runtime exceptions seem to fail silently. Program execution will continue after some type of failure and leave me totally stumped about what is not working and why. For example, in using aqueduct for dart server (this is not an issue unique to aqueduct, but dart in general) I have this code:
jsonData.forEach((jsonObject) async {
ManagedObject record = Document();
record.read(jsonObject);
print(Util.jsonFormatToString(record.asMap()));
....more code
}
In this case, the application fails silently on record.read() falls out of the forEach loop and then continues code execution. During debugging, the application returns completely strange results and after debugging, presumably there is a problem with the jsonObject attempting to be read into the managed object, but Dart gives no notice that there is a problem or what that problem might be.
This is one example of many I have been running into.
Why does Dart fail so silently or am I missing some setting somewhere that is hiding critical info? I am using IntelliJ IDE.
We're testing JS apps with protractor and sometimes we get some randomly unstable tests. Sometimes the errors are Failed: stale element reference: element is not attached to the page document
Just a hunch but sometimes developers write the code like this
await element(await by.css('.pager-next')).click();
And sometimes like this
await element(by.css('.pager-next')).click();
Is either of these "more" correct? Is there any need for the inner await or does it make no difference? Could this be causing stale element reference errors?
await is only useful for functions that return a Promise. Of the three functions in your snippet, only one of them returns a Promise. (You can check what each of them returns in the Protractor api).
ElementFinder.click() returns a Promise.
element() returns an ElementFinder.
by.css() returns a ProtractorLocator.
Therefore, the only call that should have await on it is click(), which you have done correctly in your second snippet.
StaleElementReferenceException is typically caused when you keep a reference to an object that has been removed from the page, e.g. using ElementFinder.getWebElement(). This removal can be subtle sometimes. For example, Angular will sometimes invisibly remove an element from the DOM and quickly replace it with an identical-looking one. It's hard to tell that anything even changed, but from Webdriver's perspective, the element it had a reference to has disappeared.
I think a better practice would be not to be write locators like that. I'd leave awaits for functions involving the element.
I would have a locator written like this:
const myElement = element(by.css('.pager-next'));
And then using async/await in a function:
let clickMyElement = async function(){
await myElement.click();
};
Well the system we have has a bunch of dependencies, but I'll try to summarize what's going on without divulging too much details.
Test assembly in the form of a .dll is the one being executed. A lot of these tests call an API.
In the problematic method, there's 2 API calls that have an await on them: one to write a record to that external interface, and another to extract all records and then read the last one in that external interface, both via API. The test is simply to check if writing the last record was successful in an end-to-end context, that's why there's both a write and then a read.
If we execute the test in Visual Studio, everything works as expected. I also tested it manually via command lining vstest.console.exe, and the expected results always come out as well.
However, when it comes to VS Test task in VSTS, it fails for some reason. We've been trying to figure it out, and eventually we reached the point where we printed the list from the 'read' part. It turns out the last record we inserted isn't in the data we pulled, but if we check the external interface via a different method, we confirmed that the write process actually happened. What gives? Why is VSTest getting like an outdated set of records?
We also noticed two things:
1.) For the tests that passed, none of the Console.WriteLine outputs appear in the logs. Only on Failed test do they do so.
2.) Even if our Data.Should.Be call is at the very end of the TestMethod, the logs report the fail BEFORE it prints out the lines! And even then, the printing should happen after reading the list of records, and yet when the prints do happen we're still missing the record we just wrote.
Is there like a bottom-to-top thing we're missing here? It really seems to me like VSTS vstest is executing the assert before the actual code. The order of TestMethods happen the right order though (the 4th test written top-to-bottom in the code is executed 4th rather than 4th to last) and we need them to happen in the right order because some of the later tests depend on the former tests succeeding.
Anything we're missing here? I'd put a source code but there's a bunch of things I need to scrub first if so.
Turns out we were sorely misunderstanding what 'await' does. We're using .Wait() instead for the culprit and will also go back through the other tests to check for quality.
I'm using scalatest and easymock 3.4. Tried some other versions.
I'm having issues testing w/ akka using easymock.
Two strange behaviors:
Setting expectations worked if they are NOT called, but I can't get tests
to fail if no expectations are set and the method is called.
Also I'm having these weird errors where it says the method is called as it's supposed to be! Why does this fail? Mocks are used inside an actor so there is obviously async activity involved. I can't seem to use verify :( I'm guessing this is somehow related to the async nature of akka.
[info] java.lang.AssertionError: Expectation failure on verify:
[info] YourInterface.poll(0, 100): expected: 1, actual: 1
Using all of the behavior in a test works but when introducing akka, I start seeing weird behavior making it hard to post code examples.
In regard to potential runtime failures, like database queries, it seems that one must use some form of Either[String, Option[T]] in order to accurately capture the following outcomes:
Some (record(s) found)
None (no record(s) found)
SQL Exception
Option simply does not have enough options.
I guess I need to dive into scalaz, but for now it's straight Either, unless I'm missing something in the above.
Have boxed myself into a corner with my DAO implementation, only employing Either for write operations, but am now seeing that some Either writes depend on Option reads (e.g. checking if email exists on new user signup), which is a majorly bad gamble to make.
Before I go all-in on Either, does anyone have alternate solutions for how to handle the runtime trifecta of success/fail/exception?
Try Box from the fantastic lift framework. It provides exactly what you want.
See this wiki (and the links at the top) for details. Fortunately lift project is well modulized, the only dependency to use Box is net.lift-web % lift-common
Use Option[T] for the cases records found and no records found and throw an exception in the case of SQLException.
Just wrap the exception inside your own exception type, like PersistenceException so that you don't have a leaky abstraction.
We do it like this because we can't and don't want to recover from unexpected database exceptions. The exception gets caught on the top level and our web service returns a 500 Internal server error in such case.
In cases where we want to recover we use Validation from scalaz, which is much like Lift's Box.
Here's my revised approach
Preserve Either returning query write operations (useful for transactional blocks where we want to rollback on for comprehension Left outcome).
For Option returning query reads, however, rather than swallowing the exception with None (and logging it), I have created a 500 error screen, letting the exception bubble up.
Why not just work with Either result type by default when working with runtime failures like query Exceptions? Option[T] reads are a bit more convenient to work with vs Either[Why-Fail, Option[T]], which you have to fold/map through to get at T. Leaving Either to write operations simplifies things (all the more so given that's how the application is currently setup, no refactoring required ;-))
The only other change required is for AJAX requests. Rather than displaying the entire 500 error page response in the AJAX status div container, we check for the status type and display 500 error message accordingly.
if(data.status == 500)
$('#status > div').html("an error occurred, please try again")
Could probably do an isAjax check server-side prior to sending the response; in which case I can send back only status + message rather than the error page itself.