Concise yet scalable Scala/sbt testing to verify Project Euler answers - scala

I’m using Project Euler to learn functional programming in Scala, along with sbt and other best practices. My initial stab at creating tests to verify solutions using ScalaTest was:
package euler
import org.scalatest._
class SolutionsSpec extends Spec with Matchers {
"The problems" should "have the correct solutions" in {
Problem1.solve should be ( 233168 )
Problem2.solve should be ( 4613732 )
Problem3.solve should be ( 6857 )
Problem4.solve should be ( 906609 )
}
}
This is concise, but I’m afraid it might not scale if I ever get to harder problems with longer run times. As far as I can tell, I can't test individual problems this way.
I have searched on GitHub and found two helpful examples, similar to what I am trying to do:
danielmiladinov/ProjectEuler-Scala
todd-cook/Effective-Scala-with-Project-Euler
Both are more verbose than I would like, since I’m basically just testing a mapping from problem number to solutions.
My question: Is there a concise and elegant way to do this testing that is still flexible so that I can check the result for a single or group of problems?
Edit: I found this answer discouraging TDD with Project Euler. To clarify, I am writing unit tests for math helper functions as well. My purpose for checking the answers is partly to check for regressions, but most important as an exercise.
Edit 2: I am now trying to use FunSuite:
class SolutionsSuite extends FunSuite with Matchers {
test("Problem 1") { Problem1.solve should be ( 233168 ) }
test("Problem 2") { Problem2.solve should be ( 4613732 ) }
test("Problem 3") { Problem3.solve should be ( 6857 ) }
test("Problem 4") { Problem4.solve should be ( 906609 ) }
}
This might be closer to what I want. The output looks better:
$ sbt test
...
[info] SolutionsSuite:
[info] - Problem 1 (28 milliseconds)
[info] - Problem 2 (4 milliseconds)
[info] - Problem 3 (38 milliseconds)
[info] - Problem 4 (172 milliseconds)
But I have still not been able to figure out how to run a subset of the problems. This question seems to indicate that what I am looking for is not yet possible with sbt: Run just a specific scalatest test from sbt.
Edit 3: It looks like the ability to run single test function as opposed to an entire suite is targeted for sbt 0.13.3 per issue Allow running of single junit test #911.
Edit 4: For reference, my Project Euler GitHub project.

You might like to copy my setup at https://github.com/sethtisue/project-euler . Each test looks like:
class Problem1 extends Problem(1, "12345") {
def solve: Int = ...
}
Where the "12345" is the expected answer (always a string), and the return type of solve can be anything (since it's just going to get .toStringed, since a string is what you're going to end up entering at the Euler site).
Problem is defined such that each problem is its own suite, so you can easily use sbt's test-only command to run just the problems you want.

Related

Refactoring gradle test class so that test inside can be run in parallel

I have some integration test that are run by gradle in parallel but one of the test classes is significantly slower than others - not much can be done as it iterates in for loop on every element in list ( and it will get bigger ).
As I learned gradle can only parallelise on class level.
The easiest solution would be to refactor every element in list as a class with communal base class - but that would mean that I would need to create a new class for every element in list and for every new added one and I don't feel good about it.
Is there a way to do it better ? For example make scala create runtime classes that would be recognised by gradle ? Maybe parallelise on scala level ? I tried using scala parallel collections and making list par but didn't work - I assume I was missing something.
Here is an example of the class:
class TestWithForEach {
val reports: Se[String] = Seq(
"name",
"name2",
....
)
reports.foreach(reportName => {
it should s"generate $reportName" in {
generateReport(reportName)
}
})
}
One of workarounds I have found is using sparkTest.ParallelTestExecution trait for the Class in question. This is slightly slower than doing new classes for every element so I'm still searching for alternatives.
Found out about scala reflections and it seems that it could maybe work ? Didn't had luck using them yet

How to execute scala tests programmatically

I'm looking for a way to execute scala tests (implemented in munit, but it could be also ScalaTest) programmatically. I want to perform more or less what sbt test does out-of-the box inside my own scala code, without running sbt (focusing on test discovery and execution and getting back a report).
I some something like this in mind:
object Test extends App {
val tests = TestDiscovery.discover("package.that.has.tests")
val reports = tests.foreach(test => test.execute())
// do something with the reports, maybe print to console
}
Is there any documentation related to this?
Scala Test has execute() and run().
In order to understand the impact of all the args it's worth looking at the Scala Test shell as well

Logging the time for each test (and/or testSuite) with scala test

I'm using scala test for my tests, and some time ago the builds started to fail sometimes because some (new, probably) test takes more than the limit allowed by the CI.
At first sight no test should take more than the timeout (10 minutes btw), and by the CI output it not easy to see which test is taking so much time (or how much time every test takes)
I can put simple println-based statements to print the current hour at the beginning/end of each test, but I was wondering if scala test provides some builtin feature for that?
If no, does any other scala test framework/library does?
EDIT: tried to wrap the test
I prefere some native method from the lib itself. I was trying to wrap a specs2 test in a method that would get the description and log the times, but I'm having difficulties in running the test in the moment I want to. I did something Like this:
class SomeSpec {
private def withTimeLog(x: Fragment) = {
println(s"Starting test: ${x.description}(${DateTime.now()})")
//what now?
x.executionResult
println(s"End of test: ${x.description}(${DateTime.now()})")
}
withTimeLog("Feature" should {
"stuff" in {
println(s"test: ${DateTime.now()}")
None.isEmpty must beTrue
}
})
}
and the result:
Starting test: End(2016-07-28T18:11:53.101+01:00)
End of test: End(2016-07-28T18:11:53.191+01:00)
[info] FeatureSpecV2
[info]
test running at 2016-07-28T18:11:54.037+01:00
[info] A test should
[info] + stuff
meaning, test was executed in the end, and not between the 2 print statements. Maybe Fragment.execution result doesn't run the test. Tried with several others. Any idea which method should I be using?
You can display execution times in specs2 by passing the showtimes option on the command line
sbt>testOnly *MySpec -- showtimes
Also if you use future matchers you can adjust a "time factor" with the timeFactor command line option.

Why use Specs2 over JUnit? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
We went through online material such as scalatest site. What are advantages of using scala testing framework Specs2? How does it add to the project over JUnit?
I'm a little reluctant to answer this because it's a quite subjective thing.
But I could summarise it as "In the same way Scala gives you new and cleaner ways to solve problems over Java, Specs2 gives you new and cleaner ways to test over JUnit".
Some quick examples:
Mockito is a first-class citizen
Mockito is my favourite JVM mocking library; using it with Specs2 is as simple as mixing in a trait and you get a nice DSL for verification:
class MySpec extends Specification with Mockito {
// Setup:
val mockConnection = mock[Connection]
...
// Verification: (in a test)
there was one(mockConnection).get("www.google.com")
}
ScalaCheck is a first-class citizen
ScalaCheck can save a heap of time "fuzzing" inputs to your functions, and again, a simple trait mixin gets you all of its power within your Specs2 tests.
Run a test in a Scope
Ever had a problem where tests would work on your machine but break elsewhere, because your machine ran them in a certain order? Or got into "mocking hell" due to interactions between stubbed collaborator behaviour? Specs2 lets you put the entire test into a Scope to contain all of that state and stop it from leaking into other tests:
class AnotherSpec extends Specification with Mockito {
class FooScope extends Scope {
val mockResponse = mock[Response]
val mockConnection = mock[Connection]
...
mockConnection.execute returns mockResponse
def frobulate(s:String) = {
// Use mockResponse, mockConnection, etc ...
}
}
"My thing" should {
"Do stuff" in new FooScope {
frobulate("hello")
there was one(mockConnection).post(...)
}
}
Scopes help you DRY up your tests as well as stop state leaks between them.
Plus
"Blocks" to group related tests ("My thing" should)
Readable, space-separated test names ("throw an exception if argument < 0" in)
Readable DSL for matching (result must have length(7))
Matchers for Scala-idiomatic types (result must beNone)
Easily temporarily disable a test with pending
Generate HTML reports/output with a DSL
Downsides
In an attempt to make this answer slightly more objective, I should also point out this article by the esteemed Bill Venners about compile time which points out that a Specs2 mutable.Specification, due to large numbers of implicits and function declarations, is almost an order-of-magnitude slower to compile than a roughly-equivalent JUnit test. It also does not scale well over large numbers of tests. Or at least it didn't when the article was written in early 2013. There are now approaches you can use to address this.

eclipse does not treat a scalatest flatspec as junit test

here is my test case , while i right click the file eclipse doest not show any run as junit test option. I try to manual create run configuration but does not take any sense.
scala version:2.8.1 scalatest:1.3 eclipse:3.6.2
package org.jilen.cache.segment
import org.junit.runner.RunWith
import org.scalatest.junit.JUnitRunner
import org.scalatest.FlatSpec
import org.scalatest.matchers.ShouldMatchers
#RunWith(classOf[JUnitRunner])
class RandomSegmentSpec extends FlatSpec with ShouldMatchers {
val option = SegmentOptions()
"A Simple Segment" should "contains (douglas,lea) after put into" in {
val segment = RandomSegment.newSegment(option)
segment.put("douglas", "lea")
segment("douglas") should be("lea")
}
it should "return null after (douglas,lea) is remove" in {
val segment = RandomSegment.newSegment(option)
segment.put("douglas", "lea")
segment -= ("douglas")
segment("douglas") should equal(null)
}
it should "contains nothing after clear" in {
val segment = RandomSegment.newSegment(option)
segment.put("jilen", "zhang")
segment.put(10, "ten")
segment += ("douglas" -> "lea")
segment += ("20" -> 20)
segment.clear()
segment.isEmpty should be(true)
}
}
I've encountered this seemingly randomly, and I think I've finally figured out why.
Unfortunately the plugin doesn't yet change package declarations when you move files, nor the class names when you rename files. (Given you can put multiple classes in one file, the latter will likely never be done.) If you are used to the renamings being done automagically in Eclipse, like I am, you're bound to get caught on this.
So... check carefully the following:
the package declaration in your Scala file matches the Eclipse package name
the name of the test class in the Scala file matches the name of the Scala file
I just ran into this, fixed both, and now my test runs!
This is a known problem with the Eclipse IDE for Scala. I'm currently working on the plugin for this. Watch this space.
I found Scalatest to be very bad at integrating with Eclipse (running the tests from eclipse showed that it ran them - but they would not pass or fail, but simply show up as passive blank boxes).
For some reason I could NOT get it to work after 3 hours of trying things!
Finally I tried specs2 - and it worked (Scala 2.9, Junit4 and Eclipse 3.6)!
They have a great doc here:
http://etorreborre.github.com/specs2/guide/org.specs2.guide.Runners.html#Runners+guide
Since I don't care which testing framework to use, I will try Specs2 purely from the convenience point of view.