I am looking for tests which can be used for custom collections. Ideally these are behaviour tests.
For example implementing a new Map I would like to test if it follows all required Map-rules and
methods like map, filter, view, etc.
What is Scala using to test its own collections?
This is a good question that has been asked before on SO.
There are some collections tests under test/files/scalacheck and others under test/files/run/*coll* in the source repository.
There is no conformance test or TCK per se for custom collections. Integration with collections usually involves a specific implementation requirement.
For example, the ScalaDoc for immutable.MapLike tells you to implement get, iterator and + and -. In theory, if you test the template methods, you can rely on everything you get for free from the library.
But the doc adds:
It is also good idea to override methods foreach and size for
efficiency.
So if you care about that, you'll be adding performance tests too. The standard library doesn't include automated performance testing.
Related
This question is formulated in Scala 3/Dotty but should be generalised to any language NOT in MetaML family.
The Scala 3 macro tutorial:
https://docs.scala-lang.org/scala3/reference/metaprogramming/macros.html
Starts with the The Phase Consistency Principle, which explicitly stated that free variables defined in a compilation stage CANNOT be used by the next stage, because its binding object cannot be persisted to a different compiler process:
... Hence, the result of the program will need to persist the program state itself as one of its parts. We don’t want to do this, hence this situation should be made illegal
This should be considered a solved problem given that many distributed computing frameworks demands the similar capability to persist objects across multiple computers, the most common kind of solution (as observed in Apache Spark) uses standard serialisation/pickling to create snapshots of the binded objects (Java standard serialization, twitter Kryo/Chill) which can be saved on disk/off-heap memory or send over the network.
The tutorial itself also suggested the possibility twice:
One difference is that MetaML does not have an equivalent of the PCP - quoted code in MetaML can access variables in its immediately enclosing environment, with some restrictions and caveats since such accesses involve serialization. However, this does not constitute a fundamental gain in expressiveness.
In the end, ToExpr resembles very much a serialization framework
Instead, Both Scala 2 & Scala 3 (and their respective ecosystem) largely ignores these out-of-the-box solutions, and only provide default methods for primitive types (Liftable in scala2, ToExpr in scala3). In addition, existing libraries that use macro relies heavily on manual definition of quasiquotes/quotes for this trivial task, making source much longer and harder to maintain, while not making anything faster (as JVM object serialisation is an highly-optimised language component)
What's the cause of this status quo? How do we improve it?
Does anybody know how to get list of categories (provided with 'where' filter to nunit-console) at runtime?
Depending on this, I need to differently initialize the test assembly.
Is there something static like TestExecutionContext that may contain such information?
The engine doesn't pass information on to the framework about "why" it's running a particular test... i.e. if it's running all tests or if it was selected by name or category. That's deliberately kept as something the test doesn't know about with the underlying philosophy being that tests should just run based on the data provided to them.
On some platforms, it's possible to get the command-line, which ran the test. With that info you could decode the various options and make some conclusions but it seems as if it would be easier to restructure the tests so they didn't need this information.
As a secondary reason, it would also be somewhat complicated to supply the info you want and to use it. A test may have multiple categories. Imagine a test selected because two categories matched, for example!
Is it possible that what you really want to do is to pass some parameters to your tests? There is a facility for doing that of course.
I think this is a bit of an XY problem. Depending on what you are actually trying to accomplish, the best approach is likely to be different. Can you edit to tell us what you are trying to do?
UPDATE:
Based on your comment, I gather that some of your initialization is both time-consuming and not needed unless certain tests are run.
Two approaches to this (or combine them):
Do less work in all your initialization (i.e. TestCase, TestCaseSource, SetUpFixture. It's generally best not to create your classes under test or initialize databases. Instead, simply leave strings, ints, etc., which allow the actual test to do the work IFF it is run.
2.Use a SetUpFixture in some namespace containing all the tests, which require that particular initialization. If you dont' run any tests from that namespace, then the initialization won't be done.
Of course both of the above may entail a large refactoring of your tests, but the legacy app won't have to be changed.
Is it possible to set different rules for main versus test code in Codacy? I know I can eliminate inspection of test code. But I don't want to do that. However, there are many rules, especially regarding duplication, that just don't make sense for tests.
My main language is Scala.
No. The best you can do is ignore folders (for instance the test folder).
We typically relax on test code, but it makes sense to avoid duplication on test code as well. Your (real) code will evolve over time, and eventually will make you need to change tests. Why change in 100 places instead of a single method that is shared among several tests?
I am taking course Functional Programming Principles in Scala | Coursera on Scala.
I fail to understand with immutability , so many functions and so much dependencies on recursion , how is Scala is really suitable for real world applications.
I mean coming from imperative languages I see a risk of StackOverflow or Garbage Collection kicking in and with multiple copies of everything I am running Out Of Memory
What I a missing here?
Stack overflow: it's possible to make your recursive function tail recursive. Add #tailrec from scala.annotation.tailrec to make sure your function is 100% tail recursive. This is basically a loop.
Most importantly recursive solutions is only one of many patterns available. See "Effective Java" why mutability is bad. Immutable data is much better suitable for large applications: no need to synchronize access, client can't mess with data internals, etc. Immutable structures are very efficient in many cases. If you add an element to the head of a list: elem :: list all data is shared between 2 lists - awesome! Only head is created and pointed to the list. Imagine that you have to create a new deep clone of a list every time client asks for.
Expressions in Scala are more succinct and maybe more lazy - create filter and map and all that applied as needed. You can do the same in Java but ceremony takes forever so usually devs just create multiple temp collections on the way.
Martin Odersky defines mutability as a dependence on time/history. That's very interesting because you can use var inside of a function as long as no other code can be affected in any way, i.e. results are always the same.
Look at Option[T] and compare to null. Use them in for comprehensions. Exception becomes really exceptional and Option, Try, Box, Either communicate failures in a very nice way.
Scala allows to write more modular and generic code with less effort compared to Java.
Find a good piece of Scala code and try to see how you would do it in Java - it will be self evident.
Real world applications are getting more event-driven which involves passing around data across different processes or systems needing immutable data structures
In most of the cases we are either manipulating data or waiting on a resource.
In that case its easy to hook in a callback with Actors
Take a look at
http://pavelfatin.com/scala-for-project-euler/
Which gives you some examples on using functions like map fllter etc. Functions like these are used routinely by Ruby applications
Combination of immutability and recursion avoids a lot of stackoverflow problems. This come in handly while dealing with event driven applications
akka.io is a classic example which could have been build very concisely in scala.
I am hoping that Combinator parsers, (http://debasishg.blogspot.com/2008/04/external-dsls-made-easy-with-scala.html), will work for a design to process the routing rules for a REST service that is implemented with Scalatra,(http://tutorialbin.com/tutorials/80408/infoq-scalatra-a-sinatra-like-web-framework-for-scala).
This REST service is to serve as a proxy so external applications can get access to services within the firewall, as it will have additional layers of security that can be customized for the business requirements of each REST service.
So, if a person wants to access their class schedule there will be less security than if you want to look at the transcript of someone.
I would like the rules for where to go to actually get the information, and how to return it, as well as what security is needed, in a DSL.
But, the first problem is how to dynamically change the routing rules for the REST service based on a DSL, as I am trying to create a framework that doesn't require a great deal of recompiling to add new rules, but just write the appropriate scripts and just let it be processed.
So, can a DSL be implemented using the Combinator Parser, in Scala, that will allow JAX-RS (http://download.oracle.com/javaee/6/tutorial/doc/giepu.html) to have dynamically changed routing?
UPDATE:
I haven't designed the language yet, but this is what I am trying to do:
route /transcript using action GET to
http://inside.com/transcript/{firstparam}/2011/{secondparam}
return json encrypt with public key from /mnt/publickey.txt
for /education_cost using action GET combine http://combine.com/SOAP/costeducate with
http://combine.com/education_benefit/2010 with
http://combine.com/education_benefit/2011 return html
These are two possible ideas where rules for a request for a transcript is sent to a different site, such as within a firewall, and the data is encrypted and returned.
The second would be more complicated in that the results of a SOAP and two REST requests will be combined, and there would need to be additional commands on how this is combined, but the idea is to put all of this in files that can be parsed on the fly.
If I used Groovy then some new classes could be generated for the routing, which would remove some performance hits, but I think using Scala would be the best bet, even if I took a performance hit.
My hope is to make a framework that is more maintainable so new routing rules can be written by people that don't know any OOP or functional languages, but the specifications could be written using Specs (http://code.google.com/p/specs/) so that the functional side could be certain that their requirements are tested on a regular basis.
UPDATE 2:
When I start working on a design I may intuitively understand some options, but not know why. Today I realized that the reason that Groovy may be a better fix for this is that I could then generate the classes for routing, using the metaprogramming (http://www.justinspradlin.com/programming/groovy-metaprogramming-adding-behavior-dynamically/), then I would be able to use Scala or Groovy to dynamically use the routing that was generated. I am not certain how to get Scala to generate the classes if they don't already exist.
In Groovy, as well as some other languages, as shown here (http://langexplr.blogspot.com/2008/02/handling-call-to-missing-method-in.html) if a method is missing you can dynamically generate the method and it will henceforth exist, so it will be missing one time.
It almost seems that I should be mixing Groovy with Java to make this work, but then the result may be that some of the code is in Scala and some in Java, for the routing of REST services.
Splitting the question in two parts:
can a DSL be implemented using the Combinator Parser
Yes. There are things that cannot be implemented using a combinator parser, or even other kinds of parser. For instance, Perl itself cannot be parsed (it must be evaluated). And combinator parsers are also not particularly good for complex languages (such as Scala -- its compiler is not based on combinator parsers), or if you demand top performance (such as the compilers used to compile hundreds of thousands of lines of code).
If, however, you plan to go to such extremes, choosing the parser is not going to be your main problem. For DSLs of average complexity, they'll do just fine.
that will allow JAX-RS to have dynamically changed routing
Well, I don't know JAX-RS, but if dynamically changed routing can be done with it, then combinators parsers will be able to provide whatever input is needed.
EDIT
Seeing your example, I think parser combinators are certainly enough. From their results, I expect you could dynamically create BlueEyes binders -- I haven't used BlueEyes, so I'm not sure how dynamic they are.
Another alternative would be go with Lift. Lift's binders are partial functions, and they can be combined in all the usual ways -- f1 orElse f2, f1 andThen f2, etc. I didn't suggest it at first because it is most often used with sessions, but it has a RESTful model which, I think, is stateless.
I don't know Scalatra, so I don't know if it would be adaptable to this or not.