Does Scala provide a built-in class, utility, syntax, or other mechanism for converting (by wrapping) an Iterator with an Iterable?
For example, I have an Iterator[Foo] and I need an Iterable[Foo], so currently I am:
val foo1: Iterator[Foo] = ....
val foo2: Iterable[Foo] = new Iterable[Foo] {
def elements = foo1
}
This seems ugly and unnecessary. What's a better way?
Iterator has a toIterable method in Scala 2.8.0, but not in 2.7.7 or earlier. It's not implicit, but you could define your own implicit conversion if you need one.
You should be very careful about ever implicitly converting an Iterator into an Iterable (I normally use Iterator.toList - explicitly). The reason for this is that, by passing the result into a method (or function) which expects an Iterable, you lose control of it to the extent that your program might be broken. Here's one example:
def printTwice(itr : Iterable[String]) : Unit = {
itr.foreach(println(_))
itr.foreach(println(_))
}
If an Iterator were somehow implicitly convertible into an Iterable, what will the following would print?
printTwice(Iterator.single("Hello"))
It will (of course) only print Hello once. Very recently, the trait TraversableOnce has been added to the collections library, which unifies Iterator and Iterable. To my mind, this is arguably a mistake.
My personal preference is to use Iterator explicitly wherever possible and then use List, Set or IndexedSeq directly. I have found that I can rarely write a method which is genuinely agnostic of the type it is passed. One example:
def foo(trades: Iterable[Trade]) {
log.info("Processing %d trades", trades.toList.length) //hmmm, converted to a List
val shorts = trades.filter(_.side.isSellShort)
log.info("Found %d sell-short", shorts.toList.length) //hmmm, converted to a List again
//etc
Related
In scala exercies I have found the following example:
val set = Set(4, 6, 7, 8, 9, 13, 14)
val result = set.toIterable
with the following description:
toIterable will convert any Traversable to an Iterable. This is a base trait for all Scala collections that define an iterator method to iterate through the collection's elements
But Set is already an Iterable, so what's the point of this method? If this isn't the valid case, could you point me one?
In Scala 2.13 there is no more Traversable:
Simpler type hierarchy
No more Traversable and TraversableOnce. They remain only as
deprecated aliases for Iterable and IterableOnce.
Calling toIterable on Set is redundant as it will simply return this same collection:
This collection as an Iterable[A]. No new collection will be built if
this is already an Iterable[A].
Examples where toIterable would have an effect would be
"Hello".toIterable
Array(1).toIterable
which implicitly converts to
wrapString("Hello").toIterable
wrapIntArray(Array(1)).toIterable
and make these Java-like types into Scala collections proper.
In addition to Mario Galic's answer, the other thing it does is change the static type. If you and the compiler knew it was a Set before the call, you don't know afterwards. Though the same can be achieved with a type ascription
val result: Iterable[Int] = set
(and this will work for strings and arrays as well), then you need to write out the type parameter, which may much more complex than Int.
Why would I use it? If i know it's a Set, why would I change the type to Iterable?
it can be in a method which can be overridden and doesn't have to return Set in subclasses:
class Super {
def someValues = {
val set = ... // you want to avoid duplicates
set
}
}
class Sub : Super {
override def someValues = {
List(...) // happens to have duplicates this time
}
doesn't compile, but would if Super#someValues returned set.toIterable (though it's generally good practice to have explicit return types).
It can influence later inferred types:
val arr = Array(set)
arr(0) = List(0, 1, 2, 3)
doesn't compile, but would with Array(set.toIterable).
I just had a look into the cats library in scala, more specifically the State Monad.
As a toy example I wanted to create some logic that splits a potentially large string (the StringBuilder) and returns the split up String and the remaining StringBuilder:
object Splitter {
def apply(maxSize: Int, overlap: Int): State[StringBuilder, String] = State(
builder => {
val splitPoint = math.min(builder.size, maxSize + overlap)
(builder.drop(maxSize), builder.substring(0, splitPoint))
}
)
}
Running one step of the State monad works fine but I wanted to chain all the steps until the StringBuilder eventually is empty:
val stop = State.inspect((s: StringBuilder) => s.isEmpty)
Splitter(3, 2).untilM[Vector](stop).run(new StringBuilder("tarsntiars"))
However, this doesn't work as untilM is a member of the Monad trait and there are no implicit conversions in scope. What works is:
val monad = StateT.catsDataMonadForStateT[Eval, StringBuilder]
monad.untilM[List, String](Splitter(3, 2))(stop).run(new StringBuilder("tarsntiars"))
However, I think the shorter is much more readable so I am wondering why it doesn't work? Why does the usual MonadOps mechanism doesn't work here?
After SI-2712 was fixed, the Unapply workaround was removed from Cats: https://github.com/typelevel/cats/pull/1583. Now you need the -Ypartial-unification compiler flag (assuming you are using Scala 2.11 or 2.12) in order for State to be treated as a Monad.
Scalaz still has the Unapply machinery so your code should work with Scalaz without the compiler flag.
I have an interface defined using a structural type like this:
trait Foo {
def collection: {
def apply(a: Int) : String
def values() : collection.Iterable[String]
}
}
}
I wanted to have one of the implementers of this interface do so using a standard mutable HashMap:
class Bar {
val collection: HashMap[Int, String] = HashMap[Int, String]()
}
It compiles, but at runtime I get a NoSuchMethod exception when referring a Bar instance through a Foo typed variable. Dumping out the object's methods via reflection I see that the HashMap's apply method takes an Object due to type erasure, and there's some crazily renamed generated apply method that does take an int. Is there a way to make generics work with structural types? Note in this particular case I was able to solve my problem using an actual trait instead of a structural type and that is overall much cleaner.
Short answer is that the apply method parameter is causing you grief because it requires some implicit conversions of the parameter (Int => Integer). Implicits are resolved at compile time, the NoSuchMethodException is likely a result of these missing implicits.
Attempt to use the values method and it should work since there are no implicits being used.
I've attempted to find a way to make this example work but have had no success so far.
EDIT
OK, #Drexin brings up a good point re: loss of type safety/surprising results when using implicit converters.
How about a less common conversion, where conflicts with PreDef implicits would not occur? For example, I'm working with JodaTime (great project!) in Scala. In the same controller package object where my implicits are defined, I have a type alias:
type JodaTime = org.joda.time.DateTime
and an implicit that converts JodaTime to Long (for a DAL built on top of ScalaQuery where dates are stored as Long)
implicit def joda2Long(d: JodaTime) = d.getMillis
Here no ambiguity could exist between PreDef and my controller package implicits, and, the controller implicits will not filter into the DAL as that is in a different package scope. So when I do
dao.getHeadlines(articleType, Some(jodaDate))
the implicit conversion to Long is done for me, IMO, safely, and given that date-based queries are used heavily, I save some boilerplate.
Similarly, for str2Int conversions, the controller layer receives servlet URI params as String -> String. There are many cases where the URI then contains numeric strings, so when I filter a route to determine if the String is an Int, I do not want to stringVal.toInt everytime; instead, if the regex passes, let the implicit convert the string value to Int for me. All together it would look like:
implicit def str2Int(s: String) = s.toInt
get( """/([0-9]+)""".r ) {
show(captures(0)) // captures(0) is String
}
def show(id: Int) = {...}
In the above contexts, are these valid use cases for implicit conversions, or is it more, always be explicit? If the latter, then what are valid implicit conversion use cases?
ORIGINAL
In a package object I have some implicit conversions defined, one of them a simple String to Int:
implicit def str2Int(s: String) = s.toInt
Generally this works fine, methods that take an Int param, but receive a String, make the conversion to Int, as do methods where the return type is set to Int, but the actual returned value is a String.
Great, now in some cases the compiler errors with the dreaded ambiguous implicit:
both method augmentString in object Predef of type (x: String)
scala.collection.immutable.StringOps and method str2Int(s: String) Int
are possible conversion functions from java.lang.String to ?{val
toInt: ?}
The case where I know this is happening is when attempting to do manual inline String-to-Int conversions. For example, val i = "10".toInt
My workaround/hack has been to create an asInt helper along with the implicits in the package object: def asInt(i: Int) = i and used as, asInt("10")
So, is implicit best practice implicit (i.e. learn by getting burned), or are there some guidelines to follow so as to not get caught in a trap of one's own making? In other words, should one avoid simple, common implicit conversions and only utilize where the type to convert is unique? (i.e. will never hit ambiguity trap)
Thanks for the feedback, implicits are awesome...when they work as intended ;-)
I think you're mixing two different use cases here.
In the first case, you're using implicit conversions used to hide the arbitrary distinction (or arbitrary-to-you, anyway) between different classes in cases where the functionality is identical. The JodaTime to Long implicit conversion fits in that category; it's probably safe, and very likely a good idea. I would probably use the enrich-my-library pattern instead, and write
class JodaGivesMS(jt: JodaTime) { def ms = jt.getMillis }
implicit def joda_can_give_ms(jt: JodaTime) = new JodaGivesMS(jt)
and use .ms on every call, just to be explicit. The reason is that units matter here (milliseconds are not microseconds are not seconds are not millimeters, but all can be represented as ints), and I'd rather leave some record of what the units are at the interface, in most cases. getMillis is rather a mouthful to type every time, but ms is not too bad. Still, the conversion is reasonable (if well-documented for people who may modify the code in years to come (including you)).
In the second case, however, you're performing an unreliable transformation between one very common type and another. True, you're doing it in only a limited context, but that transformation is still liable to escape and cause problems (either exceptions or types that aren't what you meant). Instead, you should write those handy routines that you need that correctly handle the conversion, and use those everywhere. For example, suppose you have a field that you expect to be "yes", "no", or an integer. You might have something like
val Rint = """(\d+)""".r
s match {
case "yes" => println("Joy!")
case "no" => println("Woe!")
case Rint(i) => println("The magic number is "+i.toInt)
case _ => println("I cannot begin to describe how calamitous this is")
}
But this code is wrong, because "12414321431243".toInt throws an exception, when what you really want is to say that the situation is calamitous. Instead, you should write code that matches properly:
case object Rint {
val Reg = """([-]\d+)""".r
def unapply(s: String): Option[Int] = s match {
case Reg(si) =>
try { Some(si.toInt) }
catch { case nfe: NumberFormatException => None }
case _ => None
}
}
and use this instead. Now instead of performing a risky and implicit conversion from String to Int, when you perform a match it will all be handled properly, both the regex match (to avoid throwing and catching piles of exceptions on bad parses) and the exception handling even if the regex passes.
If you have something that has both a string and an int representation, create a new class and then have implicit conversions to each if you don't want your use of the object (which you know can safely be either) to keep repeating a method call that doesn't really provide any illumination.
I try not to convert anything implicitly just to convert it from one type to another, but only for the pimp my library pattern. It can be a bit confusing, when you pass a String to a function that takes an Int. Also there is a huge loss of type safety. If you would pass a string to a function that takes an Int by mistake the compiler could not detect it, as it assumes you want to do it. So always do type conversion explicitly and only use implicit conversions to extend classes.
edit:
To answer your updated question: For the sake of readability, please use the explicit getMillis. In my eyes valid use cases for implicits are "pimp my library", view/context bounds, type classes, manifests, builders... but not being too lazy to write an explicit call to a method.
The source code of map for Array is:
override def map[B](f: A => B): Array[B] = throw new Error()
But the following works:
val name : Array[String]= new Array(1)
name(0)="Oscar"
val x = name.map { ( s: String ) => s.toUpperCase }
// returns: x: Array[java.lang.String] = Array(OSCAR)
Generally, when you see throw new Error() in the source code of a library class, it represents a point where the compiler is intervening and implementing the method by bridging to a facility of the platform (remember this could be Java or .NET).
The Array SID explains how arrays used to be treated in Scala 2.7.x, and how they have changed in 2.8. The compiler used to magically convert the object to a BoxedArray if you called map.
In 2.8, integration of Arrays into the Scala collections framework is largely handled with use of normal langauges features -- implicit conversions from Array[T] to WrappedArray[T] or ArraySeq[T], depending on the context, and implicit parameters of type Manifest[T] to support creation of arrays of a generic type T. Array indexing, length and update still appear as throw new Error(). Array#map no longer exists, instead you find this on WrappedArray and ArraySeq as a regular method.
UPDATE
If you're interested to know this compiler magic is defined, take a look at Scala 2.8 incarnation of Cleanup.scala.
Looks like it's just dummy code, as Scala arrays are really Java ones.