I have the following working code to solve Advent of Code 2021 day 1 (requires Python 3.10 due to itertools.pairwise), but I am stuck on how to correctly add type hints.
from itertools import pairwise, tee
from typing import Iterator, TypeVar, Tuple, Generator
_T_co = TypeVar("_T_co", covariant=True)
def tripletwise(iterable: Iterator[_T_co]) -> zip[Tuple[_T_co, _T_co, _T_co]]:
"""tripletwise('ABCDEFG') --> ABC BCD CDE DEF EFG"""
a, b, c = tee(iterable, 3)
next(b, None)
next(c, None)
next(c, None)
return zip(a, b, c)
def sonar_sweep(sea_floor_depths: Generator[int, None, None], part: int = 1) -> int:
if part == 1:
return sum(b > a for a, b in pairwise(sea_floor_depths))
return sonar_sweep((a + b + c for a, b, c in tripletwise(sea_floor_depths)), part=1)
def sea_floor_depths(report: str) -> Generator[int, None, None]:
return map(int, report.splitlines())
def main(part: int = 1) -> int:
with open("2021/data/day01.txt") as f:
report = f.read()
return sonar_sweep(sea_floor_depths(report), part)
if __name__ == "__main__":
report = """199
200
208
210
200
207
240
269
260
263"""
assert sonar_sweep(sea_floor_depths(report), part=1) == 7
assert sonar_sweep(sea_floor_depths(report), part=2) == 5
assert list(tripletwise("ABCDEFG")) == [
("A", "B", "C"),
("B", "C", "D"),
("C", "D", "E"),
("D", "E", "F"),
("E", "F", "G"),
]
assert list(tripletwise(sea_floor_depths(report))) == [
(199, 200, 208),
(200, 208, 210),
(208, 210, 200),
(210, 200, 207),
(200, 207, 240),
(207, 240, 269),
(240, 269, 260),
(269, 260, 263),
]
print(main())
print(main(part=2))
Currently, when I run mypy on this, I get:
2021/day01.py:22: error: Incompatible return value type (got "map[int]", expected "Generator[int, None, None]")
2021/day01.py:44: error: Argument 1 to "tripletwise" has incompatible type "str"; expected "Iterator[<nothing>]"
I want to indicate with the type hints that tripletwise() can accept any iterable (for example both the string "ABCDEFG" or the output of sea_floor_depths(report)). I'm also unsure about the correct return type of sea_floor_depths().
As for the first error, the map builtin is only documented to return an iterator, so the most specific type hint that you can use is Iterator[int]. Generator is a subtype of Iterator that only arises when using generator expressions or generator functions, which your function does not.
For the second error, the issue is that strs are not themselves iterators, but rather only iterables, so the appropriate type hint would be Iterable[_T_co].
Related
val list = List("1","10","12","30","40","50")
based on parameter n in eg "12" here , the elements ahead of "12" should form a
list List("30,40,50") and final list should be created as below
Expected Output
List("1","10","12",List("30,40,50") )
list.dropWhile(_!="12").tail gives `List("30,40,50")` but i am not above the achieve the desired output
partition will give the closest output to what you are looking for.
scala> list.partition(_ <= "12")
res21: (List[String], List[String]) = (List(1, 10, 12),List(30, 40, 50))
All elements of List must have the same type. splitAt or partition can accomplish this albeit with a different return type than you want. I suspect that the desired return type List[String, ..., List[String]] is a "type smell" that may indicate a another issue.
Maybe span could help:
val (a, b) = list.span(_ != "12")
val h :: t = b
val res = a :+ h :+ List(t.mkString(","))
produces for input List("123", "10", "12", "30", "40", "50"):
List("123", "10", "12", List("30,40,50"))
If you handle the output type(which is List[java.io.Serializable]), here is the acc method, which takes the desired parameter, a String s in this case :
def acc(list:List[String],s:String) = {
val i = list.indexOf(s)
list.take(i+1):+List(list.drop(i+1).mkString(","))
}
In Scala REPL:
scala> val list = List("1","10","12","30","40","50")
list: List[String] = List(1, 10, 12, 30, 40, 50)
scala> acc(list,"12")
res29: List[java.io.Serializable] = List(1, 10, 12, List(30,40,50))
scala> acc(list,"10")
res30: List[java.io.Serializable] = List(1, 10, List(12,30,40,50))
scala> acc(list,"40")
res31: List[java.io.Serializable] = List(1, 10, 12, 30, 40, List(50))
scala> acc(list,"30")
res32: List[java.io.Serializable] = List(1, 10, 12, 30, List(40,50))
Two newbie questions.
It seems that for comprehension knows about Options and can skip automatically None and unwrap Some, e.g.
val x = Map("a" -> List(1,2,3), "b" -> List(4,5,6), "c" -> List(7,8,9))
val r = for {map_key <- List("WRONG_KEY", "a", "b", "c")
map_value <- x get map_key } yield map_value
outputs:
r: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 6), List(7, 8, 9))
Where do the Options go? Can someone please shed some light on how does this work? Can we always rely on this behaviour?
The second things is why this does not compile?
val x = Map("a" -> List(1,2,3), "b" -> List(4,5,6), "c" -> List(7,8,9))
val r = for {map_key <- List("WRONG_KEY", "a", "b", "c")
map_value <- x get map_key
list_value <- map_value
} yield list_value
It gives
Error:(57, 26) type mismatch;
found : List[Int]
required: Option[?]
list_value <- map_value
^
Looking at the type of the first example, I am not sure why we need to have an Option here?
For comprehensions are converted into calls to sequence of map or flatMap calls. See here
Your for loop is equivalent to
List("WRONG_KEY", "a", "b", "c").flatMap(
map_key => x.get(map_key).flatMap(map_value => map_value)
)
flatMap in Option is defined as
#inline final def flatMap[B](f: A => Option[B]): Option[B]
So it is not allowed to pass List as argument as you are notified by compiler.
I think the difference is due to the way for comprehensions are expanded into map() and flatMap method calls within the Seq trait.
For conciseness, lets define some variables:
var keys = List("WRONG_KEY", a, b, c)
Your first case is equivalent to:
val r = keys.flatMap(x.get(_))
whereas your second case is equivalent to:
val r= keys.flatMap(x.get(_).flatMap{ case y => y })
I think the issue is that Option.flatMap() should return an Option[], which is fine in the first case, but is not consistent in the second case with what the x.get().flatMap is passed, which is a List[Int].
These for-comprehension translation rules are explained in further detail in chapter 7 of "Programming Scala" by Wampler & Payne.
Maybe this small difference, setting parenthesis and calling flatten, makes it clear:
val r = for {map_key <- List("WRONG_KEY", "a", "b", "c")
| } yield x get map_key
r: List[Option[List[Int]]] = List(None, Some(List(1, 2, 3)), Some(List(4, 5, 6)), Some(List(7, 8, 9)))
val r = (for {map_key <- List("WRONG_KEY", "a", "b", "c")
| } yield x get map_key).flatten
r: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 6), List(7, 8, 9))
That's equivalent to:
scala> List("WRONG_KEY", "a", "b", "c").map (x get _)
res81: List[Option[List[Int]]] = List(None, Some(List(1, 2, 3)), Some(List(4, 5, 6)), Some(List(7, 8, 9)))
scala> List("WRONG_KEY", "a", "b", "c").map (x get _).flatten
res82: List[List[Int]] = List(List(1, 2, 3), List(4, 5, 6), List(7, 8, 9))
The intermediate value (map_key) vanished as _ in the second block.
You are mixing up two different monads (List and Option) inside the for statement. This sometimes works as expected, but not always. In any case, you can trasform options into lists yourself:
for {
map_key <- List("WRONG_KEY", "a", "b", "c")
list_value <- x get map_key getOrElse Nil
} yield list_value
Why does the following code prints only 1 and not the rest of the list elements?
scala> val l: List [Int] = List(1,2,3)
l: List[Int] = List(1, 2, 3)
scala> l.toStream.map(x => print(x))
1res3: scala.collection.immutable.Stream[Unit] = Stream((), ?)
What is the correct way to write this code?
I'll divide my answer to two:
1. The map method in Scala:
you're using map, which expects a function with no side-effects (printing is a side effect). What you're looking for is:
l.toStream.foreach(x => print(x))
Basically, the general idea is that map takes something and converts it to something else (for example, increasing its value). while foreach is performing some action on that value that isn't supposed to have a return value.
scala> l.toStream.foreach(x => print(x))
123
2. Stream in Scala:
Streams are lazy, so Scala only computes the values it needs. Try this:
scala> l.toStream.map(x => x+1)
res2: scala.collection.immutable.Stream[Int] = Stream(2, ?)
You can see it computed the first value, and the question marks states that it has no idea what comes after it, because it didn't compute it yet. In you're example the first value is nothing, as the print returns no value.
Stream is on demand data structure which means not all the values will be evaluated until you need them.
example,
scala> val stream = (1 to 10000).toStream
stream: scala.collection.immutable.Stream[Int] = Stream(1, ?)
Now if you access head and tail, stream will be evaluated upto 2nd index.
scala> stream.head
res13: Int = 1
scala> stream.tail
res14: scala.collection.immutable.Stream[Int] = Stream(2, ?)
scala> stream
res15: scala.collection.immutable.Stream[Int] = Stream(1, 2, ?)
If you access index 99,
scala> stream(99)
res16: Int = 100
Now if you print stream, stream will be evaluated upto 99th index,
scala> stream
res17: scala.collection.immutable.Stream[Int] = Stream(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, ?)
It's always good to process only those in stream, which you need. you can use take() for that.
scala> stream.take(50).foreach(x => print(x + " "))
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
So, answer to your question can be,
scala> List(1,2,3).toStream.take(3).foreach(x => print(x + " "))
1 2 3
Reference
https://www.coursera.org/learn/progfun2/home/week/2
to print complete stream use
l.toStream.print
Output: 1, 2, 3, empty
to print first n values, you may use take(n)
l.toStream.take(2).print
prints output: 1, 2, empty
You can print it with
l.toStream.foreach(println)
But generally speaking is not a good idea trying to print or even processing without being careful a whole Stream since it may be infinite and cause an error while doing so.
More info about Streams here
Streams in Scala are lazy data structures which means that they tend to perform only the as needed work.
scala> val stream1 = Stream.range(1, 10)
// stream1: scala.collection.immutable.Stream[Int] = Stream(1, ?)
In this case, only the first element is computed. The stream knows how to compute rest of the elements and will compute them only when it actually needs them. for example ("consumers"),
scala> val list1 = stream1.toList
// list1: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9)
scala> stream1.foreach(print)
// 123456789
But when faced with "transformers", Streams will just keep the new transformation with them but will not apply to whole Stream. The map method is supposed to provide transforms
scala> val stream2 = stream1.map(i => i + 5)
// stream2: scala.collection.immutable.Stream[Int] = Stream(6, ?)
So, it just know that it has to apply this i => i + 5 function to respective elements of stream1 to get element of stream2. And will do that when required (facing any consumer).
Lets consider something similar to your example,
scala> val stream3 = stream1.map(i => println("element :: " + i))
// element :: 1
// stream3: scala.collection.immutable.Stream[Unit] = Stream((), ?)
Here your "transform" function takes an element Int, prints it in line and returns nothing which is called Unit or () in Scala. Out lazy stream here, will compute this transform for first element and will not do for rest. And this computation here will result in that element :: 1 being printed.
Now, lets see what happens when we apply some consumer to it,
scala> val list3 = stream3.toList
// element :: 2
// element :: 3
// element :: 4
// element :: 5
// element :: 6
// element :: 7
// element :: 8
// element :: 9
// list3: List[Unit] = List((), (), (), (), (), (), (), (), ())
Which will look wrong to most people. All I wanted to convert my stream to list but why are all these lines getting printed.
Which is why, when you are using map, you should provide a pure function.
What is a pure function? The simple answer is that a pure function only does the things it is supposed to do and nothing else. It does not cause any change out of its scope. It just takes something and give something else back.
All of the following are pure functions,
scala> val pf1 = (i: Int) => i + 1
// pf1: Int => Int = $$Lambda$1485/1538411140#6fdc53db
scala> val pf2 = (i: Int) => {
| val x = 100
| val xy = 200
| xy + i
| }
// pf2: Int => Int = $$Lambda$1487/14070792#7bf770ba
scala> val pf3 = (i: Int) => ()
// pf3: Int => Unit = $$Lambda$1486/1145379385#336cd7d5
Where as following is not a pure function,
val npf1 = (i: Int) => println(i)
// npf1: Int => Unit = $$Lambda$1488/1736134005#7ac97ba6
Because it causes a "magical" change in the environment.
Why "magical"? Because, it claims to be a function of type Int => Unit which means it should just be transforming an Int to an Unit. But it also printed something on our console, which was outside of its environment.
A real world example of this magic will be that - whenever you put a bread in your toaster it causes a rain storm on the Hulk's current location. And nobody wants the Hulk to come looking for their toaster.
In general, the bottom line is, that you should not use side effects in .map. When you do foo.map(bar) that just returns another collection, that contains element, generated by applying bar to the original collection. It may or may not be lazy. The point is, you should treat the elements of any collection as undefined until something looks at them.
If you want side effects, use foreach: Seq(1,2,3).toStream.foreach(println)
I have the following Array[Int]: val array = Array(1, 2, 3), for which I have the following mapping relation between an Int and a String:
val a1 = array.map{
case 1 => "A"
case 2 => "B"
case 3 => "C"
}
To create a Map to contain the above mapping relation, I am aware that I can use a foldLeft method:
val a2 = array.foldLeft(Map[String, Int]()) { (m, e) =>
m + (e match {
case 1 => ("A", 1)
case 2 => "B" -> 2
case 3 => "C" -> 3
})
}
which outputs:
a2: scala.collection.immutable.Map[String,Int] = Map(A -> 1, B -> 2, C
-> 3)
This is the result I want. But can I achieve the same result via the map method?
The following codes do not work:
val a3 = array.map[(String, Int), Map[String, Int]] {
case 1 => ("A", 1)
case 2 => ("B", 2)
case 3 => ("C", 3)
}
The signature of map is
def map[B, That](f: A => B)
(implicit bf: CanBuildFrom[Repr, B, That]): That
What is this CanBuildFrom[Repr, B, That]? I tried to read Tribulations of CanBuildFrom but don't really understand it. That article mentioned Scala 2.12+ has provided two implementations for map. But how come I didn't find it when I use Scala 2.12.4?
I mostly use Scala 2.11.12.
Call toMap in the end of your expression:
val a3 = array.map {
case 1 => ("A", 1)
case 2 => ("B", 2)
case 3 => ("C", 3)
}.toMap
I'll first define your function here for the sake of brevity in later explanation:
// worth noting that this function is effectively partial
// i.e. will throw a `MatchError` if n is not in (1, 2, 3)
def toPairs(n: Int): (String, Int) =
n match {
case 1 => "a" -> 1
case 2 => "b" -> 2
case 3 => "c" -> 3
}
One possible way to go (as already highlighted in another answer) is to use toMap, which only works on collection of pairs:
val ns = Array(1, 2, 3)
ns.toMap // doesn't compile
ns.map(toPairs).toMap // does what you want
It is worth noting however that unless you are working with a lazy representation (like an Iterator or a Stream) this will result in two passes over the collection and the creation of unnecessary intermediate collections: the first time by mapping toPairs over the collection and then by turning the whole collection from a collection of pairs to a Map (with toMap).
You can see it clearly in the implementation of toMap.
As suggested in the read you already linked in the answer (and in particular here) You can avoid this double pass in two ways:
you can leverage scala.collection.breakOut, an implementation of CanBuildFrom that you can give map (among others) to change the target collection, provided that you explicitly provide a type hint for the compiler:
val resultMap: Map[String, Int] = ns.map(toPairs)(collection.breakOut)
val resultSet: Set[(String, Int)] = ns.map(toPairs)(collection.breakOut)
otherwise, you can create a view over your collection, which puts it in the lazy wrapper that you need for the operation to not result in a double pass
ns.view.map(toPairs).toMap
You can read more about implicit builder providers and views in this Q&A.
Basically toMap (credits to Sergey Lagutin) is the right answer.
You could actually make the code a bit more compact though:
val a1 = array.map { i => ((i + 64).toChar, i) }.toMap
If you run this code:
val array = Array(1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 0)
val a1 = array.map { i => ((i + 64).toChar, i) }.toMap
println(a1)
You will see this on the console:
Map(E -> 5, J -> 10, F -> 6, A -> 1, # -> 0, G -> 7, L -> 12, B -> 2, C -> 3, H -> 8, K -> 11, D -> 4)
I am looking for a function that looks like collect. This function must keep the element that do not satisfy the predicate.
This behaviour be could expressed using map. Example :
# (0 to 10).map{
case e if e > 5 => e * e
case e => e // I want to keep elements to does not satisfy the predicate !
}
res3: collection.immutable.IndexedSeq[Int] = Vector(0, 1, 2, 3, 4, 5, 36, 49, 64, 81, 100)
I would like to be able to write this function like this :
# (0 to 10).map{
case e if e > 5 => e * e
}
scala.MatchError: 0 (of class java.lang.Integer)
$sess.cmd4$$anonfun$1.apply$mcII$sp(cmd4.sc:1)
$sess.cmd4$$anonfun$1.apply(cmd4.sc:1)
$sess.cmd4$$anonfun$1.apply(cmd4.sc:1)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
scala.collection.immutable.Range.foreach(Range.scala:160)
scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
scala.collection.AbstractTraversable.map(Traversable.scala:104)
$sess.cmd4$.<init>(cmd4.sc:1)
$sess.cmd4$.<clinit>(cmd4.sc:-1)
Unfortunately, I have not found a function that takes a PartialFunction to avoid MatchErrors.
Do you know a function with such a behaviour ?
I don't think there is a pre-defined method that does what you want, but you can use a PartialFunction's .applyOrElse to do it:
scala> implicit class MySeq[A](r: Seq[A]) {
def mapIfDefined(f: PartialFunction[A, A]): Seq[A] = {
r.map(f.applyOrElse[A, A](_, identity))
}
}
scala> (0 to 10).mapIfDefined{
case e if e > 5 => e * e
}
res1: Seq[Int] = Vector(0, 1, 2, 3, 4, 5, 36, 49, 64, 81, 100)
Unless I'm mistaken, collect does exactly what you're asking:
scala> List(1,2,3,4,5,6,7,8,9,10).collect {
| case e if e > 5 => e * e
| case e => e
| }
res0: List[Int] = List(1, 2, 3, 4, 5, 36, 49, 64, 81, 100)