Go: range receiving only odd number of values from channel - range

I am running this code in the sandbox in http://tour.golang.org/
I thought that once I launched the goroutine that ranges over the channel, all values I would send through would be printed.
package main
import "fmt"
func main() {
c := make(chan int)
go (func(c chan int){
for v := range c {
fmt.Println(v)
}
})(c)
c <- 1
c <- 2
c <- 3
c <- 4
// c <- 5 // uncomment to send more values
// c <- 6
close(c)
}
But if I send an odd number of values (say, 1, 2 and 3), all values get printed.
If I send an even number of values(say, 1, 2, 3 and 4), the last one does not get printed.
It seems that the channel creation line:
c := make(chan int)
Changes the behavior of the range expression when I add a buffer of different sizes:
(I am sending 4 values)
c := make(chan int) // prints 1,2,3
c := make(chan int, 1) // same behavior, prints 1,2,3
c := make(chan int, 2) // prints 1,2
c := make(chan int, 3) // prints 1,2,3
c := make(chan int, 4) // [no output]
c := make(chan int, 5) // [no output]
c := make(chan int, 20) // [no output]
Why is it not receiving the last value when I send an even number?
More in this:
I tested this offline also, compiling under 64 bit Linux.
I changed the program a little:
package main
import (
"fmt"
)
func main() {
c := make(chan int)
cc := make(chan int)
p := func (c chan int){
for v := range c {
fmt.Println(v)
}
}
go p(c)
go p(cc)
c <- 1
c <- 2
c <- 3
c <- 4
// c <- 5
// c <- 6
cc <- 1000
// cc <- 2000
close(c)
close(cc)
}
If I decomment the line cc <- 2000 then everything gets printed. But if I leave it commented out, I only get 1, 2 and 3.
Seems like a timing issue. I thought that the line cc <- 1000 would block the main function until all channels get read.

You are thinking of a close as being more like a send than it is, according to the memory model:
The closing of a channel happens before a receive that returns a zero
value because the channel is closed.
So you are guaranteed that those close statements will complete before their corresponding loops terminate. Since you also know the close statements must happen after the last send on those channels (since they are in the same go-routine) you know that all but the last value sent on them will be guaranteed to print. I think what you were expecting was the close to act more like a send, so that the loop is forced to print its last value. If you replace the close statements with a send of -1, for example, this will guarantee all of the values (except the -1's, possibly) will get printed. Whether or not the -1's get printed is not guaranteed.
Obviously this is a simplification of something, but I think the 'proper' way to write your example would be to use a sync.WaitGroup. It's very easy and is perfect for firing off several go-routines and waiting for them all to complete. Here is your code re-written to use a WaitGroup:
package main
import (
"fmt"
"sync"
)
func main() {
c := make(chan int)
cc := make(chan int)
var wg sync.WaitGroup
p := func(c chan int) {
for v := range c {
fmt.Println(v)
}
wg.Done()
}
wg.Add(2)
go p(c)
go p(cc)
c <- 1
c <- 2
c <- 3
c <- 4
cc <- 1000
cc <- 2000
close(c)
close(cc)
wg.Wait()
}

Writing the last line I think I realised what the problem was.
When main ends all other goroutines end.
The for loop inside the goroutine is not atomic.
The line cc <- 1000 DOES block main, but the range itself unlocks it, and main dies (and kills the goroutine too) not allowing fmt.Println to execute.

Related

Reg of Vec of Valid(Vec

I am continually getting an error due to the use of different sized Vecs. Is there a way I could implement this so that b, c, and d would all take just their respective # of Vec elements from a.
val a = Reg(Vec(10, Valid(Vec(32, UInt(8.W)))))
val b = Reg(Vec(10, Valid(Vec(16, UInt(8.W)))))
val c = Reg(Vec(8, Valid(Vec(24, UInt(8.W)))))
val d = Reg(Vec(7, Valid(Vec(32, UInt(8.W)))))
for (i <- 0 to 10) {
b(i).bits := a(i).bits
}
for (i <- 0 to 8) {
c(i).bits := a(i).bits
}
for (i <- 0 to 7) {
d(i).bits := a(i).bits
}
Here is the error message I am receiving.
Connection between sink (UInt<8>[16](Reg in file)) and source (UInt<8>[32](Reg in file)) failed #: Sink and Source are different length Vecs.
You can use the slice method to take a portion of any Scala Seq (Chisel's Vec extends Seq):
val a = Reg(Vec(10, Valid(Vec(32, UInt(8.W)))))
val b = Reg(Vec(10, Valid(Vec(16, UInt(8.W)))))
for (i <- 0 until 10) {
b(i).bits := a(i).bits.slice(0, 16)
}
This all is very index heavy though, relying on knowing the sizes of your Vecs, we could use the Seq method zip which allows you to iterate on corresponding elements in two Seqs. If one is longer than the other, it just truncates to the shorter one. The below is functionally identical to the for loop with the slice:
for ((bb, aa) <- b zip a) { // I'm using infix notation: b zip a == b.zip(a)
for ((x, y) <- bb.bits zip aa.bits) {
x := y
}
}
Note the lack of any indices, this just works no matter the size, so we could use this in a reusable function:
def nestedConnect(x: Vec[Valid[Vec[UInt]]], y: Vec[Valid[Vec[UInt]]]): Unit = {
for ((xx, yy) <- x zip y) {
for ((l, r) <- xx.bits zip yy.bits) {
l := r
}
}
}
We can then do all of your connects by using this function:
nestedConnect(b, a)
nestedConnect(c, a)
nestedConnect(d, a)
Executable example showing that this works: https://scastie.scala-lang.org/SFh1PratTCWhxHc55VGeHg

How to count the number of iterations in a for comprehension in Scala?

I am using a for comprehension on a stream and I would like to know how many iterations took to get o the final results.
In code:
var count = 0
for {
xs <- xs_generator
x <- xs
count = count + 1 //doesn't work!!
if (x prop)
yield x
}
Is there a way to achieve this?
Edit: If you don't want to return only the first item, but the entire stream of solutions, take a look at the second part.
Edit-2: Shorter version with zipWithIndex appended.
It's not entirely clear what you are attempting to do. To me it seems as if you are trying to find something in a stream of lists, and additionaly save the number of checked elements.
If this is what you want, consider doing something like this:
/** Returns `x` that satisfies predicate `prop`
* as well the the total number of tested `x`s
*/
def findTheX(): (Int, Int) = {
val xs_generator = Stream.from(1).map(a => (1 to a).toList).take(1000)
var count = 0
def prop(x: Int): Boolean = x % 317 == 0
for (xs <- xs_generator; x <- xs) {
count += 1
if (prop(x)) {
return (x, count)
}
}
throw new Exception("No solution exists")
}
println(findTheX())
// prints:
// (317,50403)
Several important points:
Scala's for-comprehension have nothing to do with Python's "yield". Just in case you thought they did: re-read the documentation on for-comprehensions.
There is no built-in syntax for breaking out of for-comprehensions. It's better to wrap it into a function, and then call return. There is also breakable though, but it works with Exceptions.
The function returns the found item and the total count of checked items, therefore the return type is (Int, Int).
The error in the end after the for-comprehension is to ensure that the return type is Nothing <: (Int, Int) instead of Unit, which is not a subtype of (Int, Int).
Think twice when you want to use Stream for such purposes in this way: after generating the first few elements, the Stream holds them in memory. This might lead to "GC-overhead limit exceeded"-errors if the Stream isn't used properly.
Just to emphasize it again: the yield in Scala for-comprehensions is unrelated to Python's yield. Scala has no built-in support for coroutines and generators. You don't need them as often as you might think, but it requires some readjustment.
EDIT
I've re-read your question again. In case that you want an entire stream of solutions together with a counter of how many different xs have been checked, you might use something like that instead:
val xs_generator = Stream.from(1).map(a => (1 to a).toList)
var count = 0
def prop(x: Int): Boolean = x % 317 == 0
val xsWithCounter = for {
xs <- xs_generator;
x <- xs
_ = { count = count + 1 }
if (prop(x))
} yield (x, count)
println(xsWithCounter.take(10).toList)
// prints:
// List(
// (317,50403), (317,50721), (317,51040), (317,51360), (317,51681),
// (317,52003), (317,52326), (317,52650), (317,52975), (317,53301)
// )
Note the _ = { ... } part. There is a limited number of things that can occur in a for-comprehension:
generators (the x <- things)
filters/guards (if-s)
value definitions
Here, we sort-of abuse the value-definition syntax to update the counter. We use the block { counter += 1 } as the right hand side of the assignment. It returns Unit. Since we don't need the result of the block, we use _ as the left hand side of the assignment. In this way, this block is executed once for every x.
EDIT-2
If mutating the counter is not your main goal, you can of course use the zipWithIndex directly:
val xsWithCounter =
xs_generator.flatten.zipWithIndex.filter{x => prop(x._1)}
It gives almost the same result as the previous version, but the indices are shifted by -1 (it's the indices, not the number of tried x-s).

Output in Yield statement is different

for (c <- "Hello"; i <- 0 to 1) yield (c + i).toChar
why the above line yield is different from the below line yield
for (i <- 0 to 1; c <- "Hello") yield (c + i).toChar
Even though (c + i).toChar is same for the above lines and the output should be the same but it is different.
The demonstrates that the order of generators is significant. (Every <- is a "generator".)
The 1st generator is "slower" in that it advances only after the 2nd generator completes a cycle.
The 1st generator also guides the output collection type. If it iterates through a String then the output is a String, if the output elements are still Chars. If it iterates through a Range then the output is an IndexedSeq[]. (According to the Scala docs, Range is "a special case of an indexed sequence.")

Scala error "value map is not a member of Double"

Explanation: I accepted gzm0's answer because it rocked!
#Eduardo did come in with a comment suggesting:
(for(i <- 10..20; j=runTest(i)) yield i -> j).toMap
which also lets me run build, he just never posted an answer and #gzm0 answer was conceptually AWESOME so I accepted it.
Once I get this other issue figured out relating to "can't call constructor" I will be able to test these out by actually running the program LOL
Question: I have an error in this expression, specifically how to fix it but more generally what am I missing about FP or Scala to make this mistake?
timingsMap = for (i <- powersList; j <- runTest(i)) yield i -> j
I am Writing my first Gradle/Scala project for a Analysis of Algorithms assignment. Scala is not part of the assignment so I am not using a homework tag. Except for my work with Spark in Java, I am brand new to Functional Programming I am sure that's the problem.
Here's a snippet, the full .scala file is on GitHub, is that OK or I will post the full program here if I get flailed :)
val powersList = List(10 to 20)
// create a map to keep track of power of two and resulting timings
var timingsMap: Map[Integer, Double] = Map()
// call the runTest function once for each power of two we want, from powersList,
// assign timingsMap the power of 2 value and results of runTest for that tree size
timingsMap = for (i <- powersList; j <- runTest(i)) yield i -> j
The error is:
/home/jim/workspace/Scala/RedBlackTree4150/src/main/scala/Main.scala:36: value map is not a member of Double
timingsMap = for (i <- powersList; j <- runTest(i)) yield i -> j
What I think I am doing in that timingsMap = ... line is get all the elements of powersList mapped onto i for each iteration of the loop, and the return value for runTest(i) mapped onto j for each iteration, and then taking all those pairs and putting them into timingsMap. Is it the way I am trying to use i in the loop to call runTest(i) that causes the problem?
runTest looks like this:
def runTest(powerOfTwo: Range): Double = {
// create the tree here
var tree = new TreeMap[Int, Double]
// we only care to create a tree with integer keys, not what the value is
for (x <- powerOfTwo) {
// set next entry in map to key, random number
tree += (x -> math.random)
}
stopWatchInst.start()
// now go through and look up all the values in the tree,
for (x <- powerOfTwo) {
// get the value, don't take the time/overhead to store it in a var, as we don't need it
tree.get(x)
}
// stop watch check time report and return time
stopWatchInst.stop()
val totalTime = stopWatchInst.elapsed(TimeUnit.MILLISECONDS)
loggerInst.info("run for 2 to the power of " + powerOfTwo + " took " + totalTime + " ms")
return totalTime
}
Note: I've had a suggestions to change the j <- to = in j <- in this line: timingsMap = for (i <- powersList; j <- runTest(i)) yield i -> j
Another suggestion didn't like using yield at all and suggested replacing with (10 to 20).map...
The strange part is the existing code does not show an error in the IntelliJ editor, only breaks when I run it. The suggestions all give type mismatch errors in the IDE. I am really trying to figure out conceptually what I am doing wrong, thanks for any help! (and of course I need to get it to work!)
After trying gzm0 answer I am getting down the same road ... my code as presented doesn't show any type mismatches until I use gradle run ... whereas when I make the suggested changes it starts to give me errors right in the IDE ... but keep em coming! Here's the latest error based on gzm0s answer:
/home/jim/workspace/Scala/RedBlackTree4150/src/main/scala/Main.scala:37: type mismatch;
found : List[(scala.collection.immutable.Range.Inclusive, Double)]
required: Map[Integer,Double]
timingsMap = for (i <- powersList) yield i -> runTest(i)
You want:
for (i <- powersList)
yield i -> runTest(i)
The result of runTest is not a list, therefore you can't give it to the for statement. The reason that you get a bit of a strange error message, is due to how your for is desugared:
for (i <- powersList; j <- runTest(i)) yield i -> j
// turns into
powersList.flatMap { i => runTest(i).map { j => i -> j } }
However, the result of runTest(i) is a Double, which doesn't have a map method. So looking at the desugaring, the error message actually makes sense.
Note that my point about runTest's result not being a list is not really correct: Anything that has a map method that will allow the above statement (i.e. taking some kind of lambda) will do. However, this is beyond the scope of this answer.
We now have successfully created a list of tuples (since powersList is a List, the result of the for loop is a List as well). However, we want a Map. Luckily, you can call toMap on Lists that contain tuples to convert them into a Map:
val tups = for (i <- powersList) yield i -> runTest(i)
timingsMap = tups.toMap
A note aside: If you really want to keep the j inside the for-loop, you can use the equals sign instead of the left arrow:
for (i <- powersList; j = runTest(i)) yield i -> j
This will turn into:
powersList.map { i =>
val j = runTest(i)
i -> j
}
Which is what you want.

How to select just one first or last record compliant to a where clause with ScalaQuery?

Having the following query template to select all:
val q = for {
a <- Parameters[Int]
b <- Parameters[Int]
t <- T if t.a == a && t.b == b
_ <- Query.orderBy(t.c, t.d)
} yield t
I need to modify it to select the very first (with minimum c and d minimum for this c) or the very last (with maximum c and d maximum for this c) record of those matching the where condition. I'd usually strongly prefer no other (than the last/first) records to be selected as there are hundreds thousands of them...
There's a potential danger here in how the OP's query is currently constructed. Run as is, getting the first or last result of a 100K result set is not terribly efficient (unlikely, yes, but the point is, the query places no limit on number of number of rows returned)
With straight SQL you would never do such a thing; instead you would tack on a LIMIT 1
In ScalaQuery, LIMIT = take(n), so add take(1) to get a single record returned from the query itself
val q = (for {
a <- Parameters[Int]
b <- Parameters[Int]
t <- T if t.a == a && t.b == b
_ <- Query.orderBy(t.c, t.d)
} yield t) take(1)
q.firstOption
There is method firstOption defined on the Invoker trait, and by some magic it is available on the Query class. So maybe you can try it like this:
val q = for {
a <- Parameters[Int]
b <- Parameters[Int]
t <- T if t.a == a && t.b == b
_ <- Query.orderBy(t.c, t.d)
} yield t
q.firstOption