I would like to be able to grow an Array-like structure up to a maximum size, after which the oldest (1st) element would be dropped off the structure every time a new element is added. I don't know what the best way to do this is, but one way would be to extend the ArrayBuffer class, and override the += operator so that if the maximum size has been reached, the first element is dropped every time a new one is added. I haven't figured out how to properly extend collections yet. What I have so far is:
class FiniteGrowableArray[A](maxLength:Int) extends scala.collection.mutable.ArrayBuffer {
override def +=(elem:A): <insert some return type here> = {
// append element
if(length > maxLength) remove(0)
<returned collection>
}
}
Can someone suggest a better path and/or help me along this one? NOTE: I will need to arbitrarily access elements within the structure multiple times in between the += operations.
Thanks
As others have discussed, you want a ring buffer. However, you also have to decide if you actually want all of the collections methods or not, and if so, what happens when you filter a ring buffer of maximum size N--does it keep its maximum size, or what?
If you're okay with merely being able to view your ring buffer as part of the collections hierarchy (but don't want to use collections efficiently to generate new ring buffers) then you can just:
class RingBuffer[T: ClassManifest](maxsize: Int) {
private[this] val buffer = new Array[T](maxsize+1)
private[this] var i0,i1 = 0
private[this] def i0up = { i0 += 1; if (i0>=buffer.length) i0 -= buffer.length }
private[this] def i0dn = { i0 -= 1; if (i0<0) i0 += buffer.length }
private[this] def i1up = { i1 += 1; if (i1>=buffer.length) i1 -= buffer.length }
private[this] def i1dn = { i1 -= 1; if (i1<0) i1 += buffer.length }
private[this] def me = this
def apply(i: Int) = {
val j = i+i0
if (j >= buffer.length) buffer(j-buffer.length) else buffer(j)
}
def size = if (i1<i0) buffer.length+i1-i0 else i1-i0
def :+(t: T) = {
buffer(i1) = t
i1up; if (i1==i0) i0up
this
}
def +:(t: T) = {
i0dn; if (i0==i1) i1dn
buffer(i0) = t
this
}
def popt = {
if (i1==i0) throw new java.util.NoSuchElementException
i1dn; buffer(i1)
}
def poph = {
if (i1==i0) throw new java.util.NoSuchElementException
val ans = buffer(i0); i0up; ans
}
def seqView = new IndexedSeq[T] {
def apply(i: Int) = me(i)
def length = me.size
}
}
Now you can use this easily directly, and you can jump out to IndexedSeq when needed:
val r = new RingBuffer[Int](4)
r :+ 7 :+ 9 :+ 2
r.seqView.mkString(" ") // Prints 7 9 2
r.popt // Returns 2
r.poph // Returns 7
r :+ 6 :+ 5 :+ 4 :+ 3
r.seqView.mkString(" ") // Prints 6 5 4 3 -- 7 fell off the end
0 +: 1 +: 2 +: r
r.seqView.mkString(" ") // Prints 0 1 2 6 -- added to front; 3,4,5 fell off
r.seqView.filter(_>1) // Vector(2,6)
and if you want to put things back into a ring buffer, you can
class RingBufferImplicit[T: ClassManifest](ts: Traversable[T]) {
def ring(maxsize: Int) = {
val rb = new RingBuffer[T](maxsize)
ts.foreach(rb :+ _)
rb
}
}
implicit def traversable2ringbuffer[T: ClassManifest](ts: Traversable[T]) = {
new RingBufferImplicit(ts)
}
and then you can do things like
val rr = List(1,2,3,4,5).ring(4)
rr.seqView.mkString(" ") // Prints 2,3,4,5
Related
I want to write a for loop in scala, but the counter should get incremented by more than one (the amount is variable) in some special cases.
You can do this with a combination of a filter and an external var. Here is an example:
var nextValidVal = 0
for (i <- 0 to 99; if i >= nextValidVal) {
var amountToSkip = 0
// Whatever this loop is for
nextValidVal = if (amountToSkip > 0) i + amountToSkip + 1 else nextValidVal
}
So in the main body of your loop, you can set amountToSkip to n according to your conditions. The next n values of i´s sequence will be skipped.
If your sequence is pulled from some other kind of sequence, you could do it like this
var skip = 0
for (o <- someCollection if { val res = skip == 0; skip = if (!res) skip - 1 else 0; res } ) {
// Do stuff
}
If you set skip to a positive value in the body of the loop, the next n elements of the sequence will be skipped.
Of course, this is terribly imperative and side-effecty. I would look for other ways to to this where ever possible, by mapping or filtering or folding the original sequence.
You could implement your own stream to reflect step, for example:
import scala.collection.immutable.Stream
import ForStream._
object Test {
def main(args: Array[String]): Unit = {
val range = 0 to 20 by 1 withVariableStep; // in case you like definition through range
//val range = ForStream(0,20,1) // direct definition
for (i<- range) {
println(s"i=$i")
range.step = range.step + 1
}
}
}
object ForStream{
implicit def toForStream(range: Range): ForStream = new ForStreamMaster(range.start, range.end,range.step)
def apply(head:Int, end:Int, step:Int) = new ForStreamMaster(head, end,step)
}
abstract class ForStream(override val head: Int, val end: Int, var step: Int) extends Stream[Int] {
override val tailDefined = false
override val isEmpty = head > end
def withVariableStep = this
}
class ForStreamMaster(_head: Int, _end: Int, _Step: Int) extends ForStream(_head, _end,_Step){
override def tail = if (isEmpty) Stream.Empty else new ForStreamSlave(head + step, end, step, this)
}
class ForStreamSlave(_head: Int, _end: Int, _step: Int, val master: ForStream) extends ForStream(_head, _end,_step){
override def tail = if (isEmpty) Stream.Empty else new ForStreamSlave(head + master.step, end, master.step, master)
}
This prints:
i=0
i=2
i=5
i=9
i=14
i=20
You can define ForStream from Range with implicits, or define it directly. But be carefull:
You are not iterating Range anymore!
Stream should be immutable, but step is mutable!
Also as #om-nom-nom noted, this might be better implemented with recursion
Why not use the do-while loop?
var x = 0;
do{
...something
if(condition){change x to something else}
else{something else}
x+=1
}while(some condition for x)
I have a LinkedHashMap which I've been using in a typical way: adding new key-value
pairs to the end, and accessing them in order of insertion. However, now I have a
special case where I need to add pairs to the "head" of the map. I think there's
some functionality inside the LinkedHashMap source for doing this, but it has private
accessibility.
I have a solution where I create a new map, add the pair, then add all the old mappings.
In Java syntax:
newMap.put(newKey, newValue)
newMap.putAll(this.map)
this.map = newMap
It works. But the problem here is that I then need to make my main data structure
(this.map) a var rather than a val.
Can anyone think of a nicer solution? Note that I definitely need the fast lookup
functionality provided by a Map collection. The performance of a prepending is not
such a big deal.
More generally, as a Scala developer how hard would you fight to avoid a var
in a case like this, assuming there's no foreseeable need for concurrency?
Would you create your own version of LinkedHashMap? Looks like a hassle frankly.
This will work but is not especially nice either:
import scala.collection.mutable.LinkedHashMap
def prepend[K,V](map: LinkedHashMap[K,V], kv: (K, V)) = {
val copy = map.toMap
map.clear
map += kv
map ++= copy
}
val map = LinkedHashMap('b -> 2)
prepend(map, 'a -> 1)
map == LinkedHashMap('a -> 1, 'b -> 2)
Have you taken a look at the code of LinkedHashMap? The class has a field firstEntry, and just by taking a quick peek at updateLinkedEntries, it should be relatively easy to create a subclass of LinkedHashMap which only adds a new method prepend and updateLinkedEntriesPrepend resulting in the behavior you need, e.g. (not tested):
private def updateLinkedEntriesPrepend(e: Entry) {
if (firstEntry == null) { firstEntry = e; lastEntry = e }
else {
val oldFirstEntry = firstEntry
firstEntry = e
firstEntry.later = oldFirstEntry
oldFirstEntry.earlier = e
}
}
Here is a sample implementation I threw together real quick (that is, not thoroughly tested!):
class MyLinkedHashMap[A, B] extends LinkedHashMap[A,B] {
def prepend(key: A, value: B): Option[B] = {
val e = findEntry(key)
if (e == null) {
val e = new Entry(key, value)
addEntry(e)
updateLinkedEntriesPrepend(e)
None
} else {
// The key already exists, so we might as well call LinkedHashMap#put
put(key, value)
}
}
private def updateLinkedEntriesPrepend(e: Entry) {
if (firstEntry == null) { firstEntry = e; lastEntry = e }
else {
val oldFirstEntry = firstEntry
firstEntry = e
firstEntry.later = oldFirstEntry
oldFirstEntry.earlier = firstEntry
}
}
}
Tested like this:
object Main {
def main(args:Array[String]) {
val x = new MyLinkedHashMap[String, Int]();
x.prepend("foo", 5)
x.prepend("bar", 6)
x.prepend("olol", 12)
x.foreach(x => println("x:" + x._1 + " y: " + x._2 ));
}
}
Which, on Scala 2.9.0 (yeah, need to update) results in
x:olol y: 12
x:bar y: 6
x:foo y: 5
A quick benchmark shows order of magnitude in performance difference between the extended built-in class and the "map rewrite" approach (I used the code from Debilski's answer in "ExternalMethod" and mine in "BuiltIn"):
benchmark length us linear runtime
ExternalMethod 10 1218.44 =
ExternalMethod 100 1250.28 =
ExternalMethod 1000 19453.59 =
ExternalMethod 10000 349297.25 ==============================
BuiltIn 10 3.10 =
BuiltIn 100 2.48 =
BuiltIn 1000 2.38 =
BuiltIn 10000 3.28 =
The benchmark code:
def timeExternalMethod(reps: Int) = {
var r = reps
while(r > 0) {
for(i <- 1 to 100) prepend(map, (i, i))
r -= 1
}
}
def timeBuiltIn(reps: Int) = {
var r = reps
while(r > 0) {
for(i <- 1 to 100) map.prepend(i, i)
r -= 1
}
}
Using a scala benchmarking template.
I'm curious if Scala has some gem hidden in its collection classes that I can use. Basically I'm looking for something like a FIFO queue, but that has an upper-limit on its size such that when the limit is hit, the oldest (first) element is removed from the queue. I've implemented this myself in Java in the past, but I'd rather use something standard if possible.
An often preferable alternative to subclassing is the (unfortunately named) "pimp my library" pattern. You can use it to add an enqueueFinite method to Queue, like so:
import scala.collection.immutable.Queue
class FiniteQueue[A](q: Queue[A]) {
def enqueueFinite[B >: A](elem: B, maxSize: Int): Queue[B] = {
var ret = q.enqueue(elem)
while (ret.size > maxSize) { ret = ret.dequeue._2 }
ret
}
}
implicit def queue2finitequeue[A](q: Queue[A]) = new FiniteQueue[A](q)
Whenever queue2finitequeue is in scope, you can treat Queue objects as though they have the enqueueFinite method:
val maxSize = 3
val q1 = Queue(1, 2, 3)
val q2 = q1.enqueueFinite(5, maxSize)
val q3 = q2.map(_+1)
val q4 = q3.enqueueFinite(7, maxSize)
The advantage of this approach over subclassing is that enqueueFinite is available to all Queues, including those that are constructed via operations like enqueue, map, ++, etc.
Update: As Dylan says in the comments, enqueueFinite needs also to take a parameter for the maximum queue size, and drop elements as necessary. I updated the code.
Why don't you just subclass a FIFO queue? Something like this should work: (pseudocode follows...)
class Limited(limit:Int) extends FIFO {
override def enqueue() = {
if (size >= limit) {
//remove oldest element
}
super.enqueue()
}
}
Here is an immutable solution:
class FixedSizeFifo[T](val limit: Int)
( private val out: List[T], private val in: List[T] )
extends Traversable[T] {
override def size = in.size + out.size
def :+( t: T ) = {
val (nextOut,nextIn) = if (size == limit) {
if( out.nonEmpty) {
( out.tail, t::in )
} else {
( in.reverse.tail, List(t) )
}
} else ( out, t::in )
new FixedSizeFifo( limit )( nextOut, nextIn )
}
private lazy val deq = {
if( out.isEmpty ) {
val revIn = in.reverse
( revIn.head, new FixedSizeFifo( limit )( revIn.tail, List() ) )
} else {
( out.head, new FixedSizeFifo( limit )( out.tail, in ) )
}
}
override lazy val head = deq._1
override lazy val tail = deq._2
def foreach[U]( f: T => U ) = ( out ::: in.reverse ) foreach f
}
object FixedSizeFifo {
def apply[T]( limit: Int ) = new FixedSizeFifo[T]( limit )(List(),List())
}
An example:
val fifo = FixedSizeFifo[Int](3) :+ 1 :+ 2 :+ 3 :+ 4 :+ 5 :+ 6
println( fifo ) //prints: FixedSizeFifo(4, 5, 6)
println( fifo.head ) //prints: 4
println( fifo.tail :+ 7 :+8 ) //prints: FixedSizeFifo(6, 7, 8)
This is the approach I toke with extending Scala's standard mutable.Queue class.
class LimitedQueue[A](maxSize: Int) extends mutable.Queue[A] {
override def +=(elem: A): this.type = {
if (length >= maxSize) dequeue()
appendElem(elem);
this
}
}
And simple use-case
var q2 = new LimitedQueue[Int](2)
q2 += 1
q2 += 2
q2 += 3
q2 += 4
q2 += 5
q2.foreach { n =>
println(n)
}
You'll get only 4 and 5 in the console as the old elements were dequeued beforehand.
Is it possible to have angle brackets in method names , e.g. :
class Foo(ind1:Int,ind2:Int){...}
var v = new Foo(1,2)
v(1) = 3 //updates ind1
v<1> = 4 //updates ind2
The real situation is obviously more complicated than this!!I am trying to provide a convenient user interface.
This response is not meant to be taken too seriously - just a proof that this can almost be achieved using some hacks.
class Vector(values: Int*) {
val data = values.toArray
def < (i:Int) = new {
def `>_=`(x: Int) {
data(i) = x
}
def > {
println("value at "+ i +" is "+ data(i))
}
}
override def toString = data.mkString("<", ", ", ">")
}
val v = new Vector(1, 2, 3)
println(v) // prints <1, 2, 3>
v<1> = 10
println(v) // prints <1, 10, 3>
v<1> // prints: value at 1 is 10
Using this class we can have a vector that uses <> instead of () for "read" and write access.
The compiler (2.9.0.1) crashes if > returns a value. It might be a bug or a result of misusing >.
Edit: I was wrong; kassens's answer shows how to do it as you want.
It is not possible to implement a method that would be called when you write v<1> = 4 (except, maybe, if you write a compiler plugin?). However, something like this would be possible:
class Foo {
def at(i: Int) = new Assigner(i)
class Assigner(i: Int) {
def :=(v: Int) = println("assigning " + v + " at index " + i)
}
}
Then:
val f = new Foo
f at 4 := 6
With a little trickery you can actually get quite close to what you want.
object Foo {
val a:Array[Int] = new Array(100)
def <(i:Int) = new Updater(a, i)
}
class Updater(a:Array[Int], i:Int) {
def update(x:Int) {
a(i) = x
}
def >() = this
}
Foo<1>() = 123
I am not sure why Scala requires the () though. And yes, this is a bit of a hack...
I need to compress/decompress some Data with a old, in house developed Algorithm.
There i have a lot of operations like:
if the next bit is 0 take the following 6 Bits and interpret them as an Int
if the next bits are 10 take the following 9 Bits and interpret them as an Int
etc.
Knows somebody something like a "Bitstrem" class in Scala? (I didn't found anything and hope that i didn't have to implement it by myself.)
Thanks
Edit:
I combined the answer with http://www.scala-lang.org/node/8413 ("The Architecture of Scala Collections") If somebody needs the samething:
abstract class Bit
object Bit {
val fromInt: Int => Bit = Array(Low, High)
val toInt: Bit => Int = Map(Low -> 0, High -> 1)
}
case object High extends Bit
case object Low extends Bit
import collection.IndexedSeqLike
import collection.mutable.{Builder, ArrayBuffer}
import collection.generic.CanBuildFrom
import collection.IndexedSeq
// IndexedSeqLike implements all concrete methods of IndexedSeq
// with newBuilder. (methods like take, filter, drop)
final class BitSeq private (val bits: Array[Int], val length: Int)
extends IndexedSeq[Bit]
with IndexedSeqLike[Bit, BitSeq]
{
import BitSeq._
// Mandatory for IndexedSeqLike
override protected[this] def newBuilder: Builder[Bit, BitSeq] =
BitSeq.newBuilder
//Mandatory for IndexedSeq
def apply(idx: Int): Bit = {
if(idx < 0 || length <= idx)
throw new IndexOutOfBoundsException
Bit.fromInt(bits(idx/N) >> (idx % N) & M)
}
}
object BitSeq {
// Bits per Int
private val N = 32
// Bitmask to isolate a bit
private val M = 0x01
def fromSeq(buf: Seq[Bit]): BitSeq = {
val bits = new Array[Int]((buf.length + N - 1) / N)
for(i <- 0 until buf.length) {
bits(i/N) |= Bit.toInt(buf(i)) << (i % N)
}
new BitSeq(bits, buf.length)
}
def apply(bits: Bit*) = fromSeq(bits)
def newBuilder: Builder[Bit, BitSeq] = new ArrayBuffer mapResult fromSeq
// Needed for map etc. (BitSeq map {:Bit} should return a BitSeq)
implicit def canBuilderFrom: CanBuildFrom[BitSeq, Bit, BitSeq] =
new CanBuildFrom[BitSeq, Bit, BitSeq] {
def apply(): Builder[Bit, BitSeq] = newBuilder
def apply(from: BitSeq): Builder[Bit, BitSeq] = newBuilder
}
}
There isn't any existing class that I'm aware of, but you can leverage the existing classes to help out with almost all of the difficult operations. The trick is to turn your data into a stream of Ints (or Bytes if there wouldn't be enough memory). You then can use all the handy collections methods (e.g. take) and only are left with the problem of turning bits into memory. But that's easy if you pack the bits in MSB order.
object BitExample {
def bitInt(ii: Iterator[Int]): Int = (0 /: ii)((i,b) => (i<<1)|b)
def bitInt(ii: Iterable[Int]): Int = bitInt(ii.iterator)
class ArrayBits(bytes: Array[Byte]) extends Iterator[Int] {
private[this] var buffer = 0
private[this] var index,shift = -1
def hasNext = (shift > 0) || (index+1 < bytes.length)
def next = {
if (shift <= 0) {
index += 1
buffer = bytes(index) & 0xFF
shift = 7
}
else shift -= 1
(buffer >> shift) & 0x1
}
}
}
And then you do things like
import BitExample._
val compressed = new ArrayBits( Array[Byte](14,29,126) ).toStream
val headless = compressed.dropWhile(_ == 0)
val (test,rest) = headless.splitAt(3)
if (bitInt(test) > 4) println(bitInt(rest.take(6)))
(You can decide whether you want to use the iterator directly or as a stream, list, or whatever.)