When serializing functions in Scala, we should take care not to capture fields that are not serializable.
class Foo(s: Set[Int] = Set(1,2,3)) {
def replicate(): () => Foo = () => new Foo(s)
}
val foo = new Foo()
val fn = foo.replicate()
serialize(fn) // java.io.NotSerializableException: Foo
In the above code snippet serializing msg will fail because the function captures this and Foo is not serializable.
To avoid capturing this SIP-21 proposes spores for Scala.
Unfortunately, spores only exist for Scala 2.11.x (see this note).
Luckily, I found in this tread (this is also described in SIP-21) that one can avoid capturing the this reference, by manually creating local values as shown in the code snippet below.
class Foo(s: Set[Int] = Set(1,2,3)) {
def replicate(): () => Foo = {
() => {
{
val localS = this.s
new Foo(localS)
}
}
}
}
Unfortunately, in Scala 2.13.1 this does not work and still captures this (try out the complete code snippet below).
Hence, we cannot serialize the function returned by the replicate method.
Now, I found the following workaround, but it's a rather ugly one.
class Foo(s: Set[Int] = Set(1,2,3)) {
def indirection() = replicate(s)
def replicate(set: Set[Int]): () => Foo = () => new Foo(set)
}
val foo = new Foo()
val msg = foo.indirection()
toStr(msg) // Works :)
By adding an extra indirection, we avoid capturing this and we can serialize the function without requiring Foo to be serializable. However, I don't want to introduce these extra indirections in all the objects I want to serialize this way. How can I avoid this (in Scala 2.13.1) ?
The complete code snippet:
// Start writing your ScalaFiddle code here
import java.io._
import java.util.Base64
def serialize(o: Any) = {
val baos = new ByteArrayOutputStream();
val oos = new ObjectOutputStream( baos );
oos.writeObject( o );
oos.close();
Base64.getEncoder().encodeToString(baos.toByteArray())
}
def deserialize(s: String) = {
val data = Base64.getDecoder().decode( s );
val ois = new ObjectInputStream(new ByteArrayInputStream( data ) );
val o = ois.readObject();
ois.close();
o
}
//////////// TRY 1
try {
class Foo(val s: Set[Int] = Set(1,2,3)) {
def replicate(): () => Foo = () => new Foo(s)
}
val foo = new Foo()
val msg = foo.replicate()
val str = serialize(msg)
val reconstructedFoo = deserialize(str).asInstanceOf[() => Foo]()
println(s"Try 1: success, reconstructedFoo.s = ${reconstructedFoo.s}")
} catch {
case e: NotSerializableException => println("TRY 1: Not serializable.")
}
//////////// TRY 2
try {
class Foo(val s: Set[Int] = Set(1,2,3)) {
def replicate(): () => Foo = {
() => {
{
val localS = this.s
new Foo(localS)
}
}
}
}
val foo = new Foo()
val msg = foo.replicate()
val str = serialize(msg)
val reconstructedFoo = deserialize(str).asInstanceOf[() => Foo]()
println(s"Try 2: success, reconstructedFoo.s = ${reconstructedFoo.s}")
} catch {
case e: NotSerializableException => println("TRY 2: Not serializable.")
}
//////////// Solution (ugly)
try {
class Foo(val s: Set[Int] = Set(1,2,3)) {
def indirection() = replicate(s)
def replicate(set: Set[Int]): () => Foo = () => new Foo(set)
}
val foo = new Foo()
val msg = foo.indirection()
val str = serialize(msg)
val reconstructedFoo = deserialize(str).asInstanceOf[() => Foo]()
println(s"Try 3: success, reconstructedFoo.s = ${reconstructedFoo.s}")
} catch {
case e: NotSerializableException => println("TRY 3: Not serializable.")
}
I am trying to do the following...
I have a class defined as RefInt and a method refInt5 which should take a List of references to instances of RefInt.
I need to change the argument in such a way so that it becomes visible to the caller.
object argpass {
class RefInt (initial : Int) {
private var n : Int = initial
def get () : Int = n
def set (m : Int) : Unit = { n = m }
}
def refint5 (xs : List[RefInt]) : Unit = {
// What am I missing to modify RefInt(s)?
}
}
The test code I have to see if the argument has changed.
property ("EX05 - refint5", EX (5)) {
val rand = scala.util.Random
val xs : List[Int] = (1 to 5).toList.map (x => rand.nextInt (100))
val ys : List[RefInt] = xs.map (x => new RefInt (x))
refint5 (ys)
var changed = false
for ((x, r) <- xs.zip (ys)) {
if (x != r.get) {
changed = true
}
}
if (!changed) {
throw new RuntimeException ("xs is unchanged!")
}
}
}
One possible option: Iterate xs, call set for each element - get the current value - add 1.
def refint5(xs: List[RefInt]): Unit = {
xs.foreach { x => x.set(x.get + 1) }
}
Is there a native way to make sure that a variable can only be set once?
Currently, I am using this approach
class SetOnceVariable[T]
{
private var value : T = _
private var initialized = false
def apply(_v : T = _) : T =
{
if (value != null && !initialized) {
value = _v
initialized = true
}
value
}
}
class ClientInfo
{
val userIP : SetOnceVariable[String] = new SetOnceVariable[String]
}
There's no such language construct, but I think I can clean up your code, at least.
class SetOnce[A](var toOption: Option[A] = None) {
def set(a: A): Unit = if (toOption.isEmpty) toOption = Some(a)
def get: A = toOption.get
}
Usage:
val x = new SetOnce[Int]
x.toOption // None
x.set(1)
x.get // 1
x.set(2)
x.get // 1
I omitted the null consideration because idiomatic Scala code tends to not use or consider null outside of Java compatibility. We mostly pretend that it doesn't exist.
Approach using lazy:
class SetOnceVariable[T] {
private var v: T = _
private lazy val value: T = v
def apply(_v: T = ???): T = {
v = _v
value
}
}
val info = new ClientInfo
println(info.userIP("IP")) // IP
println(info.userIP("IP2")) // IP
println(info.userIP("IP3")) // IP
To make it threadsafe you can use:
def apply(_v: T = ???): T =
synchronized {
v = _v
value
}
You can create a constant variable by using a val. For instance:
val a = 0; // Cannot be changed
var b = 0; // Can be changed
See this answer for more details: https://stackoverflow.com/a/1792207/4380308
Edit:
A val can be declared and then initialized later as well.
val a;
a = 0;
You can try the following:
class varSingleton[A >: Null <: AnyRef] {
private[this] var _a = null: A
def :=(a: A) { if (_a eq null) _a = a else throw new IllegalStateException }
def apply() = if (_a eq null) throw new IllegalStateException else _a
}
You can use this further like:
var singleVal = new varSingleton[Integer]
singleVal := 12
singleVal() // returns 12
singleVal := 13 //IllegalStateException
you can use simple getter and setter:
class ClientInfo {
private var _userIP: Option[String] = None
def userIP: String = _userIP.get
def userIP_=(value: String): Unit = {
_userIP = _userIP.orElse(Option(value))
}
}
val clientInfo = new ClientInfo() //> clientInfo : controllers.stackoverflow.ClientInfo controllers.stackoverflow$Clien
//| tInfo#4fccd51b
clientInfo.userIP = "first"
clientInfo.userIP //> res0: String = first
clientInfo.userIP = "second"
clientInfo.userIP //> res1: String = first
I prefer to use Option that the value directly to prevent nulls and NPE. You can of course add what ever logic you need in the setter.
I want to be able to do something like this:
prepare form:
val formDescription = formBuilder(_.textField[User](_.firstName)
.textField[User](_.lastName)
).build
showForm(formDescription)
extract data from user filled form, using User:
//contains data of a form submitted by a user:
val formData: Map[String, String] = getFormData
val newUser = User(id = randomUuid, firstName = formData.extract[User](_.firstName))
One solution I see is to use a dynamic proxy that extends provided class and remembers what was invoked on him:
def getFieldName[T:Manifest](foo: T => Any) = {
val clazz = implicitly[Manifest[T]].erasure
val proxy = createDynamicProxy(clazz)
foo(proxy)
proxy.lastInvokedMethodName
}
Is there a better way to do it? Is there any lib that implements it already?
This reflective approach takes a case class and invokes its companion apply, calling getField and fetching default args if the field is not in the data.
import scala.reflect.runtime.{currentMirror => cm, universe => uni}
import uni._
def fromXML(xml: Node): Option[PluginDescription] = {
def extract[A]()(implicit tt: TypeTag[A]): Option[A] = {
// extract one field
def getField(field: String): Option[String] = {
val text = (xml \\ field).text.trim
if (text == "") None else Some(text)
}
val apply = uni.newTermName("apply")
val module = uni.typeOf[A].typeSymbol.companionSymbol.asModule
val ts = module.moduleClass.typeSignature
val m = (ts member apply).asMethod
val im = cm reflect (cm reflectModule module).instance
val mm = im reflectMethod m
def getDefault(i: Int): Option[Any] = {
val n = uni.newTermName("apply$default$" + (i+1))
val m = ts member n
if (m == NoSymbol) None
else Some((im reflectMethod m.asMethod)())
}
def extractArgs(pss: List[List[Symbol]]): List[Option[Any]] =
pss.flatten.zipWithIndex map (p => getField(p._1.name.encoded) orElse getDefault(p._2))
val args = extractArgs(m.paramss)
if (args exists (!_.isDefined)) None
else Some(mm(args.flatten: _*).asInstanceOf[A])
}
// check the top-level tag
xml match {
case <plugin>{_*}</plugin> => extract[PluginDescription]()
case _ => None
}
}
The idea was to do something like:
case class User(id: Int = randomUuid, firstName: String, lastName: String)
val user = extract[User]()
That's my own solution:
package utils
import javassist.util.proxy.{MethodHandler, MethodFilter, ProxyFactory}
import org.specs2.mutable._
import javassist.util.proxy.Proxy
import java.lang.reflect.{Constructor, Method}
class DynamicProxyTest extends Specification with MemberNameGetter {
"Dynamic proxy" should {
"extract field name" in {
memberName[TestClass](_.a) must ===("a")
memberName[TestClass](_.i) must ===("i")
memberName[TestClass](_.b) must ===("b")
memberName[TestClass](_.variable) must ===("variable")
memberName[TestClass](_.value) must ===("value")
memberName[TestClass](_.method) must ===("method")
}
}
}
trait MemberNameGetter {
def memberName[T: Manifest](foo: T => Any) = {
val mf = manifest[T]
val clazz = mf.erasure
val proxyFactory = new ProxyFactory
proxyFactory.setSuperclass(clazz)
proxyFactory.setFilter(new MethodFilter {
def isHandled(p1: Method) = true
})
val newClass = proxyFactory.createClass()
var lastInvokedMethod: String = null
val mh = new MethodHandler {
def invoke(p1: Any, p2: Method, p3: Method, p4: Array[AnyRef]) = {
lastInvokedMethod = p2.getName
p3.invoke(p1, p4: _*)
}
}
val constructor = defaultConstructor(newClass)
val parameters = defaultConstructorParameters(constructor)
// val proxy = constructor.newInstance("dsf", new Integer(0))
val proxy2 = constructor.newInstance(parameters: _*)
proxy2.asInstanceOf[Proxy].setHandler(mh)
foo(proxy2.asInstanceOf[T])
lastInvokedMethod
}
private def defaultConstructor(c: Class[_]) = c.getConstructors.head
private def defaultConstructorParameters(constructor: Constructor[_]) = {
val parameterTypes = constructor.getParameterTypes
parameterTypes.map{
case Integer.TYPE => Integer.valueOf(0)
case _ => null
}
}
}
case class TestClass(a: String, i: Int, b: Boolean) {
var variable = "asdf"
val value = "asdfasdfasd"
def method = "method"
}
val mh = new MethodHandler {
def invoke(p1: Any, p2: Method, p3: Method, p4: Array[AnyRef]) = {
lastInvokedMethod = p2.getName
p3.invoke(p1, p4: _*)
}
}
val constructor = defaultConstructor(newClass)
val parameters = defaultConstructorParameters(constructor)
// val proxy = constructor.newInstance("dsf", new Integer(0))
val proxy2 = constructor.newInstance(parameters: _*)
proxy2.asInstanceOf[Proxy].setHandler(mh)
foo(proxy2.asInstanceOf[T])
lastInvokedMethod
}
private def defaultConstructor(c: Class[_]) = c.getConstructors.head
private def defaultConstructorParameters(constructor: Constructor[_]) = {
val parameterTypes = constructor.getParameterTypes
parameterTypes.map{
case Integer.TYPE => Integer.valueOf(0)
case java.lang.Double.TYPE => java.lang.Double.valueOf(0)
case java.lang.Long.TYPE => java.lang.Long.valueOf(0)
case java.lang.Boolean.TYPE => java.lang.Boolean.FALSE
case _ => null
}
}
}
case class TestClass(a: String, i: Int, b: Boolean) {
var variable = "asdf"
val value = "asdfasdfasd"
def method = "method"
}
I'm using some Java code to do fast prefix lookups, using java.util.TreeSet, could I be using scala's TreeSet instead? Or a different solution?
/** A class that uses a TreeSet to do fast prefix matching
*/
class PrefixMatcher {
private val _set = new java.util.TreeSet[String]
def add(s: String) = _set.add(s)
def findMatches(prefix: String): List[String] = {
val matches = new ListBuffer[String]
val tailSet = _set.tailSet(prefix)
for ( tail <- tailSet.toArray ) {
val tailString = tail.asInstanceOf[String]
if ( tailString.startsWith(prefix) )
matches += tailString
else
return matches.toList
}
matches.toList
}
}
Use a Trie. Nobody's actually posted a Trie here yet, despite the fact that some people have posted sorted TreeMap data structures that they have misnamed as tries. Here is a fairly representative sample of a Trie implementation in Java. I don't know of any in Scala. See also an explanation of Tries on Wikipedia.
The from & takeWhile approach:
class PrefixMatcher {
private val _set = new TreeSet[String]
def add(s: String) = _set.add(s)
def findMatches(prefix: String): Iterable[String] =
_set from prefix takeWhile(_ startsWith prefix)
}
An alternative is to select a subset from prefix to prefix++ (the smallest string after the prefix). This selects only the range of the tree that actually starts with the given prefix. Filtering of entries is not necessary. The subSet method will create a view of the underlying set.
There's still some work (overflow and empty strings won't work) in the increment method but the intent should be clear.
class PrefixMatcher {
private val _set = new java.util.TreeSet[String]
def add(s: String) = _set.add(s)
def findMatches(prefix: String) : Set[String] = {
def inc(x : String) = { //ignores overflow
assert(x.length > 0)
val last = x.length - 1
(x take last) + (x(last) + 1).asInstanceOf[Char]
}
_set.subSet(prefix, inc(prefix))
}
}
The same works with the scala jcl.TreeSet#range implementation.
As I understand it, the Scala TreeSet is backed by the Java TreeSet, but using the Scala variant would allow you to shorten up the code using a sequence comprehension (http://www.scala-lang.org/node/111) giving you an implementation that looked something like (for Scala 2.7):
import scala.collection.jcl.TreeSet;
class PrefixMatcher
{
private val _set = new TreeSet[String]
def add(s: String) = _set.add(s)
def findMatches(prefix: String): Iterable[String] =
for (s <- _set.from(prefix) if s.startsWith(prefix)) yield s
}
object Main
{
def main(args: Array[String]): Unit =
{
val pm = new PrefixMatcher()
pm.add("fooBar")
pm.add("fooCow")
pm.add("barFoo")
pm.findMatches("foo").foreach(println)
}
}
Apologies for any bad Scala style on my part, I'm just getting used to the language myself.
I blogged about finding matches for a combination of prefixes a while ago. It's a harder problem, as you don't know when one prefix ends and the other begins. It might interest you. I'll even post below the code that I did not blog (yet, hopefully :), though it is stripped of all comments, none of which were made in English:
package com.blogspot.dcsobral.matcher.DFA
object DFA {
type Matched = List[(String, String)]
def words(s : String) = s.split("\\W").filter(! _.isEmpty).toList
}
import DFA._
import scala.runtime.RichString
class DFA {
private val initialState : State = new State(None, "")
private var currState : State = initialState
private var _input : RichString = ""
private var _badInput : RichString = ""
private var _accepted : Boolean = true
def accepted : Boolean = _accepted
def input : String = _input.reverse + _badInput.reverse
def transition(c : Char) : List[(String, Matched)] = {
if (c == '\b') backtrack
else {
if (accepted) {
val newState = currState(c)
newState match {
case Some(s) => _input = c + _input; currState = s
case None => _badInput = c + _badInput; _accepted = false
}
} else {
_badInput = c + _badInput
}
optionList
}
}
def transition(s : String) : List[(String, Matched)] = {
s foreach (c => transition(c))
optionList
}
def apply(c : Char) : List[(String, Matched)] = transition(c)
def apply(s : String) : List[(String,Matched)] = transition(s)
def backtrack : List[(String, Matched)] = {
if(_badInput isEmpty) {
_input = _input drop 1
currState.backtrack match {
case Some(s) => currState = s
case None =>
}
} else {
_badInput = _badInput drop 1
if (_badInput isEmpty) _accepted = true
}
optionList
}
def optionList : List[(String, Matched)] = if (accepted) currState.optionList else Nil
def possibleTransitions : Set[Char] = if (accepted) (currState possibleTransitions) else Set.empty
def reset : Unit = {
currState = initialState
_input = ""
_badInput = ""
_accepted = true
}
def addOption(s : String) : Unit = {
initialState addOption s
val saveInput = input
reset
transition(saveInput)
}
def removeOption(s : String) : Unit = {
initialState removeOption s
val saveInput = input
reset
transition(saveInput)
}
}
class State (val backtrack : Option[State],
val input : String) {
private var _options : List[PossibleMatch] = Nil
private val transitions : scala.collection.mutable.Map[Char, State] = scala.collection.mutable.Map.empty
private var _possibleTransitions : Set[Char] = Set.empty
private def computePossibleTransitions = {
if (! options.isEmpty)
_possibleTransitions = options map (_.possibleTransitions) reduceLeft (_++_)
else
_possibleTransitions = Set.empty
}
private def computeTransition(c : Char) : State = {
val newState = new State(Some(this), input + c)
options foreach (o => if (o.possibleTransitions contains c) (o computeTransition (newState, c)))
newState
}
def options : List[PossibleMatch] = _options
def optionList : List[(String, Matched)] = options map (pm => (pm.option, pm.bestMatch))
def possibleTransitions : Set[Char] = _possibleTransitions
def transition(c : Char) : Option[State] = {
val t = c.toLowerCase
if (possibleTransitions contains t) Some(transitions getOrElseUpdate (t, computeTransition(t))) else None
}
def apply(c : Char) : Option[State] = transition(c)
def addOption(option : String) : Unit = {
val w = words(option)
addOption(option, w.size, List(("", w.head)), w)
}
def addOption(option : String, priority : Int, matched : Matched, remaining : List[String]) : Unit = {
options find (_.option == option) match {
case Some(pM) =>
if (!pM.hasMatchOption(matched)) {
pM.addMatchOption(priority, matched, remaining)
if (priority < pM.priority) {
val (before, _ :: after) = options span (_ != pM)
val (highPriority, lowPriority) = before span (p => p.priority < priority ||
(p.priority == priority && p.option < option))
_options = highPriority ::: (pM :: lowPriority) ::: after
}
transitions foreach (t => pM computeTransition (t._2, t._1))
}
case None =>
val (highPriority, lowPriority) = options span (p => p.priority < priority ||
(p.priority == priority && p.option < option))
val newPM = new PossibleMatch(option, priority, matched, remaining)
_options = highPriority ::: (newPM :: lowPriority)
transitions foreach (t => newPM computeTransition (t._2, t._1))
}
computePossibleTransitions
}
def removeOption(option : String) : Unit = {
options find (_.option == option) match {
case Some(possibleMatch) =>
val (before, _ :: after) = options span (_ != possibleMatch)
(possibleMatch.possibleTransitions ** Set(transitions.keys.toList : _*)).toList foreach (t =>
transition(t).get.removeOption(option))
_options = before ::: after
computePossibleTransitions
case None =>
}
}
}
class PossibleMatch (val option : String,
thisPriority : Int,
matched : Matched,
remaining : List[String]) {
private var _priority = thisPriority
private var matchOptions = List(new MatchOption(priority, matched, remaining))
private var _possibleTransitions = matchOptions map (_.possibleTransitions) reduceLeft (_++_)
private def computePossibleTransitions = {
_possibleTransitions = matchOptions map (_.possibleTransitions) reduceLeft (_++_)
}
def priority : Int = _priority
def hasMatchOption(matched : Matched) : Boolean = matchOptions exists (_.matched == matched)
def addMatchOption(priority : Int, matched : Matched, remaining : List[String]) : Unit = {
if (priority < _priority) _priority = priority
val (highPriority, lowPriority) = matchOptions span (_.priority < priority)
val newMO = new MatchOption(priority, matched, remaining)
matchOptions = highPriority ::: (newMO :: lowPriority)
computePossibleTransitions
}
def bestMatch : Matched = matchOptions.head.matched.reverse.map(p => (p._1.reverse.toString, p._2)) :::
remaining.tail.map(w => ("", w))
def possibleTransitions : Set[Char] = _possibleTransitions
def computeTransition(s: State, c : Char) : Unit = {
def computeOptions(state : State,
c : Char,
priority : Int,
matched : Matched,
remaining : List[String]) : Unit = {
remaining match {
case w :: ws =>
if (!w.isEmpty && w(0).toLowerCase == c.toLowerCase) {
val newMatched = (w(0) + matched.head._1, matched.head._2.substring(1)) :: matched.tail
val newPriority = if (matched.head._1 isEmpty) (priority - 1) else priority
if (w.drop(1) isEmpty)
s.addOption(option, newPriority - 1, ("", ws.head) :: newMatched , ws)
else
s.addOption(option, newPriority, newMatched, w.substring(1) :: ws)
}
if (ws != Nil) computeOptions(s, c, priority, ("", ws.head) :: matched, ws)
case Nil =>
}
}
if(possibleTransitions contains c)
matchOptions foreach (mO => computeOptions(s, c, mO.priority, mO.matched, mO.remaining))
}
}
class MatchOption (val priority : Int,
val matched : Matched,
val remaining : List[String]) {
lazy val possibleTransitions : Set[Char] = Set( remaining map (_(0) toLowerCase) : _* )
}
It really needs some refactoring, though. I always do it when I'm start to explain it for the blog.
Ok, I just realized what you want is pretty much what a friend of mine suggested for another problem. So, here is his answer, simplified for your needs.
class PrefixMatcher {
// import scala.collection.Set // Scala 2.7 needs this -- and returns a gimped Set
private var set = new scala.collection.immutable.TreeSet[String]()
private def succ(s : String) = s.take(s.length - 1) + ((s.charAt(s.length - 1) + 1)).toChar
def add(s: String) = set += s
def findMatches(prefix: String): Set[String] =
if (prefix.isEmpty) set else set.range(prefix, succ(prefix))
}