I have some Scala code roughly analogous to this:
object Foo {
val thingA = ...
val thingB = ...
val thingC = ...
val thingD = ...
val thingE = ...
val thingsOfAKind = List(thingA, thingC, thingE)
val thingsOfADifferentKind = List(thingB, thingD)
val allThings = thingsOfAKind ::: thingsOfADifferentKind
}
Is there some nicer way of declaring a bunch of things and being able to access them both individually by name and collectively?
The issue I have with the code above is that the real version has almost 30 different things, and there's no way to actually ensure that each new thing I add is also added to an appropriate list (or that allThings doesn't end up with duplicates, although that's relatively easy to fix).
The various things are able to be treated in aggregate by almost all of the code base, but there are a few places and a couple of things where the individual identity matters.
I thought about just using a Map, but then the compiler looses the ability to check that the individual things being looked up actually exist (and I have to either wrap code to handle a failed lookup around every attempt, or ignore the problem and effectively risk null pointer exceptions).
I could make the kind each thing belongs to an observable property of the things, then I would at least have a single list of all things and could get lists of each of the kinds with filter, but the core issue remains that I would ideally like to be able to declare that a thing exists, has a name (identifier), and is part of a collection.
What I effectively want is something like a compile-time Map. Is there a good way of achieving something like this in Scala?
How about this type of pattern?
class Things[A] {
var all: List[A] = Nil
def ->: (x: A): A = { all = x :: all; x }
}
object Test {
val things1 = new Things[String]
val things2 = new Things[String]
val thingA = "A" ->: things1
val thingB = "B" ->: things2
val thingC = "C" ->: things1
val thingD = ("D" ->: things1) ->: things2
}
You could also add a little sugar, making Things automatically convertible to List,
object Things {
implicit def thingsToList[A](things: Things[A]): List[A] = things.all
}
I can't think of a way to do this without the var that has equally nice syntax.
Related
I am trying to prevent empty values being inserted into my mongoDB collection. The field in question looks like this:
MongoDB Field
"stadiumArr" : [
"Old Trafford",
"El Calderon",
...
]
Sample of (mapped) case class
case class FormData(_id: Option[BSONObjectID], stadiumArr: Option[List[String]], ..)
Sample of Scala form
object MyForm {
val form = Form(
mapping(
"_id" -> ignored(Option.empty[BSONObjectID]),
"stadiumArr" -> optional(list(text)),
...
)(FormData.apply)(FormData.unapply)
)
}
I am also using the Repeated Values functionality in Play Framework like so:
Play Template
#import helper._
#(myForm: Form[models.db.FormData])(implicit request: RequestHeader, messagesProvider: MessagesProvider)
#repeatWithIndex(myForm("stadiumArr"), min = 5) { (stadium, idx) =>
#inputText(stadium, '_label -> ("stadium #" + (idx + 1)))
}
This ensures that whether there are at least 5 values or not in the array; there will still be (at least) 5 input boxes created. However if one (or more) of the input boxes are empty when the form is submitted an empty string is still being added as value in the array, e.g.
"stadiumArr" : [
"Old Trafford",
"El Calderon",
"",
"",
""
]
Based on some other ways of converting types from/to the database; I've tried playing around with a few solutions; such as:
implicit val arrayWrite: Writes[List[String]] = new Writes[List[String]] {
def writes(list: List[String]): JsValue = Json.arr(list.filterNot(_.isEmpty))
}
.. but this isn't working. Any ideas on how to prevent empty values being inserted into the database collection?
Without knowing specific versions or libraries you're using it's hard to give you an answer, but since you linked to play 2.6 documentation I'll assume that's what you're using there. The other assumption I'm going to make is that you're using reactive-mongo library. Whether or not you're using the play plugin for that library or not is the reason why I'm giving you two different answers here:
In that library, with no plugin, you'll have defined a BSONDocumentReader and a BSONDocumentWriter for your case class. This might be auto-generated for you with macros or not, but regardless how you get it, these two classes have useful methods you can use to transform the reads/writes you have to another one. So, let's say I defined a reader and writer for you like this:
import reactivemongo.bson._
case class FormData(_id: Option[BSONObjectID], stadiumArr: Option[List[String]])
implicit val formDataReaderWriter = new BSONDocumentReader[FormData] with BSONDocumentWriter[FormData] {
def read(bson: BSONDocument): FormData = {
FormData(
_id = bson.getAs[BSONObjectID]("_id"),
stadiumArr = bson.getAs[List[String]]("stadiumArr").map(_.filterNot(_.isEmpty))
)
}
def write(formData: FormData) = {
BSONDocument(
"_id" -> formData._id,
"stadiumArr" -> formData.stadiumArr
)
}
}
Great you say, that works! You can see in the reads I went ahead and filtered out any empty strings. So even if it's in the data, it can be cleaned up. That's nice and all, but let's notice I didn't do the same for the writes. I did that so I can show you how to use a useful method called afterWrite. So pretend the reader/writer weren't the same class and were separate, then I can do this:
val initialWriter = new BSONDocumentWriter[FormData] {
def write(formData: FormData) = {
BSONDocument(
"_id" -> formData._id,
"stadiumArr" -> formData.stadiumArr
)
}
}
implicit val cleanWriter = initialWriter.afterWrite { bsonDocument =>
val fixedField = bsonDocument.getAs[List[String]]("stadiumArr").map(_.filterNot(_.isEmpty))
bsonDocument.remove("stadiumArr") ++ BSONDocument("stadiumArr" -> fixedField)
}
Note that cleanWriter is the implicit one, that means when the insert call on the collection happens, it will be the one chosen to be used.
Now, that's all a bunch of work, if you're using the plugin/module for play that lets you use JSONCollections then you can get by with just defining play json Reads and Writes. If you look at the documentation you'll see that the reads trait has a useful map function you can use to transform one Reads into another.
So, you'd have:
val jsonReads = Json.reads[FormData]
implicit val cleanReads = jsonReads.map(formData => formData.copy(stadiumArr = formData.stadiumArr.map(_.filterNot(_.isEmpty))))
And again, because only the clean Reads is implicit, the collection methods for mongo will use that.
NOW, all of that said, doing this at the database level is one thing, but really, I personally think you should be dealing with this at your Form level.
val form = Form(
mapping(
"_id" -> ignored(Option.empty[BSONObjectID]),
"stadiumArr" -> optional(list(text)),
...
)(FormData.apply)(FormData.unapply)
)
Mainly because, surprise surprise, form has a way to deal with this. Specifically, the mapping class itself. If you look there you'll find a transform method you can use to filter out empty values easily. Just call it on the mapping you need to modify, for example:
"stadiumArr" -> optional(
list(text).transform(l => l.filter(_.nonEmpty), l => l.filter(_.nonEmpty))
)
To explain a little more about this method, in case you're not used to reading the signatures in the scaladoc.
def
transform[B](f1: (T) ⇒ B, f2: (B) ⇒ T): Mapping[B]
says that by calling transform on some mapping of type Mapping[T] you can create a new mapping of type Mapping[B]. In order to do this you must provide functions that convert from one to the other. So the code above causes the list mapping (Mapping[List[String]]) to become a Mapping[List[String]] (the type did not change here), but when it does so it removes any empty elements. If I break this code down a little it might be more clear:
def convertFromTtoB(list: List[String]): List[String] = list.filter(_.nonEmpty)
def convertFromBtoT(list: List[String]): List[String] = list.filter(_.nonEmpty)
...
list(text).transform(convertFromTtoB, convertFromBtoT)
You might wondering why you need to provide both, the reason is because when you call Form.fill and the form is populated with values, the second method will be called so that the data goes into the format the play form is expecting. This is more obvious if the type actually changes. For example, if you had a text area where people could enter CSV but you wanted to map it to a form model that had a proper List[String] you might do something like:
def convertFromTtoB(raw: String): List[String] = raw.split(",").filter(_.nonEmpty)
def convertFromBtoT(list: List[String]): String = list.mkString(",")
...
text.transform(convertFromTtoB, convertFromBtoT)
Note that when I've done this in the past sometimes I've had to write a separate method and just pass it in if I didn't want to fully specify all the types, but you should be able to work from here given the documentation and type signature for the transform method on mapping.
The reason I suggest doing this in the form binding is because the form/controller should be the one with the concern of dealing with your user data and cleaning things up I think. But you can always have multiple layers of cleaning and whatnot, it's not bad to be safe!
I've gone for this (which always seems obvious when it's written and tested):
implicit val arrayWrite: Writes[List[String]] = new Writes[List[String]] {
def writes(list: List[String]): JsValue = Json.toJson(list.filterNot(_.isEmpty).toIndexedSeq)
}
But I would be interested to know how to
.map the existing Reads rather than redefining from scratch
as #cchantep suggests
I have the following scenario:
case class MyString(str: String)
val val1: ValidatedNel[String, MyString] = MyString("valid1").validNel
val val2: ValidatedNel[String, MyString] = MyString("valid2").validNel
val val3: ValidatedNel[String, MyString] = "invalid".invalidNel
val vals = Seq(val1, val2, val3)
//vals: Seq[cats.data.Validated[cats.data.NonEmptyList[String],MyString]] = List(Valid(MyString(valid)), Invalid(NonEmptyList(invalid)))
At the end I'd like to be able to do a match on the result and get any and all errors or all the valid values as a sequence.
My question is: How to convert Seq[Validated[NonEmptyList[String],MyString]] into Validated[NonEmptyList[String],Seq[MyString]]]
So, my first pass was to implement Semigroup for Seq[MyString]:
implicit val myStringsAdditionSemigroup: Semigroup[Seq[MyString]] = new Semigroup[Seq[MyString]] {
def combine(x: Seq[MyString], y: Seq[MyString]): Seq[MyString] = x ++ y
}
... which works:
Seq(val1, val2).map(_.map(Seq(_))).reduce(_ |+| _)
//res0: cats.data.Validated[cats.data.NonEmptyList[String],Seq[MyString]] = Valid(List(MyString(valid1), MyString(valid2)))
but I need to prepare my data by wrapping all valid values in Seq... which feels strange. So, maybe there's a better way of doing that?
If you use anything other than Seq, like Vector or List, you can sequence it.
sequence basically turns a type constructor inside out. Meaning turning a F[G[A]] into an G[F[A]]. For that to work, the F needs to be a Traverse and the G needs to be an Applicative. Luckily, Validated is an Applicative and List or Vector are instances of Traverse.
So in the end your code should look something like this:
import cats.implicits._
val validatedList: Validated[NonEmptyList[String],List[MyString]]] =
vals.sequence
Note: if this doesn't compile for you, you might need to enable partial-unification.
The easiest way to enable partial-unification, is to add the sbt-partial-unification plugin.
If you're on Scala 2.11.9 or newer, you can also simply add the compiler flag:
scalacOptions += "-Ypartial-unification"
We from the cats team strongly encourage you to have this flag on at all times when using cats, as it makes everything just a lot easier.
I have a variable underlying of type Option[mutable.Traversable[Field]]
All I wanted todo in my class was provide a method to return this as Sequence in the following way:
def toSeq: scala.collection.mutable.Seq[Field] = {
for {
f <- underlying.get
} yield f
}
This fails as it complains that mutable.traversable does not conform to mutable.seq. All it is doing is yielding something of type Field - in my mind this should work?
A possible solution to this is:
def toSeq: Seq[Field] = {
underlying match {
case Some(x) => x.toSeq
case None =>
}
}
Although I have no idea what is actually happening when x.toSeq is called and I imagine there is more memory being used here that actually required to accomplish this.
An explanation or suggestion would be much appreciated.
I am confused why you say that "I imagine there is more memory being used here than actually required to accomplish". Scala will not copy your Field values when doing x.toSeq, it is simply going to create an new Seq which will have pointers to the same Field values that underlying is pointing to. Since this new structure is exactly what you want there is no avoiding the additional memory associated with the extra pointers (but the amount of additional memory should be small). For a more in-depth discussion see the wiki on persistent data structures.
Regarding your possible solution, it could be slightly modified to get the result you're expecting:
def toSeq : Seq[Field] =
underlying
.map(_.toSeq)
.getOrElse(Seq.empty[Field])
This solution will return an empty Seq if underlying is a None which is safer than your original attempt which uses get. I say it's "safer" because get throws a NoSuchElementException if the Option is a None whereas my toSeq can never fail to return a valid value.
Functional Approach
As a side note: when I first started programming in scala I would write many functions of the form:
def formatSeq(seq : Seq[String]) : Seq[String] =
seq map (_.toUpperCase)
This is less functional because you are expecting a particular collection type, e.g. formatSeq won't work on a Future.
I have found that a better approach is to write:
def formatStr(str : String) = str.toUpperCase
Or my preferred coding style:
val formatStr = (_ : String).toUpperCase
Then the user of your function can apply formatStr in any fashion they want and you don't have to worry about all of the collection casting:
val fut : Future[String] = ???
val formatFut = fut map formatStr
val opt : Option[String] = ???
val formatOpt = opt map formatStr
I'm interested in understanding the pattern of modifying an immutable collection in the most effective way.
Suppose for example I want to define the following class (an actual implementation may overload the operators, but this is just for illustration). Is this the best way (with respect to performance / safety)?
import scala.collection.immutable.HashMap
class Example[A,B] {
var col = new HashMap[A, B]()
def add(k: A, v: B): Unit = col = col + (k -> v)
def remove(k: A): Unit = col = col - k
}
would this approach be possible as well, in some way I'm missing?
class Example[A,B] extends HashMap[A,B] {
def add(k: A, v: B): Unit = ???
def remove(k: A): Unit = ???
}
The comments above are correct. The idea of immutable data structures exists and there are ways to do it, however, it seems that you may just want to stick with a mutable structure since you have a var anyway.
Check out chapter 17 of the free online pdf of Programming In Scala here where Odersky discusses the designing and building of an Immutable queue.
The gist of it is that you never actually modify the structure that you're trying to change, you just take it in, look at it, and then build a new one based on the old one and whatever it is you're trying to do.
It's the same idea behind the fact that with lists:
val list = List(1,2,3)
1 :: list
println(list)
Will print out List(1,2,3) as opposed to List(1,1,2,3)
While:
val list = List(1,2,3)
val list1 = 1 :: list
println(list1)
is what prints List(1,1,2,3)
Here's a nice slide deck that discusses some popular data structures and their functional counterparts. Functional Data Structures
It sounds like you're trying to make immutable data structures mutable. There are situations where you need to think about performance to be sure, but given that Scala has persistent data structures I'd focus on the use case and not the performance model.
val a = Map("key1" -> "some value")
val b = a + ("key2" -> "some other value")
Now b contains both entries.
If you actually need the mutable structures for your case, just use the mutable.Map.
I'm wondering if it makes sense to use lazy val to prevent unnecessary heap allocations, thereby making the GC's job a little easier. For instance, I am in the habit of writing code like so:
lazy val requiredParameterObject = new Foo {
val a = new ...
val b = new ...
// etc
}
for (a <- collection) a.someFunction(requiredParameterObject)
The idea here is that by making requiredParameterObject lazy, I am saving a heap allocation in the case that collection is empty. But I am wondering: do the internal details of lazy make this not an effective performance win?
You can pass lazy val by name:
lazy val requiredParameterObject = new Foo {
val a = new ...
val b = new ...
// etc
}
class C {
def someFunction(f: => Foo) {
f //(1)
}
}
val collection = List(new C, new C)
for (a <- collection) a.someFunction(requiredParameterObject)
In the code above Foo is created once when the first item of a collection accesses it in (1) - or never if the collection is empty.
You might also get better results if requiredParameterObject is an ordinary function, effectively calling requiredParameterObject lazily only when it's needed. However it will call it as many times as the the number of elements in the collection.
A similar question was asked here a while back, but somebody points out there that the implementation of lazy varies between Scala compiler versions. Probably the most authoritative answer you'll get to this question is to write a minimal snippet, like:
def test(pred : Boolean) {
lazy val v = // ...
if (pred)
doSomething(v)
}
run it through your version of scalac and then peek at the classfile with scalap or javap to see what you get.