I have the following entry in Play for Scala application.conf:
jobs = [
{number: 0, dir: "/dir1", name: "General" },
{number: 1, dir: "/dir2", name: "Customers" }
]
I want to retrieve this list of objects in a Scala program:
val conf = ConfigFactory.load
val jobs = conf.getAnyRefList("jobs").asScala
println(jobs)
this prints
Buffer({number=0, name=General, dir=/dir1}, {number=1, name=Customers, dir=/dir2})
But how to convert the result to actual Scala objects?
Try this one:
case class Job(number: Int, dir: String, name: String)
object Job {
implicit val configLoader: ConfigLoader[List[Job]] = ConfigLoader(_.getConfigList).map(
_.asScala.toList.map(config =>
Job(
config.getInt("number"),
config.getString("dir"),
config.getString("name")
)
)
)
}
Then from Confugutation DI
Configuration.get[List[Job]]("jobs")
Here is a Config object which will extract data from a config file into a type that you specify.
Usage:
case class Job(number: Int, dir: String, name: String)
val jobs = Config[List[Job]]("jobs")
Code:
import com.typesafe.config._
import org.json4s._
import org.json4s.jackson.JsonMethods._
object Config {
private val conf = ConfigFactory.load()
private val jData = parse(conf.root.render(ConfigRenderOptions.concise))
def apply[T](name: String)(implicit formats: Formats = DefaultFormats, mf: Manifest[T]): T =
Extraction.extract(jData \\ name)(formats, mf)
}
This will throw an exception if the particular config object does not exist or does not match the format of T.
Related
I want to use JCommander to parse args.
I wrote some code:
import com.beust.jcommander.{JCommander, Parameter}
import scala.collection.mutable.ArrayBuffer
object Config {
#Parameter(names = Array("--categories"), required = true)
var categories = new ArrayBuffer[String]
}
object Main {
def main(args: Array[String]): Unit = {
val cfg = Config
JCommander
.newBuilder()
.addObject(cfg)
.build()
.parse(args.toArray: _*)
println(cfg.categories)
}
}
Howewer it fails with
com.beust.jcommander.ParameterException: Could not invoke null
Reason: Can not set static scala.collection.mutable.ArrayBuffer field InterestRulesConfig$.categories to java.lang.String
What am i doing wrong?
JCommander uses knowledge about types in Java to map values to parameters. But Java doesn't have a type scala.collection.mutable.ArrayBuffer. It has a type java.util.List. If you want to use JCommander you have to stick to Java's build-in types.
If you want to use Scala's types use one of Scala's libraries that handle in in more idiomatic manner: scopt or decline.
Working example
import java.util
import com.beust.jcommander.{JCommander, Parameter}
import scala.jdk.CollectionConverters._
object Config {
#Parameter(names = Array("--categories"), required = true)
var categories: java.util.List[Integer] = new util.ArrayList[Integer]()
}
object Hello {
def main(args: Array[String]): Unit = {
val cfg = Config
JCommander
.newBuilder()
.addObject(cfg)
.build()
.parse(args.toArray: _*)
println(cfg.categories)
println(cfg.categories.getClass())
val a = cfg.categories.asScala
for (x <- a) {
println(x.toInt)
println(x.toInt.getClass())
}
}
}
I'm trying to use the Scala mongo driver with case classes, as described at: http://mongodb.github.io/mongo-scala-driver/2.2/getting-started/quick-tour-case-classes/
However I'm getting the exception:
Can't find a codec for class com.foo.model.User$.
org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class com.foo.model.User$.
when I try to insert an item.
The case class:
case class User(
_id: ObjectId,
foo: String = "",
foo2: String = "",
foo3: String = "",
first: String,
last: String,
username: String,
pwHash: String = ""
gender: String,
isFoo: Boolean = false)
extends FooTrait
The code:
val providers = fromProviders( classOf[User])
val registry = fromRegistries(providers, DEFAULT_CODEC_REGISTRY)
val connStr = "mongodb://...."
val clusterSettings = ClusterSettings.builder().applyConnectionString(new ConnectionString(connStr)).build()
val clientSettings = MongoClientSettings.builder().codecRegistry(getCodecRegistry).clusterSettings(clusterSettings).build()
val client = MongoClient( clientSettings )
val database: MongoDatabase = client.getDatabase(dbName).withCodecRegistry(registry)
val modelCollection: MongoCollection[User] = db.getCollection("user")
val item = User(.....) //snipped
modelCollection.insertOne(item).toFuture()
Full stack trace:
Can't find a codec for class com.foo.model.User$.
org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class com.foo.model.User$.
at org.bson.codecs.configuration.CodecCache.getOrThrow(CodecCache.java:46)
at org.bson.codecs.configuration.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:63)
at org.bson.codecs.configuration.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:37)
at com.mongodb.async.client.MongoCollectionImpl.getCodec(MongoCollectionImpl.java:1170)
at com.mongodb.async.client.MongoCollectionImpl.getCodec(MongoCollectionImpl.java:1166)
at com.mongodb.async.client.MongoCollectionImpl.executeInsertOne(MongoCollectionImpl.java:519)
at com.mongodb.async.client.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:501)
at com.mongodb.async.client.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:496)
at org.mongodb.scala.MongoCollection.$anonfun$insertOne$1(MongoCollection.scala:410)
at org.mongodb.scala.MongoCollection.$anonfun$insertOne$1$adapted(MongoCollection.scala:410)
at org.mongodb.scala.internal.ObservableHelper$$anon$2.apply(ObservableHelper.scala:42)
at org.mongodb.scala.internal.ObservableHelper$$anon$2.apply(ObservableHelper.scala:40)
at com.mongodb.async.client.SingleResultCallbackSubscription.requestInitialData(SingleResultCallbackSubscription.java:38)
at com.mongodb.async.client.AbstractSubscription.tryRequestInitialData(AbstractSubscription.java:151)
at com.mongodb.async.client.AbstractSubscription.request(AbstractSubscription.java:82)
at org.mongodb.scala.ObservableImplicits$BoxedSubscription.request(ObservableImplicits.scala:474)
at org.mongodb.scala.ObservableImplicits$ScalaObservable$$anon$2.onSubscribe(ObservableImplicits.scala:373)
at org.mongodb.scala.ObservableImplicits$ToSingleObservable$$anon$3.onSubscribe(ObservableImplicits.scala:440)
at org.mongodb.scala.Observer.onSubscribe(Observer.scala:85)
at org.mongodb.scala.Observer.onSubscribe$(Observer.scala:85)
at org.mongodb.scala.ObservableImplicits$ToSingleObservable$$anon$3.onSubscribe(ObservableImplicits.scala:432)
at com.mongodb.async.client.SingleResultCallbackSubscription.<init>(SingleResultCallbackSubscription.java:33)
at com.mongodb.async.client.Observables$2.subscribe(Observables.java:76)
at org.mongodb.scala.ObservableImplicits$BoxedObservable.subscribe(ObservableImplicits.scala:458)
at org.mongodb.scala.ObservableImplicits$ToSingleObservable.subscribe(ObservableImplicits.scala:432)
at org.mongodb.scala.ObservableImplicits$ScalaObservable.headOption(ObservableImplicits.scala:365)
at org.mongodb.scala.ObservableImplicits$ScalaObservable.head(ObservableImplicits.scala:351)
at org.mongodb.scala.ObservableImplicits$ScalaSingleObservable.toFuture(ObservableImplicits.scala:410)
I think I'm doing everything right - and unless this is a bug, the code should work. My mongo-scala-driver version is 2.2.0.
Any ideas?
Here is a sample that is working on my local box with Mongo 3.6.1.
// ammonite script mongo.sc
import $ivy.`org.mongodb.scala::mongo-scala-driver:2.2.0`
import org.mongodb.scala._
import org.mongodb.scala.connection._
import org.mongodb.scala.bson.ObjectId
import org.mongodb.scala.bson.codecs.Macros._
import org.mongodb.scala.bson.codecs.DEFAULT_CODEC_REGISTRY
import org.bson.codecs.configuration.CodecRegistries.{fromRegistries, fromProviders}
trait FooTrait
case class User(_id: ObjectId,
foo: String = "",
foo2: String = "",
foo3: String = "",
first: String,
last: String,
username: String,
pwHash: String = "",
gender: String,
isFoo: Boolean = false) extends FooTrait
val codecRegistry = fromRegistries(fromProviders(classOf[User]), DEFAULT_CODEC_REGISTRY )
import scala.collection.JavaConverters._
val clusterSettings: ClusterSettings = ClusterSettings.builder().hosts(List(new ServerAddress("localhost")).asJava).build()
val settings: MongoClientSettings = MongoClientSettings.builder().clusterSettings(clusterSettings).build()
val mongoClient: MongoClient = MongoClient(settings)
val database: MongoDatabase = mongoClient.getDatabase("mydb").withCodecRegistry(codecRegistry)
val userCollection: MongoCollection[User] = database.getCollection("user")
val user:User = User(new ObjectId(), "foo", "foo2", "foo3", "first", "last", "username", "pwHash", "gender", true)
import scala.concurrent.duration._
import scala.concurrent.Await
// wait for Mongo to complete insert operation
Await.result(userCollection.insertOne(user).toFuture(),3.seconds)
When you save this snippet into a file mongo.sc, then you can run it with Ammonite using
amm mongo.sc
In case mongo is running on the default port, the mydb database should get created automatically including the new user collection.
I'm trying to get started with avro & avro4s and I'm running into trouble getting a silly example working.
I can't even get my own version of a test found in the source repo to compile - https://github.com/sksamuel/avro4s/blob/master/avro4s-core/src/test/scala/com/sksamuel/avro4s/AvroJsonInputTest.scala#L14-L20
I'm getting the error mentioned in the title could not find implicit value for parameter fromRecord: com.sksamuel.avro4s.FromRecord[Foo].
package test
import java.io.ByteArrayInputStream
import com.sksamuel.avro4s._
import org.specs2.mutable.Specification
class AvroSpec extends Specification {
"My classes" should {
"be deserialized by avro" in {
case class Foo(num: Int, name: String)
val json = """{ "num": 17, "name": "jibba-jabbba" }"""
val inst = Foo(num = 17, name = "jibba-jabba")
val in = new ByteArrayInputStream(json.getBytes("UTF-8"), 0, json.length)
val foo = new AvroJsonInputStream[Foo](in).iterator.toSet
in.close()
foo mustEqual Set(inst)
}
}
}
I'm using Scala 2.11.8 and avro 1.7.0. What am I doing wrong?
Move the case class declaration outside the method definition:
package test
import java.io.ByteArrayInputStream
import com.sksamuel.avro4s._
import org.specs2.mutable.Specification
case class Foo(num: Int, name: String)
class AvroSpec extends Specification {
"My classes" should {
"be deserialized by avro" in {
val json = """{ "num": 17, "name": "jibba-jabbba" }"""
val inst = Foo(num = 17, name = "jibba-jabba")
val in = new ByteArrayInputStream(json.getBytes("UTF-8"), 0, json.length)
val foo = new AvroJsonInputStream[Foo](in).iterator.toSet
in.close()
foo mustEqual Set(inst)
}
}
}
Suppose I have a scala case class with the ability to serialize into json (using json4s or some other library):
case class Weather(zip : String, temp : Double, isRaining : Boolean)
If I'm using a HOCON config file:
allWeather {
BeverlyHills {
zip : 90210
temp : 75.0
isRaining : false
}
Cambridge {
zip : 10013
temp : 32.0
isRainging : true
}
}
Is there any way to use typesafe config to automatically instantiate a Weather object?
I'm looking for something of the form
val config : Config = ConfigFactory.parseFile(new java.io.File("weather.conf"))
val bevHills : Weather = config.getObject("allWeather.BeverlyHills").as[Weather]
The solution could leverage the fact that the value referenced by "allWeather.BeverlyHills" is a json "blob".
I could obviously write my own parser:
def configToWeather(config : Config) =
Weather(config.getString("zip"),
config.getDouble("temp"),
config.getBoolean("isRaining"))
val bevHills = configToWeather(config.getConfig("allWeather.BeverlyHills"))
But that seems inelegant since any change to the Weather definition would also require a change to configToWeather.
Thank you in advance for your review and response.
typesafe config library has API to instantiate object from config that uses java bean convention. But as I understand case class does not follow those rules.
There are several scala libraries that wrap typesafe config and provide scala specific functionality that you are looking for.
For example using pureconfig reading config could look like
val weather:Try[Weather] = loadConfig[Weather]
where Weather is a case class for values in config
Expanding on Nazarii's answer, the following worked for me:
import scala.beans.BeanProperty
//The #BeanProperty and var are both necessary
case class Weather(#BeanProperty var zip : String,
#BeanProperty var temp : Double,
#BeanProperty var isRaining : Boolean) {
//needed by configfactory to conform to java bean standard
def this() = this("", 0.0, false)
}
import com.typesafe.config.ConfigFactory
val config = ConfigFactory.parseFile(new java.io.File("allWeather.conf"))
import com.typesafe.config.ConfigBeanFactory
val bevHills =
ConfigBeanFactory.create(config.getConfig("allWeather.BeverlyHills"), classOf[Weather])
Follow up: based on the comments below it may be the case that only Java Collections, and not Scala Collections, are viable options for the parameters of the case class (e.g. Seq[T] will not work).
A simple solution without external libraries, inspired from playframework Configuration.scala
trait ConfigLoader[A] { self =>
def load(config: Config, path: String = ""): A
def map[B](f: A => B): ConfigLoader[B] = (config, path) => f(self.load(config, path))
}
object ConfigLoader {
def apply[A](f: Config => String => A): ConfigLoader[A] = f(_)(_)
implicit val stringLoader: ConfigLoader[String] = ConfigLoader(_.getString)
implicit val booleanLoader: ConfigLoader[Boolean] = ConfigLoader(_.getBoolean)
implicit val doubleLoader: ConfigLoader[Double] = ConfigLoader(_.getDouble)
}
object Implicits {
implicit class ConfigOps(private val config: Config) extends AnyVal {
def apply[A](path: String)(implicit loader: ConfigLoader[A]): A = loader.load(config, path)
}
implicit def configLoader[A](f: Config => A): ConfigLoader[A] = ConfigLoader(_.getConfig).map(f)
}
Usage:
import Implicits._
case class Weather(zip: String, temp: Double, isRaining: Boolean)
object Weather {
implicit val loader: ConfigLoader[Weather] = (c: Config) => Weather(
c("zip"), c("temp"), c("isRaining")
)
}
val config: Config = ???
val bevHills: Weather = config("allWeather.BeverlyHills")
Run the code in Scastie
Another option is to use circe.config with the code below. See https://github.com/circe/circe-config
import io.circe.generic.auto._
import io.circe.config.syntax._
def configToWeather(conf: Config): Weather = {
conf.as[Weather]("allWeather.BeverlyHills") match {
case Right(c) => c
case _ => throw new Exception("invalid configuration")
}
}
Another tried-and-tested solution is to use com.fasterxml.jackson.databind.ObjectMapper. You don't need to tag #BeanProperty to any of your case class parameters but you will have to define a no-arg constructor.
case class Weather(zip : String, temp : Double, isRaining : Boolean) {
def this() = this(null, 0, false)
}
val mapper = new ObjectMapper().registerModule(DefaultScalaModule)
val bevHills = mapper.convertValue(config.getObject("allWeather.BeverlyHills").unwrapped, classOf[Weather])
Using config loader
implicit val configLoader: ConfigLoader[Weather] = (rootConfig: Config, path: String) => {
val config = rootConfig.getConfig(path)
Weather(
config.getString("zip"),
config.getDouble("temp"),
config.getBoolean("isRaining")
)
}
I'd like to build a generic method for transforming Scala Case Classes to Mongo Documents.
A promising Document constructor is
fromSeq(ts: Seq[(String, BsonValue)]): Document
I can turn a case class into a Map[String -> Any], but then I've lost the type information I need to use the implicit conversions to BsonValues. Maybe TypeTags can help with this?
Here's what I've tried:
import org.mongodb.scala.bson.BsonTransformer
import org.mongodb.scala.bson.collection.immutable.Document
import org.mongodb.scala.bson.BsonValue
case class Person(age: Int, name: String)
//transform scala values into BsonValues
def transform[T](v: T)(implicit transformer: BsonTransformer[T]): BsonValue = transformer(v)
// turn any case class into a Map[String, Any]
def caseClassToMap(cc: Product) = {
val values = cc.productIterator
cc.getClass.getDeclaredFields.map( _.getName -> values.next).toMap
}
// transform a Person into a Document
def personToDocument(person: Person): Document = {
val map = caseClassToMap(person)
val bsonValues = map.toSeq.map { case (key, value) =>
(key, transform(value))
}
Document.fromSeq(bsonValues)
}
<console>:24: error: No bson implicit transformer found for type Any. Implement or import an implicit BsonTransformer for this type.
(key, transform(value))
def personToDocument(person: Person): Document = {
Document("age" -> person.age, "name" -> person.name)
}
Below code works without manual conversion of an object.
import reactivemongo.api.bson.{BSON, BSONDocument, Macros}
case class Person(name:String = "SomeName", age:Int = 20)
implicit val personHandler = Macros.handler[Person]
val bsonPerson = BSON.writeDocument[Person](Person())
println(s"${BSONDocument.pretty(bsonPerson.getOrElse(BSONDocument.empty))}")
You can use Salat https://github.com/salat/salat. A nice example can be found here - https://gist.github.com/bhameyie/8276017. This is the piece of code that will help you -
import salat._
val dBObject = grater[Artist].asDBObject(artist)
artistsCollection.save(dBObject, WriteConcern.Safe)
I was able to serialize a case class to a BsonDocument using the org.bson.BsonDocumentWriter. The below code runs using scala 2.12 and mongo-scala-driver_2.12 version 2.6.0
My quest for this solution was aided by this answer (where they are trying to serialize in the opposite direction): Serialize to object using scala mongo driver?
import org.mongodb.scala.bson.codecs.Macros
import org.mongodb.scala.bson.codecs.DEFAULT_CODEC_REGISTRY
import org.bson.codecs.configuration.CodecRegistries.{fromRegistries, fromProviders}
import org.bson.codecs.EncoderContext
import org.bson.BsonDocumentWriter
import org.mongodb.scala.bson.BsonDocument
import org.bson.codecs.configuration.CodecRegistry
import org.bson.codecs.Codec
case class Animal(name : String, species: String, genus: String, weight: Int)
object TempApp {
def main(args: Array[String]) {
val jaguar = Animal("Jenny", "Jaguar", "Panthera", 190)
val codecProvider = Macros.createCodecProvider[Animal]()
val codecRegistry: CodecRegistry = fromRegistries(fromProviders(codecProvider), DEFAULT_CODEC_REGISTRY)
val codec = Macros.createCodec[Animal](codecRegistry)
val encoderContext = EncoderContext.builder.isEncodingCollectibleDocument(true).build()
var doc = BsonDocument()
val writr = new BsonDocumentWriter(doc) // need to call new since Java lib w/o companion object
codec.encode(writr, jaguar, encoderContext)
print(doc)
}
};