Can you import user-defined scala classes into proto? - scala

I have been tasked to create a general case class that would cover different kinds of config (existing already in the system).
Here's a sample of the configs:
package ngage.sdk.configs
...
object Config {
case class TokenConfig(expiry: FiniteDuration)
case class SagaConfig(timeOut: FiniteDuration, watcherTimeout: FiniteDuration)
case class ElasticConfig(connectionString: String, deletePerPage: Int)
case class S3Config(bucket: String)
case class RedisConfig(host: String, port: Int)
...
I looked to protobuf for a solution, but I don't know how to import the case classes as mentioned above.
Here's how I started:
syntax = "proto3";
package ngage.sdk.distributedmemory.config;
import "scalapb/scalapb.proto";
import "pboptions.proto";
import "google/protobuf/duration.proto";
import "ngage.sdk.configs.Config"; //this became red
option (scalapb.options) = {
flat_package: true
single_file: true
};
message ShardMemoryConfig {
option (ngage.type_id) = 791;
int32 size = 1;
oneof config { //everything inside oneof is red
ngage.sdk.configs.RedisConfig redis = 100002;
ngage.sdk.configs.ElasticConfig elastic = 100003;
ngage.sdk.configs.S3Config s3 = 100004;
}
}
Is it even possible to import user-defined scala classes to protobuf?

You'd need a library which generates .proto files from case class definitions, I don't know if one exists (but I would expect not). PBDirect lets you write/read case classes directly, but then your ShardMemoryConfig should also be a case class using RedisConfig etc.

Related

How to convert sealed trait case objects to string using circe

I am using Scala and Circe. I have the following sealed trait.
sealed trait Mode
case object Authentication extends Mode
case object Ocr extends Mode
The output of this case object when called SessionModel.Authentication is the following:
"Authentication":{}
I need to convert this to a string so it outputs "authentication"
As Andriy Plokhotnyuk notes above, you can use circe-generic-extras:
import io.circe.Codec
import io.circe.generic.extras.Configuration
import io.circe.generic.extras.semiauto.deriveEnumerationCodec
sealed trait Mode
case object Authentication extends Mode
case object Ocr extends Mode
object Mode {
private implicit val config: Configuration =
Configuration.default.copy(transformConstructorNames = _.toLowerCase)
implicit val modeCodec: Codec[Mode] = deriveEnumerationCodec[Mode]
}
And then:
scala> import io.circe.syntax._
import io.circe.syntax._
scala> (Authentication: Mode).asJson
res1: io.circe.Json = "authentication"
scala> io.circe.Decoder[Mode].decodeJson(res1)
res2: io.circe.Decoder.Result[Mode] = Right(Authentication)
(Note that Codec is new in 0.12—for earlier versions you'll have to write out both instances as in Andriy's comment.)
Unless you have a lot of these to maintain, though, I personally think writing the instances out by hand is often better than using circe-generic-extras, and in this case it's not even much more verbose:
import io.circe.{Decoder, Encoder}
sealed trait Mode
case object Authentication extends Mode
case object Ocr extends Mode
object Mode {
implicit val decodeMode: Decoder[Mode] = Decoder[String].emap {
case "authentication" => Right(Authentication)
case "ocr" => Right(Ocr)
case other => Left(s"Invalid mode: $other")
}
implicit val encodeMode: Encoder[Mode] = Encoder[String].contramap {
case Authentication => "authentication"
case Ocr => "ocr"
}
}
Which works exactly the same as the deriveEnumerationCodec version but doesn't require anything but circe-core, is less magical, compiles much faster, etc. Generic derivation can be great for simple case classes with straightforward mappings, but I think people too often try to stretch it to cover all cases when writing instances manually wouldn't be much of a burden and might even be clearer.

Scala : How to use Global Config case class across application

I am new to scala, just started with my scala first application.
I have defined my config file under the resources folder, application.conf
projectname{
"application" {
"host":127.0.0.1
"port":8080
}
}
I have wrote one config parser file to parse from config file to case class
case class AppConfig (config: Config) {
val host = config.getString("projectname.application.host")
val port = config.getInt("projectname.application.port")
}
In my grpc server file, i have declared config as
val config = AppConfig(ConfigFactory.load("application.conf"))
I want to use this config variable across application, rather than loading application.conf file everytime.
I want to have one bootstrap function which will parse this config one time, making it available across application
You can do this automatically with PureConfig.
Add Pure Config to you build.sbt with:
libraryDependencies += "com.github.pureconfig" %% "pureconfig" % "0.11.0"
and reload the sbt shell and update your dependencies.
Now, let's say you have the following resource.conf file:
host: example.com
port: 80
user: admin
password: admin_password
You can define a case class called AppConfig:
case class AppConfig(
host: String,
port: Int,
user: String,
password: String
)
And create an instance of it, populated with the application config, using the loadConfig method:
import pureconfig.generic.auto._
val errorsOrConfig: Either[ConfigReaderFailures, AppConfig] = pureconfig.loadConfig[AppConfig]
This returns Either an error or your AppConfig, depending on the values in the config itself.
For example, if the value of port above will be eighty, instead of 80, you will get a detailed error, saying that the second config line (with the port: eighty) contained a string, but the only valid expected type is a number:
ConfigReaderFailures(
ConvertFailure(
reason = WrongType(
foundType = STRING,
expectedTypes = Set(NUMBER)
),
location = Some(
ConfigValueLocation(
new URL("file:~/repos/example-project/target/scala-2.12/classes/application.conf"),
lineNumber = 2
)
),
path = "port"
)
)
You can use loadConfigOrThrow if you want to get AppConfig instead of an Either.
After you load this config once at the start of your application (as close as possible to your main function), you can use dependency injection to pass it along to all the other classes, simply by passing the AppConfig in the constructor.
If you would like to wire up your Logic class (and other services) with the config case class using MacWire, as Krzysztof suggested in one of his options, you can see my answer here.
The plain example (without MacWire), looks like this:
package com.example
import com.example.config.AppConfig
object HelloWorld extends App {
val config: AppConfig = pureconfig.loadConfigOrThrow[AppConfig]
val logic = new Logic(config)
}
class Logic(config: AppConfig) {
// do something with config
}
Where the AppConfig is defined in AppConfig.scala
package com.example.config
case class AppConfig(
host: String,
port: Int,
user: String,
password: String
)
As a bonus, when you use this config variable in your IDE, you will get code completion.
Moreover, your config may be built from the supported types, such as String, Boolean, Int, etc, but also from other case classes that are build from the supported types (this is since a case class represents a value object, that contains data), as well as lists and options of supported types.
This allows you to "class up" a complicated config file and get code completion. For instance, in application.conf:
name: hotels_best_dishes
host: "https://example.com"
port: 80
hotels: [
"Club Hotel Lutraky Greece",
"Four Seasons",
"Ritz",
"Waldorf Astoria"
]
min-duration: 2 days
currency-by-location {
us = usd
england = gbp
il = nis
}
accepted-currency: [usd, gbp, nis]
application-id: 00112233-4455-6677-8899-aabbccddeeff
ssh-directory: /home/whoever/.ssh
developer: {
name: alice,
age: 20
}
Then define a config case class in your code:
import java.net.URL
import java.util.UUID
import scala.concurrent.duration.FiniteDuration
import pureconfig.generic.EnumCoproductHint
import pureconfig.generic.auto._
case class Person(name: String, age: Int)
sealed trait Currency
case object Usd extends Currency
case object Gbp extends Currency
case object Nis extends Currency
object Currency {
implicit val currencyHint: EnumCoproductHint[Currency] = new EnumCoproductHint[Currency]
}
case class Config(
name: String,
host: URL,
port: Int,
hotels: List[String],
minDuration: FiniteDuration,
currencyByLocation: Map[String, Currency],
acceptedCurrency: List[Currency],
applicationId: UUID,
sshDirectory: java.nio.file.Path,
developer: Person
)
And load it with:
val config: Config = pureconfig.loadConfigOrThrow[Config]
There are some possibilities to handle your problem:
Use runtime dependency injection framework like guice. You can use extension for scala.
Use implicits to handle it. You just need to create an object, which will hold your implicit config:
object Implicits {
implicit val config = AppConfig(ConfigFactory.load("application.conf"))
}
And then you can just add implicit config: Config to your arguments list when you need it:
def process(n: Int)(implicit val config: Config) = ??? //as method parameter
case class Processor(n: Int)(implicit val config: AppConfig) //or as class field
And use it like:
import Implicits._
process(5) //config passed implicitly here
Processor(10) //and here
A great advantage of it is you can pass config manually for tests:
process(5)(config)
The downside of this approach is, that having a lot of implicit resolution in your app, will make compilation slow, but it shouldn't be a problem if your app is not humongous.
Make config a field of your classes (it is called constructor injection).
class Foo(config: Config).
Then you can wire-up your dependencies manually, like:
val config: AppConfig = AppConfig()
val foo = Foo(config) //you need to pass config manually to constructors in your object graph
or you can use a framework which can automate it for you, like macwire:
val config = wire[AppConfig]
val foo = wire[Foo]
You can use a pattern called cake-pattern. It works fine for small-sized applications, but the bigger your app is, the clunkier this approach gets.
What is NOT a good approach is using global singleton like this:
object ConfigHolder {
val Config: AppConfig = ???
}
And then use it like:
def process(n: Int) = {
val host = ConfigHolder.Config.host // anti-pattern
}
It is bad because it makes mocking your config for tests very difficult and the whole testing process becomes clunky.
In my opinion, if your app is not very big, you should use implicits.
If you want to learn more on this topic, check this great guide.
You should define the fields as parameters of you case class.
final case class AppConfig(host: String, port: Int)
Then you overload the apply method of your companion object
object AppConfig {
def apply(config: Config): AppConfig = {
val host = config.getString("projectname.application.host")
val port = config.getInt("projectname.application.port")
AppConfig(host, port)
}
}
However the simplest way to process configuration with case classes is to use pureconfig.
I want to use this config variable across application, rather than loading application.conf file everytime.
Just put it in an object, like
object MyConfig {
lazy val config = AppConfig(ConfigFactory.load("application.conf"))
}
I want to have one bootstrap function which will parse this config one time, making it available across application
As soon as you call MyConfig.config it is loaded just once - as object is a Singleton. So no special bootstrap is needed.
The pattern you're trying to achieve is called Dependency Injection. From Martin Fowler's post on this topic
The basic idea of the Dependency Injection is to have a separate object, an assembler, that populates a field in the lister class with an appropriate implementation for the finder interface.
Register this configuration instance in a Dependency Injection tool like Guice.
class AppModule(conf: AppConfiguration) extends AbstractModule {
override def configure(): Unit = {
bind(classOf[AppConfiguration]).toInstance(conf)
}
}
....
// somewhere in the code
import com.google.inject.Inject
class FooClass #Inject() (config: AppConfiguration)

How do I get circe to have an either/or output scenario for the generated Json?

Here's what I intend - let's say I have a field called medical_payments - it can "either" be a limit if one elects or waived
{
"medical_payments":
{
"limit_value":"one_hundred"
}
}
If it's elected as a waiver then it should be:
{
"medical_payments":
{
"waived":true
}
}
So far here's what I have:
sealed trait LimitOrWaiver
case class Limit(limit_key: String) extends LimitOrWaiver
case class Waived(waived: Boolean) extends LimitOrWaiver
case class Selection(medical_payments: LimitOrWaiver)
Sample data:
Selection(medical_payments = Limit("one_hundred")).asJson
Output:
{
"medical_payments":
{
"Limit": { "limit_value":"one_hundred" } // additional object added
}
}
Similarly for Selection(medical_payments = Waived(true)).asJson an additional Waived:{...} is added to the Json.
I'd like it to be an either/or. What's the best way to achieve this?
The only way that I've been able to think of (not to my liking) is to use forProductN functions per the doc and manually do all this - but it's way to cumbersome for a large Json.
You can almost accomplish this with generic derivation using the configuration in generic-extras:
sealed trait LimitOrWaiver
case class Limit(limitValue: String) extends LimitOrWaiver
case class Waived(waived: Boolean) extends LimitOrWaiver
case class Selection(medicalPayments: LimitOrWaiver)
import io.circe.generic.extras.Configuration, io.circe.generic.extras.auto._
import io.circe.syntax._
implicit val codecConfiguration: Configuration =
Configuration.default.withDiscriminator("type").withSnakeCaseMemberNames
And then:
scala> Selection(medicalPayments = Limit("one_hundred")).asJson
res0: io.circe.Json =
{
"medical_payments" : {
"limit_value" : "one_hundred",
"type" : "Limit"
}
}
(Note that I've also changed the Scala case class member names to be Scala-idiomatic camel-case, and am handling the transformation to snake-case in the configuration.)
This isn't exactly what you want, since there's that extra type member, but circe's generic derivation only supports round-trip-able encoders / decoders, and without a discriminator of some kind—either a member like this or the extra object layer you point out in the question—it's impossible to round-trip values of arbitrary ADTs through JSON.
This might be fine—you might not care about the extra type in your object. If you do care, you can still use derivation with a little extra work:
import io.circe.generic.extras.Configuration, io.circe.generic.extras.auto._
import io.circe.generic.extras.semiauto._
import io.circe.ObjectEncoder, io.circe.syntax._
implicit val codecConfiguration: Configuration =
Configuration.default.withDiscriminator("type").withSnakeCaseMemberNames
implicit val encodeLimitOrWaiver: ObjectEncoder[LimitOrWaiver] =
deriveEncoder[LimitOrWaiver].mapJsonObject(_.remove("type"))
And:
scala> Selection(medicalPayments = Limit("one_hundred")).asJson
res0: io.circe.Json =
{
"medical_payments" : {
"limit_value" : "one_hundred"
}
}
If you really wanted you could even make this automatic, so that type would be removed from any ADT encoders you derive.

How to list annotations (custom java ones and others) on field members of Scala case class

So I'm trying to list fields with specific annotation in a Scala case class and I'm not able to get it working... Let's see come code right away
The case class (it's a simplified version of it, mine extends another class and is also nested in my test class where I use it for unit testing only):
case class Foo(#Unique var str: String) {}
The custom Java annotation:
#Retention(RetentionPolicy.RUNTIME)
#Target({ElementType.FIELD, ElementType.PARAMETER})
public #interface Unique {}
And my class (simplified again) where I'm trying to do some stuffs with fields marked as unique
class SomeClass[T] (implicit typeTag: TypeTag[T]) {
val fields: Iterable[universe.TermSymbol] = typeOf(typeTag).members.collect { case s: TermSymbol => s }.
filter(s => s.isVal || s.isVar)
val list = fields.flatMap(f => f.annotations.find(_.tpe =:= TypeOf[Unique]).((f, _))).toList
}
But the val list in the last peace of code is always empty... fields has str listed in but without the annotation.
What am I missing?
The code listing the annotations is from the following answer:
How to list all fields with a custom annotation using Scala's reflection at runtime?
Seems the reference post is Scala 2.10 is old and is not compatible with the newest Scala version.
There is an example for how to get the specify annotation by type.
def listProperties[T: TypeTag]: List[universe.Annotation] = {
typeOf[T].typeSymbol.asClass
.asClass
.primaryConstructor
.typeSignature
.paramLists.flatten.flatMap(_.annotations)
}
val annotations = listProperties[Foo].filter(_.tree.tpe =:= typeOf[Unique])
println(annotations)
and there is way to get the annotation's field value:
case class Foo(#Unique(field = "bar") val str: String) {}
import scala.reflect.runtime.currentMirror
import scala.tools.reflect.ToolBox
val tb = currentMirror.mkToolBox()
val result = tb.eval(tb.untypecheck(head.tree)).asInstanceOf[Unique]
and need to call out your annotation class is implemented by using Java style, In Scala maybe you want to use StaticAnnotation for creating Annotation, like:
class Unique extends StaticAnnotation

Mongodb scala driver codec for trait and inherited classes

Using the following mongo-driver. I want to insert and get structures (see below) from MongoDB.
trait A {
def id: ObjectId
}
case class B(id: ObjectId) extends A
case class C(id: ObjectId, name: String) extends A
I find a solution with using sealed classes, but I want to use traits.
I want to find a solution with Codecs or something else.
I had the same concern just a few days ago but didn't find anything in the documentation regarding sealed traits for modeling ADT in MongoDB.
In the end, I used sealed class as suggested in the official scala driver github repo.
If you really want to use traits (due to the definition of abstract methods) you can do something like this:
package example.model
import example.model.adt._
import org.mongodb.scala.bson.ObjectId
import org.mongodb.scala.bson.codecs.Macros._
import org.mongodb.scala.bson.codecs.DEFAULT_CODEC_REGISTRY
import org.bson.codecs.configuration.CodecRegistries.{fromProviders, fromRegistries}
trait MongoModel {
def _id: ObjectId
}
object MongoModel {
val codecRegistery = fromRegisteries(fromProviders(classOf[A]), DEFAULT_CODEC_REGISTRY)
}
Now you can have your ADT for A defined with sealed class.
package example.model.adt
import example.model.MongoModel
import org.mongodb.scala.bson.ObjectId
sealed class A
final case class B(_id: ObjectId) extends A with MongoModel
final case class C(_id: ObjectId) extends A with MongoModel
This answer doesn't solve the question directly but provides a feasible workaround. Note that this code is just an example. For a more complete implementation, you can see this github repo.
Since release 2.7, the mongodriver is now able to serialize sealed traits.
It works exactly like serializing a sealed classes.