I have a Java program that I must implement in Scala, but I am extremely new to Scala. After reading a number of SO question & answers as well as reading through a number of Google-retrieved resources on case classes, I am still having trouble grasping how to acquire a reference to the message I received? Example code is below:
case class SpecialMessage(key: Int) {
val id: Int = Main.idNum.getAndIncrement().intValue()
def getId(): Int = {
return id
}
}
Then in another class's receive I am trying to reference that number with:
def receive() = {
case SpecialMessage(key) {
val empID = ?? getId() // Get the id stored in the Special Message
// Do stuff with empID
}
}
I cannot figure out what to put on the right sight of empID = in order to get that id. Is this really simple, or something that isn't normally done?
These are 2 ways to do what you want, pick the one that suits best
case msg: SpecialMessage => {
val empID = msg.getId() // Get the id stored in the Special Message
// Do stuff with empID
}
case msg # SpecialMessage(key) => {
val empID = msg.getId() // Get the id stored in the Special Message
// Do stuff with empID
}
Pim's answer is good.
But maybe you can modify the structure of SpecialMessage like
case class SpecialMessage(key: Int,val id: Int = Main.idNum.getAndIncrement().intValue())
so you can get id directly from pattern matching.
def receive() = {
case SpecialMessage(key, empID) {
// Do stuff with empID
}
}
Related
I'm consuming Avro serialized messages from Kafka using the "automatic" deserializer like:
props.put(
ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
"io.confluent.kafka.serializers.KafkaAvroDeserializer"
);
props.put("schema.registry.url", "https://example.com");
This works brilliantly, and is right out of the docs at https://docs.confluent.io/current/schema-registry/serializer-formatter.html#serializer.
The problem I'm facing is that I actually just want to forward these messages, but to do the routing I need some metadata from inside. Some technical constraints mean that I can't feasibly compile-in generated class files to use the KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG => true, so I am using a regular decoder without being tied into Kafka, specifically just reading the bytes as a Array[Byte] and passing them to a manually constructed deserializer:
var maxSchemasToCache = 1000;
var schemaRegistryURL = "https://example.com/"
var specificDeserializerProps = Map(
"schema.registry.url"
-> schemaRegistryURL,
KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG
-> "false"
);
var client = new CachedSchemaRegistryClient(
schemaRegistryURL,
maxSchemasToCache
);
var deserializer = new KafkaAvroDeserializer(
client,
specificDeserializerProps.asJava
);
The messages are a "container" type, with the really interesting part one of about ~25 types in a union { A, B, C } msg record field:
record Event {
timestamp_ms created_at;
union {
Online,
Offline,
Available,
Unavailable,
...
...Failed,
...Updated
} msg;
}
So I'm successfully reading a Array[Byte] into record and feeding it into the deserializer like this:
var genericRecord = deserializer.deserialize(topic, consumerRecord.value())
.asInstanceOf[GenericRecord];
var schema = genericRecord.getSchema();
var msgSchema = schema.getField("msg").schema();
The problem however is that I can find no to discern, discriminate or "resolve" the "type" of the msg field through the union:
System.out.printf(
"msg.schema = %s msg.schema.getType = %s\n",
msgSchema.getFullName(),
msgSchema.getType().name());
=> msg.schema = union msg.schema.getType = union
How to discriminate types in this scenario? The confluent registry knows, these things have names, they have "types", even if I'm treating them as GenericRecords,
My goal here is to know that record.msg is of "type" Online | Offline | Available rather than just knowing it's a union.
After having looked into the implementation of the AVRO Java library, it think it's safe to say that this is impossible given the current API. I've found the following way of extracting the types while parsing, using a custom GenericDatumReader subclass, but it needs a lot of polishing before I'd use something like this in production code :D
So here's the subclass:
import org.apache.avro.Schema;
import org.apache.avro.generic.GenericData;
import org.apache.avro.generic.GenericDatumReader;
import org.apache.avro.io.ResolvingDecoder;
import java.io.IOException;
import java.util.List;
public class CustomReader<D> extends GenericDatumReader<D> {
private final GenericData data;
private Schema actual;
private Schema expected;
private ResolvingDecoder creatorResolver = null;
private final Thread creator;
private List<Schema> unionTypes;
// vvv This is the constructor I've modified, added a list of types
public CustomReader(Schema schema, List<Schema> unionTypes) {
this(schema, schema, GenericData.get());
this.unionTypes = unionTypes;
}
public CustomReader(Schema writer, Schema reader, GenericData data) {
this(data);
this.actual = writer;
this.expected = reader;
}
protected CustomReader(GenericData data) {
this.data = data;
this.creator = Thread.currentThread();
}
protected Object readWithoutConversion(Object old, Schema expected, ResolvingDecoder in) throws IOException {
switch (expected.getType()) {
case RECORD:
return super.readRecord(old, expected, in);
case ENUM:
return super.readEnum(expected, in);
case ARRAY:
return super.readArray(old, expected, in);
case MAP:
return super.readMap(old, expected, in);
case UNION:
// vvv The magic happens here
Schema type = expected.getTypes().get(in.readIndex());
unionTypes.add(type);
return super.read(old, type, in);
case FIXED:
return super.readFixed(old, expected, in);
case STRING:
return super.readString(old, expected, in);
case BYTES:
return super.readBytes(old, expected, in);
case INT:
return super.readInt(old, expected, in);
case LONG:
return in.readLong();
case FLOAT:
return in.readFloat();
case DOUBLE:
return in.readDouble();
case BOOLEAN:
return in.readBoolean();
case NULL:
in.readNull();
return null;
default:
return super.readWithoutConversion(old, expected, in);
}
}
}
I've added comments to the code for the interesting parts, as it's mostly boilerplate.
Then you can use this custom reader like this:
List<Schema> unionTypes = new ArrayList<>();
DatumReader<GenericRecord> datumReader = new CustomReader<GenericRecord>(schema, unionTypes);
DataFileReader<GenericRecord> dataFileReader = new DataFileReader<GenericRecord>(eventFile, datumReader);
GenericRecord event = null;
while (dataFileReader.hasNext()) {
event = dataFileReader.next(event);
}
System.out.println(unionTypes);
This will print, for each union parsed, the type of that union. Note that you'll have to figure out which element of that list is interesting to you depending on how many unions you have in a record, etc.
Not pretty tbh :D
I was able to come up with a single-use solution after a lot of digging:
val records: ConsumerRecords[String, Array[Byte]] = consumer.poll(100);
for (consumerRecord <- asScalaIterator(records.iterator)) {
var genericRecord = deserializer.deserialize(topic, consumerRecord.value()).asInstanceOf[GenericRecord];
var msgSchema = genericRecord.get("msg").asInstanceOf[GenericRecord].getSchema();
System.out.printf("%s \n", msgSchema.getFullName());
Prints com.myorg.SomeSchemaFromTheEnum and works perfectly in my use-case.
The confusing thing, is that because of the use of GenericRecord, .get("msg") returns Object, which, in a general way I have no way to safely typecast. In this limited case, I know the cast is safe.
In my limited use-case the solution in the 5 lines above is suitable, but for a more general solution the answer https://stackoverflow.com/a/59844401/119669 posted by https://stackoverflow.com/users/124257/fresskoma seems more appropriate.
Whether using DatumReader or GenericRecord is probably a matter of preference and whether the Kafka ecosystem is in mind, alone with Avro I'd probably prefer a DatumReader solution, but in this instance I can live with having Kafak-esque nomenclature in my code.
To retrieve the schema of the value of a field, you can use
new GenericData().induce(genericRecord.get("msg"))
My Google-fu is letting me down, so I'm hoping you can help
I'm building some webservices is the play framework using scala and anorm for database access
One of my calls is to update an existing row in a database - i.e run a query like
UPDATE [Clerks]
SET [firstName] = {firstName}
,[lastName] = {lastName}
,[login] = {login}
,[password] = {password}
WHERE [id] = {id}
My method receives a clerk object BUT all the parameters are optional (except the id of course) as they may only wish to update a single column of the row like so
UPDATE [Clerks]
SET [firstName] = {firstName}
WHERE [id] = {id}
So I want the method to check which clerk params are defined and build the 'SET' part of the update statement accordingly
It seems like there should be a better way than to go through each param of the clerk object, check if it is defined and build the query string - but I've been unable to find anything on the topic so far.
Does anyone have any suggestions how this is best handled
As the commenters mentioned it appears to not be possible - you must build the query string yourself.
I found that examples around this lacking and it took more time to resolve this than it should have (I'm new to scala and the play framework, so this has been a common theme)
in the end this is what I implemented:
override def updateClerk(clerk: Clerk) = {
var setString: String = "[modified] = {modified}"
var params: collection.mutable.Seq[NamedParameter] =
collection.mutable.Seq(
NamedParameter("modified", toParameterValue(System.currentTimeMillis / 1000)),
NamedParameter("id", toParameterValue(clerk.id.get)))
if (clerk.firstName.isDefined) {
setString += ", [firstName] = {firstName}"
params = params :+ NamedParameter("firstName", toParameterValue(clerk.firstName.getOrElse("")))
}
if (clerk.lastName.isDefined) {
setString += ", [lastName] = {lastName}"
params = params :+ NamedParameter("lastName", toParameterValue(clerk.lastName.getOrElse("")))
}
if (clerk.login.isDefined) {
setString += ", [login] = {login}"
params = params :+ NamedParameter("login", toParameterValue(clerk.login.getOrElse("")))
}
if (clerk.password.isDefined) {
setString += ", [password] = {password}"
params = params :+ NamedParameter("password", toParameterValue(clerk.password.getOrElse("")))
}
if (clerk.supervisor.isDefined) {
setString += ", [isSupervisor] = {supervisor}"
params = params :+ NamedParameter("supervisor", toParameterValue(clerk.supervisor.getOrElse(false)))
}
val result = DB.withConnection { implicit c =>
SQL("UPDATE [Clerks] SET " + setString + " WHERE [id] = {id}").on(params:_*).executeUpdate()
}
}
it likely isn't the best way to do this, however I found it quite readable and the parameters are properly handled in the prepared statement.
Hopefully this can benefit someone running into a similar issue
If anyone wants to offer up improvements, they'd be gratefully received
Since roughly 2.6.0 this is possible directly with anorm using their macros, http://playframework.github.io/anorm/#generated-parameter-conversions
Here is my example:
case class UpdateLeagueFormInput(transferLimit: Option[Int], transferWildcard: Option[Boolean], transferOpen: Option[Boolean])
val input = UpdateLeagueFormInput(None, None, Some(true))
val toParams: ToParameterList[UpdateLeagueFormInput] = Macro.toParameters[UpdateLeagueFormInput]
val params = toParams(input)
val dynamicUpdates = params.map(p => {
val snakeName = camelToSnake(p.name)
s"$snakeName = CASE WHEN {${p.name}} IS NULL THEN l.$snakeName ELSE {${p.name}} END"
})
val generatedStmt = s"""UPDATE league l set ${dynamicUpdates.mkString(", ")} where league_id = ${league.leagueId}"""
SQL(generatedStmt).on(params: _*).executeUpdate()
producing:
UPDATE league l set transfer_limit = CASE WHEN null IS NULL THEN l.transfer_limit ELSE null END, transfer_wildcard = CASE WHEN null IS NULL THEN l.transfer_wildcard ELSE null END, transfer_open = CASE WHEN true IS NULL THEN l.transfer_open ELSE true END where league_id = 26;
Notes:
The camelToSnake function is just my own (There is an obvious ColumnNaming.SnakeCase available for parser rows, but I couldn't find something similar for parameter parsing)
My example string interpolates {league.leagueId}, when it could treat this as a parameter as well
Would be nice to avoid the redundant sets for null fields, however I don't think it's possible, and in my opinion clean code/messy auto-generated sql > messy code/clean auto-generated sql
I am working with the Lift framework and Scala. I have a form to sign up to my application, and I want to validate all the fields in it. I have a snippet where I access my form values, and one validation class where I wrote my validation functions. The following code is what I've tried so far. In my Snippet:
if(validationClassObject.validateName(first_name)){
if(validationClassObject.validateName(last_name)){
if(validationClassObject.validateEmail(email)){
if(validationClassObject.validateUserName(name)){
// Adding values to the DB
S.redirectTo("/")
}
else{
S.notice("Invalid User Name")
}
}
else{
S.notice("Invalid Mail Id")
}
}
else{
S.notice("Invalid Last name")
}
}
else{
S.notice("Invalid First Name")
}
In the validationClass I wrote the validation code looks like:
//function for validating mail address
def validateEmail(email: String): Boolean =
"""(\w+)#([\w\.]+)""".r.unapplySeq(email).isDefined
//code for validating remaining fileds like above
This is working, but I know this is not the best way of coding this operation in Scala. How could I modify my code in a more scalable way? How can I use case classes here?
You could do:
def av[T,V](validationFunction: => Boolean, error: => T)(f: => V)={
if(!validationFunction) error
else f
}
def v[V](validationFunction: => Boolean, error: => String)(f: => V)=av(validationFunction,S.notice(error))(f)
import validationCalssObject._
v(validateName(last_name),"Invalid Last name"){v(validateName(name),"Invalid User Name"){...}}
av is a abstract method with T and V as result types for the error function and continue function f. v is the more specific function what expects a string for error and encapsulates the notice() call. we give f as the part in the curly braces v(validation, errormsg){/*todo when there is no problem*/}.
I can't do formatting in comments so I'll post a new answer.
def badName() = if ("name" == "") Some("bad name") else None
def badEmail() = if ("email" == "") Some("bad email") else None
val verifications = List[() => Option[String]](badName, badEmail)
val failed = verifications.flatMap(_())
if (failed.nonEmpty) {
// handle failed
} else {
// your custom logic here
}
if (badName) S.notice
else if (badEmail) S.notice
else if (badDay) S.notice
else { // everything OK...
// return a JsCmd or what else do you wanted here
}
An alternative solution can be written using Option and flatMap, without these all "if"-s hardcoded. If you're interested in that -- ask..
I am trying to do a simple case of /author/ and get the Lift to build a Person object based on the id passed in.
Currently i have an Author snippet
class Author(item: Person) {
def render = {
val s = item match { case Full(item) => "Name"; case _ => "not found" }
" *" #> s;
}
}
object Author{
val menu = Menu.param[Person]("Author", "Author", authorId => findPersonById(authorId), person => getIdForPerson(person)) / "author"
def findPersonById(id:String) : Box[Person] = {
//if(id == "bob"){
val p = new Person()
p.name="Bobby"
p.age = 32
println("findPersonById() id = " +id)
Full(p)
//}else{
//return Empty
//}
}
def getIdForPerson(person:Person) : String = {
return "1234"
}
}
What i am attempting to do is get the code to build a boxed person object and pass it in to the Author class's constructor. In the render method i want determine if the box is full or not and proceed as appropriate.
If i change
class Author(item: Person) {
to
class Author(item: Box[Person]) {
It no longer works but if i leave it as is it is no longer valid as Full(item) is incorrect. If i remove the val s line it works (and replace the s with item.name). So how do i do this. Thanks
The Box returned from findPersonById(id:String) : Box[Person] is evaluated and if the Box is Full, the unboxed value is passed into your function. If the Box is Empty or Failure the application will present a 404 or appropriate error page instead.
You can try double boxing your return if you want to handle this error checking yourself (so that the result of this method is always a Full Box).
def findPersonById(id:String) : Box[Box[Person]] = {
if(id == "bob"){
val p = new Person()
p.name="Bobby"
p.age = 32
println("findPersonById() id = " +id)
Full(Full(p))
}else{
return Full(Empty)
}
}
and then this should work:
class Author(item: Box[Person])
I want to copy object properties to another object in a generic way (if a property exists on target object, I copy it from the source object).
My code works fine using ExpandoMetaClass, but I don't like the solution. Are there any other ways to do this?
class User {
String name = 'Arturo'
String city = 'Madrid'
Integer age = 27
}
class AdminUser {
String name
String city
Integer age
}
def copyProperties(source, target) {
target.properties.each { key, value ->
if (source.metaClass.hasProperty(source, key) && key != 'class' && key != 'metaClass') {
target.setProperty(key, source.metaClass.getProperty(source, key))
}
}
}
def (user, adminUser) = [new User(), new AdminUser()]
assert adminUser.name == null
assert adminUser.city == null
assert adminUser.age == null
copyProperties(user, adminUser)
assert adminUser.name == 'Arturo'
assert adminUser.city == 'Madrid'
assert adminUser.age == 27
I think the best and clear way is to use InvokerHelper.setProperties method
Example:
import groovy.transform.ToString
import org.codehaus.groovy.runtime.InvokerHelper
#ToString
class User {
String name = 'Arturo'
String city = 'Madrid'
Integer age = 27
}
#ToString
class AdminUser {
String name
String city
Integer age
}
def user = new User()
def adminUser = new AdminUser()
println "before: $user $adminUser"
InvokerHelper.setProperties(adminUser, user.properties)
println "after : $user $adminUser"
Output:
before: User(Arturo, Madrid, 27) AdminUser(null, null, null)
after : User(Arturo, Madrid, 27) AdminUser(Arturo, Madrid, 27)
Note: If you want more readability you can use category
use(InvokerHelper) {
adminUser.setProperties(user.properties)
}
I think your solution is quite good and is in the right track. At least I find it quite understandable.
A more succint version of that solution could be...
def copyProperties(source, target) {
source.properties.each { key, value ->
if (target.hasProperty(key) && !(key in ['class', 'metaClass']))
target[key] = value
}
}
... but it's not fundamentally different. I'm iterating over the source properties so I can then use the values to assign to the target :). It may be less robust than your original solution though, as I think it would break if the target object defines a getAt(String) method.
If you want to get fancy, you might do something like this:
def copyProperties(source, target) {
def (sProps, tProps) = [source, target]*.properties*.keySet()
def commonProps = sProps.intersect(tProps) - ['class', 'metaClass']
commonProps.each { target[it] = source[it] }
}
Basically, it first computes the common properties between the two objects and then copies them. It also works, but I think the first one is more straightforward and easier to understand :)
Sometimes less is more.
Another way is to do:
def copyProperties( source, target ) {
[source,target]*.getClass().declaredFields*.grep { !it.synthetic }.name.with { a, b ->
a.intersect( b ).each {
target."$it" = source."$it"
}
}
}
Which gets the common properties (that are not synthetic fields), and then assigns them to the target
You could also (using this method) do something like:
def user = new User()
def propCopy( src, clazz ) {
[src.getClass(), clazz].declaredFields*.grep { !it.synthetic }.name.with { a, b ->
clazz.newInstance().with { tgt ->
a.intersect( b ).each {
tgt[ it ] = src[ it ]
}
tgt
}
}
}
def admin = propCopy( user, AdminUser )
assert admin.name == 'Arturo'
assert admin.city == 'Madrid'
assert admin.age == 27
So you pass the method an object to copy the properties from, and the class of the returned object. The method then creates a new instance of this class (assuming a no-args constructor), sets the properties and returns it.
Edit 2
Assuming these are Groovy classes, you can invoke the Map constructor and set all the common properties like so:
def propCopy( src, clazz ) {
[src.getClass(), clazz].declaredFields*.grep { !it.synthetic }.name.with { a, b ->
clazz.metaClass.invokeConstructor( a.intersect( b ).collectEntries { [ (it):src[ it ] ] } )
}
}
Spring BeanUtils.copyProperties will work even if source/target classes are different types. http://docs.spring.io/autorepo/docs/spring/3.2.3.RELEASE/javadoc-api/org/springframework/beans/BeanUtils.html