I have two enum classes defined as the following.
object Rating extends Enumeration {
type Rating = Value
val LOW: Value = Value("Low")
val AVERAGE: Value = Value("Average")
val HIGH: Value = Value("High")
}
object Reason extends Enumeration {
type Reason = Value
val MISSING_PARTS: Value = Value(1)
val WRONG_ITEM: Value = Value(2)
val DEFECTIVE_ITEM: Value = Value(3)
}
How can I get the Rating based on the String values "Low", "Average" and "High"
How can I get the Reason based on the Integer values 1, 2, 3
How can I get the Reason based on the val name MISSING_PARTS, WRONG_ITEM, DEFECTIVE_ITEM
Please correct me if I am not using the correct terminology. I am from Java background and new to Scala. I did a lot of searches but either they are outdated or they use very trivial examples where the labels and values are the same strings, which does not help much.
Any help will be appreciated.
Rating.withName("Low")
Reason(1) (which is shorthand for Reason.apply(1))
Reason.withName("DEFECTIVE_ITEM")
Explanation:
Each enumeration value has an id and a name. Both are generated using a default unless you override them using the appropriate Value(...) overload. In this case, Rating has customized names and Reason has customized IDs. Given the code above, these are the names and ids assigned to each value:
val LOW: Value = Value("Low") // Id: 0; Name: Low
val AVERAGE: Value = Value("Average") // Id: 1; Name: Average
val HIGH: Value = Value("High") // Id: 2; Name: High
val MISSING_PARTS: Value = Value(1) // Id: 1; Name: MISSING_PARTS
val WRONG_ITEM: Value = Value(2) // Id: 2; Name: WRONG_ITEM
val DEFECTIVE_ITEM: Value = Value(3) // Id: 3; Name: DEFECTIVE_ITEM
Now, we can access specific values using either withName(name: String) or apply(id: Int) which references values based on names and ids respectively.
Related
Pojo classes
case class RulePojoMigration(val tolerance_id:Int,val asset_id:Int, measure: String, cond: String, function: String, threshold_value: String, rule_dts: String,
filter: util.List[Filter], is_enabled: String, mode: String, tolerance_status: String, action_key: String,
email_ids: util.List[String], rule_name:String, rule_owner:String, group_by: util.List[String], relax_variation_band:String, reconciliation_id:Int)
case class Filter(value: String, operator: String, dimension: String)
Below is the code used to fetch the object from DB
if (checkRuleIsEditOrNot) {
ps = con.prepareStatement(WSQueryConstant.RULE_MIGRATION_GET_COMPLETE_RULE_INFO_FROM_STAGE)
ps.setInt(1, rulePojo.tolerance_id)
rs = ps.executeQuery
var dbRulePojo: RulePojoMigration = null
val jsonRulePojo: RulePojoMigration = rulePojo
if (rs.next()) {
val rule_tolerance_asset_id = rs.getInt(2)
val measure = rs.getString(3)
val cond = rs.getString(4)
val function = rs.getString(5)
val threshold_value = rs.getString(6)
val rule_dts = rs.getString(7)
val filter = gson.fromJson(rs.getString(8), classOf[util.List[Filter]])
val is_enabled = rs.getString(9)
val mode = rs.getString(10)
val tolerance_status = rs.getString(11)
val email_ids = gson.fromJson(rs.getString(12), classOf[java.util.List[String]])
val rule_name = rs.getString(13)
val rule_owner = rs.getString(14)
val group_by = gson.fromJson(rs.getString(15), classOf[java.util.List[String]])
val relax_variation_band = rs.getString(16)
var reconciliation_id = rs.getString(17)
if(reconciliation_id== null)
reconciliation_id= "-1"
dbRulePojo = RulePojoMigration(jsonRulePojo.tolerance_id, rule_tolerance_asset_id, measure, cond, function, threshold_value, rule_dts, filter, is_enabled, mode, tolerance_status, jsonRulePojo.action_key, email_ids, rule_name, rule_owner, group_by, relax_variation_band, reconciliation_id.toInt)
user request with the below values.
RulePojoMigration(1274,1234,*,less than,count,100,2020-07-04 05:31:29,[Filter(Hello,equal,funnel_state)],1,static,null,EMAIL,[x],StaticRuleMigrationTesting,email,[ALL],0,-1)
Object available in MySQL database
RulePojoMigration(1274,1234,*,less than,count,100,2020-07-04 05:31:29,[{value=Hello, operator=equal, dimension=funnel_state}],1,static,null,EMAIL,[x],StaticRuleMigrationTesting,email,[ALL],0,-1)
So I want to check the equality of two objects so that the user can modify the the existing object if there is any changes in the payload or simply he/she should get a message saying duplicate entry exist. jsonRulePojo.equal(dbRulePojo) should be true, anyhow values are same of both object.
The default equals operation does not check the contents of the two classes, it just checks whether both values refer to the same object. So two instances of RulePojoMigration will always be different even if every field is the same. By default two classes are only equal if they are the same instance.
When a case class is defined, Scala will generate an equals that does check each of the fields passed to the constructor in turn. So two case classes will be equal if they have the same contents.
So you can fix this code in one of two ways:
Make RulePojoMigration a case class
Add an equals method to RulePojoMigration
I am able to resolve the bug by making the below changes.
From
val filter = gson.fromJson(rs.getString(8), classOf[util.List[Filter]])
To
val filter = gson.fromJson(rs.getString(8), classOf[Array[Filter]])
and
dbRulePojo = RulePojoMigration(jsonRulePojo.tolerance_id, rule_tolerance_asset_id, measure, cond, function, threshold_value, rule_dts, java.util.Arrays.asList(filter:_*)
, is_enabled, mode, tolerance_status, jsonRulePojo.action_key, email_ids, rule_name, rule_owner, group_by, relax_variation_band, reconciliation_id.toInt)
And instead of passing the filter directly into dbRulePojo i passed the java.util.Arrays.asList(filter:_*)
I have a data set test1.txt. It contain data like below
2::1::3
1::1::2
1::2::2
2::1::5
2::1::4
3::1::2
3::1::1
3::2::2
I have created data-frame using the below code.
case class Test(userId: Int, movieId: Int, rating: Float)
def pRating(str: String): Rating = {
val fields = str.split("::")
assert(fields.size == 3)
Test(fields(0).toInt, fields(1).toInt, fields(2).toFloat)
}
val ratings = spark.read.textFile("C:/Users/test/Desktop/test1.txt").map(pRating).toDF()
2,1,3
1,1,2
1,2,2
2,1,5
2,1,4
3,1,2
3,1,1
3,2,2
But I want to print output like below I.e. removing duplicate combinations and instead of field(2) value sum of values1,1, 2.0.
1,1,2.0
1,2,2.0
2,1,12.0
3,1,3.0
3,2,2.0
Please help me on this, how can achieve this.
To drop duplicates, use df.distinct. To aggregate you first groupBy and then agg. Putting this all together:
case class Rating(userId: Int, movieId: Int, rating: Float)
def pRating(str: String): Rating = {
val fields = str.split("::")
assert(fields.size == 3)
Rating(fields(0).toInt, fields(1).toInt, fields(2).toFloat)
}
val ratings = spark.read.textFile("C:/Users/test/Desktop/test1.txt").map(pRating)
val totals = ratings.distinct
.groupBy('userId, 'movieId)
.agg(sum('rating).as("rating"))
.as[Rating]
I am not sure you'd want the final result as Dataset[Rating] and whether the distinct and sum logic is exactly as you'd want it as the example in the question is not very clear but, hopefully, this will give you what you need.
ratings.groupBy("userId","movieId").sum(rating)
In scala, is there a way to access a class member,but the member name is a var?
Below is a code snippet, there is a class member called "one_star". I have a var whose value is "one_star", and I want use this var as the member name of the "case class".
case class Features(
// star about
var one_star: String = "0",
var two_star: String = "0",
var three_star: String = "0",
var four_star: String = "0",
var five_star: String = "0"
// other about
)
object Features {
def apply(): Features = {
val features = new Features()
var testVar = "one_star"
features.${testVar} = "1"
features
}
}
If you want to change field name dynamically, i.e. provide class variable name as value, find field that match given variable name and finally change the value for that field, there are several ways: the simple one is to use pattern match to check the field value and change instance value by yourself and return instance. However, it can be quite messy as you need to handle for every fields defined in your class and in case you have many fields, it can be quite cumbersome. Therefore, you will need some generic way to solve this problem.
Another approach is to use scala reflection. Scala reflection is designed for this, i.e. modifying your codes at runtime like your case and in more generic way. Following is a code snippet that change value of your instance for given field name.
case class Features(
// star about
var one_star: String = "0",
var two_star: String = "0",
var three_star: String = "0",
var four_star: String = "0"
// other about
) {
def copyInstance(variableName: String, value: String): Features = {
// 1. Initialize Features instance
val instance = Features()
// 2. Import scala reflection api.
import scala.reflect.runtime.{universe => ru}
// 3. Get the mirror of instance of Features class.
// Mirror will reflect to instance of Features case class, and we will use this instance mirror to modify its fields value.
val instanceMirror = ru.runtimeMirror(instance.getClass.getClassLoader).reflect(instance)
// 4. Now, Get the field whose value need to be changed - i.e. name you provide - variableName.
val field = ru.typeOf[Features].decl(ru.TermName(variableName)).asTerm
// 5. Get the mirror for that field so that we can read and write to this field.
val fieldMirror = instanceMirror.reflectField(field)
// 6. Finally, set the value to this field.
fieldMirror.set(value)
// 7. Return the changed instance.
instance
}
}
val features = Features()
val changedFeatures = features.copyInstance("one_star", "changed")
println(features)
println(changedFeatures)
//Result
Features(0,0,0,0)
Features(changed,0,0,0)
Also, note that you may need to handle the Exception for cases where invalid variable name and value is provided. In addition, if your case class contains >22 field parameters, certain features of case class are not available.
Scala is static type language and doesn't allow these language constructions. But you can use reflection (hard way) or pattern matching with code like this one (simple way):
class Features (
var one_star: String = "0",
var two_star: String = "0",
var three_star: String = "0",
var four_star: String = "0",
var five_star: String = "0") {
def setField(field: String, value: String): Unit = {
field match {
case "one_star" => one_star = value
case "two_star" => two_star = value
case "three_star" => three_star = value
case "four_star" => four_star = value
case "five_star" => five_star = value
}
}
}
This is possible using scala-reflect, although under most circumstances I would not recommend it.
import scala.reflect.runtime.universe._
val field = typeOf[Features].decl(TermName(testVar)).asTerm.accessed.asTerm
val mirror = runtimeMirror(classOf[Features].getClassLoader)
mirror.reflect(features).reflectField(field).set("1")
Are you sure you don't want to use or extend Map[String, String] for your class? So many properties is not typical.
Using Monocle I can define a Lens to read a case class member without issue,
val md5Lens = GenLens[Message](_.md5)
This can used to compare the value of md5 between two objects and fail with an error message that includes the field name when the values differ.
Is there a way to produce a user-friendly string from the Lens alone that identifies the field being read by the lens? I want to avoid providing the field name explicitly
val md5LensAndName = (GenLens[Message](_.md5), "md5")
If there is a solution that also works with lenses with more than one component then even better. For me it would be good even if the solution only worked to a depth of one.
This is fundamentally impossible. Conceptually, lens is nothing more than a pair of functions: one to get a value from object and one to obtain new object using a given value. That functions can be implemented by the means of accessing the source object's fields or not. In fact, even GenLens macro can use a chain field accessors like _.field1.field2 to generate composite lenses to the fields of nested objects. That can be confusing at first, but this feature have its uses. For example, you can decouple the format of data storage and representation:
import monocle._
case class Person private(value: String) {
import Person._
private def replace(
array: Array[String], index: Int, item: String
): Array[String] = {
val copy = Array.ofDim[String](array.length)
array.copyToArray(copy)
copy(index) = item
copy
}
def replaceItem(index: Int, item: String): Person = {
val array = value.split(delimiter)
val newArray = replace(array, index, item)
val newValue = newArray.mkString(delimiter)
Person(newValue)
}
def getItem(index: Int): String = {
val array = value.split(delimiter)
array(index)
}
}
object Person {
private val delimiter: String = ";"
val nameIndex: Int = 0
val cityIndex: Int = 1
def apply(name: String, address: String): Person =
Person(Array(name, address).mkString(delimiter))
}
val name: Lens[Person, String] =
Lens[Person, String](
_.getItem(Person.nameIndex)
)(
name => person => person.replaceItem(Person.nameIndex, name)
)
val city: Lens[Person, String] =
Lens[Person, String](
_.getItem(Person.cityIndex)
)(
city => person => person.replaceItem(Person.cityIndex, city)
)
val person = Person("John", "London")
val personAfterMove = city.set("New York")(person)
println(name.get(personAfterMove)) // John
println(city.get(personAfterMove)) // New York
While not very performant, that example illustrates the idea: Person class don't have city or address fields, but by wrapping data extractor and a string rebuild function into Lens, we can pretend it have them. For more complex objects, lens composition works as usual: inner lens just operates on extracted object, relying on outer one to pack it back.
I have a use-case where I need to define a new enum type LongShort but I need it in a way to also carry the sign so it can be directly used in mathematical expressions e.g.
object LongShortType extends Enumeration {
type Type = Value
val Long = Value(+1)
val Short = Value(-1)
}
I'd then like to use it like this:
val longShort = LongShortType.Short
val numberOfContracts: Int = 10
val vanillaOptionNotional: Double = longShort*numberOfContracts
but this leads to compiler error cannot resolve symbol * ... is there a way to extract the value of the enum? Or am I not understanding how enum types work?
The type of LongShortType.Short isn't Int, it's Value. You can either extract the underlying id of the value:
val longShort = LongShortType.Short.id
Which is a little ugly. Or you could not use an enum type at all:
object LongShortType {
val Long = 1
val Short = -1
}
And then your equation would work as is.
OK I worked out a solution to accomplish what I wanted without any compromisse and by that I mean that this solution has all the advantages of using Scala enum e.g. the withName and still allows me to define extra features on it:
object LongShortType extends Enumeration {
type Type = LongShortVal
val Long = Value("Long", +1)
val Short = Value("Short", -1)
case class LongShortVal(name: String, sign: Int) extends Val(nextId, name)
protected final def Value(name: String, sign: Int) = new LongShortVal(name, sign)
}
and now can do:
val longShort = LongShortType.Short
val numberOfContracts: Int = 10
val vanillaOptionNotional: Double = longShort.sign*numberOfContracts
and can also do:
val longShort = LongShort.withName("Long") // returns LongShort.Long