I would like to extend the BsonRecord class to handles some of its fields when they are filled in. I'm trying to do it by extending the setFieldsFrom... methods, but it doesn't seem to work...
Here is the code I have :
trait NodeBsonRecord[MyType <: BsonRecord[MyType]] extends BsonRecord[MyType]
{
self: MyType =>
override def setFieldsFromDBObject(dbo:DBObject) =
{
super.setFieldsFromDBObject(dbo)
println("setFieldsFromDBObject")
}
override def setFieldsFromJSON(json:String) =
{
val out = super.setFieldsFromJSON(json)
println("setFieldsFromJSON")
out
}
override def setFieldsFromJsonString(json:String) =
{
val out = super.setFieldsFromJsonString(json)
println("setFieldsFromJsonString")
out
}
override def setFieldsFromJValue(jval:JValue) =
{
val out = super.setFieldsFromJValue(jval)
println("setFieldsFromJValue")
out
}
override def setFieldsFromReq(req:Req) =
{
val out = super.setFieldsFromReq(req)
println("setFieldsFromReq")
out
}
}
So when I request for a Record (using MongoRecord.find()), I expect to see a "setFieldFrom..." thing, but nothing is printed out...
Anybody can tell me how to do this ?
Mongo seems to use setFieldsFromDBObject in BsonMetaRecord as part of find, which iterates through each field and calls setFromAny.
Related
I need to write two functions to get the output format and the output index for file conversion. As part of this, I wrote a TransformSettings class for these methods and set the default value. And in the transformer class, I created a new object of TransformSettings class to get the default values for each job run. Also, I have another class called ParquetTransformer that extends Transformer where I want to change these default values. So I implemented like below.
class TransformSettings{
def getOuputFormat: String = {
"orc"
}
def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("orc.column.index.access")
}
}
class Transformer{
def getTransformSettings: TransformSettings = {
new TransformSettings
}
def posttransform(table: AWSGlueDDL.Table):Dateframe ={
val indexAccess = getTransformSettings.getOuputIndex(table: AWSGlueDDL.Table)
........
}
}
class ParquetTransformer extends Transformer{
override def getTransformSettings: TransformSettings = {
val transformSettings = new TransformSettings {
override def getOuputFormat: String = {
"parquet"
}
override def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("parquet.column.index.access")
}
}
}
}
Is there a way to avoid creating a brand new object of TransformSettings in Transfomer class every time this is called?
Also is there a way to rewrite the code using Scala value class?
As #Dima proposed in the comments try to make TransformSettings a field / constructor parameter (a val) in the class Transformer and instantiate them outside
class TransformSettings{
def getOuputFormat: String = {
"orc"
}
def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("orc.column.index.access")
}
}
class Transformer(val transformSettings: TransformSettings) {
def posttransform(table: AWSGlueDDL.Table): DataFrame ={
val indexAccess = transformSettings.getOuputIndex(table: AWSGlueDDL.Table)
???
}
}
val parquetTransformSettings = new TransformSettings {
override def getOuputFormat: String = {
"parquet"
}
override def getOuputIndex(table: AWSGlueDDL.Table): Option[String] = {
table.StorageDescriptor.SerdeInfo.Parameters.get("parquet.column.index.access")
}
}
class ParquetTransformer extends Transformer(parquetTransformSettings)
You don't seem to need value classes (... extends AnyVal) now. They are more about unboxing, not about life-cycle management. TransformSettings and Transformer can't be value classes because they are not final (you're extending them in class ParquetTransformer extends Transformer... and new TransformSettings { ... }). By the way, value classes have many limatations
https://failex.blogspot.com/2017/04/the-high-cost-of-anyval-subclasses.html
https://github.com/scala/bug/issues/12271
Besides value classes, there are scala-newtype library in Scala 2 and opaque types in Scala 3.
I'm trying to re-create similar functionality of Scala's [RestartSink][1] feature.
I've come up with this code. However, since we only return a SinkShape instead of a Sink, I'm having trouble specifying that it should return a Future[Done] instead of NotUsed as it's materialized type. However, I'm confused about how to do that. I'm only able to have it return [MessageActionPair, NotUsed] instead of the desired [MessageActionPair, Future[Done]]. I'm still learning my way around this framework, so I'm sure I'm missing something small. I tried calling Source.toMat(RestartWithBackoffSink...), however that doesn't give the desired result either.
private final class RestartWithBackoffSink(
sourcePool: Seq[SqsEndpoint],
minBackoff: FiniteDuration,
maxBackoff: FiniteDuration,
randomFactor: Double) extends GraphStage[SinkShape[MessageActionPair]] { self ⇒
val in = Inlet[MessageActionPair]("RoundRobinRestartWithBackoffSink.in")
override def shape = SinkShape(in)
override def createLogic(inheritedAttributes: Attributes) = new RestartWithBackoffLogic(
"Sink", shape, minBackoff, maxBackoff, randomFactor, onlyOnFailures = false) {
override protected def logSource = self.getClass
override protected def startGraph() = {
val sourceOut = createSubOutlet(in)
Source.fromGraph(sourceOut.source).runWith(createSink(getEndpoint))(subFusingMaterializer)
}
override protected def backoff() = {
setHandler(in, new InHandler {
override def onPush() = ()
})
}
private def createSink(endpoint: SqsEndpoint): Sink[MessageActionPair, Future[Done]] = {
SqsAckSink(endpoint.queue.url)(endpoint.client)
}
def getEndpoint: SqsEndpoint = {
if(isTimedOut) {
index = (index + 1) % sourcePool.length
restartCount = 0
}
sourcePool(index)
}
backoff()
}
}
Syntax error here, since types don't match:
def withBackoff[T](minBackoff: FiniteDuration, maxBackoff: FiniteDuration, randomFactor: Double, sourcePool: Seq[SqsEndpoint]): Sink[MessageActionPair, Future[Done]] = {
Sink.fromGraph(new RestartWithBackoffSink(sourcePool, minBackoff, maxBackoff, randomFactor))
}
By extending extends GraphStage[SinkShape[MessageActionPair]] you are defining a stage with no materialized value. Or better you define a stage that materializes to NotUsed.
You have to decide if your stage can materialize into anything meaningful. More on materialized values for stages here.
If so: you have to extend GraphStageWithMaterializedValue[SinkShape[MessageActionPair], Future[Done]] and properly override the createLogicAndMaterializedValue function. More guidance can be found in the docs.
If not: you can change your types as per below
def withBackoff[T](minBackoff: FiniteDuration, maxBackoff: FiniteDuration, randomFactor: Double, sourcePool: Seq[SqsEndpoint]): Sink[MessageActionPair, NotUsed] = {
Sink.fromGraph(new RestartWithBackoffSink(sourcePool, minBackoff, maxBackoff, randomFactor))
}
I am new to Scala, trying to understand the syntactic behaviors of Scala. I will be really appreciated if anybody help me. Thanks
With Anonymous Object: Here in this scenario if I want to print the value of resinside the main function body then what logic I need to apply?
package oops
object AnonymousObject
{
def main(args:Array[String]):Unit =
{
new student().detail(5,9) // Line 1
}
}
class student
{
def detail(x:Int, y:Int):Int =
{
val res = x*y
println(res)
}
}
Without Anonymous Object:
For more information, in this scenario given below, there has no problem to achieve it because of var s
class Student
{
var id:Int = 0; // All fields must be initialized
var name:String = null;
}
object MainObject
{
def main(args:Array[String])
{
var s = new Student() // Creating an object
println(s.id+" "+s.name);
}
}
An object which has no reference name. So simply you can print like this way inside the main
object AnonymousObject
{
def main(args:Array[String]):Unit =
{
val res = new student().detail(5,9)
println(res)
}
}
class student
{
def detail(x:Int, y:Int):Int =
{
x*y
}
}
Output: 45
I have the following test:
class Foo extends mutable.SpecificationWithJUnit {
sequential
"this example should run before the 'After' method" in new Context {
bar must beSome
}
class Context extends mutable.BeforeAfter with mutable.Around {
override def apply[T : AsResult](a: =>T): Result = {
lazy val result = super[Around].apply(a)
super[BeforeAfter].apply(result)
}
override def delayedInit(x: => Unit): Unit = around { try { before; x; Success() } finally { after }}
#Resource var barReader : BarReader = _
val bar = barReader.someBar
override def before : Any = { //some stuff}
def after: Any = {
bar = None
}
override def around[T : AsResult](t: =>T) = {
//spring context injection logic
AsResult.effectively(t)
}
}
}
}
I expect this test to pass but in reality what happens is that because of the delayed init, the after runs before the example. If I change the Context to a trait I lose the delayed init functionality. Is this a bug or am I doing something wrong?
**Edited:
This example will throw an NPE when the Context is a trait. What I expect to happen is that because of the delayed-init, the Context's constructor, which consequentially means the barReader.someBar will run only after the barReader has been injected.
Thanks
Netta
You should use a trait instead of a class for Context. If you use a class, delayedInit (hence after) will be triggered twice. Once for the body of the Context class and another time for the body of the anonymous class new Context. With a trait you don't get such a behavior:
class Foo extends mutable.SpecificationWithJUnit {
sequential
"this example should run before the 'After' method" in new Context {
bar must beSome
}
trait Context extends mutable.After {
var bar : Option[String] = Some("bar")
def after: Any = bar = None
}
}
Simple answer, looks like this can't be done.
Let's say I've got two traits, one of them being a factory for another:
trait BaseT {
val name: String
def introduceYourself() = println("Hi, I am " +name)
// some other members ...
}
trait BaseTBuilder {
def build: BaseT
}
Now, I want to extend BaseT:
trait ExtendedT extends BaseT {
val someNewCoolField: Int
override def introduceYourself() = {
super.introduceYourself()
println(someNewCoolField)
}
// some other extra fields
Let's say I know how to initialize the new fields, but I'd like to use BaseTBuilder for initializing superclass members. Is there a possibility to create a trait that would be able to instantiate ExtendedT somehow? This approach obviously fails:
trait ExtendedTBuilder { self: TBuilder =>
def build: ExtendedT = {
val base = self.build()
val extended = base.asInstanceOf[ExtendedT] // this cannot work
extended.someNewCoolField = 4 // this cannot work either, assignment to val
extended
}
def buildDifferently: ExtendedT = {
new ExtendedT(4) // this fails, we don't know anything about constructors of ExtendedT
}
def build3: ExtendedT = {
self.build() with {someNewCoolField=5} //that would be cool, but it cannot work either
}
}
I'd like to have such a set of traits (or objects) that when someone supplies concrete implementation of BaseT and BaseTBuilder I could instantiantiate ExtendedT by writing:
val extendedBuilder = new ConcreteBaseTBuilder with ExtendedTBuilder
val e: ExtendedT = extendedBuilder.build
ExtendedT could contain a field of type BaseT, but then it would require manually proxying all the necessary methods and fields, which is in my opinion a violation of DRY principle. How to solve that?
How about create ExtendBaseT instance in your ExtendBaseTBuilder
trait ExtendBaseTBuilder { self : BaseTBuilder =>
def build: ExtendBaseT = {
new ExtendBaseT {
val someNewCoolField: Int = 3
}
}
}