I have an object called Program which can only instantiate only one class, in my case val a = new A(), in it I have a local variable called localText, I want to convert it to global, I have created a variable outside the block called globalText and inside the block I assign it globalText = localText, when I do a println it shows it correctly "Hello World", I have another class called B that needs that value but I don't know how to access it because when I call it it appears with the null value and if I call the sum function what it returns is an Int. (The sum() def cannot be touched and the data comes like this from third party code) thank you very much for the help.
...
class A{
var globalText: String = _
def sum(): Int = {
val localText: String = "Hello World"
globalText = localText
println(globalText)
val b = new B()
3+4
}
println(globalText)
}
class B extends A{
var globalBText = globalText
println(globalBText)
}
object Program {
def main(args: Array[String]): Unit = {
val a = new A()
println(a.sum())
}
}
Output is
null
Hello World
null
null
7
...
This question has already been answered.
Please do not create duplicate posts with the same question. You can edit the previous question, if you feel like you missed something.
If you can't instantiate the class than this beats the whole purpose of being a class and you might as well model it as an object or create another object.
The idea is that a class must be instantiated in order for its fields to be initialized. There is no other way.
You could use an object so you don't have to instantiate any class at all, making sure that B.globalBText is initialized before you used it.
Here's what I mean:
object A {
var globalText: String = _
def sum(): Int = {
globalText = "Hello World"
3 + 4
}
}
object B {
var globalBText: String = {
A.sum()
A.globalText
}
println(globalBText)
}
object Program {
def main(args: Array[String]): Unit = {
println(B.globalBText)
}
}
Outputs:
Hello World
Hello World
Related
Primary goal
I want to use some static vals in a class so that I don't have to pass them as function parameters.
My approach
Since I want them to be static, I am declaring them in the companion object. But I cannot assign them values when I declare them, for some reasons. So I am following the below approach.
case class DemoParams(name: String)
class Demo {
def foo = {
println("Demo params name is: ", Demo.demoParams.name) // Works fine
anotherFoo(Demo.demoParams.name) // Throws NPE !
}
def anotherFoo(someName: String) = {
// some code
}
}
object Demo {
var demoParams: DemoParams = _ // Declare here
def apply() = new Demo()
def run = {
demoParams = DemoParams(name = "Salmon") // Define here
val demoObj = Demo()
demoObj.foo
}
def main() = {
run
}
}
Demo.main()
I am able to print Demo.demoParams but surprisingly, this throws a NullPointerException when I pass Demo.demoParams to another function, while running the Spark app on a cluster.
Questions
Firstly, is this the right way of declaring static values and defining them later? I would prefer to not use vars and use immutable vals. Is there a better alternative?
Second, could you think of any reason I would be getting a NPE while passing Demo.demoParams.name to another function?
Your code works fine and doesn't throw anything (after fixing a few compile errors).
But ... Don't do this, it's ... yucky :/
How about passing params to the class as ... well ... params instead?
case class Demo(params: DemoParams) {
def foo() = {
println("Demo params name is: " + params.name)
}
}
object Demo {
def run() = {
val demoParams = DemoParams(name = "Salmon")
val demoObj = Demo(demoParams)
demoObj.foo()
}
}
Not sure this is the best alternative, but consider using a trait, which still keeps you in the FP zone by avoiding the use of var:
case class DemoParams(name: String)
trait Demo {
val demoParams: DemoParams
}
Then just define it where you need it, and it's ready for use:
object MainApp extends App {
val demoObj = new Demo {
override val demoParams: DemoParams = DemoParams(name = "Salmon")
}
println("Demo params name is: ", demoObj.demoParams.name) // (Demo params name is: ,Salmon)
anotherFoo(demoObj.demoParams.name) // Salmon
def anotherFoo(name: String): Unit = println(name)
}
About the second question, without the actual code one can only guess (this sample code does not throw NPE). Probably somewhere you are using it without defining it previously, because var demoParams: DemoParams = _ just initializes demoParams to the default value of the reference type DemoParams, which is null in this case, and you get NPE when you try to access the name value of a null object. This is why using var is discouraged.
I am broadcasting a value in Spark Streaming application . But I am not sure how to access that variable in a different class than the class where it was broadcasted.
My code looks as follows:
object AppMain{
def main(args: Array[String]){
//...
val broadcastA = sc.broadcast(a)
//..
lines.foreachRDD(rdd => {
val obj = AppObject1
rdd.filter(p => obj.apply(p))
rdd.count
}
}
object AppObject1: Boolean{
def apply(str: String){
AnotherObject.process(str)
}
}
object AnotherObject{
// I want to use broadcast variable in this object
val B = broadcastA.Value // compilation error here
def process(): Boolean{
//need to use B inside this method
}
}
Can anyone suggest how to access broadcast variable in this case?
There is nothing particularly Spark specific here ignoring possible serialization issues. If you want to use some object it has to be available in the current scope and you can achieve this the same way as usual:
you can define your helpers in a scope where broadcast is already defined:
{
...
val x = sc.broadcast(1)
object Foo {
def foo = x.value
}
...
}
you can use it as a constructor argument:
case class Foo(x: org.apache.spark.broadcast.Broadcast[Int]) {
def foo = x.value
}
...
Foo(sc.broadcast(1)).foo
method argument
case class Foo() {
def foo(x: org.apache.spark.broadcast.Broadcast[Int]) = x.value
}
...
Foo().foo(sc.broadcast(1))
or even mixed-in your helpers like this:
trait Foo {
val x: org.apache.spark.broadcast.Broadcast[Int]
def foo = x.value
}
object Main extends Foo {
val sc = new SparkContext("local", "test", new SparkConf())
val x = sc.broadcast(1)
def main(args: Array[String]) {
sc.parallelize(Seq(None)).map(_ => foo).first
sc.stop
}
}
Just a short take on performance considerations that were introduced earlier.
Options proposed by zero233 are indeed very elegant way of doing this kind of things in Scala. At the same time it is important to understand implications of using certain patters in distributed system.
It is not the best idea to use mixin approach / any logic that uses enclosing class state. Whenever you use a state of enclosing class within lambdas Spark will have to serialize outer object. This is not always true but you'd better off writing safer code than one day accidentally blow up the whole cluster.
Being aware of this, I would personally go for explicit argument passing to the methods as this would not result in outer class serialization (method argument approach).
you can use classes and pass the broadcast variable to classes
your psudo code should look like :
object AppMain{
def main(args: Array[String]){
//...
val broadcastA = sc.broadcast(a)
//..
lines.foreach(rdd => {
val obj = new AppObject1(broadcastA)
rdd.filter(p => obj.apply(p))
rdd.count
})
}
}
class AppObject1(bc : Broadcast[String]){
val anotherObject = new AnotherObject(bc)
def apply(str: String): Boolean ={
anotherObject.process(str)
}
}
class AnotherObject(bc : Broadcast[String]){
// I want to use broadcast variable in this object
def process(str : String): Boolean = {
val a = bc.value
true
//need to use B inside this method
}
}
Is there a way to share a variable among all objects (instantiated from the same type)? Consider the following simple program. Two objects name and name2 have the same type A. Is there way to connect the properyList inside the two instantiation name and name2?
class A {
var properyList = List[String]()
def printProperties(): Unit = {
println(properyList)
}
}
object Experiment {
def main(args: Array[String]): Unit = {
val name = new A
val name2 = new A
name.properyList = List("a property")
name.printProperties()
name2.printProperties()
}
}
The output is
List(a property)
List()
Any way to change the class definition so that by just changing the .properyList in one of the objects, it is changed in all of the instatiations?
What you seem to be looking for is a class variable. Before I get into why you should avoid this, let me explain how you can do it:
You can attach propertyList to the companion object instead of the class:
object A {
var properyList = List[String]()
}
class A {
def printProperties(): Unit = {
println(A.properyList)
}
}
Now, to the why you shouldn't:
While scala let's you do pretty much anything that the JVM is capable of, its aims are to encourage a functional programming style, which generally eschews mutable state, especially shared, mutable state. I.e. the anti-pattern in A is not only that propertyList is a var, not a val but by sharing it via the companion object, you further allow anyone, from any thread to change the state of all instances at anytime.
The benefit of declaring your data as val is that you can safely pass it around, since you can be sure that nobody can change from under you at any time in the future.
You seem to be looking for something like java static fields.
In scala you usually achieve something like that by using a companion object:
object Main extends App {
class A {
import A._
def printProperties(): Unit = {
println(properyList)
}
}
object A {
private var properyList = List[String]()
def addProperty(prop: String): Unit = {
properyList ::= prop
}
}
val name = new A
val name2 = new A
A.addProperty("a property")
name.printProperties()
name2.printProperties()
}
If you want to have something similar to java's static fields you will have to use companion objects.
object Foo {
private var counter = 0
private def increment = {
counter += 1;
counter
}
}
class Foo {
val i = Foo.increment
println(i)
}
Code copied from:
"Static" field in Scala companion object
http://daily-scala.blogspot.com/2009/09/companion-object.html
Based on Arne Claassen's answer, but using private mutable collection with the companion object, which makes it visible only to the companion classes. Very simplistic example tried out in scala 2.11.7 console:
scala> :paste
// Entering paste mode (ctrl-D to finish)
object A {
private val mp = scala.collection.mutable.Map("a"->1)
}
class A {
def addToMap(key:String, value:Int) = { A.mp += (key -> value) }
def getValue(key:String) = A.mp.get(key)
}
// Exiting paste mode, now interpreting.
defined object A
defined class A
// create a class instance, verify it can access private map in object
scala> val a = new A
a: A = A#6fddee1d
scala> a.getValue("a")
res1: Option[Int] = Some(1)
// create another instance and use it to change the map
scala> val b = new A
b: A = A#5e36f335
scala> b.addToMap("b", 2)
res2: scala.collection.mutable.Map[String,Int] = Map(b -> 2, a -> 1)
// verify that we cannot access the map directly
scala> A.mp // this will fail
<console>:12: error: value mp is not a member of object A
A.mp
^
// verify that the previously created instance sees the updated map
scala> a.getValue("b")
res4: Option[Int] = Some(2)
I try to define a parametric type alias :
case class A
case class B
case class C
// We need an Int to load instances of A and B, and a String to load C
object Service {
def loadA(i: Int) : A = ???
def loadB(i: Int) : B = ???
def loadC(s: String) : C = ???
}
trait Location[T] { def get : T}
class IntLocation(val i: Int)
class StringLocation(val s: String)
trait EntityLocation[E] extends Location[_]
// Aim : make the loader typesafe
// Problem : I need something like that : type EntityLocation[Composite] = IntLocation
object Family {
trait EntityLoader[EntityT] extends (EntityLocation[EntityT] => EntityT)
val ALoader = new EntityLoader[A] {def load[A](l: EntityLocation[A]) = Service.loadA(l.get)
}
I am not sure what you are trying to achieve here. Could you please explain how you want to use these types in your code?
Assuming just want to use the types IdLocation and FileLocation in your code, maybe you want to try
trait Location[T] { def get : T }
type IdLocation = Location[Id]
type FileLocation = Location[java.io.File]
Seems rather convoluted, so I'm not sure I follow exactly what your purpose here is. You seem to go into many layers of factories that create factories, that call factory methods, etc.
Seems to me that at the end of the day you need you want to have a val ALoader value that you can use to get instances of A from Location[Int] objects, so I'll go with that assumption:
// Not sure what you want this one, but let's assume that you need a wrapper class per your example.
trait Location[P] { def get: P }
class IntLocation(val i: Int) extends Location[Int]
{
override def get: Int = i
}
// P for parameter, O for output class.
def loader[O, P](creator: P => O)(param: Location[P]) = { creator(param.get) }
object Service
{
// A function somewhere, capable of taking your parameter and creating something else (in your example, an Int to an 'A')
// here Int to String to make something concrete.
// This could be any function, anywhere
def loadA(someParam: Int) = someParam.toString
}
def main(args: Array[String])
{
val myStringLoader: Location[Int] => String = loader(Service.loadA)
// Alternatively, you could have written `val myStringLoader = loader(Service.loadA)(_)`. Either the type or the underscore are needed to tell the compiler that you expect a function, not a value.
// Some definition for you wrapper class
val location3 = new Location[Int]{
override def get: Int = 3
}
// ... or just a plain old instance of it.
val otherLocation = new IntLocation(5)
// This would 'load' the kind of thing you want using the method you specified.
val myString = myStringLoader(location3)
val myOtherString = myStringLoader(otherLocation)
// This prints "3 - 5"
print(myString + " - " + myOtherString)
}
This might seem like a long answer, but in truth the line def loader[O, P](creator: P => O)(param: Location[P]) = { creator(param.get) } is the one that does it all, the rest is to make it as similar to your sample as possible and to provide a working main you can use to start from.
Of course, this would be even simpler if you don't really need the Location wrapper for your integer.
I am trying to replicate my problem on a smaller example.
I am getting compilation error at the shown location in the following code snippet.
class Outer {
class Inner
}
object OuterUtil {
val obj = new Outer
object xyz extends obj.Inner
//do something with xyz
}
//-------------------
object OuterUtil2 {
var m_obj: Outer = null
def createOuter() = {
m_obj = new Outer
}
def anotherMethod() {
//Compilation error here: stable identifier required,
//but OuterUtil2.this.m_obj found.
object xyz extends m_obj.Inner
}
}
object Test {
OuterUtil2.createOuter
OuterUtil2.anotherMethod
}
OuterUtil is working fine.
In OuterUtil2, I am splitting the functionality into two functions. I am storing the Outer instance m_obj as a member var. The createOuter method creates and stores the Outer instance in m_obj. In anotherMethod, I am getting compilation error. How to fix OuterUtil2?
The prefix of a type (ie, the m_obj in m_obj.Inner) must be a stable value; a var doesn't cut it. You could make that a val and move the initialization out of createOuter.
If you just want to solve the problem with your function, here is a solution (by fixing the var to a val in the function)
def anotherMethod = {
val obj = m_obj
new obj.Inner
}
Another solution would be to use some options, however to overpass the stable identifier method, you have to define a val out of m_obj value. This solution is more elegant because you don't have null pointer exceptions if m_obj is not defined.
object OuterUtil2 {
var m_obj: Option[Outer] = None
def createOuter {
m_obj = Some(new Outer)
}
def anotherMethod = {
m_obj match{
case None => None
case Some(_) => val obj = m_obj.get; Some(new obj.Inner)
}
}
}