Is it possible to call a main method in one object from a main method in another? I have the following classes and was wondering how to call two separate main methods within one program run:
object MongoUpload {
def main(args: Array[String]): Unit = {
.. upload to Mongo ..
// Want to upload to Oracle here
}
}
object OracleUpload {
def main(args: Array[String]): Unit = {
.. upload to Oracle
}
}
Does anything make main unique among methods? Can I just call one from another?
You sure can. Just like any other method, main can be called between objects.
object foo {
def main(args: Array[String]): Unit = {
println("qux")
}
}
object bar {
def main(args: Array[String]): Unit = {
println("baz")
foo.main(null)
}
}
Running main in bar gives the following output:
baz
qux
The same can also be replicated to main methods with arguments, as in the following example:
object foo {
def main(args: Array[String]): Unit = {
println(args(0) + " " + args(1))
}
}
object bar {
def main(args: Array[String]): Unit = {
... some processing ...
foo.main(Array["Hello", "World"])
}
}
Running main in bar gives the following output:
Hello World
Whether or not it leads to clear and readable code is another question :)
Related
I have an object called Program which can only instantiate only one class, in my case val a = new A(), in it I have a local variable called localText, I want to convert it to global, I have created a variable outside the block called globalText and inside the block I assign it globalText = localText, when I do a println it shows it correctly "Hello World", I have another class called B that needs that value but I don't know how to access it because when I call it it appears with the null value and if I call the sum function what it returns is an Int. (The sum() def cannot be touched and the data comes like this from third party code) thank you very much for the help.
...
class A{
var globalText: String = _
def sum(): Int = {
val localText: String = "Hello World"
globalText = localText
println(globalText)
val b = new B()
3+4
}
println(globalText)
}
class B extends A{
var globalBText = globalText
println(globalBText)
}
object Program {
def main(args: Array[String]): Unit = {
val a = new A()
println(a.sum())
}
}
Output is
null
Hello World
null
null
7
...
This question has already been answered.
Please do not create duplicate posts with the same question. You can edit the previous question, if you feel like you missed something.
If you can't instantiate the class than this beats the whole purpose of being a class and you might as well model it as an object or create another object.
The idea is that a class must be instantiated in order for its fields to be initialized. There is no other way.
You could use an object so you don't have to instantiate any class at all, making sure that B.globalBText is initialized before you used it.
Here's what I mean:
object A {
var globalText: String = _
def sum(): Int = {
globalText = "Hello World"
3 + 4
}
}
object B {
var globalBText: String = {
A.sum()
A.globalText
}
println(globalBText)
}
object Program {
def main(args: Array[String]): Unit = {
println(B.globalBText)
}
}
Outputs:
Hello World
Hello World
I wrote a code below in Scala .
object Main {
def main(args: Array[String]): Unit = {
println(main)
}
}
It gave output as :
Main$$$Lambda$3/1543727556#3930015a
Why is this value returned ?
Also, When I wrote a simple method like one below :
def wow(): Unit = {
println(wow)
}
But it gives compilation error.
In general, can you declare a val in scala before assigning it a value? If not, why not? An example where this might be useful (in my case at least) is that I want to declare a val which will be available in a larger scope than when I assign it. If I cannot do this, how can I achieve the desired behavior?
And I want this to be a val, not a var because after it is assigned, it should NEVER change, so a var isn't ideal.
For example:
object SomeObject {
val theValIWantToDeclare // I don't have enough info to assign it here
def main(): Unit = {
theValIWantToDeclare = "some value"
}
def someOtherFunc(): Unit {
val blah = someOperationWith(theValIWantToDeclare)
}
}
object SomeObject {
private var tviwtdPromise: Option[Int] = None
lazy val theValIWantToDeclare: Int = tviwtdPromise.get
private def declareTheVal(v: Int): Unit = {
tviwtdPromise = Some(v)
theValIWantToDeclare
}
def main(args: Array[String]): Unit = {
declareTheVal(42)
someOtherFunction()
}
def someOtherFunction(): Unit = {
println(theValIWantToDeclare)
}
}
It will crash with a NoSuchElementException if you try to use theValIWantToDeclare before fulfilling the "promise" with declareTheVal.
It sounds to me that you need a lazy val.
A lazy val is populated on demand and the result is cached for all subsequent calls.
https://blog.codecentric.de/en/2016/02/lazy-vals-scala-look-hood/
Why not define a SomeObjectPartial that is partially constructed, and class SomeObject(theVal) that takes the value as a parameter?
Then your program has two states, one with the partial object, and another with the completed object.
I am broadcasting a value in Spark Streaming application . But I am not sure how to access that variable in a different class than the class where it was broadcasted.
My code looks as follows:
object AppMain{
def main(args: Array[String]){
//...
val broadcastA = sc.broadcast(a)
//..
lines.foreachRDD(rdd => {
val obj = AppObject1
rdd.filter(p => obj.apply(p))
rdd.count
}
}
object AppObject1: Boolean{
def apply(str: String){
AnotherObject.process(str)
}
}
object AnotherObject{
// I want to use broadcast variable in this object
val B = broadcastA.Value // compilation error here
def process(): Boolean{
//need to use B inside this method
}
}
Can anyone suggest how to access broadcast variable in this case?
There is nothing particularly Spark specific here ignoring possible serialization issues. If you want to use some object it has to be available in the current scope and you can achieve this the same way as usual:
you can define your helpers in a scope where broadcast is already defined:
{
...
val x = sc.broadcast(1)
object Foo {
def foo = x.value
}
...
}
you can use it as a constructor argument:
case class Foo(x: org.apache.spark.broadcast.Broadcast[Int]) {
def foo = x.value
}
...
Foo(sc.broadcast(1)).foo
method argument
case class Foo() {
def foo(x: org.apache.spark.broadcast.Broadcast[Int]) = x.value
}
...
Foo().foo(sc.broadcast(1))
or even mixed-in your helpers like this:
trait Foo {
val x: org.apache.spark.broadcast.Broadcast[Int]
def foo = x.value
}
object Main extends Foo {
val sc = new SparkContext("local", "test", new SparkConf())
val x = sc.broadcast(1)
def main(args: Array[String]) {
sc.parallelize(Seq(None)).map(_ => foo).first
sc.stop
}
}
Just a short take on performance considerations that were introduced earlier.
Options proposed by zero233 are indeed very elegant way of doing this kind of things in Scala. At the same time it is important to understand implications of using certain patters in distributed system.
It is not the best idea to use mixin approach / any logic that uses enclosing class state. Whenever you use a state of enclosing class within lambdas Spark will have to serialize outer object. This is not always true but you'd better off writing safer code than one day accidentally blow up the whole cluster.
Being aware of this, I would personally go for explicit argument passing to the methods as this would not result in outer class serialization (method argument approach).
you can use classes and pass the broadcast variable to classes
your psudo code should look like :
object AppMain{
def main(args: Array[String]){
//...
val broadcastA = sc.broadcast(a)
//..
lines.foreach(rdd => {
val obj = new AppObject1(broadcastA)
rdd.filter(p => obj.apply(p))
rdd.count
})
}
}
class AppObject1(bc : Broadcast[String]){
val anotherObject = new AnotherObject(bc)
def apply(str: String): Boolean ={
anotherObject.process(str)
}
}
class AnotherObject(bc : Broadcast[String]){
// I want to use broadcast variable in this object
def process(str : String): Boolean = {
val a = bc.value
true
//need to use B inside this method
}
}
I don't know how to fix this code. It "explodes" somewhere in returnFirstString but I don't know why. Also, I don't know how to properly display result using println. Is this approach ok.
So here's the code:
def returnFirstString(a: Array[String]): Option[String]=
{
if(a.isEmpty) { None }
Some(a(0))
}
val emptyArrayOfStrings = Array.empty[String]
println(returnFirstString(emptyArrayOfStrings))
You're not properly returning the None:
def returnFirstString(a: Array[String]): Option[String] = {
if (a.isEmpty) {
None
}
else {
Some(a(0))
}
}
Also, there's already a method for this on most scala collections:
emptyArrayOfStrings.headOption
The most concise way:
def returnFirstString(a: Array[String]): Option[String]= a.headOption