I've inherited a Scala project that has to be extended and instead of one monolithic monster it has to be split: some code needs to become a library that's used by all the other components and there's a few applications that may be included at some later point.
object Shared {
// vals, defs
}
=====
import Shared._
object Utils1 {
// vals, defs
}
=====
import Shared._
object Utils2 {
// vals, defs
}
=====
import Shared._
import Utils1._
class Class1(val ...) {
// stuff
}
=====
import Shared._
import Utils2._
class Class2(val ...) {
// more stuff
}
etc.
The problem is that Utils1 and Utils2 (and many other objects and classes) use the values from Shared (the singleton) and that Shared now has to be instantiated because one of the key things that happens in Shared is that the application name is set (through SparkConf) as well as reading of configuration files with database connection information.
The idea was to have a multi-project build where I can just pick which component needs to be recompiled and do it. Since the code in Utils is shared by pretty much all applications that currently exist and will come, it's nice to keep it together in one large project, at least so I thought. It's hard to publish to a local repository the code that's common because we have no local repository (yet) and getting permission to have one is difficult.
The Utils singletons are needed because they have to be static (because of Spark and serializability).
When I make Shared a proper class, then all the import statements will become useless and I have to pass an instance around, which means I cannot use singletons, even though I need those. It currently works because there is only a single application, so there really only one instance of Shared. In future, there will still be only one instance of Shared per application but there may be multiple applications/services defined in a single project.
I've looked into implicits, package objects, and dependency injection but I'm not sure that's the right road to go down on. Maybe I'm just not seeing what should be obvious.
Long story short, is there a neat way to give Shared a parameter or achieve what I hope to achieve?
Maybe as an option you can create SharedClass class and make Shared object extend it in each app with different constructor params:
class SharedClass(val param: String) { // vals ...
}
object Shared extends SharedClass("value")
One hacky way to do it, is to use a DynamicVariable and make it have a value, when Shared is initialized (i.e., the first time anything that refers to Shared fields or methods is called).
Consider the following:
/* File: Config.scala */
import scala.util.DynamicVariable
object Config {
val Cfg: DynamicVariable[String] = new DynamicVariable[String](null)
}
/**********************/
/* File: Shared.scala */
import Config._
object Shared {
final val Str: String = Cfg.value
def init(): Unit = { }
}
/*********************/
/* File: Utils.scala */
import Shared._
object Utils {
def Length: Int = Str.length
}
/********************/
/* File: Main.scala */
object Main extends App
{
// Make sure `Shared` is initialized with the desired value
Config.Cfg.withValue("foobar") {
Shared.init()
}
// Now Shared.Str is fixed for the duration of the program
println(Shared.Str) // prints: foobar
println(Utils.Length) // prints: 6
}
This setup is thread-safe, although not deterministic, of course. The following Main would randomly select one of the 3 strings, and print the selected string and its length 3 times:
import scala.util.Random
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
object Main extends App
{
def random(): Int = Random.nextInt(100)
def foo(param: String, delay: Int = random()): Future[Unit] = {
Future {
Thread.sleep(delay)
Config.Cfg.withValue(param) {
Shared.init()
}
println(Shared.Str)
println(Utils.Length)
}
}
foo("foo")
foo("foobar")
foo("foobarbaz")
Thread.sleep(1000)
}
object in Scala has "apply" method which you can use for this purpose. So, your Shared object will be like below
object Shared {
def apply(param1: String, param2: String) = ???
}
Now each client of Shared can pass different values.
Related
i originally made scripts with many functions on 2 individual scala worksheets. i got them working and now want to tie these individual scripts together by importing and using them into a third file. from what i have read you can not simply import external scripts you must first make them into a class and put them into a package. so i tried that but i still couldn't import it
i know this may be a bit basic for this site but im struggling to find much scala documentation.
i think my problem might span from a missunderstanding of how packages work. the picture below might help.
my program example
adder.scala
package adder
class adder {
def add_to_this(AA:Int):Int={
var BB = AA + 1;
return BB
}
}
build.scala
package builder
class build {
def make_numbers(){
var a = 0;
var b = 0;}
}
main.sc
import adder
import builder
object main {
adder.adder.add_to_this(10);
}
the errors i get are
object is not a member of package adder
object is not a member of package builder
Classes in scala slightly differ from classes in java. If you need something like singleton, you'll want to use object instead of class i.e.:
package com.example
object Main extends App {
object Hide{
object Adder{
def addToThis(AA:Int):Int = AA + 1
}
}
object Example{
import com.example.Main.Hide.Adder
def run(): Unit = println(Adder.addToThis(10))
}
Example.run()
}
Consider objects like packages/modules which are also regular values. You can import an object by its full path, i.e. com.example.Main.Hide.Adder you can also import contents of an object by adding .{addToThis}, or import anything from object by adding ._ after an object.
Note that classes, traits and case classes could not be used as objects, you can't do anything with it unless you have an instance - there are no static modifier.
I want to group a set of similar functions in a library in scala.
Here are two approaches I have seen elsewhere. I want to understand the
differences between the two.
Singleton object defined in a package
// src/main/scala/com/example/toplevel/functions.scala
package com.example.toplevel
object functions {
def foo: Unit = { ... }
def bar: Unit = { ... }
}
Package object
// src/main/scala/com/example/toplevel/package.scala
package com.example.toplevel
package object functions {
def foo: Unit = { ... }
def bar: Unit = { ... }
}
Comparison
As far as I can tell, the first approach will require explicitly importing
the functions object whenever you want to use its functions. While the package object approach allows anything in the package functions to access those methods without importing them.
Ie, com.example.toplevel.functions.MyClass would have access to com.example.toplevel.functions.foo implicitly.
Is my understanding correct?
If there are no classes defined within com.example.toplevel.functions,
it seems the approaches would be equivalent, is this correct?
Ansered by terminally-chill in a comment:
yes, your understanding is correct. Anything defined in your package
object will be available without having to import. If you use an
object, you will have to import it even within the same package.
I want to run some code in the body of an scala companion object before the class is instantiated. The idea is to register a bunch of object in a Set. Here is the code
trait Delegate {
def make: Ins
}
//EDIT: Changed constructor to private
//class Ins
class Ins private()
//this is the companion object that will be registered with the InsDelegate
object Ins extends Delegate{
//here is the code that do the registration but doesn't run
InsDelegate.register(this)
override def make: Ins = {
println("This is an Ins")
new Ins()
}
}
Here is the code for the InsDelegate
object InsDelegate {
private val objectSet = new mutable.HashSet[Delegate]()
def register(obj: Delegate): Unit = objectSet.add(obj)
def getRegisteredObj: Set[Delegate] = objectSet.toSet
}
When I run this test, nothing gets printed.
object test extends App {
InsDelegate.getRegisteredObj.foreach(_.make)
}
The code that register the companion object doesn't run. I know that unlike java, in order to run the companion object code you need to instantiate the class of the object. How do I accomplish what I am trying to do???
Scala objects are lazy, so they're only constructed when first used. In your example, the test application never creates any instances, so object Ins is never constructed.
Your code should work, but you would need to create an instance of class Ins in your test code:
object test extends App {
val temp = Ins.make()
InsDelegate.getRegisteredObj.foreach(_.make)
}
Incidentally, the convention for functions with side-effects (Delegate.make) is to take parentheses; a version without parentheses indicates that the function has no side-effects, which make clearly has (registering the Ins object, creating a new Ins element).
Another Scala convention is to name factory methods apply, rather than make. If you did that, you could create new Ins class instances using Ins(), instead of Ins.make(). (Ins() is interpreted to be the same as Ins.apply().)
Update: Forgot to mention this: if you want to register Ins without creating any instances first, you will need to reference it in some way. This quickly leads to ugly solutions along the lines of:
object Ins extends Delegate{
InsDelegate.register(this)
// Dummy method to get object to register itself.
def register(): Unit = {}
override def make: Ins = {
println("This is an Ins")
new Ins()
}
}
object InsDelegate {
private val objectSet = new mutable.HashSet[Delegate]()
def register(obj: Delegate): Unit = objectSet.add(obj)
def getRegisteredObj: Set[Delegate] = objectSet.toSet
// Create delegate objects...
Ins.register()
}
However, if we're going to go to that much trouble, we might as well forego registration and add objects in the InsDelegate object:
object Ins extends Delegate{
override def make: Ins = {
println("This is an Ins")
new Ins()
}
}
object InsDelegate {
// Set of delegate objects available. Note: this is public, replaces getRegisteredObj.
val objectSet: Set[Delegate] = Set(Ins)
}
The downside is that Delegate objects no longer register themselves, but that's a blessing in disguise as you can now test delegate creation separately from testing InsDelegate.
I know that unlike java, in order to run the companion object code you need to instantiate the class of the object
Actually, your Java code would have the same result. What you need to do in both cases is to load the class, and instantiating it is just one way to do it. You can also use Class.forName, ClassLoader.loadClass, load any class which uses it somewhere in a signature... One very well-known case is (or was, before JDBC 4.0) loading JDBC drivers.
Unfortunately, in Scala the class you need to load is actually Ins$ (the class of the companion object) and instantiating Ins (or loading it in some other way) isn't necessarily enough.
I have two controllers who writes and reads the same AccountModel case class. This class is an adapter for my "domain" object Account who flatten some collections and transform objects references (Map[Role, Auth]) to a explicit key reference (Set[AuthModel(rolekey:String, level:Int)]).
I would like to reuse this AccountModel and his implicits Writes and Reads but don't know how the achieve that 'the scala way'.
I would say in an object Models with my case classes as inner classes and all the related implicits but I think that this would become unreadable soon.
What are you used to do, where do you put your reusable Json classes, do you have some advices ?
Thanks a lot
There are two main approaches.
Approach 1: Put them on a companion object of your serializable object:
// in file AccountModel.scala
class AccountModel(...) {
...
}
object AccountModel {
implicit val format: Format[AccountModel] = {...}
}
This way everywhere you import AccountModel, the formatters will be also available, so everything will work seamlessly.
Approach 2: Prepare a trait with JSON formatters:
// in a separate file AccountModelJSONSupport.scala
import my.cool.package.AccountModel
trait AccountModelJsonSupport {
implicit val format: Format[AccountModel] = {...}
}
With this approach whenever you need serialization, you have to mix the trait in, like this:
object FirstController extends Controller with AccountModelJsonSupport {
// Format[AccountModel] is available now:
def create = Action(parse.json[AccountModel]) { ... }
}
EDIT: I forgot to add a comparison of the two approaches. I usually stick to approach 1, as it is more straightforward. The JSONSupport mixin strategy is however required when you need two different formatters for the same class or when the model class is not your own and you can't modify it. Thanks for pointing it out in the comments.
I have a trait, named Init:
package test
trait Init {
def init(): Any
}
There are some classes and an object, extends this trait:
package test
object Config extends Init {
def init() = { loadFromFile(...) }
}
class InitDb extends Init {
def init() = { initdb() }
}
When app has started, I will find all classes and objects which extends Init, and invoke their init method.
package test
object App {
def main(args: Array[String]) {
val classNames: List[String] = findAllNamesOfSubclassOf[Init]
println(classNames) // -> List(test.Config$, test.InitDb)
classNames foreach { name =>
Class.forName(name).newInstance().asInstanceOf[Init].init() // ***
}
}
}
Please note the "*" line. For test.InitDb, it's OK. But for test.Config$, when newInstance(), it throws an exception said we can't access its private method.
My problem is, how to get that object, and run its init method?
There's usually little point to doing this in Scala. Just put some code in the body of any object and it'll be executed when that object is first initialised, saving you the nasty performance hit of pre-initialising everything.
In general though, finding all subclasses of a particular type requires a full classpath scan. There are a few libraries to do this, but one of the more common is Apache's commons-discover
However... This is dynamic code, it uses reflection, and it's really NOT idiomatic. Scala has sharper tools than that, so please don't try and swing the blunt ones with such violence!
I don't totally agree with Kevin. There are some exceptions. For example I wrote a Scala desktop app. I split the core and the modules into two parts. At startup time the core loads all modules into GUI. At that time the core just gets the name of modules, it doesn't need to initialize something. Then I put all module's init code in an init() function. That function will be called when user executes the module.
#Freewind: About reflection in Scala, it's absolutely the same in Java. Just please note that the methods from Java which are used with reflection, are used for Java objects - not Scala. I'm sorry for my English. I mean those methods can not work with Scala object, trait.
For example:
var classLoader = new java.net.URLClassLoader(
Array(new File("module.jar").toURI.toURL),
/*
* need to specify parent, so we have all class instances
* in current context
*/
this.getClass.getClassLoader)
var clazz = classLoader.loadClass("test.InitDb")
if (classOf[Init].isAssignableFrom(clazz))
var an_init = clazz.newInstance.asInstanceOf[Init];
But you can not do it the opposite way:
if (clazz.isAssignableFrom(classOf[Init]))
Because Init is a trait, and Java method isAssignableFrom(Class) doesn't know trait.
I'm not sure if my question is useful for you, but here it is.