I'm looking at example Spark code and I'm a bit confused as to why the sample code I'm looking at requires two import statements:
import org.apache.spark._
import org.apache.spark.SparkContext._
This is Scala. As I understand it, _ is the wildcard character. So this looks like I'm importing SparkContext twice. Can anybody shed light on this?
This first line says to import all of the classes in the package org.apache.spark. This means you can use all of those classes without prefixing them with the package name.
The second line says to import all of the static members of the class SparkContext. This means you can use those members without prefixing their names with the class name.
Remember import doesn't really do anything at run time; it just lets you write less code. You aren't actually "importing" anything twice. The use of the term import comes from Java, and admittedly it is confusing.
This might help:
Without the first line, you would have to say
org.apache.spark.SparkContext
but the first import line lets you say
SparkContext
If you had only the first line and not the second, you would have to write
SparkContext.getOrCreate
but with both import lines you can just write
getOrCreate
Related
I have this code which compiles and works fine
import cats.implicits._
Cartesian[ValidResponse].product(
getName(map).toValidated,
readAge(map).toValidated
).map(User.tupled)
However I don't like the import of cats.implicits._ because there is just too many classes there. I tried importing specific things related to Cartesians like
import cats.implicits.catsSyntaxCartesian
import cats.implicits.catsSyntaxUCartesian
import cats.implicits.catsSyntaxTuple2Cartesian
But these did not work. As a newbie I find the implicit imports very confusing because there are simply 1000s of them and the names are not very obvious. My only alternative is to import the entire universe by import cats.implicits._ and stop thinking about it.
In fact I have a broader confusion about cats.implicits, cats.instances._ and cats.syntax._. So far I am just importing these via trial and error. I am not really sure of when to import what.
Do not try to pick out specific things from cats.implicits. You either import the entire thing, or you don't use it at all. Further, there's no reason to be afraid of importing it all. It can't interfere with anything.
Ok, I lied. It will interfere if you import cats.instances.<x>._ and/or cats.syntax.<x>._ alongside cats.implicits._. These groups are meant to be mutually exclusive: you either import everything and forget about it with cats.implicits._, or you specifically select what you want to import with cats.instances and cats.syntax.
These two packages are not meant to be imported completely like cats.implicits. Instead, they include a bunch of objects. Each object contains some implicit instances/syntax, and you are meant to import from those.
import cats.implicits._ // Good, nothing to fear
// RESET IMPORTS
import cats.implicits.catsSyntaxCartesian // Bad, don't pick and choose
// RESET IMPORTS
import cats.instances._ // Bad, is useless
import cats.syntax._ // Ditto
// RESET IMPORTS
import cats.instances.list._ // ok
import cats.syntax.cartesian._ // ok
// RESET IMPORTS
import cats.implicits._
import cats.syntax.monad._ // Bad, don't mix these two
Additionally each of cats.{ instances, syntax } contains an all object, with the obvious function. The import cats.implicits._ is really a shortcut for import cats.syntax.all._, cats.instances.all._.
I'll start by saying that import cats.implicits._ is safe, reasonable and the recommended approach when starting. So if the only reason for this question is that you don't like importing too many classes, then I think you should just bite the bulled at leave that as is.
Additionally, I recommend you take a look at the official cats import guide. It tries to explain the package/logical structure of cats code and might make it easier to understand.
The "cats" library is organized in several "areas" that can be easily distinguished by their package name:
cats._ - This is where most of the typeclasses live (e.g. Monad, Foldable etc.)
cats.data._ - This is the home of data structures like Validated and State.
cats.instances._ - This is where the instances for the typeclasses defined in 1 are. For example if you import cats.instances.list._ you'll bring into scope the Show, Monad etc. instances for the standard List. This is what you're most likely interested in.
cats.syntax._ - has some syntax enrichment that makes code easier to write and read.
An example of ussing cats.syntax._ would be:
import cats.Applicative
import cats.syntax.applicative._
val listOfInt = 5.pure[List]
//instead of
val otherList = Applicative[List[Int]].pure(5)
I have
import scala.slick.jdbc.{GetResult, StaticQuery}
import scala.slick.jdbc.StaticQuery.interpolation
this can be shortened to
import scala.slick.jdbc.{GetResult, StaticQuery}
import StaticQuery.interpolation
but then it's no longer sorted alphabetically
This would be nice, but does not work, any alternatives?
import scala.slick.jdbc.{GetResult, StaticQuery, StaticQuery.interpolation}
You can do
import scala.slick.jdbc.{GetResult, StaticQuery}, StaticQuery.interpolation
But that leads to confusing block imports, which also increases the probability for merge errors when you work with other developers. Therefore you should always use absolute imports at the beginning of a file and use relative imports only inside of narrower scopes (like the body of defs).
I am using scalaz7 in a project and sometimes I run into issues with imports. The simplest way get started is
import scalaz._
import Scalaz._
but sometimes this can lead to conflicts. What I have been doing until now the following slightly painful process:
work out a minimal example that needs the same imports as my actual code
copy that example in a separate project
compile it with the option -Xprint:typer to find out how the code looks after implicit resolution
import the needed implicits in the original project.
Although this works, I would like to streamline it. I see that scalaz7 has much more fine-grained imports, but I do not fully understand how they are organized. For instance, I see one can do
import scalaz.std.option._
import scalaz.std.AllInstances._
import scalaz.std.AllFunctions._
import scalaz.syntax.monad._
import scalaz.syntax.all._
import scalaz.syntax.std.boolean._
import scalaz.syntax.std.all._
and so on.
How are these sub-imports organized?
As an example, say I want to work with validations. What would I need, for instance to inject validation implicits and make the following compile?
3.fail[String]
What about making ValidationNEL[A, B] an instance of Applicative?
This blog post explains the package structure and imports a la carte in scalaz7 in detail: http://eed3si9n.com/learning-scalaz-day13
For your specific examples, for 3.failure[String] you'd need:
import scalaz.syntax.validation._
Validation already has a method ap:
scala> "hello".successNel[Int] ap ((s: String) => "x"+s).successNel[Int]
res1: scalaz.Validation[scalaz.NonEmptyList[Int],java.lang.String] = Success(xhello)
To get the <*> operator, you need this import:
import scalaz.syntax.applicative._
Then you can do:
"hello".successNel[Int] <*> ((s: String) => "x"+s).successNel[Int]
I want to take this import
import scala.slick.driver.PostgresDriver.simple._
out of my code and replace it with the various things I am actually using in my class
Examples of operators that I am using are +=,=== and the method firstOption.
What explicit imports do I need to make to be able to get my class to compile?
I am working on a small Scala project. I have the following issue with 'import':
If, at the top of one of my files, I import two thing with these commands:
import main.Main._
import main.game.Game
^^^^
it gives me the following error message at the underlined 'main' word: "missing arguments for method main in object Main; follow this method with `_' if you want to treat it as a partially applied function" which is quite strange especially that it is just an import statement. And naturally no actual importing occures. At first I thought about semicolon inference quirks again but it is not the case. If I swap the two lines and write like this:
import main.game.Game
import main.Main._
then everythinng is fine.
Could anyone shed some light on that? Is it something special about Scala?
Presumably you have a main method in object Main. So after import main.Main._ main refers to this method instead of the main package. You could avoid it in several ways:
Change import order, as in the question.
Don't import the main method, as Daniel C. Sobral's answer suggests.
Explicitly say you want the top-level main package:
import _root_.main.game.Game
Following the normal Java package naming convention should avoid this problem in most cases, as you are unlikely to have members (or subpackages) called com or org (though net could be a problem).
You do have a method named main inside main.Main, don't you? Well, since you imported it, it has now shadowed the package by the name main. You can try this to confirm:
import main.Main.{main => _, _}
import main.game.Game
This will exclude main from being imported.