I understand that cross-building across different Scala versions is easy with SBT. You just put your files which fail to compile in scala-2.10 and scala-2.11 directories instead of scala. However, If I want to cross build for different versions of Scala and for different versions of a dependency (say, Spark 1.6 and 2.1) then how can that be done?
Related
Does sbt (Scala build tool) have the concept of archetypes, similar to the way that you can define an archetype in a maven project? Basically a blueprint for a new project that might have specific dependencies already defined for you, or perhaps some prebuilt classes and traits?
Yeah, in sbt, we can use other plugins to achiecve this. sbt-native-packager can be used. As quoted in its documentation:
SBT native packager lets you build application packages in native
formats and offers different archetypes for common configurations,
such as simple Java apps or server applications
Let me know if it helps!!
.
I am using JDT to get ASTs and type resolvers for Java sources in Eclipse, is there a way to achieve the same for Scala sources in Scala projects?
No, it is not possible to parse Scala with Java Development Tools. Scala and Java are two completely different languages. They have different syntax, different semantics, different type systems.
I downloaded confluent-2.0.0-2.10.5.tar.gz, because I want to have scala 2.10 package
but still the kafka jar in /share/java/schema-registry is still kafka_2.11-0.9.0.0-cp1.jar
Is there anyway I can get a clean 2.10 scala confluent package
The 2.10 refers to the version of the Kafka subpackage, but a different version may be used by other subpackages.
The tar.gz packages use the 2.11 versions where a different subpackage requires access to the core Kafka jar that has a Scala dependency. (Actually, the version they depend on is really whichever Scala version is supported by Kafka and considered most stable and well supported upstream). This is necessary because Scala libraries aren't necessarily binary compatible between different Scala versions, which would mean that not doing this would require multiple versions of all the services that use the Kafka libraries, especially on platforms like Debian and RPM-based distros, i.e. we'd need a schema-registry-2.10 and schema-registry-2.11. Instead, we sort of vendorize the entire Kafka library for services that depend on it.
Note that the files under /share/java/kafka only use Scala 2.10 and if you need to pull in the clients, you can safely add that to your classpath. The use of 2.10 or 2.11 for any of the other services shouldn't matter as they are simply that: services that you execute. Any libraries that you might need to put on your classpath (e.g. serializers) only depend on the pure Java libraries in Kafka and are therefore safe to use with Kafka libraries compiled with any Scala version.
I am new to Spark. I am trying to run a simple spark project in local system.
So based on tutorials I have run 'sbt/sbt assembly'. Now jar file is created in core/target/scala-2.9.2/spark-core-assembly-0.7.0.jar. To run samples could you please tell where and how I have to add this jar to classpath?
Regards,
Dinesh
The Spark documentation's quick start guide has documentation on developing standalone applications using Spark with Scala and Java. Those instructions show how to add a Spark dependency to your Maven or SBT projects.
If you're not using Maven or SBT to build your project, you'll have to pass the appropriate flags to javac and java to add the Spark assembly JAR to your classpath, the same as you'd do for any other JAR dependency.
As an aside, 0.7.0 is a pretty old version of Spark (it was released almost a year ago); I'd recommend using a newer version, such as 0.9.0.
I am starting a new Lift project from scratch and would like to develop it against Scala 2.9.1 and SBT 0.11. Where can I get the current development Lift branch compatible with these versions and when is it going to be released for production?
I could also make use of sample projects (a REST web service and a normal web site) for these versions.
UPDATE: I've came to the idea Lift doesn't support Scala 2.9.1 and SBT 0.11 because standard examples come with SBT 0.5.5 and 2.9.0.1, and there is a record on the Lift web site saying "We are in process of building Lift against Scala 2.9.1". But practically, I've managed to use 2.9.1 with Lift 2.4-M4, seems working just fine (and as Debilski has said below, SBT 0.11 can be used to build a Lift project). So the question is more about when Scala 2.9.1 and SBT 0.11 will become mainstream for Lift and when will Lift examples be available with these versions?