I am encountering a compile time error while attempting to get Squeryl example code running. The following code is based on the My Adventures in Coding blog post about connecting to SQLServer using Squeryl.
import org.squeryl.adapters.MSSQLServer
import org.squeryl.{ SessionFactory, Session}
import com.company.model.Consumer
class SandBox {
def tester() = {
val databaseConnectionUrl = "jdbc:jtds:sqlserver://myservername;DatabaseName=mydatabasename"
val databaseUsername = "userName"
val databasePassword = "password"
Class.forName("net.sourceforge.jtds.jdbc.Driver")
SessionFactory.concreteFactory = Some(()=>
Session.create(
java.sql.DriverManager.getConnection(databaseConnectionUrl, databaseUsername, databasePassword),
new MSSQLServer))
val consumers = table[Consumer]("Consumer")
}
}
I believe I have the build.sbt file configured correctly to import the Squeryl & JTDS libraries. When running SBT after adding the dependencies it appeared to download the libraries need.
libraryDependencies ++= List (
"org.squeryl" %% "squeryl" % "0.9.5-6",
"net.sourceforge.jtds" % "jtds" % "1.2.4",
Company.teamcityDepend("company-services-libs"),
Company.teamcityDepend("ssolibrary")
) ::: Company.teamcityConfDepend("company-services-common", "test,gatling")
I am certain that at least some of the dependencies were successfully installed. I base this on the fact that the SessionFactory code block compiles successfully. It is only the line that attempts to setup a map from the Consumer class to the Consumer SQLServer table.
val consumers = table[Consumer]("Consumer")
This line causes a compile time error to be thrown. The compile is not able to find the table object.
[info] Compiling 8 Scala sources to /Users/pbeacom/Company/integration/target/scala-2.10/classes...
[error] /Users/pbeacom/Company/integration/src/main/scala/com/company/code/SandBox.scala:25: not found: value table
[error] val consumers = table[Consumer]("Consumer")
The version of Scala in use is 2.10 and if the table line is commented the code compiles successfully. Use of the table object to accomplish data model mappings is nearly ubiquitous in the Squeryl examples I'm been researching online and no one else seems to have encountered a similar problem.
Shortly after posting this and reviewing it I finally noticed my problem. I was not being conscious enough of the heavy use of mixins in Scala. I failed to extend the Schema class. That's why table is unknown in the scope of the SandBox class. I was able to solve the problem with the following class definition:
class SandBox extends Schema {
def tester() = {
...
Related
I am looking for a simple way to read a file (and maybe a directory) via sftp protocol in scala.
My tries:
I've looked at the Alpakka library, which is a part of akka.
But this works with streams, which is a complex topic I am not familiar with and it seems to much effort for this.
Then there is spark-sftp: this needs scala spark, which would be a bit much just to load a file.
There is the jsch library for java that could do the job, but I could not bring it to work
I am looking for actual working code that uses a library and sftp instead of a plain scp, which I am forced to do. I've found that there are not many examples for this on the web, and the ones I have found are much more complex.
Here is a working example, using sshj:
import net.schmizz.sshj.SSHClient
import net.schmizz.sshj.sftp.SFTPClient
object Main extends App {
val hostname = "myServerName"
val username = "myUserName"
val password = "thePassword"
val destinationFile = "C:/Temp/Test.txt"
val sourceFile = "./Test.txt"
val ssh = new SSHClient()
ssh.addHostKeyVerifier("xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx")
ssh.connect(hostname)
ssh.authPassword(username, password)
val sftp: SFTPClient = ssh.newSFTPClient()
sftp.get(sourceFile, destinationFile)
sftp.close()
ssh.disconnect()
}
I tested this on scala version 2.13.4 with the following entries in build.sbt:
libraryDependencies += "com.hierynomus" % "sshj" % "0.31.0"
libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.2.3"
I would not recommend to actually use it this way. Some of these steps should be wrapped in a Try and then some error checking should be done if the file didn't exists or the connection failed and so on. I intentionally left that out for clarity.
I am not saying that this is the only or the right library for this task. It is just the first one that did work for me. Especially the addHostKeyVerifier method was very helpful in my case.
There are also other libraries like JSCH, jassh, scala-ssh and scala-ftp wich also could very well do the job.
I recently updated my Spark version from 2.2 to 2.4.0
I started having an error in this block (which was working fine with 2.2 version):
object Crud_mod {
def f(df: DataFrame,
options: JDBCOptions,
conditions: List[String]) {
val url = options.url
val tables = options.table
val dialect = JdbcDialects_mod.get(url)
error: value table is not a member of org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions
[ERROR] val tables = options.table
So I took a look inside Spark sources and value table seems to exist in JDBCOptions class.
What am I missing please?
Your sources link is pointing to a constructor, that accepts table as an argument, but I can't find table value in class itself.
However, there's tableOrQuery (here) method, that can be used for your needs, I think.
I have a bunch of SBT 0.13 project definitions that look like this:
lazy val coreBase = crossProject.crossType(CrossType.Pure).in(file("core"))
.settings(...)
.jvmConfigure(_.copy(id = "core"))
.jsConfigure(_.copy(id = "coreJS"))
lazy val core = coreBase.jvm
lazy val coreJS = coreBase.js
(Mostly because I'm resentful about having to maintain Scala.js builds and don't want to have to type the JVM suffix every time I'm changing projects, etc.)
This doesn't compile in SBT 1.0 because Project doesn't have a copy method now.
Okay, let's check the migration guide.
Many of the case classes are replaced with pseudo case classes generated using Contraband. Migrate .copy(foo = xxx) to withFoo(xxx).
Cool, let's try it.
build.sbt:100: error: value withId is not a member of sbt.Project
.jvmConfigure(_.withId("core"))
^
So I asked on Gitter and got crickets.
The links for the 1.0 API docs actually point to something now, which is nice, but they're not very helpful in this case, and trying to read the SBT source gives me a headache. I'm not in a rush to update to 1.0, but I'm going to have to at some point, I guess, and maybe some helpful person will have answered this by then.
(This answer has been edited with information about sbt 1.1.0+ and sbt-crossproject 0.3.1+, which significantly simplify the whole thing.)
With sbt 1.1.0 and later, you can use .withId("core"). But there's better with sbt-crossproject 0.3.1+, see below.
I don't know about changing the ID of a Project, but here is also a completely different way to solve your original issue, i.e., have core/coreJS instead of coreJVM/coreJS. The idea is to customize crossProject to use the IDs you want to begin with.
First, you'll need to use sbt-crossproject. It is the new "standard" for compilation across several platforms, co-designed by #densh from Scala Native and myself (from Scala.js). Scala.js 1.x will allways use sbt-crossproject, but it is also possible to use sbt-crossproject with Scala.js 0.6.x. For that, follow the instructions in the readme. In particular, don't forget the "shadowing" part:
// Shadow sbt-scalajs' crossProject and CrossType from Scala.js 0.6.x
import sbtcrossproject.{crossProject, CrossType}
sbt-crossproject is more flexible than Scala.js' hard-coded crossProject. This means you can customize it more easily. In particular, it has a generic notion of Platform, defining how any given platform behaves.
For a cross JVM/JS project, the new-style crossProject invocation would be
lazy val coreBase = crossProject(JVMPlatform, JSPlatform)
.crossType(CrossType.Pure)
.in(file("core"))
.settings(...)
.jvmConfigure(_.copy(id = "core"))
.jsConfigure(_.copy(id = "coreJS"))
lazy val core = coreBase.jvm
lazy val coreJS = coreBase.js
Starting with sbt-crossproject 0.3.1, you can simply tell it not to add the platform suffix for one of your platforms. In your case, you want to avoid the suffix for the JVM platform, so you would write:
lazy val coreBase = crossProject(JVMPlatform, JSPlatform)
.withoutSuffixFor(JVMPlatform)
.crossType(CrossType.Pure)
...
lazy val core = coreBase.jvm
lazy val coreJS = coreBase.js
and that's all you need to do!
Old answer, applicable to sbt-crossproject 0.3.0 and before
JVMPlatform and JSPlatform are not an ADT; they are designed in an OO style. This means you can create your own. In particular, you can create your own JVMPlatformNoSuffix that would do the same as JVMPlatform but without adding a suffix to the project ID:
import sbt._
import sbtcrossproject._
case object JVMPlatformNoSuffix extends Platform {
def identifier: String = "jvm"
def sbtSuffix: String = "" // <-- here is the magical empty string
def enable(project: Project): Project = project
val crossBinary: CrossVersion = CrossVersion.binary
val crossFull: CrossVersion = CrossVersion.full
}
Now that's not quite enough yet, because .jvmSettings(...) and friends are defined to act on a JVMPlatform, not on any other Platform such as JVMPlatformNoSuffix. You'll therefore have to redefine that as well:
implicit def JVMNoSuffixCrossProjectBuilderOps(
builder: CrossProject.Builder): JVMNoSuffixCrossProjectOps =
new JVMNoSuffixCrossProjectOps(builder)
implicit class JVMNoSuffixCrossProjectOps(project: CrossProject) {
def jvm: Project = project.projects(JVMPlatformNoSuffix)
def jvmSettings(ss: Def.SettingsDefinition*): CrossProject =
jvmConfigure(_.settings(ss: _*))
def jvmConfigure(transformer: Project => Project): CrossProject =
project.configurePlatform(JVMPlatformNoSuffix)(transformer)
}
Once you have all of that in your build (hidden away in a project/JVMPlatformNoSuffix.scala in order not to pollute the .sbt file), you can define the above cross-project as:
lazy val coreBase = crossProject(JVMPlatformNoSuffix, JSPlatform)
.crossType(CrossType.Pure)
.in(file("core"))
.settings(...)
lazy val core = coreBase.jvm
lazy val coreJS = coreBase.js
without any need to explicitly patch the project IDs.
I face the following problem when using SBT. If I add this line to build.sbt:
unmanagedResourceDirectories in Compile <+= baseDirectory( _ / "src/main/scala" )
Incremental compilation breaks in a very tricky and not-nice way. A full example on how to reproduce the bug is here: https://github.com/vn971/sbt-incremental-bug
It's basically 2 files. Implicits.scala:
object MyImplicits {
implicit def stringToInt(str: String) = 1
}
And Usage.scala:
import MyImplicits._
object MyUsage {
def a: Int = ""
}
Now, in order to reproduce the incremental compilation bug, you have to do consequent changes to these files:
comment out the method in Usage.scala. Save file, re-compile.
uncomment it, save and re-compile.
comment out the method in Implicits.scala. Save file, re-compile.
Since MyImplicits.stringToInt is undoubtedly used in Usage.scala, it should not compile. But it does, with incremental compilation.
Thoughts? Questions? If you need more details than already provided, check out the minimalistic project I've linked to.
Is there any working solution for dropping into REPL console with for Scala 2.10?
This is mainly for debugging purpose - I want to pause in the middle of execution, and have a REPL console where I can inspect values and test the program's logic using complex expressions within my program at the current state of execution. Those who have programmed in Ruby might know similar function: the binding.pry.
AFAIK, Scala 2.9 and under used to have breakIf but it has been removed from the later versions. Using ILoop seems to be the new way but introduced issues due to sbt not adding scala-library to the classpath.
Several solutions such as this and this seem to offer a good workaround but my point is there must be a solution where I don't have to spend hours or even days just to make the REPL working.
In short, there's a lot more boilerplate steps involved - this is in contrast with binding.pry which is just a line of code with no additional boilerplate.
I am not aware if there's an issue introduced in executing the program as an sbt task as opposed to if running the program executable directly, but for development purpose I am currently running and testing my program using sbt task.
You could easily reimplement the breakIf method in your code. I don't think there is much cleaner way of doing that.
First you have to add a scala compiler library to your build.sbt
libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
Once that's done you can implement breakIf
import scala.reflect.ClassTag
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ILoop, NamedParam}
def breakIf[T](assertion: => Boolean, args: NamedParam*)(implicit tag: ClassTag[T]) = {
val repl = new ILoop()
repl.settings = new Settings()
repl.settings.embeddedDefaults[T]
repl.settings.Yreplsync.value = true
repl.in = repl.chooseReader(repl.settings)
repl.createInterpreter()
args.foreach(p => repl.bind(p.name, p.tpe, p.value))
repl.loop()
repl.closeInterpreter()
}
I think it's pretty straight forward, the only tricky part is that you have to set-up the classpath properly. You need to call embeddedDefaults with a class from your project (see my answer to another question).
You can use the new breakIf as follows:
val x = 10
breakIf[X](assertion = true, NamedParam("x", "Int", x))
Where X is just some of your classes.
I don't know if this answers your question, because it's hard to measure what is easy and what is hard.
Additionally just as a side note - if you want to use it for debugging purposes, why not use a debugger. I guess most of the debuggers can connect to a program, stop at a breakpoint and evaluate expressions in that context.
Edit
Seems like it doesn't work on current release of Scala 2.10, the working code seems to be:
import scala.reflect.ClassTag
import scala.tools.nsc.Settings
import scala.tools.nsc.interpreter.{ILoop, NamedParam}
def breakIf[T](assertion: => Boolean, args: NamedParam*)(implicit tag: ClassTag[T]) = {
val repl = new ILoop() {
override protected def postInitialization(): Unit = {
addThunk(args.foreach(p => intp.bind(p)))
super.postInitialization()
}
}
val settings = new Settings()
settings.Yreplsync.value = true
settings.usejavacp.value = true
settings.embeddedDefaults[T]
args.foreach(repl.intp.rebind)
repl.process(settings)
}
and usage is like
val x = 10
breakIf[X](assertion = true, NamedParam("x", x))
I was looking at this recently and found Ammonite to be a sufficient solution for my needs.
Add Ammonite to your library dependencies:
libraryDependencies += "com.lihaoyi" % "ammonite" % "1.6.0" cross CrossVersion.full
Invoke Ammonite where you want to drop to the REPL shell:
ammonite.Main().run()
Note that you have to pass any variables you want bound inside of run, e.g. run("var1" -> var1). Have a look at their example - Instantiating Ammonite.