When I try to run (from Play Framework):
import play.api.db.DB
import anorm._
import org.joda.time.DateTime
import AnormExtensions._ // http://stackoverflow.com/a/11975107/11236
import play.api.Play.current
import java.util.Date
var stream = SQL("SELECT amiId, created, version FROM Amis WHERE created = {maxCreated}")
.on("maxCreated" -> new Date(maxCreated.getMillis))
.apply()
val map: Stream[Ami] = stream.map { ami =>
val s: String = ami[String]("amiId")
val date: Date = ami[Date]("created")
// The following line throws a compilation error
var version: Integer = ami[Integer]("version")
new Ami(s, new DateTime(date), version)
}
I get a compilation error : could not find implicit value for parameter c: anorm.Column[Integer]
What's the problem here? If I can read a Date, why can't I read an Integer?
The problem was that I used java java.lang.Integer instead of scala.Int.
Such missing numeric conversion was fixed with release of Play 2.3. You may want upgrade to.
Related
How to print the date of a file in Scala is explained here.
My question is how I can get a variable containing this information which can be returned as a column to a dataframe. None of the conversions I would expect to be allowed, actually are allowed.
My code (using Scala 2.11):
import org.apache.spark.sql.functions._
import java.nio.file.{Files, Paths} // Needed for file time
import java.nio.file.attribute.BasicFileAttributes
import java.util.Date
def GetFileTimeFunc(pathStr: String): String = {
// From: https://stackoverflow.com/questions/47453193/how-to-get-creation-date-of-a-file-using-scala
val FileTime = Files.readAttributes(Paths.get(pathStr), classOf[BasicFileAttributes]).creationTime;
val JavaDate = Date.from(FileTime.toInstant);
return(JavaDate.toString())
}
#transient val GetFileTime = udf(GetFileTimeFunc _)
val filePath = "dbfs:/mnt/myData/" // location of data
val file_df = dbutils.fs.ls(filePath).toDF // Output columns are $"path", $"name", and $"size"
.withColumn("FileTimeCreated", GetFileTime($"path"))
display(file_df)//.select("name", "size"))
Output:
SparkException: Failed to execute user defined function($anonfun$2: (string) => string)
For some reason, Instant is not allowed as a column type, so I cannot use it as a return type.The same for FileTime, JavDate, etc.
IntelliJ complains about this code:
val document: Node // (initialized further up in the code)
val s: String = (new scala.xml.PrettyPrinter(80, 4)).format(document))
With the error:
Cannot resolve reference format with such signature
However - such a function exists. It has a default value for the second parameter and it seems IntelliJ isn't identifying it correctly.
I am not sure about that specific error you mention, but you have one parenthesis too many. You have:
val s: String = (new scala.xml.PrettyPrinter(80, 4)).format(document))
It should be:
val s: String = (new scala.xml.PrettyPrinter(80, 4)).format(document)
I just tried you code in sbt (once I made that correction) and it seems fine:
scala> import scala.xml._
import scala.xml._
scala> val document : Node = <test>blah</test>
document: scala.xml.Node = <test>blah</test>
scala> val s: String = (new PrettyPrinter(80, 4)).format(document)
s: String = <test>blah</test>
I'm trying to get basic file attributes using Scala, and my reference is this Java question:
Determine file creation date in Java
and this piece of code I'm trying to rewrite in Scala:
static void getAttributes(String pathStr) throws IOException {
Path p = Paths.get(pathStr);
BasicFileAttributes view
= Files.getFileAttributeView(p, BasicFileAttributeView.class)
.readAttributes();
System.out.println(view.creationTime()+" is the same as "+view.lastModifiedTime());
}
The thing I just can't figure out is this line of code..I don't understand how to pass a class in this way using scala... or why Java is insisting upon this in the first place instead of using an actual constructed object as the parameter. Can someone please help me write this line of code to function properly? I must be using the wrong syntax
val attr = Files.readAttributes(f,Class[BasicFileAttributeView])
Try this:
def attrs(pathStr:String) =
Files.getFileAttributeView(
Paths.get(pathStr),
classOf[BasicFileAttributes] //corrected
).readAttributes
Get file creation date in Scala, from Basic Files Attributes:
// option 1,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributes
val pathStr = "/tmp/test.sql"
Files.readAttributes(Paths.get(pathStr), classOf[BasicFileAttributes]).creationTime
res3: java.nio.file.attribute.FileTime = 2018-03-06T00:25:52Z
// option 2,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributeView
val pathStr = "/tmp/test.sql"
{
Files
.getFileAttributeView(Paths.get(pathStr), classOf[BasicFileAttributeView])
.readAttributes.creationTime
}
res20: java.nio.file.attribute.FileTime = 2018-03-07T19:00:19Z
I've found this interesting behaviour in nscala_time package (scala version of joda-time)
import com.github.nscala_time.time.Imports._
import com.github.nscala_time.time.DurationBuilder
object tests {
val x = 3 seconds
//> x : is of type com.github.nscala_time.time.DurationBuilder
val xx: DurationBuilder = 3 seconds
//> fails to compile:
// class DurationBuilder in package time cannot be accessed in package com.github.nscala_time.time
}
What I'm trying to achieve is implicit conversion from nscala_time Duration to scala.concurrent.Duration
I need this becuase I'm using RxScala and nscala_time in one application.
// e.g. the following should be implicitly converted
// to nscala_time Duration first
// than to scala.lang.concurrent.Duration
3 seconds
nscala_time offers rich time & date api for my application, while I'm using RxScala in the same class for GUI responsivness.
You can download a simple project to play around: https://dl.dropboxusercontent.com/u/9958045/implicit_vs_private.zip
From scala-user group: It's a known issue https://issues.scala-lang.org/browse/SI-1800
perhaps you can use an implicit conversion? (btw Duration in nscala is essentially org.joda.time.Duration):
scala> import com.github.nscala_time.time.Imports._
import com.github.nscala_time.time.Imports._
scala> implicit class DurationHelper(d:org.joda.time.Duration) {
| def toScalaDuration = scala.concurrent.duration.Duration.apply(d.getMillis,scala.concurrent.duration.MILLISECONDS)
| }
defined class DurationHelper
scala> val d = RichInt(3).seconds.toDuration
// toDuration method is defined for com.github.nscala_time.time.DurationBuilder
d: org.joda.time.Duration = PT3S
scala> def exfun(d:scala.concurrent.duration.Duration) = d.toString
exfun: (d: scala.concurrent.duration.Duration)String
scala> exfun(d)
res41: String = 3000 milliseconds
(not using import scala.concurrent.duration._ here to avoid name clashes with joda/nlscala stuff)
I'm having some trouble doing a Plain SQL Query using String Interpolation. I'm trying to query a MS-SQL 2008 database using Microsofts sqljdbc4.jar.
The table is simple:
[serv_num]
number (nvarchar(15)) | name (nvarchar(50))
=============================================
1234 | AA
+345 | BB
The code I'm using is as follows:
import java.sql.{Connection, DriverManager}
import org.slf4j.LoggerFactory
import scala.collection.JavaConversions._
import scala.slick.session.Database
import Database.threadLocalSession
import scala.slick.jdbc.{GetResult, StaticQuery => Q}
import Q.interpolation
object NumImport extends App {
def logger = LoggerFactory.getLogger("NumImport")
implicit val getServNumResult = GetResult(r => ServNum(r.<<, r.<<))
override def main(args: Array[String]) {
val uri = "jdbc:sqlserver://192.168.1.2;databaseName=slicktestdb;user=yyyy;password=xxxxxx"
val drv = "com.microsoft.sqlserver.jdbc.SQLServerDriver"
Database.forURL(uri, driver = drv) withSession {
def getServNum(n: String) = sql"select number, name from serv_num where number = $n".as[ServNum]
val result = getServNum("1234")
println(result.firstOption)
}
}
case class ServNum(num: String, name: String)
}
My issue is that I'm getting a java.lang.NullPointerException on result.firstOption (I think!). This happens when a row is found. If try to select a number that doesn't exist, say getServNum("34"), then result.firstOption is a None, as expected.
Stacktrace:
08:54:47.932 TKD [run-main] DEBUG scala.slick.session.BaseSession - Preparing statement: select number, name from serv_num where number = ?
[error] (run-main) java.lang.NullPointerException
java.lang.NullPointerException
at scala.slick.jdbc.StaticQuery.extractValue(StaticQuery.scala:16)
at scala.slick.jdbc.StatementInvoker$$anon$1.extractValue(StatementInvoker.scala:37)
at scala.slick.session.PositionedResultIterator.foreach(PositionedResult.scala:212)
at scala.slick.jdbc.Invoker$class.foreach(Invoker.scala:91)
at scala.slick.jdbc.StatementInvoker.foreach(StatementInvoker.scala:10)
at scala.slick.jdbc.Invoker$class.firstOption(Invoker.scala:41)
at scala.slick.jdbc.StatementInvoker.firstOption(StatementInvoker.scala:10)
at scala.slick.jdbc.UnitInvoker$class.firstOption(Invoker.scala:148)
at scala.slick.jdbc.StaticQuery0.firstOption(StaticQuery.scala:95)
I tried debugging and following the stacktrace, but I'm quite new to both Scala and Slick, so I'm lost. If anyone could help me or point me in the right direction I would appreciate it a lot.
Thanks.
In looking at the slick source code where the exception was happening, I could see that it was trying to invoke apply on the GetResult object, but that object seemed to be null. In thinking about what might of caused it, I'm suggesting that you move the implicit inside of the main method. It seems that the way things are structured, slick is losing track of the implicit ResultSet to case class converter (the GetResult)