found java.util.Date but required java.sql.Date? - scala

I'm trying to create a function to check if a string is a date. However, the following function got the error.
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import java.sql._
import scala.util.{Success, Try}
def validateDate(date: String): Boolean = {
val df = new java.text.SimpleDateFormat("yyyyMMdd")
val test = Try[Date](df.parse(date))
test match {
case Success(_) => true
case _ => false
}
}
Error:
[error] C:\Users\user1\IdeaProjects\sqlServer\src\main\scala\main.scala:14: type mismatch;
[error] found : java.util.Date
[error] required: java.sql.Date
[error] val test = Try[Date](df.parse(date))
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 2 s, completed May 17, 2017 1:19:33 PM
Is there a simpler way to validate if a string is a date without create a function?
The function is used to validate the command line argument.
if (args.length != 2 || validateDate(args(0))) { .... }

Try[Date](df.parse(date)) You are not interested in type here because you ignore it. So simply omit type parameter. Try(df.parse(date)).
Your function could be shorter. Try(df.parse(date)).isSuccess instead pattern matching.
If your environment contains java 8 then use java.time package always.
import scala.util.Try
import java.time.LocalDate
import java.time.format.DateTimeFormatter
// Move creation of formatter out of function to reduce short lived objects allocation.
val df = DateTimeFormatter.ofPattern("yyyy MM dd")
def datebleStr(s: String): Boolean = Try(LocalDate.parse(s,df)).isSuccess

use this: import java.util.Date

Related

Cannot find an implicit value for ContextShift

I am trying to create webapp with http4s that is based on Http4sServlet.
The following code does not compile:
import cats.effect._
import org.http4s.servlet.BlockingServletIo
import org.http4s.servlet.Http4sServlet
import scala.concurrent.ExecutionContext.global
import org.http4s.implicits._
class UserSvcServlet
extends Http4sServlet[IO](service = UserSvcServer.start
, servletIo = BlockingServletIo(4096, Blocker.liftExecutionContext(global)))(IOApp)
the error message:
[error] /home/developer/scala/user-svc/src/main/scala/io/databaker/UserSvcServlet.scala:12:54: Cannot find implicit value for ConcurrentEffect[[+A]cats.effect.IO[A]].
[error] Building this implicit value might depend on having an implicit
[error] s.c.ExecutionContext in scope, a Scheduler, a ContextShift[[+A]cats.effect.IO[A]]
[error] or some equivalent type.
[error] extends Http4sServlet[IO]( service = UserSvcServer.stream
[error] ^
[error] /home/developer/scala/user-svc/src/main/scala/io/databaker/UserSvcServlet.scala:13:36: Cannot find an implicit value for ContextShift[[+A]cats.effect.IO[A]]:
[error] * import ContextShift[[+A]cats.effect.IO[A]] from your effects library
[error] * if using IO, use cats.effect.IOApp or build one with cats.effect.IO.contextShift
[error] , servletIo = BlockingServletIo(4096, Blocker.liftExecutionContext(global)))
[error] ^
[error] two errors found
[error] (Compile / compileIncremental) Compilation failed
[error] Total time: 1 s, completed May 29, 2020, 8:45:00 PM
The UserSvcServer is implemented as follows:
import org.http4s.HttpApp
import cats.effect.{ConcurrentEffect, ContextShift, Timer}
import org.http4s.implicits._
import org.http4s.server.middleware.Logger
object UserSvcServer {
def start[F[_] : ConcurrentEffect](implicit T: Timer[F], C: ContextShift[F]): HttpApp[F] = {
val helloWorldAlg = HelloWorld.impl[F]
val httpApp = UserSvcRoutes.helloWorldRoutes[F](helloWorldAlg).orNotFound
Logger.httpApp(true, true)(httpApp)
}
}
How can I import ContextShift implicitly?
Context shift is just cats' wrapper over ExecutionContext. You can create one explicitly as stated in docs:
implicit val cs: ContextShift[IO] = IO.contextShift(ExecutionContext.global)
You usually create one context shift at the entry point of your app (probably in the main method).
If your app uses IOApp from cats-effect it would have already implicit for contextShift in scope. It will use execution context, which would have number of threads equal to available processors of your computer.
If you want to use created contextShift "deeper" inside your application you can pass it as an implicit parameter:
def doSomething(implicit cs: ContextShift[IO]): IO[Unit] = ???
So in order to make your code work, you need to make sure that method or class calls constructor of UserSvcServlet has implicit for contextShift:
(implicit cs: ContextShift[IO]).
You could also put it in separate object:
object AppContextShift {
implicit val cs: ContextShift[IO] = IO.contextShift(ExecutionContext.global)
implicit val t: Timer[IO] = IO.timer(ExecutionContext.global) //you will probably also need timer eventually
}
Then you can import it when contextShift is needed:
import AppContextShift._
By the way, using a global execution context for Blocker is not good idea.
Blocked is used for blocking operations and using it with ExecutionContext.global might lead to thread starvation in your app.
The most common approach is to use blocker created from cached thread pool:
Blocker.liftExecutorService(Executors.newCachedThreadPool())

Scala - making a WS http request

UPDATED: Method is returning type ANY rather than type Future[string]. Require return of type String.
I'm making a http request using the play.ws library 2.6. This was previously done with a curl request but this only uses basic authentication.
Below is my code and I'm trying to return a json string from this function to be deserialised in another method.
import java.io.{File, InputStream}
import java.nio.file.Paths
import javax.inject._
import org.apache.commons.io.FileUtils
import play.api._
import play.api.http.HttpEntity
import play.api.libs.ws._
import play.api.mvc._
import play.api.Play.current
import scala.collection.mutable.ListBuffer
import scala.concurrent.Await
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import sys.process._
#Singleton
class BuildService #Inject() (
ws: WSClient,
ec: ExecutionContext,
config: Configuration) {
def bbApiRequest(requestUrl: String, timeout: FiniteDuration):
Future[String] = {
val request = ws
.url(requestUrl)
.withAuth(
"user",
"pw1234",
WSAuthScheme.BASIC)
.get()
Await.result(request, timeout)
val returner = request.map(_.json)
} // <-- line 72 in below error points here.
}
When run it produces the error:
[error] C:\my_path\app\services\BuildService.scala:72: type mismatch;
[error] found : Unit
[error] required: scala.concurrent.Future[String]
[error] }
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[info] Compiling 1 Scala source to C:\my_path\restapi\target\scala-
2.12\classes...
[error] C:\my_path\restapi\app\services\BuildService.scala:72: type
mismatch;
[error] found : Unit
[error] required: scala.concurrent.Future[String]
[error] }
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
I'm trying to get ideally:
A string return (case class and json method to unpack a string)
Synchronous request (if async I'll need to wait to complete to progress application)
Secure (allows use of tokens for verification)
Appreciate any help!
Here is the function I use:
// first work with Futures the Play Controller will support that!
def bbApiRequest(requestUrl: String): Future[String] = {
// call the webservice with basic authentication
wsClient.url(requestUrl)
.withAuth("tester", "pwd123", WSAuthScheme.BASIC)
.get()
.map(checkStatus) // function that evaluates the HTTP Status
.map(_.json) // get the json
.map(Json.prettyPrint) // get it as string
}
I would create a case class directly like:
.map(jsValue => handleJson(jsValue.validate[YourModel])) // or create a model of it (instead) instead of .map(Json.prettyPrint)
Edit
Here an example of checkStatus:
protected def checkStatus(resp: WSResponse): WSResponse = {
resp.status match {
case Status.OK => resp
case Status.NOT_FOUND => throw WebNotFoundException()
case Status.FORBIDDEN | Status.UNAUTHORIZED => throw WebAccessForbiddenException()
case Status.NOT_ACCEPTABLE => throw WebNotAcceptableException()
case _ => throw WebBadStatusException(resp.status + " - " + resp.statusText.toString)
}
}
The Exception are created by myself.
The authorization can also be inputs into the function or retrieved from environment variables within the method (much easier to manage).
Simply needed to use .body on the Await call, which converts the output to generic type string.
package utils
import javax.inject._
import play.api._
import play.api.http.HttpEntity
import play.api.libs.ws._
import play.api.mvc._
import play.api.Play.current
import scala.collection.mutable.ListBuffer
import scala.concurrent.Await
import scala.concurrent.duration._
import scala.concurrent.ExecutionContext
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Future
import sys.process._
#Singleton
class HTTPRequest #Inject() (
ws: WSClient,
ec: ExecutionContext,
config: Configuration) {
def bbApiRequest(requestUrl: String, timeout: FiniteDuration) = {
val request = ws
.url(requestUrl)
.withAuth(
"user",
"PW123",
WSAuthScheme.BASIC)
.get()
Await.result(request, timeout).body
}
}

Combine Two Slick Futures and then execute them together

I have written this code and I am trying to combine two futures obtained from separate SQL operations.
package com.example
import tables._
import scala.concurrent.{Future, Await}
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
import slick.backend.DatabasePublisher
import slick.driver.H2Driver.api._
object Hello {
def main(args: Array[String]): Unit = {
val db = Database.forConfig("h2mem1")
try {
val people = TableQuery[Persons]
val setupAction : DBIO[Unit] = DBIO.seq(
people.schema.create
)
val setupFuture : Future[Unit] = db.run(setupAction)
val populateAction: DBIO[Option[Int]] = people ++= Seq(
(1, "test1", "user1"),
(2, "test2", "user2"),
(3, "test3", "user3"),
(4, "test4", "user4")
)
val populateFuture : Future[Option[Int]] = db.run(populateAction)
val combinedFuture : Future[Option[Int]] = setupFuture >> populateFuture
val r = combinedFuture.flatMap { results =>
results.foreach(x => println(s"Number of rows inserted $x"))
}
Await.result(r, Duration.Inf)
}
finally db.close
}
}
But I get an error when I try to compile this code
[error] /Users/abhi/ScalaProjects/SlickTest2/src/main/scala/Hello.scala:29:
value >> is not a member of scala.concurrent.Future[Unit]
[error] val combinedFuture : Future[Option[Int]] = setupFuture >>
populateFuture
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
The same code works, If I nest the populateFuture inside the map function of the setupFuture. But I don't want to write nested code because it will become very messy once there are more steps to do.
So I need a way to combine all futures into a single future and then execute it.
Edit:: I also tried combining the two actions
val combinedAction = setupAction.andThen(populateAction)
val fut1 = combinedAction.map{result =>
result.foreach{x =>println(s"number or rows inserted $x")}
}
Await.result(fut1, Duration.Inf)
but got error
/Users/abhi/ScalaProjects/SlickTest/src/main/scala/com/example/Hello.scala:31: type mismatch;
[error] found : scala.concurrent.Future[Option[Int]]
[error] required: PartialFunction[scala.util.Try[Unit],?]
[error] val combinedAction = setupAction.andThen(populateAction)
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
[error] Total time: 3 s, completed Jun 26, 2015 3:50:51 PM
Mohitas-MBP:SlickTest abhi$
According to http://slick.typesafe.com/doc/3.0.0/api/index.html#slick.dbio.DBIOAction, andThen() is what you are looking for:
val combinedAction = setupAction.andThen(populateAction)
val results = db.run(combinedAction)
populateAction will only run after setupAction completed successfully. This is crucial in your case since slick is fully non-blocking. The code you have now will cause problems at runtime. Both actions in your code will run asynchronously at the same time. There is no way to determine which action is executed first. But because populateAction depends on setupAction, you must ensure setupAction is executed first. Therefore use andThen.

Type mismatch error between Joda DateTime and sql Timestamp in Slick GetResult

I want to retrieve data of a column with type DateTime and I am using jodatime for it. Even though I have a custom TypeMapper, I am getting type mismatch error.
[error] C:\sample\modules\test\com\samp\user.scala:55: type mismatch;
[error] found : java.sql.Timestamp
[error] required: org.joda.time.DateTime
[error] result.nextTimestamp,
[error] ^
[error] one error found
Here is the code
import org.specs2.mutable.Specification
import scala.slick.session.Database
import Database.threadLocalSession
import scala.slick.jdbc.{GetResult, StaticQuery => Q}
import Q.interpolation
import org.joda.time.DateTime
import scala.slick.lifted.{MappedTypeMapper,TypeMapper}
import java.sql.Timestamp
class UserSpec
extends Specification {
"User tests " should {
"get all the user data in db" in
new WithServer() {
val db = Database.forURL(url = "jdbc:h2:/path/to/db-file", driver = "com.mysql.jdbc.Driver")
implicit def dateTimeTypeMapper = MappedTypeMapper.base[DateTime, Timestamp] (
dateTime => new Timestamp(dateTime.getMillis()),
timeStamp => new DateTime(timeStamp)
)
case class user(
id: String,
name: String,
joinedAt: DateTime,
description: Option[String]
)
implicit val getUserResult: GetResult[user] =
GetResult( result =>
user(
result.nextString,
result.nextString,
result.nextTimestamp,
result.nextStringOption)
)
db withSession {
val usr = Q.queryNA[user]("SELECT * FROM user").list()
usr foreach println
}
}
}
}
I am not sure why this is not working. Any help or pointers is greatly appreciated.
The problem here is that result.nextTimestamp returns a java.sql.Timestamp (result is the JDBC ResultSet). If you want to turn it into a DateTime, I think that there's a constructor for it. So you'd write:
implicit val getUserResult: GetResult[user] =
GetResult( result =>
user(
result.nextString,
result.nextString,
new DateTime(result.nextTimestamp),
result.nextStringOption)
)

Scalatra JSON guide code not compiling

I copied the instructions on the JSON formatting Guide found here: http://www.scalatra.org/2.2/guides/formats/json.html
Below is MyScalatraServlet.scala file with all of the code I embedded to test out JSON formatting:
package net.example.testapp
import org.scalatra._
import scalate.ScalateSupport
// JSON-related libraries
import org.json4s.{DefaultFormats, Formats}
// JSON handling support from Scalatra
import org.scalatra.json._
class MyScalatraServlet extends TestAppStack with JacksonJsonSupport {
get("/") {
FlowerData.all
}
}
// Sets up automatic case class to JSON output serialization, required by
// the JValueResult trait.
protected implicit val jsonFormats: Formats = DefaultFormats
case class Flower(slug: String, name: String)
object FlowerData {
/**
* Some fake flowers data so we can simulate retrievals.
*/
var all = List(
Flower("yellow-tulip", "Yellow Tulip"),
Flower("red-rose", "Red Rose"),
Flower("black-rose", "Black Rose"))
}
It seems the compiler does not like the following line:
protected implicit val jsonFormats: Formats = DefaultFormats
Here's the error message:
[error] /Users/test/test-app/src/main/scala/net/example/testapp/MyScalatraServlet.scala:22: expected start of definition
[error] protected implicit val jsonFormats: Formats = DefaultFormats
[error] ^
[error] one error found
[error] (compile:compile) Compilation failed
[error] Total time: 2 s, completed Jun 17, 2013 4:04:34 PM
>
Your val jsonFormats is just kinda floating there, not tied to a class or anything. A val needs to be defined as part of another construct like a trait, class or object. Try moving it inside of that servlet class, just before the call to get("/")