How to manage DB connection in Scala using functional programming style? - scala

I have a piece of Scala code using DB connection:
def getAllProviderCodes()(implicit conf : Configuration) : List[String] = {
var conn: java.sql.Connection = null
try {
conn = DriverManager.getConnection(DBInfo.dbUrl(conf), DBInfo.dbUserName(conf), DBInfo.dbPassword(conf))
return ResultSetIterator.create(
conn.prepareStatement("SELECT pcode FROM providers").executeQuery()
){_.getString("pcode")}.toList
} catch {
case e: Exception =>
logger.warn("Something went wrong with creating the connection: " + e.getStackTrace)
} finally {
if (conn != null) {
conn.close()
}
}
List()
}
It's very OOP-Java-like style, so I'd like to know is there a way to write it in more functional way? I tried to succeed in applying Try monad, but failed: my biggest concern is that we have state here, as well as finally block. Maybe there's some kind of pattern for such cases?
Thank you in advance.
UPD: Here's the example from here of what IMHO the solution will look like:
val connection = database.getConnection()
val data: Seq[Data] = Try{
val results = connection.query("select whatever")
results.map(convertToWhatIneed)
} recover {
case t: Throwable =>
Seq.empty[Data]
} get
connection.close()
But as I've mentioned in the comment, I have to close the connection, then I have to place all the things regarding to connection inside Try to keep it pure... and then I to the variant with "try-catch-finally" inside Try block.

I've never played around with the Java SQL Connection library so the syntax of my answer has been written as pseudocode, but if I understand your question correctly here is how I would implement what you have done:
def getAllProviderCodes()(implicit conf : Configuration): List[String] = {
val conn: Connection = DriverManager.getConnection(???) // replace ??? with parameters
val result: List[String] = Try {
??? // ResultSetIterator stuff
} match {
case Success(output) => output // or whatever .toList thing
case Failure(_) => List.empty // add logging here
}
if(conn != null) conn.close()
result // will be whatever List you make (or an empty List if Try fails)
}
Instead of a Java-like try-catch-finally block, one Scala-like way of doing things would be to put the stuff which could explode in a Try block and assigning the response to a value using case Success(out) and case Failure(ex).

Just pull the connection outside of the try:
val conn = getConnection()
try {
doStuff(conn)
} finally {
conn.close
}
If you want the result of whole thing to be a Try, just wrap it into a Try:
def doDBStuff = Try {
val conn = getConnection()
try {
doStuff(conn)
} finally {
conn.close
}
}
Or with a bit less nesting (but this will throw connection exceptions):
def doDBStuff = {
val conn = getConnection()
val result = Try { doStuff(conn) }
conn.close
result
}

Related

Scala "Try" return type and exception handling

I am a newbie for Scala and now am trying to complete an exercise. How can I return an InvalidCartException while the function return type is Try[Price]
//Success: return the calculated price
//Failure: InvalidCartException
def calculateCartPrice(cart:Cart): Try[Price] = {
if(isCartValid(cart)) {
//Calculations happen here
return Try(Price(totalPrice));
}
}
def isCartValid(cart: Cart): Boolean = {
//THIS WORKS FINE
}
Thank you for the help
If you mean "how to make the Try contain an exception", then use the Failure() like below:
def calculateCartPrice(cart:Cart): Try[Price] = {
if(isCartValid(cart)) {
//Calculations happen here
Success(Price(totalPrice));
} else {
Failure(new InvalidCartException())
}
}
Then, given a Try you can use getOrElse to get the value of success or throw the exception.
Try will catch the exception for you, so put the code that can throw the exception in there. For example
def divideOneBy(x: Int): Try[Int] = Try { 1 / x}
divideOneBy(0) // Failure(java.lang.ArithmeticException: / by zero)
If what you have is a Try and you want to throw the exception when you have a Failure, then you can use pattern matching to do that:
val result = divideByOne(0)
result match {
case Failure(exception) => throw exception
case Success(_) => // What happens here?
}
The Neophyte's Guide to Scala has lots of useful information for people new to Scala (I found it invaluable when I was learning).

Catching MalformedInputException during scala.io.Source.fromFile

I'm using the scala.io.Source.fromFile method to read a csv file. Sometimes the file will be encoded in a different encoding format. I'll allow the user to specify the file enconding but...if the user doesn't specify the proper encoding I'd like to catch the MalformedInputException and then my method will return a None (instead of Some[Iterator[String]]).
I'm using the onCodingException method of the Codec but it seems that is not get applied. See below my code:
def readFileAsIterator(fileName: String,
encoding: Option[String] = Some(defaultEncoding)): Option[Iterator[String]] = {
try {
val codecType = encoding.getOrElse(defaultEncoding)
implicit val codec = Codec(codecType)
codec.onCodingException {
case e: CharacterCodingException => {
throw (new MalformedInputException(2))
}
}
val fileLines = io.Source.fromFile(fileName)(codec).getLines()
Some(fileLines)
} catch {
case e: Exception => {
None
}
}
}
Someone has played around with this method and managed to make it work?
This
io.Source.fromFile(fileName)(codec).getLines()
returns Iterator[String] which is lazy. So exception happens on iterating, not immediately on iterator creation.
Think, in general case it is not possible to detect wrong encoding without parsing before, so you need either parse file first to understand if encoding is right and than return new created iterator (not one used for parsing!), or leave exception handling to caller code, which parses data.
Or kind of trade-off, e.g. read several first lines, if ok (no coding exceptions) create new iterator for caller, but understand that in some cases caller will get exception on later wrong encoding line.
Update
Response to your comment to me under another answer.
Check this:
def readFileAsIterator(fileName: String,
encoding: Option[String] = Some("IBM1098"),
touchIterator: Boolean = false): Option[Iterator[String]] = {
try {
val codecType = encoding.getOrElse("IBM1098")
implicit val codec = Codec(codecType)
codec.onCodingException {
case e: CharacterCodingException => {
throw new MalformedInputException(2)
}
case e: java.nio.charset.UnmappableCharacterException => {
throw new MalformedInputException(3)
}
}
if (!touchIterator) {
Some(scala.io.Source.fromFile(fileName)(codec).getLines())
} else {
val i = scala.io.Source.fromFile(fileName)(codec).getLines()
if (i.hasNext) {
Some(i)
} else {
None
}
}
} catch {
case e: Exception => {
log.info(s"Handled exception in func", e)
None
}
}
}
Two calls on file which cause exception (in my case it was UnmappableCharacterException) with touching iterator and without depending on additional argument.
Under the hood you have iterator as I said. It is lazy buffered iterator. So it is initialized on first call (in modified method I force to initialize it with hasNext).
I do not think that it reads whole file, just buffer part of it (so it is automated implementation of my "trade-off case").
There are two things which you should think about modifying here,
1 - Return a Try[Iterator[String]] instead of Option[Iterator[String]]
2 - encoding can be a String with a default value.
def readFileAsIterator(fileName: String, encoding: String = "UTF-8"): Try[Iterator[String]] = Try({
implicit val codec = Codec(encoding)
codec.onCodingException({
case e: CharacterCodingException =>
throw (new MalformedInputException(2))
})
io.Source.fromFile(fileName)(codec).getLines()
})
Had same error. I handled it using onMalformedInput() as shown below:
implicit val codec = Codec("UTF-8")
codec.onMalformedInput(CodingErrorAction.REPLACE)
codec.onUnmappableCharacter(CodingErrorAction.REPLACE)
for(line <- Source.fromFile("..").getLines()) {
...
}

closing file pointer in Scala in Finally

In the following code, I am reading no. of lines from a file. If something goes wrong, I'll like to close the file pointer. But how can I find out if f contains valid pointer or not?
def countLines(filename:String):Option[Int] = {
try{
val f = Source.fromFile(filename)
println(s"no. of lines ${f.getLines().size}")
Some(f.getLines.size)
} catch {
case ex: FileNotFoundException => {
println(s"file ${filename} not found")
None
}
} finally {
//f might not be a valid pointer depending on when the error occured
}
}
The book I am reading uses var to maintain state (if f is valid or not) but I am trying to avoid it for sake of using only immutable variables.
def countLines(filename:String):Option[Int] = {
var f:Option[Source] = None
try{
f = Some(Source.fromFile(filename))
println(s"no. of lines ${f.get.getLines().size}")
Some(f.get.getLines.size)
} catch {
case ex: FileNotFoundException => {
println(s"file ${filename} not found")
None
}
} finally {
for(i<-f){
println("closing file")
i.close()
}
}
}
A double Try(). This closes the io resource even if the getLines() fails, but only if the fromFile() succeeds.
import scala.util.Try
def countLines(filename: String): Option[Int] =
Try(io.Source.fromFile(filename)).fold(_ => None, {f =>
val count = Try(f.getLines().length)
f.close()
count.toOption
})
What do you think about this?
If you want Scala-way - i think it's good example for your task:
def countLines(filename: String): Try[Int] = Try(Source.fromFile(filename).getLines.toList.size)
def test() = {
val filename = "/etc/passwd"
countLines(filename) match {
case Success(n) => println(n)
case Failure(f) => println(f)
}
}
When n - is a number of our lines, and f - is a Throwable.
How about this:
def countLines(filename: String): Option[Int] = {
val file = Try(Source.fromFile(filename))
val count = file.map(_.getLines().size)
(for {
_ <- count.recoverWith { case _ => file.map(_.close()) }
lineCount <- count
} yield lineCount).toOption
}
Let's analyze it:
If file does not exist we will have failed Try instance and method returns None. In this case you do not need to clear any resources as no actual stream was created.
If getLines fails for any reason or anything else during processing goes south we will close created stream in first line of for comprehension
Hope it helps
Simply, how about this:
def numLines(fileName:String):Option[Int] = {
try {
val f = scala.io.Source.fromFile(fileName)
try { Some(f.getLines.size) }
catch { case ex: IOException =>
Console.err.println("i/o excetion")
None
}
finally { f.close() }
}
catch {
case ex: FileNotFoundException =>
Console.err.println("file not found")
None
}
}

Why a Thread.sleep or closing the connection is required after waiting for a remove call to complete?

I'm again seeking you to share your wisdom with me, the scala padawan!
I'm playing with reactive mongo in scala and while I was writting a test using scalatest, I faced the following issue.
First the code:
"delete" when {
"passing an existent id" should {
"succeed" in {
val testRecord = TestRecord(someString)
Await.result(persistenceService.persist(testRecord), Duration.Inf)
Await.result(persistenceService.delete(testRecord.id), Duration.Inf)
Thread.sleep(1000) // Why do I need that to make the test succeeds?
val thrownException = intercept[RecordNotFoundException] {
Await.result(persistenceService.read(testRecord.id), Duration.Inf)
}
thrownException.getMessage should include(testRecord._id.toString)
}
}
}
And the read and delete methods with the code initializing connection to db (part of the constructor):
class MongoPersistenceService[R](url: String, port: String, databaseName: String, collectionName: String) {
val driver = MongoDriver()
val parsedUri: Try[MongoConnection.ParsedURI] = MongoConnection.parseURI("%s:%s".format(url, port))
val connection: Try[MongoConnection] = parsedUri.map(driver.connection)
val mongoConnection = Future.fromTry(connection)
def db: Future[DefaultDB] = mongoConnection.flatMap(_.database(databaseName))
def collection: Future[BSONCollection] = db.map(_.collection(collectionName))
def read(id: BSONObjectID): Future[R] = {
val query = BSONDocument("_id" -> id)
val readResult: Future[R] = for {
coll <- collection
record <- coll.find(query).requireOne[R]
} yield record
readResult.recover {
case NoSuchResultException => throw RecordNotFoundException(id)
}
}
def delete(id: BSONObjectID): Future[Unit] = {
val query = BSONDocument("_id" -> id)
// first read then call remove. Read will throw if not present
read(id).flatMap { (_) => collection.map(coll => coll.remove(query)) }
}
}
So to make my test pass, I had to had a Thread.sleep right after waiting for the delete to complete. Knowing this is evil usually punished by many whiplash, I want learn and find the proper fix here.
While trying other stuff, I found instead of waiting, entirely closing the connection to the db was also doing the trick...
What am I misunderstanding here? Should a connection to the db be opened and close for each call to it? And not do many actions like adding, removing, updating records with one connection?
Note that everything works fine when I remove the read call in my delete function. Also by closing the connection, I mean call close on the MongoDriver from my test and also stop and start again embed Mongo which I'm using in background.
Thanks for helping guys.
Warning: this is a blind guess, I've no experience with MongoDB on Scala.
You may have forgotten to flatMap
Take a look at this bit:
collection.map(coll => coll.remove(query))
Since collection is Future[BSONCollection] per your code and remove returns Future[WriteResult] per doc, so actual type of this expression is Future[Future[WriteResult]].
Now, you have annotated your function as returning Future[Unit]. Scala often makes Unit as a return value by throwing away possibly meaningful values, which it does in your case:
read(id).flatMap { (_) =>
collection.map(coll => {
coll.remove(query) // we didn't wait for removal
() // before returning unit
})
}
So your code should probably be
read(id).flatMap(_ => collection.flatMap(_.remove(query).map(_ => ())))
Or a for-comprehension:
for {
_ <- read(id)
coll <- collection
_ <- coll.remove(query)
} yield ()
You can make Scala warn you about discarded values by adding a compiler flag (assuming SBT):
scalacOptions += "-Ywarn-value-discard"

Scala try with finally best practice

I have the following implementation where I'm trying to handle proper resource closing during any fatal exceptions:
private def loadPrivateKey(keyPath: String) = {
def tryReadCertificate(file: File): Try[BufferedReader] = Try { new BufferedReader(new FileReader(file)) }
def tryLoadPemParser(reader: BufferedReader): Try[PEMParser] = Try { new PEMParser(reader) }
def createXXX(buffReader: BufferedReader, pemParser: PEMParser) = try {
...
} finally {
buffReader.close()
pemParser.close()
}
tryReadCertificate(new File(keyPath, "myKey.pem")) match {
case Success(buffReader) => tryLoadPemParser(buffReader) match {
case Success(pemParser) => createXXX(buffReader, pemParser)
case Failure(fail) =>
}
case Failure(fail) =>
}
}
I already see that my nested case blocks are a mess. Is there a better way to do this? In the end, I just want to make sure that I close the BufferedReader and the PEMParser !
You could restructure your code a little like this, using a for-comprehension to clean up some of the nested case statements:
def tryReadCertificate(file: File): Try[BufferedReader] = Try { new BufferedReader(new FileReader(file)) }
def tryLoadPemParser(reader: BufferedReader): Try[PEMParser] = Try { new PEMParser(reader) }
def createXXX(buffReader: BufferedReader, pemParser: PEMParser) = {
...
}
val certReaderTry = tryReadCertificate(new File(keyPath, "myKey.pem"))
val pemParserTry = for{
certReader <- certReaderTry
pemParser <- tryLoadPemParser(certReader)
} yield {
createXXX(certReader, pemParser)
pemParser
}
certReaderTry foreach(_.close)
pemParserTry foreach (_.close)
Structured like this, you will only ever end up calling close on things you are sure were opened successfully.
And even better, if your PEMParser happened to extend java.io.Closeable, meaning that the Trys both wrapped Closeable objects, then you could swap those last two lines for a single line like this:
(certReaderTry.toOption ++ pemParserTry.toOption) foreach (_.close)
EDIT
In response to the OP's comment: In the first example, if tryreadCertificate succeeds, then certReaderTry will be a Success[BufferedReader] and because it's successful, calling foreach on it will yield the BufferedReader which will then have close called on it. If certReaderTry is Success, then (via the for-comp) we will call tryLoadPemParser and if that also succeeds, we can move on to createXXX and assign the tryLoadPemParser to the pemParserTry val. Then, later, if pemParserTry is a Success, the same thing happens where foreach yields the PEMParser and we can close it. Per this example, as long as the those Trys are successes and something else unexpected does not happen (in createXXX for example) that would throw an exception all the way out, then you can guarantee that the closing related code at the end will do its job and close those resources.
EDIT2
If you wanted the value from createXXX in a separate Try, then you could do something like this:
val certReaderTry = tryReadCertificate(new File(keyPath, "myKey.pem"))
val pemParserTry = certReaderTry.flatMap(tryLoadPemParser)
val resultTry = for{
certReader <- certReaderTry
pemParser <- pemParserTry
} yield createXXX(certReader, pemParser)