I'm trying to get basic file attributes using Scala, and my reference is this Java question:
Determine file creation date in Java
and this piece of code I'm trying to rewrite in Scala:
static void getAttributes(String pathStr) throws IOException {
Path p = Paths.get(pathStr);
BasicFileAttributes view
= Files.getFileAttributeView(p, BasicFileAttributeView.class)
.readAttributes();
System.out.println(view.creationTime()+" is the same as "+view.lastModifiedTime());
}
The thing I just can't figure out is this line of code..I don't understand how to pass a class in this way using scala... or why Java is insisting upon this in the first place instead of using an actual constructed object as the parameter. Can someone please help me write this line of code to function properly? I must be using the wrong syntax
val attr = Files.readAttributes(f,Class[BasicFileAttributeView])
Try this:
def attrs(pathStr:String) =
Files.getFileAttributeView(
Paths.get(pathStr),
classOf[BasicFileAttributes] //corrected
).readAttributes
Get file creation date in Scala, from Basic Files Attributes:
// option 1,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributes
val pathStr = "/tmp/test.sql"
Files.readAttributes(Paths.get(pathStr), classOf[BasicFileAttributes]).creationTime
res3: java.nio.file.attribute.FileTime = 2018-03-06T00:25:52Z
// option 2,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributeView
val pathStr = "/tmp/test.sql"
{
Files
.getFileAttributeView(Paths.get(pathStr), classOf[BasicFileAttributeView])
.readAttributes.creationTime
}
res20: java.nio.file.attribute.FileTime = 2018-03-07T19:00:19Z
Related
I am working on a Gatling test framework that can be parameterized through external config objects. One use case I have is that there may be zero or more CSS selector checks that need to be saved to variables. In my config object, I've implemented that as a Map[String,(String, String)], where the key is the variable name, and the value is the 2-part css selector.
I am struggling with how to dynamically assemble the check. Here's what I got so far:
val captureMap: Map[String, (String, String)] = config.capture
httpRequestBuilder.check(
captureMap.map((mapping) => {
val varName = mapping._1
val cssSel = mapping._2
css(cssSel._1, cssSel._2).saveAs(varName)
}).toArray: _* // compilation error here
)
The error I'm getting is:
Error:(41, 10) type mismatch;
found : Array[io.gatling.core.check.CheckBuilder[io.gatling.core.check.css.CssCheckType,jodd.lagarto.dom.NodeSelector,String]]
required: Array[_ <: io.gatling.http.check.HttpCheck]
}).toArray: _*
apparently, I need to turn my CheckBuilder into a HttpCheck, so how do I do that?
Update:
I managed to get it to work by introducing a variable of type HttpCheck and returning it in the next line:
httpRequestBuilder.check(
captureMap.map((mapping) => {
val varName = mapping._1
val cssSel = mapping._2
val check:HttpCheck= css(cssSel._1, cssSel._2).saveAs(varName)
check
}).toArray: _*
)
While this works, it's ugly as hell. Can this be improved?
I had the same issue.
I had the following imports:
import io.gatling.core.Predef._
import io.gatling.http.Predef.http
I changed these imports to:
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.http.request.builder.HttpRequestBuilder.toActionBuilder
which made it work.
I need to fetch the date from a file.
Below is my spark program:
import org.apache.spark.sql.SparkSession
import scala.xml.XML
import java.text.SimpleDateFormat
object Active6Month {
def main(args:Array[String]){
val format = new SimpleDateFormat("yyyy-MM-dd'T'hh:mm:ss.SSS")
val format1 = new SimpleDateFormat("yyyy-MM")
val spark = SparkSession.builder.appName("Active6Months").master("local").getOrCreate()
val data = spark.read.textFile("D:\\BGH\\StackOverFlow\\Posts.xml").rdd
val date = data.filter{line => {
line.toString().trim().startsWith("<row")
}}.filter{line=>{
line.contains("PostTypeId=\"1\"")
}}.map{line=>{
val xml = XML.loadString(line)
var closedDate = format1.format(format.parse(xml.attribute("ClosedDate").toString())).toString()
(closedDate,1)
}}.reduceByKey(_+_)
date.foreach(println)
spark.stop
}
}
And I am getting this error:
java.text.ParseException: Unparseable date: "Some(2014-05-14T14:40:25.950)"
The format of date in file is perfect i.e:
CreationDate="2014-05-13T23:58:30.457"
But in error it shows the String "Some" attached to it.
And my other question is why same working in below code:
val date = data.filter{line => {
line.toString().trim().startsWith("<row")
}}.filter{line=>{
line.contains("PostTypeId=\"1\"")
}}.flatMap{line=>{
val xml = XML.loadString(line)
xml.attribute("ClosedDate")
}}.map{line=>{
(format1.format(format.parse(line.toString())).toString(),1)
}}.reduceByKey(_+_)
My guess is that xml.attribute("ClosedDate").toString() is actually returning a string containing Some attached to it. Have you debugged that to make sure?
Maybe you shouldn't use toString(), but instead, get the attribute value, by using the proper method.
Or you can do it the "ugly" way and include "Some" in the pattern:
val format = new SimpleDateFormat("'Some('yyyy-MM-dd'T'hh:mm:ss.SSS')'")
Your second approach works because (and that's a guess because I don't code in Scala), probably the xml.attribute("ClosedDate") method returns an object, and calling toString() on this object returns the string with "Some" attached to it (why? ask the API authors). But when you use map on this object, it sets the line variable to the correct value (without the "Some" part).
So I was wondering how I might use scalaz-stream to generate the digest of a file using java.security.MessageDigest?
I would like to do this using a constant memory buffer size (for example 4KB). I think I understand how to start with reading the file, but I am struggling to understand how to:
1) call digest.update(buf) for each 4KB which effectively is a side-effect on the Java MessageDigest instance, which I guess should happen inside the scalaz-stream framework.
2) finally call digest.digest() to receive back the calculated digest from within the scalaz-stream framework some how?
I think I understand kinda how to start:
import scalaz.stream._
import java.security.MessageDigest
val f = "/a/b/myfile.bin"
val bufSize = 4096
val digest = MessageDigest.getInstance("SHA-256")
Process.constant(bufSize).toSource
.through(io.fileChunkR(f, bufSize))
But then I am stuck!
Any hints please? I guess it must also be possible to wrap the creation, update, retrieval (of actual digest calculatuon) and destruction of digest object in a scalaz-stream Sink or something, and then call .to() passing in that Sink? Sorry if I am using the wrong terminology, I am completely new to using scalaz-stream. I have been through a few of the examples but am still struggling.
Since version 0.4 scalaz-stream contains processes to calculate digests. They are available in the hash module and use java.security.MessageDigest under the hood. Here is a minimal example how you could use them:
import scalaz.concurrent.Task
import scalaz.stream._
object Sha1Sum extends App {
val fileName = "testdata/celsius.txt"
val bufferSize = 4096
val sha1sum: Task[Option[String]] =
Process.constant(bufferSize)
.toSource
.through(io.fileChunkR(fileName, bufferSize))
.pipe(hash.sha1)
.map(sum => s"${sum.toHex} $fileName")
.runLast
sha1sum.run.foreach(println)
}
The update() and digest() calls are all contained inside the hash.sha1 Process1.
So I have something working, but it could probably be improved:
import java.io._
import java.security.MessageDigest
import resource._
import scodec.bits.ByteVector
import scalaz._, Scalaz._
import scalaz.concurrent.Task
import scalaz.stream._
import scalaz.stream.io._
val f = "/a/b/myfile.bin"
val bufSize = 4096
val md = MessageDigest.getInstance("SHA-256")
def _digestResource(md: => MessageDigest): Sink[Task,ByteVector] =
resource(Task.delay(md))(md => Task.delay(()))(
md => Task.now((bytes: ByteVector) => Task.delay(md.update(bytes.toArray))))
Process.constant(4096).toSource
.through(fileChunkR(f.getAbsolutePath, 4096))
.to(_digestResource(md))
.run
.run
md.digest()
However, it seems to me that there should be a cleaner way to do this, by moving the creation of the MessageDigest inside the scalaz-stream stuff and have the final .run yield the md.digest().
Better answers welcome...
When I try to run (from Play Framework):
import play.api.db.DB
import anorm._
import org.joda.time.DateTime
import AnormExtensions._ // http://stackoverflow.com/a/11975107/11236
import play.api.Play.current
import java.util.Date
var stream = SQL("SELECT amiId, created, version FROM Amis WHERE created = {maxCreated}")
.on("maxCreated" -> new Date(maxCreated.getMillis))
.apply()
val map: Stream[Ami] = stream.map { ami =>
val s: String = ami[String]("amiId")
val date: Date = ami[Date]("created")
// The following line throws a compilation error
var version: Integer = ami[Integer]("version")
new Ami(s, new DateTime(date), version)
}
I get a compilation error : could not find implicit value for parameter c: anorm.Column[Integer]
What's the problem here? If I can read a Date, why can't I read an Integer?
The problem was that I used java java.lang.Integer instead of scala.Int.
Such missing numeric conversion was fixed with release of Play 2.3. You may want upgrade to.
I am trying to downloading image file for Latex formula. Following is the code I am using
var out: OutputStream = null;
var in: InputStream = null;
try {
val url = new URL("http://latex.codecogs.com/png.download?$$I=\frac{dQ}{dt}$$")
val connection = url.openConnection().asInstanceOf[HttpURLConnection]
connection.setRequestMethod("GET")
in = connection.getInputStream
val localfile = "sample2.png"
out = new BufferedOutputStream(new FileOutputStream(localfile))
val byteArray = Stream.continually(in.read).takeWhile(-1 !=).map(_.toByte).toArray
out.write(byteArray)
} catch {
case e: Exception => println(e.printStackTrace())
} finally {
out.close
in.close
}
I am able to download but it is not downloading complete image, expected image size is around 517 bytes but it is downloading only 275 bytes. What might be going wrong in it. Attached the incomplete and complete images. Please help me. I have used same code to download files more than 1MB size it worked properly.
You're passing a bad string, the "\f" is interpreted as an escape sequence and gives you a single "form feed" character.
Better:
val url = new URL("http://latex.codecogs.com/png.download?$$I=\\frac{dQ}{dt}$$")
or
val url = new URL("""http://latex.codecogs.com/png.download?$$I=\frac{dQ}{dt}$$""")
An alternative option is to use the system commands which is much cleaner
import sys.process._
import java.net.URL
import java.io.File
new URL("""http://latex.codecogs.com/png.download?$$I=\frac{dQ}{dt}$$""") #> new File("sample2.png") !!
An example using standard Java API and resource releasing with Using.
import java.nio.file.Files
import java.nio.file.Paths
import java.net.URL
import scala.util.Using
#main def main() =
val url = URL("http://webcode.me/favicon.ico")
Using(url.openStream) { in =>
Files.copy(in, Paths.get("favicon.ico"))
}