I want to execute the sh file from Scala application.
Let's say I have createPassword.sh file and I need to invoke this sh file from Scala application and get the output back.
How can I achieve through scala application?
This should do the trick if the script is in the current working directory (otherwise specify the full path of the script)
import sys.process._
val result = "./createPassword.sh" !!
result is then a String containing the standard output (and standard error)
EDIT: If you want to use ProcessBuillder from Java SE7, you can also use this in scala:
import java.io.{BufferedReader, InputStreamReader}
val p = new ProcessBuilder("/bin/bash","createPassword.sh")
val p2 = p.start()
val br = new BufferedReader(new InputStreamReader(p2.getInputStream()))
var line:String = ""
while ({line = br.readLine(); line!= null}) {
println(line)
}
Given your dir has a script,
`val path = "./src/test/tests/Integration/"`
`val output = Process("sh test.sh", new File("path")).!!`
Related
I'm trying to list files within a directory that match a regular expression, e.g. ".csv$" this is very similar to Scala & DataBricks: Getting a list of Files
I've been running in circles for hours trying to figure out how Scala can list a directory of files and filter by regex.
import java.io.File
def getListOfFiles(dir: String):List[File] = {
val d = new File(dir)
if (d.exists && d.isDirectory) {
d.listFiles.filter(_.isFile).toList
} else {
List[File]()
}
}
val name : String = ".csv"
val files = getListOfFiles("/home/con/Scripts").map(_.path).filter(_.matches(name))
println(files)
gives the error
/home/con/Scripts/scala/find_files.scala:13: error: value path is not a member of java.io.File
val files = getListOfFiles("/home/con/Scripts").map(_.path).filter(_.matches(name))
I'm trying to figure out the regular Scala equivalent of dbutils.fs.ls which eludes me.
How can list files in a regular directory in Scala?
The error is reporting that path is not defined in java.io.File which it isn't.
If you want to match by name, why don't you get file names? Also, your regex is a bit off if you want to match based on file extension.
Fixing these two problems:
val name : String = ".+\\.csv"
val files = getListOfFiles("/path/to/files/location")
.map(f => f.getName)
.filter(_.matches(name))
will output .csv files in the /path/to/files/location folder.
I need get all user input in stdin as single string in scala 2.12 (supposed the data would copy-pasted by single action), something like this:
please copy data:
word1
word2
word3
And I need get string with following data:
val str = "word1\nword2\nword3"
my current approach is not working, just hanging forever:
import scala.collection.JavaConverters._
val scanner: Iterator[String] = new Scanner(System.in).asScala
val sb = new StringBuilder
while (scanner.hasNext) {
sb.append(scanner.next())
}
val str = sb.toString()
Although this can print the input:
import scala.collection.JavaConverters._
val scanner: Iterator[String] = new Scanner(System.in).asScala
scanner foreach println
I'm looking for idiomatic way of doing the job
Try
LazyList
.continually(StdIn.readLine())
.takeWhile(_ != null)
.mkString("\n")
as inspired by https://stackoverflow.com/a/18924749/5205022
On my machine I could terminate the input with ^D.
In Scala 2.12 replace LazyList with Stream.
I am really messing with syntax here need help...
I have a URL, on clicking of which a sample.csv.gz file gets downloaded
Please can someone help me fill the syntactic gaps below:
val outputFile = "C:\\sampleNew" + ".csv"
val inputFile = "C:\\sample.csv.gz"
val fileUrl = "someSamplehttpUrl"
// On hitting this Url, sample.csv.gz file should download at destination 'outputFile'
val in = new URL()(fileUrl).openStream()
Files.copy(in, Paths.get(outputFile), StandardCopyOption.REPLACE_EXISTING)
val filePath = new File(outputFile)
if(filePath.exists()) filePath.delete()
val fw = new FileWriter(outputFile, true)
var bf = new BufferedReader(new InputStreamReader(new GZIPInputStream(new FileInputStream(inputFile)), "UTF-8"))
while (bf.ready()) fw.append(bf.readLine() + "\n")
I have been getting several errors with syntax... Any corrections here? I basically have an http get request that returns a URL, which I must open to download this gz file
Thanks!
Here are two possible solutions:
import java.io.{File, PrintWriter}
import scala.io.Source
val outputFile = "out.csv"
val inputFile = "/tmp/marks.csv"
val fileUrl = s"file:///$inputFile"
// Method 1, a traditional copy from the input to the output.
val in = Source.fromURL(fileUrl)
val out = new PrintWriter(outputFile)
for (line <- in.getLines)
out.println(line)
out.close
in.close
Here is a one liner which basically pipes the data from the input to the output.
import sys.process._
import java.net.URL
val outputFile = "out.csv"
val inputFile = "/tmp/marks.csv"
val fileUrl = s"file:///$inputFile"
// Method 2, pipe the content of the URL to the output file.
new URL(fileUrl) #> new File(outputFile) !!
Here is a version using Files.copy
val outputFile = "out.csv"
val inputFile = "/tmp/marks.csv"
val fileUrl = s"file:///$inputFile"
import java.nio.file.{Files, Paths, StandardCopyOption}
import java.net.URL
val in = new URL(fileUrl).openStream
val out = Paths.get(outputFile)
Files.copy(in, out, StandardCopyOption.REPLACE_EXISTING)
Hopefully one (or more) of the above will address your needs.
I'm trying to get basic file attributes using Scala, and my reference is this Java question:
Determine file creation date in Java
and this piece of code I'm trying to rewrite in Scala:
static void getAttributes(String pathStr) throws IOException {
Path p = Paths.get(pathStr);
BasicFileAttributes view
= Files.getFileAttributeView(p, BasicFileAttributeView.class)
.readAttributes();
System.out.println(view.creationTime()+" is the same as "+view.lastModifiedTime());
}
The thing I just can't figure out is this line of code..I don't understand how to pass a class in this way using scala... or why Java is insisting upon this in the first place instead of using an actual constructed object as the parameter. Can someone please help me write this line of code to function properly? I must be using the wrong syntax
val attr = Files.readAttributes(f,Class[BasicFileAttributeView])
Try this:
def attrs(pathStr:String) =
Files.getFileAttributeView(
Paths.get(pathStr),
classOf[BasicFileAttributes] //corrected
).readAttributes
Get file creation date in Scala, from Basic Files Attributes:
// option 1,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributes
val pathStr = "/tmp/test.sql"
Files.readAttributes(Paths.get(pathStr), classOf[BasicFileAttributes]).creationTime
res3: java.nio.file.attribute.FileTime = 2018-03-06T00:25:52Z
// option 2,
import java.nio.file.{Files, Paths}
import java.nio.file.attribute.BasicFileAttributeView
val pathStr = "/tmp/test.sql"
{
Files
.getFileAttributeView(Paths.get(pathStr), classOf[BasicFileAttributeView])
.readAttributes.creationTime
}
res20: java.nio.file.attribute.FileTime = 2018-03-07T19:00:19Z
I am trying to append to a file such that I first want to delete the last line and then start appending. But, I can't figure how to delete the last line of the file.
I am appending the file as follows:
val fw = new FileWriter("src/file.txt", true) ;
fw.write("new item");
Can anybody please help me?
EDIT:
val lines_list = Source.fromFile("src/file.txt").getLines().toList
val new_lines = lines_list.dropRight(1)
val pw = new PrintWriter(new File("src/file.txt" ))
(t).foreach(pw.write) pw.write("\n")
pw.close()
After following your method, I am trying to write back to the file, but when I do this, all the contents, with the last line deleted come in a single line, however I want them to come in separate lines.
For very large files a simple solution relies in OS related tools, for instance sed (stream editor), and so consider a call like this,
import sys.process._
Seq("sed","-i","$ d","src/file1.txt")!
which will remove the last line of the text file. This approach is not so Scalish yet it solves the problem without leaving Scala.
Return random access file in position without last line.
import java.io.{RandomAccessFile, File}
def randomAccess(file: File) = {
val random = new RandomAccessFile(file, "rw")
val result = findLastLine(random, 0, 0)
random.seek(result)
random
}
def findLastLine(random: RandomAccessFile, position: Long, previous: Long): Long = {
val pointer = random.getFilePointer
if (random.readLine == null) {
previous
} else {
findLastLine(random, previous, pointer)
}
}
val file = new File("build.sbt")
val random = randomAccess(file)
And test:
val line = random.readLine()
logger.debug(s"$line")
My scala is way off, so people can probably give you a nicer solution:
import scala.io.Source
import java.io._
object Test00 {
def main(args: Array[String]) = {
val lines = Source.fromFile("src/file.txt").getLines().toList.dropRight(1)
val pw = new PrintWriter(new File("src/out.txt" ))
(lines :+ "another line").foreach(pw.println)
pw.close()
}
}
Sorry for the hardcoded appending, i used it just to test that everything worked fine.