Wait for a "Process" in Scala to complete - scala

Question

So there is this function in my program, that downloads a quite heavy zip file (about 500 megabytes) and then extracts the file and removes the zip file itself.
And obviously I want to wait for the file to download completely, then wait to unzip it completely and then remove the zip file itself (I just need the Json files inside). This is the code that I use currently:
import java.io.File
import java.net.URL
import scala.sys.process._
/* other functions */
// downloading, unzipping and removing are in separate functinos, but I
// aggregated them all here for simplicity
def downloadZipThenExtract(link: String, filePath: String): Future[Int] = {
val urlObject = new URL(link)
val file = new File(filePath)
Future {
val download: ProcessBuilder = urlObject #> file
val unzip: ProcessBuilder = s"unzip ${file.getPath} -d ${file.getParent}"
val delete: ProcessBuilder = s"rm ${file.getPath}"
/*
I've already tried this:
(download ### unzip ### delete) !
And every other solution, none of them worked
*/ // =>
download !
Thread.sleep(900000) // wait 15 minutes to download
unzip !
Thread.sleep(60000) // One minute to unzip
delete !
}
}
And as you can see, I found no other approach than freezing the thread to complete the download and unzipping, which of course sucks. So I wanted to know if you guys know any better approach, thanks.

Answer 1

I am not sure why you think that "freezing the thread" sucks. If you have to wait for the process to finish, then you have to wait for it.
But if it is just Thread.sleep that you see as problematic, then I have good news for you: you don't need it :)
download !
unzip !
delete !
I know you said you already tried "every other solution, and nothing worked", but if you have indeed tried this one, you gotta elaborate, because then I don't know what you mean when you say it "doesn't work".

Answer 2

I almost forgot about this question of mine, which I'd like to answer it in case anybody else faces this problem, since someone just upvoted the question.
My solution was to use akka, then put three streams inside my code, where obviously, first one downloads the data and puts it in the desired zip file, the second one extracts the zip file using java nio, and the last one just deletes the zip file.
Note that there's some extra code which you might not need, like the println's in case of exceptions and other stuff, I just wanted to share the idea and the solution.
here is my code:
def downloadFilesInto(fullPath: String, link: String)(implicit materializer: Materializer): Future[Int] = {
val directory: File = new File(fullPath).getParentFile
val urlObj = new URL(link)
val fileOutputStream = new FileOutputStream(fullPath)
val readableChannel = Channels.newChannel(urlObj.openStream())
val source = Source.single(1)
val downloaderFlow: Flow[Int, Long, NotUsed] = Flow[Int].map { _ =>
val size: Long = fileOutputStream.getChannel.transferFrom(readableChannel, 0, Long.MaxValue)
fileOutputStream.close()
readableChannel.close()
size
}
val unzipFlow: Flow[Long, Int, NotUsed] = Flow[Long].map { size =>
println(s"file downloaded, properties:\n type: zip\n size: ${size.toString.take(3)} MB")
val zipFile = new ZipFile(new File(fullPath))
val entries = zipFile.entries().asScala.toList
val eachFileExtractionResult: List[Int] =
entries.map { entry =>
try {
val path = new File(directory + "/" + entry.getName).toPath
Files.copy(zipFile.getInputStream(entry), path)
0
} catch {
case ex: Exception =>
println(
s"""
| Caught an Exception while unzipping file: ${entry.getName}
| Type: ${ex.getClass}
| Cause: ${ex.getCause}
| Message: ${ex.getMessage}
|""".stripMargin
)
1
case th: Throwable =>
println(
s"""
| Caught a throwable while unzipping file: ${entry.getName}
| Type: ${th.getClass}
| Cause: ${th.getCause}
| Message: ${th.getMessage}
|""".stripMargin
)
2
}
}
val unsuccessfulTries = eachFileExtractionResult.filterNot(_ == 0)
if (unsuccessfulTries.isEmpty)
0
else unsuccessfulTries.head
}
val removeFlow: Flow[Int, Int, NotUsed] = Flow[Int].map {
case 0 => // successfully unzipped
s"rm $fullPath".!
case whatever => whatever
}
val sink = Sink.fold[Int, Int](0)(_ + _)
source.viaMat(downloaderFlow)(Keep.right)
.viaMat(unzipFlow)(Keep.right)
.viaMat(removeFlow)(Keep.right)
.toMat(sink)(Keep.right)
.run()
}