Reading files from directory in Play framework on Heroku - scala

I have a directory with deep structure (lot of sub-directories and files) that I read from my play application. On my PC I read the directory using -
val directory = Play.getFile("directory")
for(file <- directory.listFiles) {
val lines = Source.fromFile(file).getLines()
}
This works perfectly on my PC but not on Heroku. On Heroku I get a NPE on line#2 (above code) which means that the directory object is not getting made.
This suggestion of a similar issue suggests that I could put my directory in public and read it as using the Play.resource API. But I DONT want to put my directory in public. And I have a need to list the contents of a directory as it could be changing... how can I do this in Play on Heroku?

As this suggests,in order to make Play.getFile("location_from_project_root") (or any other similar one) working in production environment, you have to explicitly ask from play to add the directory where you put the file ,to the dist.
Ex:
You want to access a file Project_Root/resources/mtFile.json, (inside /resources folder you created), you can use Play.getFile("/resources/mtFile.json")method.
But in order to make it works in any production environment, you have to add /resources folder to the dist. Otherwise that file will not be there. In order to do that you have to add these lines into your build.sbt file.
import com.typesafe.sbt.packager.MappingsHelper._
mappings in Universal ++= directory(baseDirectory.value / "resources")
Now, if you take a dist using activator dist command you can see that a directory called /resources has added into the root of the dist. Now Play.getFile("/resources/mtFile.json") should work in production environment.

Play.getFile(relativePath) retrieves a file relative to the current app's root path. Current app's root path may not be same on your PC and Heroku.
One more thing: Your statement that the directory object is not getting made is not true. listFiles operation is throwing NPE in your case.
Try this to get a list of files from your directory:
val listOfFilesInDirectory: List[java.io.File] = Option(getClass.getResource("/directory")).map(_.toURI).map(new java.io.File(_)).map(_.listFiles).flatten.toList

You can put your files under your conf folder, so that the files are not publicly available. Then use :
Play.resource("conf/directory"): Option[URL]
or
Play.resourceAsStream("conf/directory"): Option[InputStream]

Related

How to run `forest schema:update` outside project directory?

I'm trying to use the forest-cli schema:update command, but when I do, I keep getting the error:
× We are not able to detect a Forest CLI project file architecture at this path: /PATH/TO/REPO/ROOT.: Error: No "routes" directory.
There is a routes directory, but within src/ below the repo root. I have tried running forest schema:update from inside there, but I get the exact same error. The command only has options for a config file and an output directory.
Googling has turned up nothing, and there's no obvious hint from forestadmin's documents. Thanks in advance for any assistance!
According to the forest-cli code available here, the forest schema:update command requires the package.json file to be directly accessible in order to run (In the same folder you run the command), to check that the version of the agent you are running is indeed compatible with schema:update.
You can also use the -c/--config option in order to use another location of your config/database.js, and the -o/--outputDirectory to output the result to a new location.
In your case, I would say that forest schema:update -c src/config/database.config.js -o tmp should allow you to generate the files in the tmp directory (Be aware that this directory should not exist).
This command should be run where your package.json is located.
However, I don't think you will be able to export files directly at the right location when using a custom folder structure.

How to add a folder and its content to the standard paths of Playframework and Heroku?

I have a Scala Play framework 2.7.x application which I deploy in Heroku. I use Lucene to index the WebApp and since there is no JdbcDirectory in Lucene I need to use their FSDirectory instead and that leads to issues with Heroku because I can't generate the index files under $APP_HOME/lucene-index/* in Heroku otherwise it will be wiped out each time. This leads me to two possible solutions and this is the simpler one:
Generate the $APP_HOME/lucene-index locally before deployment and save it in GIT, this folder will be at the same level as $APP_HOME/app and $APP_HOME/public.
Integrate the new nonstandard Play folder $APP_HOME/lucene-index so that it gets copied by Heroku (the purpose of this OP).
Upon startup the application checks for this folder and if doesn't exist (local case) gets generated otherwise it opens it (Heroku case).
Do I need to do something special on #2 to have Heroku recognize $APP_HOME/lucene-index/ as a folder that needs to be packaged together with the application? e.g. I would not like to put the $APP_HOME/lucene-index/ under $APP_HOME/conf/ for this to work.
Here I find the Anatomy of a Play 2.7.x application but there is no word on how to add extra path folders to it.
The solution I was after was to include the ./lucene-index folder as part of the Play dist. This is accomplished by changing the build.sbt file adding:
//********************************************************
// Add lucene-index to the dist
//********************************************************
import com.typesafe.sbt.packager.MappingsHelper._
mappings in Universal ++= directory(baseDirectory.value / "lucene-index")
Now it deploys to Heroku and it all works nicely.

play framework reading file from conf folder with routing

I have a web application with play framework. All images used in the application are kept in public folder and are accessed with the help of a routing defined in the conf/route file. So all the images I used are present in a jar file after build. But my requirement is that the admin will be placing few images that the UI should be able to access. For obvious reasons I can ask them to add images into the jar.
My plan is to ask the admin to add images to a folder inside the conf folder and read it from there using routing (I believe its possible because currently there is a routing defined that's reading a json from the config file).
# Home page
GET / controllers.Application.index
GET /clientConfig controllers.Application.clientConfiguration
GET /testImg controllers.Application.testImg
# Map static resources from the /public folder to the /assets URL path
GET /assets/*file controllers.Assets.versioned(path="/public", file: Asset)
Being totally new to Play framework I can't figure out a way to define routes for that and read images from a folder inside the conf folder . Any help is highly appreciated.
conf folder is a configuration folder so it's not a good idea to expose it as a public folder. For example from security perspective the application.conf or the database migration files contain sensitive data that shouldn't be accessible on any api endpoint.
What you could do is add an additional root folder to the public managed resources for example create a folder called publicImages in the root folder and add the following line to build.sbt
unmanagedResourceDirectories in Assets += baseDirectory.value / "publicImages"
Also remember that adding images to a folder on the deployed server means you won't be able to cluster, i.e. add additional nodes if/when load grows. So your best option is to use some 3rd party storage service, e.g. amazon s3 (it's pretty simple for the admin to upload images to a folder on s3)
If however you insist on adding the images to conf folder, what you could do is create a standard controller that accepts file name in its parameters (path) and stream the content of that file from the conf folder using
Play.getFile('conf/{myfile}') just make sure to enforce some security constraints like verifying no path is provided so malicious user can't traverse the machine's file system. And support only predefined 'safe' file types like images.
As LiorH said, it's usually not a good idea to expose conf folder to public. But if this is really what you want, then you can try to implement your own controller.
In your route file:
GET /configuration ConfController.loadConf(path)
In ConfController:
def loadConf(path: String) = Action {
Ok.sendFile(Play.getFile("conf$path"))
}

Scala io.Source.fromfile not finding my file even with absolute path specified

I am trying to access a file in Scala using io.Source.fromfile.
I have specified the full path, but i am still getting a no such directory or file error.
This is a general version of what my code looks like:
val lines = io.Source.fromFile("~/top/next/source/resources/desiredFile.txt").getLines()
I'm running Ubuntu if that makes any difference.
It probably because you are using tilde sign, use full absolute path.
If you want to avoid hard coding your home directory, you can get it from environment variables:
val home = System.getProperty("user.home")
val s = Source.fromFile(s"${home}/.....").getLines()
The compiler was assuming it should start in the project folder I was already in, so when I specified the directories above that folder, it tried to find them all as a sub-directory of my root project folder.
This would obviously cause an error.
I now have:
val lines = io.Source.fromFile("source/resources/desiredFile.txt")
which is working properly

file copy in scala play

i am to copy a file in scala But getting FileNotFound error, The assets folder is in the same directory where is src:
val src = new File("/assets/public/images/default/male.jpg")
val dest = new File("/assets/public/images/profile/male1.jpg")
new FileOutputStream(dest) getChannel() transferFrom(
new FileInputStream(src) getChannel, 0, Long.MaxValue )
In your code you are trying to copy the file using FileOutputStream, which requires a valid path to the existing file else it'll throw FileNotFoundException. (see the doc here)
val src = new File("/assets/public/images/default/male.jpg")
val dest = new File("/assets/public/images/profile/male1.jpg")
new FileOutputStream(dest) //dest should exist
Nevertheless, Play has its own utility to copy files. Here is the link.
import play.api.libs.Files
Files.copyFile(src, dest, true, true)
println(dest.getAbsolutePath()) // filepath of copied file
Also, since the files get copied to the working directory, you might not be able to see the new file in the folder structure of your favorite IDE.
Aside, you may get the path for public assets by using routes
val srcPath = routes.Assets.at("public/images/default/male.jpg").url
A general advise
When copying files on Java, use the FileUtils.copy(...) from the Apache commons project.
For your specific problem
You get "File not found" if the file cannot be found by the running process. This could be because your file is indeed not there or because the process is lacking rights to see the file.
Your wording is a little ambiguous, it seems you meant to give paths relative to the current working directory. That means that this code should be executed from a directory which contains the assets directory. If this is so, then you have made a mistake and given absolute paths to your file objects, not relative ones. All you have to do is remove the initial forward slash from those paths and it should work.
As it is, you are telling Scala/Java to look in the root directory for assets.