I have an akka project, in which I am building a Properties object using the code:
import java.io.FileNotFoundException
import java.util.Properties
import scala.io.Source
val configPath = "hikari.properties"
val url = getClass.getResource(hikariConfigPath)
val properties: Properties = new Properties()
if (url != null) {
val source = Source.fromURL(url)
properties.load(source.bufferedReader())
} else {
println("properties file cannot be loaded at path " + configPath)
throw new FileNotFoundException("Properties file cannot be loaded”)
}
And this is giving me “properties file cannot be loaded at path hikari.properties”
with the exception:
java.io.FileNotFoundException: Properties file cannot be loaded
at model.daos.JavaConnectionFactoryBuilder$.buildDataSource(JavaConnectionFactory.scala:25)
I have put the hikari.properties file in the src/main/resources directory as advised by https://zetcode.com/articles/hikaricp/
How can I solve this issue?
You are loading a non existent file hikari. instead of hikari.properties.
It should be:
val hikariConfigPath = "hikari.propertiez"
Related
I tried to read a properties file in spark where my file location is available while run the job getting below error
code is
object runEmpJob {
def main(args: Array[String]): Unit = {
println("starting emp job")
val props = ConfigFactory.load()
val envProps = props.getConfig("C:\\Users\\mmishra092815\\IdeaProjects\\use_case_1\\src\\main\\Resource\\filepath.properties")
System.setProperty("hadoop.home.directory", "D:\\SHARED\\winutils-master\\hadoop-2.6.3\\bin")
val spark = SparkSession.builder().
appName("emp dept operation").
master(envProps.getString("Dev.executionMode")).
getOrCreate()
val empObj = new EmpOperation
empObj.runEmpOperation(spark, "String", fileType = "csv")
val inPutPath = args(1)
val outPutPath = args(2)
}
}
getting error:
Exception in thread "main"
com.typesafe.config.ConfigException$BadPath: path parameter: Invalid path C:\Users\mmishra092815\IdeaProjects\use_case_1\src\main\Resource\filepath.properties':
Token not allowed in path expression: ':' (you can double-quote this token if you really want it here)
at com.typesafe.config.impl.PathParser.parsePathExpression(PathParser.java:155)
at com.typesafe.config.impl.PathParser.parsePathExpression(PathParser.java:74)
at com.typesafe.config.impl.PathParser.parsePath(PathParser.java:61)
at com.typesafe.config.impl.Path.newPath(Path.java:230)
at com.typesafe.config.impl.SimpleConfig.find(SimpleConfig.java:192)
at com.typesafe.config.impl.SimpleConfig.getObject(SimpleConfig.java:268)
at com.typesafe.config.impl.SimpleConfig.getConfig(SimpleConfig.java:274)
at com.typesafe.config.impl.SimpleConfig.getConfig(SimpleConfig.java:41)
at executor.runEmpJob$.main(runEmpJob.scala:12)
at executor.runEmpJob.main(runEmpJob.scala)
Process finished with exit code 1
Loading happens in ConfigFactory.load(). If you want to load configuration from specific file, pass it like:
val props = ConfigFactory.load("C:\\Users\\mmishra092815\\IdeaProjects\\use_case_1\\src\\main\\Resource\\filepath.properties")
As described in API documentation, getConfig method does not load configuration from file - it returns a Config object for given config path (not filesystem path!)
I have one app.conf file like below:
some_variable_a = some_infohere_a
some_variable_b = some_infohere_b
Now I need to write scala function to load this app.conf file and create a scala case class to store all these properties. with try-catch and file checking conditions and corner cases.
I am very new to scala do not have much knowledge on this please provide me a correct way to do this.
Whatever I tried I am writing below:
import java.io.File
import com.typesafe.config.{ Config, ConfigFactory }
import com.typesafe.config._
import java.nio.file.Paths
private def ReadConfFile(path: String) = {
val fileTemp = new File(path)
if (fileTemp.exists) {
val confFile = Paths.get(Path).toFile
val config = ConfigFactory.parseFile(confFile)
val some_variable_a = config.getString("some_variable_a")
val some_variable_b = config.getString("some_variable_b")
}
}
Assuming that app.conf is in root folder of your application, this should be enough to access those variables from config file:
val config = ConfigFactory.load("app.conf")
val some_variable_a = config.getString("some_variable_a")
val some_variable_b = config.getString("some_variable_b")
In case you need to load it from the file using absolute path, you can do that in this way:
val myConfigFile = new File("/home/user/location/app.conf")
val config = ConfigFactory.parseFile(myConfigFile)
val some_variable_a = config.getString("some_variable_a")
val some_variable_b = config.getString("some_variable_b")
Or something similar as you did. In your code there is a typo I guess in Paths.get(Path).toFile. "Path" should be "path". If you If you don't have some variable Path, you should get compiling error for that. If that is not the problem, then check if you are providing correct path:
private def ReadConfFile(path: String) = {
val fileTemp = new File(path)
if (fileTemp.exists) {
val confFile = Paths.get(path).toFile
val config = ConfigFactory.parseFile(confFile)
val some_variable_a = config.getString("some_variable_a")
val some_variable_b = config.getString("some_variable_b")
}
}
ReadConfFile("/home/user/location/app.conf")
I'm trying to read text file located in resources directory using Scala version 2.12.3.
However I'm getting file not found error.
my project in eclipse
my scala code:
package main.scala
import scala.io.Source
import scala.io.Codec
object Application {
def main(args: Array[String]) {
try {
val source = Source.fromFile("sample.txt")(Codec.UTF8)
for (line <- source.getLines) {
println(line.toUpperCase)
}
source.close
} catch {
case e: Throwable => e.printStackTrace()
}
}
}
I also tried using
val source = Source.fromFile("sample.txt")(Codec.UTF8)
but got the same error.
If you want to read file from src/main/resources directory you should use Source.fromResource method, so try this:
Source.fromResource("sample.txt")(Codec.UTF8)
Update
In your case you have to use either Source.fromFile("src/main/resources/sample.txt") or
Source.fromFile("sample.txt") if you put your file in root project directory
I have a spark/scala project named as Omega
I have a conf file inside Omega/conf/omega.config
I use API's from typesafe to load the config file from conf/omega.config.
It was working fine and I was able to read the respective value for each key
Now today, For the first time I added some more key-value pairs in my omega.config file and tried to retrieve them from my scala code. It throws
Exception in thread "main" com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'job_name'
This issue started happening after adding new value for the key job_name in my omega.config file.
Also I am not able to read the newly added -key-values, I am still able to read all old values using config. getString method
I am building my spark/scala application using maven.
Omega.config
input_path="/user/cloudera/data
user_name="surender"
job_name="SAMPLE"
I am Not able to access the recently added key "job_name" alone
package com.pack1
import com.pack2.ApplicationUtil
object OmegaMain {
val config_loc = "conf/omega.config"
def main(args: Array[String]): Unit = {
val config = ApplicationUtil.loadConfig(config_loc)
val jobName = ApplicationUtil.getFromConfig(config,"job_name")
}
}
package com.pack2
import com.typesafe.config.{Config, ConfigFactory}
object ApplicationUtil {
def loadConfig(filePath:String):Config={
val config = ConfigFactory.parseFile(new File(filePath))
config
}
def getFromConfig(config:Config,jobName:String):String={
config.getString(jobName)
}
}
Could some one help me what went wrong?
You can try something like:
def loadConfig(filename: String, syntax: ConfigSyntax): Config = {
val in: InputStream = getClass.getResourceAsStream(filename)
if (in == null) return null
val file: File = File.createTempFile(String.valueOf(in.hashCode()), ".conf")
file.deleteOnExit()
val out: FileOutputStream = new FileOutputStream(file)
val buffer: Array[Byte] = new Array(1024)
var bytesRead: Int = in.read(buffer)
while (bytesRead != -1) { out.write(buffer, 0, bytesRead); bytesRead = in.read(buffer) }
out.close()
val conf: Config = ConfigFactory.parseFile(file, ConfigParseOptions.defaults().setSyntax(syntax).setAllowMissing(false).setOriginDescription("Merged with " + filename))
conf
}
filename is some file path in the classpath. If you want to update this method to taking some external file into account, change update the 4th with val file: File = new File("absolute Path of he file")
I am guessing the file isn't on the classpath after you build with Maven.
Since you are using Maven to build a jar, you need your omega.config to be in the classpath. This means that you either have to put it into src/main/resources by default or explicitly tell Maven to add conf to the default resources classpath.
In PlayFramework 2.2.x after using play dist I am having issues reading and writing to /public directory. Is that a known problem? Is the only solution to read/write to another directory with a global path?
This is my sample code:
val imageDirectory = "images/twitpics/"
val localPrefix = "/public/"
val publicPrefix = "/assets/"
val files = Play.getFile(localPrefix + imageDirectory)
.listFiles.filter(_.getName.takeRight(3) == "jpg")
val randomIndex = _rand.nextInt(files.length)
val imageFile = files(randomIndex)
Also
private val _jsonConfigFile = "/public/data/data.json"
def writeJsonToFile(content: String) = {
import java.io._
val pw = new PrintWriter(Play.getFile(_jsonConfigFile))
pw.write(content)
pw.close
}
After dist the public/ directory is packaged into the application jar, which is put on the classpath, so you cannot access it through the filesystem or write to it.