I am getting a java.util.NoSuchElementException: None.get on the following code:
// Deserialize a directory bundle
val bundle = (for(bundleFile <- managed(BundleFile(bundle_path))) yield {
bundleFile.loadMleapBundle().get
}).opt.get
The error is on the opt.get line
An example from one of MLeap's tests deserializes a bundle this way:
val bundle = (for(bundle <- managed(BundleFile(new File(lrUri.getPath)))) yield {
bundle.loadMleapBundle().get
}).tried.get
Perhaps you should use tried instead of opt.
Related
I am trying to use the reactivemongo driver in my play application without using the play module for reactive mongo.
So when I try and get the parsedURI from my config, I am getting the below error:
import reactivemongo.api.MongoConnection.ParsedURI
import reactivemongo.api.AsyncDriver
import com.typesafe.config.Config
val driver = new AsyncDriver(Some(config.get[Config]("mongodb")))
val parsedUri = config.get[ParsedURI]("mongodb.uri")
Error message:
could not find implicit value for parameter loader:
play.api.ConfigLoader[reactivemongo.api.MongoConnection.ParsedURI]
[error] val parsedUri = config.getParsedURI
[error] ^ [error] one error
found
My application.conf has:
mongodb {
uri = "mongodb://127.0.0.1:27017/mydb"
mongo-async-driver = ${akka}
}
ConfigLoader is a Play type class (like Reads) which tells play how to read a type from the config file.
You can find an explanation here: https://www.playframework.com/documentation/2.8.x/ScalaConfig#ConfigLoader
Generally you would define this by doing something like:
// Config
{
config {
url = "https://example.com"
}
}
// Config class
case class AConfig(url: String)
// Config Loader
implicit val configLoader: ConfigLoader[AConfig] = ConfigLoader {root => key =>
val config = root.getConfig(key)
AConfig(config.get[String]("url"))
}
// Usage
val aConfig = config.get[AConfig]("config")
In this case I would not suggest attempting to make one for ParsedURI because it is quite a complex type. Instead I would suggest doing something like:
val parsedUri: Try[ParsedURI] = MongoConnection.parseURI(config.get[String]("mongodb.uri"))
I'm trying to read text file located in resources directory using Scala version 2.12.3.
However I'm getting file not found error.
my project in eclipse
my scala code:
package main.scala
import scala.io.Source
import scala.io.Codec
object Application {
def main(args: Array[String]) {
try {
val source = Source.fromFile("sample.txt")(Codec.UTF8)
for (line <- source.getLines) {
println(line.toUpperCase)
}
source.close
} catch {
case e: Throwable => e.printStackTrace()
}
}
}
I also tried using
val source = Source.fromFile("sample.txt")(Codec.UTF8)
but got the same error.
If you want to read file from src/main/resources directory you should use Source.fromResource method, so try this:
Source.fromResource("sample.txt")(Codec.UTF8)
Update
In your case you have to use either Source.fromFile("src/main/resources/sample.txt") or
Source.fromFile("sample.txt") if you put your file in root project directory
I have a spark/scala project named as Omega
I have a conf file inside Omega/conf/omega.config
I use API's from typesafe to load the config file from conf/omega.config.
It was working fine and I was able to read the respective value for each key
Now today, For the first time I added some more key-value pairs in my omega.config file and tried to retrieve them from my scala code. It throws
Exception in thread "main" com.typesafe.config.ConfigException$Missing: No configuration setting found for key 'job_name'
This issue started happening after adding new value for the key job_name in my omega.config file.
Also I am not able to read the newly added -key-values, I am still able to read all old values using config. getString method
I am building my spark/scala application using maven.
Omega.config
input_path="/user/cloudera/data
user_name="surender"
job_name="SAMPLE"
I am Not able to access the recently added key "job_name" alone
package com.pack1
import com.pack2.ApplicationUtil
object OmegaMain {
val config_loc = "conf/omega.config"
def main(args: Array[String]): Unit = {
val config = ApplicationUtil.loadConfig(config_loc)
val jobName = ApplicationUtil.getFromConfig(config,"job_name")
}
}
package com.pack2
import com.typesafe.config.{Config, ConfigFactory}
object ApplicationUtil {
def loadConfig(filePath:String):Config={
val config = ConfigFactory.parseFile(new File(filePath))
config
}
def getFromConfig(config:Config,jobName:String):String={
config.getString(jobName)
}
}
Could some one help me what went wrong?
You can try something like:
def loadConfig(filename: String, syntax: ConfigSyntax): Config = {
val in: InputStream = getClass.getResourceAsStream(filename)
if (in == null) return null
val file: File = File.createTempFile(String.valueOf(in.hashCode()), ".conf")
file.deleteOnExit()
val out: FileOutputStream = new FileOutputStream(file)
val buffer: Array[Byte] = new Array(1024)
var bytesRead: Int = in.read(buffer)
while (bytesRead != -1) { out.write(buffer, 0, bytesRead); bytesRead = in.read(buffer) }
out.close()
val conf: Config = ConfigFactory.parseFile(file, ConfigParseOptions.defaults().setSyntax(syntax).setAllowMissing(false).setOriginDescription("Merged with " + filename))
conf
}
filename is some file path in the classpath. If you want to update this method to taking some external file into account, change update the 4th with val file: File = new File("absolute Path of he file")
I am guessing the file isn't on the classpath after you build with Maven.
Since you are using Maven to build a jar, you need your omega.config to be in the classpath. This means that you either have to put it into src/main/resources by default or explicitly tell Maven to add conf to the default resources classpath.
I'm following the Scalazon example at here to create a Kinesis stream. The following piece of code:
val streamListFuture = for {
s <- Kinesis.streams.list
} yield s
gives the following error:
[error] KinesisStatsWriter.scala:51: value map is not a member of object io.github.cloudify.scala.aws.kinesis.Requests.ListStreams
[error] s <- Kinesis.streams.list
If I don't use a for comprehension and call val createStream = Kinesis.streams.list, there's no error. Can't seem to figure out why.
Similarly, the following bit of code:
val createStream = for {
s <- Kinesis.streams.create(name)
} yield s
produces a similar error:
[error] KinesisStatsWriter.scala:64: value map is not a member of io.github.cloudify.scala.aws.kinesis.Requests.CreateStream
[error] s <- Kinesis.streams.create(name)
Appreciate the help!
Author here, the for-comprehension works only if you include the module that implicitly converts requests to Futures (it's called ImplicitExecution). Try adding the following import statement (looks at the sample code in the library README).
import io.github.cloudify.scala.aws.kinesis.Client.ImplicitExecution._
I am having issues running Apache Spark 1.0.1 within a Play! app. Currently, I am trying to run Spark within the Play! application and use some of the basic Machine Learning within Spark.
Here's my app creation:
def sparkFactory: SparkContext = {
val logFile = "public/README.md" // Should be some file on your system
val driverHost = "localhost"
val conf = new SparkConf(false) // skip loading external settings
.setMaster("local[4]") // run locally with enough threads
.setAppName("firstSparkApp")
.set("spark.logConf", "true")
.set("spark.driver.host", s"$driverHost")
new SparkContext(conf)
}
And here's an error when I try to do some basic discovery of a Tall and Skinny Matrix:
[error] o.a.s.e.ExecutorUncaughtExceptionHandler - Uncaught exception in thread Thread[Executor task launch worker-3,5,main]
java.lang.NoSuchMethodError: breeze.linalg.DenseVector$.dv_v_ZeroIdempotent_InPlaceOp_Double_OpAdd()Lbreeze/linalg/operators/BinaryUpdateRegistry;
at org.apache.spark.mllib.linalg.distributed.RowMatrix$$anonfun$5.apply(RowMatrix.scala:313) ~[spark-mllib_2.10-1.0.1.jar:1.0.1]
at org.apache.spark.mllib.linalg.distributed.RowMatrix$$anonfun$5.apply(RowMatrix.scala:313) ~[spark-mllib_2.10-1.0.1.jar:1.0.1]
at scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:144) ~[scala-library-2.10.4.jar:na]
at scala.collection.TraversableOnce$$anonfun$foldLeft$1.apply(TraversableOnce.scala:144) ~[scala-library-2.10.4.jar:na]
at scala.collection.Iterator$class.foreach(Iterator.scala:727) ~[scala-library-2.10.4.jar:na]
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157) ~[scala-library-2.10.4.jar:na]
The error above is triggered by the following:
def computePrincipalComponents(datasetId: String) = Action {
val datapoints = DataPoint.listByDataset(datasetId)
// load the data into spark
val rows = datapoints.map(_.data).map { row =>
row.map(_.toDouble)
}
val RDDRows = WorkingSpark.context.makeRDD(rows).map { line =>
Vectors.dense(line)
}
val mat = new RowMatrix(RDDRows)
val result = mat.computePrincipalComponents(mat.numCols().toInt)
Ok(result.toString)
}
It looks like a dependency issue, but no idea where it starts. Any ideas?
Ah this was indeed caused by a dependency conflict. Apparently the new Spark uses new Breeze methods that were not available in a version I had pulled in. By removing Breeze from my Play! Build file I was able to run the function above just fine.
For those interested, here's the output:
-0.23490049167080018 0.4371989078912155 0.5344916752692394 ... (6 total)
-0.43624389448418854 0.531880914138611 0.1854269324452522 ...
-0.5312372137092107 0.17954211389001487 -0.456583286485726 ...
-0.5172743086226219 -0.2726152326516076 -0.36740474569706394 ...
-0.3996400343756039 -0.5147253632175663 0.303449047782936 ...
-0.21216780828347453 -0.39301803119012546 0.4943679121187219 ...