I imported scala notebook from databricks into Intellij but for below lines
dbutils.widgets.text("sourcePath", "xyz", "Source Path:");
import org.apache.spark.sql.DataFrame;
I'm getting error as cannot resolve symbol dbutils and cannot resolve symbol apache for second line.
Could you please advise what might be issue?
Related
I am trying to read data from neo4j using spark job. The import itself is showing this error. I tried to import org.neo4j.spark._ in intellij IDEA, but it is showing an error "Cannot resolve symbol spark". While trying in the spark-shell, it is throwing an error,
":23: error: object neo4j is not a member of package org
import org.neo4j.spark._"
Spark version - 3.1.1
dependencies
I am running databricks 7.3LTS and having errors while trying to use scala bulk copy.
The error is:
object sqldb is not a member of package com.microsoft.
I have installed correct sqlconnector drivers but not sure how to fix this error.
The installed drivers are:
com.microsoft.azure:spark-mssql-connector_2.12:1.1.0.
also i have installed the JAR dependencies as below:
spark_mssql_connector_2_12_1_1_0.jar
i couldnt find any scala code example for the above configurations on the internet.
my scala code sample is as below:
%scala
import com.microsoft.azure.sqldb.spark.config.Config
as soon as i run this command i get the error
Object sqldb is not a member of package com.microsoft.azure
any help please
In the new connector you need to use com.microsoft.sqlserver.jdbc.spark.SQLServerBulkJdbcOptions class to specify bulk copy options.
I am facing issue in compiling 'TokenizerExample' that comes along with Spark-Scala package.
I have setup my environment in IntelliJ and I am able to successfully compile other Spark-Scala examples such as NaiveBayes, CosineSimilarity etc.
But when I load the 'TokenizerExample' into IntelliJ IDE, system displays an error message on the below line stating 'Cannot resolve reference transform with such signature':
val tokenized = tokenizer.transform(sentenceDataFrame)
val regexTokenized = regexTokenizer.transform(sentenceDataFrame)
I have not done any edit and I could observe that the issue is with the transform method. Could you please help me address this issue? Appreciate your support.
Thanks!
SBT/Maven, make sure you have all the dependencies listed in the .sbt or pom.xml file for the mllib module and rebuild. I personally didn't work on mllib, but this kind of errors will show up if we don't resolve the dependencies well enough. Thanks.
I am using Scala on Eclipse Luna and trying to connect to Cassandra. My code is showing the error apache object is not a member of package org on the following line:
import org.apache.spark.SparkConf
I already imported the Scala and Spark libraries into the project. Does someone know how can I make my program import Spark libraries?
I am trying to compile Orbeon Forms in IntelliJ IDEA 14.0.1 Community Edition, but I am getting compile errors.
Checked out the branch origin/4.7-ce
Opened the project with IntelliJ
Got on error: Load error: undefined path variables. MESSAGE is undefined. Fix it
Fixed the MESSAGE error by entering C:\temp as the value. I don't know if this matters.
Installed the Scala and File Watchers plugins
Configured the Java SDK as jdk1.7.0_09
Removed the scala-sdk that was referencing non-existent scala compiler, library, reflect jars in my $USER_HOME\.ivy2\cache directory.
Installed the Scala 2.10.3 SDK and configured as a library in the project.
Make Project
Got 94 errors, all related to the util and xml packages. I would have expected the packages to be scala.util and scala.xml.
Example errors:
orbeon-forms\src\main\scala\org\orbeon\oxf\common\PEVersion.scala
Error:(34, 8) object Try is not a member of package util<br/>
import util.Try
orbeon-forms\src\main\scala\org\orbeon\oxf\fb\AlertsAndConstraintsOps.scala<br/>
Error:(46, 9) type mismatch;<br/>
found : scala.xml.Elem<br/>
required: org.orbeon.saxon.om.NodeInfo<br/>
toXML currentLang<br/>
What am I doing wrong?
I was able to fix this error by changing the import from:
import util.Try to import scala.util.Try
There were other files that had the same issue with the scala.xml package.
The full list of files I changed is:
src\main\scala\org\orbeon\oxf\common\PEVersion.scala
src\main\scala\org\orbeon\oxf\fr\mongodb\MongoDBPersistence.scala
src\main\scala\org\orbeon\oxf\fr\process\ProcessInterpreter.scala
src\main\scala\org\orbeon\oxf\fr\process\SimpleProcess.scala
src\main\scala\org\orbeon\oxf\fr\process\XFormsActions.scala
src\main\scala\org\orbeon\oxf\xforms\action\XFormsAPI.scala
src\main\scala\org\orbeon\oxf\xforms\analysis\ElementAnalysis.scala
src\main\scala\org\orbeon\oxf\xforms\analysis\PathMapXPathAnalysis.scala
src\main\scala\org\orbeon\oxf\xforms\control\controls\XFormsInputControl.scala
src\main\scala\org\orbeon\scaxon\XML.scala
src\test\scala\org\orbeon\oxf\common\VersionTest.scala