Pig-Scala UDF exception- ScalaObject not found - scala

I have a scala .class file that I convert to a jar and try to register to a pig script. It is able to find that class now BUT it throws a ClassNotFoundException for scala.ScalaObject.
I notice that there is a scala.ScalaObject.class entry in the scala-library jar in the littlepiggy/lib folder.
Question 1
Shouldn't this jar be directly accessible anyways? Or do I have to add this path to an equivalent of a CLASSPATH for Pig?
Question 2
After this, I forcefully registered that jar as well.
I got this error:
java.lang.NoSuchMethodError: scala.collection.JavaConversions$.asScalaIterator(Ljava/util/Iterator;)Lscala/collection/Iterator
This doesn't look right to me. Any ideas?
PS - This source suggests that I should include the scala-library jar but Pig should already be able to find it and anyways, its not really working for me.
(http://mehack.com/levenshtein-distance-function-for-pig-and-had-0)

The answer's here. Should have checked out a more exhaustive set of keywords.
NoSuchMethodError when attempting to implicitly convert a java to scala collection
I was using different scala versions to build the class file and then in pig.
Thanks!

Related

scala & intellij : difficulties to create a runnnable jar

I can't launch a scala jar; when I launch it I get the error "Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/immutable/List" which seems to mean the scala library is not loaded...
this is a screenshot showing a lot of informations on the artifact window.
here is the manifest:
Manifest-Version: 1.0
Class-Path: libs/scala-library-2.10.0.jar libs/commons-logging-1.1.1.j
ar libs/jcip-annotations-1.0.jar libs/jwnl-1.4_rc3.jar libs/laf-plugi
n-7.2.1.jar libs/laf-widget-7.2.1.jar libs/miglayout-core-4.2.jar lib
s/miglayout-swing-4.2.jar libs/scala-actors.jar libs/scala-library.ja
r libs/scala-swing.jar libs/slf4j-api-1.6.4.jar libs/slick_2.10-1.0.0
.jar libs/sqlite-jdbc-3.7.2.jar libs/substance-7.2.1.jar libs/trident
-7.2.1-swing.jar
Main-Class: Fenetre
and when I enter "java xf myJar.jar", there are extracted files in the directory:
- .class files
- in the libs folder, there are the libraries INCLUDING scala-library.jar & scala-library-2.10.0.jar(I specified only one of these two files in the manifest to avoid conflicts)
can you help me?
I'm new to Scala and don't know what the problem is however I've been compiling "fat jars" which include all the required libs.
I've been using https://github.com/sbt/sbt-assembly to do this successfully.
Despite what your manifest is telling you, when you run the application you either do not have the scala-library included in your class path or there's some confusion when you attempt to import List. Scala should automatically import the immutable collection classes in your project with the root Predef implementation.
Predef provides type aliases for types which are commonly used, such as the immutable collection types scala.collection.immutable.Map, scala.collection.immutable.Set, and the scala.collection.immutable.List constructors (scala.collection.immutable.:: and scala.collection.immutable.Nil). The types Pair (a scala.Tuple2) and Triple (a scala.Tuple3), with simple constructors, are also provided.
Predef in core Scaladocs
Try printing the classpath from within your app to confirm.
Although not pertinent to your question, I would recommend using SBT for dependency management now that IntelliJ IDEA 13 has full SBT integration support. Your collaborators not using IntelliJ will also be happier because SBT gives them more options for build technologies, editors, and other tooling when working on the project.

import lift mapper caught an error missing jar file

hi im creating a lift scala application
this is my problem
when i import net.liftweb.mapper._ package the compiler gives me error...
i tried to add lift-mapper_2.9.1-2.4-M4.jar jar file to build path ,but the error exists..!
the compiler shows..
bad symbolic reference. A signature in package.class refers to term db in package net.liftweb which is not available. It may be completely missing from
the current classpath, or the version on the classpath might be incompatible with the version used when compiling package.class.
Try to add the mapper via SBT or maven configuration?
Anyway, Scala-2.9.1 and Lift 2.4-M4 are very-very out of date. Better use something like that for a good start: https://github.com/lift/lift_25_sbt/
It works out of the box, and you'll only have to add 1 line to add the mapper support for it. (After that, all dependencies would be downloaded and added to the project automatically, by SBT.)

Eclipse JAR at runtime

I'm importing several JARs all sitting in the same directory. When my program runs I'm able to access the classes in several of those JARs, no problem. So my classpath is fine (I assume). But there is one JAR giving me trouble. When I try to run:
Configuration conf = new BaseConfiguration();
I get a NoClassDefFoundError error. The searching I did on "NoClassDefFound" typically points to classpath problems but as mentioned above, other JARs in the same directory are being found so I think that's not the problem.
This is in Eclipse 3.8.
The import statements:
import org.apache.commons.configuration.BaseConfiguration;
import org.apache.commons.configuration.Configuration;
And the name of the JAR is:
commons-configuration-1.6.jar
Any ideas?
Tom
Perhaps it's failing to load a dependency of one of the classes in commons-configuration-1.6.jar? Maybe from another commons- JAR? The NoClassDefFoundError should tell you which class the runtime failed to load. Have you checked this?

Scala : trying to get log4j working

Scala newb here (it's my 2nd day of using it). I want to get log4j logging working in my Scala script. The script and the results are below, any ideas as to what's going wrong?
[sean#ibmp2 pybackup]$ cat backup.scala
import org.apache.log4j._
val log = LogFactory.getLog()
log.info("started backup")
[sean#ibmp2 pybackup]$ scala -cp log4j-1.2.16.jar:. backup.scala
/home/sean/projects/personal/pybackup/backup.scala:1: error: value apache is not a member of package org
import org.apache.log4j._
^
one error found
I reproduce it under Windows: delimiter of '-classpath' must be ';' there (not ':'). Are you use cygwin or some sort of unix emulator?
But Scala script works anywhere without current dir in classpath. Try to use:
$ scala -cp log4j-1.2.16.jar backup.scala
JFI: LogFactory is a class of slf4j library (not log4j).
UPDATE
Another possible case: broken jar in classpath, maybe during download or something else. Scala interpreter does report only about unavailable member of the package.
$ echo "qwerty" > example.jar
$ scala -cp example.jar backup.scala
backup.scala:1: error: value apache is not a member of package org
...
Need to inspect content of the jar-file:
$ jar -tf log4j-1.2.16.jar
...
org/apache/log4j/Appender.class
...
Did you remember to put log4j.jar in your classpath?
Had Similar issue when started doing Scala Development using Eclipse, doing a clean build solved the problem.
Guess the Scala tools are not matured et.
Instead of using log4j directly, you might try using Configgy. It's the Scala Way™ to work with log4j, as well as configuration files. It also plays nicely with SBT and Maven.
I asked and answered this question myself, have a look:
Put it under src/main/resources/logback.xml. It will be copied to the right location when SBT is doing the artifact assembly.

scala compilation problem - with java libs

Im trying to put together a simple ant build file for compiling a scala project (scala 2.7.6)...
I have everything working correctly except that its bringing up the
"fatal error: class java.lang.Object not found." both the scalac and scala libs are all in the project folder, so Im refering to them relativelly to the project root.
Im hoping that someone could help with which command line scalac option I can use to point the compiler towards the java core lib (if thats what it needs). Im hoping not to use any environemnt variables, and instead have the needed java libs in my project root folder...
that way I can have everything needed to compile movable with the project, and not need any setup for others to be able to compile
thank you
Scala 2.7.6 is a broken release.
Please upgrade to 2.7.7
Also, do you have javac on PATH and JAVA_HOME set?