In scala it is common practice to stack package statements to allow shorter imports, but when I load a file using stacked packages into the scala ide and I attempt to use an import starting with the same organization I get a compiler error from what appears to be the presentation compiler. The code compiles fine in sbt outside of the IDE.
An example code snippet is as follows:
package com.coltfred
package util
package time
import com.github.nscala_time.time.Imports._
On the import I get the error object github is not a member of package com.coltfred.util.com.
If I move the import to a single line the error will go away, but we've used this practice frequently in our code base so changing them all to be single line package statements would be a pain.
Why is this happening and is there anything I can do to fix it?
Edit:
I used the eclipse-sbt plugin to generate the eclipse project file for this. The directory structure is what it should be and all of the dependencies are in the classpath.
Edit 2:
It turns out there was a file in the test tree of the util package (which should have been in the same package), but had a duplicate package statement at the top. I didn't check the test tree because it shouldn't affect the compilation of the main tree, but apparently I was wrong.
Not sure why the Scala IDE is not liking this, but you can force the import to start at the top level using _root_:
import _root_.com.github.nscala_time.time.Imports._
See if that avoids irritating the IDE.
This is a common annoyance that annoyed paulp into an attempt to fix it. His idea was that a dir that doesn't contribute class files shouldn't be taken as a package. If you can take util as scala.util, you should do so in preference to foo.util where that util is empty.
The util dir is the usual suspect, because who doesn't have a util dir lying around, and in particular, ./util?
apm#mara:~/tmp/coltfred$ mkdir -p com/coltfred/util/time
apm#mara:~/tmp/coltfred$ mkdir -p com/coltfred/util/com
apm#mara:~/tmp/coltfred$ vi com/coltfred/util/time/test.scala
apm#mara:~/tmp/coltfred$ scalac com/coltfred/util/time/test.scala
./com/coltfred/util/time/test.scala:5: error: object github is not a member of package com.coltfred.util.com
import com.github.nscala_time.time._
^
one error found
apm#mara:~/tmp/coltfred$ cat com/coltfred/util/time/test.scala
package com.coltfred
package util
package time
import com.github.nscala_time.time._
class Test
apm#mara:~/tmp/coltfred$
To debug, find out where the offending package is getting loaded from.
Related
I have two modules, both named connection.py in two separate environments listed below. Both of the folders containing connection.py are in my PYTHONPATH system environment variable.
However, if that of spec is not placed above that of bvbot, spec's test_connection.py attempts to import from the connection.py of bvbot.
If in cmd, I can resolve this by moving the path of spec above that of bvbot. But, in Visual Studio Code, spec's test_connection.py still imports from bvbot's connection.py.
The two environments of interest are:
C:\Users\You_A\Desktop\2016Coding\VirtualEnviroments\spec\spec_trading
C:\Users\You_A\Desktop\2016Coding\VirtualEnviroments\bvbot\Legacy_bvbot
Structure of the spec path above:
src/
spec_trading/
__init__.py
connection.py
tests/
__init__.py
connection.py
spec test_connection.py:
import pytest
from connection import Connection, OandaConnection
class TestConnection:
def test_poll_timeout(self):
connection = Connection()
timeout = 10.0
connection.set_poll_timeout(timeout)
assert connection.poll_timeout == timeout
What I am doing wrong here? How can I resolve this without resorting to manually faffing with my systems environment variables and resolve the VSC issue?
Easiest solution is to not use implicit relative imports (I assume this is Python 2.7). Basically use explicit relative imports and make sure the imports resolve within the package they are contained within instead of Python having to search sys.path for the module.
And if you are using Python 2.7, put from __future__ import absolute_import at the top of the file.
I'm having trouble importing magellan-1.0.4-s_2.11 in spark notebook. I've downloaded the jar from https://spark-packages.org/package/harsha2010/magellan and have tried placing SPARK_HOME/bin/spark-shell --packages harsha2010:magellan:1.0.4-s_2.11 in the Start of Customized Settings section of the spark-notebook file of the bin folder.
Here are my imports
import magellan.{Point, Polygon, PolyLine}
import magellan.coord.NAD83
import org.apache.spark.sql.magellan.MagellanContext
import org.apache.spark.sql.magellan.dsl.expressions._
import org.apache.spark.sql.Row
import org.apache.spark.sql.types._
And my errors...
<console>:71: error: object Point is not a member of package org.apache.spark.sql.magellan
import magellan.{Point, Polygon, PolyLine}
^
<console>:72: error: object coord is not a member of package org.apache.spark.sql.magellan
import magellan.coord.NAD83
^
<console>:73: error: object MagellanContext is not a member of package org.apache.spark.sql.magellan
import org.apache.spark.sql.magellan.MagellanContext
I then tried to import the new library like any other library by placing it into the main script like so:
$lib_dir/magellan-1.0.4-s_2.11.jar"
This didn't work and I'm left scratching my head wondering what I've done wrong. How do I import libraries such as magellan into spark notebook?
Try evaluating something like
:dp "harsha2010" % "magellan" % "1.0.4-s_2.11"
It will load the library into Spark, allowing it to be imported - assuming it can be obtained though the Maven repo. In my case it failed with a message:
failed to load 'harsha2010:magellan:jar:1.0.4-s_2.11 (runtime)' from ["Maven2 local (file:/home/dev/.m2/repository/, releases+snapshots) without authentication", "maven-central (http://repo1.maven.org/maven2/, releases+snapshots) without authentication", "spark-packages (http://dl.bintray.com/spark-packages/maven/, releases+snapshots) without authentication", "oss-sonatype (https://oss.sonatype.org/content/repositories/releases/, releases+snapshots) without authentication"] into /tmp/spark-notebook/aether/b2c7d8c5-1f56-4460-ad39-24c4e93a9786
I think file was to big and connection was interrupted before whole file could be downloaded.
Workaround
So I downloaded the JAR manually from:
http://dl.bintray.com/spark-packages/maven/harsha2010/magellan/1.0.4-s_2.11/
and copied it into the:
/tmp/spark-notebook/aether/b2c7d8c5-1f56-4460-ad39-24c4e93a9786/harsha2010/magellan/1.0.4-s_2.11
And then :dp command worked. Try Calling it first, and if it will fail copy JAR into the right path to make things work.
Better solution
I should investigate why download failed to fix it in the first place... or put that library in my local M2 repo. But that should get you going.
I would suggest to check this:
https://github.com/spark-notebook/spark-notebook/blob/master/docs/metadata.md#import-download-dependencies
and
https://github.com/spark-notebook/spark-notebook/blob/master/docs/metadata.md#add-spark-packages
I think the :dp magic command is depreciated, instead you should add your custom dependencies in the notebook metadata. You can go in the menu Edit > Edit notebook metadata, there add something like:
"customDeps": [
"harsha2010 % magellan % 1.0.4-s_2.11"
]
Once done, you will need to restart the kernel, you can check in the browser console if the package is being downloaded properly.
The easy way, you should set or add the EXTRA_CLASSPATH environnent variable to point to your .jar file downloaded :
export EXTRA_CLASSPATH = </link/to/your.jar> or set EXTRA_CLASSPATH= </link/to/your.jar> in wondows OS. Here find the detailed solution.
I think there is a bug with respect to how PyDev (version 4.6) recognizes intra-package imports when selecting Grammar 3.x for the project preferences. I have a project like this:
foobar
mypack
__init__.py
mod1.py
mod2.py
mod2.py simply says
from mod1 import fun1
mod1.py simply says
def fun1():
print("Hey we are in fun1 in mod1")
If the project Python project preferences are set to use Grammar 3.0-3.5, with a Python 3.4 interpreter, and I open up mod2.py the line from mod1 import fun1 is highlighted with an error Unresolved import: fun1. If I change the Python project preferences to use Grammar 2.7, close the file mod2.py and reopen it, the error disappears. Just by changing the grammar back and forth, and closing/reopening the file, I can make the error appear/disappear.
So it seems that setting the Grammar to 3.x in PyDev causes intra-package imports to be incorrectly flagged as having an import error.
Any suggestions?
PyDev is doing right... on Python 3, relative imports must be written as:
from .mod1 import fun1
If the import doesn't start with a dot, it'll consider it an absolute import (and will properly show the error for you as with that absolute path the imported file cannot be resolved).
So my real issue was getting PyDev to not report errors about the imports, and to be able to debug modules buried in packages that have a main() in there for debugging. The solution for me was to use relative imports (as said in Fabio's answer), and then to do the following for debugging purposes. Let's say I want to run a module pack1.subpack2.subpack3.subpack4.modtodebug using PyDev, that has relative imports, and has a main() function. At the top level of my project, I have a module debugmain.py that reads
from pack1.subpack2.subpack3.subpack4.modtodebug import main as debugmain
if __name__ == '__main__':
debugmain()
I then have one run configuration for debugmain.py and each time I want to debug a different module, I just have to edit the code in debugmain.py to point to that module.
I hope this helps someone else with this kind of problem.
I can't use functions of custom subdirectories.
My Code Organziation
I have under "src" a path hierarchy like
a/b
with all my directories and go-Files (it is the "root" of my project). The directories contain no subdirectories and it works fine. So the deepest path is "a/b/c". E.g. I have
a/b/c
and
a/b/d
with some go-files. Import of "a/b/d" and calling a function with "d.DoSomething()" from a file in "a/b/c" works fine.
Problem description
Now I want ot reorganize "a/b/d". I move some files from "a/b/d" to
a/b/d/e
and the rest of the files to
a/b/d/f
If try to import "a/b/d/e" with import-statement
import ( "a/b/d/e" )
from the same file in "/a/b/c" and want to call "e.DoSomething()" (it is the place, where the file with the "DoSomething-function" moved to), I get an error at the line, where I call "e.DoSomething()": "undefined: e".
While searching for a result, I've nowhere seen examples with deeper path hierarchies. Is it generally not possible to use/import subdirectories or what's the problem?
go-version I used: go1.2.2 linux/amd64
Thanks for any advices
Your approach is completely wrong. Go has absolutely no concept of importing files or directories, all you can import in Go are packages. It now happens that the name of a package is it's path relative to GOPATH and you import packages by that name. But the identifier under which an imported package is available in the importing code depends on the package declaration of the package. You cannot simply "move" files between directories as each directory (for the go tool) is a single package without changing the package declaration.
You can have package x under path a/b/c. When you import package x with import ( "a/b/c" ) all the exported stuff from package x is available to you as x.ExportedName.
Please read http://blog.golang.org/organizing-go-code carefully.
Try and do a go build in a/b/d/e first, before trying to build in a/b: that will generate the compiled classes you want to import.
I get the following warning whenever I start my Scala application:
WARN - imported `SVNProperties' is permanently hidden by definition of object SVNProperties in package core, at line 4 of app/core/SVNResource.scala
What could this mean?
You probably have code that looks something like this:
object Hidden {
import scala.collection.immutable
object immutable { def x = 7 }
}
except in a less obvious way. You're importing something--in my example, the package immutable--and then you go and define something else with the same name that prevents you from using what you imported.
In particular, it looks like you tried to import SVNProperties into SVNResource.scala, except that SVNResource.scala defines its own SVNProperties which hides the import.
I encountered this warning after moving some classes from one package to another. I guess there was some conflict between the new location and binaries from the old location. In my case this helped:
sbt clean
I got this warning when my class was importing classes in the same package.
Once I removed the unnecessary import, the warnings were removed.
This happened to me after moving a class from one package to another, like in astasiak's case. I ran sbt clean with no luck. For the life of me, I couldn't find the class in the old location.
However, I had other errors preventing me from building. When I fixed those, this error disappeared. My guess is that until you can build cleanly, sbt still thinks you have the class is in the old package, and includes this error with any other build errors that are preventing you from building.
My advice? Check for other compilation errors and fix those -- you might be erroneously getting this error due to sbt having an outdated view of your package structure since its last successful build.
Just to further expand on a comment posted by Nick, as this was the case for me:
Another common cause is that SVNProperties is in the same package and so is already in scope. Trying to import it explicitly results in this warning.
More concretely, you may have:
package app.core
// No need to import the following - it is already visible as
// you are in the same package
import app.core.SVNProperties
object SVNResource {
Use instead:
package app.core
object SVNResource {
(Opinion) As a matter of coding style you may want to camel case your variables differently like for eg. SvnProperties, SvnResource. It reads easier for most people, see also this question.
I had a main class with name Server and I was creating a jetty server in the main class in the following way.
import org.eclipse.jetty.server.Server
var server:Server=new Server()
I got the below warn on running sbt run
> [warn] /home/xxx/xxx/xxx/src/main/scala/com/xxx/xxx/main/Server.scala:3:
> imported `Server' is permanently hidden by definition of object Server in package main
[warn] import org.eclipse.jetty.server.Server
[warn] ^
[warn] one warning found
I renamed my main class and the warning disappeared.
If you are using Scala Eclipse IDE you can do :
Project > Clean...
After that all the warning will be removed.
Also, make sure the name of the package you import =/= object name.
I got this by having circular dependencies. Both my classes were using each other on accident.
If the warning comes from importing a class with the same name, you can always use import renaming.
package domain
case class Company (...
package models
import domain.{Company => _Company}
object Company extends SkinnyCRUDMapper[_Company] {
Issue is related to dependency conflict, When you have same class in multiple Jars compiler found one of class is hidden and gives an error.
check for your Jar version is same across project or try to change name/package for one of conflicted class