I'm trying to use the DeBerta model and I'm first trying to implement some of the code I found here (an example of how to use the model), just to make sure I have the right dependencies and know what imports I need. I'm working in Scala 2.4.0, Zeppelin 0.8.1. I've added "com.johnsnowlabs.nlp:spark-nlp-spark24_2.11:3.4.4" as a dependency, which I believe should give me spark-nlp 3.4.4. So far most of the imports I got from the sample code are working:
import com.johnsnowlabs.nlp.embeddings
import com.johnsnowlabs.nlp.annotator._
import com.johnsnowlabs.nlp.base._
import com.johnsnowlabs.nlp.training.CoNLL
import com.johnsnowlabs.nlp.util.io.ResourceHelper
import com.johnsnowlabs.util.Benchmark
import org.apache.spark.ml.Pipeline
import org.apache.spark.sql.functions.{col, explode, size}
Although two of the imports get errors
but I'm not sure I need these to load the model. But when I try to load
val embeddings = DeBertaEmbeddings.pretrained("deberta_v3_base", "en")
.setInputCols("sentence", "token")
.setOutputCol("embeddings")
.setMaxSentenceLength(512)
I get an error saying
java.lang.IllegalArgumentException: requirement failed: Can not find deberta_v3_base inside public/models to download. Please make sure the name and location are correct!
at scala.Predef$.require(Predef.scala:224)
at com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadResource(ResourceDownloader.scala:441)
at com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(ResourceDownloader.scala:499)
at com.johnsnowlabs.nlp.pretrained.ResourceDownloader$.downloadModel(ResourceDownloader.scala:492)
at com.johnsnowlabs.nlp.HasPretrained$class.pretrained(HasPretrained.scala:44)
at com.johnsnowlabs.nlp.annotator.package$DeBertaEmbeddings$.com$johnsnowlabs$nlp$embeddings$ReadablePretrainedDeBertaModel$$super$pretrained(annotator.scala:532)
at com.johnsnowlabs.nlp.embeddings.ReadablePretrainedDeBertaModel$class.pretrained(DeBertaEmbeddings.scala:329)
at com.johnsnowlabs.nlp.annotator.package$DeBertaEmbeddings$.pretrained(annotator.scala:532)
at com.johnsnowlabs.nlp.annotator.package$DeBertaEmbeddings$.pretrained(annotator.scala:532)
at com.johnsnowlabs.nlp.HasPretrained$class.pretrained(HasPretrained.scala:47)
at com.johnsnowlabs.nlp.annotator.package$DeBertaEmbeddings$.com$johnsnowlabs$nlp$embeddings$ReadablePretrainedDeBertaModel$$super$pretrained(annotator.scala:532)
at com.johnsnowlabs.nlp.embeddings.ReadablePretrainedDeBertaModel$class.pretrained(DeBertaEmbeddings.scala:326)
at com.johnsnowlabs.nlp.annotator.package$DeBertaEmbeddings$.pretrained(annotator.scala:532)
... 51 elided
I've tried to go through the documentation on the DeBerta model, and from what I've seen I have adequate versions of scala and spark-nlp according to John Snow Labs models hub. I saw on this stackoverflow thread that I might need to put "_noncontrib" after my model name. I tried that even though it says I wouldn't need it after spark-nlp 2.4.0+. So now I'm pretty sure "deberta_v3_base" is the right name, but I'm not sure what the error means by the location being incorrect. Do I need to specify a location for the pretrained model per the config helper? Does anyone know what's going on here? I apologize if I've left anything out, I'm new to Zeppelin and spark-nlp. Thanks!
Related
First allow me to say, I am trying to learn Databricks but have years of Data Factory and ETL experience. I got some code that uses AIS data to map logistics movements.
The code uses Uber libraries for H3Core functionality. I did see a demo of this code on a coworkers laptop so I know it CAN work.
I am having trouble finding the uber objects. I assume this is a newbie thing. I imagine my problem is environmental.
I cannot post all the code but the include lines that are throwing the error:
import com.uber.h3core.H3Core
import com.uber.h3core.util.GeoCoord
import com.uber.h3core.LengthUnit
Those lines produce the following errors on execution:
command-2206228078162026:1: error: object uber is not a member of package com
import com.uber.h3core.H3Core
^
command-2206228078162026:2: error: object uber is not a member of package com
import com.uber.h3core.util.GeoCoord
^
command-2206228078162026:3: error: object uber is not a member of package com
import com.uber.h3core.LengthUnit
^
Below I think it is trying to reference objects created with missing class libraries for H3Core:
command-2206228078162026:15: error: not found: value H3Core
val h3 = H3Core.newInstance()
^
command-2206228078162026:26: error: not found: value H3Core
val h3 = H3Core.newInstance()
^
I have also had similar issues, usually occurring when the size of the jar is a little bigger where in the notebook starts running before the jar is fully loaded. But I have not been able to reproduce it consistently. (So don't quote the answer on it.)
Couple of Options we have tried. Please validate and see if it work for you.
Have the jar available on the DBFS path and then install it from DBFS as a dependent library at the job level. Please refer to the link - Dependent-Libraries
We have seen that sometimes even this fails. If it is possible, you can check if the library has been loaded and sleep and then check again. Not sure how you would do that in scala.
Please validate what works....
Cheers...
I'm having trouble using the Backendless plugin for Flutter.
I include
import 'package:backendless_sdk/backendless_sdk.dart';
(as per the instructions) and can then use e.g. Backendless.UserService. But if I try to generate a user to register, e.g.:
var user = new BackendlessUser();
user.setEmail("info#example.org");
user.setPassword("password");
Backendless.UserService.register(user);
I get an error Undefined class 'BackendlessUser' on the first line. This class is defined in src/modules/user_service.dat, which is exported by src/modules/modules.dartlike this:
library modules;
export 'cache.dart';
...
export 'user_service.dart';
which in turn is imported by backendless_sdk.dart like this:
import 'package:backendless_sdk/src/modules/modules.dart';
I would have thought that it would get imported indirectly by the import of backendless_sdk.dart, but apparently not. When I import it explicitly (with the same import statement, but now in my own code and not just indirectly in backendless_sdk.dart), I get a warning Don't import implementation files from another package. But it's not an implementation file; it's exported as part of the public API (at least that's what I understand the export statement to mean).
The Dart tutorial for creating packages suggests to place the export statements directly under lib, not in lib/src, so I'm wondering whether this is an error in the structure of the plugin, or whether I'm doing something wrong.
I'd be grateful both for a solution to this particular problem and also for pointers to how I can better understand packages, libraries, imports and exports in dart; unfortunately I don't find the language specification particularly helpful in this regard.
(The error and the warning are the same whether I use flutter analyze or IntelliJ IDEA.)
The problem has been fixed in the 0.0.3 version of the plugin. Please update the backendless_sdk version in your pubspec.yaml.
You can include the only one import now:
import 'package:backendless_sdk/backendless_sdk.dart';
Please also note, that there are some changes in the syntax. So for your example you should use:
var user = new BackendlessUser()
..email = "info#example.org"
..password = "password";
Backendless.userService.register(user);
Thanks for using Flutter SDK and pointing out this issue.
It's indeed the problem in the structure of the plugin. The Backendless team is aware of it and this problem will be fixed in the next release of the plugin.
For now you can import explicitly and suppress the warning.
I'm new to flash development, so I'm watching a tutorial on how to use FlashDevelop. The video recommended I use Box2D and explained how to use it as a global classpath, which I have done.
I was messing around with the code using what the person in the video was showing, just trying to get an output. As I typed, FlashDevelop was adding in the import statements for me.
import Box2D.Collision.Shapes.b2CircleShape;
import Box2D.Common.Math.b2Vec2;
import Box2D.Dynamics.b2BodyDef;
import Box2D.Dynamics.b2FixtureDef;
import Box2D.Dynamics.b2World;
import Box2D.Dynamics.b2Body;
When I run the program though, it's returning this:
col: 31 Error: Definition Box2D.Collision.Shapes:b2CircleShape could not be found.
It's returning a variation of that for each import.
I've checked and the files are indeed there. I'm really not certain what this could be; it's possible I just missed a step.
Any ideas?
(Sorry if I formatted this question incorrectly, I'm new to this site.)
It's maybe cause you are using an old version
I think these are your choices :
1) you have to do an update
or
2) use "b2CircleDef"
See the code source in this link the change are commented
http://www.emanueleferonato.com/2010/01/27/box2dflash-2-1a-released-what-changed/
Hope that was helpful !
Good luck
I'm writing a small simulation program in Scala. It is actor-based so I've created a file messages.scala that contains all of the messages that are valid in the system.
Outside of this, I have a management component, management.scala and a file that defines the nodes and links classes nodes.scala. Management and node files both import sim.messages._ and then management does import sim.nodes._ as it needs to be able to instantiate things from that file.
The problem comes with one message type Tick that is used by both management.scala and nodes.scala. Upon compiling the management component, I get:
error: reference to Tick is ambiguous;
it is imported twice in the same scope by
import sim.nodes._
and import sim.messages._
I tried removing the import of messages in the management component since they were apparently already imported in to this scope but then they couldn't find them anymore. Ideas?
Try
import sim.nodes._
import sim.nodes.{ Tick => NodesTick }
and/or
import sim.messages._
import sim.messages.{ Tick => MessagesTick }
Of course, you will have to rename the references to Tick with the right one.
I have a class uses the following lines, it works fine in a Google App Engine project:
import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.IdentityType;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;
But when I included this class in another project, it cause error :
package javax.jdo.annotations does not exist
What should I do to find javax.jdo.* ?
Add the JDO jar file to the class path.
The star notation for imports isn't working the way you think it does.
It's not recursive - it only applies the child classes in javax.jdo, not the child packages.
If you want all the classes in javax.jdo.annotations, you'll need to import javax.jdo.annotations.*, too.
I'd recommend not using the star notation. Better to type out the imports for every class individually. Use an IDE to help you. It's clearer for you and other programmers who come after you where those classes came from.