I have a transformation which outputs a type with a wildcard: Feature<? extends Geomery>. I have specified a coder for this class and created a pipeline.
final Pipeline pipeline = Pipeline.create();
final TypeDescriptor<Feature<? extends Geometry>> featureTypeDescriptor =
new TypeDescriptor<Feature<? extends Geometry>>() {
};
pipeline.getCoderRegistry().registerCoderForType(featureTypeDescriptor, FeatureCoder.of());
final List<String> data = Arrays.asList("a", "b");
final PCollection<Feature<? extends Geometry>> features =
pipeline.apply(Create.of(data).withCoder(StringUtf8Coder.of()))
.apply(ParDo.of(new DoFn<String, Feature<? extends Geometry>>() {
#ProcessElement
public void process(ProcessContext processContext) {
final String name = processContext.element();
processContext.output(new FeatureImpl(name));
}
}));
pipeline.run().waitUntilFinish();
When I run this pipeline, I get the following error:
Exception in thread "main" java.lang.ClassCastException: org.apache.beam.sdk.repackaged.com.google.common.reflect.Types$WildcardTypeImpl cannot be cast to java.lang.reflect.TypeVariable
at org.apache.beam.sdk.coders.CoderRegistry.getCoderFromTypeDescriptor(CoderRegistry.java:623)
at org.apache.beam.sdk.coders.CoderRegistry.getCoderFromParameterizedType(CoderRegistry.java:656)
at org.apache.beam.sdk.coders.CoderRegistry.getCoderFromTypeDescriptor(CoderRegistry.java:618)
at org.apache.beam.sdk.coders.CoderRegistry.getCoder(CoderRegistry.java:252)
at org.apache.beam.sdk.values.PCollection.inferCoderOrFail(PCollection.java:149)
at org.apache.beam.sdk.values.PCollection.finishSpecifyingOutput(PCollection.java:89)
This can be traced back in the Beam code to the following line in org.apache.beam.sdk.coders.CoderRegistry#getCoderFromTypeDescriptor where type gets eventually assigned to ? extends Geometry:
else if (type instanceof WildcardType) {
// No coder for an unknown generic type.
throw new CannotProvideCoderException(
String.format("Cannot provide a coder for type variable %s"
+ " (declared by %s) because the actual type is unknown due to erasure.",
type,
((TypeVariable<?>) type).getGenericDeclaration()),
ReasonCode.TYPE_ERASURE);
}
The example below is a simplification of the real problem. In reality, I do not read from a list of strings but from HBase, so I cannot simply specify my coder in Create.of(data).withCoder(...). However, the behavior is the same.
Is this expected behavior and should I avoid using wild cards? Or should I approach this in another way? Why is my specified coder not used for this?
Related
I have a bit of code I'm using to handle any type of Map, copied from here: MongoDb Map<K, V> codec - Maps MUST have string keys FIX
#Override
public void encode(final BsonWriter writer, final Map<K, T> map, final EncoderContext encoderContext) {
try (var dummyWriter = new BsonDocumentWriter(new BsonDocument())) {
dummyWriter.writeStartDocument();
writer.writeStartDocument();
for (final Map.Entry<K, T> entry : map.entrySet()) {
var dummyId = UUID.randomUUID().toString();
dummyWriter.writeName(dummyId);
keyCodec.encode(dummyWriter, entry.getKey(), encoderContext);
//TODO: could it be simpler by something like JsonWriter?
writer.writeName(dummyWriter.getDocument().asDocument().get(dummyId).asString().getValue());
valueCodec.encode(writer, entry.getValue(), encoderContext);
}
dummyWriter.writeEndDocument();
}
writer.writeEndDocument();
}
This seems to work fine most of the time, except for when the Map is of type Map<ObjectId, *>, causing the following error:
Value expected to be of type STRING is of unexpected type OBJECT_ID
org.bson.BsonInvalidOperationException: Value expected to be of type STRING is of unexpected type OBJECT_ID
on the dummyWriter.getDocument().asDocument().get(dummyId).asString() line, in encode().
I have attempted to work arond this via the following code, but also fails; looks like there is an issue with how the type is written...
BsonValue cur = dummyWriter.getDocument().asDocument().get(dummyId);
if(cur.isObjectId()){
writer.writeName(
dummyWriter.getDocument().asDocument().get(dummyId).asObjectId().getValue().toHexString()
);
} else {
writer.writeName(
dummyWriter.getDocument().asDocument().get(dummyId).asString().getValue()
);
}
An exception occurred when decoding using the AutomaticPojoCodec.
Decoding into a '' failed with the following exception:
Cannot find a public constructor for ''.
I am beginning to suspect this code shouldn't work, and the default codec for Map<String, *> is actually being used where I thought I tested this codec before... What can get my codecs to play nice?
I'm defining some custom Exceptions in Dart.
I want in my logic to check the type of exception and base my processing on that, so I want to create distinct classes for each, for example like this :
class FailedToLoadCriticalDataException implements Exception { } // app cannot continue
class FailedToLoadNonCriticalDataException implements Exception { } // app can continue
However I also want to pass 2 parameters when I create these types of exceptions, the type of API call, and the API url, and the definition for that would look like this :
class UrlCallFailedException implements Exception {
String _dataTypeName;
String _urlEndpoint;
UrlCallFailedException([this._dataTypeName, this._urlEndpoint]);
#override
String toString() {
return "(${this.runtimeType.toString()}) Failed to fetch $_dataTypeName ($_urlEndpoint)";
}
}
Now what I want to do is (replace the initial definitions I made earlier and re)define my FailedToLoadCriticalDataException and FailedToLoadNonCriticalDataException classes so that they are exactly the code that is in the UrlCallFailedException class.
Is there any way to simply say something like class FailedToLoadCriticalDataException **is** UrlCallFailedException; and not need to duplicate the code that defines UrlCallFailedException ?
class FailedToLoadCriticalDataException implements UrlCallFailedException{ } is wrong because it is "Missing concrete implementations of 'getter UrlCallFailedException._dataTypeName',.."
class FailedToLoadCriticalDataException extends UrlCallFailedException{ } is wrong because when I got to throw FailedToLoadNonCriticalDataException("Foo", url); it's expectation is that there are no params ("Too many positional arguments: 0 expected, but 2 found.").
Is there a way to create multiple classes that behave exactly the same as another type and differ only in their class, without duplicating all the code ?
I've come up with this as a decent compromise :
class FailedToLoadCriticalDataException extends UrlCallFailedException {
FailedToLoadCriticalDataException([dataTypeName, urlEndpoint]) {
super._dataTypeName = dataTypeName;
super._urlEndpoint = urlEndpoint;
}
}
class FailedToLoadNonCriticalDataException extends UrlCallFailedException {
FailedToLoadNonCriticalDataException([dataTypeName, urlEndpoint]) {
super._dataTypeName = dataTypeName;
super._urlEndpoint = urlEndpoint;
}
}
Some, but minimal, code duplication, and I can now call throw FailedToLoadNonCriticalDataException("Foo", url); in my code later.
Is there any sample Scala code available for creating stored procedures in Neo4j-3.0.3 ?
I have been trying to create one simple Scala based stored procedure. Below is the Error message I get when I copy my scala-jar file to the neo4j-plugins directory and start the neo4j server :
=================
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.kernel.impl.proc.Procedures#1ac0223' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:444)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:140)
... 10 more
Caused by: org.neo4j.kernel.api.exceptions.ProcedureException: Unable to find a usable public no-argument constructor in the class `neoscala`. Please add a valid, public constructor, recompile the class and try again.
=================
The scala class that I have used is :
package neoproc
import org.neo4j.graphdb.GraphDatabaseService
import org.neo4j.procedure.Procedure;
import javax.ws.rs.core.{Context, Response}
class neoscala(#Context db: GraphDatabaseService) {
#Procedure
def alice():String = {
String.valueOf(db.execute( "MATCH (n:User) return n" ));
}
}
Your Scala class declares a constructor with a GraphDatabaseService argument, and the exception tells you that it only wants a no-argument constructor.
It's documented in both
the user documentation:
Only static fields and #Context-annotated fields are allowed in Procedure classes.
the Javadoc:
The procedure method itself can contain arbitrary Java code - but in order to work with the underlying graph, it must have access to the graph API. This is done by declaring fields in the procedure class, and annotating them with the Context annotation. Fields declared this way are automatically injected with the requested resource. This is how procedures gain access to APIs to do work with.
All fields in the class containing the procedure declaration must either be static; or it must be public, non-final and annotated with Context.
Apparently it's not possible to create a class with a public field in Scala, so you'll have to create a parent Java class with the public field, and extend it with your Scala class:
// ProcedureAdapter.java
public abstract class ScalaProcedureAdapter {
#Context
public GraphDatabaseService db;
}
// neoscala.scala
class neoscala extends ScalaProcedureAdapter {
// ...
}
Here is the solution for this :
We will create Class in scala :
class FullTextIndex extends JavaHelper {
#Procedure("example.search")
#PerformsWrites
def search(#Name("label") label: String,
#Name("query") query: String): Stream[SearchHit] = {
//declare your method
}
val nodes: Stream[Node] = db.index.forNodes(index).query(query).stream
val newFunction: java.util.function.Function[Node, SearchHit] = (node: Node) => new SearchHit(node)
nodes.map {
newFunction
}
}
private def indexName(label: String): String = {
"label-" + label
}
}
Procedure in Neo4j always return result in Stream and it is a latest feature in Java8 so we will also used Java Class for return the final result and For defining the public variable.
We will create Java class for result :
public class JavaHelper {
#Context
public GraphDatabaseService db;
#Context
public Log log;
public static class SearchHit {
//your result code here
}
You can refer knoldus blog for Neo4j User Defined Procedure for creating and storing Neo4j Procedure with Scala. Here you will also find sample code with git hub repository.
I'd like to generate a List field into my class generated from my DSL and initialize it like this:
private List<MyObject> myObjects= Lists.newArrayList();
The only way I know for this, is to append some text to the initializer:
members += appRule.toField("myObjects", appRule.newTypeRef(List, it.newTypeRef(MyObject))) [
initializer = [append('''Lists.newArrayList()''')]
]
However, using this approach the JvmModelInferrer won't import the Guava Strings library, thus will raise compilation issues. Is there any way to overcome this obstacle?
If I understand your issue (as you are referring to the Guava Strings library that is not used in the code :) ), your problem is, that the class reference Lists is not imported.
For such constructs, we have a helper method in EMF-IncQuery that serializes a type reference the same way parameters are serialized. This functionality relies on the injectable TypeReferenceSerializer class.
def referClass(ITreeAppendable appendable, EObject ctx, Class<?> clazz, JvmTypeReference... typeArgs) {
val ref = ctx.newTypeRef(clazz, typeArgs)
if (ref != null) {
appendable.serialize(ref, ctx)
} else {
//Class resolution error - error handling required here
//A fallback to writing out the fqn of the class
appendable.append(clazz.canonicalName)
}
}
def serialize(ITreeAppendable appendable, JvmTypeReference ref, EObject ctx) {
typeReferenceSerializer.serialize(ref, ctx, appendable)
}
I would like to have an embedded document referred to by a map (as in 'class A' below). The environment is Grails + GORM + MongoDB.
is that possible, and if yes, how?
class A { // fails with IllegalArgumentException occurred when processing request: can't serialize class X in line 234 of org.bson.BasicBSONEncoder
static mapWith = "mongo"
Map<String, X> map = new HashMap<String, X>()
}
class B { // works
static mapWith = "mongo"
List<X> list = new ArrayList<X>()
}
class C { // works with primitive type values
static mapWith = "mongo"
Map<String, String> map = new HashMap<String, String>()
}
class X {
String data
public X(String data) {
this.data = data
}
}
The embedding works perfectly,as Art Hanzel advised.
However your problem comes from the fact that you try and use List genericity as a sort of constraint :
Map<String, X>
The problem is that Grails couldn't cope well with this syntax, first because Groovy doesn't support genericity.
However, the MongoDB plugin offers a very powerful functionality that lets you define custom type as Domain Class Properties : see here.
In your case you could have
class A {
static mapWith = "mongo"
MyClass map = new MyClass()
}
Then in your src/java for example you could for example implement a
class MyClass extends HashMap<String,X> { }
Then, of course, you have to define a special AbstractMappingAwareCustomTypeMarshaller to specify how to read and write the property in the DB.
An additional step could also be to add a custom validator to class A to check the validity of data...
The MongoDB Grails plugin documentation describes how to make embedded documents:
class Foo {
Address address
List otherAddresses
static embedded = ['address', 'otherAddresses']
}
Off the top of my head, you should be able to access these via the object graph. I don't see any reason why you shouldn't.
myFoo.address.myAddressProperty...