I am trying to create a dummy PCollection with my custom objects as follows:
PCollection<MyClass> pipelineProcessingResults = pipeline.apply(Create.of(new MyClass(.., ..)));
MyClass class is as follows:
#DefaultSchema(JavaBeanSchema.class)
public class MyClass {
AnotherComplexClass _obj;
Urn _urn
}
I am getting the following exception:
java.lang.StackOverflowError
at java.util.HashMap.hash(HashMap.java:339)
at java.util.HashMap.put(HashMap.java:612)
at java.util.HashSet.add(HashSet.java:220)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.TypeVisitor.visit(TypeVisitor.java:66)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.Types.getComponentType(Types.java:197)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.TypeToken.getComponentType(TypeToken.java:563)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.TypeToken.isArray(TypeToken.java:512)
at org.apache.beam.sdk.values.TypeDescriptor.isArray(TypeDescriptor.java:193)
at org.apache.beam.sdk.schemas.utils.ReflectUtils.getIterableComponentType(ReflectUtils.java:196)
at org.apache.beam.sdk.schemas.FieldValueTypeInformation.getIterableComponentType(FieldValueTypeInformation.java:274)
at org.apache.beam.sdk.schemas.FieldValueTypeInformation.forGetter(FieldValueTypeInformation.java:189)
at org.apache.beam.sdk.schemas.JavaBeanSchema$GetterTypeSupplier.get(JavaBeanSchema.java:74)
at org.apache.beam.sdk.schemas.utils.StaticSchemaInference.schemaFromClass(StaticSchemaInference.java:92)
at org.apache.beam.sdk.schemas.utils.StaticSchemaInference.fieldFromType(StaticSchemaInference.java:166)
The class AnotherComplexClass may contain multiple fields which in turn are composed of other classes.
Which coder will best suit my purpose? Should I create a custom coder? Using the #DefaultSchema annotation did not help me much. I tried using SerializableCoder, but it throws a compiler error:
Cannot resolve method 'of(java.lang.Class<MyClass>)'
Option 1 - Custom Coder
Since you have complex nested data types, you can define a custom coder and use it with the #DefaultCoder annotator. Details see https://beam.apache.org/documentation/programming-guide/#annotating-custom-type-default-coder.
public class MyCoder implements Coder {
public static Coder<T> of (Class<T> clazz) {...}
}
#DefaultCoder(MyCoder.class)
public class MyClass {...}
Option 2 - Serializable
You can also make sure that all your POJO classes implement Serializable and by default, Java SDK uses the SerializableCoder. But it's inefficient and non-deterministic.
Option 3 - Avro
You can use AvroCoder and use Avro to generate your classes. https://avro.apache.org/docs/current/gettingstartedjava.html
Option 4 - Protocol Buffer
Similar to Avro, you can use Protocol Buffer to define your schema and classes. https://developers.google.com/protocol-buffers/docs/javatutorial.
Related
I am trying to use the in built methods of MongoRepository<T,ID> interface to interact with mongo.
interface MovieRepository : MongoRepository<Movie, String> {
}
But when I try to implement the "MovieRepository" using class. Its asking me to implement all the member functions defined in "MongoRepository" as well
class ControllerClass(private val MovieRepository: MovieRepository): MovieRepository {}
This is what i get when i initialize my controller class:
Class 'ControllerClass' is not abstract and does not implement abstract member public abstract fun <S : Movie!> save(entity: S): S
Is there any way so that i do not need to defined all those MongoRepository's functions again in my ControllerClass?
You don't usually implement a repository interface yourself: you let Spring do it for you!
First, you define your interface, as you have done:
interface MovieRepository : MongoRepository<Movie, String> {
// Add any abstract methods you'll need here…
}
Then, you autowire a property of that type. In Kotlin, you can either do it in the primary constructor, e.g.:
#Controller
class ControllerClass #Autowired constructor(
private val movieRepository: MovieRepository
) {
// …code…
}
Or as a plain property. (In this case, because you can't specify an initial value, you have to make the property a var; it must either be nullable — requiring !! everywhere you use it — or, better, make it lateinit.)
#Controller
class ControllerClass {
#Autowired private lateinit var movieRepository: MovieRepository
// …code…
}
Spring will then create some synthetic class implementing that interface, and set your property to it. (You don't need to worry about how it does that — just as you don't need to worry about all the other magic it does, much of which involves creating synthetic subclasses. That's why Spring objects generally need to be made open — and why there's a Spring plugin which takes care of doing that.)
It's more usual to use the repository in a service class, and then call that from your controller class — at least, that pattern tends to scale better, and be easier to follow and to test. But doing so directly should work too. Either way, you can call whichever repository method you need, e.g. movieRepository.findAll().
See the Spring docs; they use Java, but it's mostly trivial to convert to Kotlin.
Pardon me as I am new to Scala.
I have created a case class which encapsultes some information. One of the objects i want to take in for that is of JavaClass. As i am using in spark, i would need it to be serializable. How can i do that?
Java class
public class Currency {
public Currency(final BigDecimal amount, final CurrencyUnit unit) {
//Doing Something
}
}
case class ReconEntity(inputCurrency : Currency, outputCurrency : Currency)
Using implicit i want to have my serialization code for Currency so that spark can work on ReconEntity.
Firstly, have you tried some RDD operations using your Currency and ReconEntity classes? Do you actually get an error? Spark is able to handle RDD operations with apparently non-serializable Scala classes as values, at least (you can try this in the spark-shell, though possibly this might require the Kryo serializer to be enabled).
Since you state that you don't own the Currency class, you can't add extends Serializable, which would be the simplest solution.
Another approach is to wrap the class with a serializable wrapper, as described in this article: Beating Serialization in Spark - example code copied here for convenience:
For simple classes, it is easiest to make a wrapper interface that
extends Serializable. This means that even though UnserializableObject
cannot be serialized we can pass in the following object without any
issue
public interface UnserializableWrapper extends Serializable {
public UnserializableObject create(String prm1, String prm2);
}
The object can then be passed into an RDD or Map function using the
following approach
UnserializableWrapper usw = new UnserializableWrapper() {
public UnserializableObject create(String prm1, String prm2) {
return new UnserializableObject(prm1,prm2);
} }
If the class is merely a data structure, without significant methods, then it might be easier to unpack its fields into your RDD types (in your case, ReconEntity) and discard the class itself.
If the class has methods that you need, then your other (ugly) option is to cut-and-paste code into a new serializable class or into helper functions in your Spark code.
I am very new to GWT.
I am using ext-gwt widgets.
I found many places in my office code containing like,
class A extends BaseModel{
private UserAccountDetailsDto userAccountDetailsDto = null;
//SETTER & GETTER IN BASEMODEL WAY
}
Also, the DTO reference is unused.
public class UserAccountDetailsDto implements Serializable{
private Long userId=null;
private String userName=null;
private String userAccount=null;
private String userPermissions=null;
//NORMAL SETTER & GETTER
}
Now, I am able to get the result from GWT Server side Code and things Work fine, but when I comment the DTO reference inside the class A, I am not getting any Result.
Please explain me the need of that.
Thanks
Well the problem is in implementation of GXT BaseModel and GWT-RPC serialization.
BaseModel is based around special GXT map, RpcMap. This map has defined special serialization rules, which let's avoid RPC type explosion, but as side effect, only some simple types stored in map will be serialized. E.g. you can put any type inside the map, but if you serialize/deserialize it, only values of type Integer, String ,Double,Byte, Float and Short (and arrays of this types) will be present. So the meaning behind putting reference to the DTO inside BaseModel, is to tell GWT-RPC that this type is also have to be serialized.
Detailed explanation
Basically GWT-RPC works like this:
When you define an interface for service, GWT-RPC analyzes all the classes used in parameters/ return type, to create serializers/deserializers. If you return something like Map<Object,Object> from your service, GWT-RPC will have to create a serializer for each class which implements Map and Serializable interfaces, but also it will generate serializers for each class which implements Serializable. In the end it is quite a bad situation, because the size of your compiled js file will be much biggger. This situation is called GWT-RPC type explosion.
So, in the BaseModel, all values are stored in RpcMap. And RpcMap has custom written serializer (RpcMap_CustomFieldSerializer you can see it's code if you interested how to create such things), so it doesn't cause the problem described above. But since it has custom serializer GWT dosn't know which custom class have been put inside RpcMap, and it doesn't generate serializers for them. So when you put some field into your BaseModel class, gwt knows that it might need to be able to serialize this class, so it will generate all the required stuff for this class.
Porting GXT2 Application code using BaseModel to GXT3 Model is uphill task. It would be more or less completely rewrite on model side with ModelProviders from GXT3 providing some flexibility. Any code that relies on Model's events, store, record etc are in for a rewrite.
I noticed DbMetal generates classes that don't implement the ISerializable interface, nor are marked with DataContractAttribute. What is the easiest way of serializing such classes? Is there any DbMetal parameter that could help me?
try this:
[Serializable]
public partial class YourClassName { }
See Partial Classes for further details
GWT.create() is the reflection equivalent in GWT,
But it take only class literals, not fully qualified String for the Class name.
How do i dynamically create classes with Strings using GWT.create()?
Its not possible according to many GWT forum posts but how is it being done in frameworks like Rocket-GWT (http://code.google.com/p/rocket-gwt/wiki/Ioc) and Gwittir (http://code.google.com/p/gwittir/wiki/Introspection)
It is possible, albeit tricky. Here are the gory details:
If you only think as GWT as a straight Java to JS, it would not work. However, if you consider Generators - Special classes with your GWT compiler Compiles and Executes during compilation, it is possible. Thus, you can generate java source while even compiling.
I had this need today - Our system deals with Dynamic resources off a Service, ending into a String and a need for a class. Here is the solutuion I've came up with - btw, it works under hosted, IE and Firefox.
Create a GWT Module declaring:
A source path
A Generator (which should be kept OUTSIDE the package of the GWT Module source path)
An interface replacement (it will inject the Generated class instead of the interface)
Inside that package, create a Marker interface (i call that Constructable). The Generator will lookup for that Marker
Create a base abstract class to hold that factory. I do this in order to ease on the generated source code
Declare that module inheriting on your Application.gwt.xml
Some notes:
Key to understanding is around the concept of generators;
In order to ease, the Abstract base class came in handy.
Also, understand that there is name mandling into the generated .js source and even the generated Java source
Remember the Generator outputs java files
GWT.create needs some reference to the .class file. Your generator output might do that, as long as it is referenced somehow from your application (check Application.gwt.xml inherits your module, which also replaces an interface with the generator your Application.gwt.xml declares)
Wrap the GWT.create call inside a factory method/singleton, and also under GWT.isClient()
It is a very good idea to also wrap your code-class-loading-calls around a GWT.runAsync, as it might need to trigger a module load. This is VERY important.
I hope to post the source code soon. Cross your fingers. :)
Brian,
The problem is GWT.create doen't know how to pick up the right implementation for your abstract class
I had the similar problem with the new GWT MVP coding style
( see GWT MVP documentation )
When I called:
ClientFactory clientFactory = GWT.create(ClientFactory.class);
I was getting the same error:
Deferred binding result type 'com.test.mywebapp.client.ClientFactory' should not be abstract
All I had to do was to go add the following lines to my MyWebapp.gwt.xml file:
<!-- Use ClientFactoryImpl by default -->
<replace-with class="com.test.mywebapp.client.ClientFactoryImpl">
<when-type-is class="com.test.mywebapp.client.ClientFactory"/>
</replace-with>
Then it works like a charm
I ran into this today and figured out a solution. The questioner is essentially wanting to write a method such as:
public <T extends MyInterface> T create(Class<T> clz) {
return (T)GWT.create(clz);
}
Here MyInterface is simply a marker interface to define the range of classes I want to be able to dynamically generate. If you try to code the above, you will get an error. The trick is to define an "instantiator" such as:
public interface Instantiator {
public <T extends MyInterface> T create(Class<T> clz);
}
Now define a GWT deferred binding generator that returns an instance of the above. In the generator, query the TypeOracle to get all types of MyInterface and generate implementations for them just as you would for any other type:
e.g:
public class InstantiatorGenerator extends Generator {
public String generate(...) {
TypeOracle typeOracle = context.getTypeOracle();
JClassType myTYpe= typeOracle.findType(MyInterface.class.getName());
JClassType[] types = typeOracle.getTypes();
List<JClassType> myInterfaceTypes = Collections.createArrayList();
// Collect all my interface types.
for (JClassType type : types) {
if (type.isInterface() != null && type.isAssignableTo(myType)
&& type.equals(myType) == false) {
myInterfaceTypes.add(type);
}
for (JClassType nestedType : type.getNestedTypes()) {
if (nestedType.isInterface() != null && nestedType.isAssignableTo(myType)
&& nestedType.equals(myTYpe) == false) {
myInterfaceTypes.add(nestedType);
}
}
}
for (JClassType jClassType : myInterfaceTypes) {
MyInterfaceGenerator generator = new MyInterfaceGenerator();
generator.generate(logger, context, jClassType.getQualifiedSourceName());
}
}
// Other instantiator generation code for if () else if () .. constructs as
// explained below.
}
The MyIntefaceGenerator class is just like any other deferred binding generator. Except you call it directly within the above generator instead of via GWT.create. Once the generation of all known sub-types of MyInterface is done (when generating sub-types of MyInterface in the generator, make sure to make the classname have a unique pattern, such as MyInterface.class.getName() + "_MySpecialImpl"), simply create the Instantiator by again iterating through all known subtypes of MyInterface and creating a bunch of
if (clz.getName().equals(MySpecialDerivativeOfMyInterface)) { return (T) new MySpecialDerivativeOfMyInterface_MySpecialImpl();}
style of code. Lastly throw an exception so you can return a value in all cases.
Now where you'd call GWT.create(clz); instead do the following:
private static final Instantiator instantiator = GWT.create(Instantiator.class);
...
return instantiator.create(clz);
Also note that in your GWT module xml, you'll only define a generator for Instantiator, not for MyInterface generators:
<generate-with class="package.rebind.InstantiatorGenerator">
<when-type-assignable class="package.impl.Instantiator" />
</generate-with>
Bingo!
What exactly is the question - i am guessing you wish to pass parameters in addition to the class literal to a generator.
As you probably already know the class literal passed to GWT.create() is mostly a selector so that GWT can pick and execute a generator which in the end spits out a class. The easist way to pass a parameter to the generator is to use annotations in an interface and pass the interface.class to GWT.create(). Note of course the interface/class must extend the class literal passed into GWT.create().
class Selector{
}
#Annotation("string parameter...")
class WithParameter extends Selector{}
Selector instance = GWT.create( WithParameter.class )
Everything is possible..although may be difficult or even useless. As Jan has mentioned you should use a generator to do that. Basically you can create your interface the generator code which takes that interface and compile at creation time and gives you back the instance. An example could be:
//A marker interface
public interface Instantiable {
}
//What you will put in GWT.create
public interface ReflectionService {
public Instantiable newInstance(String className);
}
//gwt.xml, basically when GWT.create finds reflectionservice, use reflection generator
<generate-with class="...ReflectionGenerator" >
<when-type-assignable class="...ReflectionService" />
</generate-with>
//In not a client package
public class ReflectionGenerator extends Generator{
...
}
//A class you may instantiate
public class foo implements Instantiable{
}
//And in this way
ReflectionService service = GWT.create(ReflectionService.class);
service.newInstance("foo");
All you need to know is how to do the generator. I may tell you that at the end what you do in the generator is to create Java code in this fashion:
if ("clase1".equals(className)) return new clase1();
else if ("clase2".equals(className)) return new clase2();
...
At the final I thought, common I can do that by hand in a kind of InstanceFactory...
Best Regards
I was able to do what I think you're trying to do which is load a class and bind it to an event dynamically; I used a Generator to dynamically link the class to the event. I don't recommend it but here's an example if it helps:
http://francisshanahan.com/index.php/2010/a-simple-gwt-generator-example/
Not having looked through the code of rocket/gwittir (which you ought to do if you want to find out how they did it, it is opensource after all), i can only guess that they employ deferred binding in such a way that during compile time, they work out all calls to reflection, and statically generate all the code required to implement those call. So during run-time, you cant do different ones.
What you're trying to do is not possible in GWT.
While GWT does a good job of emulating Java at compile time the runtime is of course completely different. Most reflection is unsupported and it is not possible to generate or dynamically load classes at runtime.
I had a brief look into code for Gwittir and I think they are doing their "reflection stuff" at compile time. Here: http://code.google.com/p/gwittir/source/browse/trunk/gwittir-core/src/main/java/com/totsp/gwittir/rebind/beans/IntrospectorGenerator.java
You might be able to avoid the whole issue by doing it on the server side. Say with a service
witch takes String and returns some sort of a serializable super type.
On the server side you can do
return (MySerializableType)Class.forName("className").newInstance();
Depending on your circumstances it might not be a big performance bottleneck.