As per apache beam documentation, I can find data type specific coders and also custom coders. It provides feasibility to create custom coders by registering with code registry. But I would like to know if there is any coder available for POJO/bean. For example,what is the coder for PCollection
If your POJO is defined within your project, then take a look at DefaultSchema. The example there does exactly what you want, registering a schema (which implicitly registers a coder) by inspecting JavaBean-compliant methods:
#DefaultSchema(JavaBeanSchema.class)
class MyClass {
public String getFoo();
void setFoo(String foo);
....
}
Note that coders are for elements of a collection, so there is no coder for PCollection. Rather, a PCollection can have a coder set, determining how individual elements are serialized and deserialized.
Related
I am trying to create a dummy PCollection with my custom objects as follows:
PCollection<MyClass> pipelineProcessingResults = pipeline.apply(Create.of(new MyClass(.., ..)));
MyClass class is as follows:
#DefaultSchema(JavaBeanSchema.class)
public class MyClass {
AnotherComplexClass _obj;
Urn _urn
}
I am getting the following exception:
java.lang.StackOverflowError
at java.util.HashMap.hash(HashMap.java:339)
at java.util.HashMap.put(HashMap.java:612)
at java.util.HashSet.add(HashSet.java:220)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.TypeVisitor.visit(TypeVisitor.java:66)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.Types.getComponentType(Types.java:197)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.TypeToken.getComponentType(TypeToken.java:563)
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.reflect.TypeToken.isArray(TypeToken.java:512)
at org.apache.beam.sdk.values.TypeDescriptor.isArray(TypeDescriptor.java:193)
at org.apache.beam.sdk.schemas.utils.ReflectUtils.getIterableComponentType(ReflectUtils.java:196)
at org.apache.beam.sdk.schemas.FieldValueTypeInformation.getIterableComponentType(FieldValueTypeInformation.java:274)
at org.apache.beam.sdk.schemas.FieldValueTypeInformation.forGetter(FieldValueTypeInformation.java:189)
at org.apache.beam.sdk.schemas.JavaBeanSchema$GetterTypeSupplier.get(JavaBeanSchema.java:74)
at org.apache.beam.sdk.schemas.utils.StaticSchemaInference.schemaFromClass(StaticSchemaInference.java:92)
at org.apache.beam.sdk.schemas.utils.StaticSchemaInference.fieldFromType(StaticSchemaInference.java:166)
The class AnotherComplexClass may contain multiple fields which in turn are composed of other classes.
Which coder will best suit my purpose? Should I create a custom coder? Using the #DefaultSchema annotation did not help me much. I tried using SerializableCoder, but it throws a compiler error:
Cannot resolve method 'of(java.lang.Class<MyClass>)'
Option 1 - Custom Coder
Since you have complex nested data types, you can define a custom coder and use it with the #DefaultCoder annotator. Details see https://beam.apache.org/documentation/programming-guide/#annotating-custom-type-default-coder.
public class MyCoder implements Coder {
public static Coder<T> of (Class<T> clazz) {...}
}
#DefaultCoder(MyCoder.class)
public class MyClass {...}
Option 2 - Serializable
You can also make sure that all your POJO classes implement Serializable and by default, Java SDK uses the SerializableCoder. But it's inefficient and non-deterministic.
Option 3 - Avro
You can use AvroCoder and use Avro to generate your classes. https://avro.apache.org/docs/current/gettingstartedjava.html
Option 4 - Protocol Buffer
Similar to Avro, you can use Protocol Buffer to define your schema and classes. https://developers.google.com/protocol-buffers/docs/javatutorial.
I use Enterprise Architect for code generation and I would like to automatically retrieve all tags (in my case Java annotations) of the interfaces that a class realizes. Consider the following example:
From this model, I want to generate a class that looks like this:
#AnnotationOfMyInterface
public class MyClass {
...
}
So I want to add annotations as tags to MyInterface that should be applied to MyClass during code generation. In the UI, tags of implemented interfaces are shown so I was hoping there is a way to get these tags during code generation.
I tried to edit the code generation templates and found macros to get
All interfaces that a class implements: %list="ClassInterface" #separator=", "%
All tags with a given name (of the class that code is being generated for): %classTag:"annotations"%
But unfortunately, I cannot combine these macros, i.e., I cannot pass one interface to the classTag macro so that I can retrieve the tags of that particular interface (and not the one I'm generating code for). Is there a way to get classTags of a specific class / interface?
I also tried to create a separate code generation template and "call" it from the main class code generation template. But inside my template, the classTag macro still only gets the tags of the class.
Thanks to the comments above and especially because of an answer to my question in EA's forum, I was able to setup a little proof of concept achieving what I wanted. I'm answering my question to document my solution in case someone has a similar problem in the future.
After Eve's hint in EA's forum I looked into creating an AddIn for Enterprise Architect to use this AddIn from a code generation template. I started by writing a basic AddIn as explained by #Geert Bellekens in this tutorial. Afterwards I changed the AddIn to fit my needs. This is how I finally got the tagged values (annotations) of the interfaces a class realizes:
First step:
Inside a code generation template, I get all the interfaces a class realizes and pass them to my AddIn:
$interfaces=%list="ClassInterface" #separator=", "%
%EXEC_ADD_IN("MyAddin","getInterfaceTags", $interfaces)%
Second step:
As documented here the repository objects gets passed along with the EXEC_ADD_IN call. I use the repository object and query for all interfaces using the names contained in $interfaces. I can then get the tagged values of each interface element. Simple prototype that achieves this for a single interface:
public Object getInterfaceTags(EA.Repository repo, Object args)
{
String[] interfaceNames = args as String[];
String firstInterfaceName = interfaceNames[0];
EA.Element interfaceElement = repo.GetElementsByQuery("Simple", firstInterfaceName).GetAt(0);
String tag = interfaceElement.TaggedValues.GetAt(0);
return interfaceElement.Name + " has tag value" + tag.Value;
}
I know, there are a couple of shortcomings but this is just a simple proof of concept for an idea that will most likely never be production code.
I'm writing a Kotlin library for an app. This library gathers information like events, errors, device info to analyze later.
I used Objects instead of Classes for ease of use. Example object from library:
internal object LoggingModule { ...lots of methods to use internally... }
But i wonder if it's better to use Classes instead of Objects in libraries? Because there is a public method to stop info gathering on library initializing and since there is this feature, also there is no need to create classes on app initializing (meaning Objects) if this feature selected. But i don't think there will be a huge impact on app either.
init example:
class App: Application() {
override fun onCreate() {
super.onCreate()
MyLibrary.setThings(myParameters).setEnabled(true).init()
}
}
What should i do? Are Objects not a good idea in this scenario?
Objects are singleton and static in Kotlin.
The answer to the question has to do with shared state. Singletons should be stateless, with nothing shared.
I am suspicious of dogmatic ideas like "I only use X; I never use Y."
Better to understand the differences and apply them appropriately on a case-by-base basis.
I was doing some research into PolymerDart and the various annotations which can be applied to the dart files. Be it: #Property, #property, #observe, #reflectable, #PolymerRegister, OR #HtmlImport.
So, I started to look into the concept of Dart and how to make annotations for them. I saw on stackoverflow that you can do something like this.
class Datatable {
final String name;
const DataTable(this.name);
}
which can easily do some additional information inside the constructor optionally.
class Datatable {
final String name;
const DataTable(this.name) {
console.log("$name was referenced.");
}
}
So, I can create and implement a variety of Annotations we could leverage, but this is where it starts to get fishy.
I was curious if there was a way to create annotations for polymerdart? is that mostly locked down, or can is there a way to create ones which do simple functions, maybe even for example: creating an Annotation which executes the #Property(computed:"") functionality.
I was wanted to create some sort of customization for our team to use.
For the record, I know that i can do something like
const myCustomAnnotation = const Property();
which would allow me to do:
#myCustomAnnotation
I was thinking I could then do something like like:
class myCustomComputed extends Property {
final String functionName;
const myCustomComputed() : Property(computed: this.functionName);
}
to allow me to do something like:
#myCustomComputed("testFunction(varA)")
This is a big topic, but the brief answer is, yes, what you describe is technically possible but not trivial.
Dart annotations are available at runtime via reflection, but are most often used by pub transformers during the build process. The purpose of a transformer is to modify assets (e.g. dart code) to some new form before runtime.
The Polymer annotations you mentioned above are handled by the Polymer transformer. This transformer handles identifying the annotations you mentioned above and automatically rewriting annotated code to include all the necessary implementations and wiring such that everything behaves as we expect as Polymer elements.
So there's nothing stopping you from defining your own annotations and transformers, including those that build upon the existing Polymer transformers, and packaging it up for your own or others' use.
I will note though that it is somewhat complex topic (ref the Polymer transformer) and there seem to be few simple code-rewriting transformer examples from which to build on.
In Guvnor documentation, I know how to define data enumeration and use it in Guvnor. Is it possible to fetch data enumeration from my own Java code?
From Guvnor's documentation:
Loading enums programmatically: In some cases, people may want to load their enumeration data entirely from external data source (such as a relational database). To do this, you can implement a class that returns a Map. The key of the map is a string (which is the Fact.field name as shown above), and the value is a java.util.List of Strings.
public class SampleDataSource2 {
public Map<String>, List<String> loadData() {
Map data = new HashMap();
List d = new ArrayList();
d.add("value1");
d.add("value2");
data.put("Fact.field", d);
return data;
}
}
And in the enumeration in the BRMS, you put:
=(new SampleDataSource2()).loadData()
The "=" tells it to load the data by executing your code.
Best Regards,
I hope its not too late to answer this.
To load enum from application to guvnor,
Build the enum class dynamically from string (in my case enum values is provided by user via GUI)
Add it to a jar, convert it to byte array
POST it to guvnor as asset (model jar) via REST call
Call save repository operation (change in source code of guvnor)
Now enum will be visible as fact in your rule window.
Editing/Deletion of model jar and validation of rules aftermath is something you have to take care.