ASN.1 compiler with support for Extended XER/RXER - encoding

I am desperately searching for an ASN.1 compiler that will successfully parse a predefined ASN.1 definition I got from a customer.
None of the products (free or commercial) so far was able to parse the definition, which is XER based and has some transient RXER dependency - when I specify the relevant ASN.1-definitions manually (taken from the RFCs), the compiler also emits parser errors.
The relevant code line is (simplified):
MYMSG ::= SEQUENCE
{
msgID [ATTRIBUTE] [250] UTF8String OPTIONAL,
msgType UTF8String
}
Every compiler complains about the [ATTRIBUTE] token, so I found this is part of the ASN.X specification defined in RFC 4912 and also depends on support for RXER in RFC 4910, see also X.680-1.
Problem is every compiler I tried does not seem to support these encoding schemas out of the box and fails to parse the definitions mentioned in the RFCs, e.g. for RXER and all definitions that use this encoding I get:
"RXER.asn", line 20
(AdditionalBasicDefinitions): A1139W:
The default encoding reference 'RXER'
for the module
'AdditionalBasicDefinitions' is
unknown and all encoding instructions
within the module identified by this
encoding reference will be ignored.
RXER INSTRUCTIONS
(Note: all dependent modules like ASN.X include the instruction "RXER INSTRUCTIONS" immediately after the "DEFINITIONS"-tag, which is not understood by any compiler I tried).
I tried openasn1 (www.openasn1.org) - funny thing is that I have some old and partially functional Java mapping objects in the code I got that was generated by openasn1! - the online compiler at http://lionet.info/asn1c/asn1c.cgi and various commercial tools like Objective Systems ASN1C v6.4.1 at http://www.obj-sys.com/Cnge641Dwld/acv64kits.php (they even have a current Eclipse plugin), Marben http://www.marben-products.com/asn.1/tce_java.html and unigone http://www.unigone.com/en/products/Asn1Compiler/description
I always get an error similar to this:
ASN.1 grammar parse error near line 13 (token "ATTRIBUTE"): parse error, unexpected TOK_capitalreference, expecting TOK_number
Am I missing something obvious like IMPORTs or other definitions/compiler flags?

I managed to compile the schema and generate the needed Java mapping classes. I had to use the commercial OSS Nokalva compiler, as all free tools I tried failed on (E)XER encoded schemata.
There were also some errors in the schema I received, so here is what I had to do:
first I added the XER-instructions in the DEFINITIONS-line along with the usual Tags
directive:
DEFINITIONS XER INSTRUCTIONS AUTOMATIC TAGS ::=
in the footer, I added the XER encoding directive:
ENCODING-CONTROL XER GLOBAL-DEFAULTS
MODIFIED-ENCODINGS
when using XER encoding, you have to explicitly specify tags to avoid ambiguities in syntax:
[ATTRIBUTE] [TAG: 0]
instead of the ambiguous definition
[ATTRIBUTE] [0]
All these problems were resolved by consulting this nice writeup on EXER-encoding, definitely a good read and thanks to Nokalva for this helpful documentation!

Related

What is #_currentArena? (Syntax)

I found this code in arena.dart in the ffi examples of flutter:
/// The last [Arena] in the zone.
factory Arena.current() {
return Zone.current[#_currentArena];
}
I am puzzled and could not find anything about the #_currentArena thing. What type of language construct is that, and how does it work? Trying to name anything (else) starting with # gives immediate errors; auto completion doesn't find it, and trying to go to its definition doesn't work either.
So this seems to be something extremely special, and extremely undocumented... leaving me extremely curious!
Link to source file in Flutter SDK Sqlite Example
As #pskink pointed out in his comment, this is part of the symbols.
A Symbol object represents an operator or identifier declared in a
Dart program. You might never need to use symbols, but they’re
invaluable for APIs that refer to identifiers by name, because
minification changes identifier names but not identifier symbols.
To get the symbol for an identifier, use a symbol literal, which is
just # followed by the identifier:
#radix
#bar
Symbol literals are compile-time constants.

Mixing vocabularies, Google's structured data testing tool and Schema.org extensions

We are using several vocabularies along with schema.org and struggle with the structured data testing tool from Google. Is is even possible to completely pacify it when mixing vocabularies?
Some of the classes and properties we use are specializations of classes and properties of schema.org.
I have read the page about the extension mechanism. It is completely unclear to me what external extensions actually are. It is completely unclear to me if and how it is possible to communicate to Google that a class/property is a specialization of a schema.org class/property (so that Google uses RDFS reasoning to get statements involving the schema.org namespace).
The example I am using is http://www.netestate.de/imgtag_schema_example/lio.html
The RDFa in that page describes the image shown. The <img> tag in the source has a typeof attribute.
If I use typeof="lio:Image", I get 1 error about lio:Image not being known to Google. Makes sense. Validation URL: http&colon;//www.netestate.de/imgtag_schema_example/lio.html
If I use typeof="lio:Image schema:ImageObject", I get exactly the same error. Validation URL: http&colon;//www.netestate.de/imgtag_schema_example/lioschema.html
If I use typeof="schema:ImageObject", I get 19 errors about properties not recognized as compatible with ImageObject. Validation URL: http&colon;//www.netestate.de/imgtag_schema_example/schema.html
If I use typeof="schema:ImageObject lio:Image", I get 1 error about a class that is not known to Google (the class is not named but "ImageObject" is red!). Validation URL: http&colon;//www.netestate.de/imgtag_schema_example/schemalio.html
If I use typeof="lio:Image" and add the statement lio:Image rdfs:subClassOf schema:ImageObject to the RDFa, the validator separates the triples about http&colon;//purl.org/net/lio#Image ("class not defined, no errors") and the image (unknown class #__sid=rd0, 1 error). Validation URL: http&colon;//www.netestate.de/imgtag_schema_example/liosubclass.html
Where does the relative URI #__sid=rd0 come from?
Why is the error about #__sid=rd0 missing in this simpler example?
http://www.netestate.de/imgtag_schema_example/minimal.html
Don't let any Google Structured Data Testing Tool complaints about unknown vocabulary bother you. Its main purpose it to help publishers understand when they are using structures which Google products/features expect and use. Generally it will only understand the schema.org parts (and won't exploit subtypes to other vocabularies). You might find using the additionalType property helps make some errors go away. The __sid=rd0 ID is just a generated URI for what RDF would consider a 'blank node' in the graph.

using protobuf as a textual configuraton file

I recently encountered a very large mission-critical project where all the configuration
files were defined using textual protobuf definitions. The configuration files are meant to be
human readable and editable.
For example
message ServerSettings {
required int32 port = 3022;
optional string name = "mywebserver";
}
Personally I found this humorous.
But is it in fact a reasonable keep-it-simple technique, or clearly moronic ?!
In other words, are there REAL, ACTUAL problems with this ?
If that is the text proto if format, then... Whatever, I guess. If it works, then it is as reasonable as any other serialization format.
If that is meant to be proto schema, then it is illegal (the value after the = is meant to be the field number).
Json or XML might be more typical, but as long as it works it isn't "moronic". So the ultimate question is: does it work?
I think it's quite clever. I am guessing they pass it through protoc --encode to generate a binary which is what is actually parsed.
Pros:
1. Code is generated to parse configuration
2. Type validation
3. More robust configuration file compared to a key/value as it supports structs, unions, maps and arrays
4. The configuration data is now serializable meaning it can be easily exposed to an RPC or IPC interface.
Cons:
1. The syntax can be a little verbose for maps/arrays.
2. It requires protoc to be installed on the target as well as libprotobuf.so if you are on a system with tight memory limits.

How to internationalize java source code?

EDIT: I completely re-wrote the question since it seems like I was not clear enough in my first two versions. Thanks for the suggestions so far.
I would like to internationalize the source code for a tutorial project (please notice, not the runtime application). Here is an example (in Java):
/** A comment */
public String doSomething() {
System.out.println("Something was done successfully");
}
in English , and then have the French version be something like:
/** Un commentaire */
public String faitQuelqueChose() {
System.out.println("Quelque chose a été fait avec succès.");
}
and so on. And then have something like a properties file somewhere to edit these translations with usual tools, such as:
com.foo.class.comment1=A comment
com.foo.class.method1=doSomething
com.foo.class.string1=Something was done successfully
and for other languages:
com.foo.class.comment1=Un commentaire
com.foo.class.method1=faitQuelqueChose
com.foo.class.string1=Quelque chose a été fait avec succès.
I am trying to find the easiest, most efficient and unobtrusive way to do this with the least amount of manual grunt work (other than obviously translating the actual text). Preferably working under Eclipse. For example, the original code would be written in English, then externalized (to properties, preferably leaving the original source untouched), translated (humanly) and then re-generated (as a separate source file / project).
Some trails I have found (other than what AlexS suggested):
AntLR, a language parser / generator. There seems to be a supporting Eclipse plugin
Using Eclipse's AST (Abstract Syntax Tree) and I guess building some kind of plugin.
I am just surprised there isn't a tool out there that does this already.
I'd use unique strings as methodnames (or anything you want to be replaced by localized versions.
public String m37hod_1() {
System.out.println(m355a6e_1);
}
then I'd define a propertyfile for each language like this:
m37hod_1=doSomething
m355a6e_1="Something was done successfully"
And then I'd write a small program parsing the sourcefiles and replacing the strings. So everything just outside eclipse.
Or I'd use the ant task Replace and propertyfiles as well, instead of a standalone translation program.
Something like that:
<replace
file="${src}/*.*"
value="defaultvalue"
propertyFile="${language}.properties">
<replacefilter
token="m37hod_1"
property="m37hod_1"/>
<replacefilter
token="m355a6e_1"
property="m355a6e_1"/>
</replace>
Using one of these methods you won't have to explain anything about localization in your tutorials (except you want to), but can concentrate on your real topic.
What you want is a massive code change engine.
ANTLR won't do the trick; ASTs are necessary but not sufficient. See my essay on Life After Parsing. Eclipse's "AST" may be better, if the Eclipse package provides some support for name and type resolution; otherwise you'll never be able to figure out how to replace each "doSomething" (might be overloaded or local), unless you are willing to replace them all identically (and you likely can't do that, because some symbols refer to Java library elements).
Our DMS Software Reengineering Toolkit could be used to accomplish your task. DMS can parse Java to ASTs (including comment capture), traverse the ASTs in arbitrary ways, analyze/change ASTs, and the export modified ASTs as valid source code (including the comments).
Basically you want to enumerate all comments, strings, and declarations of identifiers, export them to an external "database" to be mapped (manually? by Google Translate?) to an equivalent. In each case you want to note not only the item of interest, but its precise location (source file, line, even column) because items that are spelled identically in the original text may need different spellings in the modified text.
Enumeration of strings is pretty easy if you have the AST; simply crawl the tree and look for tree nodes containing string literals. (ANTLR and Eclipse can surely do this, too).
Enumeration of comments is also straightforward if the parser you have captures comments. DMS does. I'm not quite sure if ANTLR's Java grammar does, or the Eclipse AST engine; I suspect they are both capable.
Enumeration of declarations (classes, methods, fields, locals) is relatively straightforward; there's rather more cases to worry about (e.g., anonymous classes containing extensions to base classes). You can code a procedure to walk the AST and match the tree structures, but here's the place that DMS starts to make a difference: you can write surface-syntax patterns that look like the source code you want to match. For instance:
pattern local_for_loop_index(i: IDENTIFIER, t: type, e: expression, e2: expression, e3:expression): for_loop_header
= "for (\t \i = \e,\e2,\e3)"
will match declarations of local for loop variables, and return subtrees for the IDENTIFIER, the type, and the various expressions; you'd want to capture just the identifier (and its location, easily done by taking if from the source position information that DMS stamps on every tree node). You'd probably need 10-20 such patterns to cover the cases of all the different kinds of identifiers.
Capture step completed, something needs to translate all the captured entities to your target language. I'll leave that to you; what's left is to put the translated entities back.
The key to this is the precise source location. A line number isn't good enough in practice; you may have several translated entities in the same line, in the worst case, some with different scopes (imagine nested for loops for example). The replacement process for comments, strings and the declarations are straightforward; rescan the tree for nodes that match any of the identified locations, and replace the entity found there with its translation. (You can do this with DMS and ANTLR. I think Eclipse ADT requires you generate a "patch" but I guess that would work.).
The fun part comes in replacing the identifier uses. For this, you need to know two things:
for any use of an identifier, what is the declaration is uses; if you know this, you can replace it with the new name for the declaration; DMS provides full name and type resolution as well as a usage list, making this pretty easy, and
Do renamed identifiers shadow one another in scopes differently than the originals? This is harder to do in general. However, for the Java language, we have a "shadowing" check, so you can at least decide after renaming that you have an issues. (There's even a renaming procedure that can be used to resolve such shadowing conflicts
After patching the trees, you simply rewrite the patched tree back out as a source file using DMS's built-in prettyprinter. I think Eclipse AST can write out its tree plus patches. I'm not sure ANTLR provides any facilities for regenerating source code from ASTs, although somebody may have coded one for the Java grammar. This is harder to do than it sounds, because of all the picky detail. YMMV.
Given your goal, I'm a little surprised that you don't want a sourcefile "foo.java" containing "class foo { ... }" to get renamed to .java. This would require not only writing the transformed tree to the translated file name (pretty easy) but perhaps even reconstructing the directory tree (DMS provides facilities for doing directory construction and file copies, too).
If you want to do this for many languages, you'd need to run the process once per language. If you wanted to do this just for strings (the classic internationalization case), you'd replace each string (that needs changing, not all of them do) by a call on a resource access with a unique resource id; a runtime table would hold the various strings.
One approach would be to finish the code in one language, then translate to others.
You could use Eclipse to help you.
Copy the finished code to language-specific projects.
Then:
Identifiers: In the Outline view (Window>Show View>Outline), select each item and Refactor>Rename (Alt+Shift+R). This takes care of renaming the identifier wherever it's used.
Comments: Use Search>File to find all instances of "/*" or "//". Click on each and modify.
Strings:
Use Source>Externalize strings to find all of the literal strings.
Search>File for "Messages.getString()".
Click on each result and modify.
On each file, ''Edit>Find/Replace'', replacing "//\$NON-NLS-.*\$" with empty string.
for the printed/logged string, java possess some internatization functionnalities, aka ResourceBundle. There is a tutorial about this on oracle site
Eclipse also possess a funtionnality for this ("Externalize String", as i recall).
for the function name, i don't think there anything out, since this will require you to maintain the code source on many version...
regards
Use .properties file, like:
Locale locale = new Locale(language, country);
ResourceBundle captions= ResourceBundle.getBundle("Messages",locale);
This way, Java picks the Messages.properties file according to the current local (which is acquired from the operating system or Java locale settings)
The file should be on the classpath, called Messages.properties (the default one), or Messages_de.properties for German, etc.
See this for a complete tutorial:
http://docs.oracle.com/javase/tutorial/i18n/intro/steps.html
As far as the source code goes, I'd strongly recommend staying with English. Method names like getUnternehmen() are worse to the average developer then plain English ones.
If you need to familiarize foreign developers to your code, write a proper developer documentation in their language.
If you'd like to have Javadoc in both English and other languages, see this SO thread.
You could write your code using freemarker templates (or another templating language such as velocity).
doSomething.tml
/** ${lang['doSomething.comment']} */
public String ${lang['doSomething.methodName']}() {
System.out.println("${lang['doSomething.message']}");
}
lang_en.prop
doSomething.comment=A comment
doSomething.methodName=doSomething
doSomething.message=Something was done successfully
And then merge the template with each language prop file during your build (using Ant / Gradle / Maven etc.)

Zend_Json_Server vs JSON-RPC spec

The ZF implementation of the JSON-RPC 2.0 protocol only allows error codes:
const ERROR_PARSE = -32768;
const ERROR_INVALID_REQUEST = -32600;
const ERROR_INVALID_METHOD = -32601;
const ERROR_INVALID_PARAMS = -32602;
const ERROR_INTERNAL = -32603;
const ERROR_OTHER = -32000;
plus, range(-32099, -32000)
These are defined in the JSON-RPC spec as pre-defined and/or reserved. At least this is what get out of the spec:
The error codes from and including -32768 to -32000 are reserved for pre-defined errors. Any code within this range, but not defined explicitly below is reserved for future use. The error codes are nearly the same as those suggested for XML-RPC at the following url: http://xmlrpc-epi.sourceforge.net/specs/rfc.fault_codes.php
code message meaning
-32700 Parse error Invalid JSON was received by the server.
An error occurred on the server while parsing the JSON text.
-32600 Invalid Request The JSON sent is not a valid Request object.
-32601 Method not found The method does not exist / is not available.
-32602 Invalid params Invalid method parameter(s).
-32603 Internal error Internal JSON-RPC error.
-32000 to -32099 Server error Reserved for implementation-defined server-errors.
The remainder of the space is available for application defined errors.
Nowhere does it say that you cannot, for example use -100 or 100. Am I missing something?
Somewhere I think that "server error" and "application error" have been confused by ZF as the same thing, while when reading the sourcefourge link above, clearly the authors of the protocol had something different in mind, allowing the application developers A LOT of space:
In addition, the range -32099 .. -32000, inclusive is reserved for implementation defined server errors. Server errors which do not cleanly map to a specific error defined by this spec should be assigned to a number in this range. This leaves the remainder of the space available for application defined errors.
I use the JSON-RPC component of ZF for a few projects. It works pretty well - but I would hardly consider it an exemplar of the JSON-RPC spec. As far as I know, only a couple clients out there actually test their implementations against Zend_Json_Server, so it's hardly a widely adopted implementation. At one point, I actually had to patch Zend_Json_Server to make it work with one client, as it was not following the spec properly (that's since been fixed).
So basically what I'm saying is "good point, you're probably right." If it itches enough, just fork zf2 and submit a pull request with a better implementation of the spec - much easier to get positive / negative feedback when you're looking at a diff.
If they accept it, submit a patch for zf1 to merge in downstream.