I am developing a general purpose CRUD code generator application. The idea is that codes/files (model, controller, view) for common insert, update, list, delete etc. operations will be automatically generated from model definition (like the definition used in Grails). But the generated code can be for any framework, e.g. Play (Scala or Java version) or Django or Grails or whatever framework user wants to use it for, even AngularJs. That is, same model definition can be used for generating code for any framework.
My question is, what can I use for this task - Scala or Groovy or some DSL specialized tools like Xtext?
This seems like a good case for a DSL. A DSL can be summed up as the following 3 elements:
Abstract Syntax: the concepts of your DSL. Here you want to specify CRUD applications.
Concrete Syntax(es): a way to materialize your Abstract Syntax. As a programmer, the first thought is often text-based syntaxes, but you could also use a graphical or tree-like syntax, or even simply a GUI with text fields and checkboxes.
Semantics: the meanings of your DSL. Here you want to generate code.
I'll now suggest some solutions which are based on Java and come from the Eclipse Modeling ecosystem.
Eclipse EMF implements standards for the definition of so-called "metamodels" (basically abstract syntaxes). In the Eclipse world, EMF is the base for a lot of tooling.
Assuming you have an EMF metamodel, textual syntaxes can be specified using Eclipse Xtext, and graphical syntaxes using Eclipse Sirius. Note that you can also develop your own GUI in Java and create your model using the EMF Java APIs. Also note that Xtext can create your metamodel for you based on the grammar you want for your text-based syntax. This is nice if you don't want to dive too deep into EMF it self (thus steps 1 and 2 are one and the same).
Eclipse Acceleo provides a template language specifically designed to generate text, including code. Once again, you can also write your code generator using plain Java, or any JVM-based language thanks to the EMF Java APIs. If you use Xtext, there is also a facility for including an Xtend-based code generator alonside your syntax.
Related
I want to ask the advantages mps and xtext have over each other and the main features when writing a language. I know when working with mps you are directly editing the AST and xtext uses a parser. I have read an advantage of using a AST allows for multiple languages to be extended for the language you are making, I don’t really understand what this means, could this be explained further and why would someone want to extend multiple language ?
Also i have read that the AST cut out ambiguous code, how does it do this?
I know that both MPS and xtext have features like underlining and highlighting code is their any other feature relating to code validation ?
Any other main differences and general feature of them are welcome ?
I have no practical experience with Xtext, so I will talk mostly about MPS.
LWB
Both Xtext and MPS are language workbenches, so they have their own schema used to metamodel the abstract syntax (structure of concepts), some way to define the concrete syntax (notations) and some way to define generators (M2M or M2T transformations) or less usually interpreters. Then they have provide the IDE itself with highlighting, smart actions like refactoring and contextual error fixes, advanced search and navigation (go to declaration etc.), checking for errors (type errors, static code analysis, checking of defined constraints & rules, checking cardinality, dataflow analysis), ... So yes, lots of options for validation. I have mentioned things, that are in MPS, not sure if Xtext provides everything. However, all of these features are organised in so-called aspects which you can check out in a summary table which shortly describes each aspect.
Projectional editor
As you have mentioned, MPS uses a projectional editor. You directly manipulate the AST, parser-based post-IntelliJ smart IDEs are able to provide you with intelligent actions like recaftoring and go to declaration etc. only because they parse the language in memory and construct an AST behind the scenes anyway. Projectional editors skip the parsing step.
Dodging ambiguity
It uses no parser at all, so all of the downsides of having a parser are gone. First of all, the language developer does not need to be an expert in syntax analysis, so you don't need to hire them specifically. But the best win is to have infinite language composability. This is achieved, as you have mentioned, by totally avoiding ambiguities which could appear in grammars (MPS does not use a grammar, but a model). Let's say you use language A and language B. For demonstration, let's say both languages extend BaseLanguage (abbr. BL, the MPS-equivalent of Java) and they have both defined a statement to log. Concept a logs to stderr and b logs to a file. However, both a and b have an identical concrete syntax (i.e. editor definition in MPS) which just says log. Now if you had a parser and it encounters the token log it cannot decide from which language the concept is, so it's ambiguous - not even a look-ahead parser can do it. In a projectional editor this cannot happen, because only the projection is identical and under the hood the AST has an instance of either a or b (you can think of it as always using the whole FQN of a class in Java, just the package is hidden in the IDE, so you can use identically named classes from different packages). The "ambiguity" is resolved at the time of writing by the user: when he writes log a dropdown menu appears clearly showing that one of them is a and the other is b (maybe even shows a description which would say "Log to file" / "Log to stderr").
Modularity
Consequently, MPS has very good modularity, composability and extensibility of languages. You have mentioned
allows for multiple languages to be extended for the language you are making [...]
why would someone want to extend multiple language
You need to differentiate between using a language and extending it
(if you are interested more Völter talks about 4 kinds of composition techniques regarding languages: referencing, extension, reuse and embedding). Using a lanugage is just the ability to write programs in it. If you extend a language, it's kind of like inheritance, you add new concepts to it, f.e. create a new type of Java (BL) statement. And it has been done in the standard languages shipped with MPS too. You have for example the checkedDots language which extends BL with an operation .? which is null-safe (similarly to null-conditional operator ?. in C#). So why extend a language? Because you can use new constructs, add new functionality or syntactic sugar. Another ready-to-use language in BL is the tuples language, which has both indexed and named tuples. Then there is the collections language, which kind-of replaces the Java Stream API. All of these little languages are extensions which you can start using with a simple Ctrl+L. You could also embed another language to your language - use a regex inside an SQL statement inside your Java code.
Generation
Another kind of language dependency in MPS is to have a "generation target" language. Generators in MPS work in a way that you transform your language sentence (i.e. model) into another MPS language. You can invent your own little language, or implement LOLcode and setup the generator to transform it into valid Java code. However, this language must already exist in MPS, so you cannot generate it to Python, if there is no Python implementation in MPS. The other alternative is to generate text (M2T), this way you could theoretically generate Python source code, or just print the LOLcode as-is.
Multiple notations
The second great difference in projectional vs. parser-based editors is that the latter inherently supports only textual notation. Maybe there are some external tools you can use. On the other side, MPS provides textual, tabular, symbolic (math symbols) and graphical (diagrams) notations. There is a possibility to swap your view from one notation to another, per concept or for the whole "file" (program).
Drawbacks
It's not all roses though. Projectional editors have some limitations, or challenges to solve. There is an analysis of challenges in projectional editors which points out mainly usability and infrastructure integration. They are mostly solved in MPS, f.e. regarding infrastructure you have a good VCS diff/merge tool. For automatic/cmd builds there is a language that generates Ant. Gradle or Maven does not work with MPS directly, but through Ant. Regarding usability "MPS takes a
while to get used to, but then its usability is comparable to ParEs."3 You should use a language called GrammarCells (available through MPS-extensions or mbeddr.platform) which makes it easy to build good editors (mainly for arithemtic expressions), otherwise by default you must enter concepts in prefix order (+ first, not the number). Comments in MPS cannot be placed willy-nilly. Cannot establish references to non-existing nodes... (see the Table 1. in 3)
MPS currently does not have a web-based version. There are some planned, though. Jetbrains works on WebMPS, then there is modelix.
Portability
Generally, you are stuck to working in MPS. By default it is not really portable, unless you explicitly define generators which produce portable output. If you want to input a program , you can code a paste-handler where you could put your parser, or you can change the format in which the AST is stored (from XML to maybe directly your language, but this would again require a parser to read). I am currently working on a solution which enables to import an MPS language from a YAJCo model (model-based parser generator, where the input is not a grammar, but Java classes representing the semantic model). Then you can import a sentence (file) which creates and populates a model (AST). From the program in MPS you can generate Java source code which fills the original Java classes if you need it.
BTW the mbeddr project has implemented importing from ECore check here
Dictionary
M2M = model to model
M2T = model to text
I want to generate code from my state machine in Magic Draw. Magic Draw supports code generation for classes but does not include an option for state machine. I tried using SinelaboreRT software. However, it generates limited code. We need to manually add 'Main' function and other functions defined inside states. I wanted to know if it is possible to generate an executable or a C/C++/Java code file with all the code mentioned inside states as well a 'Main' function?
Yes, there are three main options that I am aware of 1) make your own code generator, 2) buy a commercial code generator e.g. LieberLieber provides what appears to be a fairly sophisticated one, or 3) use one of the open source code generators such as Papyrus-RT.
The first option isn't actually that difficult depending on your target language and framework. For my work at MITRE, I have written a generator to take properly formed Magicdraw models and create deploy-able Spring microservices. I used the Spring state machine library to simplify the STM code generation.
I personally found most convenient way to create state machine from models is UML or any other DSL is the combination of
Eclipse Papyrus / Eclipse XText / Eclipse XTend
there is also new kind in the block, if you don't want to work Eclipse based
Langium
but they are suffering at the moment little bit from being new but I will advice you check them every 6 months, they seem promising.
If you want to see how is it done, I have several blogs about it.
UML Based:
Akka Finite State Machine Generation Blog2
Papyrus and Spring State Machine
DSL Based:
XText and Spring State Machine
I have several internal logic dependencies in my source code. For example
Class A accepts an object and that object to be valid in Class A needs to have particular interfaces such as InterfaceOne, InterfaceTwo
I would like a way to represent the Interface dependencies for Class A visually in enterprise architect. Right now i'm generating the base class by importing the source code then I'm manually creating the dependencies between the Classes and Interfaces.
In my source code these dependencies are all within a variable of the class
$requiredDependencies = array('InterfaceOne', 'InterfaceTwo')...
Is there a way to programatically either parse this code or maybe enterprise architect has a way to read comments (like doxygen) and I could specify such relation in comments?
The Grammar Framework lets you generate in-EA parsers for custom languages, allowing you to reverse-engineer code in whatever language you choose. This is a pretty complex beast, but have a look in the help file under Extending UML Models -- MDG Technology SDK -- Grammar Framework.
If the language is already supported by EA, then that reverse-engineering process cannot be modified (other than what's available in the options), although you can of course write your own parser from scratch using the grammar framework.
If you want to do additional processing for a reverse-engineered class based on what's in its source file, then you can find the source file in Element.GenFile. You would then have to parse it yourself, of course.
I've a DSL written using Xtext. What I want is to execute that DSL to perform something good out of it.
I wrote myDslGenerator class implementing the interface IGenerator in xtend to generate java code and it's working fine.
I've two questions;
What is the difference between Interpreter and Code Generator?
Aren't both for executing DSL?
How to write an interpreter? Any step by step tutorial link? I found many tutorial to generate code using xtend but couldn't find any for writing an interpreter.
Thank you,
Salman
Basically, interpreters and code generators work really differently. Code generators are like a compiler: they create executable code of your DSL in another language; on the other hand, interpreters are used to traverse your DSL and execute them in your own environment. This means, the generated code does not have to (but of course it can) depend on your DSL, can be faster/more optimized; while interpreters need to understand the constructs of your language, but can be executed in your development IDE, not required to run an additional application.
AFAIK Xtext does not support writing interpreters, its somewhat out of their scope (not entirely - for Xbase expressions there is an XbaseInterpreter instance, that can be reused - provided you set its classpath correctly), as they are extremely language-specific.
I also don't know any step-by-step tutorial about interpreting Xtext DSLs (not even for the XbaseInterpreter), but it basically boils down to a traversal of the AST, and as a node is traversed, the corresponding statement is executed dynamically. For this traversal to work, as expected, the interpreter has to maintain a (possibly hierarchic) context of variables and other references.
Both of those frameworks deal with meta-model:
XText (Eclipse)
MPS (JetBrain)
Do you have example of practical applications based on meta-model transformation with those tools?
We created whole bug tracker using MPS. Code generation is not the goal but mean to get some executable code. The goal is to give a tool to developer that allows creating DSLs with minimum effort.
Cool thing about MPS is that it also provides you with an IDE for your language. And different DSLs you create are compatible, i.e. you can create DSL that extends Java with closures and another DSL that enables external methods, and these extensions will work together.
They are different in term of document storing the metamodel.
Regarding XText, this article illustrates one usage, when it comes to y create your own programming languages and domain-specific languages (DSLs).
Once you have a language, you want to process it and this means usually to transform your model into another representation.
The facility responsible for this transformation is called generator and consists of a bunch of transformation templates (e.G. XPand) and some code executing them. On some event, the model is read in and the transformations are applied to produce code.
Example of such a model transformation:
dot3zest, which comes with a DOT to Zest interpreter (which now uses the Xtext switch API generated for the DOT grammar) is support for ad-hoc DOT edge definitions.
Regarding MPS, you have here a serie of practical examples,
like this code generation to GPL such as Java, C#, C++ or XML:
(source: googlecode.com)
I think the main usage of XText is firstly to create a DSL from the grammer you defined and an eclipse workbench auto-generated for you. Secondly, it can transform the scrpit written in your DSL into java. The built-in expressions from XText2 is a plus.
The framework gives you a free IDE to support your writing DSL you created. And the DSL is the ulimate product to provide. It can be used to abstract the rules and logics from the real world. For example, in our project, the product config rule. Only specialist knows them, so they write some in the DSL you create.