Change the multi-line comment characters in Xtext - eclipse

I want a Xtext grammar that allows me to write MIME media types this way:
mediaType application/atom+xml
specURL "http://www.rfc-editor.org/rfc/rfc4287.txt",
This is not a problem, but the following is:
mediaType application/*
specURL "http://www.iana.org/assignments/media-types/application",
You can guess of the troubles ahead with the /* characters that usually define a multi-line comment. The terminal for it is defined in the default Terminals provided by Xtext, more specifically in the ML_COMMENT terminal:
terminal ML_COMMENT : '/*' -> '*/';
I customized it by copying the default terminals to a new one of my own, where the ML_COMMENT terminal is defined this way instead:
terminal ML_COMMENT : '"""' -> '"""';
This produces a more Pythonistic way to have multi-line comments. It works fine in the generated DSL. But the /* characters still pose problem when I try to define the media type for application/*, as shown above. I get an error message of mismatched input '/*' expecting '}' (the } character would specify the end of the media types listing).
Even more troubling is that the content assist of the Xtext editor still auto-fill an ending */ multi-line comment characters when I type a (supposedly obsolete) /* combo characters. As I overrode the multi-line comment terminal, I am wondering why the auto-complete still flirts with the older ML_COMMENT definition. Do I need to override something else?
Here are some fragments for the media type grammar:
MediaType returns restapi::MediaType:
{restapi::MediaType}
'mediaType' name=MediaTypeQualifier ('specURL' specURL=EString)?;
MediaTypeQualifier:
MediaTypeFragment ('/' MediaTypeFragment)?(';' MediaTypeFragment'='MediaTypeFragment)*;
MediaTypeFragment:
(ID ( ('-'|'+'|'.') ID )* ) | '*'
I am using Xtext version 2.3.1 within Eclipse 4.2.2. Does anyone have experience with overriding the multi-line comment terminal? Is there something that I missed?

It's hard to tell from the grammar snippet that you provided, but it appears to me that you still have a keyword /* somewhere in your grammar.

Related

VS code interprets wrong comma from my keyboard

When I type a comma in VS code I get
,
but what I really want is
,
I'm not sure why there's two types of commas, but the first one is giving me run time errrors in python
"I'm not sure why there's two types of commas"
Well ... it is because Unicode defines them1. The first one is U+FF0C : FULLWIDTH COMMA. The second one is U+002C : COMMA
The latter is the one that should be bound to the "comma" key on your keyboard, but it is possible that something has changed your VSCode key bindings. This page describes how to examine your VSCode key bindings.
But I think the more likely explanation is that you have copy-pasted some source code from (for example) a PDF file that is using U+FF0C instead of U+002C for cosmetic reasons ... or something. It ia also possible that they were placed there by the author of the original document, or by their word-processing software.
You could try using the Gremlins extension to highlight any potentially troublesome characters in your source code.
1 - According to Wikipedia, the purpose is "so that older encodings containing both halfwidth and fullwidth characters can have lossless translation to/from Unicode.".

change '#' key in freemarker templates

In order to use if statements in Freemarker templates, the following syntax is used;
[#if ${numberCoupons} <= 1]
[#assign couponsText = 'coupon']
[/#if]
Is there a way to replace the '#' character with something else, because I am trying to integrate it with drools (a java based rule engine) and the '#' character is used to mark start of comments so the formatting breaks?
There isn't anything for that out of the box (it uses a JavaCC generated parser, which is static). But you can write a TemplateLoader that just delegates to another TemplateLoader, but replaces the Reader with a FilterReader that replaces [% and [/% and [%-- and --%] with [#, etc. Then then you can use % instead of # in the FreeMarker tags. (It's somewhat confusing though, as error messages will still use #, etc.)
As #ddekany wrote, you can write code that tranform the template without the pound sign, But notice it can clash with HTML or XML (and similar) tags, at least from an editor prespective.

.tmlanguage escape sequences and rule priorities

I'm implementing a syntax highlighter in Apple's Swift language by parsing .tmlanguage files and applying styles to a NSMutableAttributtedString.
I'm testing with javascript code, a javascript.tmlanguage file, and the monokai.tmtheme theme (both last included in sublime text 3) to check that the syntax get highlighted correctly. By applying each rule (patterns) in the .tmlanguage file in the same order they come, the syntax is almost perfectly highlighted.
The problem I'm having right now is that I don't know how to know that a quote (") should be escaped when it has a backslash before it (\"). Am I missing something in the .tmlanguage file that specifies that?. Other problem is that I have no idea how to know that other rules should be ignored when inside others, for example:
I'm getting double slashes taken as comments when inside strings: "http://stackoverflow.com/" a url is recognised as comment after //
Also double or single quotes are taken as strings when inside comments: // press "Enter" to continue, the word "Enter" gets highlighted as string when should be same color as comments
So, I don't know if there is some priority for some rules over others in the convention, or if there is something in the files that I haven't noticed.
Help please!
Update:
Here is a better example of what I meant by escape quotes:
I'm getting this: while all the letters should be yellow except for the escaped sequence (/") which should be blue.
The question is. How do I know that /" should be escaped? The rule for that piece of code is:
Maybe I am late to answer this. You can apply the following method.
(Ugly) In your end regex, use ([^/])(") and in your endCaptures, it would be
1 = string.quote.double.js
2 = punctuation.definition.string.end.js
If the string must be single line, you can use match=(")(.*)("), captures=
1 = punctuation.definition.string.begin.js
2 = string.quote.double.js
3 = punctuation.definition.string.end.js
and use your patterns
You can try applyEndPatternLast and see if it is allowed. Set applyEndPatternLast=1 will do.
The priority is that earlier rules in the file are prioritized over later rules. As an example, in my Python Improved language definition, I have a scope that contains a series of all-caps constants used in Django, a popular Python web framework. I also have a generic constant.other.allcaps.python scope that recognizes (just about) anything in all caps. Since the Django constants rule is before the allcaps rule in the .tmLanguage file, I can color it with a theme using one color, while the later-occurring "highlight everything in all caps" only grabs identifiers that are NOT part of the first list.
Because of this, you should put your "comments" scope(s) as early in the file as possible, then write your parser in such a way that it obeys the rule I described above. However, it's slightly more complicated than that, as I believe items in the repository are prioritized based on where their include line is, not where the repository rule is defined in the file. You may want to do some testing to verify that, though.
Unfortunately I'm not sure what you mean about the escaped quotes - could you expand on that, and maybe add an example or two?
Hope this helps.
Assuming that / is the correct character for escaping a double quote mark, the following should work:
"str_double_quote": {
"begin": "\"",
"end": "\"",
"name": "string.quoted.double.swift",
"patterns": [
{
"name": "constant.character.escape.swift",
"match": "/[\"/]"
}
]
}
You can match an escaped double quote mark (/") and a literal forward slash (//) in the patterns to consume them before the end marker is used to handle them.
If the character for escaping is actually a backslash, then the tricky bit is that there are two levels of escaping, for the JSON encoding as well as the regular expression syntax. To match \", the regular expression requires you to escape the backslash (\\"). JSON requires you to escape backslashes and double quotes, resulting in \\\\\" in a TextMate JSON grammar file. The match expression would thus be \\\\[\"\\\\].

How to use unicode characters in Eclipse File Search?

We have some XML file that contains some invalid character, and the program says neither which file it is, nor which line number or character offset. It would be a few seconds work to fix the problem if I could just search for exactly that character, but I cannot find how to express a Unicode character in the file search (or at least I assume so, since the search returns nothing).
Neither 0x1e nor \u001e seem to match anything.
[EDIT] I mean, I can still change the code, and eventually find which file it is by catching the Exception, and using some kind of script/tool to find where exactly the character is, but I do believe it should be possible to search with Unicode in Eclipse, and that is what I am asking in this question.
It may be a problem with the character encoding.
As you're going to need to perform a global / site-wide search to find the , you'll probably need to set the global text file encoding:
Preferences -> Workspace -> Text file encoding
This option may be under the 'General' section in Eclipse, depending on your setup and installed plugins etc.
Ensure that the encoding is set to UTF-8.
You will also need to escape the unicode character sequences, like so:
\u2665
(which I see you have tried)

Netbeans : Auto Format : prevent it for a section of my code

I use Netbeans auto format (ctrl+alt+f) a lot. It's a very nice function!.
But I use StringBuffer.append() to generate some xml. I indent the .append parameter to represente the node structure of my xml.
msg.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
msg.append( "<root>");
msg.append( "<subNode/>");
my problem : the autoformat move all my parameters to the same column.
msg.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
msg.append("<root>");
msg.append("<subNode/>");
My question : How can I prevent the auto format to to modify my code on a section of my file. I'm hoping to found something similar to "editor-fold".
//<editor-noAutoFormatting>"
msg.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
msg.append( "<root>");
msg.append( "<subNode/>");
The question has already been ask :
Netbeans: Auto Format - Prevent Space Formatting on Variable Assignment
Netbeans auto format issue with method parameter indention
editing the NetBeans source formatting standard
There is unfortunately no answer for that. The idea of annotation is not implemented for formatting (or I don't find it).
So from now the only way to avoid this, is to select the text you want to format, without your xml part and then use format.
EDIT :
The only things I found to avoid autoformat to delete spaces is to use comments /* */.The spaces between them will not be trim by Netbeans formatter.
Example :
msg.append(/* */"<subNode/>");.