All,
I have a specified type of pattern in my lex file:
"#"[ \\t]*"ifdef".* { action_ifdef_manager(yytext);}
if a text like this encoutred #ifdef GLOBALVAR the action action_ifdef_manager is not called
Thanks for any Help.
The only problem is ambiguity of patterns. You should have a similar pattern like '#ifdef'
a |
ab |
abc |
abcd ECHO; REJECT;
The lexer here returns in "abcd" stream all four validated patterns a , ab , abc, abcd
Take a look at the Flex manual
Related
I wrote such code
"State": {
"prefix": "state",
"body": [
"const [$1, set${1:/capitalize}] = useState($2);"
],
"description": "Adds state"
},
I expect that the result will be (if I enter test in $1) like this:
const [test, setTest] = useState($2);
But I get such result:
const [/capitalize, set/capitalize] = useState();
In official docs I found such rule: '${' int ':' '/upcase' | '/downcase' | '/capitalize' '}'.
Could you please tell what I am doing wrong?
You can use below snippet for the requested output:
const [$1, set${1/(.*)/${1:/capitalize}/}] = useState($2);
Output will be (in case I enter $1 as test):
const [test, setTest] = useState();
Lets look at why your version ${1:/capitalize} doesn't work:
Here is a portion of the snippet grammar you cited from https://code.visualstudio.com/docs/editor/userdefinedsnippets
tabstop ::= '$' int
| '${' int '}'
| '${' int transform '}'
-snip-
transform ::= '/' regex '/' (format | text)+ '/' options
format ::= '$' int | '${' int '}'
| '${' int ':' '/upcase' | '/downcase' | '/capitalize' '}'
So initially it looks like ${1:/capitalize} is correct, just looking at the last line of the grammar above it seems
${' int ':' '/capitalize'
is a valid option. But you have to track through the grammar to use it properly. The format syntax can only be used in a transform. We see this in:
transform ::= '/' regex '/' (format | text)+ '/' options
So right there your version does not include a transform. You do not have the necessary regex preceder. So those '/upcase' | '/downcase' | '/capitalize' options can only be used as part of a transform with a regex (although you can have an empty regex but that doesn't help you and you still need to have the regex entry point anyhow).
Here is the general form of a transform:
${someInt/regex captures here/do something with the captures here, like ${1:/capitalize} /}
Note that the first $someInt is a tabstop - it could be $1 for example, but the second $1 (with the capitalize) is NOT a tabstop but a reference to the first capture group from the preceding regex. So a transform can only transform something that has been captured by a regex.
The grammar requires that the format option be part of a transform, and a transform requires a regex and $n's in the format part refer to capture groups and not tabstop variables.
I hope this all makes sense.
I am trying to consolidate an email list, but I want to uniq (or uniq -i -u) by the email address, not the entire line so that we don't have duplicates.
list 1:
Company A <companya#companya.com>
Company B <companyb#companyb.com>
Company C <companyc#companyc.com>
list 2:
firstname lastname <firstname#gmail.com>
Fake Person <companyb#companyb.com>
Joe lastnanme <joe#gmail.com>
the current output is
Company A <companya#companya.com>
Company B <companyb#companyb.com>
Company C <companyc#companyc.com>
firstname lastname <firstname#gmail.com>
Fake Person <companyb#companyb.com>
Joe lastnanme <joe#gmail.com>
the desired output would be
Company A <companya#companya.com>
Company B <companyb#companyb.com>
Company C <companyc#companyc.com>
firstname lastname <firstname#gmail.com>
Joe lastnanme <joe#gmail.com>
(as companyb#companyb.com is listed in both)
How can I do that?
given your file format
$ awk -F'[<>]' '!a[$2]++' files
will print the first instance of duplicate content in angled brackets. Or if there is no content after the email, you don't need to un-wrap the angled brackets
$ awk '!a[$NF]++' files
Same can be done with sort as well
$ sort -t'<' -k2,2 -u files
side-effect is output will be sorted which can be desired (or not).
N.B. For both alternatives the assumption is angled brackets don't appear anywhere else than the email wrappers.
Here is one in awk:
$ awk '
match($0,/[a-z0-9.]+#[a-z.]+/) { # look for emailish string *
a[substr($0,RSTART,RLENGTH)]=$0 # and hash the record using the address as key
}
END { # after all are processed
for(i in a) # output them in no particular order
print a[i]
}' file2 file1 # switch order to see how it affects output
Output
Company A <companya#companya.com>
Company B <companyb#companyb.com>
Company C <companyc#companyc.com>
Joe lastnanme <joe#gmail.com>
firstname lastname <firstname#gmail.com>
Script looks for very simple emailish string (* see the regex in the script and tune it to your liking) which it uses to hash the whole records,last instance wins as the earlier onse are overwritten.
uniq has an -f option to ignore a number of blank-delimited fields, so we can sort on the third field and then ignore the first two:
$ sort -k 3,3 infile | uniq -f 2
Company A <companya#companya.com>
Company B <companyb#companyb.com>
Company C <companyc#companyc.com>
firstname lastname <firstname#gmail.com>
Joe lastnanme <joe#gmail.com>
However, this isn't very robust: it breaks as soon as there aren't exactly two fields before the email address as the sorting will be on the wrong field and uniq will compare the wrong fields.
Check karakfa's answer to see how uniq isn't even required here.
Alternatively, just checking for uniqueness of the last field:
awk '!e[$NF] {print; ++e[$NF]}' infile
or even shorter, stealing from karakfa, awk '!e[$NF]++' infile
Could you please try following.
awk '
{
match($0,/<.*>/)
val=substr($0,RSTART,RLENGTH)
}
FNR==NR{
a[val]=$0
print
next
}
!(val in a)
' list1 list2
Explanation: Adding explanation of above code.
awk ' ##Starting awk program here.
{ ##Starting BLOCK which will be executed for both of the Input_files.
match($0,/<.*>/) ##Using match function of awk where giving regex to match everything from < to till >
val=substr($0,RSTART,RLENGTH) ##Creating variable named val whose value is substring of current line starting from RSTART to value of RLENGTH, basically matched string.
} ##Closing above BLOCK here.
FNR==NR{ ##Checking condition FNR==NR which will be TRUE when 1st Input_file named list1 will be read.
a[val]=$0 ##Creating an array named a whose index is val and value is current line.
print $0 ##Printing current line here.
next ##next will skip all further statements from here.
}
!(val in a) ##Checking condition if variable val is NOT present in array a if it is NOT present then do printing of current line.
' list1 list2 ##Mentioning Input_file names here.
Output will be as follows.
Company A <companya#companya.com>
Company B <companyb#companyb.com>
Company C <companyc#companyc.com>
firstname lastname <firstname#gmail.com>
Joe lastnanme <joe#gmail.com>
Perhaps I don't understand the question !
but you can try this awk :
awk 'NR!=FNR && $3 in a{next}{a[$3]}1' list1 list2
public
class FirstExample { // Here in this example we define some properties in our class
WRITE // This WRITE command follows a rule type defined in my grammar
Hello World
private
String firstTitle ;
private
String secondTitle ;
private
Integer firstAmount ;
private
Integer secondAmount ;
SET
firstTitle = "Ramesh"
SET
secondTitle = "Suresh"
SET
firstAmount = 100
SET
secondAmount = 200
When I change the command 'WRITE' to 'WRITEE' then error highlights in every line.
In eclipse java editor when we write a wrong syntax then error shows on that line only.
I am searching solution for this issue for past 2 days but not able to find the solution.
Kindly help me.I am not able to upload the picture due to some 10 reputation message comes while posting the question..
I am sending the grammar.Kindly check it once.
grammar org.xtext.example.mydsl.MyPoc with org.eclipse.xtext.common.Terminals hidden (GUESS_COMMENT,WS)
generate myPoc "http://www.xtext.org/example/mydsl/MyPoc"
Domainmodel:
(elements+=Type)*;
Type:Class;
// This is the starting point of the grammar
Class:
(packageList+=packageList)*
(packageList+=Import)*
(directives+=Directives) 'class' name=ID ('extends' superType=[Class])? '{'
(greetings+=Greeting)*
(features+=Feature)*
setValues+=SetValues*
operations+=Operation*
functionCall+=ArithmeticOperation*
directives+=Directive*
allMethod+=AllMethod*
allMethodInClass+=AllMethodInClass*
samples+=Samples*
'}';
//Package list consist of all the packages we want to include in our class
packageList:
('package' name=ID) ('.'?) (classes+=[Class])?;
//Import list consist of all the packages we want to include in our class
Import:
'import' importedNamespace=QualifiedNameWithWildcard;
QualifiedNameWithWildcard:
'QualifiedName' '.*'?;
Greeting:
(directives+=Directives)'Hello World'';'
(directives+=Directives)?'bye'?';'
(directives+=Directives)?';'
;
// Features defines the properties of that class
// In our grammar by default we make the property private
// our grammar takes two types of DataType String or Integer
// Name of the property given as per rule ID defined in Terminal file
Feature:
(directives+=Directives)* type=DATA_TYPE name=ID ';';
// In SetValues we initialize the properties by using SET directive
SetValues:
(directives+=Directives) name=ID ('='?) (INT | STRING)';';
//In Operation we define the function,arguments with there Data Type
Operation:
'Do' name=ID ('(' types+=DATA_TYPE args+=ID (',' types+=DATA_TYPE
args+=ID)* ')')?';';
//In Arithmetic Operation we call the function using $ passing the parameters
ArithmeticOperation:
(directives+=Directives)? name=ID '=' (directives+=Directives)? ('$')? types+=[Operation] ('(' (args+=ID) ','
(args+=ID)* ')')?';';
//In Samples we use Do While loop and inside that loop we use if else block
Samples:
name=ID (directives+=Directives) ('!,' STRING)? loopConstruct+=LoopConstruct* operations+=Operation*;
// In AllMethod we can call any method by using class name with '.' operator
AllMethod:
(classes+=[Class]) "." types+=[Operation] ('(' (args+=INT | ID) ',' (args+=INT | ID)* ');')?;
// In AllMethodInClass we can see all methods present in that class
AllMethodInClass:
types+=[Operation] ('(' (args+=INT | ID) ',' (args+=INT | ID)* ');')?;
//In LoopConstruct we define how we can use Do while loop with directive types given below
LoopConstruct:
(directives+=Directives) ("{"
(directives+=Directives ('!,' STRING))*
(directives+=Directive)*
"}")
(directives+=Directives) ("{"
(directives+=Directives ('!,' STRING))*
(directives+=Directive)*
"}")?
(directives+=Directives)?;
Directive:
(directives+=Directives) ('(' name=ID LOOP_CODITION_CHECK name=ID ')')?
'{' (directives+=Directives ('!,' STRING)?)* '}';
Directives:
name=DirectiveType;
DirectiveType:
{DirectiveType} value=DIRECTIVES_TYPE;
// From here all are rules which shows that what we can use in our grammar
terminal DATA_TYPE:
('Integer' | 'String');
// These are some directives
// WRITE is following this rule
terminal DIRECTIVES_TYPE:
('SET' | 'WRITE' | 'READ' | 'QUIT' | 'If' | 'ElseIf' | 'Else' | 'EndIf' | 'Do' | 'while' | 'private' | 'public' | 'do');
terminal LOOP_CODITION_CHECK:
('=' | '>' | '<' | '!')+;
terminal GUESS_COMMENT : '//' !('\n'|'\r')* ('\r'? '\n');
I want to make it clear that WRITE is not only the issue.The line from where grammar breaks it shows error on the remaining lines comes after that line.
After changing my class rule with your code it gives lots of warning.I am sending the warning messages.
warning(200): ../org.xtext.example.myPoc/src-gen/org/xtext/example/mydsl/parser/antlr/internal/InternalMyPoc.g:416:2: Decision can match input such as "RULE_ID {RULE_ID..RULE_DATA_TYPE, RULE_DIRECTIVES_TYPE, 'Do'}" using multiple alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
error(201): ../org.xtext.example.myPoc/src-gen/org/xtext/example/mydsl/parser/antlr/internal/InternalMyPoc.g:416:2: The following alternatives can never be matched: 9
warning(200): ../org.xtext.example.myPoc/src-gen/org/xtext/example/mydsl/parser/antlr/internal/InternalMyPoc.g:1166:1: Decision can match input such as "RULE_DIRECTIVES_TYPE '{' RULE_DIRECTIVES_TYPE '!,' RULE_STRING '}' RULE_DIRECTIVES_TYPE" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.xtext.example.myPoc/src-gen/org/xtext/example/mydsl/parser/antlr/internal/InternalMyPoc.g:1184:1: Decision can match input such as "'Do' RULE_ID '(' RULE_DATA_TYPE RULE_ID ',' RULE_DATA_TYPE RULE_ID ')' ';'" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.xtext.example.myPoc/src-gen/org/xtext/example/mydsl/parser/antlr/internal/InternalMyPoc.g:1546:3: Decision can match input such as "RULE_DIRECTIVES_TYPE" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.xtext.example.myPoc.ui/src-gen/org/xtext/example/mydsl/ui/contentassist/antlr/internal/InternalMyPoc.g:568:1: Decision can match input such as "RULE_ID {RULE_ID, RULE_DATA_TYPE..RULE_DIRECTIVES_TYPE, '}', 'Do'..'('}" using multiple alternatives: 8, 9
As a result, alternative(s) 9 were disabled for that input
error(201): ../org.xtext.example.myPoc.ui/src-gen/org/xtext/example/mydsl/ui/contentassist/antlr/internal/InternalMyPoc.g:568:1: The following alternatives can never be matched: 9
warning(200): ../org.xtext.example.myPoc.ui/src-gen/org/xtext/example/mydsl/ui/contentassist/antlr/internal/InternalMyPoc.g:2810:42: Decision can match input such as "RULE_DIRECTIVES_TYPE '{' RULE_DIRECTIVES_TYPE '!,' RULE_STRING '}' RULE_DIRECTIVES_TYPE" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.xtext.example.myPoc.ui/src-gen/org/xtext/example/mydsl/ui/contentassist/antlr/internal/InternalMyPoc.g:2838:39: Decision can match input such as "'Do' RULE_ID '(' RULE_DATA_TYPE RULE_ID ',' RULE_DATA_TYPE RULE_ID ')' ';'" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
warning(200): ../org.xtext.example.myPoc.ui/src-gen/org/xtext/example/mydsl/ui/contentassist/antlr/internal/InternalMyPoc.g:3561:1: Decision can match input such as "RULE_DIRECTIVES_TYPE" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
After the warning code gives an exception
GeneratorException: (Element: -UNKNOWN-; Reported by: Generator)
org.eclipse.emf.common.util.WrappedException: java.io.FileNotFoundException: ..\org.xtext.example.myPoc\src-gen\org\xtext\example\mydsl\parser\antlr\internal\InternalMyPocLexer.java (The system cannot find the file specified)
I have reduced my grammar and remove all the error and warnings from my code but the error highlighting in every line is still there.
Now my grammar is
grammar org.xtext.example.mydsl.MyPoc with org.eclipse.xtext.common.Terminals hidden (GUESS_COMMENT,WS)
generate myPoc "http://www.xtext.org/example/mydsl/MyPoc"
Domainmodel:
(elements+=Type)*;
Type:Class;
// This is the starting point of the grammar
Class:
(packageList+=packageList)*
(packageList+=Import)*
(directives+=Directives) 'class' name=ID ('extends' superType=[Class])? '{'
(greetings+=Greeting
|features+=Feature
|setValues+=SetValues)*
'}';
//Package list consist of all the packages we want to include in our class
packageList:
('package' name=ID) ('.'?) (classes+=[Class])?;
//Import list consist of all the packages we want to include in our class
Import:
'import' importedNamespace=QualifiedNameWithWildcard;
QualifiedNameWithWildcard:
'QualifiedName' '.*'?;
Greeting:
(directives+=Directives)'Hello World'';'
(directives+=Directives)?'bye'?';'
(directives+=Directives)?';'
;
// Features defines the properties of that class
// In our grammar by default we make the property private
// our grammar takes two types of DataType String or Integer
// Name of the property given as per rule ID defined in Terminal file
Feature:
(directives+=Directives)* type=DATA_TYPE name=ID ';';
// In SetValues we initialize the properties by using SET directive
SetValues:
(directives+=Directives) name=ID ('='?) (INT | STRING)';';
Directives:
name=DirectiveType;
DirectiveType:
{DirectiveType} value=DIRECTIVES_TYPE;
// From here all are rules which shows that what we can use in our grammar
terminal DATA_TYPE:
('Integer' | 'String');
// These are some directives used by us taken from there website
terminal DIRECTIVES_TYPE:
('SET' | 'WRITE' | 'READ' | 'QUIT' | 'If' | 'ElseIf' | 'Else' | 'EndIf' | 'Do' | 'while' | 'private' | 'public' | 'do');
terminal LOOP_CODITION_CHECK:
('=' | '>' | '<' | '!')+;
terminal GUESS_COMMENT : '//' !('\n'|'\r')* ('\r'? '\n');
And in eclipse xtext editor when I write private as privatee it shows error in every line.
My code is
public
class MyPoc {
private
String firstTitle ;
**privatee**
String title ;
private
String secondTitle ;
private
Integer firstAmount ;
private
Integer secondAmount ;
SET
firstTitle = "Ramesh";
SET
secondTitle = "Suresh";
SET
firstAmount = 100;
SET
secondAmount = 200;
}
Now I add some more grammar in my class attribute.The console shows no warning and no exception.But in eclipse editor again when we break grammar syntax it shows error in every line.
Class:
(packageList+=packageList)*
(packageList+=Import)*
(directives+=Directives) 'class' name=ID ('extends' superType=[Class])? LBRACKET
(directives+=Directives* (features+=Feature | setValues+=SetValues | operations+=Operation | functionCall+=ArithmeticOperation | allMethod+=AllMethod
| allMethodInClass+=AllMethodInClass | samples+=Samples) SEMICOLON)*
RBRACKET;
Kindly check it once and correct me if I am doing something wrong.
I have found that problem is not in the class part.The problem is in the parts where we define the rule in the grammar.
Like for ArithmeticOperation
ArithmeticOperation:
(directives+=Directives)? name=ID '=' (directives+=Directives)? ('$')? types+=[Operation] ('(' (args+=ID) ','
(args+=ID)* ')')?';';
Actually I have to write this code in eclipse.Here Concat is the opeartion name means method name.
SET Result = WRITE $ Concat (firstTitle , secondTitle)
Kindly correct me if I ma doing something wrong to achieve my target code.
Regards
Xtext error recovery system is not the same than Java Editor (JDT). Its behavior depends on how you write your rules.
Maybe your grammar is too much ambiguous and you should add a line termination symbol.
Also, you should read this article about error recovery with Xtext: Parser error recovery
Edit
Currently the class rule is very ambiguous with chaining rules with the same first token. The first thing you should do is to rewrite your class rule and remove all ambiguities.(warnings & erros when compiling grammar).
See the following example of class rule:
Class:
(packageList+=packageList)*
(packageList+=Import)*
(directives+=Directives) 'class' name=ID ('extends' superType=[Class])? '{'
(greetings+=Greeting | features+=Feature | setValues+=SetValues
| operations+=Operation | functionCall+=ArithmeticOperation
| directives+=Directive | allMethod+=AllMethod
| allMethodInClass+=AllMethodInClass | samples+=Samples)*
'}';
Edit 2
I made some refactorings to your grammar. And now it has the behavior you asked for.
Domainmodel:
(elements+=Type)*;
Type:Class;
// This is the starting point of the grammar
Class:
(packageList+=packageList)*
(packageList+=Import)*
(directives+=Directives) 'class' name=ID ('extends' superType=[Class])? LBRACKET
(directives+=Directives* (features+=Feature | setValues+=SetValues) SEMICOLON)*
RBRACKET;
terminal SEMICOLON:
';'
;
terminal LBRACKET:
'{'
;
terminal RBRACKET:
'}'
;
//Package list consist of all the packages we want to include in our class
packageList:
('package' name=ID) ('.'?) (classes+=[Class])?;
//Import list consist of all the packages we want to include in our class
Import:
'import' importedNamespace=QualifiedNameWithWildcard;
QualifiedNameWithWildcard:
'QualifiedName' '.*'?;
// Features defines the properties of that class
// In our grammar by default we make the property private
// our grammar takes two types of DataType String or Integer
// Name of the property given as per rule ID defined in Terminal file
Feature:
type=DATA_TYPE name=ID ;
// In SetValues we initialize the properties by using SET directive
SetValues:
name=ID ('='?) (INT | STRING);
Directives:
name=DirectiveType;
DirectiveType:
{DirectiveType} value=DIRECTIVES_TYPE;
// From here all are rules which shows that what we can use in our grammar
terminal DATA_TYPE:
('Integer' | 'String');
// These are some directives used by us taken from there website
terminal DIRECTIVES_TYPE:
('SET' | 'WRITE' | 'READ' | 'QUIT' | 'If' | 'ElseIf' | 'Else' | 'EndIf' | 'Do' | 'while' | 'private' | 'public' | 'do');
terminal LOOP_CODITION_CHECK:
('=' | '>' | '<' | '!')+;
terminal GUESS_COMMENT : '//' !('\n'|'\r')* ('\r'? '\n');
Details of what i changed:
- Removed (useless?) greeting rule.
- Made line/block separators terminals.
- Moved directive to parent rule.
Sample code for editor:
public
class MyPoc {
private
String firstTitle ;
privatee
String title ;
private
String secondTitle ;
private
Integer firstAmount ;
private
Integer secondAmount ;
SET
firstTitle = "Ramesh";
SET
secondTitle = "Suresh";
SET
firstAmount = 100;
SET
secondAmount = 200;
}
I'm trying to define a rule where one element is the perfect candidate for the common terminal rule ID. When I launch the editor to test with sample code, the ID rule is not matched for the string "ABC":
Define : ABC : Holder_1
I get "mismatched input 'ABC' expecting RULE_ID".
Is there something in my grammar conflicting with the ID rule to cause this error?
This is my grammar file:
grammar com.testco.Test with org.eclipse.xtext.common.Terminals
import "http://www.eclipse.org/emf/2002/Ecore" as ecore
generate defwiz "http://www.testco.com/Test"
Define_Holder:
'Definition' ':' holder_name=ID ':' holder_number=HOLDER_NUMBER (':' attribute=define_attr)? (':' pad=holder_pad)?
;
holder_pad:
HEX
;
Type:
TYPE_TERM ':' type_value=TYPE_VAL
;
//***************** TERMINALS *****************
terminal TYPE_TERM:
'Type_' INT+
;
terminal PROFILE:
(PROFILE_ID)(PROFILE_ID)'.'(PROFILE_ID)(PROFILE_ID)
;
terminal HOLDER_NUMBER returns ecore:: EString:
'Holder_' INT+;
terminal HEX returns ecore:: EString :
('0'..'9'|'A'..'F')
;
terminal PROFILE_ID : '^'?('a'..'z'|'A'..'Z') ('a'..'z'|'A'..'Z'|'0'..'9')*;
//***************** ENUMS *****************
enum define_attr:
BINARY='Binary' |
SCRAMBLE='Scramble' |
FORCESIZE='ForceSize' |
FIXEDSIZE='FixedSize'
;
Your rule PROFILE_ID shadows the ID rule for almost every case (except for the underscore).
Yes, it's likely that lexer scans ABC as HEX terminal. Try to define the latter, for example, as follows:
terminal HEX returns ecore:: EString :
'0x' ('0'..'9'|'A'..'F')
;
I've a problem with LIKE operator in PostgreSQL. It does not match patterns that contain the - character. I've tried to escape these characters using the ESCAPE option, but it still does not work.
Query:
select * from footable where trascrizione like '% [---]is Abraam %';
Sample data (contents of column trascrizione):
[---]is Abraam [e]t Ise[---] / ((crux quadrata))
How can I solve this problem?
That pattern would not match because there is no space before [---]is Ambraam. There is a space in your pattern, between the % and [ characters, and it's requiring that space to be in your data. Try LIKE '%[---]is Abraam %'.
Please show the code. I have no problem:
SELECT * FROM a WHERE a LIKE '%-' AND b LIKE '%-%' AND c LIKE '-%';
a | b | c | d | e
------+---+------+---+---
foo- | - | -bar | |
(1 Zeile)